Automação de Auditoría
Ryan Teeter, Ph.D. Student, Rutgers University
18th World Continuous Auditing Symposium
6th CONTECSI São Paulo, Brasil – June 4, 2009
Aiding the Audit
Outline
1. Introduction
– Continuous Controls Monitoring (CCM) and COSO
Guidance
2. Automating the IT Audit
– Evaluating monitoring software platforms
3. Implementation of CCM at Siemens PLM
– Classifying audit requirements into degrees of
automation
– Creating rules from Audit Action Sheets
– Reengineering audit processes
– Feedback loop
4. Preliminary Results
– Time and resource commitments
– Successes & Challenges
18th WCARS – June 4, 2009
5. Conclusion
2
Aiding the Audit
1. Introduction
• Continuous Controls Monitoring (CCM)
– Evaluating control settings on business processes
that provide compliance with regulation and/or
internal management objectives.
– Proof of concept expands existing research in
continuous audit streams (see Brown, Wong, and
Baldwin 2007, Alles et al 2006)
• COSO “Guidance on Monitoring Internal Control
Systems” (2008):
– Effective monitoring involves (1) establishing an
effective foundation for monitoring, (2) designing
and executing monitoring procedures that are
prioritized based on risk, and (3) reporting the
results, and following up on corrective action where 3
18th WCARS – June 4, 2009
necessary.
Aiding the Audit
2. Automating the IT audit
• Why the IT audit?
– Cost considerations and effectiveness
– Internal audit team spends approximately 70 days
manually checking tables, authorizations, and
documentation
• Vasarhelyi et al (2004) indicate firms are
likely to adapt existing internal audit
programs
• Alles et al (2006) suggest utilizing the
expertise of experienced audit professionals
• Verifiability of automated controls against
results from the manual audit
18th WCARS – June 4, 2009
4
Aiding the Audit
2.1 Evaluating monitoring software
platforms
• CCM aids compliance for SOX sec. 201
and 404
• Large accounting firms unable to
source CCM
• Third-party platforms installed as
monitoring and control layer
– Minimal impact on
Type
ERPperformance
Topography
– See Vasarhelyi, Homogenous
2004
System-specific
Example
Modular and mapping
Homo/heterogenous
ACL IDEA, OverSight
Custom
Homo/heterogenous
Siemens’ e-Audit
Alles, et al 2006
18th WCARS – June 4, 2009
Approva BizRights
5
Aiding the Audit
Siemens’ Current SAP Audit Model
• Use text file output and transaction checks on
line to audit SAP
• Report findings and recommendations for
remediation
• Use follow-up audits to assure appropriate
Company A are Company B
Company C
Company D
controls
SYS.
SAP SYS.
SAP SYS.
SAP SYS.
in SAP
place
and
remain
in
place
PD2
P88
P51
P40
Common –“E -Audit” Extractions on a request basis.
Text File
Store
18th WCARS – June 4, 2009
Text File
Store
Text File
Store
Text File
Store
6
Aiding the Audit
3. Implementation of CCM at
Siemens PLM
• Rules were created in Approva BizRights based
on ~300 audit action sheets provided by
Siemens
• Siemens Corporation wants universallyadaptable sets of rules and control tests for
use in different divisions
18th WCARS – June 4, 2009
7
Aiding the Audit
AAS Audit Program Test of Effectiveness:
1. /nSA38 report RSUSR002, user SAPCPIC.
2. Check whether SAPCPIC is used as a
dialogue user
(>>eAudit: 1.02.060_2 SAP* data in
USR02 – last login date, UFLAG <<)
3. Check which profiles have been assigned
(>>eAudit:
18th WCARS – June
4, 2009
1.02.060_3
profiles of SAP* in USR04<<)
8
Aiding the Audit
18th WCARS – June 4, 2009
9
Aiding the Audit
3.1 Classifying rules by degree of
automation
• Authorization
– users with access to screens or functions
– Approx 30% of audit effort
• Baseline
• Separation of duties
• Transaction
– Frequency of code use
• User Activity Insight
– Timeliness and correctness
• Configuration
– ERP settings
• Manual
18th WCARS – June 4, 2009
10
Aiding the Audit
3.2 Creating rules from Audit
Action Sheets
• Low-hanging fruit
– Authorization requests
– Separation of duties checks
– Example: See who can create and approve purchase
orders
• Partial automation
– Example: See who has access and whether that is
appropriate
• Non-automatable
– Evaluation of documentation
– Interviews with managers
18th WCARS – June 4, 2009
11
Aiding the Audit
3.3 Reengineering audit processes
• Creation of custom rules in Approva
InsightStudio
• Combination of existing controls tests
• Partial automation of manual controls
– “Gain an understanding of X process. Verify Y
function isn’t allowed.”
18th WCARS – June 4, 2009
12
Aiding the Audit
3.4 Feedback loop
• Rule descriptions were added to aid the audit
• Rules were tested and compared to results
from the manual audit
• Adjustments were made based on results
Rule Type
AAS Ref
#
Short Description
Description of rule to be
made
Conditions
used
Status
Authorization
1.02.X
Unauthorized access to
SAP system – emergency
user concept
Test these authorizations:
1. S_TCODE=SM18,
S_ADMIN_FCD=AUDA
2. …
AI rules
Rule Built
1.02.X
Configuration
1.02.X
System
admin/completeness
verification
Set up 3 rules to test the
following:
1. parameter rdisp/vbdelete=0
2. parameter rdisp/vbreorg=0
3. …
Parameters
are listed in
report
RSPFPAR
Rule Built
1.02.X
18th WCARS – June 4, 2009
13
Aiding the Audit
4. Results
•
•
•
•
Time and resource commitments
Successes
Challenges
Firm characteristics
18th WCARS – June 4, 2009
14
Aiding the Audit
4.1 Time and resource commitments
• Time commitments:
– 70 days for the manual audit
– 3 months preparation
• Platform installation
• AAS classification
• Resource commitments
– Travel, lodging, etc.
•
•
•
•
3-5 researchers – 3 Full-time equivalent
2 internal auditors at PLM
2 IT auditors from Siemens
1 support staff from Approva
18th WCARS – June 4, 2009
15
Aiding the Audit
4.2 Successes
• Initially approximately 63% of controls
automated
• Rules were used to provide support for the IT
audit
• Initial evaluation of cost savings (A&D PL
specific)1:
–
For 3 of every 4 years, eliminate ~ 500 man-hours
of IT GCC and
application control testing (@ $137/hr =
$68,750/year)
– With system certified, 80% reduction in 500 manhours of annual
external IT audit hours
(@ ~$200/hr, $80,000/year)
18th WCARS – June 4, 2009
1
Siemens IT audit pool billing rate is $137/hour; Approx $200/hr Big 4
16
Aiding the Audit
4.2 Successes
Module
Total
Controls
Authoriz
ations
Controls
Percent
Automated
(%)
Business
Process
Controls
Percent
Automated
(%)
Overall
Percent
Automated
(%)
Basis System (BC)
104
20
100%
84
44%
55%
Financial Accounting (FI)
55
8
100%
47
51%
58%
Asset Accounting (AA)
26
4
100%
22
64%
69%
Sales and Distribution (SD)
21
5
100%
16
50%
62%
Materials Management (MM)
32
8
100%
24
54%
66%
Project System (PS)
32
9
100%
23
70%
78%
Human Resources (HR)
14
14
100%
N/A
N/A
100%
Total
284
68
100%
216
52%
63%
18th WCARS – June 4, 2009
17
Aiding the Audit
4.3 Challenges
• Audit priority
– Non-applicable rules ignored because of time
constraints
• CCM platform issues
– Bugs or unimplemented features
– Identified when comparing automated with manual
results
– Vendor vs. auditor priorities
– Issues addressed in future releases
• Properly functioning controls
– Control failure resulted in lack of support for the
audit
18th WCARS – June 4, 2009
18
Aiding the Audit
4.4 Firm characteristics
• Siemens PLM
– Technology firms generally have better IT controls
– Already using SAP R/3
• Degree of success may depend on the amount of
IT systems and support.
18th WCARS – June 4, 2009
19
Aiding the Audit
5. Conclusion
• The IT audit is a feasible starting point for
CCM implementation
– Existing audit plan
– Knowledge of experienced auditors
– Real-time performance comparison
• 63% of audit controls automated
– 100% of authorizations, which comprise 30-35% of
audit commitment
– vs. 75% proposed by Alles et al. (2006)
18th WCARS – June 4, 2009
20
Aiding the Audit
Expanding this paper
• Weighting control risk?
– (Cushing 1974, Cash et al, 1977, Vasarhelyi 1980,
Srindini and Vasarhelyi 1986, Vasarhelyi and
Srindini 1989)
• Cost savings reallocation to auditing
rulebooks?
18th WCARS – June 4, 2009
21