Expect the Unexpected
Planning the Scope of an IT Performance Audit
Robin Garity, C.P.A., C.I.S.A.
October 2014
Agenda
Standards
Importance
Audit Assignment #1 – Michigan Business One Stop
System
Audit Assignment #2 – Branch Office System
What do the standards say about
Performance Audit Planning?
 Generally Accepted Governmental Auditing Standards
(GAGAS) states
 6.07 Auditors must plan the audit to reduce audit risk to an
appropriate level for the auditors to obtain reasonable
assurance that the evidence is sufficient and appropriate to
support the auditors’ findings and conclusions.
 6.09 The scope defines the subject matter that the auditors will
assess and report on, such as a particular program or aspect of
a program, the necessary documents or records, the period of
time review, and the locations that will be included.
Why is planning the audit scope
important in a performance audit?
 Determines direction of audit (many possibilities)
 Security
 Accurate processing
 Efficiency of system
 Governance
 Determines audit value
 What will change if the conclusion is that the auditee/system is not
effective?
 Will recommendations be useful?
Why is planning the audit scope
important in a performance audit?
(continued)
 Ensures that all significant risks are identified and addressed
during the audit
 Poor scope planning can result in a stressful audit
 Inadequate resources
 Inefficient testing
 No pressure…But don’t mess up when planning the audit
scope!
Audit Assignment Example #1
Michigan Business One Stop System (MBOS)
 Assignment based on criticality to audit entity
 System mission - Create a one-stop shop for individuals or
businesses doing business with the State of Michigan
 No prior audits
 Implemented in 2009
 Known costs of $21.3 million to date for development and
maintenance
Scope Planning Ideas
 Confidential and critical licensing information in the system.




Operating System Access and Configurations
Database Access and Configurations
Application Access
Monitoring Processes
Scope Planning Procedures
 Interviewed project manager, DBA, and system administrators
 Reviewed system documentation
 Data dictionary
 Network diagram
 Development contracts
 Reviewed policies and procedures for managing the system
 Interviewed users/stakeholders
What We Heard
 Very few customers liked or used MBOS
 Process was much more complex for customers
 Applicant data must be reentered into secondary systems
 New development projects on hold because of uncertainty
regarding MBOS’s future
 Departments unsure of what license information is available in
the system
Scope U-Turn
 FROM:
Operating System Access and Configurations
Database Access and Configurations
Application Access
 TO:
Project Planning - Is there a plan for making the system more effective?
Governance - Is there leadership to make decisions on the future of the
system?
Updating of System - If departments are unsure of licenses in the system,
are license applications really up to date in MBOS?
What We Learned About
Planning the Audit Scope
 Always interview users of the system during planning.
 Keep in mind the future impact.
 Be flexible.
Outcome
 Findings
 No strategic plan for continued development and use of the system.
 No post-implementation review to determine if expected benefits were realized.
 Lack of an effective governance structure.
 No process to periodically review and update the content (out-of-date fees,
applications, etc.)
 Latest update – DTMB is shutting down the system because it
is not providing the expected benefits.
Audit Assignment Example #2
Branch Office System
 System used in branch offices for vehicle registrations, driver
licensing, etc.
 The Department of State collects approximately $2.2 billion per year
through the various systems that process driver and vehicle related
transactions.
 Audit assignment based on revenue and criticality of system
Scope Planning Ideas
Branch Office System
 Application controls
 Access/segregation of Duties
 Proper input of licensing and registration data
 Change management
Scope Planning Procedures
 Interviewed project managers, DBA, and system administrators.
 Reviewed system documentation




Data dictionary
Network diagram
Development contracts
System flows
 Reviewed policies and procedures for managing the system.
 Interviewed system users.
What We Found Out
 Branch Office System scheduled for replacement.
 Many systems process driver and vehicle related data on the back
end and store confidential data. The Branch Office System is
primarily data input.
 Complex flow of information between departments for use in
processing driver and vehicle-related data.
 Prior non-IT audit of fee calculations (audited around systems) but no
actual IT audits.
A New Focus
 FROM:
Branch Office System Application controls

Access/Segregation of duties

Proper input of licensing, registration data
 TO:
 Excluding Branch Office System (being replaced)
 Security for other driver and vehicle related systems that store confidential data

Operating System

Database
 Reviewing actual processing of data outside of Branch Office System

Are matches and input of information proper to ensure no registrations to suspended licenses, deceased, stolen
vehicles, etc.

Excluding fee calculations
What We Learned About
Planning the Audit Scope
 Consider new development projects
 Consider entire process
 Understand in detail what has already been audited
Potential Audit Conclusions
 Security weaknesses
 Access issues
 Data processing inconsistencies
Final Suggestions For
Planning the Audit Scope
 Be sure to:
 Spend sufficient time in planning
 Obtain complete understanding of business processes and flow of
system data
 Listen to what auditee and users think are the problems
 Evolve your scope
 To ensure:
 Audit value
 Impact on future processes
 An efficient audit