F E AT U R E
Information Technology Auditing
and Facilitated Control Self-assurance
By Ken Doughty, CISA, CBCP, and John O’Driscoll, CISA, CIA
A
fundamental requirement for effective auditing is to
provide an opinion to the executive team and the board
audit committee on the adequacy of the internal control framework operating within the organisation’s information
technology and telecommunications (IT&T) environment. This
requirement, while ongoing, may have specific meaning at
some point, e.g., financial year-end when management is
required to sign off on the end of year accounts.
IT auditors have used a range of audit methodologies and
techniques to support their audit opinions. This paper will outline an approach recently utilised within a financial services
organisation to provide an annual assessment of the IT&T
internal control framework.
The approach used is based on a set of internationally
recognised IT service delivery and support process models
called ITIL (Information Technology Infrastructure Library)
and relies extensively on the use of control self-assurance
(CSA) workshops facilitated by IT audit staff. This approach
can be applied with equal success to internal and outsourced
IT&T environments.
CSA is a risk management program where risks and controls are examined and assessed to provide reasonable assurance to management that business objectives will be met. IT
management and staff involved in the delivery of services and
products to an organisation participate in all phases of the
process. For CSA to be effective it must have support from
IT&T management and staff.
The strengths of this approach are that it:
• Engages stakeholders in the review process
• Provides an end-to-end process perspective
• Increases participants’ knowledge and understanding of their
processes and potential risk exposure
• Provides process participants with an opportunity to air common concerns and participate in process improvement
Preparation
Training and ITIL Certification
A certified ITIL trainer presents a three-day in-house training program on ITIL. The training participants include all
internal IT auditors and representatives from the organisation’s
external auditors, IT department and IT&T service providers.
The training program also incorporates a number of teambuilding activities.
At the end of the three-day training program, all the attendees have completed the ITIL Foundation Level Certification
examination.
The main reason for inviting representatives outside of the
organisation’s IT audit department is to establish and maintain
an open and trusted relationship with all stakeholders. Further,
it helps gain acceptance of the new approach being adopted.
Senior Management Buy-in
A number of presentations are made to senior management
within the organisation and the IT&T service providers to
explain the purpose and approach and gain their acceptance.
A brochure to aid the communication on the purpose and
context of the process is prepared and distributed to all key
stakeholders.
ITIL Processes
The IT&T environment covers a large range of processes that
deliver services and products to the organisation. To ensure there
is effective coverage of the IT&T environment in using FCSA
(facilitated control self-assurance), the auditor utilises a best
practice IT framework with defined supporting processes.
One such framework with well-defined supporting processes is ITIL (www.itil.co.uk), a series of best practice guidelines
in IT management. Developed in the late 1980s when IT professionals gathered to develop a comprehensive life cycle for
all areas of IT service management, it created a common set of
terminology for the IT team to put services into business terms
and align them with key objectives.
This nonproprietary framework consists of a number of
interrelated processes that provide an end-to-end perspective to
IT&T service provision, with the aim to drive continuous
improvement. Today, many industry leaders recognise ITIL as
the de facto world standard in IT service management.
Service management is defined by ITIL as “the process of
maintaining and gradually improving business-aligned IT service quality, through the constant cycle of agreeing, monitoring, reporting and reviewing IT service achievements and
through instigating actions to eradicate unacceptable levels of
service.”
Since the mid-1990s, ITIL has had a proven track record
with corporations and governments worldwide, such as
Microsoft, Procter & Gamble, AXA and ABN AMRO.
The benefits of using ITIL include:
• IT services meeting business requirements, i.e., the IT effort
is focused on assisting the organisation achieve its strategic
objectives
• Improved efficiency and quality of service delivery
• Clear understanding of service delivery and support priorities
• Improved relationship among customers, IT and vendors
• Improved lines of communication between IT and its
customers
• Monitoring of service delivery which facilitates the identification of areas of process weakness
ITIL is particularly effective in an outsourcing environment,
where a business relationship between IT&T supplier(s) and
customers aims to provide an optimal balance between highquality services and controlled costs.
Figure 1 indicates the interrelationships among service
delivery, service support, application management, infrastructure management and the business perspective.
Figure 1—The ITIL Jigsaw
Service
Support
Service
Delivery
Applications
Management
The Business
Perspective
Infrastructure
Management
The ITIL service delivery process model is set out in
figure 2 and further description of each component is included
in table 1.
Table 1
ITIL
Component
Goal
Service level To maintain and gradually improve businessmanagement aligned IT service quality through a constant
cycle of agreeing, monitoring, reporting and
reviewing IT service achievements and
through instigating actions to eradicate
unacceptable levels of service
Availability To optimise the capability of IT infrastructure
management and supporting organisation to deliver a costeffective and sustained level of availability that
enables the business to satisfy its objectives
Capacity
To understand the future business
management requirements (the required service delivery),
the organisation’s operation (the current
serviced delivery), the IT infrastructure (the
means of service delivery), and ensure that
all current and future capacity and
performance aspects of the business
requirements are provided cost-effectively
Financial To provide cost-effective stewardship of the
management IT assets and the financial resources used in
for IT services providing IT services
IT service
continuity
To support the overall business continuity
management process by ensuring that the
required IT technical and services facilities
can be recovered within required and agreed
business time-scales
Figure 3 describes the ITIL service support process model.
Each component is broken down in table 2.
Figure 2—ITIL Service Delivery Process Model
Figure 3—ITIL Service Support Process Model
The Organisation, Customers and Users
The Organisation, Customers and Users
Service Level
Management
Availability
Management
Capacity
Management
Incident
Management
Financial
Management
Problem
Management
Change
Management
IT Service
Continuity
Release
Management
Configuration
Management
Table 2
ITIL
Component
Figure 4
Goal
Incident
To restore normal service operation as
management quickly as possible with minimum disruption
to the business, thus ensuring that the best
achievable levels of availability and service
are maintained
Problem
To minimise the adverse effect on the business
management of incidents and problems caused by errors in
the infrastructure, and to proactively prevent
the occurrence of incidents, problems and errors
Change
To ensure that standardised methods and
management procedures are used for efficient and prompt
handling of all changes, to minimise the
impact of any related incidents upon service
Release
To take a holistic view of change to an IT
management service and ensure that all aspects of a
release, both technical and nontechnical, are
considered together
Configuration To provide a logical model of the IT
management infrastructure by identifying, controlling,
maintaining and verifying the versions of all
configuration items in existence
There are two other critical supporting processes to the ITIL
service management model that the auditor needs to be aware
of when using ITIL as the best practice IT management framework: service desk and security (table 3).
Overview of FCSA Process
WORKSHOP
PARTICIPANTS
RESPONSE TO
STATEMENTS
DEVELOP
PROCESS
CONTROL
STATEMENTS
PROCESS
RISK AND
CONTROL
PROFILE
IDENTIFY
CONTROL RISKS
SELFASSURANCE
CONFIRM
CONTROL
EFFECTIVENESS
REPORT
RESULTS
DEVELOP
SERVICE
IMPROVEMENT
PLAN
workshop. This information is important, as it will assist in
alleviating any misconception the workshop participants may
have about the workshops and its outcomes.
It is important to remember that the workshops should be
structured to include:
• An introduction (overview of workshop process, tools and a
clearly defined purpose and context)
• A description of the IT&T process to be covered
• A description of the process (e.g., problem management) to
capture responses from participants (i.e., information)
• Workshop feedback (i.e., outcomes)
• The way forward
Table 3
ITIL
Component
Goal
Service desk To act as the central point of contact between
the user and IT service management; to
handle incidents and requests and provide an
interface for other activities such as change,
problem, configuration, release service level,
and IT service continuity management
Security
To manage the defined level of security on
information and IT services
FCSA Workshops
Facilitated CSA workshops are conducted by the auditor to
assist IT&T management and staff in identifying risks, assessing the control environment and developing a service improvement plan (SIP) to mitigate the risks.
Workshop Structure
To facilitate workshop participation and outcomes, the
participants should receive a handout describing the FCSA
process (figure 4) including the tools being used in the
Experience shows that:
• The workshop duration should not be more than two hours,
as it requires a large amount of concentration by participants.
• The number of workshop participants should be between five
and nine.
• Workshop statements should be limited to a maximum of 25.
• The workshop should be held in a closed room to minimise
disruptions.
Workshop Participants
It is important that the appropriate participants are selected
to attend all workshops. The participants should include IT&T
management and staff, including the IT&T department managers and senior/experienced staff members who are involved
in the delivery of services and products to the organisation. It
is critical that they know their processes and are empowered to
implement the appropriate controls.
The IT&T department managers are selected, as they are the
process owners. They have the accountability for managing the
IT&T risks on a day-to-day basis. They must take ownership
of the information technology risk and control processes and
proactively manage these over time.
Workshop Statements and Assessment Criteria
One of the critical parts of the FCSA is developing the statements to be asked of the workshop participants for response.
The statements are developed using information technology best
practice standards for service delivery and support.
Up to 25 statements per process form the basis of the FCSA
workshops. Individual workshop participants are asked to
respond to each statement using the following seven-point
scale:
• Strongly disagree
• Disagree
• Slightly disagree
• Neither agree or disagree
• Slightly agree
• Agree
• Strongly agree
In particular, the answers from participants should elicit
discussion based upon their knowledge of and experience
with the processes. An example of a statement in relation
to problem management is: “There are clear criteria for
prioritising a problem.”
Workshop Tool (Optional)
In this approach, an anonymous voting tool is used during
each workshop. Each participant uses a separate keypad to vote
on each statement. A bar chart analysis of all responses is displayed after each statement. This promotes open discussion
and drives suggestions for process improvement. The outcome
of the discussion, i.e., the process risk and control issues, are
clearly identified and recorded.
Workshop Facilitator
It is essential that the workshop facilitator is an experienced
IT audit practitioner with strong communication skills. The
quality of information collected in the workshops is dependent
upon the facilitator gaining the participants’ confidence in their
ability to manage the workshop. Further, the facilitator will be
required to clarify and elicit additional information as needed,
based upon the participants’ responses to the statements.
Deliverables
Workshop Documentation
Each workshop requires a resource to document the risk and
control issues from the discussions based upon the participants’
responses to the statements. It is important that the resource
also is an experienced IT audit practitioner as this person will
assist and support the workshop facilitator seeking clarification
of responses that may otherwise have been either misunderstood or not clearly enunciated.
From the responses collected for each workshop, a detailed
worksheet is prepared. The worksheet details the following
information:
• Workshop participants’ names, job titles and contact details
(this will facilitate follow-up)
• Responses to each statement (option finder)
• Risk and control issues discussed
• Analysis of responses (detailed information of process
control issues)
Figure 5
Response
Strongly
Disagree
Disagree
Slightly
Disagree
Neither
Agree
nor
Disagree
Slightly
Agree
Agree
Strongly
Agree
Total No.
of
Responses
2
2
1
1
2
0
0
8
Control Issues
•
•
•
•
Miscategorisation of severity level by help desk staff
Severity levels definitions are too broad
Severity levels definitions are too loosely applied
“Customers” insist on making all issues a severity level 1
Details from
participants
in support
of their
responses
Service Improvement
• Help desk staff to receive greater training to meet operational requirements
• Existing severity level definitions to be reviewed and revised where appropriate
• Guidelines for the application of severity levels to be developed, documented and training
given to staff on application
• Communications plan to be developed to educate users on severity levels
Recommendations
to address issues
Figure 5 is an example of a workshop worksheet with a
participant’s response to a specific statement in relation to
problem management. The statement used for the example is
“There are clear criteria for prioritising a problem.”
Audit Testing
If evidence is required to support the workshop outcomes,
then audit testing may be performed, particularly where the
radar map (refer to the Audit Report section of this article) indicates that the IT&T service provider exceeds the benchmark.
The extent and type of testing performed, i.e., compliance
and/or substantive, are dependent upon the level of comfort
required to support the workshop outcomes. Experience has
shown that if the IT&T service provider has failed to meet or
exceed the benchmark, then additional evidence to support the
workshop outcome is not required.
Action Plans
Action plans are developed based on the outcome of each
workshop. It is essential that these plans are agreed upon and
accepted by the appropriate parties. Figure 6 indicates the
process model utilised in the FSCA.
Figure 6
Audit
IT Service
Provider
Audit
IT Service
Provider
FCSA
Workshop
Workshop
Worksheet
Workshop
Worksheet
Audit Report
& SIP
– Documentation of
control issues
– Determine if tesing
required
– Prepare
recommendations
for improvement
Review Workshop
Docs for accuracy &
completeness.
Response to
recommendations
Review response,
& prepare Service
Improvement Plan
(SIP) & Audit
Report
Review &
approve for issue
Workshop
Worksheet
Workshop
Worksheet
Audit Report
& SIP
Audit Report
& SIP
Figure 7—Service Improvement Plan Process—
Problem Management
Target
Service
Impact
Date
Improvement
H – High
Action
M – Medium
L - Low
30 June
1. Help desk staff are
H
1. Miscategorisation
to receive additional
of severity level by
training to meet
help desk staff
operational
requirements.
1 May
2. Existing severity
M
2. Severity level
level definitions are
definitions are too
to be reviewed and
broad
revised where
appropriate.
30 June
3. Guidelines for the
H
3. Severity level
application of
definitions are too
severity levels are
loosely applied
to be developed,
documented and
training given to
staff on application.
4. A communications
1 May
L
4. “Customers” insist
plan is to be develon making all
oped to educate
issues a severity
users on severity
level 1
levels.
Control/
Service
Issue
Figure 8—Audit Rating Scale
Executive
Management & IT
Service Provider
Service Improvement Plan
A service improvement plan (SIP) provides the necessary
details upon which the IT&T service provider can act. IT provides details of the control issue and the agreed-upon action to
be taken to address the issue and names the designate who has
responsibility for implementing the agreed action and the
target date for implementation (see figure 7).
Audit Report
After working through the control issues and responses with
the IT&T service provider, a report is prepared. A radar map is
used in the executive summary to assist executive management
to gain an understanding of the issues without having to read
all the detail that supports the auditor’s opinion. A radar map
provides a high-level overview assessment of the results of
FCSA.
Rating
Description
5
Unsatisfactory The auditable activity was not in compliance with policies,
systems and procedures. Major risks were identified that
have adversely impacted the auditable activity’s
contribution to the organisation’s strategies.
4
The auditable activity was not always in compliance with
Needs
policies, systems, and procedures. Major risks were
improvement identified which adversely impact or are likely to adversely
impact the auditable activity’s contribution to the
organisation’s core strategies.
3
The auditable activity was generally in compliance with
Average
policies, systems, and procedures. Some control
deficiencies were identified that, if not promptly
corrected, may lead to major risks adversely impacting
the auditable activity’s contribution to the organisation’s
core strategies.
2
The auditable activity was in compliance with
Good
policies, systems, and procedures. Some control
deficiencies were identified but these did not/are not
expected to lead to major risks or adversely impact the
auditable activities contribution to the organisation’s core
strategies.
1
The auditable activity has achieved its goals and
Commendable objectives. The auditable activity was in compliance with
policies, systems, and procedures. No control
deficiencies that would hinder the attainment of goals
and objectives of the organisation were identified.
The workshop results are converted into a standard fivepoint audit scale to provide consistency for all audit reports
within the organisation (see figure 8).
The following is a hypothetical example of what can be
expected from this type of review. An arbitrary audit rating
benchmark is set, say 2.5. The radar map indicates where the
IT&T service provider’s performance is for each ITIL process.
Two colours are used to indicate performance. Green means it
meets or exceeds benchmark, and red means it failed to meet
or achieve the benchmark.
Figure 9 indicates that two processes exceeded the benchmark and identifies a number of processes that require further
remediation or improvement to achieve or exceed the benchmark.
Figure 9
“…Everyone involved in the review has been very positive
about the approach used, and the value of the outcomes….”
This approach should be considered as a key strategy in the
delivery of IT audit services to an organisation.
References
IT Service Management—IT Service Management Forum Ltd
ISBN 0-9524706-1-6
Ken Doughty, CISA, CBCP
is executive audit manager, IT&T Commonwealth Bank
Group. He has more than 20 years’ IS audit experience with
more than 10 years’ business continuity experience in the
public and private sectors. He speaks regularly at seminars and
conferences in Australia and internationally. He also has published papers on IS auditing and business continuity in
Australia and the US. He can be reached at
doughtke@cba.com.au.
Internal Control Framework Assessment
Incident Management
5.00
Security Management
Problem Management
4.00
3.00
Continuity Management
Change Management
2.00
Service Support
Processes
1.00
0.00
Financial Management
John O’Driscoll, CISA, CIA
is executive audit manager, IT&T Commonwealth Bank
Group. He has more than 15 years’ IS audit experience in
the public and private sectors. He regularly speaks at security
and IS audit seminars and conferences and presents training
courses on behalf of ISACA and IIA Sydney chapters. He can
be reached at john.odriscoll@cba.com.au.
Release Management
© Copyright K. Doughty, J. O’Driscoll 2002
Service Delivery
Processes
Capacity Management
Configuration Management
Areas of Risk
Service Level Management
Availability Management
Exceeded Benchmark
Conclusion
The use of FCSA workshops with an industry-recognised
IT best practice process model provides a valid assessment of
the internal control framework of the IT&T environment which
meets the demands of the organisation’s senior executives.
The experience of using FCSA for IT auditing that was documented in this article has resulted in a saving of more than
35 percent in resources and time for this type of audit compared to a traditional approach. While this is impressive, the
biggest benefit resulted from the enthusiastic acceptance of the
approach by the organisation’s IT&T service providers and key
stakeholders. Commitment to resolving control deficiencies
and implementing process improvement has been outstanding
at the workshop participant and senior management level.
Some of the comments the organisation has received regarding the use of this approach include:
“…The processes and tools used encouraged transparency,
collaboration and contribution from all parts of the organisation involved….”
“…The report and related documents are of excellent quality and easily provide a basis for our organisation to progress
with improving critical areas and there were many areas identified that span the end-to-end service delivery process….”