Training in Portugal (1)
EUROSAI IT Working Group - Michel Huissoud
1
Project:
« Design a self-assessment
tool for SAI’s based
on
»
EUROSAI IT Working Group - Michel Huissoud
2
What we want
 Improve IT audit (methodology and
practical approach with CobiT)
 IT Governance (with selfassessment) by the SAIs
EUROSAI IT Working Group - Michel Huissoud
3
1. Genesis of a success story
the Hague, 1 October 2002
EUROSAI IT Working Group - Michel Huissoud
4
Our mandate
The objective of this project is to design and pilot a selfassessment tool for all SAIs. It is based on 'CobiT', which
is a governance (and audit) framework for the domain of
information technology. The self-assessment tool we are
developing should enable us to measure the maturity of
the IT control of our own offices.
EUROSAI IT Working Group - Michel Huissoud
5
2. Why...
... a self-assessment ?
... of Information Technologies ?
... based on CobiT ?
EUROSAI IT Working Group - Michel Huissoud
6
Why a self-assessment?
 It allows « proximity ». The evaluation is
carried out by the people :
–
–
who know the subject
who are interested in solving the problems
 It is confidential. The organization is in control
of the results of the evaluation and their
distribution. Self-assessment is not an audit.
 The extern moderation encourages the
people to speak freely.
EUROSAI IT Working Group - Michel Huissoud
7
Why IT?


As in every organisation or company, it is in the
interest of the SAI to maintain control of its IT system.
The latter is of fundamental importance, whether this
has to do with managing dossiers, planning auditor
tasks, communication or knowledge management.
Issues concerning communication and defining the
roles between the different partners represent one of
the main challenges in IT governance. The SAIs,
together with other enterprises, need better
communication between the sponsors and the IT
specialists.
EUROSAI IT Working Group - Michel Huissoud
8
like the other organisations...

we lose time because of system shutdowns...
 we type the same information in different
systems two or three times...
 we develop projects which don’t meet
expectations...
 we manage expensive service providers...
 we use IT without enough training...
EUROSAI IT Working Group - Michel Huissoud
9
Why “based on CobiT”?
 CobiT is a well accepted standard
 Cobit can be downloaded free from
www.isaca.org
 CobiT is also available in French
www.afai.asso.fr , German www.isaca.ch and
Spanish www.isaca.org
 but our group wanted to be sure that CobiT is the
best choice ...
EUROSAI IT Working Group - Michel Huissoud
10
What have we done?
 Studies of other tools:
 ISO 9001
 European Foundation for Quality Management (EFQM) Excellence
Model
 ITIL / Process Maturity Self-Assessment & Action Plan
 CMM Capability Maturity Model
 Common Assessment Framework (CAF), result of the cooperation
among the EU Ministers responsible for Public Administration
 Contact with specialists:
 Philips, The Netherlands
 Swisslife, Switzerland
 Prof. W. van Grembergen (University of Antwerp, Belgium)
.... our research confirmed the legitimacy of choosing CobiT
EUROSAI IT Working Group - Michel Huissoud
11
3. Looking for the gaps and
use CobiT as a bridge!
...the problem is always by the interface

Management  IT

IT  Audit

IT audit  Financial audit
EUROSAI IT Working Group - Michel Huissoud
12
COBIT includes 36 national
and international standards



Codes of conduct issued by
Council of Europe, OECD,
ISACA, etc.
Qualification criteria for IT
systems and processes:
ITSEC, TCSEC, ISO 9000,
SPICE, TickIT, Common
Criteria, etc.
Professional standards in
internal control and
auditing: COSO Report,
IFAC, AICPA, IIA, ISACA,
PCIE, GAO standards, etc.
EUROSAI IT Working Group - Michel Huissoud



Industry practices and
requirements from industry
forums (ESF, I4) and
government-sponsored
platforms (IBAG, NIST, DTI),
etc.
Technical standards from
ISO, EDIFACT, etc.
Emerging industry-specific
requirements such as from
banking, electronic commerce
and IT manufacturing
13
the three most
important sources:
qualification standards
(ISO, SPICE, ITIL,...)
IT security standards
(ITSEC, BS7799, etc...)
EUROSAI IT Working Group - Michel Huissoud
audit standards (IFAC,
IIA, COSO, GAO, ...)
14
with CobiT,
they can
Control
OBjectives
for Information
communicate
together!...
and Related Technology
EUROSAI IT Working Group - Michel Huissoud
15
Service level
for example
Control Objectives
•The service level agreement
should cover at least the following
aspects: availability, reliability,
performance, capacity for growth,
levels of support provided to users,
continuity planning, security,
minimum acceptable level of
satisfactorily delivered system
functionality, restrictions (limits on
the amount of work), service
charges, central print facilities
(availability), central print
distribution and change
procedures. (...)
EUROSAI IT Working Group - Michel Huissoud
Management Guideline
Key Performance Indicators
•Time lag of resolution of a service level
change request
•Time lag to resolve a service level issue
•Number of times that root cause analysis
of service level procedure and subsequent
resolution is completed within required
period
•Significance of amount of additional
funding needed to deliver the defined
service level (...)
Audit Guideline
•Considering whether recourse process is
identified for non-performance
•Testing that historical performance
against prior service improvement
commitments is tracked (...)
16
or Information
Architecture
Management Guideline
Key Goal Indicators
•(...)
•Reduction of data redundancy
•Increased interoperability between
systems and applications (...)
Control Objectives
•Data Classification Scheme
A general classification framework
should be established with regard
to placement of data in information
classes (i.e., security categories) as
well as allocation of ownership. The
access rules for the classes should
be appropriately defined.(...)
EUROSAI IT Working Group - Michel Huissoud
Audit Guideline
•Considering whether a medium is used to
distribute the data dictionary to ensure
that it is accessible to development areas
and that changes are reflected
immediately
•Identifying data items where ownership is
not clearly and/or appropriately defined.
(...)
17
or manage
the operations
Control Objectives
•Job Scheduling
IT management should ensure that
the continuous scheduling of jobs,
processes and tasks is organised
into the most efficient sequence,
(...). The initial schedules as well as
changes to these schedules should
be appropriately authorised.
•Remote Operations
For remote operations, specific
procedures should ensure that the
connection and disconnection of
the links to the remote site(s) are
defined and implemented..(...)
EUROSAI IT Working Group - Michel Huissoud
Management Guideline
Critical Success Factors
•Changes to job schedules are strictly
controlled
•There are strict acceptance procedures
for new job schedules, including
documentation delivered
•Clear and concise detection, inspection
and escalation procedures are
established(...)
Audit Guideline
•Review of a sample of limited IT
operations and determining whether they
meet policy and procedures requirements.
•Identifying a sample of abnormal ends
(ABENDS) for jobs and determining
resolution of problems which occurred.
(...)
18
CobiT is special



Quality
Cost
Delivery



Confidentiality
Integrity
Availability

Effectiveness and Efficiency of
operations or projects
Reliability of Information
Compliance with laws and regulations


.... this framework goes further than
the other ones!
EUROSAI IT Working Group - Michel Huissoud
19
Navigation in CobiT: How can you select the right process?
« availability » for example
EUROSAI IT Working Group - Michel Huissoud
20
or « human ressources » ?
EUROSAI IT Working Group - Michel Huissoud
21
Warm up…
 Who doesn't know what the EUROSAI IT
Working Group is?
 Who doesn't know what CobiT is?
 Who doesn't know what self-assessment is?
 Is self-assessment a questionnaire or an
interview method?
 Are we looking for problems in efficiency or
in security?
EUROSAI IT Working Group - Michel Huissoud
22
4. Our method
EUROSAI IT Working Group - Michel Huissoud
23
How do we proceed?
Documentation
to study will be
provided on
CobiT, selfassessment,
etc..)
The instructor
will provide
more
information, the
structure of
your business
will be
discussed and
then forms will
be filled in
2 weeks before
EUROSAI IT Working Group - Michel Huissoud
The instructor
will
consolidate
the results
and a
discussion of
the results
will follow
Workshop
An action
plan for the
future will be
prepared
together and
the exercise
will then be
evaluated
The results
of the
workshop
are then
presented to
the top
managemen
t of the SAI
Post ws
24
The problem has 2 dimensions
Planing ans
organisation
Etc…
acquisition and
implementation
business process 1
business process 2
Etc…
Etc...
AI2
AI1
PO2
business process 3
business process 4
business process 5
PO1
first dimension = business
second dimension = IT
business process 6
business process 7
EUROSAI IT Working Group - Michel Huissoud
25
the first form
identify the business process
Business added-value chain
BVC Form 1. Does
the IT help to achieve
the SAI's strategic
goals?
B1
Audit Risk Management
B2
Organise the missions
B3
Analyse the data
B4
Test the IT by the IT-Audit
B5
Report the results to the auditee
B6
Track the implementation of the recommandations
B7
Manage the knowledge
B8
Manage finances and human ressources
B9
Administer and archive the dossiers
B10
Publish the results of the audits
B11
Communicate
EUROSAI IT Working Group - MichelB12
Huissoudother …
B13
other…
What is the quality
of the current IT
systems ?
very low (0)
quality level (1)
quality level (2)
quality level (3)
quality level (4)
very high (5)
What is the
importance of the
future IT systems
for this business
process?
high (5)
importance level (4)
importance level (3)
importance level (2)
low (1)
no application software (0)
high (5)
importance level (4)
importance level (3)
importance level (2)
low (1)
no application software (0)
What is the
importance of the
current IT systems
for this business
process?
In which IT-process (see in Form
2) is the problem (especially if
quality level = 0 or 1)?
26
What do we understand by
“business process”? examples:
Audit Risk Management
Organise the missions
Analyse the data
Test the IT by the IT-
 Manage finances and
Audit
Report the results to the
auditee
Track the implementation
of the recommendations
Manage the knowledge

EUROSAI IT Working Group - Michel Huissoud




human resources
Administer and archive
the dossiers
Publish the results of
the audits
Communicate
Automated data inputs
Automated relations
between different audits
27
EUROSAI IT Working Group - Michel Huissoud
28
then, we evaluate the importance and
the quality of the current IT systems
Business added-value chain
BVC Form 1. Does
the IT help to achieve
the SAI's strategic
goals?
B1
Audit Risk Management
B2
Organise the missions
B3
Analyse the data
What is the quality
of the current IT
systems ?
very low (0)
quality level (1)
quality level (2)
quality level (3)
quality level (4)
very high (5)
What is the
importance of the
future IT systems
for this business
process?
high (5)
importance level (4)
importance level (3)
importance level (2)
low (1)
no application software (0)
high (5)
importance level (4)
importance level (3)
importance level (2)
low (1)
no application software (0)
What is the
importance of the
current IT systems
for this business
process?
In which IT-process (see in Form
2) is the problem (especially if
quality level = 0 or 1)?
Importance Test the IT by the IT-Audit
Quality of
B5
Report the results to the auditee
of theB6 IT Track the implementation the
IT
of the recommandations
B7
Manage the knowledge
systems?
systems?
B4
B8
Manage finances and human ressources
B9
Administer and archive the dossiers
B10
Publish the results of the audits
B11
Communicate
EUROSAI IT Working Group - Michel B12
Huissoud other …
B13
other…
29
the second form
COBIT's Domains and Processes
Planning and Organisation
PO1
PO2
PO3
PO4
Which business processes (see
in Form 1) are affected by this
problem (especially if level = 0 o
1)?
Importance
Define a Strategic IT Plan Quality of
of Define
the IT
the IT
the information architecture
Determine the technological
direction
systems?
systems?
...
Define the IT Organisation and Relationships
EUROSAI IT Working Group - Michel Huissoud
PO5
Maturity level of
the process
non-existent (0)
initial / ad hoc (1)
repeatable but intuitive (2)
defined process (3)
managed and measurable (4)
optimised (5)
CobiT Form 2: What is the maturity level of the
IT-processes?
very important (2)
important (1)
not important (0)
not sure (0)
Importance
of the
process
Manage the IT investment
30
6 maturity levels
EUROSAI IT Working Group - Michel Huissoud
31
Maturity model?
Example: “DS04 Ensure continuous service”
0 Non-existent.
There is no understanding of the risks, vulnerabilities and threats to IT operations or the
impact of loss of IT services to the business. Service continuity is not considered as
needing management attention.
5 Optimised
Integrated continuous service processes are proactive, self-adjusting, automated and
self-analytical and take into account benchmarking and best external practices.
Continuous service plans and business continuity plans are integrated, aligned and
routinely maintained. Buy-in for continuous service needs is secured from vendors and
major suppliers. Global testing occurs and test results are feed back as part of the
maintenance process. Continuous service cost effectiveness is optimized through
innovation and integration. Gathering and analysis of data is used to identify
opportunities for improvement. Redundancy practices and continuous service planning
are fully aligned. Management does not allow single points of failure and provides
support for their remedy. Escalation practices are understood and thoroughly enforced.
EUROSAI IT Working Group - Michel Huissoud
32
Example 2: “PO10 Manage projects”
0 Non-existent.
Project management techniques are not used and the organization does not consider
business impacts associated with project mismanagement and development project
failures.
5 Optimised
A proven, full life-cycle project methodology is implemented and enforced, and is
integrated into the culture of the entire organization. An on-going program to identify and
institutionalize best practices has been implemented. There is strong and active project
support from senior management sponsors as well as stakeholders. IT management has
implemented a project organization structure with documented roles, responsibilities and
staff performance criteria. A long term IT resources strategy is defined to support
development and operational outsourcing decisions. An integrated program
management office is responsible for projects from inception to post implementation.
The program management office is under the management of the business units and
requisitions and directs IT resources to complete projects. Organization-wide planning of
projects ensures that user and IT resources are best utilized to support strategic
initiatives.
EUROSAI IT Working Group - Michel Huissoud
33
matching the results ...
COBIT's Domains and Processes
Planning and Organisation
Business added-value chain
BVC Form 1. Does
the IT help to achieve
the SAI's strategic
goals?
Where are the
What is the quality
of the current IT
systems ?
very low (0)
quality level (1)
quality level (2)
quality level (3)
quality level (4)
very high (5)
What is the
importance of the
future IT systems
for this business
process?
high (5)
importance level (4)
importance level (3)
importance level (2)
low (1)
no application software (0)
high (5)
importance level (4)
importance level (3)
importance level (2)
low (1)
no application software (0)
What is the
importance of the
current IT systems
for this business
process?
PO1
Define a Strategic IT Plan
PO2
Define the information architecture
PO3
Determine the technological direction
PO4
Define the IT Organisation and Relationships
PO5
Manage the IT investment
PO6
Communicate management aims and direction
PO7
Manage human resources
PO8
Ensure compliance with external requirements
PO9
Assess risks
PO10
Manage projects
PO11
Manage quality
AI1
Identify automated solutions
AI2
Acquire and maintain application SW
AI3
Acquire and maintain technology infrastructure
AI4
Develop and maintain procedures
AI5
Install and accredit systems
AI6
Manage changes
Delivery and Support
reasons for the
dissatisfaction?
In which IT-process (see in Form
2) is the problem (especially if
quality level = 0 or 1)?
What impacts do the
IT problems have?
DS1
Define and manage service levels
DS2
Manage third-party services
DS3
Manage performance and capacity
DS4
Ensure continuous service
DS5
Ensure system security
DS6
Identify and allocate costs
DS7
Educate and train users
Audit Risk Management
B2
Organise the missions
B3
Analyse the data
B4
Test the IT by the IT-Audit
DS8
Assist and advise customers
B5
Report the results to the auditee
DS9
Manage the configuration
B6
Track the implementation of the recommandations
DS10
Manage problems and incidents
B7
Manage the knowledge
DS11
Manage data
Manage finances and human ressources
DS12
Manage facilities
DS13
Manage operations
B9
Administer and archive the dossiers
B10
Publish the results of the audits
M1
Monitor the processes
B11
Communicate
M2
Assess internal control adequacy
B12
other …
M3
Obtain independent assurance
B13
other…
M4
Provide for independent audit
EUROSAI IT Working Group - Michel Huissoud
Which business processes (see
in Form 1) are affected by this
problem (especially if level = 0 or
1)?
Acquisition and Implementation
B1
B8
non-existent (0)
initial / ad hoc (1)
repeatable but intuitive (2)
defined process (3)
managed and measurable (4)
optimised (5)
CobiT Form 2: What is the maturity level of the Maturity level of
IT-processes?
the process
very important (2)
important (1)
not important (0)
not sure (0)
Importance
of the
process
Monitoring
34
5. what you get
gaps analysis
a good discussion !
action plan
EUROSAI IT Working Group - Michel Huissoud
35
For example: satisfaction with the IT support
of the business processes
B10
confidential....
B12
confidential....
B6
confidential...
B5
confidential...
B3
confidential...
B4
confidential...
B1
confidential...
B9
confidential...
B7
confidential...
B2
confidential...
B11
confidential...
B8
confidential...
EUROSAI IT Working Group - Michel Huissoud
2.29
2.00
1.75
1.38
1.33
1.29
1.00
1.00
0.86
0.83
0.60
0.00
36
identification of the problems
(business point of view)
6
5
4
What is the quality of the
current IT systems ?
3
What is the importance of
the future IT systems ?
2
1
B13
B12
B11
B10
B9
B8
B7
B6
B5
B4
B3
B2
B1
0
process
EUROSAI IT Working Group - Michel Huissoud
37
identifying the problems
(IT point of view)
Importance of the process ?
Maturity level of the process ?
6
5
4
3
2
1
process
EUROSAI IT Working Group - Michel Huissoud
38
M4
M3
M2
M1
DS13
DS12
DS11
DS10
DS9
DS8
DS7
DS6
DS5
DS4
DS3
DS2
DS1
AI6
AI5
AI4
AI3
AI2
AI1
PO11
PO10
PO9
PO8
PO7
PO6
PO5
PO4
PO3
PO2
PO1
0
An action plan
Findings and Actions form
Finding/
Description
Gap Risk / implication
Recommendation / action Person in
description
charge
EUROSAI IT Working Group - Michel Huissoud
Deadline for
finishing
activity
Priority
1-10
39
and perhaps in the future:
a benchmarking
Big SAIs
Middle SAIs
Small SAIs
EUROSAI IT Working Group - Michel Huissoud
40
A reasonable time management (first day)
14.00
Start of the workshop
15.00
Identify the business processes
15.30
Coffee break
(moderator) Adaptation of the form 1 and print them
16.00
Fill form 1
16.15
Presentation CobiT
17.15
Select the most important IT processes
18.00
Fill form 2
18.30
End of the first day
Then, put the results in your EXCEL sheet, prepare
the presentation of the results and the
discussion of tomorrow…
EUROSAI IT Working Group - Michel Huissoud
41
A reasonable time management (second day)
09.00
09.30
10.00
10.15
10.45
11.30
12.30
15.00
Presentation of the results
Discussion (validation of the results, looking
for consensus)
Listing the most important problems and
strengths
Coffee break
Prepare an action plan
Fill the evaluation forms
Finalization of the action plan
Discussion and end of the workshop
Preparation of the final presentation
Presentation and discussion with the head of
the SAI
Write the evaluation report!
EUROSAI IT Working Group - Michel Huissoud
42
We will now focus
on the following points
Get the
right
Documentation to
persons!
study will be
provided on
CobiT, selfassessment, etc..)
Identify
The instructor the
will provide
more
processes!
information, the
structure of
your business
will be
discussed and
then forms will
be filled in
2 weeks before
EUROSAI IT Working Group - Michel Huissoud
The instructor
will
consolidate
the results
and a
discussion of
the results
will follow
Use the
EXCEL sheet
correctly!
An action
plan for the
future will be
prepared
together and
the exercise
will then be
evaluated
Ask the
right
questions!
Workshop
Get a
good
action
The results
plan!
of the
workshop
are then
presented to
the top
managemen
t of the SAI
Post ws
43