Uploaded by ashutosh.iiitm

CMMI

advertisement
Measuring Control Effectiveness
John Mitchell
LHS Business Control
47 Grangewood
Potters Bar
Hertfordshire
EN6 1SL
England
© John Mitchell
PhD, MBA, CEng, CITP, FBCS, CFIIA, CISA, CGEIT, QiCA, CFE
Tel: +44 (0)7774 145638
john@lhscontrol.com
www.lhscontrol.com
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
CMM & ISO 15504 Levels
CMM
ISO 15504
5 – Optimised
5 - Optimised
4 – Managed and Measurable
4 – Predictable
3 – Defined
3 – Established
________________________________________
2 – Repeatable
2 - Managed
1 – Ad Hoc
1 - Performed
0 – Non existent
0 - Incomplete
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
# 2 2014
ISACA Ireland Conference – 3 Slide
October
Components of the Control Environment
Monitoring
Control
Activity
Information &
Communication
© John Mitchel
Risk
Analysis
Control
Objectives
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Generic Risk Management Process
H
i
g
h
Senior Management
Attention
Local Management
Attention
No Action
L
I
K
E
L
I
H
O
O
D
L
o
w
E
Inherent
Risk
D
Likelihood
Reduction
C
B
Residual
Risk
A
A
Low
© John Mitchel
Consequence
Reduction
B
C
CONSEQUENCE
D
E
High
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Which Risk
Would You Want Assurance Over?
Inherent Risk
Controls
Risk 1
None
Risk 2
Some
Risk 3
Lots
© John Mitchel
Residual
Risk
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
What Is This Control Stuff?
Anything which monitors or modifies a process to ensure its
predictability
A control is basically a test against a prediction
You can only test for what you can predict
Sometimes the prediction is absolute
(gender must be ‘F’)
Sometimes the prediction is variable
(within the range of 50 to 50,000)
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Control Classifications
Class
Ability to detect the event and take recovery action
1
Prevents the event, or detects it as it happens and prevents further
impact
2
Detects the event and reacts fast enough to fix it well within the
specified time window
3
Detects the event and reacts just fast enough to fix it within the
specified time window
4
Detects the event but cannot react fast enough to fix it within the
specified time window
5
Fails to detect the event but has a partially deployed business
continuity plan
6
Fails to detect the event but does have a business continuity plan
7
Fails to detect the event and does not have a business continuity
plan
© John Mitchel
Source: D Brewer & W List
Type
Preventive
Detective
Reactive
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Anatomy of a Control
Design
Implementation
Monitoring
Evaluation
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Measuring Control Design
How well the control should work, in theory, if it is always
applied in the way intended:
3 – designed to reduce risk aspect entirely
2 – designed to reduce most aspects of risk
1 – designed to reduce some areas of risk
0 – very limited or badly designed, even where used correctly
provides little or no protection
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Measuring Control Implementation
The way in which the control performs in practice:
3 – control is always applied as intended
2 – control is generally operational but on occasions is not
applied as intended
1 – control is sometimes correctly applied
0 – control is not applied or applied incorrectly
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Measuring Control Monitoring
How we know that the control is continuing to operate
(embedded monitor):
3 – operation is always monitored
2 – operation is usually monitored, but on occasions is not
1 – operation is monitored on an ad-hoc basis
0 – operation is not monitored at all
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Measuring Control Evaluation
How frequently control effectiveness & efficiency is
evaluated:
3 – control is regularly evaluated for effectiveness/efficiency
2 – control is occasionally evaluated for
effectiveness/efficiency
1 – control is evaluated very infrequently
0 – control is never evaluated
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Scoring Control Effectiveness Example
(No Weighting)
Apply DIME:
Design
Implementation
Monitoring
Evaluation
= 2 (3)
= 3 (3)
= 2 (3)
= 1 (3)
TOTAL
= 8 (12) = 0.75 (75% total effectiveness)
NOTE: If either Design, or Implementation is zero then
total score becomes zero
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Risk & Control Documentation
LHS Business Control
47 Grangewood, Potters Bar, EN6 1SL, England
+44 (0)1707 851454
csa@lhscontrol.com
www.lhscontrol.com
RISK & CONTROL DOCUMENTATION
Company:
Division:
Location:
Score the Effectiveness of the
Controls in Mitigating the Risk
N/A
1
2
3
4
5
Business Area/Activity:
A
Controls for managing the risk of
B
As a minimum these should include the
following standard controls
Is it performed?
Contr.
Class
Contr.
Score
N/A
Yes
Who/what performs it?
How
Often?
How is it evidenced?
Who/what performs it?
How
How is it evidenced?
Who/what will perform
it?
How
Often?
How will it be evidenced?
No
1) Control 1
2) Control 2
3) Control 3
4) Control 4
C
Where the answer to a minimum requirement is
NO:
Is it performed?
Contr.
Class
Please give details of any alternative controls
providing assurance
D
Where the score for control effectiveness is < 3
Please detail the control which is to be
implemented to improve the result
© John Mitchel
Contr.
Score
N/A
Contr.
Class
Yes
No
Proposed
Implementation
Date
Pot.
Score
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Assessing Overall
Control Effectiveness
Analyse each control to arrive at an overall score for all of
the controls mitigating a risk
1 = Poor level of control - management attention required
2 = Very basic control - enhancement required
3 = Adequate level of control - scope for improved
effectiveness
4 = Good control - scope for increased efficiency
5 = Excellent control - no improvement possible
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Control Effectiveness Reporting
H
i
g
h
L
I
K
E
L
I
H
O
O
D
L
o
w
E
8
D
2,18
C
16
B
1
A
Low
B
C
CONSEQUENCE
12
12) Power
Loss
14) 3rd Party
Support
17
A
© John Mitchel
3,4,5,6,7,9,
10,11,13,14
15
D
15) Loss of
Data Centre
E
High
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Graphical Representation
(Multiple Risk Areas)
Changes in control over the Internet
4.5
3.5
2.5
1.5
0.5
2010
© John Mitchel
2012
2013
M/F Ops.
Network
Disaster Rec.
Change Control
Internet
EPOS
HR
EDI
Sys. Dev.
Sys. Maint.
Help Desk
Mngt. Info.
Cap. Plan.
Tech. Support
17
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Summary

Whether you use CMM or ISO 15504 you still need to assess control
effectiveness

Evaluation should be against the controls mitigating a risk

Evidence must be available that the control is effectively working

The evidence must show who/what operates the control and the
frequency of operation

Control effectiveness can be consistently assessed by applying the
DIME method
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Questions?
John Mitchell
PhD, MBA, CEng, CITP, FBCS, CFIIA, CISA, CGEIT, QiCA, CFE
LHS Business Control
47 Grangewood
Potters Bar
Hertfordshire EN6 1SL
England
Tel: +44 (0)7774 145638
john@lhscontrol.com
www.lhscontrol.com
© John Mitchel
GRC 2.0 - Breaking Down
The Silos
rd
ISACA Ireland Conference – 3 October 2014
Download