Measuring Control Effectiveness John Mitchell LHS Business Control 47 Grangewood Potters Bar Hertfordshire EN6 1SL England © John Mitchell PhD, MBA, CEng, CITP, FBCS, CFIIA, CISA, CGEIT, QiCA, CFE Tel: +44 (0)7774 145638 john@lhscontrol.com www.lhscontrol.com GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 CMM & ISO 15504 Levels CMM ISO 15504 5 – Optimised 5 - Optimised 4 – Managed and Measurable 4 – Predictable 3 – Defined 3 – Established ________________________________________ 2 – Repeatable 2 - Managed 1 – Ad Hoc 1 - Performed 0 – Non existent 0 - Incomplete © John Mitchel GRC 2.0 - Breaking Down The Silos rd # 2 2014 ISACA Ireland Conference – 3 Slide October Components of the Control Environment Monitoring Control Activity Information & Communication © John Mitchel Risk Analysis Control Objectives GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Generic Risk Management Process H i g h Senior Management Attention Local Management Attention No Action L I K E L I H O O D L o w E Inherent Risk D Likelihood Reduction C B Residual Risk A A Low © John Mitchel Consequence Reduction B C CONSEQUENCE D E High GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Which Risk Would You Want Assurance Over? Inherent Risk Controls Risk 1 None Risk 2 Some Risk 3 Lots © John Mitchel Residual Risk GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 What Is This Control Stuff? Anything which monitors or modifies a process to ensure its predictability A control is basically a test against a prediction You can only test for what you can predict Sometimes the prediction is absolute (gender must be ‘F’) Sometimes the prediction is variable (within the range of 50 to 50,000) © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Control Classifications Class Ability to detect the event and take recovery action 1 Prevents the event, or detects it as it happens and prevents further impact 2 Detects the event and reacts fast enough to fix it well within the specified time window 3 Detects the event and reacts just fast enough to fix it within the specified time window 4 Detects the event but cannot react fast enough to fix it within the specified time window 5 Fails to detect the event but has a partially deployed business continuity plan 6 Fails to detect the event but does have a business continuity plan 7 Fails to detect the event and does not have a business continuity plan © John Mitchel Source: D Brewer & W List Type Preventive Detective Reactive GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Anatomy of a Control Design Implementation Monitoring Evaluation © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Measuring Control Design How well the control should work, in theory, if it is always applied in the way intended: 3 – designed to reduce risk aspect entirely 2 – designed to reduce most aspects of risk 1 – designed to reduce some areas of risk 0 – very limited or badly designed, even where used correctly provides little or no protection © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Measuring Control Implementation The way in which the control performs in practice: 3 – control is always applied as intended 2 – control is generally operational but on occasions is not applied as intended 1 – control is sometimes correctly applied 0 – control is not applied or applied incorrectly © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Measuring Control Monitoring How we know that the control is continuing to operate (embedded monitor): 3 – operation is always monitored 2 – operation is usually monitored, but on occasions is not 1 – operation is monitored on an ad-hoc basis 0 – operation is not monitored at all © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Measuring Control Evaluation How frequently control effectiveness & efficiency is evaluated: 3 – control is regularly evaluated for effectiveness/efficiency 2 – control is occasionally evaluated for effectiveness/efficiency 1 – control is evaluated very infrequently 0 – control is never evaluated GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Scoring Control Effectiveness Example (No Weighting) Apply DIME: Design Implementation Monitoring Evaluation = 2 (3) = 3 (3) = 2 (3) = 1 (3) TOTAL = 8 (12) = 0.75 (75% total effectiveness) NOTE: If either Design, or Implementation is zero then total score becomes zero © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Risk & Control Documentation LHS Business Control 47 Grangewood, Potters Bar, EN6 1SL, England +44 (0)1707 851454 csa@lhscontrol.com www.lhscontrol.com RISK & CONTROL DOCUMENTATION Company: Division: Location: Score the Effectiveness of the Controls in Mitigating the Risk N/A 1 2 3 4 5 Business Area/Activity: A Controls for managing the risk of B As a minimum these should include the following standard controls Is it performed? Contr. Class Contr. Score N/A Yes Who/what performs it? How Often? How is it evidenced? Who/what performs it? How How is it evidenced? Who/what will perform it? How Often? How will it be evidenced? No 1) Control 1 2) Control 2 3) Control 3 4) Control 4 C Where the answer to a minimum requirement is NO: Is it performed? Contr. Class Please give details of any alternative controls providing assurance D Where the score for control effectiveness is < 3 Please detail the control which is to be implemented to improve the result © John Mitchel Contr. Score N/A Contr. Class Yes No Proposed Implementation Date Pot. Score GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Assessing Overall Control Effectiveness Analyse each control to arrive at an overall score for all of the controls mitigating a risk 1 = Poor level of control - management attention required 2 = Very basic control - enhancement required 3 = Adequate level of control - scope for improved effectiveness 4 = Good control - scope for increased efficiency 5 = Excellent control - no improvement possible © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Control Effectiveness Reporting H i g h L I K E L I H O O D L o w E 8 D 2,18 C 16 B 1 A Low B C CONSEQUENCE 12 12) Power Loss 14) 3rd Party Support 17 A © John Mitchel 3,4,5,6,7,9, 10,11,13,14 15 D 15) Loss of Data Centre E High GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Graphical Representation (Multiple Risk Areas) Changes in control over the Internet 4.5 3.5 2.5 1.5 0.5 2010 © John Mitchel 2012 2013 M/F Ops. Network Disaster Rec. Change Control Internet EPOS HR EDI Sys. Dev. Sys. Maint. Help Desk Mngt. Info. Cap. Plan. Tech. Support 17 GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Summary Whether you use CMM or ISO 15504 you still need to assess control effectiveness Evaluation should be against the controls mitigating a risk Evidence must be available that the control is effectively working The evidence must show who/what operates the control and the frequency of operation Control effectiveness can be consistently assessed by applying the DIME method © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014 Questions? John Mitchell PhD, MBA, CEng, CITP, FBCS, CFIIA, CISA, CGEIT, QiCA, CFE LHS Business Control 47 Grangewood Potters Bar Hertfordshire EN6 1SL England Tel: +44 (0)7774 145638 john@lhscontrol.com www.lhscontrol.com © John Mitchel GRC 2.0 - Breaking Down The Silos rd ISACA Ireland Conference – 3 October 2014
