CERT® Resilience Management Model
CERT-RMM Overview
David White
CERT Resilient Enterprise Management Team
© 2011 Carnegie Mellon University
Notices
NO WARRANTY
THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS
FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF
ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED
TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS
OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY
WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR
COPYRIGHT INFRINGEMENT.
Use of any trademarks in this presentation is not intended in any way to infringe on the rights of the
trademark holder.
This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or
electronic form without requesting formal permission. Permission is required for any other use. Requests
for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu.
This work was created in the performance of Federal Government Contract Number FA8721-05-C-0003
with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded
research and development center. The Government of the United States has a royalty-free governmentpurpose license to use, duplicate, or disclose the work, in whole or in part and in any manner, and to have
or permit others to do so, for government purposes pursuant to the copyright license under the clause at
252.227-7013.
© 2011 Carnegie Mellon University
2
CERT | Software Engineering Institute | Carnegie Mellon
Software Engineering Institute (SEI)
•
Federally funded research and development center
based at Carnegie Mellon University
•
Basic and applied research in partnership with
government and private organizations
•
Helps organizations improve development,
operation, and management of software-intensive
and networked systems
CERT – Anticipating and solving our
nation’s cybersecurity challenges
•
Largest technical program at SEI
•
Focused on internet security, secure systems,
operational resilience, and coordinated response to
security issues
© 2011 Carnegie Mellon University
3
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
4
Operational resilience
and operational risk
Setting context
© 2011 Carnegie Mellon University
5
Operational resilience defined
Resilience: The physical property of a
material when it can return to its original
shape or position after deformation that
does not exceed its elastic limit
[wordnet.princeton.edu]
Operational resilience: The
emergent property of an organization
that can continue to carry out its
mission in the presence of
operational stress and disruption
that does not exceed its limit
[CERT-RMM]
Where does the stress and disruption come from? Risk.
© 2011 Carnegie Mellon University
6
Operational resilience and operational risk
Operational resilience emerges from effective operational
risk management
Operational risk categories:
Actions of
people
Systems
and
technology
failures
Failed
internal
processes
External
events
© 2011 Carnegie Mellon University
7
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
8
CERT® Resilience Management
Model (CERT-RMM)
A platform for improvement and measurement
© 2011 Carnegie Mellon University
9
What is CERT®-RMM?
CERT-RMM is a capability
model for managing and
improving operational
resilience.
• Guides implementation and
management of operational
resilience activities
• Converges key operational risk
management activities: security,
BC/DR, and IT operations
• Defines maturity through
“…an extensive superset of the things an
organization could do to
be more resilient.”
- CERT-RMM adopter
capability levels (like CMMI)
• Enables measurement
• Improves confidence in how an
organization responds in times
of operational stress
© 2011 Carnegie Mellon University
10
Imperatives for building CERT-RMM
Increasingly complex operational
environments; traditional approaches failing
Siloed nature of operational risk activities; a
lack of convergence
Tech reliance
Global economy
Overreliance on technical approaches
Open boundaries
Complexity
Cultural shifts
Lack of common language or taxonomy
Lack of means to measure organizational
capability
Inability to confidently predict outcomes,
behaviors, and performance under times
of stress
© 2011 Carnegie Mellon University
11
CERT-RMM background
Collaboration
with high
maturity
organizations
800+
practices for
security, BC,
& IT ops
20+ years of
security mgmt
knowledge at
CERT
CERTRMM
CMMI
architecture
and
experience
Piloting in
private and
government
organizations
© 2011 Carnegie Mellon University
12
Organizational context
Organization
Mission
Service
Service
Service
Mission
Mission
Mission
Assets in Production
people
information technology
facilities
Four asset types:
• People – the human capital of the organization
• Information – data, records, knowledge in physical or digital form
• Technology – software, systems, hardware, network
• Facilities – offices, data centers, labs – the physical places
© 2011 Carnegie Mellon University
13
Organizational context - disruption
Organization
Mission
Service
Service
Service
Mission
Mission
Mission
people
info
tech
facilities
Operational risk can disrupt an asset
And lead to organizational disruption
© 2011 Carnegie Mellon University
14
Building resilience at the asset level
tech
Protect
Security Domain
Sustain
BC/DR Domain
Protection strategies
Sustainment strategies
Keep assets from
exposure to disruption
Keep assets productive
during adversity
Typically implemented as
“security” activities
Typically implemented as
“business continuity” activities
© 2011 Carnegie Mellon University
15
Building resilience at the asset
tech
Protect
Sustain
Security Domain
BC/DR Domain
The optimal “mix” of
these strategies
depends on the value
of the asset and the
cost of deploying
and maintaining the
strategy.
Manage Risk
Manage Condition
Manage Consequence
© 2011 Carnegie Mellon University
16
Organizational context
Organization
Mission
Service
Service
Service
Mission
Mission
Mission
people
Sustain
Protect
info
Sustain
Protect
tech
Sustain
Protect
facilities
Sustain
Protect
Operational Resilience
Management System
CERT-RMM
focuses here
© 2011 Carnegie Mellon University
17
Resilience management in the life cycle
Resilience management covers the life cycle of an asset.
Operational resilience management focuses on the deploy,
operate, and decommission phases, but must reach back to
address issues during development.
Design
Plan
Develop
Deploy
Operate
Retire
Acquire
Asset in Production
© 2011 Carnegie Mellon University
18
CERT-RMM position in life cycle
Design
Plan
Develop
Deploy
Operate
Retire
Acquire
CERT-RMM
CMMI-DEV
CMMI-ACQ
CMMI-SVC
DEVELOPMENT
OPERATION
© 2011 Carnegie Mellon University
19
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
20
CERT-RMM Architecture
How the model is put together
© 2011 Carnegie Mellon University
21
CERT-RMM: 26 process areas in 4 categories
Engineering
Operations Management
ADM
Asset Definition and Management
AM
Access Management
CTRL
Controls Management
EC
Environmental Control
RRD
Resilience Requirements Development
EXD
External Dependencies Management
RRM
Resilience Requirements Management
ID
Identity Management
RTSE
Resilient Technical Solution Engineering
IMC
Incident Management & Control
SC
Service Continuity
KIM
Knowledge & Information Management
PM
People Management
Enterprise Management
TM
Technology Management
COMM
Communications
VAR
Vulnerability Analysis & Resolution
COMP
Compliance
EF
Enterprise Focus
Process Management
FRM
Financial Resource Management
MA
Measurement and Analysis
HRM
Human Resource Management
MON
Monitoring
OTA
Organizational Training & Awareness
OPD
Organizational Process Definition
RISK
Risk Management
OPF
Organizational Process Focus
© 2011 Carnegie Mellon University
22
CERT-RMM process area architecture
Process
Area
Focused Activity
What to do to achieve
the capability
How to accomplish the goal
How to implement the practice
Points of connection to other
practice bodies
Specific
Goals
Three
Generic
Goals
Specific
Practices
Generic
Practices
Subpractices
Subpractices
Maturity
Elements
© 2011 Carnegie Mellon University
23
CERT-RMM links to codes of practice
Process
Area
Codes of Practice:
BS25999-1:2006
CMMI v1.2
CMMI for Services
Specific
Goals
CobiT 4.1
COSO ERM
DRII GAP
FFIEC Handbooks (Security, BCP)
Specific
Practices
ISO 20000-2:2005(E) (ITIL-related)
ISO 24762:2008(E)
ISO 27002:2005
NFPA 1600 (2007)
Subpractices
PCI DSS v1.1
Val-IT
© 2011 Carnegie Mellon University
24
CERT-RMM numbers
© 2011 Carnegie Mellon University
25
Where to start
To use the model, start by selecting any number of process
areas (or even parts of process areas) that align with your
objectives.
Starting with 1 process area or a few specific goals is
completely acceptable.
There is no requirement to use the entire model—use
whatever parts of the model make sense for your
situation.
© 2011 Carnegie Mellon University
26
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
27
Measuring maturity — the
CERT-RMM capability dimension
Measuring process institutionalization to
determine capability under stress
© 2011 Carnegie Mellon University
28
Institutionalization
What does institutionalization look like?
It describes when something has become ingrained in the way
an organization operates.
”institutionalize.” Dictionary.cambridge.org Advanced Learner's Dictionary. Cambridge University
Press. 14 Sep. 2010. <http://dictionary.cambridge.org/dictionary/british/institutionalize_2>.
© 2011 Carnegie Mellon University
29
Process institutionalization in CERT-RMM
Capability levels are used in CERT-RMM to measure process institutionalization
Processes are
acculturated,
defined,
measured,
and
governed
Practices are
performed
Level 3
• Defined
• Managed
Higher degrees of
institutionalization
translate to more
stable processes that
Level 1
• produce consistent
Level 2
• Performed
• are retained during
Level 0
Practices are
incomplete
results over time
times of stress
• Incomplete
© 2011 Carnegie Mellon University
30
Capability Levels and Generic Goals
Capability levels apply independently to each process area
• An organization could target level 1 in one process area and level 3
in another
• Provides for very flexible application of the model
Generic goals define capability levels:
To achieve:
An organization must satisfy:
Capability Level 1
Generic Goal 1
Capability Level 2
Generic Goals 1 and 2
Capability Level 3
Generic Goals 1, 2, and 3
© 2011 Carnegie Mellon University
31
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
32
COMP: Compliance
One process area in-depth
© 2011 Carnegie Mellon University
33
COMP – Compliance process area
Purpose: ensure awareness of and compliance with an established
set of relevant internal and external guidelines, standards, practices,
policies, regulations, and legislation, and other obligations (such as
contracts and service level agreements) related to managing
operational resilience
Collect once — comply many times
•
Data collection is one of the most expensive activities for compliance
•
Understand intersecting requirements to leverage compliance data
•
Develop a compliance knowledgebase with strong data validation
© 2011 Carnegie Mellon University
34
Compliance: specific goals & practices
Specific Goals
Specific Practices
COMP:SG1
SG1.SP1: Establish a compliance plan
Prepare for compliance SG1.SP2: Establish a compliance program
management
SG1.SP3: Establish compliance guidelines and
standards
COMP:SG2
Establish compliance
obligations
SG2.SP1: Identify compliance obligations
SG2.SP2: Analyze obligations
SG2.SP3: Establish ownership for meeting obligations
COMP:SG3
SG3.SP1: Collect and validate compliance data
Demonstrate
SG3.SP2: Demonstrate the extent of compliance
satisfaction of
obligation satisfaction
compliance obligations
SG3.SP3: Remediate areas of non-compliance
COMP:SG4
Monitor compliance
activities
SG4.SP1: Evaluate compliance activities
© 2011 Carnegie Mellon University
35
Achieving capability level 1 in COMP
Generic Goals
Generic Practices
GG1 Achieve Specific Goals√ GG1.GP1
Perform Specific Practices
√
Achieve capability level 1 by satisfying generic goal 1,
which means:
• Perform the COMP specific practices (all 10 of them) so that you
• Satisfy the COMP specific goals (all 4 of them)
© 2011 Carnegie Mellon University
36
Achieving capability level 2 in COMP
Generic Goals
Generic Practices
GG1 Achieve Specific Goals√ GG1.GP1
GG2 Institutionalize
a Managed Process
Perform Specific Practices
√
Establish Process Governance
√
GG2.GP2
Plan the Process
√
GG2.GP3
Provide Resources
√
GG2.GP4
Assign Responsibility
√
GG2.GP5
Train People
√
GG2.GP6
Manage Work Product Configurations
√
GG2.GP7
Identify and Involve Relevant Stakeholders √
GG2.GP8
Monitor and Control the Process
√
GG2.GP9
Objectively Evaluate Adherence
√
√ GG2.GP1
GG2.GP10 Review Status with Higher Level Managers √
Achieve capability level 1 plus satisfy generic goal 2 by
performing the associated 10 generic practices.
© 2011 Carnegie Mellon University
37
Achieving capability level 3 in COMP
Generic Goals
Generic Practices
GG1 Achieve Specific Goals√ GG1.GP1
GG2 Institutionalize
a Managed Process
Perform Specific Practices
√
Establish Process Governance
√
GG2.GP2
Plan the Process
√
GG2.GP3
Provide Resources
√
GG2.GP4
Assign Responsibility
√
GG2.GP5
Train People
√
GG2.GP6
Manage Work Product Configurations
√
GG2.GP7
Identify and Involve Relevant Stakeholders √
GG2.GP8
Monitor and Control the Process
√
GG2.GP9
Objectively Evaluate Adherence
√
√ GG2.GP1
GG2.GP10 Review Status with Higher Level Managers √
GG3 Institutionalize
a Defined Process
√ GG3.GP1
GG3.GP2
Establish a Defined Process
√
Collect Improvement Information
√
© 2011 Carnegie Mellon University
38
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
39
SC: Service Continuity
One process area in-depth
© 2011 Carnegie Mellon University
40
SC – Service Continuity
Purpose:
To ensure the continuity of
essential operations of services
and related assets if a
disruption occurs as a result of
an incident, disaster, or other
disruptive event.
Contains
• 7 specific goals
• 20 specific practices
• ~40 pages
© 2011 Carnegie Mellon University
41
SC specific goals 1-3 and practices
Specific Goals
Specific Practices
SG1 Prepare for Service
Continuity
SG1.SP1 Plan for Service Continuity
SG2 Identify and Prioritize
High-Value Services
SG2.SP1 Identify the Organization’s High-Value Services
SG1.SP2 Establish Standards and Guidelines for Service
Continuity
SG2.SP2 Identify Internal and External Dependencies and
Interdependencies
SG2.SP3 Identify Vital Organizational Records and
Databases
SG3 Develop Service
Continuity Plans
SG3.SP1 Identify Plans to be Developed
SG3.SP2 Develop and Document Service Continuity Plans
SG3.SP3 Assign Staff to Service Continuity Plans
SG3.SP4 Store and Secure Service Continuity Plans
SG3.SP5 Develop Service Continuity Plan Training
© 2011 Carnegie Mellon University
42
SC specific goals 4-7 and practices
Specific Goals
Specific Practices
SG4 Validate Service
Continuity Plans
SG4.SP1 Validate Plans to Requirements and Standards
SG5 Exercise Service
Continuity Plans
SG5.SP1 Develop Testing Program and Standards
SG4.SP2 Identify and Resolve Plan Conflicts
SG5.SP2 Develop and Document Test Plans
SG5.SP3 Exercise Plans
SG5.SP4 Evaluate Plan Test Results
SG6 Execute Service
Continuity Plans
SG6.SP1 Execute Plans
SG7 Maintain Service
Continuity Plans
SG7.SP1 Establish Change Criteria
SG6.SP2 Measure the Effectiveness of the Plans in
Operation
SG7.SP2 Maintain Changes to Plans
© 2011 Carnegie Mellon University
43
Achieving capability level 1 in SC
Generic Goals
Generic Practices
GG1 Achieve Specific Goals√ GG1.GP1
Perform Specific Practices
√
Achieve capability level 1 by satisfying generic goal 1,
which means:
• Perform the SC specific practices (all 20 of them) so that you
• Satisfy the SC specific goals (all 7 of them)
© 2011 Carnegie Mellon University
44
Achieving capability level 2 in SC
Generic Goals
Generic Practices
GG1 Achieve Specific Goals√ GG1.GP1
GG2 Institutionalize a
Managed Process
Perform Specific Practices
√
Establish Process Governance
√
GG2.GP2
Plan the Process
√
GG2.GP3
Provide Resources
√
GG2.GP4
Assign Responsibility
√
GG2.GP5
Train People
√
GG2.GP6
Manage Work Product Configurations
√
GG2.GP7
Identify and Involve Relevant Stakeholders √
GG2.GP8
Monitor and Control the Process
√
GG2.GP9
Objectively Evaluate Adherence
√
√ GG2.GP1
GG2.GP10 Review Status with Higher-Level Managers √
Achieve capability level 1 plus satisfy generic goal 2 by
performing the associated 10 generic practices.
© 2011 Carnegie Mellon University
45
Achieving capability level 3 in SC
Generic Goals
Generic Practices
GG1 Achieve Specific Goals√ GG1.GP1
GG2 Institutionalize a
Managed Process
Perform Specific Practices
√
Establish Process Governance
√
GG2.GP2
Plan the Process
√
GG2.GP3
Provide Resources
√
GG2.GP4
Assign Responsibility
√
GG2.GP5
Train People
√
GG2.GP6
Manage Work Product Configurations
√
GG2.GP7
Identify and Involve Relevant Stakeholders √
GG2.GP8
Monitor and Control the Process
√
GG2.GP9
Objectively Evaluate Adherence
√
√ GG2.GP1
GG2.GP10 Review Status with Higher-Level Managers √
GG3 Institutionalize a
Defined Process
√ GG3.GP1
GG3.GP2
Establish a Defined Process
√
Collect Improvement Information
√
© 2011 Carnegie Mellon University
46
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
47
Using CERT-RMM
A process for improvement
© 2011 Carnegie Mellon University
48
Using CERT-RMM for improvement
Recognize
Objective
Evaluate
Results
Determine
Scope
Implement
Changes
Identify
Gaps
Analyze
Gaps
© 2011 Carnegie Mellon University
49
Recognize
Objective
Recognizing objectives
Objectives frame and provide context
Evaluate
Results
Determine
Scope
Implement
Changes
Identify
Gaps
Analyze
Gaps
Answer the question: What are we trying to accomplish with
the improvement effort?
Typical themes:
• Are we doing all that we should to manage business continuity (or
security, IT ops, or a combination)?
• How can we minimize the potential disruption from <some known
risk or category of risk>?
• How can we improve the efficiency, effectiveness, or consistency of
our operational risk management activities (security, BC, & IT ops)?
• Do our policies and guidelines produce the risk management
activities that we want them to? How can we improve policy?
© 2011 Carnegie Mellon University
50
Recognize
Objective
Determining scope
Two elements:
Evaluate
Results
Determine
Scope
Implement
Changes
Identify
Gaps
Analyze
Gaps
• Organizational scope:
On which part of the organization will we focus?
• Model scope:
Which parts of the CERT-RMM will we use?
—
—
Whole process areas (1-6 typically)
Parts of process areas (a set of practices)
Both elements should align with objectives and sponsorship
Model scoping can be easily accomplished by walking the
model outline in a small workshop or meeting
© 2011 Carnegie Mellon University
51
Organizational scope
Determine
Scope
Where, in the organization,
process improvement will be
focused
Must consider
• Span of sponsorship
developed in Initiating phase
• Span of authority of the
improvement team
• Schedule feasibility for
desired improvements
© 2011 Carnegie Mellon University
52
Organizational scoping example -1
1
1.1
1.1.1
1.1.1.1
1.1.2.1
Organizational Unit
1.2
1.1.2
1.1.2.2
Determine
Scope
1.3
1.2.1
1.3.1
1.2.1.1
1.3.1.1
1.3.2
1.3.2.1
1.3.3
1.3.2.2
1.3.3.1
Suppose that we are performing process improvement on the part
of the organization defined by 1.3 and its subunits
First, we have to understand where the CERT-RMM practices are
performed or designate where they will be performed
© 2011 Carnegie Mellon University
53
Determine
Scope
Model scope
Determines which areas of the model will be selected for
process improvement
When selecting, consider process areas that
• May be causing “pain” or perceived weakness
• Align with regulatory or industry initiatives and objectives
• Align with organizational objectives or initiatives
• Support other organizational process improvement initiatives such
as Six Sigma or ITIL
• Explore areas in which the organization needs to develop
competency
© 2011 Carnegie Mellon University
54
CERT-RMM model scope in detail -1
Determine
Scope
Process
Areas
People
Capability
Level Targets
Information
Asset Scope
Technology
Model Scope
Facilities
Business
Continuity
Resilience
Scope
Security
IT Operations
© 2011 Carnegie Mellon University
55
CERT-RMM model scope in detail -2
Process
Areas
Determine
Scope
Fine-grained model scoping options
People
Capability
Level Targets
Information
Asset Scope
Technology
Model Scope
Facilities
Business
Continuity
Resilience
Scope
Security
IT Operations
© 2011 Carnegie Mellon University
56
Determine
Scope
PA-level scope example
Capability Profile
Scoping Caveats
Information and technology assets only
ADM
COMP
Information security compliance only
IMC
Information security incidents only
KIM
None
TM
None
0
1
2
3
© 2011 Carnegie Mellon University
57
CERT-RMM model scope in detail -3
Determine
Scope
Fine-grained model scoping options
Specific &
Generic Practices
People
Information
Asset Scope
Technology
Model Scope
Facilities
Business
Continuity
Resilience
Scope
Security
IT Operations
© 2011 Carnegie Mellon University
58
Determine
Scope
Practice-level scope example
Example scope for IT Disaster Recovery activities.
Note: PAs with no selected practices are hidden.
© 2011 Carnegie Mellon University
59
Recognize
Objective
Identifying gaps
Methods:
Evaluate
Results
Determine
Scope
Implement
Changes
Identify
Gaps
Analyze
Gaps
Rigorous: CERT-RMM Capability Appraisals
• Three classes: A (most rigorous), B, and C (least)
• Outputs include detailed practice-level characterizations and written
findings statements
Lightweight: CERT-RMM Compass
• Questionnaire-based gap analysis instrument from CERT
• In development now
Informal: gap analysis roundtable or workshop
• Assemble a group of internal experts
• Informally evaluate the organization’s implementation of the model
practices in a workshop setting
© 2011 Carnegie Mellon University
60
CERT-RMM appraisal comparison
Process
Area
Generic
Goals
Specific
Practices
Generic
Practices
Resource requirements:
Effort
Appraisal team:
Depth of investigation:
Class A
Class B
Class C
Capability Level
Ratings
--
--
(Satisfied or
Not Satisfied)
--
--
Characterization of
implementation on
5-point scale
Characterization
of approach on
3-point scale
Characterization
of intent on 3point scale
(Fully, Largely, Partially,
Not, Not Yet Implemented)
(High, medium, low)
(High, medium, low)
Findings statements
Statements
Statements
(strengths & weaknesses)
(strength/weakness)
(strength/weakness)
4 or more
2 or more
1 or more
High
Medium
Low
High
Medium
Low
(0, 1, 2, or 3)
Model-Related Outputs
Specific
Goals
Identify Gaps
Goal Ratings
© 2011 Carnegie Mellon University
61
CERT-RMM appraisal comparison
Specific
Goals
Generic
Goals
Specific
Practices
Generic
Practices
Model-Related Outputs
Process
Area
Depth of investigation:
Resource requirements:
Effort
Appraisal team:
Identify Gaps
Class A
Class B
Class C
Capability Level
Scoped at the
Ratings
process
area level
(0, 1, 2, or 3)
--
--
--
--
Goal Ratings
(Satisfied or
Not Satisfied)
Characterization of
implementation on
5-point scale
(Fully, Largely, Partially,
Not, Not Yet Implemented)
Characterization
Characterization
of approach on
of intent on 33-point scale
point scale
May be
scoped
practice
level
(High,
medium,
low) at the(High,
medium,
low)
Findings statements
Statements
Statements
(strengths & weaknesses)
(strength/weakness)
(strength/weakness)
4 or more
2 or more
1 or more
High
Medium
Low
High
Medium
Low
© 2011 Carnegie Mellon University
62
Sample class B/C scope
Identify Gaps
Example scope for IT Disaster Recovery activities.
Note: PAs with no practices in scope are hidden.
© 2011 Carnegie Mellon University
64
Sample class B/C appraisal output
Identify Gaps
For IT Disaster Recovery activities:
Note: PAs with no practices in scope are hidden.
© 2011 Carnegie Mellon University
65
Sample class A appraisal output
Identify Gaps
© 2011 Carnegie Mellon University
66
A appraisals must
be scoped
Sample class A Class
appraisal
output
to include full process areas
Identify Gaps
Class A appraisals
include goal ratings
Class A appraisals include
Capability Level ratings. These
results would yield Capability
Level 0 because at least one
specific goal is not satisfied.
© 2011 Carnegie Mellon University
67
Sample appraisal findings
Identify Gaps
Strengths
• The service continuity testing program is complete, rigorous, well-
implemented, consistently-followed, and provides valuable feedback
for the improvement of preparedness activities across the
organization.
• …
Weaknesses
• Internal dependencies are well-identified in support of service
continuity planning, but external dependencies are not.
• While service continuity plans are being executed appropriately in
the organization, no evidence was provided to show that plans are
being evaluated for their effectiveness in operation.
• …
© 2011 Carnegie Mellon University
68
Sample appraisal findings
Identify Gaps
Strengths
• The service continuity testing program is complete, rigorous, well-
implemented, consistently-followed, and provides valuable feedback
for the improvement of preparedness activities across the
organization.
• …
Findings statements are generated
for class A, B, and C appraisals
Weaknesses
• Internal dependencies are well-identified in support of service
continuity planning, but external dependencies are not.
• While service continuity plans are being executed appropriately in
the organization, no evidence was provided to show that plans are
being evaluated for their effectiveness in operation.
• …
Findings statements are agreed by
consensus of the full appraisal team
© 2011 Carnegie Mellon University
69
Appraisal process
Preparation
Lead
•
Appraiser
•
•
•
Customer •
Identify Gaps
Onsite
Reporting
Develops appraisal
plan
Trains appraisal team
Coaches and
monitors evidence
preparation*
Plans and schedules
interviews
Appraisal team:
• Reviews evidence
(may collect
additional evidence)
• Performs interviews
• Characterizes
practices by
consensus
Appraisal team:
• Presents final findings
to sponsor – typically
in MS Powerpoint
• Optionally produces a
written report which
may include detailed
recommendations
Collects and prepares
evidence*
•
Supports interviews
and additional
evidence collection
* Evidence collection in advance of the onsite is the most efficient appraisal process, but may require
substantial effort by the customer – this mode is called “verification.” Alternatively, the evidence can be
collected during the onsite period in a mode called “discovery.”
© 2011 Carnegie Mellon University
70
Recognize
Objective
Analyzing gaps
To make sure that closing gaps makes sense,
gaps should be analyzed:
Evaluate
Results
Determine
Scope
Implement
Changes
Identify
Gaps
Analyze
Gaps
• Is the cost for closing a gap worth the investment?
• Are there any efficiencies that can be realized by making the
changes to close one or more gaps (efficiencies may include
streamlining controls or compliance activities)?
• Which gaps are most important in the context of the objective?
• Are the organizational changes necessary to close the gaps within
the bounds of sponsorship?
Output is a set of prioritized gaps to be closed
© 2011 Carnegie Mellon University
71
Recognize
Objective
Implementing changes
Use model guidance
Evaluate
Results
Determine
Scope
Implement
Changes
Identify
Gaps
Analyze
Gaps
• Subpractices and other informative material provide implementation
guidance
• Code of Practice Crosswalk highlights connections between CERT-
RMM and relevant standards and codes of practice, which can serve
as additional implementation guidance
• Generic practices in the model provide guidance for having the
changes persist in the organization
Consider measurements that could be implemented with the
changes to help monitor results and inform management
© 2011 Carnegie Mellon University
72
Recognize
Objective
Evaluating results
Evaluate
Results
Determine
Scope
Implement
Changes
Identify
Gaps
Did we achieve the objective?
Analyze
Gaps
Did the changes stick? Can we be sure the new state will
persist?
Are additional needs or objectives now apparent?
When should we make another improvement cycle?
If measurements were implemented, are they revealing
positive trends?
© 2011 Carnegie Mellon University
73
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Compliance process area
Service Continuity process area
Using CERT-RMM
Summary and resources
© 2011 Carnegie Mellon University
74
Summary and resources
© 2011 Carnegie Mellon University
75
Key benefits of using CERT-RMM
Improve efficiency and
effectiveness of operational risk
management
Lower risk, lower cost
Institutionalize resilience
management processes using
proven techniques
Confidence that processes
will be sustained in times
of stress
Establish a common language
for resilience in your
organization (or community)
Effectively communicate
and collaborate to achieve
resilience
Access an extensive body of
knowledge for managing
operational risk and resilience
Confidence in
completeness, flexibility,
and scalability of approach
© 2011 Carnegie Mellon University
76
But I’m already using ________
Most organizations already use one or more standards or
practice bodies to support security and continuity activities.
CERT-RMM can complement your current efforts
• Completeness: CERT-RMM may provide coverage or guidance not
included in your current practice bodies
• Scalability & flexibility: use only the parts that you need to support
your improvement objective
• Stickiness: institutionalization guidance can be deployed to help
you make current and improved practices persist and collaborate
© 2011 Carnegie Mellon University
77
Potential next steps
Get the book
Take the course
Select a subset of the model that matches your current
improvement objectives
Convene a small team to review the model content and
identify gaps in your current activities
© 2011 Carnegie Mellon University
78
Resources
Training
Book
Introduction to the CERT Resilience
Management Model (3-day course)
Includes full
model (v1.1)
plus adoption
guidance and
perspectives
from realworld use of
the model.
• Public courses
- Feb 14-16, 2012 (DC)
- July 16-18, 2012 (Pittsburgh)
- Oct 2-4, 2012 (DC)
• Private onsite courses are also
available
www.sei.cmu.edu/training/P66.cfm
Lead appraiser apprenticeship program
is also available to certify people in
leading CERT-RMM-based appraisals
Available at Amazon.com
www.cert.org/resilience
email: info@sei.cmu.edu
© 2011 Carnegie Mellon University
79
Contact information
David White
SEI Customer Relations
CERT Resilient Enterprise
Management Team
For general inquiries
dwhite@cert.org
412-268-5800
David Ulicne
Joe McLeod
For information about training
For information about working
with us
deu@sei.cmu.edu
customer-relations@sei.cmu.edu
jmcleod@sei.cmu.edu
© 2011 Carnegie Mellon University
80
© 2011 Carnegie Mellon University
81
CERT-RMM Use Scenario
Using selected process areas to improve incident
management
© 2011 Carnegie Mellon University
82
Scenario: improve incident management
Objective: improve incident management capability
A quick scan through CERT-RMM reveals several process
areas that would assist with this objective
• Incident Management and Control
• Risk Management
• Monitoring
• Service Continuity
© 2011 Carnegie Mellon University
83
Incident Management and Control defines
Event – one or
more occurrences,
possibly minor, that
affect assets and
have the potential
to disrupt operations
Incident – an event
(or series of events)
of higher magnitude
that significantly
affects assets and
requires action to
limit impact
Incident
Criteria
Event
Crisis – an incident
where the impact is
rapidly escalating or
immediate
Crisis
Criteria
Incident
Closure
Crisis
Closure – should actively occur for all events, incidents, and crises
when no further actions are needed.
© 2011 Carnegie Mellon University
84
Incident Management and Control
In most organizations, many event streams need to be
watched to effectively provide early warning and to detect
incidents and crises.
How do we build an effective approach?
Event stream
Event stream
Event stream
Event stream
Event stream
© 2011 Carnegie Mellon University
85
Risk Management -1
Risk Management guides
Network intrusions
Supply disruption
the identification of sources
Malware
and categories of risk that
Mass illness
matter to the organization,
Extreme weather
for example:
© 2011 Carnegie Mellon University
86
Risk Management -2
These sources of risk
should inform the event
streams if they are likely to
lead to incidents or crises
Network intrusions
Supply disruption
Malware
Mass illness
Extreme weather
Event stream
Event stream
Event stream
Event stream
Event stream
© 2011 Carnegie Mellon University
87
Monitoring
Monitoring guides the implementation of data collection and
sharing activities. In this example, it will provide guidance on
implementing the infrastructure to monitor these event
streams.
Network intrusions
Malware
Mass illness
Supply disruption
Extreme weather
© 2011 Carnegie Mellon University
88
Risk Management -3
Risk Management practices produce criteria for measuring
the potential impact of risks.
Risk measurement criteria
inform
Incident
Criteria
Crisis
Criteria
Network intrusions
Malware
Mass illness
Supply disruption
Extreme weather
© 2011 Carnegie Mellon University
89
Incident Management and Control process
Practices from Incident management and Control produce a
consistent process for managing incidents and crises
Consistent incident management process, including closure
Incident
Criteria
Crisis
Criteria
Closure
Network intrusions
Malware
Incident
Crisis
Mass illness
Supply disruption
Incident
Extreme weather
© 2011 Carnegie Mellon University
90
Service Continuity
Service Continuity practices produce plans to ensure the continuity
of operations in the event of disruptions. Continuity plans will be
triggered during incidents or crises. Collaboration is needed to
ensure that plans are effectively triggered.
Service continuity plans
Triggers
Incident
Criteria
Crisis
Criteria
Triggers
Closure
Network intrusions
Malware
Incident
Crisis
Mass illness
Supply disruption
Incident
Extreme weather
© 2011 Carnegie Mellon University
91
Incident Management system
Four process areas that can
help us develop an effective
incident management
system in our organization
Incident
Criteria
• Incident Management and Control
• Risk Management
• Monitoring
• Service Continuity
Crisis
Criteria
Closure
Network intrusions
Malware
Incident
Crisis
Mass illness
Supply disruption
Incident
Extreme weather
© 2011 Carnegie Mellon University
92
CERT-RMM for Assurance
Focusing CERT-RMM on early life-cycle activities
for building resilience in
© 2011 Carnegie Mellon University
93
RTSE – Resilient Technical Solution Engineering
Ensure that software and systems
are developed to satisfy their
resilience requirements
© 2011 Carnegie Mellon University
94
RTSE specific goals
Goal
Goal Title
RTSE:SG1
Establish guidelines for resilient
technical solution development
RTSE:SG2
Develop resilient technical solution
development plans
RTSE:SG3
Execute the plan
© 2011 Carnegie Mellon University
95
RTSE: Building in versus bolting on
Requires organizational
intervention
Extends resilience
requirements to assets that
are to be developed
Creates requirements for
quality attributes
Attempts to reduce the level
of operational risk
Extends across the
life cycle
© 2011 Carnegie Mellon University
96
RTSE: Designing and testing for resilience
• Performing resilience controls planning and design
• Incorporating resilience controls into architecture design
• Designing resilience-specific architecture
• Adopting secure coding practices
• Processes for detecting and removing defects
• Designing testing criteria to attest to asset resilience
• Testing resilience controls
• Designing service continuity plans during the
development process
© 2011 Carnegie Mellon University
97
RTSE influences
BSIMM2
bsimm.com
Open Web Applications Security Project (OWASP) Software
Assurance Maturity Model www.owasp.org
Microsoft Security Development Life Cycle
www.microsoft.com/security/sdl/
DHS Process Reference Model for Assurance Mapping to
CMMI-DEV V1.2
https://buildsecurityin.us-cert.gov/swa/procresrc.html
© 2011 Carnegie Mellon University
98
CERT-RMM for software assurance
© 2011 Carnegie Mellon University
99