Cybersecurity: Engineering a
Secure Information Technology
Organization, 1st Edition
Chapter 8
Standard Process Models for Securing
ICT Organizations
Objectives
• Distinguish between process definition and process
improvement
• Understand the purpose of standard models for
process improvement
• Understand how process improvement enhances
system and software security
• Understand the basic concepts of process capability
maturity
• Understand the Software Engineering Institute’s
Capability Maturity Models (CMM and CMMI)
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
2
Underwriting Trust and Competence in
ICT
• The software industry has developed
comprehensive models of best practices to address
ICT product integrity
– Called capability models or process improvement
models
• A formal model is necessary
– Activities within any organization have to be logically
related and effectively coordinated
• A model of best practice ensures that coordination
is logical, complete, and correct
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
3
The Problems that Capability Models
Address
• ICT security issues fall into five categories:
– Installation of malicious logic on hardware or
software
– Installation of counterfeit hardware or software
– Failure or disruption in the production or distribution
of a critical product or service
– Reliance upon a malicious or unqualified service
provider for the performance of a technical service
– Installation of unintentional vulnerabilities on
software or hardware
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
4
The Problems that Capability Models
Address
• Malicious code is embedded in a product to fulfill
some hostile purpose
– Rigorous testing and inspection are required to find
and eliminate instances
• Counterfeit parts threaten product security and
integrity because they are not authentic parts
• Unintentional vulnerabilities occur in software and
hardware because of failures in the development
and sustainment process
– Weaknesses that can be exploited by a given threat
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
5
Putting Capability into Practice
• Adopting and following a commonly accepted
capability model is the approach that is most
frequently chosen to address the problem
• Process capability calls out three common-sense
principles:
– Control the development and sustainment work
using common best practice
– Adopt rigorous assurance practice at the component
construction level
– Rationally plan for contingencies
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
6
Putting Capability into Practice
• A large percentage of breakdowns caused by
counterfeiting activity can be mitigated by ensuring
all entities in supply chain are under strict
management control
• Control processes: explicitly designated
behaviors designed to ensure proper performance
of a product or related process
• The most common characteristic of a capability
model is that it can enforce trust through a
universally recognized third party assessment or
audit
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
7
Putting Capability into Practice
• Standard assessment underwrites two of the most
important factors in global business: trust and
competence
• According to Watts Humphrey of SEI, three
variables that serve as a basis for trust in business
are:
– History, understanding, and awareness
• A formally defined process has to be available to
assess and certify the supplier’s competence
– This role is filled by capability models
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
8
A Distinction: Why We Need to Build a
Standard Infrastructure First
• Generic capability maturity models are not intended
to define the general infrastructure of the ICT
organization
– They are considered necessary to refine that
structure
• Capability models specify key processes for
performing software work
– Describe minimum requirements in carrying out
those processes
• Key processes: operations that an organization
performs to conform to industry best standards
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
9
Why Use a Process Capability Model?
• The role of ICT management is to ensure that
faults do not occur in the first place
• Managers have to use a commonly accepted
means to ensure product integrity
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
10
The History of Best Practice Models
• Early Models of the CMM and ISO 9000
– 1987, the International Standards Organization (ISO)
published ISO 9000
– 1987, Watts Humphrey of SEI published an article
on assessing software engineering capability
• Would later develop into an early version of the CMM
– Version 1.0 of the CMM was released in two
technical reports by SEI: The Capability Maturity
Model for Software and Key Practices of the
Capability Maturity Model
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
11
Expanding the Application of the CMM
During the Late 1990s
• The CMM was used throughout the 1990s as the
model of best practice for U.S. software industry
• A separate version, called Capability Maturity
Model Integrated (CMMI) was developed in the
mid-1990s
• CMMI version 1.1 was released in 2002
– Version 1.2 was released in August 2006
• The current version, CMMI 1.3, was released in
November 2010
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
12
ISO 15408: The Common Criteria
• In 2009, ISO published ISO/IEC 15408
– Defined a set of criteria for rigorous, technically
based evaluation of ICT products
• This standard was called the common criteria
– Established a basis for evaluating the security of ICT
products and systems
• 15408 is one of the earliest examples of a true ICT
security standard
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
13
The 21st Century
• A range of models was published through the early
2000s
• These models are the basis for discussion in the
rest of this book
• All of them provide an excellent basis for
developing a fully functional process that ensures
best practices in ICT development, sustainment,
and acquisition
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
14
Families of Prominent Capability
Models
• CMMI, ISO 15408, and ISO 15504 are families of
standards
– they are referred to by their generic titles for
convenience
• Neither CMMI nor ISO 15504 is specifically a
product standard
– Designed to guide the way an organization
approaches its work
– Not to shape the outcomes of that work
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
15
The Capability Maturity Model (CMM)
• CMM is flexible and assessment based
– It defines five levels of capability and assesses an
organization’s current level of process maturity
against these levels
• Process maturity: the level of capability of a given
process based on routine key practices
• The CMM can be used for software process
improvement
– Or for software capability evaluations
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
16
Background of the CMM
• CMM is a commercial model and is a direct
outgrowth of ideas that originated in the software
industry
• The CMM is grounded in a set of practices that a
software organization can use to plan and manage
its software development and maintenance
operations
• The CMM specifies five levels of increasing
capability from ad hoc and immature operation to
mature, disciplined systematic processes
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
17
Evolution of the CMM
• The CMM is called a framework or model rather
than a standard
– It is promulgated by an organization that is not a
formal standards body
• The CMM is probably SEI’s best-known product
• CMMI was developed independently of the CMM in
the mid-1990s
• CMMI is fully dependent on the original CMM in
form and structure
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
18
Components of the CMM
• The current CMM is designated as SEI/CMM
version 1.1
• CMM 1.1 is based on progress through five
process maturity levels
• Each maturity level is characterized by a distinctive
set of key process areas (KPAs)
• Common features establish the basis for proving
that the organization is meeting its goals within
each KPA
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
19
Components of the CMM
• The CMM includes the following components
–
–
–
–
–
–
Maturity levels
Process capability
Key process areas
Goals
Common features
Key practices
• The most visible concept in the CMM is the
maturity levels
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
20
Maturity Levels of the CMM
• Each key process area can be distinguished
through a precise set of goals for installing a
requisite element of a good software process
• The Initial Level (1) - the organization can be
chaotic and unmanaged
– The only measure of capability is individual
competence
– Project success depends strictly on individual efforts
and the professionalism of each staff member
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
21
Maturity Levels of the CMM
• The Repeatable Level (2) - processes at this level
are capable of being improved
– Overall goal is to manage its projects more
effectively
– Project scheduling, staffing, and costing are more
predictable, and problems are addressed using
knowledge generated from an organization’s own
experience
– Important aspect of this level is the practice of
configuration management, which supports the
ability to make stable and rational decisions
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
22
Maturity Levels of the CMM
• The Defined Level (3) - The goal is to create an
environment where software managers and
technical personnel can do their jobs effectively
– The organization’s processes for developing and
maintaining software are fully defined, documented,
and integrated into a body of knowledge
– Workers have a precise understanding of the
organization’s software engineering and
management processes
– Organizations formalize a body of universally
accepted best practices for software engineering
work
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
23
Maturity Levels of the CMM
• The Managed Level (4) - feature of this level is the
development and use of a targeted set of
productivity and quality metrics
– The organization formulates and deploys an
assessment and feedback mechanism to gauge
effectiveness of its software products and processes
– Formally establishes an empirically based
management information system (MIS)
– Organization must be able to monitor and detect
significant variations between desired and actual
behavior
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
24
Maturity Levels of the CMM
• The Optimizing Level (5) - the organization has
access to all mechanisms necessary to identify and
react to problems and then take steps to improve
the process
– All outcomes are predictable at this level and all
processes are repeatable
– New technologies or software methods can be
seamlessly integrated into the software operation
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
25
Key Process Areas (KPAs)
• A KPA resides at one level of maturity
• Each KPA can be viewed as a particular capability
that the organization must be able to document to
demonstrate a given level of defined maturity
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
26
The Repeatable Level
• Requirements Management - to establish the
required consensus between the customer and the
software supplier
• Software Project Planning - to establish the
operational basis for the software project through a
set of explicit plans
• Software Project Tracking and Oversight establishes and maintains an adequate level of
understanding of project activity
• Software Subcontract Management - defines a
mechanism for subcontractor selection
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
27
The Repeatable Level
• Software Quality Assurance - enables managers to
have complete visibility into the evolving software
process and provides a more complete
understanding of product quality
• Software Configuration Management - establishes
and maintains the integrity of the software
throughout the lifecycle
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
28
The Defined Level
• Organization Process Focus - establishes and
assigns responsibilities for refining an
organization’s software processes
• Organization Process Definition - develops and
maintains a collection of software process assets
that provide a foundation for process improvements
• Training Program - develops skills and knowledge
so workers can carry out assignments
• Integrated Software Management - integrates the
organization’s software engineering and
management into a set of best practices
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
29
The Defined Level
• Software Product Engineering - consistently carries
out a well-defined engineering process
• Inter-group Coordination - establishes a means for
the software engineering group to participate
actively with other engineering units
• Peer Reviews - removes defects from software
products as early and efficiently as possible
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
30
The Managed Level
• Quantitative Process Management - adds formal,
comprehensive measurements to the practices
defined in the last KPAs in the Defined level
• Software Quality Management - applies a
comprehensive measurement program to the
software products described in the Software
Product Engineering KPA
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
31
The Optimizing Level
• Defect Prevention - identifies the causes of defects
and prevents them from recurring through activities
such as defect evaluation, causal assessment, and
process change
• Technology Change Management - also called
technology transfer
– Identifies new technologies, methods, or processes
and helps transition them into the organization
• Process Change Management - takes
improvements and disseminates them throughout
the organization
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
32
Explaining the KPAs
• KPAs are the best-practice areas that distinguish
the CMM
– Each KPA exists at a single maturity level
• KPAs in this model can be classified as
implementing three types of processes:
Management, Organizational, and Engineering
• The Management process contains project
management as it evolves from planning and
tracking at Level 2 to managing at Level 3
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
33
Explaining the KPAs
• The Organizational process category contains
wider responsibilities that are necessary as the
organization matures
• The Engineering process category contains the
more common technical activities of software
engineering
– Includes requirements analysis, design, coding, and
testing
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
34
Key Practices
• The purpose of key practices is to state the
fundamental policies, procedures, and activities
that help create the infrastructure for effective
implementation of a given KPA
• The goal set summarizes the key practices of a
KPA and is used to determine whether an
organization or project has effectively implemented
the KPA
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
35
Common Features of KPAs
• Five common features of KPAs:
–
–
–
–
–
Commitment to Perform
Ability to Perform
Activities Performed
Measurement and Analysis
Verifying Implementation
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
36
Determining Capability: The CMM
Assessment Process
• The CMM process assessment establishes a
baseline for determining the process maturity level
of each software organization
• The basic approach is to conduct a structured
series of interviews using a questionnaire
• Two types of assessment methods are employed
with the CMM:
– Software Capability Evaluation (SCE)
– Software Process Assessment (SPA)
• Both types use the CMM as the basis for
determining maturity of a particular process
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
37
Determining Capability: The CMM
Assessment Process
• SPAs tend to be more open and collaborative
– Used to identify problems and help managers make
improvements
• SCEs are rooted in the original practical intent of
the CMM (to select a capable supplier)
– Focus on risks associated with a supplier
– Necessary when important contracts are being bid
• SCEs are costly and tend to resemble audits
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
38
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
39
Specific Conduct of the Assessment
Process
• A maturity questionnaire (MQ) is administered
– Typically to 4-10 people
• Outcomes are assessed, not scored
• Respondents are briefed about the:
– Role of CMM appraisals in process improvement
– Objectives and principles of the appraisal
– Activities that might take place
• Following the assessment, the lead auditor selects
a form for reporting problems or areas of concern
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
40
Maturity Rating Schemes
• Based on the assessment results, each component
can be assigned one of the following ratings:
–
–
–
–
Satisfied
Unsatisfied
Not applicable
Not rated
• Each maturity level contains several KPAs that
must be satisfied
• The assessment team uses documents and
interviews to decide whether an organization
complies with a certain key process
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
41
Maturity Rating Schemes
• Practices that every member of the organization
should understand and use:
– Size of the organization and costing procedures
– Standard reporting practices required across the
organization
– Standard metrics required for projects
– Tailoring guidelines and waiver procedures
– Training plans for the organization
– Policies, procedures, and standards for engineering
– Standard lifecycle activities such as design,
programming, and testing
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
42
Maturity Rating Schemes
• Project-level documents can include:
–
–
–
–
–
–
–
Minutes from project management meetings
Project status reports and schedules
Software change request forms
Test records
Training records
Software development folders
Historical data derived by comparing plans vs. actual
trends
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
43
Maturity Rating Schemes
• At the end of assessment, a final meeting to
compile findings into a report takes place
• Elements of this report include:
–
–
–
–
The scope and objectives of the assessment
Details of the assessment program
Copies of nonconformity reports
The team’s recommendations for each area under
study
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
44
Assessor Qualifications
• CMM qualification requirements are less
regimented than they are for governmentmandated compliance standards
• SEI offers CMM assessor courses
– Has licensed companies to conduct SEI-compliant
CMM assessment and assessor training
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
45
CMMI
• CMMI is the current benchmark for the CMM
• The two types of CMMI:
– Staged - provides a sequence of staged
improvements
• Permits comparisons between units based on maturity
levels
• Can be integrated with other CMMS
– Continuous - allows an organization to select the
order of improvement that bests meets objectives
• Enables an organization to evaluate an internal
process based on a desired profile of capability
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
46
CMMI Disciplines and Environments
• CMMI includes two disciplines and one
development environment:
– System Engineering Discipline
– Software Engineering Discipline
– Integrated Product and Process Development
Environment
• CMMI provides guidance for improving the
development, acquisition, and maintenance of
software products and services
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
47
CMMI Maturity Levels
• Initial
• Managed (in the CMM, this level is known as
Repeatable)
• Defined
• Quantitatively Managed (in the CMM, this level is
known as Managed)
• Optimizing
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
48
CMMI Key Process Areas (KPAs)
• KPAs are slightly different in CMMI
• Instead of six KPAs in Level Two of the Software
CMM, CMMI has seven
• CMMI features 13 KPAs in Level Three instead of
the seven in the Software CMM
• Level Four and Level Five KPAs are very similar to
those of the Software CMM
– They use different names
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
49
CMMI Common Features
• CMMI has the following four common features:
–
–
–
–
Commitment to Perform
Ability to Perform
Directing Implementation
Verifying Implementation
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
50
ISO 15504 (also known as the
Security Engineering CMM)
• ISO 15504 establishes a migration path for existing
assessment models and methods
• Aim of 15504 is to perform process assessment,
process improvement, and capability
determinations
• Software process domains assessed by 15504 are:
– Acquisition, Supply, Development, Operations,
Maintenance, Supporting processes, and Service
support
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
51
Summary
• To develop successful, defect-free software, an
organization must adopt and follow a disciplined set of
practices
• Capability maturity models warrant that an
organization’s security features are correct
• The capability maturity process is defined by policies,
and it passes through five standard stages called
maturity levels: Initial, Repeatable, Defined, Managed,
and Optimizing
• Capability models have existed for the past 25 years
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
52
Summary
• The common features of the CMM delineate
management qualities
• The ISO 15408 standard is the first true security
standard for software
• Capability criteria define all aspects of correct product
and process performance
• The outcome of capability evaluation is an explicit
understanding and documented description of every
KPA, the requirements for implementing a capable
organization, and the relationships between those
elements
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
53
Summary
• The routine assessment of an organization’s activity
using a capability model produces quantitative data
that managers can use to improve their processes
Cybersecurity: Engineering a Secure Information Technology
Organization, 1st Edition
© Cengage Learning 2015
54