SCRLC
Metrics / Quantifying
Risk (Track #4)
Edward Erickson
Track Co-leader
June 7, 2007
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
1
Agenda
 Overview
 Scope
 Deliverables
 Schedule / Milestones
 What we need from the Council
 Case Study
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
2
Overview
 Participation Excellent from thought leaders – lacking from practitioners
Track
4 Quantifying Risk / Metrics
Track Leaders
Feryal Erhun, Stanford
Edward Erickson, Cisco
Track Members to Date*
Hau Lee, Stanford
Ely Kahn, TSA
Andrew Cox, TSA
Tim Astley, Zurich
Lance Solomon, Cisco
 Survey Response Rate Poor
3 companies (P&G, Boeing, Cisco) + TSA
2 thought leaders (Stanford, Zurich)
 Despite this track members believe that:
this is a critical focus area
it will lag the other tracks and will have a longer payoff time frame
 Research members will lead the effort in the early phases
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
3
Scope
 In Scope
How to portray SC risk modeling & analysis results in an impactful way
Methods for quantifying SC risk to support decision making & measuring
the impact of actions
Methods for modeling SC risk & identifying potential improvement
actions
Tools & techniques for determining important risk events and the scope
of models
How to ground SC risk data in reality
 Out of Scope
Standards definitions
Tool/Modeling development
Industry specific methods
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
4
Deliverables – To Date
 Survey practitioners to understand current SC risk
metric practices
 Survey thought leaders to determine Best Known
Methods (BKMs)
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
5
Metrics/Quantifying Researcher Risk Survey
Who:
All SCRLC research organizations – 1 survey per organization
Why:
Get a good sample of all of the metrics/quantifying risk best practices
from a research/theoretical point of view.
Questions:
1.
What is the best way known way to quantify SC risk?
2.
What is the best way you’ve seen in practice to measure SC risk?
3.
What are the major gaps you see between the best methods and what
you’ve seen in practice?
4.
What are your current area of expertise and interest in measuring SC risk?
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
6
Summary of Researcher Survey Results (2 out 5 Responded)
Where We Need to Be
•
•
•
•
Integrated view of supply chain risk
Utilize distributions for occurrence and intensity
Driven by historical loss/occurrence data
Application of expert knowledge to address gaps in
data
• Lack of data-driven analysis on key areas of supply chain risk
• Lack of understanding for all risks affecting the supply chain
• Focus on consequences rather than vulnerabilities and triggers
• Focus narrowly on cost – should include customer impact
• Focus only on most recent disruptions
• Minimal use of stochastic modeling
Where We Are
• Independent focus on supplier, disaster and IT
risks
• Focus on easy to measure risks
• Lack of data
• Limited to analysis of the averages
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
7
Metrics/Quantifying Practitioner Risk Survey
Who:
All SCRLC companies & government agency members – 1 survey per
organization
Why:
Get a good sample of all of the metrics/quantifying risk practices across all
member companies
Questions:
1.
To what degree is SC risk management driven at your company (e.g. not at all, a
strategic program, an ongoing part of the business, etc)?
2.
Where do you want see your company in 2 years with respect to SC risk
measurement and metrics
3.
Do you use metrics/measurement as part of your SC risk management organization?
If you don't, what metrics/measurements could you envision as part of an effective process
for managing risk?
If you do, what metrics/measurements do you currently use?
4.
What data do you use to manage SC risk and manage your SC risk programs?
5.
How do you use these data to manage SC risk and manage your SC risk programs?
6.
What tools do you use to drive SC risk management decisions?
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
8
Summary of Practitioner Survey Results (4 out 10 Responded)
Question
P&G
Boeing
TSA
Cisco
1. To what degree is SC risk management
driven at your company (e.g. not at all, a
strategic program, an ongoing part of the
business, etc)?
On-going component of
several business functions
Varies by subject and the
division within the
company. Mature in
strategic planning and
materials
Current - by each mode of
transportation
Subset of enterprise risk
management group
2. Where do you want see your company in
2 years with respect to SC risk
measurement and metrics
Continuing to use existing
metrics in organizations that
have risk responsibilities; will
add other metrics as identified
by the SCRLC if we believe
they will add value
More focused, capable, and
armed with more facts and
data to more effectively
guide SC risk management.
Accurately identify critical
vulnerabilities and
propose/develop
countermeasures
3. Do you use metrics/measurement as part
of your SC risk management organization?
If you don't, what metrics / measurements
could you envision as part of an effective
process for managing risk? If you do, what
metrics / measurements do you currently
use?
Identification and assessment,
Audit Scores, Site risk
assessment (risk identified,
likelihood, business impact,
risk rating) and plan against
high risk rated scenarios
Volume of imports by
supplier, country risk
ratings based on a variety
of criteria, metrics showing
anticipated increases or
decreases in supplier
shipments.
Proxy metrics to determine
effectiveness of risk
management efforts
Risk scores/maps Time
to recover, probabilistic
revenue at risk
4. What data do you use to manage SC risk
and manage your SC risk programs?
Data from the programs
mentioned in question #3 &
new ideas from industry
leaders, consultants,
academia, daily news
Individual Procurement
Agents manage risk but
higher level org. might
oversee a collective SC risk
program.
Classified intelligence
information. Industry
supplied transportation
data.
Natural Hazard data,
Geopolitical data, expert
opinion
5. How do you use these data to manage
SC risk and manage your SC risk
programs?
Typically Scorecards &
Leadership Reviews
N/A
Proxy measures to estimate
the effectiveness of various
regulations or security
programs.
Metrics drive SC risk
priorities
6. What tools do you use to drive SC risk
management decisions?
Internal standards, culture and
business unit financial
accountability and agreement
at the right level of
management
N/A
Checklist tools in the field.
Moving toward more
advanced simulation
models @ HQ.
Macroeconomic models for
costing.
Scorecards, Risk
Ratings and Simulation
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
Future - “systems” focused
approach to risk
management.
SC risk part of the DNA
within the business and
operations groups
Better quantification of the
“ROI” for risk management
activities.
9
Deliverables - Planned
 BKMs for portraying SC risk modeling & analysis
results in an impactful way
 BKMs for measuring SC risk and deciding what
mitigation actions to pursue
 BKMs and tools used for modeling risk and how to
manage scope of these models
 BKMs on SC risk data collection
 BKMs for how to measure risk improvement based
on supply chain improvements
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
10
Schedule / Milestones
 May’07
Kickoff & Agreement on Scope/Deliverables/Milestones/Meeting Schedule
Complete survey on Metrics/Quantifying metrics
Session to review survey results and prepare for June core team update
 June’07
Session on post core team update, change scope, etc
July’07
Session on Best Known Methods (BKMs) for measuring risk & deciding what
mitigation actions to pursue
August’07
BKMs & tools used for modeling risk & how to manage scope of these models
September’07
BKMs on event probability data collection
November’07
BKMs for how to measure risk improvement based on supply chain
improvements
Monthly teleconference except for months with core team meeting (9 meetings/yr)
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
11
What we need from the Council
1. Are you supportive of the longer
term view required?
2. Are you supportive of the defined
deliverables?
3. Fill out the survey
4. Join the team
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
12
Cisco
Case Study
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
13
Supply Chain Risk Mgmt. (SCRMx)
The Challenge
Risk
Measures
& Processes
Process / DNA
Risk
Tolerance
Strategic
Partner
Site Risk
Mgmt
(PSRM)
Tactical
Focus &
Governance
Foundational
Responsive
Business Continuity
Plans (BCP) - Partner
Components
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Crisis
Drills
Comparative
Risk
Mitigation
Quantify
Risks
Risk Map
& Modeling
Crisis Mgmt.
Plan
Pandemic
Plan
Transformation
Cisco Confidential
Risk
Strategy
Risk
Budget
Business Continuity
Mgmt. (BCM) - Process
Trans. & Logistics
Customers
14
High Level Process
Quantify
Assess
Measure
 Iterative process combining metrics and probabilistic modeling
 Use exposure and recovery metrics to assess and determine focus areas
 Use probabilistic modeling to quantify and measure the impact to the
business and pareto key drivers
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
15
Probabilistic Revenue Impact
Site X
Prod.
RevenueY
Company
($/Wk)
$50
Mil /Qtr
52 Week
Time to
Time
to Recover
X
Recover
(TTR)
(Wks)
Probability of an
Event
Catastrophic
Occurring
Site Fire
(%)= %.01
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
$2.6 Bil
Revenue
Revenue
Impact
($)
Impact
Probabilistic
Revenue Impact
= $26
($) Mil
16
Exec. Mgmt. / Finance
Manufacturing Operations
Product Operations
Cisco Case Study – Key Metrics
Risk Map
Rev. vs Risk
(Prod. View)
What products
should I be most
concerned
about?
Risk Map
Rev vs Risk
(Site View)
What sites
should I be most
concerned
about?
Rev @ Risk (E2E)
What is my Risk?
TTR
(Product View)
Rev @ Risk
(Prod. View)
Pareto of
Drivers
ROI
BCP
What are the
most critical
components?
What is their
impact &
likelihood?
What are the
drivers?
What will be my
ROI?
Are my partners
resilient?
TTR
(Site View)
Rev @ Risk
(Site View)
Pareto of
Drivers
ROI
BCP
What are the
most critical
issues?
What is the
impact &
likelihood?
What are the
drivers?
What will be my
ROI?
Are my partners
resilient?
Risk Map
Rev. vs Risk (Event)
ROI
TTR (Top Product)
What should I be most
concerned about?
What are my costed
options?
What is the impact to
my customer?
What has it cost me?
How has it changed?
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
17
Cisco Case Study - Probabilistic Modeling
Methodology
Inputs
Integrated Model
Outputs
Disruption
Revenue @ Risk (Prod)
Site/Region Events &
Frequency
Revenue @ Risk (Horiz.)
Capacity Impact
Revenue @ Risk (E2E.)
Time to Recover
Revenue @ Risk (Event)
Expected Capacity Loss
Supply chain redundancies
•
•
•
•
Excel Based
Monte Carlo
Crystal Ball Engine
Direct Data Links
Financial Impact
Sensitivity Analysis
identifying risk drivers
What-if Analysis
Site Revenue
Objective: Quantify drivers of risk and potential improvement from mitigations
Presentation_ID
© 2007 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
18