Palisade @RISK Conference 2007
Significance of Risk Quantification
The Smart Decision-making Process
John G. Zhao
Abstract
The “significance of risk quantification in decision-making process” reflects the
emergence of modern risk management within project management discipline. Using
commercial @RISK suite, the integrated process forces decision makers to quantify
risk impacts rather than, as current practice, to qualify risk effects on their decisions.
The author has integrated Risk Register, Monte Carlo Simulation, Decision Trees and
Force Field Analysis to facilitate a decision-making process. This system has been
used and applied to Suncor Bitumen Selection Strategy and other applications, the
“case study” proved to be successful. In addition, the result of this case study, along
with further research work, may have potential commercial values if the processes
are properly generalized, theorized and formalized; it will be valuable to Suncor or
any other energy company who desires a proven methodology for their future major
capital project selection decisions, because “many organizations continue using
decision practices that are decades out of date” (Schuyler, 2001, p.29).
Why Risk Quantification ?
Many papers and books have narrated in length on the topic of project risk
management in recent years, thanks to the PMI’s promotion on risk management.
However, most of the topics have focuses on the risk management process, paying
more attention to qualitative risk assessment and leaving quantitative risk analysis an
area of vacuum. The importance of quantifying identified risks can not be
emphasized more as it is the foundation of a meaningful decision making process.
Many researchers and scholars have written extensively on decision theories, few
however have attempted to answer the questions of what the risk quantification is,
how risks are quantified and why they are necessary in decision-making process.
Decision theories, and empirical studies under risk and uncertainty have produced a
John Zhao
Page (1)
September 11, 2007
Palisade @RISK Conference 2007
rich collection of findings (Simon1, 2006), their practicality in directing real-life project
decision-makings has yet to stand the industry’s tests.
The significance of risk quantification, its correlation to other decision variables, and
importance of risk categorization, and the use of integrated risk management
techniques are paramount in deriving future project option selection decisions during
an organization’s business case study phase. Unfortunately, many organizations,
including some predominant large firms, still lack scientific approaches in their
decision strategy. Static or deterministic numerical calculations still prevail large
portion of decision-makings which may not truly reflect the complexity of the
scenarios that are often than not risky, uncertain and probabilistic. Precision in
calculating these uncertainties is nearly impossible simply because risks are hard to
quantify, needless to mention computation of risks.
The author proposed the following hypothesis, which is being both theoretically and
practically proved, through an integrated risk modelling technique using @RISK
suite, including Monte Carlo simulation and PrecisionTree:
If risks are only identified but not quantified, decisions under uncertainty become
intuitive and a “gut feel”; if identified risks are better quantified, the credibility and the
quality of decisions made are significantly enhanced.
When talking about risk quantification, Bernstein (1998, xxiii) unambiguously stated
that “without numbers, risk is wholly a matter of gut”. To avoid arbitrary decisions
with absolute subjectivity or pure instinct, the succinct risks which are associated with
business decision objectives must be identified and quantified for its probabilistic
value, that often reflect on the business case NPV (net present value) or IRR (internal
rate of return). To achieve this objective, a stipulated process, or procedural step to
derive the conclusion, has to be detailed and communicated clearly to the business
stakeholders.
Background of the Case Study
The author has developed the project risk management processes and successfully
implemented across Suncor Major Projects, a business division of Suncor Energy
Inc., the 2nd largest integrated oil and gas company in Canada. At the request of a
business development group, an option selection decision process aided by @RISK
software was designed and proposed to Growth and Planning group. Upon their
acceptance, I enthusiastically facilitated the 3-day “business case” workshop which
1
Dr. Herbert Simon received his Nobel Memorial Prize in 1978 and national Medal of Science in 1986.
John Zhao
Page (2)
September 11, 2007
Palisade @RISK Conference 2007
formulated the specific but strategic decisions on bitumen supply to Suncor’s future
refining business. As the result of this decision, a multi-million dollar new project was
initiated and being developed. This process required interpreting the implied and
veiled descriptive risks and translating the qualitative consequences to quantitative
numerical figures for “calculation or simulation”. Fortunately as the workshop
facilitator, I have accumulated rich experiences in collecting raw data using personal
interview techniques, and thorough understanding of theories behind “translating
qualitative text” to quantitative numerical data for the purpose of risk analysis (Taylor,
1990, p.211), therefore the above process was a scientific approach backed up by
academic theory in terms of epistemology, and a practical one easily understood and
accepted by 20 some workshop attendees.
Suncor has produced raw bitumen from varied oilsands mining sources since 1967
as feedstock to its refineries, and an environmental friendly SAGD2 operation was
added to its bitumen production capacity in 2003. To reach the goal of 500,000
barrels per day (bpd) production by 2012 from the current 260,000 bpd, it was very
critical to properly and wisely decide what bitumen sources, mining or SAGD, should
be invested on as capital growth project (Suncor, 2005).
“Increasing business complexity, speed, and competition pressure us to improve
decision making” (Schuyler, 2001, p.4). In-depth understanding of risks and their
quantified impacts or opportunities is the prerequisite for making intelligent decisions.
It is the author’s hope that this paper will contribute to the existing knowledge base of
decision theory from the narrowest perspective of project management, and the most
practical aspect of methodology.
Current Gaps in Decision Science
Many literatures have been produced and researches conducted about decision
analyses. CCPS (1995, p. 43) categorized all recognized risk decision aids into five
branches which have totalled to 17 techniques, and most of which are based on
“fuzzy set theory”. The essential ingredients of modern techniques for quantifying
risk had been developed in 1730, all the tools we use today in the risk management
and in the analysis of decisions and choices, stem from the development that took
place between 1654 and 1760 (Bernstein, 1998, p.5). Early development of Game
Theory, Utility Theory, Payoff Matrix and Bayes’ Theorem, and so on built solid
foundation for decision science; modern decision theories of Fuzzy Decision Analysis
(Zadeth,1965 cited by French 1986, p.361), Subjective Expected Utility (Simon,
2
Steam Assisted Gravity Drain (SAGD): Inject high temperature and pressure steam into about 400m deep
bitumen reservoir (wells) to produce raw bitumen.
John Zhao
Page (3)
September 11, 2007
Palisade @RISK Conference 2007
2006), Decision Support System (DSS) (Bodily, 1985) and etc. have been developed
mostly in academia without much practical meanings. About 60 years ago, von
Neumann’s Monte Carlo simulation which was applied in Atomic Design during WWII
was a big practical leap in modern decision analysis. Morris (1994, p. 217) even
concluded that only risk management has received new development since 1960s in
the discipline of project management, however, the advancement was made mostly
in general terms, lacking specifics of risk quantification to aid decision-making
process.
Bernstein (1998,p.73) affirmed that we all have to make decisions on the basis of
limited data, and most critical decisions would be impossible without sampling. He
credited statistics for its crucial role in decision making. Statistics exists because we
live in an uncertain world, and decisions are made in the face of probabilities.
Business decisions, either using Value at Risk theory or the theory of Real Options,
seek to promote various business objectives (Mclaney, 2006, p.19), however, often
risks are not sufficiently taken into consideration or inadequately quantified. New
computer-based decision tools, such as Palisade PrecisionTree3 for example,
addressed subjective probability distribution quite well, but neglected to mention how
risk values can be objectively quantified (Palisade, 2000, p.53). Prichard (2001,
p.39) twice theorized in his writing that risk quantification is to produce project
contingency reserve, but failed to mention anywhere its function in decision-making
process.
French (1986, p.324) impatiently stated that “we have developed enough theory. The
time has come to call a halt and interpret that theory”. A practical decision-making
framework is what the industry needed to guide decision makers in their day-to-day
job. Contrary to meeting the practical demands, academia continued to battle each
other on theories, such as Shackle’s strongest objection to probability-based decision
making (Dembo and Freeman, 1998, p.58). The lack of recognized theoretical basis
on practical decision-making process still haunts lots of corporations, and in spite of
the emergence of decision analysis during 1960s from modern management science,
evaluation practices and decision policy remain a weak link in many organizations
(Schuyler, 2001, p.4), needless to mention risk quantification technique.
“Decisions are made at every step in the analysis of a business and of its economic
dimensions” (Drucker, 1964, p.195). Due to project risk management’s “novelty” and
“infancy”, a mature and systematic process to identify, quantify and apply risks to the
science of decision-making has yet to be generalized and theorized in academic
3
Developed and commercialized by Palisade Corporation, New York, USA
John Zhao
Page (4)
September 11, 2007
Palisade @RISK Conference 2007
world, practical process implemented as a decision policy within organizations,
however it proves not only to be exigent but very challenging as well.
Isolated Single Decision Method
In today’s computer technology advancing era, one needs to remember that only
combination of scientific methods and experienced heuristics can provide better end
results. “No evidence exists that complex models do better than heuristics (Mowen,
1993, p.132). Having stated that, the computer aided methods do offer tremendous
advantages in the face of decision science, because uncertainty analysis using
traditional deterministic calculus and algorithm proves to be not only labour-intensive
and time consuming, but also impractical and unachievable. With the helps of
@RISK Monte Carlo simulation and PrecisionTree® package, an integrated process
is proposed hereof to support business decisions embracing the concepts of risk
management.
Hammond et al (1999, p. 2) stated that “the ability to make smart decisions is a
fundamental life skill” for both professionals and personnel. Decisions can be fairly
obvious and intuitive requiring non-brain work; however life is far more complex than
simple instinct therefore we always face touch decisions. Businesses are even more
so when they come to encounter decisions which may seem unilateral ones but can
impact many facets across the organization. Worse yet when decisions involving
uncertainties and risks, since decisions are normally made for future events and
decision-makers expect favourite future outcomes, that decision-makers may loose
their sensible “intuition” and “logics”. The future is inherent with risks and
unnecessarily coherent, “wise decision-makers supplement intuition with logical and
rational decision (analysis) methods” (Schuyler, 2001, p.140). One of methods
proposed and used by many scholars and practitioners is Decision Tree approach.
Decision Tree is very popular among decision makers and business consultants.
Palisade PrecisionTree® is a book by itself and Palisade (2000, p.15) says “it brings
advanced modelling and decision analysis to Excel worksheets” to set up systematic
process for decision-making. Interestingly some world scale large business mergers,
acquisitions and new mega project decisions are made solely dependent upon
decision tree conclusions, plus decision-makers smart and wise intuitive judgement.
Probabilistic decision trees are constructed based upon an expert opinion at his
discretion at any give “chance node” for the probability of event occurrence, and the
subjectively estimated single deterministic consequences should the risky event
occur. Qualitative risk identification, itemization, categorization, severity ranking and
John Zhao
Page (5)
September 11, 2007
Palisade @RISK Conference 2007
evaluation are not taken into consideration in decision derivation, neither are built into
decision tree formulae for quantification.
Another well-liked stochastic method for decision-making is Monte Carlo simulation
for which Palisade is famed for its @RISK suite. In many cases risks are random
and conjecture but often continuous and correlated instead of discrete and distinct,
therefore this type of risks are better suited for simulation. For some applications in
Oil and Gas industry, random simulations have far more advantages over decision
tree method, such as cost estimate contingency simulation based on certain risk
tolerability, but the reverse proves to be true as well, for instance the chances of dry
reservoir well in exploration project. In recent years, however, advanced computer
technology and developed software allow the combination of Monte Carlo simulation
with decision Tree method. Palisade (2000, p.188) and Murtha (2000, p.17) both
talked about possibilities of integrating the two, however, their practical application in
the industry has not yet been common and visible.
Can an integrated and risk-based decision-making process be made practical and
workable in the industry? The author, through a live case study4 in Suncor Energy
Inc, the 2nd largest integrated energy firm in Canada, demonstrates in the proceeding
sections procedural steps to integrate qualitative risk assessment and quantitatively
risk-adjusted NPV, feeding the both into PrecisionTree® for smart option decisions.
An Integrated Approach
“Most firms operate a hierarchical decision making process. Each level of the firm
has a range of actions and is accountable to higher levels”. (Unpublished Thesis,
Year U/A, p. 7). Conventionally the highest levels in an organizational hierarchy
make final decisions and middle / lower levels participate in decision-making process,
often lacking scientific and thorough decision back-ups. Although the force-field
analysis is probably adopted at managerial level for option selection decisions,
relative subjectivity in the static process heavily affects proportional scores assigned
against or for each option. The option with the highest positive scores usually wins,
but risks associated with each option are not necessarily or adequately addressed,
needless to mention quantified.
Real Option Theory blossomed in recent years following Back-Scholes invention in
early 1970s. “There is an options way of thinking in the financial decision-making
process through the use of an options framework” (ed. Fusaro, 1998, Miller, p. 134).
4
The true numbers in the case study have been replaced with fake figures for confidentiality reasons.
John Zhao
Page (6)
September 11, 2007
Palisade @RISK Conference 2007
Within business decision realm, decision-makes are often challenged with various
options, and the decisions on different options could steer the organization in a totally
opposite direction, deviating from companies’ vision, mission and business
objectives. A proven probabilistic model and business process for option selections
at the business case study or project feasibility study phase is essential, paramount
and of necessity.
The model is built to follow business process. In the pyramid diagram (Figure 1-1),
the Force Field Analysis becomes a depository and a useful apparatus for the
purposes of options’ comparison and analysis, when hard decisions must be made in
selecting one of many optional choices. Contradictory to the static force-field
analysis method, the risk-based force-field is used to solicit collective opinions of
decision workshop attendees who are not necessarily in decision-making levels but
subject matter of experts. These “opinions” are interpreted into quantifiable numerical
figures as inputs to feed the PrecisionTree® and risk-adjusted NPV Monte Carlo
simulation model, whose results are correlated and linked to the PrecisionTree®.
Qualified risks for each option are further quantified for their expected monetary
values (EMV), and WTI-driven5 NPVs for each choice are further simulated based on
defined risk ranges. Both results are tabulated in the Force Field Analysis Table and
will be analyzed and presented to the higher levels in an organization as decision
recommendations.
Figure 1-1
Suncor major projects group has devised and labelled their risk management process
RISCOR™ which has three distinct functions. RISCOR™ Decision has integrated
risk register (RISCOR™ PM), PrecisionTree® and risk-adjusted NPV economic
Monte Carlo simulation (@RISK) model, and this integrated decision analysis model
5
In any economic model, WTI pricing is the most sensitive variable and has bigger vulnerability.
John Zhao
Page (7)
September 11, 2007
Palisade @RISK Conference 2007
is Excel spreadsheet based, and operated on Palisade @RISK platform. The
business process mapping is designed as illustrated in the following flow diagram to
direct decision stakeholders to avoid making obvious mistakes, because “people
make a variety of systematic and predictable mistakes by following their heuristics”
(Bazerman, 2002, p.11).
The process flow map, supported by computer-aided
stochastic decision model, takes stakeholders through each procedural steps to
identify key risks in the category of P (political), E (economical), S (social), and T
(technological) during various business case studies for each case option.
Figure 1-2
Decision
Objective
Options
Options’
Description:
Clearly lay out
Each Option’s
Assumptions
Feasibility
Study
Risk-adjusted NPV
@RISK
Monte Carlo
Simulation Model
Force Field
Analysis:
Options
Criteria:
PEST
RAM
NPV/IRR
Other
Risk
(Negative)
Opportunity
(Positive)
Qualitative “R” & “O” Assessment:
Risk Matrix (Prob x Conq + R + P)
# R = (- EV$)
# O = (+EV$)
Decision
Recommendation
PRECISIONTREE
The Best EMV of Risked NPV
TABLE
These key risks are assessed based on the pre-established Risk Matrix, and
quantified for its Expected Value (Probability x Consequence $) and total risk severity
scores. These scores of qualitative risk / opportunity assessment for each option are
entered into the Force Field Analysis table to derive qualitative conclusions /
decisions on the optimal option solution. An algorithm is built in the Force Field table
using raw data from the scores to calculate the objective probabilities of occurrence
of each risk category at the Chance Node of the PrecisionTree® for each option. The
quantitative economic analysis model based NPV values simulated from Monte Carlo
@RISK for each option are also entered as end results into the PrecisionTree®. This
simulation model, which is used to simulate an option’s net present value, contains
the variables of WTI oil pricing, production volume, Capex and Opex, Tax and
Royalties, etc. whose risk ranges and their correlations have been addressed in
@RISK simulation formulae. After all, the business objective is the maximization of
John Zhao
Page (8)
September 11, 2007
Palisade @RISK Conference 2007
the market value of owner’s wealth (Baker, 1981, p.21).
The best EV of risk
adjusted NPV associated with a particular option after running Tree Analysis shall
yield the final conclusion.
Figure 1-3
Force Field
Scores
Against - Risks
-24
Natural Gas Intensity Political pressure to reduce
NG, GNG Emissions/Carbon
Tax, Commodity
Decoupling/NEPII (National
Energy Program)
1PR
-29
Supply Costs ($/bbl),
Natural Gas Price/Oil Price,
CAPEX, Labour Productivity,
SOR/Reservoir performance.
1ER
-28
Remote location - Ability to
attract/retain skilled
workforce. Community
reaction to mitigations.
Infrastructure.
1SR
24
-22
SAGD/Reservoir Reservoir performance and
understanding of the
geotechnical issues.
1TR
92
-103
Opportunities - For
1PO
Tax Policy (Royalties) - More
favorable treatment due to provinces
desire to preferentially develop large insitu resource base.
1EO
Share Value/Credit, Increased value
in product due to flexibility in marketing
(reduced upgrading costs)
1SO
Reduced Footprint / Ecological
Impact = Better company image
1TO
Young Technology Opportunities Upside realized as knowledge of
process/reservoir increases.
Benefit SUM (1)
15
30
23
NET FORCE
Impact SUM (1)
-11
Legend: 1PO = Option one Political Opportunities; 1ER = Option One Economic Risk; (P.E.S.T.)
Summary and Conclusion
Fishhoff et al. (1981, by Bazerman, 2002, p.43) pointed out that perceptions of risk
are often faulty, resulting in misdirected risk reduction efforts by decision makers.
This further stressed the importance of risk quantification in the context of risk
management, which is often referred to as descriptive and qualitative process. Risk
management process can also be quantitative based, and risk severity levels and risk
tolerance levels are virtually measurable and quantifiable, though subjectivity may
skew the results. Risk quantification plays a significant role in business decision
making process in the circumstance of risk and uncertainty, more over, Baker (1981,
p.48) distinguished risk from uncertainty, and the latter was used to describe
situations in which probabilities could not be assigned against outcomes. While
decisions under uncertainty are subject to other theories such pay-off matrix, game
John Zhao
Page (9)
September 11, 2007
Palisade @RISK Conference 2007
theory and MinMax regret theory, to “quantify” the degree of uncertainty is as
important as to measure risk.
The decision complexity often surpasses human ability, both psychologically and
physiologically, in many aspects. Calculation or simulation of considerable amount
of quantitative data in various forms and graphs can cause enormous stress in
humans, and in many cases wise decisions are only feasible with the help of
computer-aided decision supporting system or models, after all, all we need is the
information-based knowledge to make right business decisions, most of which are
optimization problems where there is a range of choices for several decision
variables (Schuyler, 2001, p.199).
Quantitative decision analysis using appropriate stochastic tools, such as @RISK,
provides a more realistic and more accurate depiction of business option values for
making smart decisions. And the integrated process, as illustrated in the case study,
makes smart decisions possible.
END ( 3,188 words)
REFERENCES
Baker, A. (1981) Business Decision Making. Published by Croom Helm Ltd., London, UK
Bazerman, M. (2002) Judgment in Managerial Decision Making. 5th Ed. Published by John
Wiley & Sons, Inc. New York, USA
Bernstein, P. (1998) Against The Gods – The Remarkable Storey of Risk. Published by John
Wiley & Sons, Inc. New York, USA
Bodily, A. (1985) Modern Decision Making – A Guide to Modelling with Decision Support
System. Published by McGraw-Hill, Inc., USA
CCPS (1995) Tools for Making Acute Risk Decisions with Chemical Process Safety
Applications. Published by Centre for Chemical Process Safety (CCPS) / AIChE
John Zhao
Page (10)
September 11, 2007
Palisade @RISK Conference 2007
Dembo, R. and Freeman, A. (1998) The Rules of Risk – A Guide for Investors. Published by
John Wiley & Sons, Inc., New York, USA
Drucker, P. (1964) Managing for Results – Economic Tasks and Risk-taking Decisions.
Published by Harper & Row, Publishers, New York, USA
French, S. (1986) Decision Theory – An Introduction to the Mathematics of rationality.
Published by Ellis Horwood Ltd., West Sussex, England
Hammond, J., Keeney, R. and Raiffa, H. (1999) Smart Choices – A Practical Guide to
Making Better Decisions.
Published by Harvard Business School Press, Boston,
Massachusetts
McLaney, E. (2006) Business Finance – Theory and Practice.
Publications, Inc.
Published by Sage
Morris, P (1994) The Management of Projects. Published by Thomas Telford Services Ltd.,
Heron Quay, London, UK
Mowen, J. (1993) Judgement Calls – High stake Decisions in a Risky World. Published by
Simon & Schuster, New York, USA
Murtha, J. (2000) Decisions Involving Uncertainties – An @RISK Tutorial for Petroleum
Industry. Published by Palisade Corporation, New York, USA
Pritchard, C. (2001) Risk Management – Concepts and Guidance. 2nd Edition, Published by
ESI International, Virginia, USA
Palisade (2000) PrecisionTree – Powerful Decision Analysis for Spreadsheets. Published
by Palisade Corporation, New York, USA
Schuyler, J. (2001) Risk and Decision Analysis in Projects. 2nd Edition, Published by Project
Management Institute, Pennsylvania, USA
Simon, H. (2006)
Empirical Studies of Choice Under Uncertainty. Available at:
http://dieoff.org/page163.htm Accessed on April 16, 2007.
Suncor (2005) Strategic Risk Management Process - Proposal for Bitumen Supply Strategy.
Unpublished document, Suncor Energy Inc., Canada
John Zhao
Page (11)
September 11, 2007