Risk and opportunity
Part 1
Tor Stålhane
Torbjørn Skramstad
Time
Topics
09:00
Risk and opportunity

What is it and why do we need to manage it?

Why is opportunity assessment important?

Why should we worry about risk and opportunity in SPI?
The SWIR model and how to use it.
Exercise – Application of the SWIR model
09:50
Coffee break
10:00
Assessment and brainstorming

The human bias

Qualitative assessment

Simple brainstorming techniques

Some important diagrams for risk assessment
Exercise – Build an event tree
10:50
Coffee break
11:00
Simple risk and opportunity assessment
Risk and opportunity management - barriers and enablers.
The ROP - Risk and Opportunity Pattern
Exercise – Application of the ROP in SPI
11:50
Coffee break
12:00
Leverage as a decision tool
Extended risk and opportunity assessment
The ALARP and GALE concepts – when is enough really enough?
The CORAS model – quantitative and qualitative assessment
Exercise – Application of the GALE concept
Important things to remember – summing it up
12:30
Lunch
EuroSPI 2006 - Part 1
2
Contents of part 1
•
•
•
•
What is risk and what is opportunity
Why should we care
Assessing risk and opportunity
Risk and opportunity in SPI – the SWIR
and the SWIRO models
• More on assessment
• Brainstorming techniques
EuroSPI 2006 - Part 1
3
Risk and opportunity
Risk and opportunity have three things in
common:
• They are concerned with events that may
– or may not – happen in the future.
• The events are identifiable but their effect
are uncertain, although less uncertain than
the probabilities.
• The outcome of the events can be
influenced by our actions
EuroSPI 2006 - Part 1
4
What is risk
A risk is something that can be a problem in
the future. It is defined by two parameters
• The consequences - C. What will happen
if the risk becomes a problem?
• The probability - p. What is the probability
that the risk will become a problem?
The risk – R – is defined as R = C*p
EuroSPI 2006 - Part 1
5
What is opportunity
An opportunity is something that can be
beneficial in the future. It is defined by two
parameters
• The value - V. What will happen if the
opportunity becomes a reality?
• The probability - p. What is the probability
that the opportunity will be realized?
The opportunity – O – is defined as O = V*p
EuroSPI 2006 - Part 1
6
Why should we care -1
Risks may turn into problems. We can reduce or
avoid future problems by reducing their
consequences or their probabilities. This can be
done by
• Changing the way we work to
– Replace a high risk activity with a low risk activity.
– Remove the risk possibility
• Adding risk avoidance activities to the way we
work
EuroSPI 2006 - Part 1
7
Why should we care - 2
Opportunities may turn into benefits. We can
increase future benefits by increasing their
probabilities. This can be done by
• Changing the way we work – replace a low
opportunity activity with a high opportunity
activity.
• Adding opportunity enabling activities to
the way we work
EuroSPI 2006 - Part 1
8
Assessing risk and opportunity
Both risk and opportunity is defined by value
and probability.
Experience and data are important for two
reasons. They can:
• Be used to estimate values and
probabilities.
• Serve as an anchor for assessment – e.g.
“How bad can it get?”
EuroSPI 2006 - Part 1
9
Risk and improvement
All SPI activities implies change and all
changes carries their own risks and
opportunities.
We will present two relevant models called
SWIR and SWIRO respectively.
The purpose of these models is to identify
risks and opportunities in SPI work.
EuroSPI 2006 - Part 1
10
The SWIR model -1
The SWIR model is the SPI version of the
SWOT model.
• SWOT – Strengths, Weaknesses,
Opportunities and Threats.
• SWIR – Strengths, Weaknesses,
Improvements and Risks.
EuroSPI 2006 - Part 1
11
The SWIR model - 2
Strengths
Were shall we win?
Weaknesses
What are our weak sides?
Improvements
Where shall we improve
ourselves?
Risks
What can go wrong?
Which opportunities can
we loose
EuroSPI 2006 - Part 1
12
The SWIR components - 1
• Strengths – we need to know and understand
our strong sides so that we
– do not destroy them in the SPI process
– can build on them and improve them
• Weaknesses – must be known so that we
understand what we are up against.
• Improvements – what we want to achieve. They
must be discussed and understood together with
our strengths and weaknesses.
EuroSPI 2006 - Part 1
13
The SWIR components - 2
• Risks – potential problems that we have to
cope with. They can stem from:
– Our weak sides
– Changes that are a necessary part of the SPI
process.
– Threats to our strong side – things that must
be kept the way they are.
EuroSPI 2006 - Part 1
14
The SWIRO model - 1
The SWOT model includes opportunities but
ignore improvements
The SWIR model includes improvements but
ignores opportunities.
It might be a good idea to merge these two
models so that we have a unified
presentation of strengths, weaknesses,
risk, opportunities and improvements.
EuroSPI 2006 - Part 1
15
The SWIRO model - 2
Strengths
Where shall we win?
Weaknesses
What are our weak
sides?
Improvements
Risks
Where shall we improve What can go wrong?
ourselves?
Current opportunities
Which opportunities do
we have now?
New opportunities
Which new opportunities
will the change open
up?
EuroSPI 2006 - Part 1
16
A caveat
None of the presented models – SWOT,
SWIR or SWIRO – will help us to assess
the risks and opportunities.
The models are just used to get a complete
picture of the situation.
Assessment is the logical next step.
EuroSPI 2006 - Part 1
17
Exercise
You are considering the introduction of an
ISO conform process into your company.
Fill in the SWIR or SWIRO diagram.
EuroSPI 2006 - Part 1
18
Assessment - 1
Even though assessment is a subjective
activity it is not about throwing out any
number that you like.
To be useful, an assessment must be
• Based on relevant experience.
• Anchored in real world data.
• The result of a documented and agreedupon process.
EuroSPI 2006 - Part 1
19
Assessment - 2
Risk and opportunity assessment is critically
dependent on the persons who participate,
their experience and their knowledge.
Experiments have shown that people have
some biases which implies that we need to
be careful when we look at the identified
risk events and their assessed
consequences and probabilities.
EuroSPI 2006 - Part 1
20
The human bias
Two human biases are important:
• Omission bias - most persons prefer doing
nothing instead of an action if the
consequences have equal values.
• Status quo bias - people assign a larger
risk to change than to maintaining status
quo. This bias increases if the change
action has the potential to create victims.
EuroSPI 2006 - Part 1
21
Qualitative assessment
We can assess consequences, probabilities
and benefits qualitatively in two ways. We
can use:
• Categories – e.g. High, Medium and Low
• Numbers – e.g. values from 1 to 10.
EuroSPI 2006 - Part 1
22
Categories – 1
When using categories, it is important to
give a short description as to what each
category implies. E.g. it is not enough to
say “High consequences”. We must relate
it to something already known, e.g.
• Project size
• Company turn-over
• Company profit
EuroSPI 2006 - Part 1
23
Categories – 2
Two simple examples:
• Consequences: we will use the category
“High” if the consequence will gravely
endanger the profitability of the project.
• Probability: we will use the category “Low”
if the event can occur but only in extreme
cases.
EuroSPI 2006 - Part 1
24
Impact and probability - 1
Impact
Probability
H
M
L
H
H
H
M
M
H
M
L
L
M
L
L
EuroSPI 2006 - Part 1
25
Impact and probability - 2
The multiplication table is used to rank risks
and opportunities. It can not tell us how
large they are.
We should only use resources on risks and
opportunities that are above a certain,
predefined level.
EuroSPI 2006 - Part 1
26
Numbers as categories -1
We can use numbers instead of names. This
does not make the assessment more
precise but will free us from the need to
define a multiplication table in order to
identify risks.
In principle we can use any numbers. The
best solution is, however, to just assign
number to the three aforementioned
categories
EuroSPI 2006 - Part 1
27
Numbers as categories – 2
The following values are often used in
practice, both for consequences, benefits
and probabilities:
• 10 – high
• 4 – medium
• 1 – low
Thus, a medium consequence and a low
probability will give a risk of 4*1 = 4.
EuroSPI 2006 - Part 1
28
Numbers as categories – 3
Impact
Probability
H / 10
M/3
L/1
H / 10
H / 100
H / 30
M / 10
M/3
H / 30
M/9
L/3
L/1
M / 10
L/3
L/1
EuroSPI 2006 - Part 1
29
Simple brainstorming techniques
Brainstorming is an efficient way to use the
creative abilities that each person have.
In its simplest form, people just generate
ideas and a person registers the ideas on
a whiteboard or a flip-over.
We can, however, use techniques to do
better.
EuroSPI 2006 - Part 1
30
Brainstorming and risks - 1
We can use previous experiences to answer
questions such as
• Can this really happen; e.g. has it
happened before?
• Can we describe a possible cause consequence chain for the event?
• How bad can it get?
• How often has this happened in the past?
EuroSPI 2006 - Part 1
31
Brainstorming and risks - 2
We can use techniques such as:
• Affinity diagrams – “post it notes”
• Cause – consequence diagrams, such as
– Ishikawa diagrams – also called fishbone
diagrams
– Event trees
– Cause – consequence networks
EuroSPI 2006 - Part 1
32
Ishikawa diagram
Planning
Resources
Wrong
personnel
Estimation
Follow-up
Loose key
personnel
Too late
delivery
Tool X is not
working
Changes
Misunderstandings
Requirements
Reuse problems
Development
EuroSPI 2006 - Part 1
33
Event trees
Found in unit test
Found in
integration test
Coding error
Found in
systems test
Not found in
unit test
Not found in
integration test
Not found in
systems test
EuroSPI 2006 - Part 1
Delivered
to customer
34
Cause – consequence diagram
E4
C1
C6
E1
C2
E5
Acc
E7
E2
C3
E6
C4
C7
E3
E8
E6
C5
EuroSPI 2006 - Part 1
35
Change and risk
Changes can introduce risks. The main
reasons are that:
• Any effect of a change is related to the
future and can thus not be certain
• It is difficult to completely understand the
effect of changes in a complex,
sociological system
EuroSPI 2006 - Part 1
36
Change and opportunities
Changes can create new opportunities. The
opportunities are mostly
• Indirect effects of what we do to achieve
our goals – e.g. a new tool that can be
used in several ways
• Additional effects of having achieved the
goals – e.g. less need for rework frees
resources for developing a new product.
EuroSPI 2006 - Part 1
37
Risk and opportunity in SPI
Risk and opportunity are important in SPI.
We need to consider:
• Cost related to the change.
• Benefit, which is its planned purpose
• Risk related to the change, since we are
going to work in a new way.
• New opportunities that are opened up by
the changes
EuroSPI 2006 - Part 1
38
Exercise
You want to study the effect of document
inspection on the number of defects
delivered to the customer.
Build an event tree for the starting event
“A defect has been introduced in high level
design”
EuroSPI 2006 - Part 1
39
Next session
The next session will focus on
• How to do simple risk and opportunity
assessment.
• The introduction of barriers and enablers
into risk and opportunity assessment
• How to use leverage to prioritize our
actions
EuroSPI 2006 - Part 1
40
Risk and opportunity
Part 2
Tor Stålhane
Torbjørn Skramstad
Contents of part 2
•
•
•
•
•
•
•
•
Simple risk assessment
Simple opportunity assessment
The total picture – risk and opportunity
The risk and opportunity pattern
Barriers, enablers and leverage
Extended risk analysis
Extended opportunity analysis
Risk and regret
EuroSPI 2006 - Part 1
42
Simple risk assessment
In order to a simple risk assessment we
need to identify:
• Dangerous events
• Each event’s
– consequence – C
– probability – p
• Possible barriers – changes or controls
• Person responsible for each risk - Resp.
EuroSPI 2006 - Part 1
43
Simple risk table
Event
C
p
R Barriers
EuroSPI 2006 - Part 1
Resp
44
Events
We start by identifying dangerous events.
The simple way to do this is to use
brainstorming.
The process is simple – just sit down and
envisage your worst nightmares related to
the activities under consideration.
Be realistic – only consider things that you
believe can happen.
EuroSPI 2006 - Part 1
45
Barriers
Barriers can be realized through:
• Prevention – we change our process so
that the event cannot occur.
• Mitigation – we can
– change the process in order to reduce the
event’s probability or consequences.
– define activities that will reduce the problems
if the event occurs.
EuroSPI 2006 - Part 1
46
Event
Barrier 6
Prob.
Barrier 5
Handling barriers
Prevent event from having
bad consequences
Barrier 4
Barrier 3
Barrier 2
Risk
Barrier 1
Prevention barriers
Prevent risk from becoming
a problem
Reduction barriers
Reduce effect
of event
EuroSPI 2006 - Part 1
47
Simple opportunity assessment
In order to assess opportunities, we need to
identify:
• The event that opens up opportunities enablers
• Each opportunity’s
– realizable value – V
– probability - p
• The activity needed to realize the value
• Person responsible for each opportunity
EuroSPI 2006 - Part 1
48
Simple opportunity table
Enabler
Opportunity
V
p
O Enabling
activity
EuroSPI 2006 - Part 1
Resp.
49
Enablers
Any action – e.g. a change – can create an
opportunity enabler. Each enabler opens
up a set of opportunities.
Further actions are needed in order to
realize value.
Both enablers, opportunities and enabling
actions can be identified through
brainstorming.
EuroSPI 2006 - Part 1
50
Opportunity and risk
Assessing consequences and value:
• H – High. Will have large impact
• M – Medium. Should not be ignored
• L – Low. Can be ignored
Assessing Probability:
• H – High. Will happen quite often
• M – Medium. Will happen now and then
• L – Low. Will almost never happen
EuroSPI 2006 - Part 1
51
The total picture - 1
The total picture of the situation shows the
risks and the benefits that stem from a
planned change.
This is not a mechanism that can be used to
identify the best solution.
It is, however, an important input when we
want to make a decision.
EuroSPI 2006 - Part 1
52
The total picture - 2
The total picture shows risks, benefits and
opportunities. Risk can be shown in two
ways:
1. Unmitigated risks
2. Mitigated risks – include the effect of risk
reduction activities, e.g. barriers. This
can be done by
– Modifying the risk assessment
– Indicate how the risk will move in the
diagram
EuroSPI 2006 - Part 1
53
Costs and benefits
Reduced number of
MMI-related defects
H
B
M
L
p
L
M
Extra work needed for
MMI-specification
L
C
H
M
H
EuroSPI 2006 - Part 1
54
Unmitigated risks
Reduced number of
MMI-related defects
H
B
M
L
p
L
M
Extra work needed for
MMI-specification
L
C
H
M
H
Large disagreements
between designers and
MMI experts
Partnership does not
work
EuroSPI 2006 - Part 1
55
The mitigation effect
Reduced number of
MMI-related defects
H
B
M
L
p
L
M
Extra work needed for
MMI-specification
L
C
M
H
H
2
Large disagreements
between designers and
MMI experts
Partnership does not
work
1
EuroSPI 2006 - Part 1
56
Including opportunities
Reduced number of
MMI-related defects
Better MMI for existing
products
H
B
Better MMI
requirements will
reduce imp. costs
M
L
p
L
M
Extra work needed for
MMI-specification
L
C
M
H
H
2
Large disagreements
between designers and
MMI experts
Partnership does not
work
1
EuroSPI 2006 - Part 1
57
The tyranny of “either – or”
All too often we are confronted by the
statement that we can get only get X if
we are willing to suffer Y.
This is the wrong attitude. The right attitude
is that we will
1. Do what is needed to get X
2. Perform activities that will remove or
reduce the bad effects of Y.
EuroSPI 2006 - Part 1
58
The risk and opportunity pattern
A pattern is a description of a standard way
to solve a common problem. The Risk and
Opportunity Pattern – ROP – is a way to
analyze and manage risk and opportunity.
ROP has two components:
• A set of assessment and management
activities
• A process that describe an activity
sequence
EuroSPI 2006 - Part 1
59
The ROP process
ROP consists of the following activities:
1. Define the job and its borders
2. Perform a risk assessment
3. Perform an opportunity assessment
4. Implement the identified barriers
5. Do the job while
–
–
controlling risks and preventing problems
searching for opportunity enables and harvesting
benefits
EuroSPI 2006 - Part 1
60
ROP activities – risk part
•
•
•
•
Define the job and its borders. We
cannot consider everything – only what is
inside the defined borders.
Perform a risk assessment.
Implement the barriers identified in the
previous step.
Do the job - control risks and prevent
problems.
EuroSPI 2006 - Part 1
61
Exercise
Your company consider buying a new test
administration tool. Management is unsure
whether this is a wise investment.
Use the risk part of ROP to help
management in their decision.
EuroSPI 2006 - Part 1
62
Barriers and enablers
Barriers and enablers will define actions that
will help us to
• Avoid problems – barriers
• Reap benefits – enablers
Identification of barriers and enablers is,
however, not enough. We also need to
assess how effective they are.
EuroSPI 2006 - Part 1
63
Leverage
Leverage is a prioritizing mechanism:
Leverage = (Benefit – Cost) / Cost
Leverage will prioritize activities with
• Large net benefits
• Small costs
EuroSPI 2006 - Part 1
64
Extended risk table -1
We can use cause – consequence chains or
event trees for a risk to identify the best
place to insert a barrier.
For each barrier, we need to assess:
• Cost - the cost of implementing it. We will
use the scale H = 10, M = 3 and L = 1.
• E – how effective is the barrier? We will
use the scale h = 1.0, m = 0.5 and l = 0.2
EuroSPI 2006 - Part 1
65
Extended risk table - 2
Event
C
p
R Barrier Cost
EuroSPI 2006 - Part 1
E
L
Resp.
66
Barrier leverage
Leverage = (C*p*E – Cost) / Cost
The leverage will prioritize barriers which:
• Have low costs – Cost is small
• Have high efficiency – E is large
• Attack important risks – C*p is high
EuroSPI 2006 - Part 1
67
Barrier – example
Event
Partnership does
not work – business
conflicts
Customers do not
prioritize project
participation
Cons
.
10
10
p
3
3
R
Mitigation
Do a thorough research
on selected partner’s
30
business goals
E
L
Resp
John
0.5
State the conditions and
consequences of customer
30 participation in the
1.0
contract
EuroSPI 2006 - Part 1
Cost
10
0.5
Pete
3
9.0
68
Some comments on barriers
It is important to remember that:
• Each risk will usually need a different barrier – a
barrier that works against one risk can be
valueless against another risk.
• It is important to consider the three main barrier
strategies:
– Prevent the risk from becoming a problem
– Control the problem to avoid the consequences
– Reduce the consequences
EuroSPI 2006 - Part 1
69
Extended opportunity table - 1
Even if an opportunity arises, nothing will
really happen if we do not do something to
realize it.
An enabler is an event that will help us to
reap a benefit.
Just as barriers, the activities linked to an
enabler have costs and effectiveness.
Thus, we can compute the leverage and
use this as a basis for our decisions.
EuroSPI 2006 - Part 1
70
Extended opportunity table - 2
Enabler
Opportu V p
nity
O
Action Cost
EuroSPI 2006 - Part 1
E
L
Resp.
71
Opportunity leverage
Leverage = (V*p*E – Cost) / Cost
The enabling activity leverage will prioritize
activities which:
• Have low costs – Cost is small
• Have high efficiency – E is large
• Enable valuable opportunities – V*p is
high
EuroSPI 2006 - Part 1
72
Enabler - example
Enabler
Opportunity
Better MMI
requirements,
which will
reduce imp.
costs
Use MMI more
actively to
create more
popular
products
Better understanding of how MMI requirements are implemented
and adapted
Value
10
10
p
10
3
O
100
30
Action
Use new
knowledge to
make better
MMI
requirements
spec
Redesign user
interface for
products A and
B
EuroSPI 2006 - Part 1
E
Cost
L
Resp
Peter
1
3
32
Brian
1
10
2
73
An alternative presentation - 1
We have earlier used the cost-benefit
diagram to show benefits, opportunities,
costs and risks.
By including the efficiency of barriers and
enabling actions, we get a better picture of
the overall situation.
Since we already have performed the
necessary multiplications, we can use a
one-dimensional representation.
EuroSPI 2006 - Part 1
74
An alternative presentation - 2
The alternative representation is just a representation. It is
thus just one of several inputs to a decision.
100
30
10
10
Costs and
risks
30
100
Benefits and
opportunities
EuroSPI 2006 - Part 1
75
A small example - 1
We have the following assessed values:
• Cost: C = medium, p = high, Cost = 30.0
• Benefit: V = high, p = high, Benefit = 100.0
• Risks
– R1: C = medium, p = low, barrier efficiency = medium,
R1 = 1.5
– R2: C = high, p = low, barrier efficiency = low, R2 =
8.0
• Opportunities
– O1: V = medium, p = high, enabling activity efficiency
= medium, O1 = 15.0
– O2: V = high, p = high, enabling activity efficiency is
low, O2 = 20.0
EuroSPI 2006 - Part 1
76
A small example - 2
Cost R2 R1
100
30
10
O1 O2
10
Costs and
risks
30
Benefit
100
Benefits and
opportunities
EuroSPI 2006 - Part 1
77
Regret and risk - 1
Instead of just looking at cost and value of
an opportunity, we can include risk and
regret in the leverage expression.
Regret is the, often indirect, cost of skipping
or ignoring an opportunity.
Priority = (Value + Regret) / (Cost + Risk)
EuroSPI 2006 - Part 1
78
Regret and risk - 2
Just as cost, value and risk, regret has to
bee assessed, for instance on a scale
from 1 to 10 or just using three values
such as 10, 3 and 1.
As should be expected
• High regret and low risk will give high
priority.
• Low regret and high risk will give low
priority
EuroSPI 2006 - Part 1
79
Next session
The next session will focus on
• Two risk assessment concepts – ALARP
and GALE
• How to use the GALE method
• Quantitative assessment and the CORAS
model
• Summing up - some important things to
remember
EuroSPI 2006 - Part 1
80
Risk and opportunity
Part 3
Tor Stålhane
Torbjørn Skramstad
Contents
•
•
•
•
•
•
•
•
ALARP and GALE
Using GALE
How to do risk assessment with GALE
A small example
Quantitative assessment
The CORAS model
A small example
Important things to remember
EuroSPI 2006 - Part 1
82
ALARP and GALE
There are two competing principles in the
assessment of risk:
• ALARP – As Low As reasonably PossibleWe have done all that is reasonable to
prevent problems and dangers.
• GALE – Globally At Least Equivalent. E.g.
introducing a new process will not
increase the risks compared to what it is
today.
EuroSPI 2006 - Part 1
83
ALARP
ALARP requires that we analyze each risk
separately and then implement mitigation
activities.
A reasonable goal is to reduce each risk
until the extra mitigation costs exceed the
value of the risk reduction achieved.
All that we have seen up till now fits into an
ALARP policy .
EuroSPI 2006 - Part 1
84
GALE
GALE requires us to look at the total risk of
a change. In this way we can start by
attacking the cheapest risk or the risk with
the largest leverage.
The problem with the GALE principle is that
we need to perform arithmetic on risks.
E.g. we need to decide how many medium
risks we need before we have a large risk
EuroSPI 2006 - Part 1
85
ALARP vs. GALE - 1
There is no such thing as the right risk
principle. It is always a matter of company
choice and company policy.
The two principles will lead to different
prioritization of mitigation activities.
• ALARP – each risk is reduced as much as
possible.
• GALE – we need to be below the present
risk level.
EuroSPI 2006 - Part 1
86
ALARP vs. GALE - 2
The one important thing with using the
GALE principle is that it forces us to ask
“What is the current risk level?”
All too often we act as it the current way of
doing things is risk free and all risk stems
from changes.
This stance is enforced by the human
tendency to underestimate the risk of
status quo.
EuroSPI 2006 - Part 1
87
Using GALE
Important points
• GALE is a method for risk analysis.
Benefits must be included elsewhere
• We need to look at both our current risk
and the risk resulting from the proposed
changes.
• Always perform a sensitivity analyses.
EuroSPI 2006 - Part 1
88
Risk – status quo vs. change
In many cases, maybe even in most of
them, we do risk assessment because we
want to compare two or more alternatives,
e.g.:
• Status quo – no changes
• One or more changes - improvements
EuroSPI 2006 - Part 1
89
Event identification
• All significant dangerous events must have
been identified.
• There must be a minimal overlap between
the dangerous events .
• There must be a maximum of commonality
between the dangerous events considered
for the status quo and for the system after
the proposed changes
EuroSPI 2006 - Part 1
90
The three event sets
The previous rules split the dangerous
events into three sets – dangerous events
that:
• Apply both to the status quo and to the
new system.
• Are unique to the status quo
• Are unique to the new system
EuroSPI 2006 - Part 1
91
GALE and risk assessment - 1
GALE uses the following parameters for risk
assessment:
• FE – the event frequency
• PE – the probability that the event will lead
to an accident
• S – the severity score of an event
EuroSPI 2006 - Part 1
92
GALE and risk assessment - 2
We can compute individual and
accumulated risk indices:
IE = FE + PE + S
IGR = log Sumi(10I)
IE is the risk index for a hazardous event
IGR is the global risk index
EuroSPI 2006 - Part 1
93
The GALE scoring scheme
The scoring scheme of GALE
• Focuses on deviations from current
average. This is reasonable, given that it is
mainly concerned with comparing status
quo to a new situation.
• Must be tailored to each situation. The
next slide shows an example from road
safety. We need a scheme adapted to SPI.
EuroSPI 2006 - Part 1
94
Road safety - frequency score for
event
Frequency
Occurrences / year on M42 ATM section
classification
Very frequent
10000
Hourly
FE
6
Frequent
1000
A few times a day
5
Probable
100
Every few days
4
Occasional
10
Monthly
3
Remote
1
Annually
2
Improbable
0.1
Every 10 years
1
Incredible
0.01
Every 100 years
0
EuroSPI 2006 - Part 1
95
SPI and GALE
We need a special scoring scheme for
development projects. For events that can
lead to problems we need to consider:
• How often does the event occur - FE?
• If the event occurs, what is the probability
that it will cause a real problem - PE?
• If the problem occurs, how severe will the
consequences be – S?
EuroSPI 2006 - Part 1
96
SPI goals
Based on the GALE parameters, we can
also identify possible SPI goals:
• S: reduce the consequences – reduction
and handling barriers
• FE: reduce the number of event
occurrences – problem opportunities
• PE: reduce the probability that the event
will cause a problem – prevention barriers
EuroSPI 2006 - Part 1
97
Frequency score for event
Frequency
class
Occurrences per project
FE
Very frequent
200
Every project
6
Frequent
100
Every few projects
5
Probable
40
Every 10th project
4
Occasional
10
Every 100th project
3
Remote
1
A few times in the company’s
lifetime
2
Improbable
0.2
One or two times during the
company’s lifetime
1
Incredible
0.01
Once in the company’s
lifetime
0
EuroSPI 2006 - Part 1
98
Probability score for event
Classification
Interpretation
PE
Probable
It is probable that this event, if it
occurs, will cause a problem
3
Occasional
The event, if it occurs, will
occasionally cause a problem
2
Remote
There is a remote chance that this
event, if it occurs, will cause a
problem
1
It is improbable that this event, if it
occurs, will cause a problem
0
Improbable
EuroSPI 2006 - Part 1
99
Severity score for event
Severity
class
Severe
Average
Minor
Interpretation
The portion of occurring problems that
have serious consequences is much
larger than average
The portion of occurring problems that
have serious consequences is similar
to our average
The portion of occurring problems that
have serious consequences is much
lower than average
EuroSPI 2006 - Part 1
S
2
1
0
100
Sensitivity analysis
The global risk index is made of many
indices. Each index will have a certain
degree of uncertainty connected to it.
Usually, a few indices will have a large
influence on the result while the rest will
have but little influence.
Pareto’s rule applies - we need to identify
the few important indices.
EuroSPI 2006 - Part 1
101
A small example - 1
Status quo
Event
After process
improvement
S
FE
PE
S
FE
PE
Too late delivery – 1
1
5
3
1
4
3
Too high cost – 2
1
5
3
2
4
3
Low customer satisfaction – 3
1
4
3
0
3
2
Low developer satisfaction – 4
1
4
2
0
3
2
Too low product quality - 5
1
4
2
0
3
2
EuroSPI 2006 - Part 1
102
A small example - 2
• Status quo:
I1 = 9, I2 = 9, I3 = 8, I4 = 7, I5 = 7
• After SPI activity:
I1 = 8, I2 = 9, I3 = 5, I4 = 5, I5 = 5
IGR = log Sum(10I)
• Status quo: log Sum(10I) = 9.3
• After SPI activity: log Sum(10I) = 9.0
EuroSPI 2006 - Part 1
103
A small example - 3
We see from the results that the risk
reduction is small – from 9.3 to 9.0.
We also see that the main reason for this is
that we have increased quality but
increased the cost.
The main result from the GALE process is
that we need to find ways to increase the
quality without increasing our development
cost.
EuroSPI 2006 - Part 1
104
Quantitative assessment -1
In some cases, we can use numerical
values. This occurs if we can use
• Experience to identify the cost of a
problem – e.g. correcting an error or
loosing a customer.
• Old data to identify a probability – e.g. the
probability of missing a defect during
inspection.
EuroSPI 2006 - Part 1
105
Quantitative assessment - 2
Quantitative risks and opportunities give us
real values.
The usefulness of this is, however, limited
since it is difficult to find real values for all
risks and opportunities.
It is not obvious how we can compare
qualitative and quantitative risks or
opportunities
EuroSPI 2006 - Part 1
106
The CORAS model
CORAS was developed as a framework for
assessment of security risks.
What should concern us here, however, is
how they related the qualitative risk
categories, not to absolute values, but to
the company’s turn-over.
EuroSPI 2006 - Part 1
107
The CORAS consequence table
Consequence values
Category
Measured
related to
income
Measured
loss due to
impact on
business
Insignificant
Minor
Moderate
Major
Catastrophic
0.0 – 0.1%
0.1 – 1.0%
1 – 5%
5 – 10%
10 – 100%
Lost profits
Reduce the
resources of one
or more
departments
Loss of a couple
of customers
Close down
departments or
business
sectors
No impact on
business.
Minor delays
EuroSPI 2006 - Part 1
Out of
business
108
The CORAS frequency table - 1
As we will see on the next slide, CORAS
allows us to interpret frequency in two
ways:
• The number of incidents per year
• The failing portion of demands
We will use the second interpretation but
instead of focusing on a system, we
related it to the number of projects, e.g.
SPI projects.
EuroSPI 2006 - Part 1
109
The CORAS frequency table - 2
Frequency values
Category
Rare
Unlikely
Possible
Likely
Almost
certain
Number of
Unwanted
incidents per
Year
1/100
1/100 – 1/50
1/50 - 1
1 - 12
> 12
Number of
Unwanted
incidents per
Demand
1/1000
(1/500)
1/50
(1/25)
1/1
Interpretation
of number of
demands
Unwanted
incident
never
Occurs
Each
thousand
time the
system is
used
Each five
times the
system is
used
EuroSPI 2006 - Part 1
Each tenth
time the
system is
used
Every
second
time the
system is
used
110
A small example
We have a company with 10 developers and an
estimated yearly turnover of NOK 10 millions.
We decide that the consequences of a late
delivery is “medium”, which gives a
consequence of 1 – 5% or NOK 100 000 to 500
000.
We decide that the event is “likely” to occur, which
gives us a p-value of 0.04
The expected loss is thus 4 000 to 20 000.
EuroSPI 2006 - Part 1
111
Exercise
Your company has decided to change
development process.
• List all important events
• Find the risk index for each event for
– Status quo
– The new development process
EuroSPI 2006 - Part 1
112
Important things to remember - 1
The most important things to remember:
• Risk assessment is by its nature subjective.
• Use group techniques and include all
stakeholders
• Use simple techniques so that you do not
exclude one or more stakeholders
• Anchor it in experience and available data will,
however, improve the quality
• Subjective values like “High” must be anchored
in each company’s reality. One company’s
“High” may be another company’s “Low”.
EuroSPI 2006 - Part 1
113
Important things to remember - 2
• Include the effect of choosing status quo in
all SPI risk analyses.
• Always include opportunities
• Consider the three barrier categories –
prevention, handling and reduction
• Rank risks and opportunities according to
their leverage
• The results from a risk assessment is just
one of several inputs to a decision
EuroSPI 2006 - Part 1
114