How ESSA has successfully
used Decision Analysis to
overcome challenges in multiobjective resource management
problems
Developed by
ESSA Technologies
Ltd.
General overview
January 10 2002
David Marmorek,
Calvin Peters,
Ian Parnell,
Clint Alexander
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Common challenges in resource
management
• Getting stakeholder groups to agree on a course of
action, given multiple values and objectives
• Getting scientists to agree on which uncertainties most
critically affect management decisions, and what
decisions are most robust to these uncertainties
• Evaluating the costs and benefits of adaptive
management - is it worth it?
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
How decision analysis can help with these
challenges
• It provides a toolbox for handling multiple objectives /
values, and analyzing tradeoffs among these objectives
• It systematically analyzes the impacts of uncertainties on
decisions
• It can be used to evaluate the ability of Adaptive
Management experiments to improve decisions
• It provides a helpful way to integrate many techniques
employed by managers and scientists (i.e. models,
interactive workshops, sensitivity analysis) into products
that better clarify management decisions
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Three examples
• Getting scientists to agree: PATH
• Getting stakeholders to agree: Cheakamus
• Evaluating adaptive management: Keenleyside
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
PATH: Decision Context
Multiple historical changes in Columbia and Snake
River ecosystems and fisheries management
practices
Endangered species listings for Snake River
salmon populations
Multiple hypotheses and uncertainties held by
different groups of scientists
Duelling models representing these hypotheses
and uncertainties
Best management policies for species recovery?
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
PATH: Washington State, US
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Decision Analysis: 8 elements
1. List of alternative management actions
2. Management objectives composed of performance
measures (to rank management actions)
3. Uncertain states of nature (different hypotheses)
4. Probabilities of those states (to account for uncertainty);
5. Model to calculate outcomes of each combination of
management action and hypothesised state of nature;
6. Decision tree;
7. Rank actions based on expected value of the
performance measures; and,
8. Sensitivity analyses.
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Decision Analysis: Basic Elements
Management
actions
Probabilities of
states of nature
States of nature
or hypotheses
Outcomes or
consequences
Hypothesis 1
C11
Hypothesis 2
C12
Hypothesis 1
C21
Hypothesis 2
C22
P1
Action 1
Action 2
P2
P1
P2
MoF Adaptive Management Training Course
Overview of Decision Analysis with examples - Jan 10, 2002
Module 3 - 36
ESSA Technologies
PATH Decision Tree
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Benefits of decision analysis in PATH
• Allowed evaluation of multiple hypotheses for 14
uncertainties - scientists did not have to agree!
• Only 3 of these turned out to make a difference to the
decision - created a common focus for AM, research
• Preferred actions were those which were most robust to
the critical uncertainties (drawdown A3)
• Sensitivity analyses defined how much belief you would
have to have in a given hypothesis to change decision
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Recent Publications on PATH
•
Marmorek, David R. and Calvin Peters. 2001. Finding a PATH towards scientific
collaboration: insights from the Columbia River Basin. Conservation Ecology 5(2):
8. [online] URL: <http://www.consecol.org/vol5/iss2/art8>
•
Deriso, R.B., Marmorek, D.R., and Parnell, I.J. 2001. Retrospective Patterns of
Differential Mortality and Common Year Effects Experienced by Spring Chinook of
the Columbia River. Can. J. Fish. Aquat. Sci. 58(12) 2419-2430
http://www.nrc.ca/cgi-bin/cisti/journals/rp/rp2_tocs_e?cjfas_cjfas12-01_58
•
Peters, C.N. and Marmorek, D.R. 2001. Application of decision analysis to
evaluate recovery actions for threatened Snake River spring and summer chinook
salmon (Oncorhynchus tshawytscha). Can. J. Fish. Aquat. Sci. 58(12):2431-2446.
<same web site as above>
•
Peters, C.N., Marmorek, D.R., and Deriso, R.B. 2001. Application of decision
analysis to evaluate recovery actions for threatened Snake River fall chinook
salmon (Oncorhynchus tshawytscha). Can. J. Fish. Aquat. Sci. 58(12):24472458. <same web site as above>
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Cheakamus WUP: Decision Context
• British Columbia Hydro, Water Use Planning:
Stakeholder driven multi-objective consultation / decision
process.
• No formal incorporation of uncertainty as for PATH
• Emphasis: values, objectives, performance measures,
trade off analysis (DA steps 1, 2, 5 and 7).
• Used PrOACT approach (Smart Choices, Hammond et
al 1999)
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Cheakamus WUP: Process
WUP Steps
PrOACT Approach
Problem
Many choices
Objectives
Alternatives
Consequences
Tradeoffs
Clear choice
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Cheakamus WUP:Decision Problem
Select operating alternatives for Daisy Lake Dam that:
1) recognize multiple water uses in the Cheakamus and
Squamish Rivers, and
2) achieve a balance between competing interests and
needs.
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Cheakamus WUP:Objectives and PMs
Fundamental
Objectives
Power
First Nations
Recreation
Flooding
Fish
Performance Measures
Average power revenue ($M/yr)
1. Maximize economic
Power production (GWh)
returns from power
generation.
Greenhouse Gas emission reductions (Ktonnes/yr)
2. Protect integrity of
Flood and erosion risk to ancestral burial grounds
SFN heritage sites and and culturally important locations
cultural values.
Rafting (Avg. #days/yr)
3. Maximize physical
conditions / access for Kayaking (Avg. #days/yr)
recreation (kayaking,
Sportfishing (Avg. #days/yr)
rafting, sportfishing).
4. Minimize adverse
Flooding (# floods >450cms at Brackendale)
effects of flood events.
5. Maximize wild fish
Anadromous rearing Habitat Availability (m2),
populations
Resident rearing Habitat Availability (m2)
Anadromous Effective Spawning Area (m2),
Adult Migration flows (Avg. #days <10CMS)
Aquatic Ecosystem
6. Maximize area and
integrity of aquatic
ecosystem
Anadromous Riffle Benthic Biomass (kg benthos),
Resident Riffle Benthic Biomass (kg benthos)
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Cheakamus: WUP Alternatives
• Consultative Committee specifies operating
alternatives for Hydro operations model (AMPL).
• Basic constraints: minimum flow at Brackendale
gauge, minimum dam release.
• AMPL model produces 32 water years of flow data
for these control points
• Flow data and other models used to calculate
performance measures.
• Performance measures summarize consequences
of alternatives for objectives.
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Cheakamus WUP: Consequences
Fundamental
Objectives
1. Maximize economic
returns from power
generation.
Performance
Measures
Average power
revenue ($M/yr)
2. Protect integrity of SFN
heritage sites and cultural
values.
3. Maximize physical
conditions / access for
recreation (kayaking,
rafting, sportfishing).
Alternatives
15Min3Dam
15Min5Dam
15-20Min37Dam "Hybrid"
20Min7Dam
10Dam
35.6
34.8
34.3
32.3
31.8
Partly considered by Flood PMs, will be addressed in future if necessary.
Kayaking (Avg.
#days/yr)
123.9
137.7
199.8
242.0
204.1
Sportfishing (Avg.
#days/yr)
57.6
72.0
82.7
192.8
122.0
5. Maximize wild fish
populations
(x 103 m2)
RUA Resident Habitat
Rainbow Parr
35.8
37.7
42.5
42.5
45.2
Effective Spawning Area
Chum
9.8
9.2
9.7
7.3
6.5
Resident Riffle
Benthic Biomass (g
x 106)
3.4
3.5
2.9
2.9
3.0
6a. Maximize area and
integrity of aquatic
ecosystem
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Tradeoffs (or not)
RB Parr Habitat Availability
(m2)
Tradeoff: VOE vs. RB Parr
50000
10Dam
20Min7Dam
40000
20Min3Dam
7Dam
15Min5Dam
5Dam
20Min
15Min3Dam
30000
20000
Win-Lose
10000
0
31.00
32.00
33.00
34.00
35.00
36.00
VOE ($M/yr)
Tradeoff: VOE vs. Chum Effective Spawning Area
Chum Eff. Spwn. Area
12000
15Min3Dam
10000
6000
20Min
20Min7Dam
8000
15Min5Dam
7Dam
10Dam
5Dam
20Min3Dam
4000
Win-Win
2000
0
31.00
32.00
33.00
34.00
35.00
36.00
VOE ($M/yr)
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Cheakamus WUP: Filtering
• Use PMs to Eliminate clearly inferior alternatives.
• Drop insensitive PMs (e.g., rafting).
• Drop Objectives that don’t help the decision (e.g.,
flooding).
• Tradeoff analysis: Even Swaps
• Elicit values behind decisions (e.g., rating exercises)
• Develop new alternatives to address concerns (e.g.,
chum spawning vs. rainbow trout rearing).
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Keenleyside Problem : Increased egg
mortality from dam operation
Flow during spawning
Flow during
incubation
 stage
Proportion
eggs in dewatered
area
Risk
Biological
flows too high reduce productive capacity, may drive
population towards extinction
Economic
smaller flows may reduce de-watering mortality but
reduce potential $ and operational flexibility
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Problem II: Uncertainty
True whitefish recruitment dynamics?
Alternative Hypotheses
25,000
Age 4 Whitefish
20,000
Given typical
egg mortality,
LARGE differences
in abundance
associated with
these curves
15,000
Very Sensitive
Sensitive
10,000
Neutral
5,000
Insensitive
Very Insensitive
0
5
10
15
20
25
Eggs Just Prior to Hatching (millions)
No reliable baseline information
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Stage 1 - Decision Analysis w current uncertainty
Management
Actions
Columbia River
Flows During
Whitefish
SPAWNING
States of Nature and their Probabilities
Kootenay River
Flows During
Whitefish
SPAWNING
Min. Columbia River
Flows Prior to
Whitefish
HATCHING
Min. Kootenay
River Flows Prior
to Whitefish
HATCHING
Outcomes
Egg-Age4
Recruitment
Relationship
Egg
Abundance
Abundance
4+ Recruits
Foregone
Power
Revenues
20 kcfs
30 kcfs
10 kcfs
15 kcfs
10 kcfs
a1, b1
15 kcfs
20 kcfs
15 kcfs
a2, b2
40 kcfs
45 kcfs
50 kcfs
55 kcfs
60 kcfs
65 kcfs
20 kcfs
..
.
55 kcfs
25 kcfs
..
.
85 kcfs
20 kcfs
a3, b3
..
.
a4, b4
Model
a5, b5
55 kcfs
70 kcfs
80 kcfs
85 kcfs
Natural variability in flow
Uncertainty due to lack
of understanding / data
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Stage 1 Results: Current Uncertainty
Expected adult N, year 50
50,000
Minimum desired
Base Case: Current Uncertainty
45,000
N
40,000
Objective:
Maintain “least cost”
whitefish population nearest
to or greater than 45,000
adults
35,000
0.4
30,000
Whitefish recruitment dynamics:
Current state of knowledge
20 30 40 45 50 55 60 65 70 80 85
HKD Spawning Q (kcfs)
P 0.2
0
H1
(sensitive)
Overview of Decision Analysis with examples - Jan 10, 2002
H2
H3
H4
H5
(insensitive)
ESSA Technologies
Stage 2 - Simulated learning from flow
experiments and monitoring
Uses same model and uncertain components but...
Actions are now alternative experimental
flow regimes + monitoring programs
Assume a true relationship for population
dynamics with process error
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
What would you change if you knew the “truth”?
If population insensitive, then maximize power revenues (85 kcfs)
If population sensitive, then minimize biological risk (~60 kcfs)
Minimum desired
Expected adult N, year 50
Current Uncertainty
Sensitive
Insensitive
50,000
N
45,000
10
40,000
7.5
$Cnd mil
5
Max. potential
power revenues (per
yr)
35,000
30,000
2.5
25,000
20
30
40
45
50
55
60
65
70
80
85
HKD Spawning Q (kcfs)
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Example Stage 2 Results: Good monitoring is
critical for differentiating hypotheses; flow
manipulation had less effect than expected.
Probability identify insensitive population
(10-year experiments)
Natural Variability and Measurement Error
Low Nat Variability
Flow
manipulation
Constant
Passive
Active
High Meas.
Error
0.55 ($0.48)
0.60 ($1.23)
0.63 ($3.48)
High Natural Variability
Low Meas.
High Meas.
Error
Error
0.88 ($1.55) 0.51 ($0.48)
0.92 ($2.3) 0.57 ($1.23)
0.92 ($4.55) 0.54 ($3.48)
Low Meas.
Error
0.74 ($1.55)
0.85 ($2.3)
0.85 ($4.55)
$ CDN Millions
Blue = things under AM practitioners control
Red = beyond AM practitioners control
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
AM can “pay for itself”
Increase in annual power revenues from
operating with experimental information
(insensitive population only, 10-year
experiments)
Natural Variability and Measurement Error
Low Nat Variability
Flow
manipulation
High Meas.
Error
Low Meas.
Error
High Natural Variability
High Meas.
Error
Low Meas.
Error
Constant
$0.2 (2.4)
$0.6 (2.6)
$0.2 (2.4)
$0.2 (7.7)
Passive
$0.2 (6)
$0.6 (3.8)
$0.2 (6.15)
$0.6 (3.8)
Active
$0.6 (5.8)
$0.6 (7.6)
$0.2 (17.4)
$0.6 (7.6)
$Cnd millions
Numbers in brackets = experimental pay-back interval in years
Blue = things under AM practitioners control
Red = beyond AM practitioners control
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Is AM and monitoring worth it?
“Yes” If
New information leads to choice of a
different management action that
better satisfies a particular objective,
or
rigorously confirms that current
management action is appropriate.
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
No definitive “yes/no”
Factor
Under AM
practitioners control
Can evaluate implications
using decision analysis?
Management objective
(fish vs. power $)
Yes
Yes
Ability to do well designed
experiments
Yes
Yes
Initial level of uncertainty in
alternative hypotheses
Maybe
Yes
Magnitude of natural variability
in the system
No
Yes
What “truth” really is
No (can’t know without
doing the experiment)
Yes
Inherent sensitivity of best
action to uncertainty
No
Yes
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
General Conclusions
• Value of AM potentially large
• Whether to proceed depends on “the kind” of
system you are in (i.e. previous factors)
• Decision Analysis is very helpful for evaluating
these benefits
– Determine which uncertainties have strongest effect on
choice of “best” management decision
– Decisions more robust to uncertainties
(reduces risk - integrates broader range of possible
outcomes included)
– Include new information as revised probabilities on
hypotheses
Overview of Decision Analysis with examples - Jan 10, 2002
ESSA Technologies
Decision Analysis - Summary
PATH –
scientist
consensus
Cheakamus –
stakeholder
consensus
Keenleyside –
AM evaluation
Actions



Objectives



Uncertainties



Probabilities

Element of Decision
Analysis
Model




Decision Tree

Rank Actions






Sensitivity Analyses
Overview of Decision Analysis with examples - Jan 10, 2002

ESSA Technologies