What goes wrong with ecosystem
models as tools for policy design
And what can we do about it?
Carl Walters, Fisheries Centre, University of British Columbia
Changing Currents, February 26, 2005
Policy design involves choices, and choice
necessary involves predictions about how impacts
may vary with policy choice. So there is no way to
avoid predictive models of some sort. The only
issue is which one(s) to use:
• Intuitive assessments of response directions and
magnitudes
• Simple models that emphasize “major” interactions
and dependencies
• Detailed models that pretend to capture “all” the
relevant factors
An explosion of ecosystem
modeling over the past five years
has produced many apparently
credible models that fit historical
data well and make reasonable
policy predictions
But habitat changes (including those
caused by fishing) and intensive fishery
removals are creating essentially novel
situations. Only a fool would pretend to
understand the “mechanics” of
ecological response well enough to be
able to predict “all” important responses
to these novel situations.
Getting 90% of your model equations right
does not mean that your model is 90% sure
to give the right answers.
• “Details” that are hard to model can
sometimes have huge effects
• Examples:
– Clouds and particulates, in climate models
– “minor” prey that are only eaten in small
space-time windows, in ecosystem models
– Compensatory behaviors in response to
habitat change, in fish habitat models
A score card for using models to address
ecosystem management questions
CONCERN
Bycatch impacts
GRAD COMMENT
E
We are not bad at predicting direct
Aeffect of fishing in general
C
Trophic effects of fishing can be
classified as “top down” or “bottom
up” with respect to where
management controls are exerted
-on valued prey
B
Changes in M for prey species
already subject to assessment
-on “rare” prey
F
Outbreaks of previously rare
species
Top-down
effects
(of predator culling or
protection)
Modeling scorecard (continued)
CONCERN
GRADE COMMENT
Uncertainty here is about flexibility of
Bottom-up effects
C
predators to find alternative food
(effects of prey harvesting
on predator stocks)
Multiple stable
states
Habitat damage
Production
regime changes
Selective fishing
practices/policies
Regime shifts
sources when prey are fished
B
Cultivation-depensation mechanism
appears to be main mechanism that
could cause “flips”
D
Lack of understanding about real
habitat dependencies, bottlenecks
C
Models look good when fitted to
data, but have not stood test of time
F
Fine-scale changes in competition
and predation regimes
C
Policy adjustments in response to
ecosystem-scale productivity change
All of the predictive approaches are
failing, for some obvious reasons
• Lack of long term monitoring data on non-target species
and life stages
• Concentration of interaction effects (trophic, habitat) on
early life stages (recruitment) that are difficult to monitor
• Confounding of fishery, environmental, and trophic
effects in historical data
• Failure to anticipate “new problems” (emergent novelty)
due to unpredictable changes in system structure (exotic
invasions, fisheries inventions)
• Unpredictable pre-adaptations to habitat alterations
Failure to anticipate new problems
(emergent novelty)
• Exotic invasions, eg zebra mussel and
white perch in Great Lakes
• Unexplained “explosions” of small flatfish
and ratfish off B.C. coast
• Innovations in fisheries technologies and
markets, e.g. mothership dropline
operations and live trade of reef piscivores
in the Great Barrier Reef.
Lack of long term monitoring data
on nontarget species
• There be vampires in the basement.
• Example: Myers points out that
comparison of longline surveys in the
central North Pacific shows massive
increase in small piscivores (pomfrets)
since 1950. Is this change real (models
do not predict it) and is it causing changes
in tuna recruitments (cultivationdepensation effects)?
The cultivation-depensation
hypothesis
Cod, tuna, …
Reduction in large piscivore
abundance can “release” increases
in smaller piscivore species, which
then cause reduced survival of
juvenile piscivores.
Adult piscivores
Predation (-)
Growth (+),
reproduction
Arctic cod, pomfret,…
Small piscivore
species
Juvenile piscivores Competition, predation (-)
Concentration of interaction effects
on early life stages that are difficult
to monitor
• High juvenile mortality rates, thought to be
mainly by predation, concentrated in short
space-time windows
• Example: marine mammal predation on
juvenile salmon in Puget Sound-Georgia
Strait
• Example: tiger shark predation on juvenile
monk seals in NWHI
Most ecosystem models today are
age-size structured, but…
Some older models
(MSVPA, MSFOR)
concentrated on
estimating and
predicting mortality
rates for the older
ages for which data
are more readily
available, treating
recruitment rate (in
red) as a black box
like we do in most
single species
models.
Species 1
Species 2
N4
N4
N3
N3
N2
N2
R
R
Species 3
N4
N3
N2
Z
R
B
Newer models like
Ecosim try to
represent
interaction effects
at all life stages,
particularly the
higher mortality
rates (and hence
potential for
change), habitat
effects, and other
factors affecting
the recruitment
component of the
dynamics.
Confounding of fishery,
environment, and trophic effects:
monk seals in NWHI
Fishing effort:
Initial ecosim runs: fishing+
Trophic interactions only
did not explain monk seal
decline, predicted lobster
recovery
Satellite chlorophyll data
indicate persistent 40-50%
decline in primary production
around 1990; this “explains” both
continued monk seal decline and
persistent low lobster abundance
Lo chl
Are seals causing fish declines in the
Georgia Strait, or is it fishing, or is it
oceanographic change, or is it all three?
Surprising responses to habitat change: coho
salmon changes after logging
Survival
Egg to Fry Survival in Carnation Creek: red points show years
during and after experimental logging
40%
35%
30%
25%
20%
15%
10%
5%
0%
1971
1975
1979
1983
1987
1991
1995
Year
Total Smolt Output from Carnation Creek: did not decline as
expected from decline in egg-fry survival
6000
Numbers
5000
4000
3000
2000
1000
0
1971
1975
1979
1983
1987
1991
1995
Year
500
Adult and Jack Coho Returning to Carnation Creek: not
clearly related to juvenile survival or abundance
Numbers
400
300
200
100
0
1971
1975
1979
1983
Year
1987
1991
1995
Particularly when fisher behavior is included,
ecosystem models often predict highly
“counterintuitive” policy effects, even in terms of
the direction of response. Such situations cry out
for treating policies as experimental treatments:
• Bycatch reductions in Gulf of Mexico
shrimp fisheries may cause severe
decreases in recruitment of both shrimp
and red snapper.
• Marine protected areas may suffer higher
fishing pressure than surrounding open
areas.
Three definitions of “adaptive
management”
• Original Walters-Hilborn: treating management
policy options as experimental treatments in
situations where small-scale science fails to
resolve large-scale uncertainty
• US Forest Service: monitoring to identify needs
for corrective policy change
• Modern Bureaucratic: using modeling and
facilitation processes to build stakeholder
consensus about policy choices
There is catastrophic misunderstanding
about the capability of scientists to
provide advice about large-scale
dynamics
• Have you read “State of Fear”, Crichton’s analysis of
what scientists can and cannot say about climate
change? Scientists cannot resolve uncertainties about
how big systems change, using only models and timeseries data.
• It is not true that treating scientific consensus as fact is a
wise precautionary approach; there is only so much
money to spend on corrective action:
– What if climate change is being driven by land use changes
rather than CO2?
– What if salmon are declining because of ocean ecosystem
changes rather than freshwater habitat loss?
– What if tuna longline fisheries are shut down because of flawed
analysis of their impacts, while the real problem is purse-seine
fishing?
Once you get us busy trying to
defend a hypothesis, scientists can
be quite inventive:
• We can show you only those facts that fit
the hypothesis.
• We can even alter the facts to fit, by
analyzing raw data in different ways.
• We can baffle each other, and you, with
complex terminology and models.
• Trust us, we are highly trained
professionals.
Getting 90% of your model equations right
does not mean that your model is 90% sure
to give the right answers.
• “Details” that are hard to model can
sometimes have huge effects
• Examples:
– Clouds and particulates, in climate models
– “minor” prey that are only eaten in small
space-time windows, in ecosystem models
– Compensatory behaviors in response to
habitat change, in animal habitat models
High uncertainty in ecosystem model predictions
implies a clear need to treat all ecosystem
management initiatives as adaptive management
experiments. But you will probably fail to implement
such experiments, because of:
• People who fervently believe that they already have the
answers (i.e. know what policies to use)
• People who refuse to embrace uncertainty in policy
design because they fear loss of credibility, authority
• People who fear any risk of policy failure, and do not
understand that “safe” options are not safe
• The high cost of monitoring experiment responses,
and/or high risks of investing in new technologies and
institutional arrangements (e.g. data gathering by
fishers) to reduce monitoring cost
Adaptive management failures in
British Columbia
• Watershed restoration program: several hundred
million dollars to spend on many watersheds, but almost
no investment in monitoring or use of opportunities for
experimental treatment comparisons.
• B.C. Hydro Water Use Planning process: an offer of
water for fish at over 40 facilities, followed by mindless
demands for restoration of “natural” hydrographs.
• Deep water rockfish management: an offer by industry
to have large closed areas, cooperative survey
monitoring programs; offer was rejected by federal
Department of Fisheries and Oceans
Dishonesty and denial of hard decision problems has
spread like cancer through our public agencies. The
ground controllers know that some day faulty nav and
comm systems in 207 could cause an accident…