Online Resource 3: Detailed Methods
Refuge Models
The timestep for the model is seasonal. We model water depth in the spring, summer and fall seasons
using σ {1,2,3}  {spring, summer, fall} to represent the current season. We do not model winter as
few waterfowl overwinter at the refuge. Winter inflows are held as snow and released in the spring,
where they contribute as spring inflows to the refuge. The variable t {0,1,2,....} increments every 3
seasons and records the year. Between timesteps, season and year interact as follows:
  1 , if   3
 
, otherwise
1
, if   3
t
t 
t  1 , otherwise
Inflow to the refuge is based on seasonal inflow data collected at the refuge. Inflow is a stochastic
variable. We fit a gamma distribution to the inflow data for each season. This gives the probability of
obtaining an inflow of a given magnitude. A random draw from the relevant gamma distribution in each
season determines the inflow for the season during model simulation. This inflow enters the model at
the most upstream pool and is treated according to equation ( 1 ).
S ( , t )  Qi ( , t )  E ( ) , if S i ( , t )  Qi ( , t )  E ( )  Ci (d i ,  , t )
S i (  1, t )   i
, if S i ( , t )  Qi ( , t )  E ( )  Ci (d i ,  , t )
Ci (d i ,  , t )

0
Oi ( , t )  
Ci (d i ,  , t )  ( S i ( , t )  Qi ( , t )  E ( ))

(1)
, if S ( , t )  Qi ( , t )  E ( )  ...
Ci ( d i ,  , t )
, if S i ( , t )  Qi ( , t )  E ( )  ...
Ci ( d i ,  , t )
Where Si(σ ,t) is the stored volume of pool i before the inflow arrives in season σ and year t, Qi(σ ,t) is
the inflow entering the pool, E(σ) is the volume lost to evaporation, Ci(di,σ ,t) is the capacity of pool i
when the depth is d, and Oi(σ ,t) is the outflow from the pool. The outflow Oi(σ, t) flows into the next
pool downstream and becomes the inflow Qi+1(σ, t). Any outflow that leaves the system at the
downstream end is simply removed from the model.
Evaporation losses are treated as constant in the model. Average monthly evaporation data from the
National Weather Service Climate Prediction Center (National Oceanic and Atmospheric Administration
National Weather Service 2011) were summed over each season. We used a total evaporation loss of
0.8 inches, 4.5 inches and 1.0 inches (i.e., 20mm, 115mm, and 25mm) during the spring, summer and fall
respectively. Although evaporation has been predicted to increase over the Great Plains with a warming
global climate (Poiani, Johnson et al. 1995), the evaporation loss was small compared to the inflows for
these refuges in most cases because the refuge pools are filled by river inflow, unlike the surrounding
pothole wetlands that are filled by localized precipitation events.
The pool capacity Ci(di,σ,t) is determined by control structure height (CSH) at the downstream end of the
pool (pool depth). Pool depths can be converted to capacities using area-depth-volume conversions for
each refuge pool (online resource 4). We defined discretized pool depth classes for each pool (defined in
online resource 1) in each refuge with an upper limit at the maximum height of the control structure for
each pool. We denote these depths for pool i as d i {1,2,..., i } , where λi is the number of levels
included in the model for pool i. The state variable d{d1 , d 2 ,..., d n } represents the current depths of
each of the n pools in the refuge.
Refuge managers can control the capacity of the pools by raising or lowering the control structures and
changing pool depth. Pool depths can be converted to areas of different water depth classes available to
waterfowl using area-depth conversions for each pool (online resource 4). Each water depth class
represents a habitat for waterfowl and the depth classes are (1) shallow (< 2ft= <0.6m); (2) medium (25ft= 0.6-1.5m); and (3) deep (> 5ft= >1.5m). These water depth classes (habitats) are different than the
pool depth classes mentioned in the previous paragraph.
Each management action corresponds to one combination of control structure heights in each of the n
pools. For example in the model for Sand Lake NWR, there are five pool depth classes for Mud Lake and
seven pool depth classes for Sand Lake. This means that there are 5x7= 35 possible management
actions to consider. We use the notation a  (a1 , a 2 , ... , a n ) to refer to one combination of actions for
the refuge. The set of possible management actions is A, so that a  A .
The transition probability Pr( d ' | d ,  , a) determines the probability of a transition from a depth d to
depth d’ in season σ, assuming action a is taken. To determine this transition probability, we first
discretized the gamma distribution fits to inflows into M intervals of size r, and defined the bounds of
these intervals using q j  ( j  1) * r , j  {0,1,...M ) , and q M 1 
lim
Q (we used a resolution of
Q
r=100cfs). We denote the probability of an inflow between qj and qj+1 with Pr( q j  Q  q j 1 ) .
Let Ω denote the set of depths d’ that can be reached from starting depth combination d given an
inflow between qj and qj+1 and action a. We can use Ω to define an indicator function
1 , if d '  
that determines which states are accessible from state d. For a given starting
I  (d ' )  
'
0 , if d  
depth d and inflow Qi, there is a deterministic next state given by equation ( 1 ) and the volume-depth
relationship, so Ω will have just one element for each possible inflow. We can then define the transition
probability for moving from one pool depth class to another in the course of a season:
M
Pr(d | d ,  , a)   Pr(q j  Q  q j 1 ) I  (d ' ).
'
'
j 0
(2)
The model also includes a coarse resolution vegetation model. Refuge staff noted the importance of
cycling pool depth to provide a range of vegetation densities among pools (i.e., supplying a range of pool
depths throughout the year to allow occasional inundation of the areas adjacent to permanent water).
In particular, cycling the pool depth prevents the permanent establishment of high density perennial
emergent vegetation such as cattail and bulrush. Perennial emergents can crowd the wetland,
eliminating the open water that waterfowl use as habitat. In most of the refuges, perennial emergents
can be controlled by flooding the wetland and drowning the emergent vegetation. In Lacreek NWR,
emergents are controlled by reducing pool depth so the vegetation can be controlled by mechanical
means.
Three classes of emergent vegetation density are generally recognized: (1) low (< 20%); (2) medium (2080%); and (3) high (> 80%); density is the percentage of a pool surface that is occupied by emergent
vegetation. Each pool can be in one of these vegetation density classes in each season. We use the
symbol ψi to denote the vegetation state of pool i, i  {1,2,..., n} , and ψ={ψ1,ψ2,…,ψn} to represent the
current vegetation state of all n pools in the refuge.
Season-specific probabilities of transition among vegetation density classes:
Pr(d i , i' , , a | d i , i ,  , a)  Pr(i' | d i , i ,  , a)
were elicited from refuge managers (see online resource 5 for elicitation sheets). They estimated the
probability of ending the season in each vegetation density class given three refuge-specific pool depth
classes and vegetation density at the start of the season. Using continuous pool depths in elicitation was
impractical so we categorized pool depths as shallow, medium or deep. The definition of shallow,
medium and deep pools at each refuge was unique and based on the characteristics of the refuge. These
definitions were based on the point at which shallow and deep water depth classes first reached roughly
one third of the refuge area, except at Lacreek where definitions were based on the depths at which
medium and deep water depth classes first contributed to the area of each pool in the refuge (at
Lacreek deep water never reached one third of the refuge area). At Sand Lake, a pool depth < 2ft=0.6m
was shallow, a pool depth of 2-5ft=0.6-1.5m was medium and a pool depth > 5ft=1.5m was deep. At
Arrowwood and Lacreek, a pool depth < 6ft=1.8m was shallow, a pool depth of 6-10ft=1.8-3.0m was
medium and a pool depth > 10ft=3m was deep. At Tewaukon, a pool depth < 4ft= 1.2m was shallow, a
pool depth of 4-8.5ft=1.2-2.6m was medium, and a pool depth > 8.5ft=2.6m was deep. This pool depth
classification is only used in the model when evaluating vegetation state transitions. It does not affect
the objective function directly. For the water balance part of the model, the percentage of the pool
surface area in each water depth class is used to evaluate the objective.
Combining the transitions for all possible vegetation state changes, we obtain:
n
Pr( ' | d ,  ,  , a)   Pr( i' | d i ,  i , σ, a)
(3)
i 1
Finally, combining the transition probabilities for changes in both depth and vegetation state, we obtain:
Pr( d ' ,  ' | d ,  ,  , a)  Pr(d ' | d ,  , a) Pr( ' | d ,  ,  , a)
(4)
Equation ( 4 ) is the transition model that is used to determine the optimal solution.
Management Objective
The purpose of manipulating the water depth classes in the refuge pools is to provide an adequate
range of habitat types for waterfowl with different habitat requirements. Three water depth classes
were recognized and each represents a habitat for waterfowl. The depth classes are (1) shallow (<
2ft=0.6m); (2) medium (2-5ft=0.6-1.5m); and (3) deep (> 5ft=1.5m). Refuge managers agreed on a
common goal that 30% of the refuge wetlands should be in each water depth class during the spring
through fall periods. We set a management goal to maximize the area of the refuge in each water depth
class, up to a maximum of 30 for each depth class in each season. The maximum reward in any season
was 90 (≥ 30% in each depth class).
Each refuge pool needs to be occasionally manipulated to control perennial emergent vegetation that
will otherwise crowd the water surface and reduce the amount of open water available for waterfowl. A
second goal is to maintain perennial emergent vegetation at a low or intermediate level in each pool.
Summarizing these two goals, the management objective is:
Control water depths such that 30% of the wettable area (the sum of all pool surface areas when pool
depths were at maximum control structure heights) of the refuge is in shallow, medium and deep water
depth classes; and maintain the density of perennial emergent vegetation at low or medium densities.
Adaptive Management: Climate change scenarios
We do not know for certain how the inflow to the refuge will change in response to a changed climate.
To account for this, we included two possible future inflow scenarios, and attach a belief weight to each
model. Because the probability of inflow changes each time step (assuming a trend in inflow over time),
the transition probability (equation ( 4 ) must be calculated for each model at each timestep based on
the predicted inflows for each scenario at that timestep. We define the transition probability for the tth
timestep as
Pr( d ' ,  ' | d ,  ,  , a, t ) .
The belief weight bi , (0  bi  1) for model i represents the probability that model i is the correct
model (or degree of confidence in model i). In this context it would represent the probability that the
future climate will evolve most like model i. As the system is managed, new information becomes
available which is used to determine which model the system is following. This new data is used to
update the belief weights for each of the M models according to Bayes’ rule:
bi (t  1) 
Pr( d', ' | d ,  ,  t , a, t , mi )bi (t)
M
 Pr(d', '| d , ,
k 1
t
, a, t , mk )bk (t )
This study applies passive adaptive management, which means that model weights are held constant
over the period of management. Belief weights are still updated, but the new weights have no influence
on the management. In this way we gain information about the correct model, but don’t use this
information to affect the management decision until we re-calculate the optimal solution with new
weights.
Optimization
The first step in optimization is to articulate the management in terms of a reward function, which gives
the immediate benefit of a state. Let the total area of pool i be ω(Di) where Di is the maximum depth of
pool i at the control structure, and let the area of shallow, medium and deep habitat when the pool
depth is di be ω1(di), ω2(di), ω3 (di) respectively. The refuge-wide area in each water depth class is then
the sum of the areas in that class across all pools. We assign a reward equal to the percentage of the
total refuge area in each class, up to a maximum of 30 for each depth class in each season. The
maximum reward in any season was 90 (≥ 30% in each water depth class). The reward function R is given
by:
3
n c( ) (d )

i
j
i 

R(d )   min  30, 100

(
D
)
j 1
i

1
i


(5)
Where d  (d1 , d 2 , ..., d n ) is a vector of pool depth classes for the n pools in the refuge. The index j
refers to the water depth class, where j={1,2,3}= {shallow, medium, deep} respectively. The term c(ϕi) is
an indicator function that specifies if the vegetation in pool i is crowded. We define c(ϕi) using:
0 , if i  3
c(i )  
1 , if i  3
If the vegetation level is 3, then the pool surface will be crowded with cattail and bulrush. In this case,
the reward for the crowded pool is zero and it does not contribute rewards to any depth class.
The objective of the optimization is to maximize the expected sum of the immediate rewards articulated
in equation ( 5 ). This is achieved by defining a value function Vk(d, ψ, σ) that records the expected sum
of rewards for a state (d, ψ) in season σ after t years and k iterations. If the optimization is carried out
for T years, then the total number of iterations is 3 seasons x T years. If T=20 years then K=60 iterations.
To find the optimal solution, the value at the final iteration (k=K) is first set to be the reward resulting
from the action that offers maximum immediate reward:
VK (d ,, K , b)  amax
 A R(d ,  )
We used the value function to deduce the optimal management option to take for each state in the
previous iteration (i.e., iteration K-1) by computing the expected sum of rewards that resulted from each
of the possible actions. The process was repeated until the start of the management horizon (k= 0; t=0)
was reached. The process can be summarized using an iterative equation (sometimes referred to as the
finite-time Bellman equations):
M
Vk (d ,  ,  k , b)  amax
 A R(d ,  )    bi  Pr(d', ' | d ,  ,  k 1 , a, t , mi ) Vk 1 (d ' ,  ' ,  k 1 , bi )
i 1
d'
(6)
'
Where the variable bi represents the belief in model i. We solved the optimization separately for each of the
two scenarios in the manuscript, so that M=1 and bi =1. The parameter γ is called the discount factor, and is a
number that determines how we value future states compared to current states. We used a discount factor
of 0.99, which values states in all time steps similarly as well as ensuring convergence of the iterative
optimization over the long-term (Williams and Baird III 1993).
At each timestep the optimal action is the action that maximizes the value function (equation 6). The solution
gives the optimal height at which to set the control structure in each pool for each possible combination of
pool depth and vegetation density in each year and season.
References
National Oceanic and Atmospheric Administration National Weather Service (2011, Dec 1, 2011).
"Calculated Evaporation Climatology." Retrieved 3rd January, 2012, from
http://www.cpc.ncep.noaa.gov/soilmst/eclim_frame.html.
Poiani, K. A., W. C. Johnson, et al. (1995). "Sensitivity of a prairie wetland to increased temperature and
seasonal precipitation changes." JAWRA Journal of the American Water Resources Association 31(2):
283-294.
Williams, R. and L. Baird III (1993). Tight performance bounds on greedy policies based on imperfect
value functions.Technical Report NU-CCS-93-14. Boston, MA, Northeastern University, College of
Computer Science.