Designs for estimating variability
structure and implications for
detecting watershed restoration
effectiveness
• David P. Larsen
– Western Ecology Division, NHEERL, USEPA
– 200 SW 35th St. Corvallis, OR 97333
• N. Scott Urquhart
– Department of Statistics
– Colorado State University
– Ft. Collins, CO 80523
Topics
• Linear trend detection
– Applying the tools to restoration
monitoring
• Organizing variability
• Expanding the linear trend detection
model
• Variance summary
• Trend detection
A 2% per Year Linear Trend
(each point is a regional mean value)
For any patterned trend, there is an underlying
linear component.
A 2% per Year Linear Trend
(each point is a regional mean value)
Can we detect a difference is slope between
“treated” and “untreated” systems?
Reference
Linear trend detection
• Hypothesis test: Slope = 0?
• Power: If a trend is present, what is
the likelihood of detecting it?
• Hypothesis: Slope between treated
and reference = 0
• Power: likelihood of detecting if
different?
Linear trend detection
• Power depends on:
- magnitude of the trend (slope),
- variability of our measurements,
- number of sites,
- the duration of the study (how long
we can wait for the information).
Variance of a trend slope: How
precisely can we estimate it?
var( slope) 

 (X
2

X
)
i
2
Organizing Variability
• Four major components:
– Spatial
• Site-to-site
– Temporal (year to year)
• Year
• Site x Year Interaction
– Residual
----------Stream Size -----------> .
“SITE” VARIANCE:
Persistent Site-to-Site Differences
due to
Different Landscape/Historical Contexts
Different Levels of Human Disturbance
Year variation
• Concordant year-to-year variation
across all sites
• Caused by regional
phenomena such as:
– Wet/Dry years
– Ocean conditions
– Major volcanic
eruptions
Interaction variation
• Independent
year-to-year
variation among
sites
• Driven by local
factors
Residual variation
• The rest of it including:
– Temporal or seasonal variation during
sampling window
– Fine scale spatial variation
– Crew-to-crew differences in applying the
protocol
– Measurement error
–…
Design framework
• Multiple sites with revisits within and
among years
• Need a sample size of 30-50 to get
reasonable estimate of variance, i.e.,
30 – 50 sites; 30-50 revisits within
year; at least 5 years with some sites
visited annually, or at least in pairs of
adjacent years.
AUGMENTED SERIALLY ALTERNATING
PANEL
0
1
2
3
4
TIME PERIOD ( ex: YEARS)
1 2 3 4 5 6 7 8 9 10 11 12 13 ...
X X X X X X X X X X X X X
X
X
X
X
X
X
X
X
X
X
X
X
X
SERIALLY ALTERNATING WITH
CONSECUTIVE YEAR REVISITS
PANEL
1
2
3
4
TIME PERIOD ( ex: YEARS)
1 2 3 4 5 6 7 8 9 10 11 12 13 ...
X X
X X
X X
X
X X
X X
X X
X X
X X
X X
X X
X X
X X
Variance of a trend slope
site
(New sites each year)
residual
year interaction
2
r
2
2
i
s
v
2
y
s
s
var( slope) 

 

N
 
N
N
 (X
i
 X)
2
Xi = Year ; Ns= Number of sites in region;
Nv= Number of within-year revisits
(Urquhart and Kincaid. 1999. J. Ag., Biol., and Env. Statistics 4:404-414)
Variance of a trend slope
(Revisiting the same sites each year)
residual
interaction
year
2
2
r
i
2
v
y
s
var( slope) 

 
N
 
N
( X
i
 X)
Xi = Year ; Ns= Number of sites in region;
Nv= Number of within-year revisits
(See Urquhart & Kincaid, 1999)
2
Implications
• Effect of site = 0 if sites are revisited
across years
• Year is not sensitive to “sample size”and its
effect can become dominant
• Residual is affected by within year revisits
• Interaction and residual are affected by
number of sites in survey, therefore other
factors being equal, better to add sites to
the survey rather than revisit sites
Some options
(after adding sites doesn’t help)
• Extend survey interval
• Focus on subpopulations to manage
variance
• Monitor hypothesized covariates
controlling “year”
Adaptations for Effectiveness
Monitoring
• Context
– Comparing two
watersheds
Adaptations for Effectiveness
Monitoring
• Context
– Comparing
multiple
watersheds
– Some treated
( )
– Some
reference ( )
12
10
8
6
4
Indicator
Power to detect a
2% per year “drift”
from reference?
5
10
15
Year
20
25
Variance of the difference in two trend slopes
(New sites each Year)
residual
interaction
site
var( slope)  2



2
s
Nv

Ns
Ns
2
i
( X
i
 X)
Xi = Year ; Ns= Number of sites in each region;
Nv= Number of within-year revisits
2
r
2
Variance of the difference in two trend slopes
(Revisiting the same sites Each Year)
residual
interaction
2
2
r
i
v
var( slope)  2

 
N
Ns
( X
i
 X)
Xi = Year ; Ns= Number of sites in each region;
Nv= Number of within-year revisits
2
Denominator’s effect
Duration (yrs) S(Xi – X)2
9
60
10
82.5
11
110
12
143
13
182
14
228
15
280
Variance Summary
(Large wood)
Monitoring Site
area
Year
Interaction Residual
North
Coast
0.003
0.009
0.033
Mid-Coast 0.081
0
0.003
0.014
Mid-South 0.234
0.007
0.004
0.020
South
Coast
0.166
---
0.006
0.019
Umpqua
0.138
0.002
0
0.020
0.131
Design for power curves
• Annual visits, # sites varies
• Serially alternating design, with annual
panel
• Variance components values were selected
as low and high for Log10(LW+0.1)
• Alpha = 0.1
POWER CURVES FOR LOW VALUES OF
VARIANCE COMPONENTS
1
0.8
POWER
n = 25
0.6
n=5
0.4
0.2
0
0
5
10
YEAR
15
20
POWER CURVES FOR HIGH VALUES OF
VARIANCE COMPONENTS
1
0.8
POWER
n = 25
0.6
n=5
0.4
0.2
0
0
5
10
YEAR
15
20
POWER CURVES FOR HIGH VALUES OF VARIANCE
COMPONENTS; AUGMENTED ROTATING PANEL
DESIGN
1
0.8
POWER
n = 6, 24
0.6
n = 2,8
0.4
The first number gives the number of
sites in the "always revisit" panel;
the second number gives the size
of each of the rotating panels.
0.2
0
0
5
10
YEAR
15
20
Summary
• Characterization of spatial and temporal variation
• Design framework for estimating components of
variation
• A framework for evaluating linear trend
• How variation affects trend detection
• Modifying the framework for evaluating
restoration
• An example using large wood as an indicator