WE PROBABLY COULD HAVE MORE
FUN TALKING ABOUT THESE
TRAFFIC STOPPERS
NPS Conference 2006 # 1
WHO CLEARLY HAVE
THE RIGHT OF WAY!
BUT…
NPS Conference 2006 # 2
DESIGNING SURVEYS OVER
TIME
(PANEL SURVEYS)
VARIANCE, POWER and RELATED
TOPICS
N. Scott Urquhart
Senior Research Scientist
Department of Statistics
Colorado State University
Fort Collins, CO 80527-1877
NPS Conference 2006 # 3
BRIEF COMMENTS ON MONITORING
 Monitoring is a long-term endeavor for most
(living) natural resources

Think of it as a legacy for your grandchildren
 Many of you have tried to use solid data someone
else gathered, but documented poorly.

Much good data “dies” when the person
who gathered it dies or retires!
 Monitoring data requires three things to retain its
value:



Metadata
Storage in a retrievable, maintained, data system
Backed up – safe from fires, floods, or earthquakes
NPS Conference 2006 # 4
SHORT OUTLINE OF THE REST OF
THIS TALK
 Inference Perspectives & monitoring
 Trend, Variance Structures, and Power to Detect Trend
 Panels and Panel Structures
 Putting it all together = Power Curves
{and standard errors of estimated current status}
 Status Estimates from Data Over Years
NPS Conference 2006 # 5
INFERENCE PERSPECTIVES
 Design Based

Inferences rest on the probability structure
incorporated in the sampling plan
 Completely defensible; very minimal assumptions
 Limiting relative to using auxiliary information
 Model Assisted


Uses models to complement underlying sampling
structure
Has opportunities for use of auxiliary information
 Model Based (eg: spatial statistics)


Ignores sampling plan
Defensibility lies in defense of model
NPS Conference 2006 # 6
APPROACH OF THIS PRESENTATION
 Use tools from the arena of


Model-assisted and
Model-based analyses
 To study the performance of

Design based &
 Model-assisted analyses
 WHY?

Without models,
 performance evaluations need simulation
(Steve Garman’s topic)
 Before substantial data have been gathered

Minimal basis for values to enter into simulation studies
NPS Conference 2006 # 7
STATUS & TRENDS OVER TIME
IN ECOLOGICAL RESOURCES
OF A REGION
MAJOR POINTS
 Regional trend  site trend
 Detection of trend requires substantial elapsed time
 Regional OR intensive site
 Almost all indicators have substantial patterns in
their variability

Design to capitalize on this; don’t fight it.
 Minimize effect of site variability with planned
revisits – specific plans will be illustrated
 Design tradeoffs: TREND vs STATUS
NPS Conference 2006 # 8
REGIONAL TREND  SITE TREND
 The predominant theme of ecology:


Ecological processes
How does a specific kind of ecosystem function
 Energy flows
 Food webs
 Nutrient cycling

Most studies of such functions must be
 Temporally intensive
– What material goes from where to where?
 Consequently spatially restrictive

In this situation: Temporal trend = site trend
NPS Conference 2006 # 9
REGIONAL TREND  SITE TREND
( - CONTINUED)
 The predominant theme of ecology
versus
 A Substantial (any) Agency Focus:

All of an ecological resource
 In an area or region
 Across all of the variability present there
 For Example, National Park Service


All riparian areas in Olympic National Park
All riparian areas in National Parks in the
coastal Northwest
NPS Conference 2006 # 10
TREND ACROSS TIME - What is it?
 Any response which changes across time in a
generally


Increasing or
Decreasing
Manner shows trend

Monotonic change is not essential.
 If trend of this sort is present,
it WILL BE detectable as linear trend.

This does NOT mean trend must be linear (examples
follow)
 Any specified form is detectable
 Time = years, here
NPS Conference 2006 # 11
TREND ACROSS TIME - What is it?
(continued)
TREND = YES
TREND = NO; PATTERN = YES
90
90
70
70
50
1989
1991
1993
1995
50
1989
1991
Year
1993
1995
1997
Year
TREND = YES, PATTERN = YES
TREND = NO; PATTERN = YES
400
350
300
300
CARBON DIOXIDE
CONCENTRATION (ppm)
CARBON DIOXIDE
CONCENTRATION (ppm)
350
250
200
150
100
50
0
1955
1965
1975
1985
YEAR
1995
2005
250
200
150
100
50
0
1955
1965
1975
1985
1995
2005
YEAR
NPS Conference 2006 # 12
TREND = NO; PATTERN = YES
DETRENDED CARBON DIOXIDE
CONCENTRATION (ppm)
350
300
TREND
= NO; PATTERN = YES
250
200
150
100
50
0
1955
1965
1975
1985
1995
YEAR
NPS Conference 2006 # 13
TREND DETECTION REQUIRES
SUBSTANTIAL ELAPSED TIME
 IT IS NEARLY IMPOSSIBLE TO DETECT
TREND IN LESS THAN FIVE YEARS. WHY?
var ( ˆ ) 
2
2
(
t

t
)
 i
YEARS
2
(
t

t
)
 i
3
4
5
6
7
8
9
10
2
5
10
17.5
28
42
60
82.5
NPS Conference 2006 # 14
VARIANCE HAS A LOT OF STUCTURE
IMPORTANT COMPONENTS OF VARIANCE
 POPULATION VARIANCE:
2
( SITE
)
2
(

 YEAR VARIANCE:
YEAR )
2
(

 RESIDUAL VARIANCE:
RESIDUAL )
NPS Conference 2006 # 15
HOW SHOULD YOU ESTIMATE
VARIANCE?
 Alternatives:

Designed-based
 Horwitz-Thompson – extremely variable = don’t use
 Local Neighborhood Variance Estimator (NBH) – Stevens
– Gives estimate of variance something like residual
component of variance, only
– Power nearly impossible to evaluate in this context

Model-assisted
 Linear models – Urquhart & Courbois (& Williams, now)
 Gives estimates of site, year and residual variances

Which should you use?
NPS Conference 2006 # 16
HOW SHOULD YOU ESTIMATE
VARIANCE?
(Continued)
 We really aren’t sure, but this topic is under
active investigation
 Don Stevens and I are good friends, so this topic
isn’t a professional conflict.

We both want to know the answer!
 Results by Courbois (JABES, 2004) suggest this:

Unless response values and inclusion probabilities (p)
are highly correlated, they (p) can be ignored.
 If this stands up, as I expect it to, practically, linear model
estimates of components of variance will be fine.
 Answer expected by summer.
NPS Conference 2006 # 17
IMPORTANT COMPONENTS OF VARIANCE
( - CONTINUED)
2
 POPULATION VARIANCE: ( SITE
)

Variation among values of an indicator (response) across
all sites in a park or group of related parks, that is, across
a population or subpopulation of sites
NPS Conference 2006 # 18
IMPORTANT COMPONENTS OF VARIANCE
( - CONTINUED II)
2
(

 YEAR VARIANCE:
YEAR )

Concordant variation among values of an indicator
(response) across years for ALL sites in a regional
population or subpopulation

NOT variation in an indicator across years at a single site

Detrended remainder, if trend is present
 Effectively the deviation away from the trend line (or other
curve)
NPS Conference 2006 # 19
IMPORTANT COMPONENTS OF VARIANCE
( - CONTINUED - III)
 Residual component of variance

Has several contributors

Year*Site interaction
2
( RESIDUAL
)
 This contains most of what ecologists would call year to year
variation, i.e. the site specific part

Index variation
 Measurement error
 Crew-to-crew variation
(minimize with well documented protocols and training)
 Local spatial = protocol variation
 Short term temporal variation
NPS Conference 2006 # 20
SOURCE OF DATA FOR ESTIMATES OF
COMPONENTS OF VARIANCE
 EMAP Surface Waters:
Northeast Lakes Pilot 1991 - 1994
 About 450 observations



Over four years
Including about 350 distinct lakes
Design allowed estimation of several residual
components
 Lakes illustrate what is generally referred to
in this presentation as “sites.”

Similar patterns appear in other data sets
 Both aquatic and terrestrial
NPS Conference 2006 # 21
COMPOSITION OF TOTAL VARIANCE - NE LAKES
Acid Neutralizing Capacity
LAKE COMPONENT OF VARIANCE
Ln(Conductance)
Ln(Chloride)
pH(Closed system)
Secchi Depth
Ln(Total Nitrogen)
Ln(Total Phosphorus)
Ln(Chlorophyll A)
YEAR
Ln( # zooplankton taxa)
Ln( # rotifer taxa)
Maximum Temperature
0.00
RESIDUAL COMPONENT OF VARIANCE
0.20
0.40
0.60
0.80
1.00
PROPORTION OF VARIANCE
NPS Conference 2006 # 22
ALL VARIABILITY IS OF INTEREST
 The site component of variance is one of the major
descriptors of the regional population
 The year component of variance often is small, too
small to estimate. It is a major enemy for
detecting trend over time.
If it has even a moderate size, “sample size”
reverts to the number of years.
 In this case, the number of visits and/or number of sites
has no practical effect.

NPS Conference 2006 # 23
ALL VARIABILITY IS OF INTEREST
( - CONTINUED)
 Residual variance characterizes the inherent
variation in the response or indicator.
 But some of its subcomponents may contain
useful management information

CREW EFFECTS ===> training
 VISIT EFFECTS ===> need to reexamine definition of
index (time) window or evaluation protocol
 MEASUREMENT ERROR ===> work on
laboratory/measurement problems
NPS Conference 2006 # 24
DESIGN TRADE-OFFS: TREND vs STATUS
 How do we detect trend in spite of all of this
variation?
 Recall two old statistical “friends.”


Variance of a mean, and
Blocking
NPS Conference 2006 # 25
DESIGN TRADE-OFFS: TREND vs STATUS
( - CONTINUED)
 VARIANCE OF A MEAN:
var (mean) 
2
m

Where m members of the associated population have
been randomly selected and their response values
averaged.

Here the “mean” is a regional average slope, so "2"
refers to the variance of an estimated slope ---
NPS Conference 2006 # 26
DESIGN TRADE-OFFS: TREND vs STATUS
( - CONTINUED - II)
 Consequently
var (mean) 
 Becomes
2
m
1
2
var (regional mean slope) 
m  ( ti  t ) 2
 Note that the regional averaging of slopes has the
same effect as continuing to monitor at one site for
a much longer time period.
NPS Conference 2006 # 27
DESIGN TRADE-OFFS: TREND vs STATUS
( - CONTINUED - III)
 Now, 2, in total, frequently is large.
 If we take one regional sample of sites at one time,
and another at a subsequent time, the site
component of variance is included in 2.
 Enter the concept of blocking, familiar from
experimental design.

Regard a site like a block

Periodically revisit a site

The site component of variance vanishes from the
variance of a slope.
NPS Conference 2006 # 28
PANEL DESIGNS
 Question: “ What kind of temporal design should
you use for National Parks?
 A Panel is a Set of Sites which have the same
Revisit Schedule

Each panel ordinarily should have as good a
spatial coverage as possible (GRTS)
 You have many usable and defensible temporal
designs


Choose one which fits your needs and resources
Evaluation tools are available, and demonstrated here
NPS Conference 2006 # 29
A SINGLE PANEL
NUM
OBS
11
YEAR
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
 Conventional for trend detection
 Not very good for status
 (Call this design 1)
NPS Conference 2006 # 30
AN “AUGMENTED” PANEL PLAN (Design 2)
NUM
OBS
5
YEARS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
…
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
…
 Add sites to the above annual revisit plane, as
NPS Conference 2006 # 31
AN “AUGMENTED” PANEL PLAN (Design 2)
NUM
OBS
YEARS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
…
5
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
…
6
X
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
…
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
…
NPS Conference 2006 # 32
A POSSIBLE NPS PANEL PLAN (Design 3)
NUM
OBS
YEARS
1
5
5
5
5
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
X
17
18
19
X
X
X
X
NPS Conference 2006 # 33
…
A POSSIBLE NPS PANEL PLAN (Design 3)
NUM
OBS
YEARS
1
5
2
3
4
5
6
7
8
9
10
11
12
13
14
15
X
1
…
19
X
5
1
18
X
5
1
17
X
5
4
16
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
…
NPS Conference 2006 # 34
A POSSIBLE NPS PANEL PLAN (Design 3)
NUM
OBS
YEARS
1
5
2
3
4
5
6
7
8
9
10
11
12
13
14
15
X
X
X
1
X
X
1
1
2
1
1
1
2
…
X
X
X
1
1
…
19
X
5
1
18
X
5
2
17
X
5
4
16
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
…
X
X
X
X
X
X
X
X
X
X
…
X
X
X
X
X
X
X
X
X
X
X
X
…
NPS Conference 2006 # 35
WHY LOOK AT POWER?
 Power provides a tool for comparing various designs
 Looking at it does not imply that we have to conduct
tests of hypotheses
 The computations displayed here use

A components of variance statistical model
 To evaluate the underlying variances of estimated slopes
 And the temporal design being considered

These computations are much more complex, and
suitable to your problems, than any of the
web-available tools
 Use of those tools here would commit “sins of
pseudoreplication”!
NPS Conference 2006 # 36
Power for Trend Detection
1
POWER FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{large is good}
0.8
Design 1: One panel n = 11
0.6
0.4
0.2
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 37
Power for Trend Detection
1
POWER FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{large is good}
0.8
Design 1: One panel n = 11
0.6
Design 2: Augmented, n = 5 + 6
0.4
0.2
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 38
Power for Trend Detection
1
POWER FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{large is good}
0.8
Design 1: One panel n = 11
0.6
Design 2: Augmented, n = 5 + 6
0.4
0.2
Design 3: A NPS proposal, n varies from 3 to 12
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 39
Power for Trend Detection
1
POWER FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{large is good}
0.8
Design 1: One panel n = 11
0.6
Design 2: Augmented, n = 5 + 6
0.4
0.2
Design 2A: Augmented n = 3 + 2 or 3
Design 3: A NPS proposal, n varies from 3 to 12
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 40
STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{small is good}
Power for Trend Detection
0.6
0.5
0.4
Design 1: One panel n = 11
0.3
0.2
0.1
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 41
STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{small is good}
Power for Trend Detection
0.6
0.5
0.4
Design 1: One panel n = 11
0.3
0.2
Design 2: Augmented, n = 5 + 6
0.1
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 42
STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{small is good}
0.6
Power for Trend Detection
Design 3: A NPS proposal, n varies from 3 to 12
0.5
0.4
Design 1: One panel n = 11
0.3
0.2
Design 2: Augmented, n = 5 + 6
0.1
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 43
STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A
WITH PARAMETER CONDITIONS BILLY GAVE
{small is good}
0.6
Power for Trend Detection
Design 3: A NPS proposal, n varies from 3 to 12
0.5
0.4
Design 1: One panel n = 11
0.3
Design 2A: augmented n = 3 + 2 or 3
0.2
Design 2: Augmented, n = 5 + 6
0.1
0
0
5
10
15
20
25
30
Elasped Years
NPS Conference 2006 # 44
DESIGN 3 – A PROPOSED NPS
TEMPORAL DESIGN??
 Defensible design, given restrictions of


Field resources = # sites visited per year
Need to split field resources across two type of
aquatic systems:
 Streams
 Wetlands
NPS Conference 2006 # 45
STATUS ESTIMATES
FROM DATA OVER YEARS
 Idea:



Estimate year effects, and
Adjust all sites to the latest year
Idea very similar to “adjusted treatment means”
in the analysis of covariance
 For example, if a line approximates the trend,



The array of { Sitei + (yearnow – yearinitial)slope }
Describes current status, accounting for documented
trend
This approach extends to models other than lines.
 Linkage of site revisits (=connectedness) is necessary
to allow estimation of all differences in year effects
NPS Conference 2006 # 46
FUNDING ACKNOWLEDGEMENT
The work reported here today was developed under the STAR Research Assistance
Agreement CR-829095 awarded by the U.S. Environmental Protection Agency (EPA) to
Colorado State University. This presentation has not been formally reviewed by EPA. The
views expressed here are solely those of presenter and STARMAP, the Program he
represents. EPA does not endorse any products or commercial services mentioned in this
presentation.
This research is funded by
U.S.EPA – Science To Achieve
Results (STAR) Program
Cooperative
# CR - 829095
Agreement
NPS Conference 2006 # 47
RELATED INFORMATION ON THE WEB

Web-available information from a graduate course in environmental
Sampling (ST571) I taught at OSU

http://oregonstate.edu/instruct/st571/urquhart/index.html














Environmental Sampling
Anatomy
Variable Probability Sampling
Cost Effective Resource Allocation
Sampling Macroinvertebrates
Maps & Grids
Spatial Sampling
Support Regions
Statistical Power - Concepts
Power to Detect Trend in Ecological Resources
Representative Sampling
Statistical Aspects of Taxonomic Richness
Sample Size to Estimate Taxonomic Richness
Evaluating a Protocol for "Measuring" Physical Habitat
 Also see: http://www.stat.colostate.edu/starmap/
 Also see NPS I&M site, from Port Angeles meeting, 2003
NPS Conference 2006 # 48
VISUALIZING LINES AND
YEAR EFFECTS
N. Scott Urquhart
STARMAP
Colorado State University
Fort Collins, Co 80523-1877
NPS Conference 2006 # 49
WHAT WE SEE – NO LINES &
NO DECOMPOSITION
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 50
A LINE
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 51
A LINE WITH YEAR EFFECTS
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 52
A LINE WITH YEAR EFFECTS & ANNUAL
RESIDUALS
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 53
A LINE WITH ONLY ANNUAL
RESIDUALS SHOWN
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 54
A LINE WITH ANNUAL RESIDUALS
& END MARKERS
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 55
A LINE WITH YEAR EFFECTS,
JIGGERED RESIDUALS & END
MARKERS
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 56
A REALITY
 With a single site



The year effect
And residual effect
Can not be separated
 But with several sites

These effects can be separated
 Following figures show the patterns
NPS Conference 2006 # 57
TWO LINES
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 58
TWO LINES WITH YEAR EFFECTS
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 59
TWO LINES WITH YEAR EFFECTS &
JIGGERED RESIDUALS
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 60
TWO LINES WITH YEAR EFFECTS &
JIGGERED RESIDUALS – ONLY SOME YEARS
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 61
WHAT WE SEE – NO LINES &
NO DECOMPOSITION
12
RESPONSE VALUES
10
8
6
4
2
0
0
5
10
YEARS
15
20
NPS Conference 2006 # 62