Research and Experimental Design
Edward O. Garton, Jon S. Horne,
Jocelyn L. Aycrigg, and John T. Ratti
2012
Department of Fish and Wildlife Sciences, University of
Idaho, Moscow, Idaho
Chapter 1 In Silvy, Nova (ed.). Techniques For Wildlife Investigations
and Management. Seventh edition, Vol. I Wildlife Research Methods.
The Wildlife Society, Bethesda, MD.
Crisis!
► There
is a crisis in wildlife, fisheries, ecology and
conservation biology.
► Our practice of scientific method is broken.
► Our rigor in designing research has been lost in a rush
to investigate.
► We have failed to lay the proper foundation for
experiments, models and samples.
► Even worse, we have failed to confront our
understanding (theories) with data to improve and
correct them.
Twelve-Step Program for
Recovering Naturaholics to Bring
Back Rigor
►
►
►
►
►
►
1. Identify problem and set objectives.
2. Conduct literature review and contact experts.
3. Collect preliminary observations and apply exploratory
data analysis to them.
4. Formulate a theory (conceptual model or research
hypothesis).
5. Formulate multiple predictions from conceptual model
as testable hypotheses or alternate models.
6. Design tests/experiments/models to test the theory.
Twelve-Step Program (cont.)
►
►
►
►
►
►
7. Do a pilot study (techniques, variances, costs).
8. Revise proposal with a statistician based on results of
pilot study and reviews by experts.
9. Do experiment, collect observations and/or build model.
10. Confront theories with data (analyze it).
11. Evaluate, interpret and draw conclusions.
12. Use results and publish them for all to use and build
on.
Scientific Method (Conceptual)
Overview: Formal Sequence
of 18 Steps
Step
Example
1.
Identify the research problem.
1.
What are the influences of
environmental factors, such as
wildfire and winter severity, on
the carrying capacity of elk
winter range?
2.
Conduct literature review of
relevant topics.
2.
Identify broad and basic research
objectives.
Excellent earlier work by
Houston 1982, Merrill & Boyce
1991, …
3.
Determine temporal and spatial
differences in food…
3.
See additional steps and remainder of example in Box 1.1 of book
chapter.
Initial Steps
► Problem
Identification
 Applied research
 Basic research
► Literature
Review
 Professional Society Publications:
►TWS, AFS, ESA, SCB, SAF, SRM, others
 Google Scholar, Web of Science, etc.
 Use Google, Yahoo and other general search engines
cautiously and critically
Initial Steps: Defining Populations
(Biological, Political, and Research)
► Delimiting
populations
 Defines potential applications (inference
space)
 Required for sampling
 Equally important for experiments (esp. field
experiments) and models
Hierarchically Structured
Biological Units:
►
Individual
► Deme
► Population
►Metapopulation
►Subspecies
►Species
Deme
► “A
group of individuals where breeding is random”
(Emlen 1984).
► “A panmictic population” (Ehrlich and Holm 1963)
► Identification
of demes and other groupings of
individuals should be based on demography, movement,
genetics and geography of individuals and habitats.
Deme - Definition
► The
smallest grouping of individuals
approximating random breeding, within
constraints of breeding system, where it is
reasonable to estimate birth, death, immigration
and emigration rates.
Deme
► Genetics:
Random breeding within constraints of social
system.
► Demography: Smallest grouping where its feasible to
estimate birth, death, immigration and emigration rates.
► Movement: Restricted to home ranges in key seasons.
► Geography: Continuous distribution of individuals
within one patch of habitat or a closely spaced set of
habitat patches.
Population
►A
collection of demes or individuals at one point
in time, typically the breeding season, with
strong connections demographically (very high
correlations in vital rates), geographically (close
proximity), genetically (Manel et al. 2005) and
through frequent dispersal.
Population
► Movement: High
rates of dispersal between
adjacent demes.
► Genetics: Very closely related genetically.
► Geography: A collection of closely-spaced
occupied patches of habitat without great
expanses of non-habitat intervening between
them.
► Demography: Very high correlations between
demes and small groups of individuals.
Metapopulation
►
►
►
A collection of populations sufficiently close together that
dispersing individuals from source populations occasionally
colonize empty habitat resulting from local population extinction
(Levins 1969).
Populations within a single metapopulation may show low or
high correlations in demographic rates, but the low rates of
dispersal are still high enough to maintain substantial genetic
similarity.
Numerous types of metapopulations have been described from
source-sink to non-equilibrium to Levins’ classic (Harrison and
Taylor 1997).
Metapopulation
► Movement:
Probability of dispersal between
populations is low but colonization occurs.
► Demography: Possible low correlations in rates
produces high independence whereas demographics of
“source” populations may drive demography of “sink”
populations.
► Genetics: Genetic differentiation occurs between
populations but not enough to become subspecies.
► Geography: Substantial areas of non-habitat may
separate populations and patches of suitable but
unoccupied habitat.
Biological, Political, and Research Populations
Statistical/Research Population = Sampling
Universe = Sampling Frame
► Entire
set of units (i.e., elements, potential
observations) that exist and from which we want to
make inference.
► Even
though we want to make statements (i.e.,
estimates) for animal or plant populations we often
recast this population spatially:
 i.e., statistical population = all the spatial units
occupied by our biological population
Statistical Population
► GIS
a key tool for handling sampling frame
because we often lay it out spatially.
► We use sampling frame to define statistical
population to which we apply our estimates
► We also draw our samples randomly from the
sampling frame.
► But note that randomly doesn’t mean stupidly –
we can draw them cleverly while we draw them
randomly.
Sampling or Experimental Units
► Non-overlapping collections of
elements of
statistical population that cover entire statistical
population.
► Note that these sampling units may be
individuals, plots, spatial polygons, or
collections of elements.
Initial Steps
► Define
Population(s)
► Preliminary Data Collection
 Get our feet wet and try out ideas
► Exploratory Data
Analysis
► Search literature for others’ ideas
► Put on thinking cap
 Theory, Models, Predictions and Hypotheses
Conceptual Model of Waterfowl
Population Dynamics
Components of Theory
(after
Pickett et al. 2007)
► Domain:
Scope in space, time and phenomena
addressed by a theory.
► Assumptions: Conditions needed to build
theory.
► Concepts: Labeled regularities in phenomena.
► Definitions: Conventions and descriptions
necessary for the theory to work with clarity
Components of Theory
► Confirmed generalizations:
Condensations and
abstractions from a body of facts that have been
tested or systematically observed.
► Laws or principles: Conditional statements of
relationship or causation, statements of identity,
or statements of process that hold within a
domain.
Components of Theory
► Models:
Conceptual constructs that represent or
simplify the structure and interactions in the
material world.
 Scientific models can project consequences of
ideas
 Statistical models draw inferences and
discriminate between competing ideas based on
limited observations
Components of Theory
► Framework: Nested
causal or logical structure of
a theory.
► Translation: Procedures and concepts needed to
move from the abstractions of a theory to the
specifics of applications or test or vice versa.
► Hypotheses: Testable statements derived from or
representing various components of theory.
Study Designs for Investigation Process
Experiments
Strengths and weaknesses of different types of experiments (modified from Diamond 1986).
__________________________________________________________
Experiment Type
__________________________________________________________
Laboratory
Field
Natural
_____________________________
Control of independent variablesa
Highest
Medium
Low
Ease of inference
High
Medium
Low
Potential scale (time and space)
Lowest
Medium
Highest
Scope (range of manipulations)
Lowest
Medium
High
Realism
Low
High
Highest
Generality
Low
Medium
High
__________________________________________________________
aActive regulation and/or site matching.
See Table 1.1 in book chapter.
Experimental Design
► Focus
of statistics programs, especially in
agricultural regions
► Dependent experimental units
► Fixed, random, mixed, and nested effects
► Controls
► Replication
► Determining required sample size
Checklist for Experimental Design
► 1.
What is hypothesis to be tested?
► 2. What is response or dependent variable(s) and
how should it be measured?
► 3. What is independent or treatment variable(s)
and what levels of variable(s) should we test?
► 4. To which population do we want to make
inferences?
Checklist for Experimental Design
► 5.
What will be our experimental unit?
► 6. Which experimental design is best?
► 7. How large should the sample size be?
► 8. Have you consulted a statistician and received
peer review on your design?
Modeling
► Powerful




alternative for:
solutions to pressing problems
selecting best of alternatives
effects from multiple simultaneous causes
evaluating population viability
► Goal:
build simplest scientific model with ability
to produce useable predictions
► Statistical models used for estimation,
hypothesis testing, and comparison
Scientific Modeling Strategies
Description
Simple
Complex
Quantification
Conceptual (verbal)------------Quantitative
Theoretical
General-----------------Complex Simulation
Relationships
Linear------------------------------Non-linear
Variability
Deterministic----------------------Stochastic
Time Scale
Time-specific------------------------Dynamic
Mathematical
Formulation
Difference ------------------------Differential
Equations
Equations
Factors
Single------------------------------Multifactor
Spatial
Single Site---------------------------Multi-site
No. of Species
Single Species------------------Multi-species
Statistical Modeling Strategies
Description
Sampling
Simple
Complex
Simple random----------Stratified, clustered, or
multi-stage
Hypothesis testing Fixed or -----------------Mixed fixed and random
random effects
effects
Independence of Complete ----------------Dependence between
observations
independence
observations in space,
time, or both
Errors
Single term---------------Separate process and
observation errors
Steps to Build a Scientific Model
► Problem
definition
► System identification
► Select model type
► Mathematical formulation
► Parameter estimation
► Model validation
► Model experimentation
Basic Concepts of Survey Sampling
►Survey
sampling is a magic sword;
 but it’s a two-edged sword:
►Sharp
edge slices through all kinds of problems producing
unbiased estimates even if some assumptions about
population are wrong
►Dull
edge requires applying some type of random
sampling
Designing a Survey
► 1.
What is objective of survey?
► 2. What is best technique?
► 3. To which population will we make inferences?
► 4. What will we sample (sample unit)?
► 5. What is the size of population we are sampling
(N)?
► 6. Which sample design is best?
► 7. How large should sample be?
Sampling Designs and Types of
Sampling Units or Experimental Units
Key Terms and Concepts
►Precision: deviation of repeated samples from
each other.
►Bias: deviation of repeated samples from true
value.
►Accuracy: deviation of an individual sample
from true value.
Bias, Precision, and Accuracy
Confronting Theories with Data
► Hypothesis testing
(classic frequentist)
► Effect size and interval estimation
► Information theoretic model selection
► Regression and General Linear Models (GLM Fisherian)
► Bayesian Approaches
► Validating parametric and simulation models
Confronting Theories with Data
Finishing
► Speculation
and new hypotheses
► Publication will
 Correct errors
 Insure sound
 Facilitate
controversial practices
conclusions
defending
 Help personnel grow in skills
 Make a permanent contribution to wildlife
knowledge
Common Problems to Avoid
►Procedural
inconsistency
►Non-uniform treatments
►Pseudoreplication
►Insufficient sample size
Adaptive Management: Connecting
Research and Management
► Specify
management goals formally
► Use predictive models to:
 Summarize existing knowledge of factors and processes
producing desired responses
 Forecast results of alternative management actions
 Select best management action to implement, treating it as an
experiment to evaluate
► Monitor
what happens to:
 Ensure goals are met
 Correct relationships and processes assumed in management
Adaptive Management: Connecting
Research and Management
► Alternatives:
 Structured Decision Making
 Scenario Planning
 Various forms of business decision-making
Summary
►
Carefully designed wildlife research will improve reliability of
knowledge base of wildlife management.
►
Research biologists must rigorously apply scientific method and
make use of powerful techniques in survey sampling, experimental
design and information theory.
►
We must move from observational studies to experimental studies,
replicated across space and time, that provide a more reliable basis
for interpretation and conclusions.
►
Modeling is an effective tool to predict consequences of
management choices, especially when based on carefully designed
field studies, long-term monitoring, and management experiments
designed to increase understanding.
Summary
►
Wildlife biologists have tremendous responsibility
associated with management of animal species
experiencing:



►
increasing environmental-degradation problems
loss of habitat
declining populations
We must face these problems armed with knowledge
from quality scientific investigations.