FEATURE:
FISHERIES
RESEARCH
Research,Monitoring, and Evaluationof Fish and Wildlife
RestorationProjectsin the ColumbiaRiver Basin:
Lessons
LearnedandSuggestions
for Large-Scale
MonitoringPrograms
IndependentScientificAdvisoryBoard(ISAB) to the NorthwestPowerandConservation
Council,
Portland,Oregon,NOAA Fisheries,Seattle,Washington,
ColumbiaRiverBasinIndianTribes
Downloaded by [Oregon State University] at 16:49 31 October 2011
IndependentScientificReviewPanel(ISRP) to the NorthwestPowerandConservation
Council,
Portland,Oregon
WesternEcoSystems
Technology,
Inc.,Cheyenne,
Wyoming.
LymanL. McDonald, McDonaldisseniorbiometrician,
He can be reached at Lmcdonald@west-inc.com.
RobertBilby,
Bilbyisan ecologist
at Weyerhaeuser
Company,Tacoma,Washington.
Peter A. Bisson,
CharlesC. Coutant,
JohnM. Epifanio,
Daniel Goodman,
SusanHanna,
NancyHundy,
Erik Merrill,
Brian Riddell,
Bisson
isseniorscientist
at the Forestry
Sciences
Laboratory
of theU.S. ForestService,Olympia,
Washington.
Coutantisdistinguished
research
ecologist,
Oak RidgeNationalLaboratory,
Oak Ridge,Tennessee.
Epifanioisdirectorandassociate
professional
scientist
forthe Divisionof Ecology
andConservation
Scienceat the IllinoisNaturalHistorySurvey,Champaign.
Goodmanisprofessor
of ecology,
MontanaStateUniversity,Bozeman.
Hannaisprofessor
of marineeconomics,
Departmentof AgriculturalandResource
Economics,
OregonStateUniversity,Corvallis.
Huntly isprofessor
of ecology,
IdahoStateUniversity,Pocatello.
Lissisprofessor
of fisheries,
OregonStateUniversity,Corvallis.
Loudenslager
isprofessor
of fisheries
biologyandhatcherymanager,HumboldtStateUniversity,
Arcata, California.
Merrillisthecoordinator
forthe Independent
ScientificReviewPanelandthe Independent
Scientific
Advisory
Board,
Northwest
Power
andConservation
Council,Portland,
Oregon.
William Liss,
Philippisprincipalscientist,Divisionof for EcologyandConservation
Scienceat the Illinois
EricJ. Loudenslager, NaturalHistorySurvey,Champaign.
Riddellisa fisheries
ecologist,
PacificBiological
Station,Departmentof Fisheries
andOceans
DavidP. Philipp,
William Smoker,
Canada,Nanaimo, BritishColumbia.
Smokerisprofessor
of fisheries,
Universityof AlaskaFairbanks,
JuneauCenterfor Fisheries
and
OceanSciences,
Juneau,Alaska.
RichardR. Whitney, Whitneyisretiredfromthe U.S. FishandWildlifeServiceandthe Schoolof Fisheries,
University
of
Washington,
Seattle,
Washington
and
Williamsisa fisheries
biologist,
CenterforSalmonids
andFreshwater
Species
at Risk,Universityof
Richard N. Williams
Idaho,Hagerman.
%l'.B/ISl•Pteam
- be
•:
'
, '}
v
ABSTRACT: The year2006 markedtwo milestonesin the
Columbia River Basin and the PacificNorthwest region's
effortsto rebuildits oncegreatsalmonand steelheadruns--
Charles
Coutant,
•ayW•jte,
and
Nancy•?untly.
•
the 25thanniversary
of the creationof the NorthwestPower
and Conservation
Counciland the l0th anniversary
of an
amendment to the Northwest
I
582
Power Act
that formalized
scientificpeerreviewof thecouncil's
FishandWildlifeProgram
and its varied individualprojects.The authorsof this article
servedaspeerreviewersin the lastdecade.Restorationefforts
in the ColumbiaRiverconstitutea massive
long-termattempt
at fisheries
andecosystem
restoration.In thisarticleweexamine
someof the lessonswe learnedin reviewingthe research,
monitoring,andevaluationeffortsof projectsand their effects
on advancingknowledge(i.e., adaptivemanagement)in the
ColumbiaRiver BasinFishand Wildlife Program,one of the
mostambitious
andexpensive
long-termecological
restoration
programs
in the United States.
Fisheries ß vot 32 NO 12 ß DECEMBER
2007
ß WWW.FISHERIES.ORG
Downloaded by [Oregon State University] at 16:49 31 October 2011
Investigaci6n,Monitoreo y Evaluaci6nde Proyectosde
Pescay Vida Silvestreen la Cuencadel
Columbia:
leccionesaprendidas
y sugerencias
paraprogramas
RESUMEN. E1afio2006 representa
en dossentidosunafechacrfticaparala cuencadel Rfo Columbiay paralosesfuerzos
de recuperaci6n
del salm6ny la truchaarcofrisen la regi6nPacfficoNoroeste:el 25 aniversariode la creaci6ndel Consejo
parala Conservaci6n
y PoderdelNoroestey el 10øaniversario
dela enmiendaal Acto dePoderdelNoroeste,queformaliza
el arbitrajecientfficodel Programade Pescay Vida Silvestrede dicho consejo,asfcomode susproyectosindividuales.
Durantela tiltima d6cada,los autoresdel presentetrabajofungieroncomoirbitros de estosproyectos.Los esfuerzos
de
recuperaci6n
en el Rfo Columbiaconstituyen
una iniciativamuyimportanteen cuantoa la rehabilitaci6nde pesquerfas
y ecosistemas.
En esteartfculoseexaminanalgunaslecciones
aprendidas
duranteel proceso
de revisi6nde investigaci6n,
monitoreoy evaluaci6nde proyectosy su repercusi6nsobreel avancedel conocimiento(manejo adaptativo)en el
Programa
de Pescay Vida Silvestrede la Cuencadel Rfo Columbia,unode losprogramas
de recuperaci6n
milsambiciosos
y de milslargoplazoen losEstados
Unidosde Norteam6rica.
INTRODUCTION
ery programs,which serve other and
sometimes
competinggoals,still constiConcern about depressedsalmon tute a substantial
fractionof the budget.
and steelheadpopulationsand plans BPA,whichfundsthe council'sprojects
for construction of additional dams throughrevenuefrom the region'selecand hydroelectric power plants in tricity consumers,
hasspentabout$3.7
The council'soriginalgoalwasto double the run size to 5 million adult fish
through mitigation actions associated
with its Fish and Wildlife Program.In
2005 and 2006, 25 yearsinto that effort,
runs averagedabout 1.1 million adults
the Columbia River Basin in the late
billion on fish and wildlife restoration
to BonnevilleDam with mostreturning
1970sled to passage
of the 1980Pacific from 1981 to 2006 (NPCC 2006), with fish (-75%) being of hatcheryorigin.
Northwest Electric Power Planning an averageannualbudgetof $130 mil- Returnsof naturally spawningfish are
and Conservation Act by Congress lion over the last 5 years.BPA suggests currentlyabout 1-2% of historicalnum(Northwest Power Act) and forma- that the recoveryprogramcostsshould bers(Williams 2006).
tton of the NorthwestPowerPlanning also include an additional $4.1 billion
Council in 1981 (renamed Northwest for power purchases
and foregonerev- PEER REVIEW COMMITTEES
MANAGEMENT
Power and Conservation Council in
enuesthat it attributesto river opera- AND ADAPTIVE
2004). The act chargesthe councilwith tionsrequiredto assistfishpassage
and
A 1996 amendment to the Northwest
developinga Fishand Wildlife Program improvefishsurvival(e.g.,spillingwater
that balancesproviding the Pacific at damsinsteadof passing
waterthrough PowerAct createda formalpeer review
process for projects seeking funding
Northwestregion with a reliable and turbines).
efficienthydroelectric
energy-producing Despite these expenditures, the under the council's Fish and Wildlife
systemand protecting, mitigating, and abundance of wild salmon and steelhead Program.The amendmentdirectsthe
enhancingfishandwildlife (Lee 1993). populationshas not recoveredto target council to appoint an Independent
The act requiresthe federalBonneville levels. Annual returns of anadromous Scientific Review Panel (ISRP), and
Power Administration (BPA) to fund salmon and steelhead to the Columbia
directs the ISRP to review proposed
fishandwildlifeprojectsconsistent
with River Basinrangedfrom about7 to 15 projects,usingthe followingfivecriteria.
the council's
fishandwildlifeprogram. million fish in the mid-nineteenth cen- Projectsare to: (1) be basedon sound
Since 1981, hundreds of restoration tury (NPPC 1986; Chapman 1986). In science,(2) benefitfishand wildlife, (3)
projectshave been fundedto imple- 1980, when the Northwest Power Act havea clearlydefinedobjectiveandoutment the council's Fish and Wildlife
was passed,total runs averagedfewer come (4) includeprovisionsfor momtoringandevaluationof results,and (5)
Program.Since the early 1990s, resto- than 1 million adults at Bonneville
ratton has been affected increasingly Dam, the firstcensuspoint in the lower be consistent with the council's Fish and
by the recovery needs of Endangered river. Total Columbia River runs, catch Wildlife Program.Basedin part on the
Species
Act-listedsalmonandsteelhead plus escapement,were estimatedat 2.5 ISRP's technical reviews, the counctl
a suiteof projectsto BPA
stocks(Williams 2006), althoughhatch- million adultfish(average1976-1981). recommends
Fisheries ß VOL32 NO 12 ß o•c•R
2007
• WWW.
F[SHERIœS.OR•
Downloaded by [Oregon State University] at 16:49 31 October 2011
for funding.The fourth criterionrequiring monitoringandevaluationhasbeen
particularlydifficult to satisfyby some
agencies
andscientists,
because
of a lack
of understanding
of whereand how individual projectsfit into the assessment
of
effectsof multiple managementactions
in largeareas.Also, provisionsfor monitoringandevaluationhavebeendifficult
to maintain through the funding process,becausethere is constantpressure
to implementadditionalprojects.
A companion committee for peer
review and adaptivemanagementis the
IndependentScientificAdvisoryBoard
(ISAB), an advisoryboardappointedby
tists,andfishandwildlifemanagers
about
what constitutesappropriateresearch,
monitoring,andevaluationat bothproject and programmatic
levels,leadingto
improvedprojectsand monitoringprogram designs.We think that the issues,
our experiences,and our recommendations have broadenoughapplicationto
meritsharingwith a largeraudience.The
designand analysisof monitoring and
evaluationprojectsare problemsamong
fish and wildlife entities for a variety of
reasons(e.g., lackof knowledgeof statistical theory,practicallimitationsof real
world ecologicalmonitoring, and constant pressureto implement additional
NOAA Fisheries, the Northwest Power management actions), in spite of the
and Conservation Council, and the fact that informationgainedis essential
Columbia River Indian Tribes. Unlike
for policy makersintending to develop
the ISRP,which reviewsindividualproj- an optimum mix of managementactions
ects, the ISAB providesscientificpeer to achievetheir goals(adaptivemanagereview of major and often controversial ment; Magnusonet al. 1996).
programmaticissuesin fishand wildlife
restoration in the Columbia River Basin.
RESEARCH,
The ISAB is chargedwith making recommendationsfor research,monitoring,
and adaptivemanagement.
The authors
AND
of this article have served on one or both
committees.
Over the 10-yearhistoryof the basinwide peer review process,committee
recommendations have resulted in a sus-
tained dialogbetweenreviewers,scien-
MONITORING
EVALUATION
ADAPTIVE
FOR
tots to know if expectedoutcomesare
being realized, and this information
can be usedto justifyfurther funding.If
desiredoutcomesare not beingrealized,
mid-course
corrections can be made
to increase the likelihood of achiev-
ing desiredresults.We recognizethe
complexitiesinherent in researchand
monitoring,especiallygiven limitations
on project scope,finances,and personnel. Moreover,it is not easyto condense
advice into a simple set of recommendations that apply to all conceivable
researchand monitoring situationsin
large ecosystems.
Different issues(e.g.,
understandingthe effectiveness
of habitat improvement or measuring increases
in the survivalrates of hatchery fish)
requiredifferentmonitoringapproaches.
Taking this into account,we describe
commonresearchand monitoringneeds
for projects related to environmental
restoration in the Columbia River Basin
and summarizeour findingsin the form
of lessons learned
about data collec-
MANAGEMENT
tion, data analysis,and the evaluation
of projecteffectiveness.
We believethat
Research, monitoring, and evalua- manyof theseissuesare sharedby other
tion arecentralto adaptivemanagement largeecosystemand natural resourceresand are the meansby which the science- toration programsand that the lessons
policy interfaceis informed(Magnuson learned in the Columbia River Basin
et al. 1996). Monitoring of individual will have broad applicationto fisheries
restoration projects allows investiga- and wildlife managementelsewhere.
Adultweir for spring-summer
Chinookon JohnsonCreek,Idaho.
$114
Fisheries e VOL 32 "O 12 * DECEMBER
2007 o WWW.FISHERIE$,OR6
Downloaded by [Oregon State University] at 16:49 31 October 2011
1. Research, monitoring,
and evaluation terminology
is used inconsistently.
actionsare not allocatedto studyunits we proposethat it be usedin placeof the
by some randomizationprocedure.In word "treatment" in observational studcontrast, the words "manipulative," ies of largeecosystems.
For example,a
"true,"
and "randomized-treatment stream might be selectedto receive a
We observed inconsistent use of
experiment"have been usedto identify certain aquatic habitat improvement
research, monitoring, and evaluation scientific studies where treatments are action, and production of naturally
(RM&E} terminologyby practitionersof allocatedbysomerandomized
procedure. spawnedfishmight be comparedbefore
variousscientificdisciplines(e.g., fisher- We determined that to avoid confusion
andafterthe actionis implemented.It is
ies, wildlife, hydrology,genetics,and it would be well to classifyresearchand usefulto refer to "managementaction"
statistics), which has led to confusion in
monitoringstudiesinto ( 1) observational to distinguishthe assessment
effort from
designand analysisof studies.We have studieswhich may include evaluation one in which habitat improvement
attemptedto maintainconsistency
with of effectsof "managementactions"and "treatments" and "controls" are ranMacDonaldet al. (1991) wheretheyuse (2) randomized-treatmentexperiments domlyassigned
to candidatestreams.
Professional statisticians often disthe phrases"implementationmonitor- where "treatments"are appliedto study
ing" to evaluate if individual manage- unitsaccordingto somerandomizedpro- tinguish betweenobservationalstudies
ment actions are installed correctly; cedure.The rolesof thesetwocoretypes with and without active applicationof
"effectiveness
monitoring"to determine of inference-supporting
studiesin assess- managementactions,calling the former
if the individualinstallationsperformas ment of large-scaleenvironmentaland "quasi-experiments."Studies of areas
expected;"projectmonitoring"to assess ecologicalprograms
are not alwayswell without active applicationof manageif the entire suiteof individualmanage- understood.
ment actions are "pure" observational
ment actions installed in a particular
studies.We do not make that distension,
project perform as planned; and "vali- Observational Studies
becausebasicdesignof samplingefforts
and statistical estimation methods do
dation monitoring"to determineif the
Observational
studies are those in
anticipatedwatershedresponseactually
not varysignificantlybetweenthe two.
occurs.
which the researchpersonlacksthe full
The primaryinconsistencies
in termi- control over the schedulingof experi- Randomized-TreatmentExperiments
nologyexistwith the methodology
used mental stimuli, i.e., impositionof manfor the assessment
effort. For example, agementactions.We found inconsistent
Randomized-treatmentexperiments
the words "observational," "mensura- use of the term "treatment" in the conrequire that treatments be randomly
tive," "quasi-experimental,"or simply texts of randomized-treatmentexperi- assignedto experimentalunits (Fisher
1935). One or more of the treatments
"monitoring"havebeenusedto identify ments and observational studies. The
scientificstudieswith no management term "managementaction" is in wide- may be designatedas a control (or refactionsor studieswhere management spreaduseby managementagencies,and erence). These experimentsgenerate
!
!
•N
Steelheadtrap on the ImnahaRiver,Oregon.
Fisheries ß VOL32 NO 12 ß DECEMBER
2007 ß WWW.FISHERIES.ORG
585
conclusions of causal relauons between
treatmentsand effectsthat are "design
based"(i.e., basedon explicit designof
the experimentto isolatea cause-effect
relationshipbetween a treatment and
a response)and are easiestto justify;
however, the mechanisms
producing a
conclusivelydemonstratedeffect may
remain conjectural.
Downloaded by [Oregon State University] at 16:49 31 October 2011
2. Implementation monitoringis often
the only monitoringpursued, but
this constrainsadaptivelearning.
Implementationmonitoringinvolves
assessing
whether or not the physical
objectivesof a restorationproject (e.g.,
addinglargewoodto a stream,fencing
a riparianzoneto excludelivestock,or
removing a barrier to fish migration)
(GIS) for trackinglong term changesin
riparianand other terrestrialhabitat in
sub-basins or watersheds.
A censusis often appropriateto documentdirectpossibleeffectsof management actions.For example,a censusin
a projectto supplementa weakstockof
naturally spawningfish with hatchery
fish might includecompletecountsof
hatcheryand naturallyproducedadults
passinga weir to the spawninggrounds
on small tributary streams.Another
example might involve counting the
numberof poolsin a streambeforeand
after habitat restoration. Censuses need
vational
studies. Because of the extra
assumptions
or modelsrequired,such
methodshave their limitations, but can
be usefulwhen assumptions
are judged
to be reasonably
well satisfied.
5. Effectivenessmonitoring of
individual managementactionscan
bestsupportbroad.scaleproject
monitoringby using commonmethods.
not be expensiveor time consuming.
4. Probabilisticsamplingis often
effectivein tracking status and
trendsfor parametersfor which
were achieved(MacDonald et al. 1991). census data are not available.
Often there is no mention of tracking
Data collectedon a probabilisticsamthe ecologicaleffectsof such proposed
managementactions to determine if ple of units within areasoften provides
in studiesrequiring
the installationperformedas expected effectiveassessment
(effectiveness
monitoringas definedby on-the-groundfield data collection in
MacDonaldet al. 1991). Implementation relativelysmallstudyareas,and infermonitoring tracks progressof projects, encescan be appliedto the total areaor
such as in terms of miles of stream fenced
populationsampled,not just the units
off, but adaptive managementrequires on which data werecollected.Sampling
that project resultsalso be measured methodsare often labor intensive, and
•n terms of benefits to fish and wildlife
it may not be economicallyfeasibleto
andtheir habitat.All projectsshouldbe collectdata on all unitsin a studyarea.
subject to effectivenessmonitoring to A goodillustrationof the useof probabiverifytheir physicalandecological
ben- listic samplingin monitoringof salmon
efits(e.g.,changesin vegetation,flow,or statusand trend is the Oregon Plan for
temperature)or shouldbe includedin Salmon and Watersheds Monitoring
an overarching integrated monitoring Program, as implementedin Oregon
kisutch)
programto evaluateoverall benefitsof for coho salmon(Oncorhynchus
in
coastal
streams
(Rodgers
2000).
The
multiple managementactions(project
Oregon
Plan
applied
a
rigorous
sampling
monitoringas definedby MacDonald et
al 1991).
design for probabilisticsite selection
to answerkey monitoringquestionsfor
estimatingcohosalmondistributionand
3. Completecensusby remotemeans
abundance.
is often effectivein assessingstatus
When probabilistic sampling is
and trends over large areas.
applied to observationalstudies(i.e.,
Complete census(data collectedon treatmentsare not randomlyassigned
all units--e.g., individual organisms, to sites), non-subjective conclusions
habitats, or sites--within areaswith no are restricted to changes,differences,
randomizationat any level), often pro- and trendsfor the sampledstudyareas
vldeseffectivelong-termprojectmoni- and time periods.Statistical inferences
toringof largeareas.Where thesetypes concerning cause and effect relationof data are available, a censuscan be a shipsthat gobeyondthe observed
"real"
very effective tool for assessing
status differencesamongstudyareasand time
and trends.In somecases,aerial pho- periodsare often made under assumptographyor other remote sensingtech- tionsthat aredifficultto justify.Putative
nologycanbe usedto createcensusdata causal relations are best stated as tentalayersin geographicinformationsystems tive workinghypotheses
warrantingfur586
ther study.There •sa largeand growing
literatureconcerning
the useof Bayesian
and other model-based
tools (e.g., geostatisticalmethods)for analysisof obser-
Effectiveness
monitoringmosteasily
provides data for monitoring of habitat at a largerscaleif similar site selection
and data collection
methods are
used. The more site selection and data
collection methodsdiffer amongindividual managementactions, the more
difficultit becomesto aggregate
data to
make inferencesabout larger regions
We persistently
have urgeduseof common methods,and progresshas been
maderecentlyby the PacificNorthwest
Aquatic Monitoring Partnership on
developinga set of commonmethods
that can be appliedto a varietyof morntoting needs.
6. Intensiveresearchprogramsare
required to answer the question
"What was the causeof the change
detectedin the monitoringprogram?"
Properlydesignedcensusor probabilistic sampling programscan provide relatively low cost, repeatable
data collection with enough accuracy
and precisionto detect changein the
face of backgroundnoise. Thus project or validation monitoring programs
(MacDonald et al. 1991) are success-
ful if they detect real overall effectsof
managementactions;however,conclusivescientificexplanationfor the effects
may b• elusive. Determination of the
causesof the effectsdetectedby project
and validationmonitoringrequiresthe
developmentof testablehypotheses
and
implementationof appropriateexperiments in more intensive research.
Fisheries ß VOL32 No 12 ß DECEMBER
2007 ß WWVV.FISHERIES.ORG
7. Randomized-treatment experimentsare
difficult to conduct on large ecosystems.
Randomized-treatment
field experimentson specieswith
complicatedlife historiesmaybe very difficult.Many species
of interest in the Columbia Basin occupylarge areasduring
their life history,requiringthe experimentalunits to be very
large.In addition,responses
to a treatmentmaytakea considerabletime to becomeapparent.For example,severalsalmon
generations
mightbe requiredto registeran effectfroma managementaction.Over this extendedperiodof time it maybe
difficult or impossibleto ensurethat environmentalfactors,
suchas streamflow,affect all studysitesapproximatelyto the
samedegree.In addition, unwantedanthropogenicimpacts
to all studysitesmust be minimizedto avoid confounding
Downloaded by [Oregon State University] at 16:49 31 October 2011
results.
A revealingexampleof the difficultiesin implementing
large scale effectivenessmonitoring in the Columbia River
Basinwith a randomized-treatment
experimentwasthe Idaho
SupplementationStudy on Chinook salmon(O. tshawytscha;
Lutch et al. 2003). During implementationof this study,randomizedassignments
of treatmentsto streamswereabandoned
(e.g.,somecontrolsitesbecametreatmentsitesin the middleof
the experiment),and the studyconvertedto an observational
one.A rigorous
randomized-treatment
studywill be difficultto
maintain when dealingwith rare or threatenedpopulations,
becausethere is an ethical dilemmabetweenletting a resource
be injuredor disappearand gaininginformationthat would
lead to bettermanagement.
8. Observationalstudiesthat corroborateeach other provide
evidencefor the actual effectsof managementactions.
Monitoring multiple, independent,but similar observational studies,and analyzingthem as if they were replicates
subjectto study-specific
randomeffects,mayprovideconvincing evidencefor the effectof an actionif resultsare consistent
across
the studies.In an inductivesense,monitoringnumerous
observational
studiescan suggest
causalrelations(e.g.,Shipley
2000). Good samplingdesignis required,and conclusions
about restoration effectivenesswill require some scientific
judgment becausemany uncontrolledfactorscan influence
the results.Ideally,further analysis,and insightfuldesignsfor
further measurements,
may allow replacementof someof the
Introducing
the VR2W Single
Channel
Now with Bluetooth
wireless
technology for significantly
faster data upload
The VR2W was designedusingthe same provenand
reliable technologyyou've come to trust in the VR2.
And like the VR2, the VR2W is affordable,compact,
easy to use, long-lastingand flexible, making it ideal
for madne researchprojects.Nowwiththe newVP,2W,
VEMCO
•
has taken the VR2 and made it even better!
Significantly faster
upload speed with
Bluetooth©
wireless
technology - after retrieving your VR2Ws
from the water, upload
your data 20 times
faster than the VR2 and
from up to 7 receivers
simultaneously
•
Increaseddata storage
capability - 8 MBytes
(1-milliondetections),4
times that of the VR2
•
Field upgradabledesign allowsthe VR2W to be
upgradedin the field
•
Safe, robustdata storage capability- the VR2W
retains all detections in non-volatilememory so
data is saved even if the unit unexpectedlyfails
•
Fullycompatiblewith all existingVR2 receivers
The VR2W also uses enhanced PC Software. The new
VEMCO User Environment (VUE) PC Software for
initialization,configurationand data
•
random effects with covariates.
r
9. Simple evaluation methodsand periodic re-evaluations
are needed,becauseevaluation methodologies
will improve.
Receiver
uploadfromVEMCOreceiversai-
•J • -- lows
users
tocombine
data
from
I
•'...............
multiple
receivers
ofvarying
types
into a singleintegrateddatabase.
VEMCO (a division
ofAMIRIXSystems
Inc.)
It is important to distinguishevaluationbasedon data
collectedas part of long-termmonitoringprograms(usually
Tel: 902-450-1700 Fax: 902-450-1704
www.vemco.com
data collectedfrom largeareasand over longtime periods)
and evaluation of data collected in more focused randomized-
treatmentor observationalresearchprojects.Many long-term
monitoringstudiesdonot yieldusefulresultsuntil a significant
periodof time haspassed.
Mostresearchprojectsarerelatively
shortterm, often three to five years,are designedto test specific hypotheses,
and requirewell-definedmethodsfor analysisand evaluation.Evaluationis an equallyimportantpart of
Fisheries ß VOL32 rio 12 ß DECEMBER
2007 ß WWW.FISHERIES.ORG
Adivision
of•
587
long-term ecologicalmonitoring, and
there is a need for clear proceduresfor
analysis.Ongoingevaluationof the data
from long-termmonitoringeffortsis also
important, becauseit allows detection
of unusualeventsor changesin time for
themto be subjectto additionalscrutiny
and to provide information for adap-
Downloaded by [Oregon State University] at 16:49 31 October 2011
tive management. Long-term monitor-
ing data, however,are expectedto have
valuefor manyyears(decadesto centuries), and relativelysimpledesignbased
analyses are recommended because
evaluationmethodologies
are likely to
evolve,and, in any case,the time duration of the study and contrastsin the
data will continue to improve. Care
shouldbe taken to assurecomparability
over time asmethodologies
evolve.
10. Use of different site selection
criteria, indicator variables, and data
collectionmethodshampersevaluation
of large-scalerestorationprograms.
Good monitoringdesignisdifficultto
attain when participatinggovernments,
agencies,and other stakeholdershave
different (and sometimesconflicting)
eomor
II!
?
ß
vested interests. Often organizations iesbecausethe actionsare typicallynot
have ongoingresearchand monitoring assignedrandomlywithin each pair of
efforts that use different site selection
sites.An alternativeextensiveapproach
criteria, indicator variables, and data is to samplesiteswith a gradientof concollection methods,and each isreluctant
ditionsof the factorsunderstudy.
to adapt its approaches.Government
Definitive conclusions (e.g., that
agenciesmayhavebeengivenclearlegal road removal reduces sedimentation in
mandatesthey have to meet and legal streams)are not justifiedby extensive
settlementsfrom previous court-cases studydesignsin the absenceof randomhave set explicit standardsfor how and ized treatment assignment.If enough
what kinds of data need to be collected.
sitesare sampled,however,and if the
Although suchdata collectionsmay in results corroborate one another, then
aggregatedevote exceptionalresources there is evidence in an inductive sense
for monitoring,their disparatemethods that a cause-and-effectrelationship is
andmetricsmeanthat the datatheycol- probablybeingmeasured.
lect are often difficult or impossibleto
The intensiveapproachfocuseson a
combineinto meaningfulevaluationsat reducednumberof samplingunits (e.g.,
larger scales.Top-down coordination, watersheds)with a high level of samchangesin legalrequirements,
andadop- pling intensity to identify the factors
tion of commonapproaches
will require responsible
for observedchangesin systime, however, changesare needed to tem condition.The basicpremiseis that
ensure that data from individual monicause-effectrelationships in complex
toring effortscan be combined.
systems
are bestunderstood
by concentrating effortsat a few locationswhere
11. There are two general approo•hes ecosystemprocessescan be studied in
for collectionof empirical data for
enough detail to reveal mechanisms.
evaluation of large scalerestoration
Concentrationof effort is necessaryin
•ctivities: extensive and intensive.
an intensivestudydesignto focussufficient resources
and researchexpertise
The
extensive
to teaseapart the complexinteractions
approach involves governing ecosystem response to restosamplingof a large ration activities. With extensivestudies,
number of study many sitesor pairsof sitesare needed.
units with none,
one, or more than
With
used
tional studies,smaller-scalerandomized-
intensive studies, there
are
repeatedmeasurements
of larger numone
restoration
bersof variableswithin a few, perhaps
action applied to more controlled,studyareas.Although
each.A commonly both approachesconstitute observa-
river pro.
/
extensive
study design calls
for a large number
of pairs of sites in
which
a
restoration
'!
certain
action
(e.g., removal of
roads) is applied
to
one
member
treatmentexperimentscanbe embedded
within either of the two designs.
There are advantages and disadvantagesto each approach(Roni et al.
2003). Extensivedesignsallow reasonable inferencesbasedon the layout of
the study,but the number of activities
or combinations of activities that can be
of a pair and is comparedis limited. Intensive designs
not applied to the limit investigationsto a smallernumother site.Changes ber of siteswith restrictedgeographical
in other manage- coverageandcombinationsof activities,
ment
actions
are
but allow detailedstudyof more variexpected to be ables,processes,
and their relationships.
applied uniformly Disadvantages
of both approachescan
to both
LittleRiVer
members
of a pair during
the study. Most
often,
ß .Reimarch
&Dsin.em.riVer.
c0m
these are
observational stud-
includecosts,limitedabilityto extrapolate resultsto largeregions,and logistical difficultiesof dealingwith relatively
large and long-term monitoring projects. Problems related
to unavoidable
Fisheries © VOL 32 NO 12 ß DECEMBER
2007 ß WWW.FISHERIES.ORG
changesin applicationsof managementactions,however,
shouldbe fewerwith the intensiveapproach,becausefewer
sitesare required.
CONCLUSIONS
We recognizethe difficultiesinherent in assessing
the
response
of ecosystems
to restorationactions,given[imitations
on mandate,finances,expertise,and personnel.Moreover,it
is not easyto providea simpleset of recommendations
that
applyto all conceivableresearchandmonitoringsituationsin
largeecosystems.
Basedon our experiences
in the Columbia
RiverBasin,we offerthesefivestepsfor developinga research,
monitoring,andevaluationplan for a largearea:
Trac' -your
_hwith
e newest, mot
advance=d aco.ustic
tracRing receiver
ava
ble tody.
Downloaded by [Oregon State University] at 16:49 31 October 2011
1. Developan extensivecensusof attributesfor large-scale
habitat trendsbasedon remotesensingand other appropriate methods,with data layersin a GIS.
2. Developcommonsite selectionproceduresand common
datacollectionprotocolsfor on-the-ground
populationand
habitatmonitoring.
3. Obtain overallconsensus
on a designstrategythat employs
legitimatetreatment-controlpairs, employsa broad gradient of treatment levels, or that randomizestreatment
assignmentover enoughsitesto averageout site differences.Implement the design,and stick with it until the
importantquestion(s)are resolvedor until datashowthat
the uncontrolledvariation is sogreatthat the designis not
adequateto resolvethe question.
4. As dataareobtainedon statusandtrendsof populationsand
habitat,developempiricalmodelsfor predictinghabitat or
populationlevelparameters
(e.g.,presence-absence
of focal
species
andselectionof "preferred"
habitat). Potentialpredictorvariablesincludenot only physicalhabitat variables
(e.g.,vegetation,flow,andtemperature),but alsothe likely
biologicaleffectsof managementactionsthat arecurrently
in placeor will be implementedin the future.
5. Employbestprofessional
judgment,basedon availabledata
and with knowledgeof the existenceof significantdata
gaps,asto whetheror not intensivewatershedmonitoring
projectsshouldbe initiated.
Thesefive stepswill help accomplish
successful
large-scale
research,long-term monitoring, and evaluation programs.
Appropriateandrelativelyintensiveresearch
projectscan be
implementedwhen the causesof changesobservedin longterm monitoringprojectsare not obviousand causalmechanismsrequireelucidation.The resultsshouldprovidethe basis
for policy-levelselectionof an optimummix of management
actions,i.e. adaptivemanagement.
VEMCO's
VR100
Acoustic
Tracking Receiver: the ultimate
fish tracking solution.
hether
you
areactively
tracking
large
pelagic
fish
or conductingpresence/absence studies, the
VR100 willget the job done.The VR100 has a flexible
systemsarchitecturewith 8MB of non-volatileinternal
memory,GPS positioning
and precisetiming,USB link
to PC or laptop,and fieldinstallablesoftwareupgrades.
Other features include:
Simultaneous, multi-frequency reception and
detectiontrackingalgorithms
Wide dynamicrange allowingmulti-tagreception
withoutgain adjustment
Splash proofcase with marine grade connectors
Codedand continuoustags
Operationfrequency10-100kHz
VEMCO (a division
ofAMIRIXSystems
Inc.)
Tel: 902-450-1700 Fax: 902-450-1704
ACKNOWLEDGEMENTS
www. vemco.com
We thank the Northwest Power and Conservation Council,
IVi • I1%
NOAA Fisheries,and the Columbia River BasinIndian Tribes
for their support.This manuscriptbenefitedfrom the helpful
commentsof RichardAlldredge,presentmemberof the ISRP
and ISAB; Stuart Hurlbert and Roland Lamberson,present
membersof the ISAB; Chris Jordan;Tracy Hillman; Bruce
Fisheries ß VOL 32 NO 12 ß DECEMBER
2007 ß WVVW.FISHERIES.ORG
•/a
•
Adivision
of•
S89
the
Marcor;David Marmorek;PeterPaquet; MacDonald, L., A. W. Smart, and R. C.
Wissmar. 1991. Monitoring guidelines
to evaluateeffectsfor forestryactivities
and Charlie Paulsen;however,we assume
full responsibilityfor our statementsand
recommendations.
Also, we appreciate
the review commentsprovidedby the
scienceeditor and anonymousreferees
on an earlierdraftof the manuscript.
Downloaded by [Oregon State University] at 16:49 31 October 2011
Basin Fish and
1978-2005.NPCC, Portland,Oregon.
Protection
Rodgers,J. D. 2000. Abundanceof juvenile coho salmon in Oregon coastal
streams,1998 and 1999. Monitoring
Program Report Number OPSWODFW-2000-1,OregonDepartmentof
Agency,Region 10, NPS section,WD139, Seattle,Washington.
Magnuson, J., F. W. Allendorf, R. L.
Beschta, P. A. Bisson, H. L. Carson,
REFERENCES
River
Power and Conservation Council,
on streams in the Pacific Northwest and
Alaska. U.S. Environmental
Columbia
Wildlife Program of the Northwest
D. W. Chapman, S.S. Hanna, A.
Fish and Wildlife, Portland.
R. Kapuscinski, K. N. Lee, D. P.
Chapman,D. W. 1986.SalmonandsteelRoni P., M. Liermann, and A. Steel. 2003.
Lettenmaeir, B. J. McCay, G. M.
head abundance in the Columbia River
Monitoringandevaluatingfishresponse
MacNabb, T. P. Quinn, B. E. Riddell,
in the nineteenth century.Transactions
to instreamrestoration.Pages318-339
and E. E. Werner. 1996. Upstream:
of the American Fisheries Society
in D. R. Montgomery,S. Bolton, D. B.
salmon and society in the Pacific
115:662-670.
Booth, and L. Wall, eds. Restoration
Northwest. National Academy Press,
Fisher, R. A. 1935. The designof experiof Puget Sound rivers. University of
WashingtonD.C.
ments.Oliver and Boyd,Edinburgh.
WashingtonPress,Seattle.
NPPC (Northwest Power Planning
Lee, K. N. 1993. Compassand gyroscope.
Shipley,
B. 2000. Causeand correlationin
Council). 1986. Council staff compiIntegratingscienceand politicsfor the
biology:
a user'sguideto path analysis,
lation of information on salmon and
environment.IslandPress,
Washington,
structuralequations,and causalinfersteelhead losses in the Columbia River
D.C.
ence. Cambridge University Press,
Basin.NPPC, Portland,Oregon.
Lutch, J., C. Beasley,and K. Steinhorst. NPCC
Cambridge.
(Northwest
Power
and
2003. Evaluation
and statistical review
of Idahosupplementationstudies.Idaho
Departmentof Fish and Game Report
Number
03-16.
Boise.
to the river: restoringsalmonto the
annualreportto the Northwestgovernorson expendituresof the Bonneville
Power Administration to implement
'YourTags
World
Williams, R. N. (editor}. 2006. Return
Conservation Council). 2006. Fifth
Leader
Columbia
ß•r in Fish Tags -FcP
e Laminated
• Dart, Fing½
Tags, Dart Tags & M,
ß GunsandT•
"WhyRisk YourResearchTo The Copy-Cats
Elsevier
Academic
Your Way
stacean
Tags
almost any other
River.
Press,Amsterdam.
;tom taggi.
I •olution
•
-7 ~ 50 Years
Tags
ner• Intramuscular
T- -
s, Extra Needles• etc,
.. ,eed.
... When YouCanHave The Original?"
View our latest catalog at www, flo.t/t•.•!.½om, or email us at• sales(•flo.•ta•.com
or call to •tiscuss your custom tagging needs: (800) 843-1172
S90
Fisheries ß VOL 32 NO 12 ß DECEMBER
2007 ß WWW.FISHERIES.ORG