Answering Questions in Management and
Research Using Large-Scale Manipulative
Experiments
Robert J. Cooper
George A. Gale
Leonard A. Brennan
Abstract—An adequate information base regarding management
practices for migratory landbirds and other nongame species does
not yet exist. Land managers, therefore, must act with inadequate
knowledge of the resources they are charged with managing. We
believe that part of the solution to this problem rests with the land
managers themselves, in that they are in a position to gain new
knowledge about this resource by combining research or monitoring
with the management activities they currently use or anticipate
using. By collaborating with researchers, they can modify some of
these manipulations to take the form of well-designed, large-scale
experiments. Ideally, such experiments should include features of
sound experimental design, such as replication, randomization, and
controls. Where such features are compromised, we offer some
suggestions on how to modify designs appropriately. They also
should include estimation of demographic parameters such as
productivity and survival, rather than just assessment of presence/
absence. We present four examples from our own work with silviculture in two forest types, prescribed fire, and insect pest management. In each case, a long-term, large-scale, manipulative experiment was developed and funded through collaborative efforts among
researchers, managers, and multiple partners. Benefits to managers include timely information directly pertinent to their lands.
Benefits to researchers include increased funding opportunities for
basic as well as applied research, and the knowledge that their
research results are being used. Both groups benefit in that they are
able to achieve more together than either could alone.
Land managers (e.g., wildlife managers, foresters, range
managers) typically carry out a variety of management
practices to promote the resources they oversee. When faced
with questions regarding management of their resource,
they often resort to what is to them common sense, based on
a career-long experience with the resource in question.
When faced with questions for which they do not readily
have answers, they often rely upon literature and/or expert
In: Bonney, Rick; Pashley, David N.; Cooper, Robert J.; Niles, Larry,
eds. 2000. Strategies for bird conservation: The Partners in Flight planning process; Proceedings of the 3rd Partners in Flight Workshop; 1995
October 1-5; Cape May, NJ. Proceedings RMRS-P-16. Ogden, UT: U.S.
Department of Agriculture, Forest Service, Rocky Mountain Research
Station.
Robert J. Cooper, Daniel B. Warnell School of Forest Resources, University
of Georgia, Athens, GA 30602 U.S.A. George A. Gale, Department of Biology,
University of Memphis, Memphis, TN 38152 U.S.A. Current address: King
Mongkut’s University of Technology, Thonburi School of Bioresources &
Technology, Division of Natural Resources Management, Bangmod,
Thungkru, Bangkok 10140 Thailand. Leonard A. Brennan, Tall Timbers
Research Station, Route 1, Box 678, Tallahassee, FL 32312 U.S.A.
220
opinion. A current problem is that an adequate information base does not yet exist regarding management practices for migratory landbirds and many other nongame
species. We often simply do not know the effect of a particular management practice on a particular bird species.
Some of the most basic information needed by land managers to effectively manage migratory landbirds involves
the effects of typical management practices on migrants.
Despite recent advances in the knowledge of management of
long-distance migrants, most reviews on the subject have
been necessarily general (Finch and Stangel 1993b; Martin
and Finch 1995). One of the most common complaints we
hear from managers regarding managing their lands for
migratory birds is the lack of such information specific to
their needs.
We maintain that the power to change this situation rests
partially with land managers themselves. Often, important
information can be obtained by monitoring, which can be
done directly by the land manager if adequate resources and
personnel exist, or by collaborating with researchers or
volunteers. For example, monitoring conducted in places
where different management practices have been applied in
the past provides information on effects of those practices. If
practices such as timber harvesting, prescribed burning, or
pesticide applications are about to be performed, then monitoring efforts aimed at assessing the effects of those practices can constitute manipulative experiments. Attention to
attributes of experimental design such as replication, randomization, and controls will help such assessments be more
meaningful, and allow extrapolation of results to other
areas. Input by experienced researchers, presumably knowledgeable in experimental design, is therefore desirable. By
communicating and collaborating with researchers, land
managers can make their areas more attractive as research
sites. From the researcher’s viewpoint, input from managers
is just as desirable to ensure that manipulations are done
properly and that they reflect realistic management practices. Also, both groups often will be needed to combine
scarce resources required to perform the project properly.
The purpose of this paper is to provide suggestions on how
such large-scale manipulative experiments can be designed
and developed, with some examples of practical application,
which come from our own unpublished work, in progress. The
lack of sound, large-scale manipulative experiments in the
literature at the time of the Cape May workshop reflected
the past failure of researchers and land managers to collaborate. We note with optimism that many such experiments are now under way, however, this fact does not diminish
USDA Forest Service Proceedings RMRS-P-16. 2000
the pertinence of this paper. It is targeted equally toward
managers, researchers, and those interested in monitoring,
because without the cooperation and collaboration of all
three groups, making the large advances in the knowledge
needed to effectively manage Neotropical migratory birds
will be difficult.
Experimental Design _____________
If the effect of a particular management practice on
migratory landbirds is to be examined, then the investigators should follow rules of good experimental design in their
investigation. By experimental design we mean “the logical
structure of an experiment” (Fisher 1971). The reader is also
directed to Hurlbert (1984), Johnson (this proceedings), and
Nichols (this proceedings). In this paper we will largely use
the terminology of Hurlbert (1984), who differentiated between a manipulative experiment, in which experimental
units (i.e., those units to which treatments are applied) are
somehow manipulated by the application, and a mensurative
experiment, in which nothing is manipulated, but plots may
be established and measured with the objective of comparing experimental units over space or time. We assume in the
following that some management practice (e.g., timber harvest, prescribed burning, pesticide application, grazing) is to
be (in the case of the manipulative experiment) or has been
(for mensurative experiments) applied over a large area
(e.g., a management compartment). For example, in the case
of a management compartment of several hundred hectares,
usually the whole compartment is scheduled to receive a
particular treatment at a particular time; nothing is random
about the selection of the site or the treatment allocated to
it. However, monitoring that site still can be meaningful if
another control site is monitored, and if a before-and-after,
control and impact (BACI) design is used (Stewart-Oaten
and others 1986). Alternatively, and this is the approach we
have adopted, if plots can be established within a management compartment or other area, and if the land manager is
willing to let various treatments be randomly assigned to the
plots, then the study takes on the characteristics of a welldesigned manipulative experiment. Below, we briefly discuss some considerations involving the design of large-scale
manipulative experiments.
Objective
This may seem obvious, but more than one major investigation has failed because the objectives of the study were not
clearly defined. Objectives will dictate the treatments to be
applied, the timing of the treatments, what should be measured, and how long the study should last.
What should be measured? Point counts are desirable in
that they cover a maximum amount of area with a minimum
amount of effort, so that larger numbers of replicates are
possible. Also, standardized methodologies have been developed (Ralph and others 1995b; Hamel and others 1996).
However, point counts also provide minimal information, and
are affected by a number of extrinsic factors. Methodologies
should be determined by objectives more than by logistical or
financial constraints. We believe that to truly understand
USDA Forest Service Proceedings RMRS-P-16. 2000
the effects of alternative management practices, demographic
data (productivity, survival) are required. Much of this section of the Cape May Proceedings is devoted to this proposition.
Controls
By control we mean a treatment that serves as a meaningful comparison to an actual management practice, such that
the effects of the practice relative to not performing the
action can be assessed. Often, but not always, developing a
control involves taking measurements in an area where no
management action is taken (see example 2 for a control
involving a manipulation). Simply monitoring birds on a
single study area before and after a management practice is
applied does not provide useful information, because other
factors that might influence the birds in question also might
have changed over that time. For this reason we believe that,
of the features of sound experimental design identified by
Hurlbert (1984), controls are the most important.
Randomization
Ideally, treatments (alternative management practices,
controls) should be allocated randomly to experimental
units. If a particular management compartment, for example, is due to be thinned, but no other compartment is due,
then one might divide the compartment designated for
treatment in half, thinning a randomly selected portion of
the compartment, but leaving the other half as a control.
Such a procedure is not always possible, because sometimes
a particular treatment must be applied to a particular area.
However, if randomization is not used, then the areas
studied are not representative of the population from which
they were selected, and meaningful extrapolation of results
to other areas is not valid. Also, an effort should be made to
intersperse treatments, to avoid an unlucky draw in which
all replicates of a particular treatment are lumped together
in space (Hurlbert 1984).
Replication
If treatments are applied to one experimental unit each,
then any differences found may result not from the treatment, but could be attributable to preexisting differences on
those sites. In other words, the effect of treatment is confounded with location, and inferences applied to the specific
site where the study is being performed are limited.
Because of expense and other considerations, only one set
of treatments may be possible at a particular site (i.e., no
replication at all). Is a study that compares a number of
treatments with no replication worthwhile? While certainly
not as desirable as a replicated study, we believe the answer
is yes. First, an unreplicated study still can provide the land
manager with some insight concerning the potential effects
of management practices on birds. Results of unreplicated
studies can be thought of as working hypotheses, which the
manager can support or refute by further investigations.
This approach can be expanded into the area of adaptive
resource management (Conroy, this proceedings). Second, if
the management practice in question is of interest to one
221
manager, it probably is of interest to others, and hopefully
the study can be, or already is, being replicated elsewhere
(perhaps also using a design that is similarly compromised).
Power
Power is defined as the probability of rejecting a false null
hypothesis, therefore, having high power in an experiment
is desirable. Unfortunately, the only way to increase the
power of an experiment while minimizing Type I error is to
increase the number of replicates. Because of the relationship between power and sample size, the power of most
large-scale manipulative experiments is low. There are
several ways to partially overcome the low power inherent in
studies with a small number of replicates. First, rather than
investigate many different levels of a particular treatment
(e.g., silvicultural options), one can reduce the number of
treatment levels, and for the same effort, increase the
number of replicates per level. Second, standardization of
methods among studies with similar objectives can increase
power by combining studies from different locations in one
analysis (meta-analysis, see Fernandez-Duque and Valeggia
1994). See Cohen (1977) for a more complete treatment of the
subject of power.
Manipulation
Sometimes, research aimed at understanding the effects
of alternative management practices on birds has involved
taking measurements in areas where alternative management practices were applied in the past. This assumes that
the different areas were similar to begin with, which probably is not the case. It also ignores the important consideration of randomization. However, short-term effects (≤3 yrs)
of alternative management practices are often not the same
as long-term effects (say, ≥10 yrs); understanding long-term
effects requires a long period of time that we often do not
have. Therefore, while mensurative experiments often are
not as rigorous or insightful as manipulative experiments,
they are sometimes necessary. Furthermore, short-term
effects can be misleading. For example, site fidelity can
cause migratory birds to occupy breeding territories in
habitats severely altered by timber cuts or other management activities that occurred during the previous winter,
even though the habitats have become suboptimal. Assessment of both short- and long-term effects are important;
therefore, we conclude that mensurative experiments can be
valuable. They are not, however, a substitute for good longterm research.
Pretreatment Data
Are pretreatment data necessary in manipulative experiments such as these? Strictly speaking, a treatment effect
can be detected on most parameters without pretreatment
data if adequate controls are used (e.g., Cooper and others
1990). Pretreatment data are desirable for several reasons,
however. First, we have found that a season or two is often
required for a project to operate smoothly. Often, we are
unsure which species we will monitor for demographic work
222
until we have gained some experience in the study area.
Second, estimation of survival necessitates a large number
of banded birds, and obtaining adequate sample sizes can
take several years. Third, because of low power or no replication, detecting treatment effects is often difficult. An
alternative to data analysis involves a largely graphical
approach, in which means and standard errors are plotted
for each treatment over time (Hurlbert 1984). Pretreatment
data are necessary in this type of analysis. Also, a BACI
analysis can be employed with this design (Stewart-Oaten
and others 1986).
Realism
Researchers tend to want to conduct experiments in an
ordered, efficient, elegant fashion. Managers, on the other
hand, generally are not as concerned with methodology as
they are with results (i.e., which management option produces the desired results). For example, in a study of the
effect of different logging practices on avian productivity
(see below), it was suggested that different treatments
should consist of removing the same total volume of timber
on each plot, but the size and number of individual cuts
should differ among plots. While this is attractive from a
research standpoint, it is not the way that timber is marked
and cut in an actual harvest operation on these national
wildlife refuges. The objective of harvesting was to promote
wildlife habitat. Different prescriptions result in different
percentages of standing timber removed. Therefore, the
individual trees that were selected for harvest were selected
and marked by the refuge silviculturalist using prescriptions typically employed on the refuge. The other approach
was simply not realistic, and inferences made from that
study design would have had limited application elsewhere.
Plot Size
The size of the study plots will vary with the study and its
objectives. We have adopted BBIRD (Breeding Bird Inventory and Research Database) protocols, which call for plots
to be at least 20 ha to allow enough area to conduct point
counts and to find a sufficient number of nests to make valid
inferences (Martin and others 1997).
Collaboration ___________________
Research such as that described above requires collaboration and cooperation between researchers and managers.
Researchers who attempt large-scale manipulative experiments without close collaboration with the managers on
whose land they are working risk conducting a presumably
well-designed experiment that has limited application elsewhere; they also risk a certain amount of resentment. Most
managers also are willing to share resources such as housing
and vehicles if available. On the other hand, most managers
are not researchers; they risk conducting a poorly designed
experiment if they do not consult with researchers wellversed in experimental design.
Large-scale manipulative experiments with adequate replication are expensive, and may be beyond the means of most
USDA Forest Service Proceedings RMRS-P-16. 2000
managers. Even on well-funded projects, replication may be
minimal. In such cases, collaboration with other land managers and researchers at other locations is helpful. By
bringing other resources to bear, a well-designed study can
be replicated at another location. If done carefully so that the
design and methodology are identical, then different locations can serve as blocks in a randomized design. Even if the
different studies are similar only in the questions they pose,
they sometimes can be combined in a meta-analysis, which
again serves to increase the power of the overall analysis.
Examples
1. Alternative Silvicultural Practices in Bottomland Hardwood Forests—This study was initiated in
1993 by the University of Memphis collaborating with the
U.S. Fish and Wildlife Service (USFWS), and was continued
in 1997 in a similar collaborative arrangement between the
USFWS and the University of Georgia. The study area is the
White River National Wildlife Refuge (NWR), one of many
federal refuges along the lower Mississippi Alluvial Valley
(MAV) originally established for management of migratory
waterfowl. Federal refuges also provide some of the largest
tracts of unfragmented bottomland hardwood forest remaining in the MAV. Most of the forest on refuges is managed to
promote within-stand diversity and the production of hard
mast; timber harvest alternatives designed to promote oak
regeneration are selective cuts (thinnings) or small (<2 ha)
patch cuts.
In 1993, six 50-ha plots were established in the same
management compartment (which was due to be harvested
in 1995), so that all plots were initially somewhat similar.
Three treatment options (thinnings, patch cuts, no cuts)
were randomly allocated to two plots each. After two seasons
of pre-treatment data collection in 1994-1995, treatments
were applied in the late summer-fall of 1995 under the
supervision of refuge silviculturalists. Post-treatment data
collection occurred in 1996-2000. Measurements taken include point counts, survival, productivity, habitat characteristics, and foraging behavior. Methods generally follow
BBIRD protocols.
Since its inception, the project has been funded by a
partnership between the University of Memphis (and later
the University of Georgia), the USFWS, the U.S. Forest
Service (USFS), the Biological Resources Division of the U.S.
Geological Survey, and the Arkansas Game and Fish Commission and the Arkansas Chapter of The Nature Conservancy. The refuge staff was present and contributed to every
aspect of planning the project, including personnel, equipment, and most importantly housing, which served to defray
costs and logistical problems.
Despite the favorable aspects of the design of this experiment, there are still only two replicates of each treatment,
providing low power. Therefore, we are collaborating with
researchers and managers at Tensas River NWR (D. J.
Twedt, unpublished data), who are performing a very similar experiment, also with two replicates of each treatment.
For some questions, then, this collaboration will double our
number of replicates.
USDA Forest Service Proceedings RMRS-P-16. 2000
2. Alternative Gypsy Moth Management Practices
in the Middle Appalachians—This study was initiated in
1994 by the University of Memphis collaborating with West
Virginia University (WVU) and the USFS, and was continued in 1997 with collaboration between the University of
Georgia, WVU, and the USFS. The study areas are located
in the George Washington National Forest in Virginia and
the Monongahela National Forest in West Virginia. The
gypsy moth (Lymantria dispar), accidentally introduced
from Eurasia in 1869, has become the most important insect
pest of eastern deciduous forests, annually defoliating millions of hectares. Pesticides are also applied to hundreds of
thousands of hectares annually. Although the pesticide most
commonly used is the bacterium Bacillus thuringiensis (Bt),
which affects larval insects only and has no direct effects on
vertebrates, it still may affect birds indirectly by eliminating
their most important prey, caterpillars. The gypsy moth is
expected to eventually invade most of the lower 48 states,
and is expected to be most devastating in the southern
states, whose forests have a high oak component (oaks are
preferred hosts).
Eighteen 200-ha plots were established in 1994, nine in
each forest. Within each plot, a 30-ha core plot was established where most of the bird and insect work is being
performed. Three treatment options (Bt, no-action [defoliation], and controls) were randomly allocated to the plots to
provide six replicates of each treatment option. Unlike many
studies, controls are not untreated treatments. Rather, we
expect the untreated areas to be defoliated, so they will not
serve as controls as defined above. Therefore, six plots will
be sprayed with Gypchek, the commercial formulation of the
nucleopolyhedrosis virus that affects only the gypsy moth (it
is not in widespread production and is available only for
experimental purposes). This will assure that the only difference between defoliated sites and controls will be defoliation (neither will have Bt), and that the only difference
between Bt-treated sites and controls will be Bt (neither will
have defoliation). Pre-treatment data collection occurred in
1995-1996, and pesticide treatments were applied in1997
and 1998. With post-treatment monitoring, the project is
scheduled to be a 10-year study. Measurements taken include point counts, territory mapping, survival, productivity, habitat, diet and foraging behavior, and arthropod
sampling. Methods again generally follow BBIRD protocols.
This project was funded by a competitively awarded grant
to a team headquartered at WVU. The team consists of
entomologists, a toxicologist, avian ecologists, and a herpetologist. Plots were located through collaboration with USFS
personnel. Spraying was done as part of the gypsy moth
eradication programs of the respective national forests, and
involved state and federal biologists. Spray volumes and
rates reflected normal applications.
3. Alternative Prescribed Burning Practices in
Longleaf Pine Forests—In the southeastern coastal plain,
applications of prescribed fire for management of northern
bobwhite (Colinus virginianus) quail in longleaf pine (Pinus
palustris) forests have been conducted during February and
March for more than 70 years (Stoddard 1931). The genesis
for this idea came from the understanding that lightningseason ignitions were a natural process of coastal plain
223
ecosystems. However, elements of controversy (fire burns
nests), as well as convenience (February and March are
pleasant times to burn, whereas June and July can be
miserable), resulted in development of a paradigm of dormant season applications of prescribed fire for many years.
Over time, enlightened managers realized that if prescribed fire was to be used to mimic an ecosystem process,
ignitions should be generated during the peak of lightning
season (April-August) rather than dormant season (late
winter). Although winter months are not outside the temporal range of natural fire occurrence in this region, research
and anecdotal observation indicate that applications of lightning season fire are essential for keystone plants like
wiregrass (Aristida stricta) to flower and set viable seed. The
existing literature on comparison of dormant and lightning
season fire applications is limited, controversial, and conflicting (Robbins and Myers 1992).
Therefore, an experiment was developed to examine how
dormant and lightning season applications of prescribed fire
influence the abundance and distribution of vertebrates in
longleaf pine forests. In one study, four pairs of 12 ha plots
were each established in Apalachicola National Forest in
Florida, and the Sandhills region of North Carolina. In each
pair, one of the two fire application options was randomly
assigned. The plots are imbedded in much larger management units that average 50-100 ha, which is the level of
management at which prescribed fire is applied. Here,
controls were not used because the “no treatment” option is
not a viable management alternative. Therefore, pre- and
post-treatment assessments are a central part of the experimental design (Engstrom and others 1996). In this study,
bird abundance was estimated by spot mapping. Other work
done on these plots and others in ancillary studies include
vegetation sampling, arthropod abundance, and quail radiotelemetry. The project is funded through a partnership
between the Department of Defense, the North Carolina
Wildlife Resources Division, National Fish and Wildlife
Foundation, four private foundations, several key donors
from the Red Hills Shooting Plantations, the USFS, USFWS,
Tall Timbers Research Station, and Mississippi State University (Brennan and others 1996).
4. Longleaf Pine Restoration/Midstory Removal Experiment—Longleaf pine forest once covered nearly 90
million acres of the southeastern coastal plain (Simberloff
1993). Today, less than 1 million ha remain. Much of the
remnant longleaf stands are degraded from lack of frequent
fire, which causes hardwoods to encroach and invade the
midstory of longleaf stands. In many cases, encroachment of
hardwood midstory causes keystone species such as redcockaded woodpecker (Picoides borealis), gopher tortoise
(Gopherus polyphemus), and bobwhite quail to leave or
undergo local extinction.
Personnel from Eglin Air Force Base, The Nature Conservancy, the University of Florida, and Tall Timbers Research
Station have been collaborating on a comprehensive, largescale hardwood midstory removal experiment in longleaf
pine stands at Eglin. The primary objective of the study is to
understand how three methods of hardwood removal (fire,
chainsaw, and herbicide) affect soil properties, vegetation,
224
arthropods, and birds. Each treatment area is 100 ha. There
are 6 replicates per treatment option (18 total), in addition
to 3 untreated stands. Thus, overall, treatments were applied to 1800 ha under various local contracts by the Jackson
Guard Natural Resources Division of Eglin. The project
features point counts, vegetation and arthropod sampling,
and measurements of avian foraging behavior.
Discussion _____________________
The benefits of large-scale manipulative experiments involving management questions are clear: Rigorous experimental design, realism, a good blend of management and
research, and immediate application of results. Another
benefit is that more resources are brought to bear to obtain
funding.
How is funding obtained? This sort of research is expensive, and is only likely to be funded extramurally through
competitive grants. The need for partnerships is critical,
both to make the research stronger and to increase the
funding base. Also, with more partners the funding base is
more secure. Even if a project is well funded but from a single
source, it is in jeopardy of ending if that single source is
suddenly gone.
Researchers who have spent their careers conducting
basic research might question the wisdom of becoming
involved with management questions. We have found that
several benefits result from conducting this sort of applied
research. First, it is generally easier to obtain funding to do
applied research, especially when agencies involved with
management are providing some of the funding. Second,
basic questions involving the ecology of migratory birds can
always be addressed within the framework of the applied
study. For example, Bt application can be viewed as an
experimental manipulation of prey abundance, providing
the basis for interesting questions involving bird/insect
interactions and habitat selection theory, among others
(e.g., Rodenhouse and Holmes 1992). Third, researchers in
conservation biology often wonder if their research is making a difference. We have found that the results of this
research are of great interest to managers, and that results
are often applied before they find their way into print.
Further, if conducted properly, the results of these studies
can be applied not only on the study site but elsewhere with
similar habitat types. Finally, collaboration among researchers and managers is the essence of Partners in Flight, an ad
hoc group whose members, while diverse in interests and
backgrounds, all work toward the single goal of conserving
populations of Neotropical migratory birds.
Acknowledgments ______________
We wish to acknowledge all of our collaborators and
contributors, too numerous to mention individually, on these
projects. We also acknowledge the contributions made by the
many technicians, interns, and graduate students on these
projects. Without all of you our work would be far less
meaningful, and this publication would not have been
possible.
USDA Forest Service Proceedings RMRS-P-16. 2000