Combining Information From Monitoring
Programs: Complications Associated With
Indices and Geographic Scale
John R. Sauer
I feel that some biologists at this conference have been too
optimistic about the use of indices of abundance (Kenneth H.
Pollock in Ralph and Scott 1981: p. 510).
Abstract—To adequately monitor Neotropical migratory birds,
information must be collected to assess population change at local,
regional, and continentwide scales. I suggest that large-scale survey
results (such as those derived from the North American Breeding
Bird Survey) should not be used to predict population attributes on
parks, refuges, and other protected areas. These areas are often
managed, and generally contain habitats that can be poorly sampled
in large scale surveys, hence local bird populations might be quite
different from those sampled in the large-scale surveys. Furthermore, we are limited in our capabilities to combine information from
local surveys with large-scale survey data. Most surveys of bird
populations collect indices of abundance which are often not comparable among surveys due to habitat and region specific differences
in probabilities of detecting birds. In assessing the effects of management, it is important to understand the limitations of monitoring at different geographic scales and to design programs to monitor
at the scale at which management is conducted.
Conservation of Neotropical migratory bird populations
requires information on population responses to land management at several different scales: At a very local scale,
such as within the plot on which management is conducted;
at an intermediate scale, such as a refuge or forest containing a series of plots; and at regional levels, such as flyways,
states, or the physiographic strata used in the North American Breeding Bird Survey (BBS, Peterjohn 1994). Point
counts or other survey methods that produce an index to
population size often are crucial components of monitoring
programs at all these levels (e.g., Ralph and others 1995a).
Of course, monitoring population size by itself is not sufficient for assessing effects of management (Nichols, this
proceedings). Direct estimates of productivity and survival
are more likely to provide insights into factors influencing
population change (Temple and Wiens 1989). However, the
difficulties in implementing demographic monitoring programs at large scales suggest that monitoring population
In: Bonney, Rick; Pashley, David N.; Cooper, Robert J.; Niles, Larry,
eds. 2000. Strategies for bird conservation: The Partners in Flight planning process; Proceedings of the 3rd Partners in Flight Workshop; 1995
October 1-5; Cape May, NJ. Proceedings RMRS-P-16. Ogden, UT: U.S.
Department of Agriculture, Forest Service, Rocky Mountain Research
Station.
John R. Sauer, Patuxent Wildlife Research Center, 11510 American Holly
Drive, Laurel, MD 20708.
124
size will play a critical role in assessing the effect of management actions.
Often, large-scale surveys such as the BBS are considered
to be primary methods of monitoring population size, with
smaller-scale surveys (e.g., within national forests or in
forest plots, Ralph and others 1995a) providing supplemental information that can contribute to a continentwide program. A frequent assumption is that information from surveys can be combined to form a composite index, and that
information from the BBS can be applied at any geographic
scale to address management issues. Unfortunately, both of
these assumptions usually are false.
In general, large-scale surveys (as exemplified by the
BBS) do not provide adequate information for addressing
local issues, and local surveys do not provide a valid sample
frame for addressing regional issues. These generalizations
may be correct for two primary reasons. First, the habitats
managed by most land managers are not random samples
from regional habitats, hence, the relative abundances and
trends of birds in these local areas may not be representative
of regional patterns. Second, counts from large-scale and
local surveys provide only an index to the population. It is
difficult to combine information from surveys unless the
relationship between the indices and the actual population
sizes associated with each survey are known (or at least
estimated without bias).
In this paper, I use examples from the BBS to discuss
limitations of our present monitoring procedures, and to
identify some crucial issues in monitoring at different scales.
I suggest that technical difficulties with combining information from different surveys will limit our ability to integrate
information among geographic scales, and that survey data
should be viewed as addressing management issues at the
scale at which they are collected.
Can Breeding Bird Survey Results
Provide Valid Predictions of Trend
and Relative Abundance at Low
Geographic Scales? _____________
The temptation to use BBS data to predict relative abundance and population trends of birds on refuges or other land
management units is overwhelming when BBS data are
displayed as GIS coverages that purport to map population
trends rangewide for species (Sauer and others 1995). A
simple overlay of the map onto the area of interest could
provide a local estimate of trend for a species, if the regional
results are valid for the lower scale. The consistency of
USDA Forest Service Proceedings RMRS-P-16. 2000
regional patterns of population change for many species
(e.g., Wood Thrush, Hylocichla mustelina) suggests that
regional patterns often might be generally applicable to
habitats within the region.
Unfortunately, there is no capability for habitat-specific
analyses within the sample units of the BBS. Large-scale
surveys provide extensive results at the expense of local
specificity. Indices of abundance from these surveys often
are aggregate, based on large sample units (e.g., in the BBS:
50 3-minute point counts/24.5 mi survey route), and represent a composite index for habitats encountered in the
landscape.
At the scale at which the data are collected, relative
abundance estimates from BBS data are correlated with a
variety of landscape-level attributes. Flather and Sauer
(1996) found associations among bird data from BBS routes
and measures of percentage cover, edge, and other landscape metrics estimated from raster images of high-altitude
photographs. Most traditional analyses of BBS data estimate trends or other components of population change using
the route as the sample unit, and aggregate route trends to
estimate regional patterns of population change. At this
scale, maps of relative abundance from the BBS show reasonable qualitative views of bird distributions (Sauer and
others 1995).
Certainly, counts from pieces of each sample unit could be
considered habitat-specific, and used as dependent variables for habitat-specific analyses. However, BBS stop locations often are difficult to assign unambiguously to habitats,
hence the definition of a population or area associated with
individual stops is not a trivial task. Also, data from individual stops tend to be very limited, and the 50-stop aggregate was originally chosen to ensure that many species
would be represented on each survey route. These difficulties emphasize that the BBS was not designed for low-scale,
habitat-specific analyses.
The landscape-level nature of BBS data means that using
the information for management at local scales is difficult.
Most refuges and parks exist because they contain some
unique habitat features, such as high mountains, roadless
areas, wetlands, or continuous forests. These features ensure that little BBS data will actually be available for the
area due to inaccessibility. Additionally, habitats in protected areas rarely experience the changes that occur outside the boundaries of the areas. Therefore, BBS data from
surrounding routes will not provide representative information for these protected areas.
Consequently, the BBS forms a regional context for evaluation of bird population trends in local areas and specialized
habitats, but should not necessarily be considered a valid
predictor of local trends. If changes in population of a
migratory species are being caused by factors on the wintering grounds, then local trends might be consistent with
regional trends; however, if changes are being influenced by
local habitat factors, then local trends might be very different from regional patterns. Of course, if a refuge or park is
acting as a source population for a region, population dynamics for that region will be greatly influenced by the
dynamics within the source. Without understanding the
causes and habitat-specificity of population change, we
cannot reasonably assess whether a local site will be consistent with regional patterns of population change.
USDA Forest Service Proceedings RMRS-P-16. 2000
Can Local Surveys be Incorporated
Into Large-Scale Analyses? _______
Partners in Flight and other programs have resulted in
development of many local surveys in areas that are not
explicitly sampled by the BBS (e.g., Howe and others 1995).
Often, we assume that these data can contribute to the largescale view of population change (e.g., Ralph and others
1995a).
Federal properties in the United States such as national
wildlife refuges and national parks are good examples of
areas that generally are not sampled by the BBS. The
enormous regional differences in relative size and dispersion of these properties suggest that land holdings of different agencies present different problems for analysis. Some
properties, such as Bureau of Land Management or Forest
Service holdings in the western United States, are incorporated into the landscape-level results of the BBS, as BBS
routes are conducted on established roads in and adjacent to
the properties. Because the lands make up a large part of the
landscape, they are part of the random sample.
To estimate trends specifically for these properties, it
seems reasonable to consider them as separate strata in a
stratified sampling design. Independent trend information
could be estimated for the areas, and they could be combined
with BBS as additional strata after eliminating overlapping
BBS data. Because most independent surveys have not been
conducted for enough years to provide long-term trends, few
opportunities exist to develop composite trends from BBS
and local survey results. Difficulties with sampling methods
discussed below also may complicate development of composite results from BBS data and these strata.
In addition, because the sizes of many federal holdings are
very small relative to the BBS strata, a composite result
based on area weights may not adequately incorporate the
information from those lands. For example, large continuous forests in national forests in the Appalachians form a
relatively small portion of the land area in the eastern
United States, hence, they likely will be only a minor part of
an area-weighted composite result. However, these lands
might play major roles as bird population sources, and many
managers might desire that these areas be given more
weight in analyses of BBS data. Unfortunately, weighting
results to appropriately incorporate relative abundance and
area is one of the most controversial aspects of existing BBS
analyses (Peterjohn 1994), and adding new strata to the
analysis with even more weighting factors will tend to
undermine the credibility of the results.
Some investigations have attempted to post-stratify BBS
survey routes in an attempt to develop sub-strata for analysis of habitat-specific trends (e.g., Ponderosa Pine (Pinus
ponderosa) habitats in the southwestern United States, or
areas with high cattle use versus areas of low cattle use,
(Sauer, unpublished analyses). Although abundances of
birds on BBS routes reflect underlying habitat characteristics along the routes (Flather and Sauer 1996), post-stratification of BBS routes is not an efficient strategy for analysis. The reclassification of route data generally results in
small sample sizes, and leads to statistical tests having low
power to detect differences among the sub-strata.
125
Complications Associated With
Combining Results From Surveys
That Collect Index Information ____
As noted above, combining information from different
surveys is problematic if index data are collected. Point
counts, such as those used in the BBS, are a population size
index, and do not represent an actual population size.
Instead, the counts underestimate the population size, that
is, the likelihood that any observer actually counts all birds
of any species present on a survey route is small. In the
eastern United States, BBS observers probably count only
an average of 50 percent of the individuals present of any
species (Sauer and others 1994a). Although we can only
speculate about the extent of the undercount, it likely differs
among survey methods, regions, and habitats (e.g., higher
proportions of the birds are observed in open habitats than
in habitats with dense vegetation, Sauer and others 1994a).
Observers also tend to differ greatly in counts they collect,
even under similar circumstances of counting (Sauer and
others 1994b). At least for some species, evidence exists that
observers have gotten better at counting over time, and that
they now count a higher proportion of the birds present
(Sauer and others 1994b).
The potential for differences in detection probabilities are
intrinsic in any survey that does not explicitly estimate the
proportion of birds that are detected (Lancia and others
1994). Any analysis of count data must at least consider the
consequences of inconsistencies in the undercount (Sauer
and others 1994a). Standardization of counting procedures
(e.g., Ralph and others 1995a) does not eliminate the problem (Barker and Sauer 1995). One important consequence of
this deficiency is that a count-based estimate of regional
population size is flawed. In BBS analyses, even estimating
a mean abundance for a route is not trivial. Commonly used
estimates of bird abundance such as mean counts on the
route are biased by observer changes and trend over time
(Flather and Sauer 1996).
Combining the results of surveys from different regions
requires estimates of the populations in each region. If the
surveys use methods of data collection that lead to different
undercounts, reconciling their results may be difficult. For
example, individuals of many species with low abundances
can be attracted by playing recordings of their calls. Some
biologists advocate special surveys in which the observer
126
plays recordings of calls and counts the responses. However,
these special surveys would be conducted only in habitats
with high probabilities of encountering the species, and over
a limited part of the species range. Any attempt to combine
these results with BBS data would require that we know how
the index derived from the special survey relates to the BBS
index. These relationships can be established only by estimating the size of the undercount associated with each survey,
and adjusting survey results to a consistent index level.
Conclusions ____________________
The BBS provides a large-scale view of bird populations
and monitors many species of birds with sufficient precision
to provide some confidence in the results (Peterjohn and
others 1995). However, attempts to use BBS information to
provide insights into local management actions is likely to
lead to imprecise and inaccurate results. Also, studies designed specifically for local areas may not be easy to combine
with the large-scale results provided by the BBS. However,
both kinds of information are needed for assessment of
management actions.
Because managers cannot rely on the landscape-level
results of the BBS to provide them with reliable information
on local consequences of management actions, they must
design surveys at the appropriate scale to monitor change.
Adaptive management provides a reasonable framework for
monitoring in the context of uncertainty about population
responses (Nichols, this proceedings). Another alternative
involves using BBS data as a “control,” and developing local
monitoring programs in management “treatments” that are
then used to test for differences in local area and regional
population changes. Because of the technical considerations
of combining different indices, local information should not
be considered as part of a regional monitoring program
without an understanding of the relative detection probabilities of the indices collected in each survey.
Unfortunately, much of the literature on monitoring has
suggested that all information collected can be fit into a
composite nationwide program. Perhaps a more productive
view would be to consider the need for habitat-specific or
species-specific programs in areas where management occurs, and to design experiments to assess the consequences
of the management.
USDA Forest Service Proceedings RMRS-P-16. 2000