Breeding Bird Survey Statistical Methods
The North American Breeding Bird Survey (BBS) is conducted annually in the United States,
Canada, and, as of 2008, northern Mexico. The survey is designed to monitor trends in relative
abundance of North American breeding birds at continental, national, and regional scales.
Environment Canada's Canadian Wildlife Service and the United States Geological Survey's
(USGS) Patuxent Wildlife Research Center jointly coordinate the BBS program.
Data Collection
BBS data are collected along randomly established roadside routes by skilled volunteer
observers. Routes consist of 50 stops spaced 0.8 km apart along a 39.2 km length of all-weather
secondary roads. The starting point and direction of routes are selected randomly within a given
degree block (degree blocks are geographic areas 1 degree longitude by 1 degree latitude in size
and are the basic sample design units for the BBS) in order to sample a range of habitats
representative of the region. In Canada, the acceptable dates for running a BBS route are
between 28 May and 7 July although most observations are made during the first three weeks of
June. Observers are encouraged to run their routes for as many consecutive years as possible in
order to reduce the effects of observer variability on trend estimates. Because the BBS is
designed to monitor long-term changes in bird populations, changes in habitat along routes are
expected. However, routes may be altered or abandoned if the road system is altered, increasing
ambient noise interferes with an observer's ability to detect birds, or if traffic conditions are
unsafe for the observer.
At each of the 50 stops, observers conduct a three-minute point count observation, during which
they record the total number and species of all birds heard, and all birds seen within 0.4 km of
their location. Start and finish times for the routes are recorded. Sky and wind conditions are
recorded at the beginning of each 10-stop block and at the end of the route. The number of
vehicles passing and other excess noise are recorded during each stop. Data are entered into the
online database by observers, or are submitted to one of the BBS national offices for entry. All
BBS data are stored in the North American BBS database housed at the USGS Patuxent Wildlife
Research Center and are available on the USGS BBS website. Original data sheets for Canada are
stored by Environment Canada's Canadian Wildlife Service. For a comprehensive description of
the BBS field methods, see Robbins et al. (1986).
Data Screening
Several factors, in addition to changes in bird populations, contribute to variation within BBS
data. These include changes in weather, date, starting and finishing times, as well as differences
among observers. To help reduce the extraneous variability in the data, the data are screened to
remove surveys run under unacceptable conditions (e.g., rain and/or high winds, outside of the
range of acceptable dates, or outside of the range of acceptable times of day). All criteria are
available on the USGS BBS website.
Geographic Analysis Strata
The data are analyzed within geographic strata defined by the intersection of Bird Conservation
Regions (BCRs) and the Provinces/Territories. For example, British Columbia is divided into four
strata by BCRs 4, 5, 9, and 10. For a given species, only strata with sufficient data are included
in the analysis. The data are considered sufficient if there are at least 3 routes on which the
species has been observed and, on average, there are at least 2 years of observations for each
of those routes. Trends and estimates of relative abundance are modeled separately within each
of these analytical strata to account for differences in a species' population status and trends
among different regions, and to allow the stratum-level estimates to be combined into composite
regional estimates based on BCRs or political units.
The geographic nature of these strata means that users of these results should carefully consider
the spatial scope of the national and regional estimates of trends and annual indices from the
BBS data. To this end, the trend and annual index estimates for every species and region are
presented with an associated map showing the regions to which the estimates apply. The maps
provide an at-a-glance assessment of the spatial scope of each trend and annual index estimate,
and will be helpful in clarifying the following two potential sources of error in interpreting BBS
results:
First, the estimates apply to the specific strata in which the data were collected and these strata
may not completely cover a given region of interest. For example, we report "national" trends;
however, the BBS does not cover Canada in its entirety, so some portion of most species'
populations is not sampled by the BBS. The same can be said for many provinces, territories,
and BCRs.
Second, the short-term and long-term estimates for a given species and region may be
calculated based on data from different portions of the region. The earliest BBS routes in Canada
were run in the Maritimes and Quebec in 1966. By the early 1970s, much of southern Canada
had adequate coverage; however, some regions have only had adequate coverage for the last 20
years. Therefore, not all strata can be appropriately included in all long-term trend estimates
(i.e., approximately 40 years). For example, there were very few BBS data in the Northwest
Territories and much of the Yukon until the late 1980s or later, so trends in these areas cannot
reasonably be extended back to 1970. For a species and region that includes these later-starting
strata and strata with data from the full time-series (e.g., most national trends for species with
some distribution in the boreal forest), the short and long-term results will differ because they
represent different parts of the region. Although this complicates comparisons between shortand long-term trends, it allows for trend estimates for both the longest possible time period (the
long-term trend for regions with data over the full time-series) and for the largest geographic
area (the short-term trend for all regions with some data). In each case where the short- and
long-term trends include different strata, lists of included and excluded strata will be supplied
directly under the associated map, reflecting these differences.
Analytical Methods
The BBS is analyzed using a Hierarchical Bayesian model that accounts for the effects of
variation among observers and routes, first-year observer effects, variations in trend and
abundance among strata, and annual variation around a long-term trend. The observed counts
are assumed to have an over-dispersed Poisson distribution. The model is fit using standard
approaches to Bayesian modeling: Markov chain Monte Carlo (MCMC) methods with Gibbs
sampling using WinBUGS software. The model used here is derived from the model described in
Sauer and Link (2011), with some small changes to better suit the Canadian data. A detailed
statistical description of the model and the model code in WinBUGS language are available in
Smith et al., 2014in .
The hierarchical structure of the model makes the most efficient use of the data and also means
estimates are much less sensitive to random sampling error (e.g., variation in the number of
routes run in a given year). The parameters that represent potential sources of error or variation
in the data-over dispersion, variation among observers and routes, first-year observer effects,
and the annual variation around the stratum trend-are modeled as random effects (i.e.,
parameters governed by hyperparameters higher up in the hierarchy). The individual parameters
(e.g., the effect of a given observers' first year, or the departure from the stratum-specific longterm trend line in a given year) are drawn from a distribution of parameters with a common
mean and variance: the hyperparameters. One particular benefit/consequence of the hierarchical
structure is that the inter-annual variation in the annual indices partially reflects the quality of
the underlying data. As a result, the annual indices will show patterns in the population's
trajectory (i.e., well-defined, inter-annual variation) if the underlying data support that pattern,
but in years with sparse or variable counts, the pattern in the trajectory is smoothed towards the
trend line.
Reliability Measures
To quantify the reliability of the BBS trend estimates in describing overall population trends,
three measures of reliability are calculated: precision, coverage and model fit. In addition to
these three credibility measures, one may also want to consider the following measures reported
in the final columns of the trend data tables: the number of routes used to generate the trends,
the estimates of the probabilities that a given population has decreased or increased, or the
probabilities that the magnitude of the population change falls within one of the six population
change categories. These probabilities provide a concise and intuitive assessment of the
uncertainty around some commonly desired interpretations. For example, a relatively strong
positive trend that is imprecisely estimated may in fact have a greater probability of being a
decline, than a more stable but precisely estimated trend.
Precision: This measure reflects the width of the 95% credible interval (CI). Smaller values
indicate a more precise trend estimate. The 95% CI defines a range of trend values that have a
95% probability of including the true population trend value, given the data and the model. For
example, the following statement is a valid interpretation of a hypothetical 95% CI with lower
limit = 1.2 and upper limit = 2.3: given the BBS data and the accuracy of the model, there is a
95% probability that the average annual increase in the population lies somewhere between
1.2%/year and 2.3%/year.
Geographic coverage: This measure indicates what proportion of the regional population had
some probability of influencing the estimated trend. It is calculated as the estimated proportion
of the regional population inside degree-blocks with BBS data. The reported value is a geometric
average annual proportion covered during the time period of the trend, where degree blocks are
considered covered for all years between the first and last years that contributing routes were
run in the degree-block. Estimates of regional and degree-block specific population proportions
were derived using maps of a species' relative density across its Canadian range. These maps
were created by overlaying existing species range maps with estimates of relative population
density within the species' range, which were derived using the average number of individuals on
BBS routes run from 1997 through 2007, and comparable counts from other monitoring
programs.
Model fit: This measure reflects the adequacy of some of the assumptions underlying the
statistical model. The fit-statistic reported in the trend tables measures the fit of the model to
the observed data. It is a Bayesian p-value, where values <0.01 or >0.99 indicate poor fit (i.e.,
a consistent difference between the observed data and a simulated dataset that fits the model).
The Bayesian p-value is used in combination with a convergence criterion, to categorize the
model fit component of overall reliability. The convergence criterion (Gelman-Rubin statistic)
measures the consistency of the results across three independent MCMC chains with randomly
chosen initial starting values. Gelman-Rubin values >1.1 were considered convergence failures
for the primary stratum-specific parameters (long-term change and average abundance).
Convergence failures are not specifically reported with the trend estimates, because they are
relatively rare, however the data are available on request from BBS@ec.gc.ca.
Precision, coverage, and model fit are combined into an overall reliability score for each trend.
These overall reliability scores provide an overview that alerts the user to potentially unreliable
trends. However, specific measures are also available to make an informed decision about the
uncertainty associated with particular conclusions based on the trends. For example, a given
trend may have a low reliability score because it is relatively imprecise (i.e., large credible
interval), but, if for example the magnitude of the trend estimate is sufficiently large, this
imprecision may not affect an overall conclusion that the species' population has increased.
Additional Information
See Smith et al. 2014, for a statistical comparison between the new hierarchical Bayesian
analysis and the previous estimating equations approach that was used to generate BBS trends
for Canada in 2009.
Literature Cited
Robbins, C. S., D. Bystrak, and P. H. Geissler. 1986. The Breeding Bird Survey: its first fifteen
years, 1965-1979. U.S. Fish and Wildlife Service Resource Publication157.
Sauer, J. R., and W. A. Link. 2011. Analysis of the North American Breeding Bird Survey using
hierarchical models. The Auk 128:87-98
Smith A. C, M-A. R Hudson, C. M Downes, and C. M Francis. 2014. Estimating breeding bird
survey trends and annual indices for Canada: how do the new hierarchical Bayesian estimates
differ from previous estimates. Canadian Field-Naturalist 128:119-134. Available (open-access):
http://www.canadianfieldnaturalist.ca/index.php/cfn/article/view/1565
Geographic Analysis Strata
The following Canadian map shows geographic strata defined by the intersection of Bird
Conservation Regions (BCRs) and the Provinces/Territories. The data are analyzed within these
geographic strata.
Definitions
Species
Indicates the species' common English name.
Note on Species Names
For some species that have undergone a change in taxonomy since the beginning of the
Breeding Bird Survey, trend results are presented for both the individual species as well as a
"combined" trend that includes all data for the species complex. Users may wish to examine
both. These species complexes are:







Sapsuckers (Yellow-bellied/Red-naped/Red-breasted Sapsucker)
Traill’s Flycatcher (Alder/Willow Flycatcher)
Western Flycatcher (Pacific-slope/Cordilleran Flycatcher): the majority of records in
Canada are Pacific-slope Flycatchers.
Solitary Vireo (Cassin’s/Blue-headed Vireo)
Northern Oriole (Bullock’s/Baltimore Oriole)
Blue Grouse (Dusky/Sooty Grouse): in 2006 the species was split into Dusky Grouse
and Sooty Grouse. Results are presented only for the Blue Grouse species complex.
Western Grebe (Western/Clark’s Grebe): in 1985 the "Western Grebe" was split into
Western and Clark’s Grebe. Results are presented only for the species complex; Clarke’s
Grebe is very rare in Canada and these results primarily reflect the trend for Western
Grebe.
Time Period
Indicates the time period of the estimated trend. Two time periods are reported for each
combination of species and region:


Long-term = first year with sufficient data (usually 1970) to present.
Short-term = most recent ten years.
Annual trend
Indicates the trend estimate (i.e., the average annual percent change in the population).
Limits
Lower
Indicates the lower 95% credible limit on the estimated annual trend (i.e., conditional on the
model and the data, there is a 95% probability that the true trend falls between the lower 95%
limit and the upper 95% limit).
Upper
Indicates the upper 95% credible limit on the estimated annual trend (i.e., conditional on the
model and the data, there is a 95% probability that the true trend falls between the lower 95%
limit and the upper 95% limit).
Overall Reliability
Indicates categories from low to high. This column combines three specific and largely
independent measures of the credibility of the estimated trend. The category reported here is the
minimum category of the three primary credibility measures: Geographic coverage, Model fit,
and Precision. See the explanations of each credibility measure for category definitions.
N Routes
Indicates the number of unique BBS routes that were included in the analysis and were run at
least once between the start and end years of the trend.
Reliability details
Geographic coverage
Indicates the proportion of the species’ regional breeding population within the area covered by
the BBS. The reported value is a geometric average annual proportion. In a given year, the area
covered is defined as the area within degree-blocks (1 degree longitude by 1 degree latitude)
that contain routes contributing data to the analysis. Degree blocks are considered to contain
routes contributing to the analysis for all years between the first and last years that routes were
run in the degree-block.
Categories for overall reliability:



High > 50% regional pop (i.e., > 0.50) within covered area
Medium 50% - 25%
Low < 25%
Note on missing coverage estimates: For some species in some regions, no estimate of
geographic coverage is provided. These missing values occur when the BBS dataset includes
observations of a species from outside the area defined by the species’ range map. For strata
with no overlap between the observations and the species’ range map, the proportion of the
species’ regional breeding population covered cannot be calculated.
Range maps provide relatively coarse estimates of a species’ distribution and can only provide a
snap-shot in time. BBS observations may, in some cases, track expanding species’ ranges, or
simply indicate errors in the range maps.
Model fit
Indicates the fit of the model to the observed data, and is a Bayesian p-value, where values
<0.01 or >0.99 indicate poor fit. This Bayesian p-value is used in combination with a
convergence criterion (not reported here) to categorize the model fit component of overall
reliability. Convergence failures occur when the results depend on randomly chosen initial
starting values, for the primary stratum-specific parameters (long-term change and average
abundance). Convergence failures are not reported here because they are relatively rare,
however the data are available on request from BBS@ec.gc.ca.
Categories for overall reliability:



High = no strata failed to converge and Bayesian p-value > 0.01 and < 0.99
Medium = at least one stratum failed to converge or Bayesian p-value < 0.01 or > 0.99
Low = at least one stratum failed to converge and Bayesian p-value < 0.01 or > 0.99
Precision
Indicates the width of the 95% credible interval (Upper limit - Lower limit).
Categories for overall reliability:



High < 3.5 width of credible interval
Medium 3.5 - 6.7
Low > 6.7
These categories indicate trend estimates that are sufficiently precise to confidently identify the
following magnitudes of population declines over 20 years:



High 30% decline
Medium 50% decline
Low category includes trends that are too imprecise to confidently identify a 50%
population decline over 20 years.
Probability of decrease
Indicates the probability that the population has decreased during the specified time period,
conditional on the model and the data. For example, a value of 0.944 indicates that there is a
94.4% probability that the population has declined since 1970.
Probability of increase
Indicates the probability that the population has increased between the specified time period,
conditional on the model and the data.
Probabilities of population change during time period.
Reflect the probabilities that the total change in population size falls into each of the 6
categories. Values can be added across categories, but may not sum to 1.0 due to rounding
errors. All probabilities are conditional on the model and the data.
> 50% decrease
Indicates the probability that the population has decreased by more than 50% during the
specified time period.
25% - 50% decrease
Indicates the probability that the population has decreased by 25 to 50% during the specified
time period.
0% - 25% decrease
Indicates the probability that the population has decreased by 0 to 25% during the specified time
period.
0% - 33% increase
Indicates the probability that the population has increased by 0 to 33% during the specified time
period.
33% - 100% increase
Indicates the probability that the population has increased by 33 to 100% during the specified
time period.
> 100% increase
Indicates the probability that the population has increased by more than 100% during the
specified time period.
Definitions - Annual Index results
Year
Indicates the year associated with the estimated annual index.
Annual Index
Indicates the estimated average abundance on BBS routes run in a given year by an average
observer.
Lower limit
Indicates the lower 95% credible limit on the annual index.
Upper limit
Indicates the upper 95% credible limit on the annual index.