Combining Information From Monitoring Programs: Complications Associated With Indices and Geographic Scale John R. Sauer I feel that some biologists at this conference have been too optimistic about the use of indices of abundance (Kenneth H. Pollock in Ralph and Scott 1981: p. 510). Abstract—To adequately monitor Neotropical migratory birds, information must be collected to assess population change at local, regional, and continentwide scales. I suggest that large-scale survey results (such as those derived from the North American Breeding Bird Survey) should not be used to predict population attributes on parks, refuges, and other protected areas. These areas are often managed, and generally contain habitats that can be poorly sampled in large scale surveys, hence local bird populations might be quite different from those sampled in the large-scale surveys. Furthermore, we are limited in our capabilities to combine information from local surveys with large-scale survey data. Most surveys of bird populations collect indices of abundance which are often not comparable among surveys due to habitat and region specific differences in probabilities of detecting birds. In assessing the effects of management, it is important to understand the limitations of monitoring at different geographic scales and to design programs to monitor at the scale at which management is conducted. Conservation of Neotropical migratory bird populations requires information on population responses to land management at several different scales: At a very local scale, such as within the plot on which management is conducted; at an intermediate scale, such as a refuge or forest containing a series of plots; and at regional levels, such as flyways, states, or the physiographic strata used in the North American Breeding Bird Survey (BBS, Peterjohn 1994). Point counts or other survey methods that produce an index to population size often are crucial components of monitoring programs at all these levels (e.g., Ralph and others 1995a). Of course, monitoring population size by itself is not sufficient for assessing effects of management (Nichols, this proceedings). Direct estimates of productivity and survival are more likely to provide insights into factors influencing population change (Temple and Wiens 1989). However, the difficulties in implementing demographic monitoring programs at large scales suggest that monitoring population In: Bonney, Rick; Pashley, David N.; Cooper, Robert J.; Niles, Larry, eds. 2000. Strategies for bird conservation: The Partners in Flight planning process; Proceedings of the 3rd Partners in Flight Workshop; 1995 October 1-5; Cape May, NJ. Proceedings RMRS-P-16. Ogden, UT: U.S. Department of Agriculture, Forest Service, Rocky Mountain Research Station. John R. Sauer, Patuxent Wildlife Research Center, 11510 American Holly Drive, Laurel, MD 20708. 124 size will play a critical role in assessing the effect of management actions. Often, large-scale surveys such as the BBS are considered to be primary methods of monitoring population size, with smaller-scale surveys (e.g., within national forests or in forest plots, Ralph and others 1995a) providing supplemental information that can contribute to a continentwide program. A frequent assumption is that information from surveys can be combined to form a composite index, and that information from the BBS can be applied at any geographic scale to address management issues. Unfortunately, both of these assumptions usually are false. In general, large-scale surveys (as exemplified by the BBS) do not provide adequate information for addressing local issues, and local surveys do not provide a valid sample frame for addressing regional issues. These generalizations may be correct for two primary reasons. First, the habitats managed by most land managers are not random samples from regional habitats, hence, the relative abundances and trends of birds in these local areas may not be representative of regional patterns. Second, counts from large-scale and local surveys provide only an index to the population. It is difficult to combine information from surveys unless the relationship between the indices and the actual population sizes associated with each survey are known (or at least estimated without bias). In this paper, I use examples from the BBS to discuss limitations of our present monitoring procedures, and to identify some crucial issues in monitoring at different scales. I suggest that technical difficulties with combining information from different surveys will limit our ability to integrate information among geographic scales, and that survey data should be viewed as addressing management issues at the scale at which they are collected. Can Breeding Bird Survey Results Provide Valid Predictions of Trend and Relative Abundance at Low Geographic Scales? _____________ The temptation to use BBS data to predict relative abundance and population trends of birds on refuges or other land management units is overwhelming when BBS data are displayed as GIS coverages that purport to map population trends rangewide for species (Sauer and others 1995). A simple overlay of the map onto the area of interest could provide a local estimate of trend for a species, if the regional results are valid for the lower scale. The consistency of USDA Forest Service Proceedings RMRS-P-16. 2000 regional patterns of population change for many species (e.g., Wood Thrush, Hylocichla mustelina) suggests that regional patterns often might be generally applicable to habitats within the region. Unfortunately, there is no capability for habitat-specific analyses within the sample units of the BBS. Large-scale surveys provide extensive results at the expense of local specificity. Indices of abundance from these surveys often are aggregate, based on large sample units (e.g., in the BBS: 50 3-minute point counts/24.5 mi survey route), and represent a composite index for habitats encountered in the landscape. At the scale at which the data are collected, relative abundance estimates from BBS data are correlated with a variety of landscape-level attributes. Flather and Sauer (1996) found associations among bird data from BBS routes and measures of percentage cover, edge, and other landscape metrics estimated from raster images of high-altitude photographs. Most traditional analyses of BBS data estimate trends or other components of population change using the route as the sample unit, and aggregate route trends to estimate regional patterns of population change. At this scale, maps of relative abundance from the BBS show reasonable qualitative views of bird distributions (Sauer and others 1995). Certainly, counts from pieces of each sample unit could be considered habitat-specific, and used as dependent variables for habitat-specific analyses. However, BBS stop locations often are difficult to assign unambiguously to habitats, hence the definition of a population or area associated with individual stops is not a trivial task. Also, data from individual stops tend to be very limited, and the 50-stop aggregate was originally chosen to ensure that many species would be represented on each survey route. These difficulties emphasize that the BBS was not designed for low-scale, habitat-specific analyses. The landscape-level nature of BBS data means that using the information for management at local scales is difficult. Most refuges and parks exist because they contain some unique habitat features, such as high mountains, roadless areas, wetlands, or continuous forests. These features ensure that little BBS data will actually be available for the area due to inaccessibility. Additionally, habitats in protected areas rarely experience the changes that occur outside the boundaries of the areas. Therefore, BBS data from surrounding routes will not provide representative information for these protected areas. Consequently, the BBS forms a regional context for evaluation of bird population trends in local areas and specialized habitats, but should not necessarily be considered a valid predictor of local trends. If changes in population of a migratory species are being caused by factors on the wintering grounds, then local trends might be consistent with regional trends; however, if changes are being influenced by local habitat factors, then local trends might be very different from regional patterns. Of course, if a refuge or park is acting as a source population for a region, population dynamics for that region will be greatly influenced by the dynamics within the source. Without understanding the causes and habitat-specificity of population change, we cannot reasonably assess whether a local site will be consistent with regional patterns of population change. USDA Forest Service Proceedings RMRS-P-16. 2000 Can Local Surveys be Incorporated Into Large-Scale Analyses? _______ Partners in Flight and other programs have resulted in development of many local surveys in areas that are not explicitly sampled by the BBS (e.g., Howe and others 1995). Often, we assume that these data can contribute to the largescale view of population change (e.g., Ralph and others 1995a). Federal properties in the United States such as national wildlife refuges and national parks are good examples of areas that generally are not sampled by the BBS. The enormous regional differences in relative size and dispersion of these properties suggest that land holdings of different agencies present different problems for analysis. Some properties, such as Bureau of Land Management or Forest Service holdings in the western United States, are incorporated into the landscape-level results of the BBS, as BBS routes are conducted on established roads in and adjacent to the properties. Because the lands make up a large part of the landscape, they are part of the random sample. To estimate trends specifically for these properties, it seems reasonable to consider them as separate strata in a stratified sampling design. Independent trend information could be estimated for the areas, and they could be combined with BBS as additional strata after eliminating overlapping BBS data. Because most independent surveys have not been conducted for enough years to provide long-term trends, few opportunities exist to develop composite trends from BBS and local survey results. Difficulties with sampling methods discussed below also may complicate development of composite results from BBS data and these strata. In addition, because the sizes of many federal holdings are very small relative to the BBS strata, a composite result based on area weights may not adequately incorporate the information from those lands. For example, large continuous forests in national forests in the Appalachians form a relatively small portion of the land area in the eastern United States, hence, they likely will be only a minor part of an area-weighted composite result. However, these lands might play major roles as bird population sources, and many managers might desire that these areas be given more weight in analyses of BBS data. Unfortunately, weighting results to appropriately incorporate relative abundance and area is one of the most controversial aspects of existing BBS analyses (Peterjohn 1994), and adding new strata to the analysis with even more weighting factors will tend to undermine the credibility of the results. Some investigations have attempted to post-stratify BBS survey routes in an attempt to develop sub-strata for analysis of habitat-specific trends (e.g., Ponderosa Pine (Pinus ponderosa) habitats in the southwestern United States, or areas with high cattle use versus areas of low cattle use, (Sauer, unpublished analyses). Although abundances of birds on BBS routes reflect underlying habitat characteristics along the routes (Flather and Sauer 1996), post-stratification of BBS routes is not an efficient strategy for analysis. The reclassification of route data generally results in small sample sizes, and leads to statistical tests having low power to detect differences among the sub-strata. 125 Complications Associated With Combining Results From Surveys That Collect Index Information ____ As noted above, combining information from different surveys is problematic if index data are collected. Point counts, such as those used in the BBS, are a population size index, and do not represent an actual population size. Instead, the counts underestimate the population size, that is, the likelihood that any observer actually counts all birds of any species present on a survey route is small. In the eastern United States, BBS observers probably count only an average of 50 percent of the individuals present of any species (Sauer and others 1994a). Although we can only speculate about the extent of the undercount, it likely differs among survey methods, regions, and habitats (e.g., higher proportions of the birds are observed in open habitats than in habitats with dense vegetation, Sauer and others 1994a). Observers also tend to differ greatly in counts they collect, even under similar circumstances of counting (Sauer and others 1994b). At least for some species, evidence exists that observers have gotten better at counting over time, and that they now count a higher proportion of the birds present (Sauer and others 1994b). The potential for differences in detection probabilities are intrinsic in any survey that does not explicitly estimate the proportion of birds that are detected (Lancia and others 1994). Any analysis of count data must at least consider the consequences of inconsistencies in the undercount (Sauer and others 1994a). Standardization of counting procedures (e.g., Ralph and others 1995a) does not eliminate the problem (Barker and Sauer 1995). One important consequence of this deficiency is that a count-based estimate of regional population size is flawed. In BBS analyses, even estimating a mean abundance for a route is not trivial. Commonly used estimates of bird abundance such as mean counts on the route are biased by observer changes and trend over time (Flather and Sauer 1996). Combining the results of surveys from different regions requires estimates of the populations in each region. If the surveys use methods of data collection that lead to different undercounts, reconciling their results may be difficult. For example, individuals of many species with low abundances can be attracted by playing recordings of their calls. Some biologists advocate special surveys in which the observer 126 plays recordings of calls and counts the responses. However, these special surveys would be conducted only in habitats with high probabilities of encountering the species, and over a limited part of the species range. Any attempt to combine these results with BBS data would require that we know how the index derived from the special survey relates to the BBS index. These relationships can be established only by estimating the size of the undercount associated with each survey, and adjusting survey results to a consistent index level. Conclusions ____________________ The BBS provides a large-scale view of bird populations and monitors many species of birds with sufficient precision to provide some confidence in the results (Peterjohn and others 1995). However, attempts to use BBS information to provide insights into local management actions is likely to lead to imprecise and inaccurate results. Also, studies designed specifically for local areas may not be easy to combine with the large-scale results provided by the BBS. However, both kinds of information are needed for assessment of management actions. Because managers cannot rely on the landscape-level results of the BBS to provide them with reliable information on local consequences of management actions, they must design surveys at the appropriate scale to monitor change. Adaptive management provides a reasonable framework for monitoring in the context of uncertainty about population responses (Nichols, this proceedings). Another alternative involves using BBS data as a “control,” and developing local monitoring programs in management “treatments” that are then used to test for differences in local area and regional population changes. Because of the technical considerations of combining different indices, local information should not be considered as part of a regional monitoring program without an understanding of the relative detection probabilities of the indices collected in each survey. Unfortunately, much of the literature on monitoring has suggested that all information collected can be fit into a composite nationwide program. Perhaps a more productive view would be to consider the need for habitat-specific or species-specific programs in areas where management occurs, and to design experiments to assess the consequences of the management. USDA Forest Service Proceedings RMRS-P-16. 2000