The good, the bad and the ugly - How to read and understand climate-health
studies
Sari Kovats, for NCAR Colloquium. 2006
Learning Objectives
By the end of this session you should be able to:
 describe the main methodological issues in environmental epidemiology, specifically
those relating to the investigation of the heath effects of weather and climate
variability.
 assess and critically interpret scientific data relating to potential climate hazards to
health
Introduction to epidemiology
Public health is the method and practice of the prevention of disease in human
populations. Epidemiology is the branch of public health which attempts to discover the
causes of disease in order to make disease prevention possible.
Epidemiological studies are:
 quantitative (rather than qualitative)
 observational (rather than experimental)
 studies of the determinants of disease in human populations (rather than
individuals).
Experimental studies (such as randomised trials) are used to determine the effectiveness
of specific health protection measures (interventions, i.e. sex education programmes,
drug treatment). They are not a common feature of environmental epidemiology but are
an important component of public health research.
Environmental epidemiology is one branch of epidemiology that is concerned with health
determinants in the environment (including pollutants of the water, soil and air).
Environmental epidemiology: methods and concepts
Measures of disease frequency
Incidence: The incidence of a disease is the rate at which new cases occur in a
population during a specified period. When the population at risk is roughly constant,
incidence is measured as:
Number of new cases
Population at risk x time during which cases were ascertained
If the “population at risk” changes during the period of study, for example, through
births, deaths, or migrations, then the incidence measure should relate the numbers of
1
new cases to the person years at risk, calculated by adding together the periods during
which each individual member of the population is at risk during the measurement
period. Thus incidence is defined as:
Number of new cases
Total person years at risk
Prevalence: The prevalence of a disease is the proportion of a population that are cases
at a point in time. Prevalence is an appropriate measure in such relatively stable
populations and is unsuitable for acute disorders.
Mortality is the incidence of death from a disease. For measures of morbidity (non-fatal
outcomes) it is important to have an agreed standard case definition and to ensure that
collection of the health data is standardised in order to avoid bias. The population at risk
(or total person years at risk) is also called the denominator.
Epidemiological study designs
Cohort studies: In a longitudinal study subjects are followed over time with continuous
or repeated monitoring of risk factors or health outcomes, or both. Such investigations
vary enormously in their size and complexity. A large population may be studied over
decades. Outcomes such as mortality and incidence of cancer have been related to
employment status, housing, and air pollution exposures.
Cross-sectional studies: A cross sectional study measures the prevalence of health
outcomes or determinants of health, or both, in a population at a point in time or over a
short period. Such information can be used to explore aetiology but associations must be
interpreted with caution. Bias may arise because of selection into or out of the study
population. A cross sectional design may also make it difficult to establish what is cause
and what is effect, because cause and effect are measured at the same point in time.
Studies that look at climate and environmental factors as determinants of the
distribution of a disease are a type of cross-sectional study.
Time series studies: the analysis of variation in health events, such as daily or weekly
counts of deaths or hospital admissions, in relation to exposures measured at similar
temporal resolution. Refer to lecture by Paul Wilkinson.
Case-control studies: In a case-control study, patients who have developed a disease
are identified and their past exposure to a certain risk factor is compared with that of
controls who do not have the disease. This permits estimation of odds ratios (but not of
attributable risks). Allowance is made for potential confounding factors by measuring
them and making appropriate adjustments in the analysis. The choice of controls is very
important and must be done to minimise bias. An example of a case-control study is the
study of risk factors for heatwave deaths in Chicago in 1995 by Semenza et al (1997). In
this study, controls were chosen from the same neighbourhoods as the decedents.
Ecologic Studies
An ecological (ecologic) study is one where the exposure information is collected on a
group rather than on individuals. For example, weather exposures for a population are
obtained from a single weather station. An ecologic study does not represent a
fundamentally different study design, but merely a particular variant of the four basic
study designs describe above. The information on average levels of exposure in a
population is used as a surrogate measure of exposure in individuals.
2
This approach is sometimes criticised as inadequate and unreliable because of the
“ecological fallacy” and other forms of bias that can occur. However, ecological studies
are important for several reasons:
 Hypothesis generation and testing. Population-level studies play an essential role
in defining the most important public health problems to be addressed, and in
generating hypotheses as to their potential causes. Many important individuallevel risk factors for disease simply do not vary enough within populations to
enable their effects to be identified or studied (Rose, 1992).
 Some important risk factors for disease genuinely operate at the population level.
In some instances they may directly cause disease, but perhaps more commonly
they may cause disease as effect modifiers or determinants of exposure to
individual-level risk factors. For example, being poor in a rich country or
neighbourhood may be worse than having the same income level in a poor country
or neighbourhood, because of problems of social exclusion and lack of access to
services and resources. Climate is another factor that only operates at the
population level.
Precision and validity
All epidemiological studies are concerned with precision and validity.
Random error (chance) will occur in any epidemiologic study. The confidence intervals
around the final estimate and hypothesis testing (p-values) are typically used to assess
the role of chance in any apparent association between exposure and health outcome.
An understanding of basic statistical principles and methods is necessary but will not be
addressed in this lecture.
In order to ensure validity, the study should be designed in order to gather information
on all known risk factors (potential confounders), and to adjust for these in the analysis.
However, there will always be other unknown or unmeasurable risk factors in any study.
Epidemiological studies are also subject to bias, where systematic errors occur in
measuring the exposure or the disease/outcome. Information bias is a common problem
when information is collected by surveys (self-reported). For example, persons who have
been flooded may be more likely to remember minor illnesses than persons who have
not been flooded.
Climate and health research: the role of epidemiology
Research on the potential health impacts of climate change must rely on observed
effects of weather and climate, primarily using epidemiological methods. As climate
change requires at least 30 years of data, the opportunities for observing directly the
effects on human health are limited.
3
Climate and weather exposures
Climate variability can be expressed at various temporal scales (by day, season and
year) and is an inherent characteristic of climate, whether the climate system is subject
to change or not. Climate "exposures" can be described in three broad temporal
categories:
 Long term changes in mean temperatures, and other climate "norms" (e.g. global
climate change).
 Climate variability about norms over periods ranging from a few years to several
decades, including:
- shifts in the frequency/probability distributions of climate variables
- recurring climate phenomena such as the El Niño-Southern Oscillation
(ENSO).
 Isolated extreme events (either simple extremes, e.g. temperature/precipitation
extremes, or complex events such as tropical cyclones, floods or droughts).
These types of "exposures" are clearly not independent. Many health outcomes are
sensitive to isolated extreme events (e.g. heavy rainfall, high temperatures) but are not
likely to be significantly affected by long-term, incremental, climate change, unless these
same meteorological extremes also change in frequency or character.
Use of meteorological data
Daily meteorological variables can be obtained for stations near to the population under
study. In cities, this is not usually a problem. In rural areas, however, it may be difficult
to find a station nearby. As a general rule, if using daily data, temperatures are
homogenous within about 300km area providing that there are no local landscape
features that affect climate, such as mountains, or water courses or coastal regions. For
monthly data, temperatures are similar up to 1200km area. Precipitation exposures are
more localised in time and space. Therefore, such data should not be used from a 50 km
radius, or 400 km radius for daily or monthly values, respectively. For these, reasons
4
care should be taken when aggregating variables such as precipitation and humidity over
large areas. Other dataset are available where stations are missing have been
interpolated, or supplemented with modelled data (also called re-analysis data).
Although these sources are readily available on online, it is preferable to use local
observed data were possible. The use of reanalysis data may give spurious results for
studies of local effects.
ENSO and health studies
Many studies have looked at the associations between health outcomes and inter-annual
variability in climate parameters (temperature and rainfall). These studies require long
time series in order to a sufficient measure of the climate signal. Many studies have
investigated the effect of ENSO, but there have also been papers on the North Atlantic
oscillation (NAO) and Quasi-Biennial Oscillation (QBO).
Recommendations1 for conducting and reporting studies for the effects of ENSO on
health outcomes:







Test and report results of association between weather parameters and ENSO
parameter in the data.
Report published assessments of ENSO teleconnections by climatologists in region of
interest
Describe the geographical area from which the health data are derived.
Test and report results of association between weather parameters and disease
outcome
Use time series with more than one ENSO event
Remove any trend and seasonal patterns in the time-series prior to assessing
relationships
Report associations both with and without adjustment for autocorrelation
Reviews: searching the peer reviewed literature
National and international assessments of the potential impact of climate change require
thorough and systematic reviews of the peer-reviewed literature. A literature search
must be systematic so that important studies (particularly negative findings) are not
missed. Without a systematic approach to identifying and reading papers, the review
may give a biased picture of current scientific knowledge.
It is important to have a clearly defined search strategy: this would include the
specification of the search terms (e.g. exposure, outcome), and the databases that will
be searched. The quality of the studies should also be judged by objective criteria.
One of the best free access databases is PubMed:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?DB=pubmed
Critical Appraisal: checklist
This is a guideline2 for the steps that should be followed in critically appraising the
quality and validity of published literature that uses health data to quantify or describe a
Kovats S et al. El Nino and health. Lancet, 2003, 361, 1481-1489
Modified from the University of Manchester resource site on health, environment and work:
http://www.agius.com/hew/resource/searchap.htm#peer
1
2
5
climate and health association. Remember that each paper is essentially unique - there
may be special aspects of methodology which may warrant focussed attention
 Title: Does this accurately reflect the main scope and nature of the work?
 Abstract: Is this a well structured, accurate and balanced summary of the work?
Does it distinguish between the results and the conclusions drawn?
Objectives
 Are these clearly stated in the introduction?
 How specific are they?
 Have they been met?
 Do they raise hypotheses or test hypotheses?
 What approach has been used? e.g. time series, cross-sectional, cohort, casecontrol.
 Was it appropriate and could it reasonably be expected to fulfil the objectives?
Health outcomes
 Was the study "population" clearly defined? Including the geographical area?
 Is the study population the most appropriate? i.e. is it based on administrative
boundaries rather than the most appropriate for the climate exposures?
 Is it representative of the group from which it is drawn?
 How were individuals in the study selected (sampling)?
 How was the sample size or geographical area chosen?
 What is the source of health data? Is a case definition clearly defined?
 Is there a comparison (control) group? This is particularly important for assessing
the impact of extreme events (floods) where the sample should include those who
were affected by the flood event, and those who were not.
 Clinical or laboratory diagnosis should be given greater credence than self-reported
diagnosis.
Methods Used
 How has the information been obtained?
 Have sources of data been clearly described?
 Have they been validated?
 Are they reproducible?
 Could they have been biased?
 Is quality control of collection of information mentioned?
 Are response rates (in health surveys) quoted? Could a poor response rate hide the
possibility of important bias?
Exposure
 How well was this 'speciated', (i.e. characterised as to its identity, and other
relevant co-exposures assessed) ?
 How was it measured?
 How well was it quantified?
 Was it studied in such a way as to explore a possible exposure-response
relationship?
 Is the observed meteorological data appropriate for the study population?
Statistical Methods
 Were they appropriate and necessary?
 Could chance have been responsible for the results?
 If a time series analysis – have results been adjusted for season/month and trend?
 Are associations quantified both with and without adjustments for spatial or
temporal autocorrelation?
 Are sensitivity analyses undertaken on the key assumptions (e.g. adjustment for
season)?
6
Results
 Do results include p-values or confidence intervals?
 Do results appear in enough detail to permit some checking for accuracy (between
the text, tables, figures etc)?
 Are they consistent?
 Are they detailed enough to justify the conclusions?
 If appropriate, are they consistent with an exposure-response relationship?
Bias and Other Distortions
 What are the most important sources? i.e. interviewer, observer, recall, selection,
response, etc.
Discussion and Conclusions
 Are the conclusions consistent with the reported results?
 Are they plausible? Is there a plausible biological explanation for association
between weather parameters and disease outcome?
 Are the limitations of the study addressed?
 Were the sources, direction and magnitude of bias adequately discussed?
 Have the confounders been adequately considered?
 Could other conclusions be drawn from the same results? (e.g. if they rely on
temporality alone).
 Has there been an adequate comparison with other relevant literature?
 How relevant were the study population and conditions of exposure to the
conclusions drawn?
 Are the implications (e.g. for policy) clearly stated and reasonable?
Key terms
Bias: Any trend in the collection analysis, interpretation, publication or review of data
that can lead to conclusions that are systematically different from the truth.
Confidence interval (CI): The computed interval with a given probability, e.g. 95%, that
the true values of a variable such as a mean, proportion, or rate is contained with
the interval.
Confounding: A situation in which a measure of the effect of an exposure on risk is
distorted because of the association between of exposure with the other factors(s)
that influence the outcome under study.
Ecological fallacy: The bias that may occur because an association observed between
variables on an aggregate level does not necessarily represent the association that
exists at an individual level.
Post hoc: (of a study hypothesis) formulation of hypothesis after making the observation
Publication bias: Bias due to the fact that studies are more likely to be published where
they report a positive association.
Based on Last (2001)
Recommended reading.
 Wilkinson P ed. Environmental Epidemiology. 2006. Open University Press, ISBN:
0335218423.
 Last JM. A Dictionary of Epidemiology. 2001. New York, Oxford University Press.
 Pearce N. A Short Introduction to Epidemiology. 2nd ed. Wellington, CPHR, 2005
7
Abstract A
Predicting high-risk years for malaria in Colombia using parameters of El Nino
Southern Oscillation.
Bouma MJ, Poveda G, Rojas W, Chavasse D, Quinones M, Cox J, Patz J. Trop Med Int
Health. 1997 Dec;2(12):1122-7.
The interannual variation in malaria cases in Colombia between 1960 and 1992 shows a
close association with a periodic climatic phenomenon known as El Nino Southern
Oscillation (ENSO).
Compared with other years, malaria cases increased by 17.3% during a Nino year and
by 35.1% in the post-Nino year. The annual total number of malaria cases is also
strongly correlated (r = 0.62, P < 0.001) with sea surface temperature (SST) anomalies
in the eastern equatorial Pacific, a principal parameter of ENSO.
The strong relation between malaria and ENSO in Colombia can be used to predict high
and low-risk years for malaria with sufficient time to mobilize resources to reduce the
impact of epidemics. In view of the current El Nino conditions, we anticipate an increase
in malaria cases in Colombia in 1998.
Further studies to elucidate the mechanisms which underlie the association are required.
As Colombia has a wide range of climatic conditions, regional studies relating climate and
vector ecology to malaria incidence may further improve an ENSO-based early warning
system. Predicting malaria risk associated with ENSO and related climate variables may
also serve as a short-term analogue for predicting longer-term effects posed by global
climate change.
8
Abstract B
Temperature and mortality in 11 cities of the eastern United States
Curriero FC, Heiner KS, Samet JM, Zeger SL, Strug L, Patz JA. Am J Epidemiol. 2002 Jan
1;155(1):80-7.
Episodes of extremely hot or cold temperatures are associated with increased mortality.
Time-series analyses show an association between temperature and mortality across a
range of less extreme temperatures. In this paper, the authors describe the
temperature-mortality association for 11 large eastern US cities in 1973-1994 by
estimating the relative risks of mortality using log-linear regression analysis for timeseries data and by exploring city characteristics associated with variations in this
temperature-mortality relation.
Current and recent days' temperatures were the weather components most strongly
predictive of mortality, and mortality risk generally decreased as temperature increased
from the coldest days to a certain threshold temperature, which varied by latitude,
above which mortality risk increased as temperature increased. The authors also found a
strong association of the temperature-mortality relation with latitude, with a greater
effect of colder temperatures on mortality risk in more-southern cities and of warmer
temperatures in more-northern cities. The percentage of households with air
conditioners in the south and heaters in the north, which serve as indicators of
socioeconomic status of the city population, also predicted weather-related mortality.
The model developed in this analysis is potentially useful for projecting the consequences
of climate-change scenarios and offering insights into susceptibility to the adverse
effects of weather.
9
Abstract C
Climate and the prevalence of symptoms of asthma, allergic rhinitis, and atopic
eczema in children
Weiland SK, Husing A, Strachan DP, Rzehak P, Pearce N; ISAAC Phase One Study Group.
Occup Environ Med. 2004 Jul;61(7):609-15.
AIMS: To investigate the association between climate and atopic diseases using
worldwide data from 146 centres of the International Study of Asthma and Allergies in
Childhood (ISAAC).
METHODS: Between 1992 and 1996, each centre studied random samples of children
aged 13-14 and 6-7 years (approx. 3000 per age group and centre) using standardised
written and video questionnaires on symptoms of asthma, allergic rhinoconjunctivitis,
and atopic eczema during the past 12 months. Data on long term climatic conditions in
the centres were abstracted from one standardised source, and mixed linear regression
models calculated to take the clustering of centres within countries into account.
RESULTS: In Western Europe (57 centres in 12 countries), the prevalence of asthma
symptoms, assessed by written questionnaire, increased by 2.7% (95% CI 1.0% to
4.5%) with an increase in the estimated annual mean of indoor relative humidity of
10%. Similar associations were seen for the video questionnaire and the younger age
group. Altitude and the annual variation of temperature and relative humidity outdoors
were negatively associated with asthma symptoms. The prevalence of eczema symptoms
correlated with latitude (positively) and mean annual outdoor temperature (negatively).
CONCLUSIONS: Results suggest that climate may affect the prevalence of asthma and
atopic eczema in children.
10