Chaos in climate change impact estimates
Abstract
Global Circulation Models incorporate chaotic dynamics to reflect real-world weather patterns. This implies that extremely small perturbations of the climate system may generate
very different weather patterns. Here I show that the SRES climate change scenarios generated
by the Coupled Model Intercomparison Project phase 3 (CMIP3) — ubiquitous in the impact
literature — display strong chaotic dynamics at regional and sub-regional level, at least until
2065. Chaos is triggered by changes to historic forcing in the year 2000 to reflect different
emissions trajectories. This suggests that large uncertainty exists on how to link local climate
change and global forcing. Furthermore, short- and mid-term differences in local climate change
across different SRES emission scenarios reflect chaotic dynamics rather than different forcing
patterns. I show that the “chaos” in the climate scenarios generates a “chaotic” relationship
between exogenous forcing and local economic impacts. “Perturbed exogenous forcing” model
ensemble would resolve this uncertainty.
1
1
Introduction
General Circulation Models (GCMs) are the workhorse of climatologists to study the climate system
and to assess the impact of past and future changes of Greenhouse Gases (GHGs) emissions and of
other forcing agents on climate. In a nutshell, GCMs are large-scale, extremely complex and computationally intensive, weather forecasting machines. Climatologists run GCMs using alternative
exogenous forcing patterns to study their effect on climate.
General Circulation Models are deterministic. This means that two runs of the same model,
with the same starting conditions and the same exogenous forcing, generate identical weather
patterns over time and space. However, GCMs embed the chaotic dynamics of weather. This
means that very small changes of initial conditions and of exogenous forcing translate into very
different predictions of weather patterns. This is the well-known “butterfly effect” in chaos theory. ?
In order to control for the effect of different initial conditions — which are highly uncertain
— climatologists run “ensembles” of scenarios. A “model ensemble” is a set of scenarios generated
by the same GCM with different initial conditions. If the number of runs is sufficiently large, the
“model ensemble” mean provides the expected characterization of climate for that GCM. There
is instead no control for the effect of small changes in external forcing. However, by introducing
differences in emission trajectories climatologists trigger chaotic dynamics in weather patterns.
Here I show the implications of omitting these controls by surveying the scenarios based on
SRES emission trajectories, prepared for the Coupled Model Intercomparison Project phase 3
(CMIP3). ? The CMIP3 scenarios were widely used by Working Group I (the science of climate
science) and by working Group II (impacts, adaptations and vulnerability) of the International
Panel on Climate Change (IPCC) for the Fourth Assessment Report of the IPCC. Most of the
impact estimates surveyed by the Fifth Assessment report also rely on CMIP3 scenarios, because
the new climate scenarios (CMIP5) are very recent. A careful analysis of the CMIP3 dataset reveals
that GCMs have a chaotic reaction to very small changes of GHGs and other forcing agents. The
perturbation introduced by climatologists when they substitute future to historical forcing are large,
persistent and widespread.
Figure 1 graphically conveys this message. For any exogenous forcing level M 0 , there exists a
2
=
| ′′,
′
′′
| ′,
′−
′
′+
′′ +
′′
′′ +
M
Figure 1: The within models uncertainty.
sufficiently small contour set [M 0 − δ, M 0 + δ ] around M 0 , with δ > 0, for which the relationship
between the climate variable x and the forcing level M is chaotic., i.e. very small variations of
M trigger large and apparently random variations of x. Experiments described below show that
a less than 1 percent change in global CO2 concentrations can generate variations of monthly
mean temperatures in excess of 11 ◦ C. This is not the result of bad science and should not be a
surprise. Chaos governs weather and GCMs faithfully reproduce weather dynamics. Remarkably,
however, the chaotic response of weather to small changes in exogenous forcing does not disappear
in long-term averages of weather in the climate projections for the A1B, A2 and B1 SRES scenarios.
Scenarios described below show that a small change in global CO2 concentrations can still generate
5 ◦ C deviations in 20 years averages of mean monthly temperature.
The sensitivity of local climate change scenarios to very small perturbations of exogenous
forcing calls for sound robustness tests. Suppose in fact that one wants to compare the impact of
two different levels of exogenous forcing, M 0 and M 00 on x. Figure 1 portrays the possibility that
x0 = g (M 0 ) > g (M 00 ) = x00 even if the distribution of the realizations of x in the two contour sets
clearly indicates that x is positively correlated to M . In this case one would rather have a robust
statistic of the level of x in the vicinity of any level of M . For example, the expected value of x,
given M and δ. Unfortunately, y = f (x|M, δ), the probability density function (pdf) of x given M
3
and δ, is unknown. This is source of what I call the “within models” uncertainty to distinguish it
from what I define the “between models” uncertainty, which is due to the impossibility of inferring
the pdf of climate change from the scenarios generated by different GCMs for the same exogenous
forcing. The “between models” uncertainty has been rarely addressed by the impacts literature. ?
The “within models” uncertainty has been outright neglected. Here I show that the “within models”
uncertainty is large and it significantly affects climate change impact estimates.
2
Chaos in climate change scenarios
In the CMIP3 exercise climatologists generated scenarios of weather from the 19th century to 2100
and beyond, using historical exogenous forcing until December 1999 and exogenous forcing as dictated by the SRES scenarios from January 2000. They also built model ensembles by repeating
each run with different initial conditions. Two SRES scenarios that share the same initial conditions have identical weather patterns until December 1999. Afterward, the weather patterns start
diverging because the forcing conditions are different.
How large are the deviations of weather patterns due to a change of exogenous forcing? All the
GCMs included in the analysis show that small variations of exogenous forcing have an immediate
and remarkable impact on weather patterns. Global mean temperature of the A1B, A2 and B1
SRES climate scenarios is the same until 1999, because they share the same historical forcing and
the same initial conditions. As exogenous forcing slightly departs after 1999, also global mean
temperatures depart along distinct trajectories, as documented in the Supplementary Material
(SM).
Figure 2 displays regional weather patterns generated by the HADCM3 model in January
and July 2000, when the exogenous forcing scenarios have just begun to diverge. The HADCM3
model of the Hadley Centre Climate Programme hosted by the UK Meteorological Office (UKMO)
is probably the more widely used model in the impacts literature and is used as an example
throughout this paper. ? The same findings apply to all other GCMs, as documented in the SM.
The scenarios reveal that minor perturbations of exogenous forcing have striking impacts on local
weather patterns in the very short-term. For example, average January temperature in the A1B
4
scenario, in Central Europe and in the Mid-West of the US is up to 10 ◦ C colder than in both the
A2 and the B1 scenarios. This is the “butterfly effect” at work in GCMs. The analysis of other
models, of other seasons and of precipitation patterns reveals similar differences, as documented in
the SM.
Are the short-term chaotic responses of weather to small variations of exogenous forcing absorbed as time goes by? Do they disappear if we consider averages of weather variables (i.e.
climatologies) over many years? Unfortunately, scenarios in which the same model is run with
small perturbations of the same exogenous forcing scenario (e.g. the “A2” HADCM3 ensemble)
are not available. However, for a fortunate coincidence, the patterns of GHGs used in the SRES
A1B, A2 and B1 scenarios are very similar until 2030 and they remain very similar until 2060 for
the A1B and A2 scenarios. In the SM I report the three SRES GHGs concentration pathways used
for the CMIP3 and I show that the differences are very small. For example CO2 concentrations in
the three SRES scenarios are virtually identical until 2020 and until 2060 for the A1B and the A2
scenarios. The difference between the A2 and the A1B scenarios from 2020 to 2050 is of 1.5 ppm
at most. The two scenarios diverge only in 2060, when the A2 has 8 ppm more (+1.4%) than the
A1B scenario. One should bear in mind that concentrations raise from about 370 ppm in 2000 to
856 ppm in 2100 in the A2 scenario and that CO2 is the most important among all GHGs. CO2 is
responsible for 63% of the radiative forcing from all long-lived gases. ?
The SRES runs thus provide a natural experiment against which I test my hypothesis. I
focus on twenty years climatologies of temperature and precipitation. I first consider climatologies
of global averages of annual mean temperature and precipitation (2011-2030 and 2046-2065). I
subsequently consider regional climatologies of monthly mean temperatures and precipitation.
We would expect the same GCM to generate scenarios that are very similar for years in which
the exogenous forcing is very similar. From the analysis of scenarios generated by 21 GCMs, I find
that this holds at global level for annual mean temperature and precipitation but it fails at regional
level for monthly temperature and precipitation.
The models disagree on the level of global mean temperature. This is source of “between
models” uncertainty. Temperature and precipitation levels are instead almost identical for two
SRES scenarios generated by the same model. If I drop an outlier, I find that the mean global
5
6
Figure 2: Differences between SRES scenarios surface air temperature in January and July 2000.
temperature range across SRES scenarios is equal to 0.08 ◦ C in 2011-2030 and 0.06 ◦ C in 20462065. I find even smaller differences in precipitation scenarios. This is comforting. It says that
chaotic dynamics do not affect climatologies of the global mean temperature. It also supports the
hypothesis that the forcing trajectories of the SRES scenarios are very similar during the time
frames under exam.
I now examine the “within models” uncertainty by focusing on scenarios generated by the
HADCM3 model. Figure 3 maps the difference between the A2 and the B1 and between the
A2 and the A1B HADCM3 temperature anomalies for 2011-2030. The striking finding is that
climatologies of monthly temperatures, at local level, display remarkable differences. For example,
the A2 January temperature anomaly is much higher than in the B1 scenario in Russia, and than
the A1B scenario is western Siberia. The A2 temperature anomaly is instead much lower than
both the A1B and B1 scenarios in the Northern America. The A2 scenario for the eastern part of
the US and of Canada is more than 3 ◦ C colder than the A1B scenario. At global level the mean
annual temperature difference between the A1B and A2 scenarios during 2011-2030 is only 0.12 ◦ C.
I find similarly large differences in other months, in 2046-2065 climatologies and in other models
(see additional maps and tables in the SM).
Extremely small perturbations of exogenous forcing may induce large deviations of weather
patterns at regional level and on sub-annual time resolution. These deviations do not vanish over
time. They persist and they affect temperature climatologies.
Figure 4 displays the precipitation anomaly for 2011-2030 climatologies in the three SRES
scenarios. Also in this case I find large differences among different SRES scenarios. For example,
the South East of Brazil has much higher rainfall in the A1B and the A2 scenarios than in the
B1 scenario. There are also large differences in precipitations along the East Coast of the US, in
Western Europe, in Southern Africa, in Northern Australia and in Southern Madagascar. This
is confirmed by the analysis of scenarios in other months, in the 2046-2065 climatologies and in
scenarios generated by other models (see additional maps and tables in the SM).
7
Notes: The temperature anomaly is calculated subtracting the temperature climatology between 2011-2030 from
the temperature climatology of 1961-1990. Temperature anomalies of the B1 and the A1B scenarios have been
subtracted from the temperature anomaly of the A2 scenario to provide a more clear comparison of the scenarios.
Figure 3: Air temperature anomaly, UKMO HADCM3 model, January 2011-2030.
8
Notes: Precipitation patterns can be very different at local level and a direct comparison of the scenarios using
maps of differences or of percentage change is often problematic.
Figure 4: Precipitation flux anomaly, UKMO HADCM3 mode, January 2011-2030.
9
3
Chaos in climate change impact estimates
The implications of the “within models” uncertainty for the impacts literature are matter of empirical investigation. Impact functions that are relatively inelastic, that cover large areas of the globe
and use climatologies of annual means of climatic variables should not be significantly affected (e.g.
impact studies of sea-level rise). Impact studies that have highly non-linear response functions
(e.g. threshold effects), that use daily or sub-daily temperature and precipitation observations or
inter-annual weather variation would instead be significantly affected (e.g. estimates of impacts on
yields from crop models).
Here, I provide an assessment of the “within models” uncertainty for estimates of climate change
impact on United States (US) land values using a “Ricardian” model ? . The model assumes that
land values reflect the long-term profitability of agricultural land. By regressing land values on
climate and other control variable it is possible to estimate how climate affects land values. The
estimated climate sensitivity of US agriculture is then used to predict the impact of climate change
on land values, at county, regional and national level.
The model uses a quadratic seasonal characterization of climate, for both temperatures and
precipitations. Since the relationship between climate and land values is assumed to be quadratic,
climate sensitivity varies across US counties. In order to control for time-varying factors I use a
pooled model based on six Agriculture Censuses, from 1978 to 2002. ?
?
Further information on the
model, on the variables and on regression results is available in the SM. Econometric estimates of
the models reveal that the seasonal coefficients are significantly different and that the relationship
between climatic variable and land values is typically quadratic in shape.
Figure 5 displays two panels, one for 2011-2030 and the other for 2046-2065. The first two rows
of each panel display maps of winter (December, January and February) and summer (June, July
and August) mean monthly temperature anomalies with respect to 1961-1990, for the A1B, A2 and
B1 scenarios of the HADCM3 model. The third row displays percentage impacts of temperature
and precipitation changes (in all four seasons) on land values across the US.
The figure reveals three striking findings. First, despite remarkable similarities between the
emission scenarios, local climate patterns are remarkably different over the US. In some cases the
10
(a) 2011-2030
Temperature anomaly – HADCM3 – DJF
A1B
A2
B1
Temperature anomaly – HADCM3 – JJA
A1B
A2
B1
Impacts
A1B
A2
B1
(b) 2046-2065
Temperature anomaly – DJF
A1B
A2
Temperature anomaly – JJA
A1B
A2
Impacts
A1B
A2
Figure 5: Temperature anomalies and impacts at county-level, 2011-2030 in panel (a) and 2046-2065
in panel b.
11
differences are in excess of 3 ◦ C. Second, the large differences across scenarios suggest that chaotic
dynamics are persistent but as the signal becomes stronger, they become less relevant. Third, the
chaotic response of the climate system to exogenous changes in forcing agents is reflected in the
distribution and the size of impact estimates.
The first two panels of Figure 6 report aggregate impact estimates in 2011-2030 and in 20462065 with 95% bootstrap confidence intervals, for all GCMs used in this study. The two panels carry
three important messages. First, different models generate very different impact estimates even if
they use the same emissions scenario. This is what I call the “between models” uncertainty. Second
and most importantly, the “within models” uncertainty is strong. Impact estimates generated using
climate scenarios produced by the same GCM, for virtually identical levels of exogenous forcing, are
dramatically different. Third, while in 2011-2030 the “within models” uncertainty is stronger than
the “between models” uncertainty, the opposite is true in 2046-2065. As the signal gets stronger,
the intrinsic differences across models matter more than chaotic weather dynamics within each
model.
The bottom panel of Figure 6 plots percentage changes of aggregate US land values and the
level of CO2 concentrations for three representative GCMs. The three scatter plots reveal very
different responses of aggregate impacts to increasing CO2 concentrations. They also reveal that
for very similar levels of CO2 concentrations, differences of impact estimates for the same model
can be very large.
4
Discussion
The three scatter plots in panel c of Figure 6 provide a striking example of the “within models”
uncertainty (see Figure 1). For any level of exogenous forcing researchers and policy makers are
left with ambiguous results and cannot easily separate “noise” from the true signal. The size and
the effect of the “within models” uncertainty is a matter of empirical investigation. Estimates of
climate change impacts on US agricultural land values reveal that the ambiguity that arises from
“within models” uncertainty is large and should not be disregarded by the impact community.
Policy makers should be aware that impact estimates in the short- and medium-term are ambiguous
12
(a) Percentage impact on United States agricultural land values – 2011-2030
(b) Percentage impact on United States agricultural land values – 2046-2065
(c) The relationship between CO2 concentration and impacts on United States agricultural land values
Notes: The bars indicate 95% bootstrap confidence intervals. Scenarios from all GCMs that have run the A1B, A2
and B1 SRES emissions scenarios for the CMIP3. The vertical axis of the figure in panel b is cut to preserve
readibility.
Figure 6: The “between” and “within” models uncertainty reflected in impact estimates.
13
because different models predict different climate scenarios and because the same models embed
noise. This suggests that investing in long-run adaptation in the short- and medium-term may
have uncertain outcomes. This has also implications for Integrated Assessment Models that use
impact functions with a high geographic resolution. Thus, the “within models” uncertainty has
also implications for cost-benefit studies of climate policy.
The analysis that I presented here can be replicated using the recent set of scenarios generated
by the CMIP5 using the new Representative Concentration Pathways (RCPs), if sufficient similarities between the RCPs exist. ? The effect of the “within” models uncertainty on impact estimates
can be studied using other world regions, other sectors and other methods.
Climatologists can reduce the “within models” uncertainty by running “perturbed exogenous
forcing scenarios”. This is costly because GCMs are so big that it takes weeks if not months to have
a complete scenario on the world’s fastest computers. However, at least in theory, it is possible to
create ensembles of scenarios driven by very similar exogenous forcing trajectories to estimate the
distribution of climate change in a small contour set of the central exogenous forcing trajectory.
This would resolve the “within models” uncertainty.
14
References
[1] Lorenz, Edward N. The Essence of Chaos. University of Washington Press, (1995).
[2] Solomon, S., D. et al. Technical Summary. In S. Solomon et al. eds., Climate Change 2007:
The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report
of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge,
United Kingdom and New York (2007).
[3] Nakićenović, N. et al. Special Report on Emissions Scenarios : a special report of Working
Group III of the Intergovernmental Panel on Climate Change Cambridge University Press,
Cambridge, United Kingdom and New York (2000).
[4] Auffhammer, M., S. Hsiang, W. Schlenker and A. Sobel. Using weather data and climate model
output in economic analyses of climate change Review of Environmental Economics and Policy
7(2), 181–198 (2013).
[5] Mendelsohn, R. and W.D. Nordhaus, D. Shaw. The impact of global warming on agriculture:
a ricardian analysis. Am. Econ. Rev. 84(4), 753–771 (1994).
[6] Massetti, E. and R. Mendelsohn. Estimating Ricardian functions with panel data. Climate
Change Economics 2(4), 301–319 (2011).
[7] Massetti, E. and R. Mendelsohn. The impact of climate change on US agriculture: a repeated
cross-sectional Ricardian analysis. In A. Dinar and R. Mendelsohn, eds. Handbook on Climate
Change and Agriculture. Edward Elgar Publishing (2012).
[8] van Vuuren, D.P., J. Edmonds, M. Kainuma, K. Riahi, A. Thomson, K. Hibbard, G.C. Hurtt,
T. Kram, V. Krey, J-F Lamarque, T. Masui, M. Meinshausen, N. Nakicenovic, S.J. Smith and
S.K. Rose. The representative concentration pathways: an overview Clim. Change 109(1-2),
5–31 (2011).
15