1
Overcoming Chaotic Behavior of Climate Models
2
3
4
S. Fred Singer
5
6
7
8
9
10
11
12
University of Virginia/ Science & Environmental Policy Project, Arlington, VA 22202 singer@sepp.org
Abstract: A fundamental issue in climate science is Attribution – determining the relative importance of
human and natural causes. The task generally involves comparing temperature trends from observations
and from greenhouse (GH) models. However, a problem arises from the chaotic uncertainties inherent in
the (non-linear) model calculations. Modelers try to overcome this problem by forming an “ensemblemean” of a number of “simulations” (runs) of the same model. Here we conduct a synthetic experiment,
and use two distinct procedures to demonstrate that no fewer than about 20 runs (of 20-yr length of an
IPCC Ggeneral-Ccirculation Mmodel) are needed to place useful constraints upon chaos-induced model
uncertainties
13
14
Introduction:
15
16
17
18
19
20
21
22
23
As Lorenz (1963, 1975: Zichichi, 2007, and also Giorgi, 2005) have demonstrated, climate models, using
non-linear partial differential equations, generate results highly sensitive to the initial conditions. To
reproduce an identical result in successive “simulations” (runs), the parameters describing the model’s
initial state must be given to a precision that is unattainable in practice. The Intergovernmental Panel on
Climate Change (IPCC-TAR; Meehl et al., 2001) acknowledges that, mathematically speaking, the
climate is a “complex, non-linear, chaotic object” and that, therefore, “the long-term prediction of future
climate states is not possible.” Accordingly, any comparison of modeled with observed temperature
trends cannot be done satisfactorily without understanding the chaotic behavior of a climate model.
24
25
26
27
28
One consequence is that successive runs of the same climate model can yield very different values for
warming trends. These trends may vary by an order of magnitude or more, and even their sign may vary.
For example, the Japanese MRI model carried out five runs for IPCC [Santer et al., 2008]: the individual
trends range from 0.042 to 0.371 K/decade [Fig. 1] -- and this error interval (‘spread’) would have been
even greater if more runs had been performed.
29
30
31
32
33
34
35
36
37
Fig. 1: Illustrating the chaotic nature of model trends, using the results of 5 runs (sometimes referred to
as “realizations” or “simulations”) from 1979 to 1999 of a particular GCM (from Japan’s MRI), as
presented in figure 1 of Santer et al 2008. The OLS trends of the five runs range from +0.042 to +0.371
(K per decade). The range of trends would likely to be even larger if more runs had been carried out.
None of the five trends, nor the ‘ensemble-mean’ shown, represents the ‘true’ model trend. As discussed
in the text, one needs to show that the cumulative ensemble-mean approaches an asymptotic value as
the number of runs increases.
38
39
40
41
Modelers, therefore, carry out several runs (“simulations”) and then publish the ‘model-ensemble-mean,’
E, the arithmetic average of the individual trend values generated by the several runs. Only rarely do we
learn the results of the individual simulations that are components in E. Yet how do we know that, say,
five runs are sufficient to produce a reliable EM to compare with an observed trend?
42
43
44
45
The present paper addresses a single but crucial aspect of the impact of chaoticity on the performance of
general-circulation models: the strong dependence of the error-bars in temperature-trend projections on
the number of simulations that are run on a particular model. We suggest that it may be possible to
overcome the “chaoticity barrier” by performing a sufficient number of runs.
46
47
Method:
48
49
50
51
52
The objective of this enquiry was to establish how many simulation runs of a GCM, at minimum, are
necessary to provide reasonable constraints on the value of E. For this investigation, it would have been
desirable to use climate models which had each been run at least 20 times. However, financial and time
constraints on modelers mean there are no ready examples of such multiple runs. Therefore, we
developed a synthetic approach to the problem.
53
54
55
A single, unforced (‘constant forcing’) control run of 1000 years’ duration was obtained from the
Program for Climate Model Diagnosis and Intercomparison (PCMDI) at Lawrence Livermore National
Laboratory.
Temperatures of Unforced Model
56
57
58
Fig. 2: Temperature values of an unforced 1000-year climate model control run. Source: PCMDI.
59
60
61
62
63
64
65
First, the temperatures were plotted against time in years [Fig. 2] to check for inevitable drift from what
should be a straight, zero-slope, horizontal line. Next, the temperature series was divided into 25
segments, each of 40 years’ duration (and also into 50 segments of 20-year runs). For each segment,
trend values T1 … T25 were determined. This procedure is analogous to, and (considering the chaoticity
of the climate object) equivalent to, 25 separate runs of a GCM over a single 40-year time-interval.
Another advantage of using an unforced model is that the true trend is known in advance – namely, zero
(except for drift effects).
66
67
68
69
70
71
72
73
First procedure: the cumulative ensemble mean Ecum. In the first of two procedures, a cumulative
ensemble mean Ecum was derived by adding the trend value of an additional run to the previous values,
Eq. (1), so as to determine a new value of E. Finally, the cumulative trend was plotted as a function of n,
the number of trend values used [Fig. 3]:
n
EMcum = 1/n Σ Ti
i=1
,
(1)
74
75
76
77
It was then possible to observe where this cumulative ensemble mean, Ecum, approaches an asymptotic
value that may be termed the ‘true’ trend. Results [Fig. 3] indicate that about 10 runs of the model seem
to be sufficient for 40-yr runs (and 20 runs of 20-yr length).
78
79
80

81

82

83

84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
Fig. 3: Procedure #1: Cumulative ensemble-means of trend values as a function of n, the number of runs
(of length 20 or 40 years). The cumulative ensemble mean, Ecum, is seen to reach an asymptotic value
close to zero as the number of runs exceeds about 20 (for a run-length of 20 years) – and about 10 (for a
run-length of 40 years). In the absence of model drift, the asymptotic value would presumably be zero.
Control experiments for the first procedure
100
101
102
103
104

To investigate the influence of drift, which is seen to exist in the 1000-yr model run [Fig. 2], we
have also carried out the same procedure for two additional time periods (Years 200-1000 and
400-1000), for which the drift appears to be more uniform – or at least does not change its sign.
For 40-yr runs, the asymptote of the cumulative ensemble mean Ecum is reached again after at
least 10 runs.
105
106
107
108
109

In addition to the 25 time-segments, each of 40 years, starting in years 1, 41, 81, … , a further 24
trend values may be determined by starting the 40-year time-segments in years 11, 51, 91, … ,
with still further trends determinable by starting in years 21, 61, 101, … , and at 31, 71, 111, …,
for a total of 97 trend values. All of these were found on examination to behave in the same way.
Lagged auto-correlation does not seem to be significant.
110
111

We also checked against a possible influence of contiguity by selecting non-contiguous segments
to form cumulative ensemble-means.
112
113
114

Finally, by starting in each year on the interval [1, 961], it is possible to obtain 961 (partly
overlapping) segments of 40-year length. When all 961 trend values are plotted, they are found to
form a Gaussian distribution.
115
116
Second procedure: constraining the error-interval
117
118
119
120
In the second of two procedures, the interval on which the values of E fall (‘spread’), was investigated for
(assumed) three synthetic models as a function of the number of runs. The result is shown in Fig.4, with
the spread plotted against n, the number of simulation runs. For a run length of 40 years, the trend interval
is seen to approach zero for n > 10 (Fig. 4)
3 models
6
4
2
0
-2 0
5
10
15
-4
-6
-8
-10
-12
121
122
123
124
125
126
127
Fig. 4: Procedure #2: Error interval (“spread” = Tmax - Tmin) of ensemble-means of trend values of 3
(synthetic) models as a function of n, the number of runs. (All trend values shown on the y-axis were
multiplied by 1000.) For a run length of 40 years, the spread is seen to collapse towards zero as the
number of runs exceeds 10. Similar results are obtained for the cases of 4 models and 5 models. The
dashed lines result from a different method of selecting trend values (see text).
128
Details for the second procedure
129

The time-series is truncated by removing years 1-200, to minimize possible effects of drift.
130
131

The remaining 800 years were divided into 40 segments of length 20 years, whereupon 40 trend
values T1 … T40 were determined.
132
133

We assume we have 3 models, each with the same number of runs n. We therefore assign n trend
values, arbitrarily selected from the 40 values of T, to each of the three models.
134

For n=1, the error interval among the 3 trend values is simply (Tmax – Tmin).
135
136
137

For n=2, a series of ensemble-mean trend values was constructed thus: T΄1 = ½(T1+T2); T΄2 =
½(T3+T4); T΄3 = ½(T5+T6). The trend interval among the 3 values was then determined as (T΄max–
T΄min).
138

The procedure was repeated for n = 4, 5, 8, 10, and 13, respectively. Results are plotted in Fig. 4.
139
140

To test sensitivity, the procedure was next repeated for an assumed 4 models (permitting n-values
up to 10), and then for 5 models. Similar results were obtained.
141
142
143

As a further check, we repeated all procedures by starting the selection of T-values with T40
(instead of T1) and proceeding backwards. Those max and min trends are indicated by dashed
lines in Fig. 4.
144
145
Discussion:
146
147
We suggest here that it may be possible to overcome the “chaoticity barrier” of climate models identified
by IPCC in the 20CEN intercomparison.
148
149
150
We have demonstrated that the ensemble-mean (E) trend obtained from a multi-run model is more
reliable than the trend obtained from a model that is run only once. In general, ten or more 40-yr runs
may be necessary to form a reliable E.
151
152
153
154
155
156
Sensitivity Analysis of Segment Length: Our initial choice of a segment length of 40 years was
arbitrary. We now investigate the effects of segment length on the convergence of the ensemble mean
value. We find convergence after 5 runs for model runs of length 80 years and after 20 runs (see Fig. 3)
for a segment length of 20 years (which is typical of the models in the IPCC compilation; see Fig. 1).
Empirically, it appears that convergence is achieved in 400 run-years – i.e., (20 x 20), (10 x 40), and (5 x
80). We have not discovered a theoretical explanation for this useful result.
157
158
159
160
161
162
Discussion: Few modelers have the resources to carry out ten or more runs of a particular generalcirculation model. They frequently report a temperature trend based on only a single run. For example,
the IPCC’s compilation of 22 ‘20CEN’ models has five models with just one run, five with 2 runs, and
only seven with 4 or more runs. The run lengths varied between 20 and 24 years. Modeler should be
encouraged to report not only the details of the forcings and parameterizations used in their particular
models but also the results of each run and its length (in years).
163
164
165
166
Most investigators when considering a group of models compound the problem by simply using the
average of the ensemble-mean trends of the group to compare with an observed trend. This procedure,
however, is defective in that it gives equal weight to single-run models and multi-run models, and leads to
greatly enhanced uncertainty of modeled trends. Yet we demonstrate here that the ensemble-mean trend
167
168
obtained from a multi-run model is more reliable than the trend obtained from a model that is run only
once.
169
170
171
172
173
174
175
176
The “spread” in trend values trend values discussed in Procedure #2 is akin to the “range’ of extreme
values of a Gaussian distribution. But “range” is an improper statistical metric.; it inceases as the number
of independent data points increases –while the Standard Deviation of their distribution decreases. For
example, it can be shown that the wide model uncertainty displayed as a grey region in Fig. 6 of Santer et
al. [2008] and labeled as a “2-sigma Standard Deviation” is actually an artifact, caused by the presence of
five single-run models in the IPCC compilation of models. A compilation comprising only multi-run
models would, therefore, help to constrain chaotic uncertainty, and would provide a more reliable means
of comparing the consistency of modeled with observed trends.
177
178
Acknowledgements
179
180
181
We are grateful to Roger Cohen, Curtis Covey, Robert Levine, Craig Loehle, Christopher Monckton, and
Ronald Stouffer for useful discussion and to Garrett Harmon and Will McBride for technical assistance in
preparing the figures.
182
References
183
184
185
186
187
188
189
190
191
192
193
Giorgi, F. 2005. Climate Change Prediction. Climatic Change 73: 239-265: DOI: 10.1007/s10584-0056857-4
194
195
196
197
198
199
200
201
Santer, B.D., Thorne, P.W., Haimberger, L., Taylor, K.E., Wigley, T.M.L., Lanzante, J.R., Solomon, S.,
Free, M., Gleckler, P.J., Jones, P.D., Karl, T.R., Klein, S.A., Mears, C., Nychka, D., Schmidt, G.A.,
Sherwood, S.C., Wentz, F.J. 2008. Consistency of modeled and observed temperature trends in the
tropical troposphere. Int. J. Climatol.: doi:1002/joc.1756.
Lorenz, Edward N. 1963. Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20: 130141.
Lorenz, E. N. 1975. The physical bases of climate and climate modeling, in Climate predictability, #16 in
GARP Publication Series, pp. 132-136, World Meteorological Organization.
Meehl, G., et al. 2001. Climate Change: The Scientific Basis, in Fourth Assessment Report of the
Intergovernmental Panel on Climate Change, Cambridge University Press, London.
Zichichi, A. Meteorology and climate: Problems and expectations. Pontifical Council for Justice and
Peace, The Vatican, 26-27 April 2007
********************************************************