WORLD METEOROLOGICAL ORGANIZATION
____________________
GPC-SIF/Doc. 5.5
(18.II.2003)
_________
COMMISSION FOR BASIC SYSTEMS
WORKSHOP OF GLOBAL PRODUCERS OF
SEASONAL TO INTERANNUAL FORECASTS
ITEM: 5.5
Original: ENGLISH
Geneva, 10-13 February 2003
The IRI's Climate Forecast System; Future Outlook for System Advances
(Submitted by Mr Anthony G. Barnston)
GPC-SIF/Doc. 5.5
The IRI's Climate Forecast System; Future Outlook for System Advances
Overview of Current State of Affairs
The International Research Institute for Climate Prediction (IRI) makes forecasts near the middle of
each month for the global temperature and precipitation for four overlapping future 3-month
periods extending to 6.5 months in advance. The forecasts are based mainly on ensembles of
predictions from each of five dynamical atmospheric general circulation models (AGCMs) whose
behaviors are governed largely by predicted global sea surface temperature (SST) anomalies. The
forecast procedure begins with SST anomaly prediction. Two versions of SST forecast are used: (1)
an evolving SST prediction based on a dynamical tool for tropical Pacific and statistical tools for
the other ocean regions, and (2) an SST forecast based on SST anomalies persisted from the most
recently observed month. These SST predictions are then used, in a second tier of the process, as a
basis for prediction of the climate over land. The historical skill of the atmospheric models in
hindcast mode, and of real-time forecasts over 4 recent years, are known. The hindcast skills using
perfectly known SST forcing are used to form the weights of the five models for real-time forecasts
in a multi-model ensemble setting. In addition to the dynamical tools, simple empirical tools are
also sometimes used, such as composites of climate anomalies based on years having ENSO
conditions similar to what is currently expected for a given season. Final forecasts are expressed as
probabilities of tercile categories (below, near, and above normal) whose definitions are based on
observed data over a recent 30-year period.
More Detail on the SST Prediction
The dynamical model given the most weight for tropical Pacific SST predictions is the coupled
ocean-atmosphere model of the National Centers for Environmental Prediction (NCEP). The NCEP
forecasts cover the area from 30N to 25S, and 70W to 120E. Predictions from other dynamical
models, such as the COLA model, the simpler Lamont-Doherty model, and the model run at the
European Center for Medium Range Weather Forecasts (ECMWF), are also viewed, but have been
given little weight. Plans are being designed to allow these additional models to contribute to a
multi-model SST forecast that would be weighted by the relative expected accuracies of the input
models. Forecasts of the tropical Atlantic SST are made using the statistical canonical correlation
analysis (CCA) by CPTEC/INPE in Brazil, using the tropical Atlantic and Pacific SST fields as
predictors. Similarly, forecasts of the Indian Ocean are presently done at the IRI using a CCA,
using as predictors the most recent 1-month mean observed Indo-Pacific SST anomalies, and the
forecasts of the Pacific SST field. For all extratropical latitudes, the SST anomalies of the most
recent month are damped toward climatology with an e-folding time of 3 months
More Detail on the Global Climate Predictions
As of mid-2003, five models are run for each forecast: (1) The MRF9 from the NCEP in
Washington, DC, U.S.A.; (2) the ECHAM4.5 from Max Planck Institute in Hamburg, Germany; (3)
the CCM3.2 from the National Center for Atmospheric Research (NCAR) in Boulder, Colorado,
U.S.A.; (4) the NSIPP model run at NASA/Goddard Space Flight Center (GSFC) in Greenbelt,
Maryland, U.S.A., and (5) the COLA2 model run at the Center for Ocean-Land-Atmosphere
Studies (COLA) in Calverton, Maryland, U.S.A. The NCEP model is presently run in Queensland,
Australia; the ECHAM4.5 and CCM3.2 models are run by IRI personnel at the IRI and at Scripps
Institution of Oceanography, respectively, the NSIPP at GSFC, and the COLA2 model at COLA.
Most of these models are run at T42 spatial resolution, and using about 18 vertical levels. A
minority of the models are run at T63 resolution and converted to T42 resolution at IRI. Use of
models from other modeling centers is being considered. Examples are CPTEC’s AGCM and
GPC-SIF/Doc. 5.5, p.2
CPC/NCEP’s current model (not the same as the MRF9). Ensembles of 10 runs or more are
produced from each of the currently used models, where the ensemble members are exposed to the
same predicted (or persisted) SST but initialized with differing atmospheric initial conditions. The
atmospheric initial conditions are not those observed, but are restart files that are updated as new
observed SSTs become available in real-time. Because the first period being forecast starts about
two weeks following the time of the forecast, lack of use of real initial atmospheric conditions is not
believed to appreciably affect forecast skill on average. In total, there are thus at least 50 runs from
the original five models, and 70 for the first forecast target period if the persisted SST runs are
included also. (The persisted SST scenario is run only for the ECHAM4.5 and CCM3.2 models—
the ones that are run by IRI staff.) The flow of the IRI’s forecast system is shown in Fig. 1.
IRI DYNAMICAL CLIMATE FORECAST SYSTEM
HISTORICAL DATA
Extended simulations
PERSISTED
GLOBAL
SST
Observations
GLOBAL
ATMOSPHERIC
MODELS
24
10
ECHAM4.5(MPI)
FORECAST SST
CCM3.2(NCAR)
TROP. PACIFIC
(NCEP dynamical)
NCEP(MRF9)
TROP. ATL, INDIAN
(statistical)
NSIPP(NASA)
EXTRATROPICAL
(damped persistence)
Persisted
SST
Ensembles
3 Mo. lead
COLA2.x
POST
PROCESSING
-Statistics
24
Forecast
10
SST
10 Ensembles
10 3/6 Mo. lead
10
AGCM INITIAL CONDITIONS
-Multimodel
Ensembling
-graphics
REGIONAL
MODELS
UPDATED ENSEMBLES (10+)
WITH OBSERVED SST
Figure 1. Schematic diagram showing the IRI's 2-tiered seasonal forecast operational system. The
numbers on the arrowheads indicate the actual numbers of runs of each model contributing to the
multi-model ensemble.
Format of Forecast Output
The forecasts are issued on a 2.5 x 2.5 degree latitude/longitude grid for precipitation, and on a 2 x
2 degree grid for temperature. These grids are used because the data sets used for verification are on
compatible grids. Forecasts are only issued over land areas, including small islands if they produce
observations for verification. Precipitation forecasts are issued only over regions whose
climatological precipitation exceeds 30 mm for the 3-month period in question. Probabilities of the
three terciles are given in multiples of 5% (e.g. 40%, 35%, 25% for below, near, and above normal
categories), except for the climatology forecast which is labeled as 33%, 33%, 33%.
Improving the Estimate of the Expected Skill of the Atmospheric GCMs
GPC-SIF/Doc. 5.5, p.3
An ongoing effort is underway to gain more realistic estimates of model skill in retrospective
hindcasts, in which observed SST information is not permitted once the model run has begun. So
far, long hindcast runs have been conducted for ECHAM4.5 using SST anomalies persisted from
the observed SST the previous month. Skills indicated from these hindcasts are generally somewhat
lower than those from “perfect” observed SSTs.
Skill of the IRI's Real-Time Precipitation Forecasts since Late 1997
The IRI has been issuing real-time forecasts for only about 5 years since their first set of forecasts
for October-November-December 1997
For verification, the precipitation observations used are those of the CPC Merged Analysis of
Precipitation (CMAP) on a 2.5 degree grid (Xie and Arkin 1996). Temperature forecasts are
verified using the New data from University of East Anglia (New et al. 2000), which are converted
from a 1.0 degree grid to a 2.0 degree grid.
Because the IRI’s forecasts are issued in a probabilistic format, the verification is done using a
probabilistic measure of skill. The verification measured used for the probability forecast is the
ranked probability skill score (Epstein 1969; Wilks 1995), defined as:
3
RPS =
 ( Fi  Oi) 2
i 1
where Fi is the cumulative forecast probability up to category i, and Oi is the cumulative
observation up to category i. The rank probability skill score, or RPSS, is then calculated using
climatology forecasts as the reference for comparison:
RPSS = 1 - (RPSf / RPSc)
where RPSf is the RPS for the forecast, and RPSc is the RPS of the climatology forecast. It should
be noted that the RPSS appears to be a harsh skill score. RPSS skills of 0.10 are regarded as
respectable in the business of seasonal precipitation forecasting, and skills of 0.20 as very good.
The RPSS skill for the IRI’s precipitation forecasts for the first season over the globe during the
1997-2001 period has been compared with the skill for some individual constituent forecasts tools:
(1) probability forecasts in accordance with historical probabilities associated with the ENSO state,
(2) the climate category observed in the previous 3-month period is used as the forecast, with a
100% probability given to the given category and 0% for the other two categories, and (3)
probabilities from the error-corrected ensemble distribution of the ECHAM3.6 model which was
the best individual AGCM used during the 1997-2001 period. Of the four sets of forecasts, the IRI’s
final forecast had the highest overall skill, a skill that still, however, is modest in many locations.
During specific regions and seasons, however, such as those known to be under ENSO influence,
skill was respectable. Examples are Jan-Feb-Mar for the southern U.S., southern Africa and
Indonesia, Apr-May-Jun in northeast Brazil, Jul-Aug-Sep in the Sahel and parts of India, and OctNov-Dec in southeastern South America, eastern equatorial Africa and Indonesia/Philippines. These
real-time skill results will soon be published in Goddard et al. (2003).
GPC-SIF/Doc. 5.5, p.4
Future Goals
The IRI is working toward a more robust system of forecasting tropical SST for the forecast
periods. This system will use several dynamical models as input, and will probably generate several
SST forecast scenarios from these (not literally those of the individual input SST forecasts). Use of
a mixed layer model for forecasting outside of the tropical Pacific is a possibility.
Providing access to individual model forecasts to other global climate forecast producers is
progressing at a good rate. Currently the IRI’s data library contains historical simulations of the five
AGCMs, and access to real-time forecasts is currently permitted on a password basis. Real-time
forecasts can be viewed on the web. A plan is to remove password protection; we are working out
how to field users’ inquiries.
IRI is increasing the use of post-processing of model forecast output using multivariate statistical
corrections based on CCA on a region-by-region basis. This has already been shown to increase
skill appreciably for many models, regions and seasons. A wholesale correction process would be
desirable. Some corrections use just the local region of model forecast as the predictor, while others
reach to the model’s prediction in regions more remote from the area being corrected.
A newly initiated probabilistic ENSO forecast is now posted monthly by the IRI out to 8 months in
advance. A goal is to increase the objective component of this forecast—i.e. rely more confidently
on a selected set of objective dynamical and statistical prediction tools.
References
Epstein, E. S. (1969). A scoring system for probability forecasts of ranked categories. J. Appl.
Meteor., 8, 985-987.
Goddard, L., A. G. Barnston and S. J. Mason, 2003: Evaluation of the IRI’s “Net Assessment”
Seasonal Climate Forecasts: 1997-2001. Bull. Am. Meteor. Soc., 84, accepted (to appear in middle
to late 2003).
New, M., M. Hulme and P.D. Jones, 2000: Representing twentieth-century space-time climate
variability. Part II: Development of a 1901-96 monthly grid of terrestrial surface climate. J.
Climate, 13, 2217-2238.
Wilks, D. S., 1995. Statistical methods in the atmospheric sciences. International Geophysics
Series, Vol. 59, Academic Press, San Diego, 464 pp.
Xie, P. and P. A. Arkin (1996). Analyses of Global Monthly Precipitation Using Gauge
Observations, Satellite Estimates, and Numerical Model Predictions. J. Climate, 9, 840 -858.