Appendix I Metrics for Validating Climate Models’ Performance
on Intraseasonal-to-Interannual Climate Variability
1. Principles for designing the metrics
 Selecting verifiable, objective, quantitative measures that are useful for tracking
model development a model differences.
 These measures depict essential features of the mean states and variability
with specific focus on the most important phenomena.
 Geographical focus: the global tropics with regional emphasis in the IndianPacific region.
2. Variables
(Category 1):
SST, Precipitation (Pr), surface (2 m) air temperature (Ts), surface winds, OLR,
winds at 850 and 200 hPa (V(u,v) 850, V (u,v) 200), 500 hPa Omega (O500),
Geopotential at 850 (H850), 200 (H 200), and 500 (H500) hPa.
(Category 2)
Net surface heat fluxes (regional models), Vertical profile of heating (Yanai)
3. Objective statistical measures
1-D time series (area averaged quantities):
Temporal mean, SD, correlation coefficient, RMSE.
2-D maps (Spatial distribution)
Spatial averaged standard deviation (SD)
181
Anomaly Pattern Correlation Coefficients (PCC),
Root Mean Square Error (RMSE).
Time-space (Hovemoller) diagram: PCC, RMSE
4. Domains
Global tropics (GT): 30S-30N
Equatorial regions (ER): 5S-5N.
ENSO: 30S-30N, 160E-80W
Nino 3: 5S-5N, 150W-90W
Nino 3.4: 5S-5N, 170W-120W
AAM: 30S-30N, 40E-160E.
Monsoon sub-domain:
Indian or South Asian (SA): 5N-30N, 60E-105E
Western North Pacific (WNP): 5N-20N, 105E-160E
Australian (AUS): 20S-5S, 105E-160E
East Asian (EA): 20-45N, 105E-145E
Global Tropics sub-domain:
Africa (AF): 0-50E, 30S-30N
Indian Ocean (IO): 50-110E, 30S-30N
Western Pacific (WP): 110-180E, 30S-30N
Eastern Pacific (EP): 180-280E, 30S-30N
Atlantic Ocean (AO): 280-3600E, 30S-30N
182
5. Definition of climatological mean and variability
Mean state
Defined by a 20- or 30-year mean climatology including annual and seasonal
means.
Mean annual cycle (AC):
(1) Defined by long-term monthly mean (for 2-D maps) or pentad data (for time
series)
(2) Defined by the first two EOF modes of climatological monthly mean
precipitation. The first mode is solstice global monsoon mode (well
represented by difference between JJA and DJF) and the second mode is
equinox asymmetric mode (well represented by difference between AM and
ON)
Smoothed AC:
Defined by the Fourier harmonics 1 through 3 (annual, semiannual, and period
of one-third of the year)
Interannual variability:
Obtained using seasonal mean minus the corresponding AC (seasonal mean).
Intraseasonal variability:
Daily or pentad mean anomalies minus smoothed annual cycle.
6. Long-term mean state
2-D maps of annual mean and JJA & DJF mean: SST, Pr, V 850, V200, OLR, Ts, 500
omega
183
7. Climatological annual cycle
Time-longitude diagram of Equatorial SST, 20oC isotherm depth, surface zonal wind
stress (5S-5N)
Annual cycle (AC) of regional mean precipitation (pentad time series) for AAM subdomains: ISM (7.5N-27.5N, 60-105E), WNP (7.5N-22.5N, 105-145E) (Wang et al.
2004),
8. Interannual variability (IAV)
2-D maps of the seasonal mean anomalies of Pr, OLR, SST, Ts: Compute

Anomaly pattern correlation coefficient (PCC) for each year and its temporal
mean

Spatial mean SD for each year (time series) and its temporal mean

RMSE for each year and temporal mean
8.1 Dominant Mode of A-AM Variability
Season-Reliant EOF (SEOF) to determine the major modes of variability for
precipitation
Physical consideration: anomalous climate (ENSO, monsoon) is regulated by the
seasonal march of the solar radiation forcing (annual cycle). SEOF analysis
detects seasonal evolving major modes of climate variability.
Method: construct a covariance matrix that consists of a sequence of seasonal
anomalies within a “Monsoon year” (Meehl 1987, Yasunari 1991)
8.2 AAM indices
184
Precipitation indices:

ISM, WNP, AUS, EA (regional mean precipitation)

All Indian Summer Rainfall (Pathasarathy and Carpenter, 1992)
Circulation indices

Broad scale Asian summer monsoon (Westerly vertical shear):
u200 (0-20N, 40-110E) minus u850(0-20N, 40-110E) (Webster-Yang, 1992)

ISM:
Southerly vertical shear: v850(10N-30N) minus v200 (10N-30N) (Goswami et
al. 1999);
Westerly meridional shear: u850 (5N-15N, 40E-80E)-u850 (20N-30N, 60E90E) (Wang et al. 2001);
Cross-equatorial flow: v850 (10S-10N, 40-55E) (Joseph 2005)

WNPM:
Westerly meridional shear: u850 (5-15N, 100-130E) minus u850 (20-30N,110140E) (Based on Wang and Fan 1999)

AUSM:
Zonal wind u850 (10S-0, 120E-150E)(Based on Webster 1983 and McBride
1987)
8.3 ENSO indices

Nino 3.4 (5S-5N, 120W-170W) SSTA

Nino 3 (5S-5N, 90W-150W) SSTA

Nino 4 (5S-5N, 150E-170W) SSTA

Model (Reanalysis) "Tahiti minus Darwin" like SOI: Using standardized
anomalies of sea-level pressure as per the methodology of Ropelewski and
185
Jones (1987). SLP (125E-135E, 7S-17S) minus SLP (145W-155W, 12S-22S)
(AchutaRao and Sperber, 2002, 2005)
8.4 Equatorial SST variability
EOF analysis of equatorial SST anomalies averaged between 10S-5N for each
season.
Background: The longitudinal distribution of SST anomalies along the equator is of
central importance for determining tropical and global teleconnection, thus it has
been recognized as a major source of global climate predictability.
8.5 ENSO-Monsoon relation
 Lead-lag correlation between Nino 3.4 SSTA and All Indian summer monsoon
rainfall
 Composite maps for El Nino and La Nina DJF seasons, respectively
 Composite maps for El Nino onset and decay JJA seasons, respectively
8.6 Practical predictability of global precipitation
 Quantification of practical predictability of global precipitation which relies on
identification of the “predictable” leading modes of the observed interannual
variation of global precipitation. The predictability is quantified by the
fractional variance accounted for by these “predictable” leading modes of
Season-reliant EOF analysis.
9. Intraseasonal Variability:
186

Variance of daily data

Fractional variance: Band-pass (2-10, 10-20, 20-90) day filtered data.

200 hPa zonal mean zonal winds (20-90 days) between 10S and 10N (Slingo
et al. 1995)

Propagation: RMSE add PCC of the Hovemoller diagrams. Use OLR or
precipitation averaged at (10S-10N, 75-100E) (20-90 day filtered data) as
reference to compute lead/lag correlation coefficients
(a) along the equatorial band (10S-20N) from –5 to 5 lag. (Nov.-April),
(b) along the meridional bands (75-100E) (May-Oct) (from 20S to 30N).
187