Zhan Zhang
and HWRF Team
1
Outline
 Introduction to ensemble prediction system
 What and why ensemble prediction
 Approaches to ensemble prediction
 Hurricane ensemble prediction
 HWRF-based EPS
 Methodology;
 Verification I : Ensemble vs. Deterministic;
 Statistical Characteristics of HWRF EPS;
 Verification II: HWRF EPS vs. Top-flight models;
 How ensemble perturbation affects storm intensity forecasts;
 Conclusion and Future Work.
2
What is an Ensemble Forecast ?
An ensemble forecast is simply a collection of two or more forecasts
verifying at the same time. Ensemble forecast aims to estimate the
probability density function of forecast states
Why do we need ensemble forecast ?
Uncertainties, or weak noises, acting upon a numerical weather
prediction (NWP) model system can have far-reaching consequences
due to its chaotic and nonlinear nature (Lorenz, 1963, 1965).
What are the main source of uncertainties ?
IC/BC uncertainties: observational errors, poor data coverage, and
errors in DA system;
Model uncertainties: mis-representation of model dynamics/physics,
impact of sub-grid scale features.
These uncertainties are inevitable. In a chaotic system like an
atmospheric model, non linear errors will grow - sometimes
rapidly. Eventually these growing errors cause the model forecast output
to eventually become useless.
3
Track Prediction for Hurricane Debby, 20120624 00Z
Multi-model
Large differences
in predicted storm
tracks due to:
1. multi-model
dynamics;
2. multi-physics;
3. multi-initial
analysis.
NCEP ensemble
ECMWF ensemble
CMC ensemble
4
Prediction for Hurricane Raymond, 20121024 00Z
Large differences in
predicted storm
intensity due to subgrid uncertainties in
model physics:
stochastically
perturbed cumulus
convection scheme in
HWRF
Dry air at mid-level
suppressed storm
development in one
member, while active
convective cells
overcome the dry air,
storm intensified in
anther member.
Two clusters: decaying and intensifying
5
Approaches to Ensemble Prediction
Monte Carlo Approach ---- not practically possible
sample all sources of forecast error, perturb any input variable and
any model parameter that is not perfectly known. Take into
consideration as many sources as possible of forecast error.
Reduced Sampling ----- limited resource
sample leading sources of forecast error, prioritize. Rank sources,
prioritize, optimize sampling: growing components will dominate
forecast error growth.
Existing Methods
Initial uncertainties: SV-based ensemble (ECMWF), EnKF-based
ensemble (MCS), BV-based ensemble (NCEP), ETKF-based ensemble
(UKMet), ETR-based ensemble, EOF-based ensemble.
Model uncertainties: Multi-model ensemble; Single model with
multi-physics.
Desired Ensemble Perturbations
Growing modes, Orthogonality, No bias, Similar to analysis error
6
HWRF-based Ensemble Prediction
Considerations for Hurricane EPS:
1. Uncertainties in initial storm position, intensity,
and structure;
2. Uncertainties in large scale flows (ICs/BCs);
3. Results of multi-scale interactions among subgrid scales, (~0-100m), convective clouds (~1001000m), and the large-scale environment (~1001000km)
7
Background and Motivation
Convective Trigger function in Current HWRF Cumulus
Parameterization Scheme (SAS: Simplified Arakawa-Schubert)
PCSL-PLFC <= DP(w) Convection is triggered,
PCSL-PLFC > DP(w)
No sub-grid convection
PCSL: Parcel pressure at Convection Starting Level,
PLFC: Parcel pressure at Level of Free Convection
DP(w): Convective Trigger, which is function of large scale
vertical velocity w.
DP(w) is arbitrarily confined between 120hPa-180hPa
 Storm intensity (Max Wind Speed) is found very sensitive to
the convective trigger function;
 Necessary to introduce fuzzy logic trigger to represent subgrid features.
8
Methodology
Model Physics Perturbations (large scale):
Stochastic Convective Trigger
PCSL-PLFC <= DP(w) + Rr (n)
Rr is white noise, ranging from -50hPa to +50hPa, n
is nth ensemble member, used as random seed. No
spatial and temporal correlations
IC/BC Perturbations (sub-grid scale):
20 member GEFS.
9
HWRF EPS vs. its Deterministic versions
HW01 – HW20:
Individual Ensemble Members Generated from Stochastic
Convective Trigger + GEFS IC/BC;
HWMN:
Mean of HW01-HW20 ensembles;
FY13:
2013 Operational HWRF, Deterministic version of HWRF;
H212:
2012 Operational HWRF;
2012: all storms during Aug. 1 to Oct. 31; 463 cycles
2011: 09L, 12L, 14L and 16L; 144 cycle
Total: 607 cycles.
10
ATL-Track
comparable
EP-Track
~23% improvement at day 4
ATL-Intensity
~14% improvement
EP-Intensity
~22% improvement at day 5
11
Central Pressure Verifications
Atlantic
E-Pac
12
Frequency Superior Performance Verifications
(Atlantic Basin)
Track: FY13 vs. H212
Track: HWMN vs. FY13
Intensity: FY13 vs. H212
Intensity: HWMN vs. FY13
13
Frequency Superior Performance Verifications
(E-Pac Basin)
Track: FY13 vs. H212
Track: HWMN vs. FY13
Intensity: FY13 vs.
H212
Intensity: HWMN vs.
FY13
14
Track Probability Forecasts for Hurricane Sandy
Few
members
turned
west
More
members
turned
west
FY13
All
members
turned
west
15
Ensemble Intensity Forecasts for Hurricane Sandy
16
Spaghetti Plots Comparison for Track
Forecasts Sandy, 2012
FY13
HWMN
Less outliers
17
Spaghetti Plot Comparison for Intensity
Forecasts Sandy, 2012
FY13
HWMN
Tighter clusters
18
Statistical Characteristics of HWRF EPS
Ensemble spread and forecast error relationship:
Assume the ensemble spread and forecast error
are two random variables, their 1st and 2nd
moments will be evaluated;
Spread/skill correlation: researches have shown
that the spread/skill correlation is generally not as
high as expected, less than 0.5 (Baker 1991,
Houtekamer 1993, Whitaker and Loughe 1997)
 Spread evaluation: rank histogram for ensemble
forecasts.
19
The under-dispersion
increases with forecast lead
time
1. The mean of ensemble
spread is close to the mean of
the forecast errors;
2. The difference between the
two lines indicates the level of
ensemble dispersion;
20
Relationship between Ensemble Spread and Forecast Error
Track
Intensity
0.3
Solid: Spread/Error Correlation
Dash: abs(Es-Ee)
Solid line: Spread/Error Correlation
Dash line: abs(Es-Ee)
*Es: standard deviation of ensemble spread, Ee: standard deviation of forecast error;
1. Spread/skill correlations reach to the peak at day 3 for both track and intensity;
2. The difference (Es-Ee) measures the closeness of the ‘climatological’ values of
ensemble spread and forecast errors; the ensemble spread has less predictive value
(of the forecast skill) when |Es-Ee| is small;
3. In HWRF EPS, intensity spread is a good indicator of forecast skill up to 72h, track
spread is not.
21
1. The correlation decreases with
lead time
2. The ensemble track spread is
correlated with intensity spread,
even though there is no
track/intensity error correlation;
22
Analysis of Rank Histogram for Track Forecast
Latitude
Northward bias
Southward bias
1. Only AL basin data is used;
2. The observed values are
distributed evenly in inner bins;
3. The N/S bias is mostly due to
longer lead time;
4. W (fast) track bias.
Longitude
Westward (fast) bias
Eastward (slow) bias
23
Analysis of Rank Histogram for Intensity Forecast
Strong bias
Weak bias
1. The observed values are distributed evenly in inner bins;
2.The deterministic model tends to have positive bias for strong
storms and negative bias for weaker storms.
24
Comparison HWRF EPS with last year’s
Top-Flight Models
Track:
AVNI, EMXI, GHMI
Intensity: DSHP, LGEM, GHMI
25
AL-track
EP-track
Statistically
indistinguishable with
EMXI, AVNI, SS in
improvement over
GHMI
Statistically
indistinguishable with
EMXI, AVNI, SS in
improvement over
GHMI
AL-Int
EP-Int
Better skills over GHMI and DSHP;
SS improvements over GHMI and
DSHP at some lead times
Smallest errors in all;
SS improvements over other
models at some lead times
26
Pairwise Comparison Between HWMI and Top-flight Models
Intensity For Atlantic Basin
From TCMT verification
27
Pairwise Comparison Between HWMI and Top-flight Models
Intensity For East-Pacific Basin
Sample size is not big enough
for EP
From TCMT verification
28
How stochastic convective trigger function perturbation
affects storm intensity forecasts
1. What is the cause of storm intensity ensemble
spread? Convective trigger perturbation or initial
storm cycling?
2. Impact of stochastic convective trigger perturbations
on D01 and D02? Cumulus convection is explicitly
expressed in domain 3 (~3km) in current
configuration
29
Cold-start Experiment
With GFS
~20kt diff
Several cold-start experiments
indicate there are large intensity
forecast spread even without
model cycling.
~15hPa diff
30
Impact of Cumulus Convection on Large Scale Flow
Idealized Experiment
Solid line: SAS in D01 and D02 (27km and 9km)
Dash line: no SAS in Do1 and D02.
Sub-grid convection is explicitly expressed in D03 in both exps.
The model storm will not develop when SAS scheme is turned off
in the 27- and 9-km domains even if the domain 3 (3km) resolution
is high enough to resolve the convection scheme.
31
10m Wind Speed forecast at 24h from
Individual Ensemble Members (Isaac, 2012)
Intensity or
Maximum Wind
Speed has random
feature.
32
10m Wind Speed forecast at 48h from
Individual Ensemble Members (Isaac, 2012)
Land interaction
Different predicted
tracks for individual
ensemble members due
to land interaction,
causing intensity
variations.
33
HWRF/EPS Verifications for 2013 Storms
AL-Track
AL-Intensity
EP-Track
EP-Intensity
IO-Track
IO-Intensity
34
Summary






Ensemble forecast is necessary due to the chaotic and nonlinear nature of the atmosphere;
Additional uncertainties, e.g. sub-grid scales, initial storm
positions, needs to be take into account in EPS;
HWRF-based EPS includes perturbations from large scale
flows (GEFS) and sub-grid scales (physics-based SCT);
HWRF EPS introduces no bias but inherits some biases from
the deterministic model in terms of track/intensity forecasts;
Both HWMI track and intensity forecast skills are improved
over its deterministic versions (H212 and FY13), with more
improvements in intensity forecasts;
The HWMI forecasts are statistically undistinguishable from
that of the top-flight models; The HWMI produces SS
improvements over GHMI in both track and intensity forecasts
at all lead times.
35
Summary (Continued)

In HWRF EPS, the intensity spread may be a good
predictor for intensity forecast skill up to 72h, not
the track spread;
Future work:


Experiment with SAS scheme turned on in third
domain;
Develop an ensemble ranking system that identifies
the ensemble member closest to the ensemble mean
based on their predicted storm centers, intensities,
and storm structures.
36