WORLD METEOROLOGICAL ORGANIZATION
COMMISSION FOR BASIC SYSTEMS
OPAG DPFS
EXPERT TEAM ON ENSEMBLE PREDICTION
SYSTEMS
CBS-DPFS/EPS/Doc.7(3)
(14.2.2006)
_______
Item: 7
ENGLISH ONLY
EXETER, UNITED KINGDOM
6-10 FEBRUARY 2006
Application of EPS for severe weather forecasting
(Submitted by Mr David Richardson.)
Summary and purpose of document
The Extreme forecast index (EFI) developed for the ECMWF EPS is
reviewed. Initial verification has shown encouraging results for Europe. A
new implementation of the EFI will be introduced on 1 February 2006. A
more extensive verification of the new system is required to assess the
potential benefit to users.
ACTION PROPOSED
(The meeting is invited to review the document and consider input to its conclusions
and recommendations as appropriate.)
CBS-DPFS/EPS/Doc.7(3), p. 2
1
The extreme forecast index
1.1 Introduction
The forecasting of extreme weather events has become more and more important in the last few years. The
Extreme Forecast Index (EFI) has been developed and implemented to help the forecasters to synthetize the
deviation from climatology of the EPS forecasts. This EFI index is an integral measure of the departure
between the EPS forecasts and the reference climate distributions. Because of this design, it is strongly
dependant on the climate distribution from which the level of extremity is calculated. WMO defines a
climate as a collection of observations over three decades (30 year period). Regarding the model climate it is
hard to achieve this requirement. In the Numerical Weather Prediction community a comprehensive, widely
accepted concept of the model climate has not been developed yet. Therefore the model climate information
that is used in the different applications is most of the time case specific, developed to meet the requirement
of the actual problem.
At ECMWF there was no medium range model climate available at the time of the EFI development. To
constitute the pseudo-climate for each month from only consistent model forecast as far as possible,
altogether three possible ways could be considered:
1. The best and most costly option was to run a consistent, long set of forecasts starting from reanalyses using the model version used for creating the EPS. This could not have been performed in
the early times (apart from the computational issue, the ERA-40 reanalysis was not finished).
2. To gather information from model forecasts valid for the same calendar month over the previous
three years.
3. To gather information from model forecasts valid for the previous calendar month on the current
year, and from the previous year, the current calendar month and the following one. This approach
could be useful after significant model upgrades, such as resolution changes.
In practice the 2. and 3. types of climate information were prepared operationally since the implementation
of the EFI. In both cases to build the climate distribution the ensemble analysis, the 5 and the 10 days
forecasts were kept from each 12 UTC EPS run, considered as independent realisations of the climate. This
super-ensemble had a size about 10000 members. This assumption is however less reasonable over areas
(such as the tropics), where the transient weather patterns are not losing their predictability over less than a
week, and therefore their variability can not really be explored by such a super ensemble.
Figure 1 shows an example of an Extreme Forecast Index (EFI) chart for the 2m temperature, 10m wind and
the precipitation forecasts. At the ECMWF Users’ meeting in June 2005 a request was made, via the
chairman of the WMO/CBS Expert Team on Ensemble Forecasting, to provide this product to the WMO
user community. First steps to verify the EFI against observations, using additional information from the
Meteo France (EUMETNET/ECSN) station Climate Atlas for Europe, have already been taken. From 1
February 2006 the EFI will use a new model climatology (described below, 1.3). Additional evaluation of the
EFI will then be required to provide appropriate performance assessment before wider distribution of
potential EFI products.
CBS-DPFS/EPS/Doc.7(3), p. 3
Weather anomalies predicted by EPS: 20041112 0 UTC
1000 hPa Z ensemble mean and EFI values for 24h TP, wind gust and 2m temperature
VT: 20041114
extreme warm
extreme cold
cold
40°W
windy
extreme wind
0°
Heavy precip
Extreme precip
40°E
8
8
0
16
16
warm
16
L
0
0
8
8
16
16
16
24
32
24
8
H
40°W
32
24
40°E
8
L
0
0
24
16
8
16
H
16
0°
Figure 1: Extreme Forecast Index for 2m temperature, 10 metre wind and precipitation predicted over
Europe and the Atlantic valid 14 November 2004
1.2 EFI verification
The Climate Atlas of Europe newly released by Météo-France as part of a EUMETNET/ECSN co-ordinated
action has made it possible this year to lay the foundations of what could become a European scale
verification system, using the ranking of observations within local climate records. Figure 2 shows the
Relative Operating Characteristics of the EFI in predicting extreme precipitation events, while Figure 3
provides the same information, using the Probability of exceeding 20mm/day as a criterion to warn against
the same extreme events. Clearly the EFI brings some refinement, allowing the detection of more events in
the early medium range, even if this is at the expense of relatively high false alarm rates (left) and false alarm
ratios (right).
CBS-DPFS/EPS/Doc.7(3), p. 4
Modified ROC curves
ROC curvesEFI(Annual verification, Min=20mm or 20/100 th of Upper Monthly Quintile)
6-30
54-78
?
102-126
6-30
0.9
0.9
0.8
0.8
0.7
?
102-126
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
54-78
Probability of Detection
1
Probability of Detection
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0
1
0
0.1
False Alarm Rate
False Alarm Rate
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability of False Detection
FAR
Figure 2: Verification of daily rainfall events exceeding the 99% threshold of the EUMETNET/ECSN
station climatology using the EFI. Each curve is for a different forecast range, while each point of the
curve is for a different EFI threshold. Sample contains 2622 events over the period October 2003 to May
2005.
Modified ROC curves
ROC curvesProb(Annual verification, Min=20mm or 20/100 th of Upper Monthly Quintile)
36
72
120
?
36
0.9
0.9
0.8
0.8
0.7
?
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
120
Probability of Detection
1
Probability of Detection
1
72
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
False Alarm Rate
False Alarm Rate
0.8
0.9
1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Probability of False Detection
FAR
Figure 3: Same as Figure 2, but using different numbers of EPS members exceeding 20mm/day as a
decision criterion to forecast the occurrence of the 99% threshold.
1
CBS-DPFS/EPS/Doc.7(3), p. 5
1.3 New EFI climatology
A new version of the EFI climatology will be introduced from 1 February 2006. The new EFI pseudoclimate is the realisation of the first option mentioned in Section 1.1. The system characteristics are the
following:

The EPS control version is run on a daily basis (well in advance, to provide the necessary re-forecast
files for the EFI computation), without the stochastic physics, always with the configuration
currently in operations. Therefore the climate of every single day will be unique and slightly
different. And moreover any model upgrade can easily be applied in the EFI process without any
technical or meteorological discrepancy.

In order to simulate the 30 years climate period, the model is run starting from the interpolated ERA40 analysis for the years from 1971 to 2000. For example to simulate the climate conditions for
January the 1st, the Control is run 30 times, starting from the initial conditions on 1971.01.01,
1972.01. 01 up to 2000.01.01.

The Control is run once a day only, at 12 UTC, until 48 hours to provide the best model estimate of
the actual climate conditions on a particular day for all the possible EFI parameters. For example the
30 control runs on the 1st of January, at 12 UTC, will provide the model climate for the 2nd of
January, because of the necessary time projections. Regarding for instance the 12 UTC 2 metre
temperature climate, the 24 hours timestep of the re-forecast are considered, and for the 24 hour
accumulated precipitation (from 06 UTC to 06 UTC), the forecast range of +18h to +42h has to be
taken into account.

Although the system provides the climate for every day specifically, the number of the available
forecasts for a single day is only 30. In the old EFI climate an order of 10000 forecasts was
considered. To keep the system as consistent as possible, the new EFI climate is composed of the sub
climate sample of 31 adjacent days, centred on the day for which the EFI is to be computed (±15
days). With this choice the number of forecasts constituting the climate distributions reach 930 (30
years * 31 days). This can also guarantee that the climate distribution is smooth enough, it is
constituted in a detailed form around the tails (which has great importance in order to be reliable in
terms of the extremes) and avoid the jumpiness of the climate distribution on a day to day basis
caused by too small sample size.

The tests that have been made confirmed our expectations. The climate distributions and therefore
the EFI values can show occasionally quite large differences depending on the time period and the
location considered. The climate also can vary significantly according to the applied yearly sample
size (EFI climate composed from only 1 day (±0 day) to a 31 day (±15 days) period each year).
CBS-DPFS/EPS/Doc.7(3), p. 6
1
0.9
0.8
0.7
EPS forecast
Old climate
0.6
30 years EPS control re-forecast
climate (with different number of
days)
0.5
0.4
READING
New EPS climat with 1=yellow, 3=green, 7=orange,
15=red, 31=blue days each year(1971-2000),
centered around the EPS forecast date (mmdd)
Old EPS climat (2003-2004-2005) is dashed blue
EPS forecast(black),2005.01.20 00 UTC; tstep=108h
0.3
0.2
0.1
0
-4
0
4
8
12
16
20
Figure 4: An example with the different pseudo-climate distributions. The Day-5 EPS forecast from
2005.01.20, for Reading.