Use of Ensemble Forecasting at Météo-France
J. Nicolau, Météo-France
Jean Nicolau
Météo-France (SCEM/PREVI)
42, Avenue G. Coriolis
31057 Toulouse cedex
Phone : 33 5 61 07 85 32
Fax : 33 5 61 07 84 53
E-Mail : Jean.Nicolau@meteo.fr
Summary
Intuitively, the idea that a weather forecast does not have the same reliability of one day to the other,
according to the weather situation, is well known as well by our users as by meteorologists. For a few
years, methods of numerical prediction have made possible quantitatively to estimate this reliability (one
takes often the opposed formulation, that of the estimate of uncertainty) at the time where the forecast is
performed and not only a posteriori. The main tool allowing estimating such information is the
Ensemble Forecasting. In the first part, we are presenting how and why such tool is used at MétéoFrance in the medium-range weather forecasting, while insisting on the influence which it can have as
well on the manner of working of the forecaster as on the products of the prediction. In a second part
some other applications of the Ensemble Prediction System are described.
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
1
I. Use of EPS in the Medium-Range weather forecasts
The limit of the synoptic methodology
The weather forecast is at once based on the identification of meteorological patterns. Traditionally, the
forecaster is using the synoptic scale characterised by well defined structures such as disturbances or
fronts. The forecaster has to predict the evolution of such structures in order to give sensitive weather.
This prediction is based on the use of numerical model. These models regularly controlled, show that
the forecast skill decreases with the range. Up to Day 4 less than 50% of forecasts are good and up to
Day 6 almost 90% are wrong. In this control, the synoptic scope is judged in its consequences on the
sensitive weather over France. The first point to note is that models are inconsistent from each other and
changeable from one day to another.
The supra-synoptic scale
As predictability is directly linked to the range and to the spatial scale of the prediction, the synoptic
guidance does not seem to be relevant up to Day 4. A new scale, the supra synoptic scale, is then
considered. In this scale, synoptic features are not taken into account as complex meso-scale features for
the synoptic scale. For instance, the occurrence time and the precise location of a perturbation can't be
realistically predicted. For that reason this new scale will be characteristic of the global circulation.
Weather types
In order to elaborate the weather forecast, it is necessary to associate for each "supra" pattern an
ensemble of weather parameters. Of course traditional synoptic weather parameters such as
precipitation can't be associated to such pattern. With the supra-synoptic scale, a certain number of flow
types slowly variable over a period of 2/3 days are identified. From these flow types, it is possible to
determine the main characteristics of the weather. Four main flow types have been defined:




Straight Flow : rapid, straight, perturbed, rainy often windy.
Undulating Flow : alternating ridge and trough, unsettled weather, a rainy period is followed by a
sunny one. There is at least one rainy period.
Warm Blocked Flow : anticyclonic conditions. Dry weather, sunny in summer, sometimes foggy or
dull weather in winter.
Cold Blocked Flow : low-pressure conditions, cool or cold weather, often rainy.
Estimating the uncertainty : The Ensemble Prediction System (EPS)
The ensemble forecast is a set of integrations of a deterministic numerical weather prediction (NWP)
model; these integrations differ only in their initial states, reflecting uncertainties in such starting
conditions. In essence, the ensemble prediction is an attempt to estimate the nonlinear evolution of the
forecast error probability function (PDF) through a finite sample of deterministic forecast integrations.
One of the fundamental difficulties in this approach lies in the choice of initial conditions for the
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
2
individual members of the ensemble. From the definition above, these must reflect an adequate
sampling of the probability distribution function of analysis error. However, the phase-space dimension
of current NWP models is well in excess of a million. Actually only a few perturbations (initial states
modifications) will be retained. Hence, with a limited ensemble size, it is important to sample those
trajectories starting within the initial error balls, which evolve to significantly different large-scale flow
patterns, such as weather regimes.
The operational EPS at ECMWF is presently based on the use of 25 perturbations defined from the
Singular Vectors method. These perturbations are added and retracted from the initial state. Thus a set
of 51 predictions (50 perturbed forecasts and one unperturbed, the so-called control forecast) is obtained.
This system is running daily to a 240 hours range.
The EPS classification : the tubing
The huge quantity of information in the Ensemble Prediction System makes its direct use very difficult.
Tools have been developed in order to condense the information coming from the 51 ensemble forecasts
: the tubing (figure 1) is a method designed to classify ensemble forecasts. Forecasts that are similar to
the ensemble mean are grouped into one cluster, the so-called central cluster. The other forecasts are
grouped into a few tubes indicating different deviations from the ensemble mean. Each tube is
represented by its extreme, i.e. the forecast which is the most different from the ensemble mean in the
direction of the tube. The extremes of the tubes are not alternatives; they are extreme representatives of
the tubes, almost caricatures, allowing to better visualise the different tendencies present in the
ensemble, by contrast with the central cluster mean forecast.
farthest member
central cluster
tube
central cluster mean
figure 1: tubing representation
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
3
The 7 days weather forecast
Every day, the medium range forecaster of the Central Forecast Department in Toulouse, is provided
with three sets of the tubing classification of the EPS, representing the central cluster mean and a
variable number of tubes (generally less than 5 or 6). Each set are representative for a period of 72
hours, based on +96h, +144h and +192h forecasts. The fields used are 500hPa height and temperature,
mean sea level pressure and 850 hPa temperature, on North Atlantic and Western Europe area. Other
deterministic fields and probabilistic fields of weather parameters like temperature anomalies and
precipitation amounts are also used. The forecaster describes the most likely forecast evolution by
interpreting the large scale patterns of the central cluster mean. He(she) supplies a "supra-synoptic
scenario" for D4/D5 (D+5/D+6 model range) and the evolution expected for D6/D7 (D+7/D+8 model
range), with associated comments, and a graphical document, the so-called PRESSYME (PREvision
Supra SYnoptique Moyenne Echéance) for each of these two periods. The association between flow
types and weather characteristics is based on the expertise of the forecasters (more or less a
climatological knowledge). A weather types automatic classification of all of the ensemble members is
provided to the forecaster. The kind of weather expected for France is then defined in the technical
guidance, with more precision for parameters like wind, temperatures and rain intensity. The precise
description given on a daily basis for the weather characteristics may vary for a same flow type,
depending on the season, the region (quarters of France typically) and on the different flow types that
have been identified (it is not an automatic transcription of the flow type analysis). It is here necessary
to underline the way of interpreting the central cluster mean. This field represents more or less
predictable features. It plays the role of a clever filter: some details described by T159 members are
rubbed by this way and that means that such details are not predictable. Of course the predictability is
not the same one day to the other and the filtering degree is varying with the spread of the ensemble. If
the spread is small, then the mean will describe precise details (in practice the equivalent truncation of
the central cluster mean will vary between T20 and T10).
Recently a new graphical product has been added to the technical medium range guidance, for the
national media, the so-called "Nébule MEDIA" (figure 2). This product includes information on weather
tendencies and some risks (strong winds, storms, snow).
In addition, a confidence index is defined by the forecaster. This confidence index is linked to the
number of tubes, which are, in term of weather parameters over the area of interest, significantly
different from the central cluster mean. It is then evaluated in a subjective way, on a scale of 1 to 5 :



4 if the confidence is strong (0 or 1 tube)
3 if the confidence is normal (2 or 3 tubes)
2 if the confidence is weak (more than 3 tubes)
Values of 1 and 5 are rarely used and only in very specific cases: very strong confidence, which is more
often very difficult to appreciate, or on the opposite, very weak confidence due to a great number of
tubes with significant variants over the country. Finally, the confidence index can be modified by the
forecaster according to the kind of weather expected. For example, the forecaster may have a very
strong confidence for a warm block, but a lower confidence for the corresponding kind of weather
(generally fair), due to a possible presence of low clouds, especially in winter.
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
4
figure 2 : Nébule MEDIA product
The medium range weather forecast evaluation
Subjective evaluation of the forecaster weather flow analysis is conducted in a daily basis. Four marks
are used :

 A : very good forecast.
 B : good forecast, large scale circulation well forecasted, small differences on chronology or spatial
localization of forecast flow patterns.
 C : rather bad, not necessarily a bad forecast of large scale circulation but differences are too
important on France.
 D : bad forecast of large circulation.
If A and B forecast are considered to be "good" forecasts, the percentage of good forecasts on the past
year was : 87 % for D4-D5 and 59% for D6/D7. Considering the correlation between confidence index
and skill forecast, it appears that when the confidence index is high the forecast is good, while when the
confidence index is low, the correlation with bad forecasts is not so evident. As a consequence it can be
said that forecasters are rather inclined to "overprotect" but also that there is still too many bad forecasts
which are not detected.
Finally the 7 days weather forecasts give good results in terms of scores but the public perception for
such forecast is not really positive. Firstly the information given seems to be too vague, secondly the
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
5
uncertainty quantification (confidence index) is perhaps not the appropriate way to give the weather
predictability because there is a lot of bad interpretations about it.
II. Use of EPS for professional applications
EPS allows also probabilistic prediction for weather parameters. The probability of an event can be
directly calculated as being the proportion of members of the ensemble for which this event occurs
(figure 3). However it appears that such probabilities need to be readjusted (we will speak about
"calibration") in reliability. For instance, for the event: "24h-precipitation 1 mm exceeding over
Toulouse", for the cases where the probability is 80%, the effective frequency of this event is only 60%.
It thus will be required to gauge these probabilities according to a "climatology" of the forecasted
probabilities, particularly since the request for these products is increasing on behalf of the professionals
(agriculture, buildings, transport, energy...).
Calibration
Generally, raw probabilities are not reliable. Model bias (for example rainfall over-estimating) or the
lack of statistical consistency of the ensemble (weak spread) can be at the origin of such problem. This
poor reliability is linked to the fact that the hypothesis is equi-probability of all of the ensemble
members. A very simple method to calibrate probabilities is based on the use of the reliability diagram.
The calibrated probability can be obtained by the observed frequency (y-axis) corresponding to the raw
probability (x-axis) on the reliability curve. This method has a strong disadvantage: it uses the reliability
diagram, which depends on the threshold.
The method used here is valid whatever the threshold. It was already used by Hamill et Colucci (1997,
1998) and Eckel et Walters (1998). This method is based on rank diagrams (also called "Talagrand
diagrams", figure 4). The 51 values of the EPS are ordered from the lowest to the highest. It is then
possible to determine the bin including the considered threshold. The calibrated probability corresponds to the
probability
rain height
cumul of all of the left side bins of the rank diagram before the threshold's bin. This method allows increasing
efficiently the reliability for parameters such as precipitation (figure 5).
24
25
26
27
28
29
30
31
01
02
03
days
figure 3 : 12h amount rain probability on Toulouse for different thresholds
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
6
P(verification in bin n)
bin
1
bin
10
bi n
20
bin
30
4
bin
0
bin
52
figure 4 : rank diagram
PROBABILITIES RR24 > 1mm Calibration test
raw probabilities
observed frequencies
calbrated probabilities
forecasted probabilities
distribution
forecasted probabilities (%) 1998
figure 5 : reliability curves before and after calibration
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
7
Statistic adaptation
The calibration method is not really satisfactory for such parameters as minimum and maximum
temperatures. For these particular parameters, a statistical adaptation method based on a multi-linear
regression is presently used. This method increases the results in term of reliability but also in term of
resolution, but acts in reducing the spread of the ensemble. In order to solve this problem a first trial
was undertaken. This trial consisted in applying a translation based on the previous spread (before
regression) to all the regressed members. Finally the best results were obtained in combining the
regression and the calibration methods (figure 6).
figure 6 : Brier score for different methods for the event "Minimum temperature below 0°C"
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
8
Products
With such methods, Météo-France can now supply professionals with probability products. One of the
most popular is the so-called Meteogram which provides the evolution of Maximum and Minimum
temperatures on one point with the ensemble mean value, the minimum and maximum values given by
extreme members and the uncertainty given by the interval including 75% of the members. Of course, at
Medium-Range such point is representative of a region. The risk of extreme events such as heavy rain
can also be evaluate by probability charts.
Date : Jeudi 14 Septembre 2000
figure 7 : Minimum and Maximum temperature evolution with an uncertainty quantification
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
9
III. Special applications of EPS
Two experiments based on the use of the EPS are presented here.
Low pressure centers tracking
The first one concerns the tracking of the cyclone "Connie", which passed close to the French island of
La Réunion (Indian Ocean) on the 25th of January 2000. A selection on low pressure centers was done
over the ensemble members in order to see the evolution of the cyclone from the early to the medium
range. The trajectory was well seen in the first steps of the forecast but rapidly lost by the ensemble
(figure 8).
Météo-France is waiting for the promising diabatic singular vectors computed over the tropical areas to
improve such trackings.
figure 8 : Temporal evolution of the cyclone "Connie" for EPS members
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
10
Model coupling
The second one is related to the dramatic Erika oil slick. On December 12, the tanker Erika was
wrecked near the Brittany coast. In order to track the pollutant, Météo-France used its model of
pollutant drift ("Mothy"). This model uses usually the wind speed and direction from a global
atmospheric model. The idea was here to get the more pessimistic evolution in terms of wind speed and
directions (i.e. towards the French coast) from the ensemble in order to evaluate the earlier date of the
pollutant arrival on the coast (figure 9). With this kind of forecast, it was therefore possible to announce
that there was no risk for the pollutant to arrive on the coast before the 23th of December. This
information was a posteriori verified and allowed the authorities to take some measures (such as floating
dams). The pollutant arrived on the coast on the 25th of December after 2 weeks of daily tracking of the
trajectory by Météo-France.
figure 9 : Pollutant drift model coupled with the more pessimistic ensemble member in terms of wind.
Base : 16 of december at 12 utc, step 168. Red star indicates the initial position;
grey lines indicate the trajectories of the pollutant peaces.
Large black points indicate the final position of the different surface pollutant peaces
and little black points indicate the final position of subsurface pollutant peaces.
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
11
References
Eckel F. A. and M. K. Walters, calibrated probabilistic quantitative precipitation forecasts based on the MRF
ensemble. Weather and Forecasting, vol 13, 1132-1147, 1998.
Hamill T. S. and S. J. Colucci, verification of ETA-RSM short-range ensemble forecasts. Monthly Weather
Review. vol 125, 1312-1327, 1997.
Hamill T. S. and S. J. Colucci, evaluation of ETA-RSM ensemble probabilistic precipitation forecasts. Monthly
Weather Review. vol 126, 711-724, 1998.
Workshop on Use of Ensemble Predictions Sytems
-
Beijing, 16-20 October 2000
-
12