Comparison of Several Methods for
Probabilistic Forecasting of Locally-Heavy
Rainfall in the 0-3 Hour Timeframe
Z. Sokol1, D. Kitzmiller2, S. Guan2
1Institute
of Atmospheric Physics AS CR, Prague, Czech Republic
2Hydrology Laboratory, Office of Hydrologic Development, NOAA National
Weather Service, Silver Spring, Maryland, USA
Introduction
• Description of the current operational model
used by NWS (U.S.A.)
• Aims of the study
• Alternative models tested on the selected
subregion
• Comparison of the regression model results
• Conclusions
Current Model
• Predictands: probabilities that rainfall will reach or
exceed 2.5, 12.5, 25.4, and 50.8 mm during the
succeeding 3-h period at boxes of a 40-km grid
covering the conterminous United States
• Predictors:
– Extrapolated radar reflectivity, lightning strike rate,
and cloud-top temperature by advecting the
corresponding initial-time fields at the velocity of
the forecasted 700-500 hPa mean wind vector
– Forecasts of humidity, stability indices, moisture
divergence, and precipitation from the operational
Eta (NAM) model
Current Model
• Forecasting tool:
– Linear regression model for each threshold
amount and 8 daytimes (01-03 UTC, … , 22-00
UTC)
– Separate sets of equations for warm (AprilSeptember) and cool (October-March) seasons
– One model for all boxes in the conterminous
United States
– Regression model derived from historical data
(MOS)
Example of Outputs
1800-2100 UTC, 4 June 2005
Probability of 25 mm (1 inch)
rainfall.
Categorical rain amount forecast.
Probability of 50 mm (2 inches)
rainfall.
Radar/gauge precipitation
estimates during verifying period.
Aims of This Study
• Attempt to refine existing model for U.S.
– Examine regression models not previously
considered
– Consider effects of local and regional models,
rather than single general model
• Consider implications for development of a
model for the Czech Republic
Tests
• Selected subregion: The northeastern United
States (New York, Massachusetts, Vermont, New
Hampshire, Rhode Island, and Maine) during the
warm season (May-September).
This area has a summertime precipitation regime
similar to that of the Czech Republic.
• Data: 4 years May-September, 1997-2000
Development of the model:
– 3 years – calibration data
– 1 year – independent data
Tests
• Categorical forecast (yes/no) for given thresholds
for boxes
– Mean precipitation in 40x40 km region
– Maximum 4x4 km precipitation within 40x40 km
region
• Transition from probabilistic to categorical
forecast
– Fixed threshold 0.5
– Optimum threshold derived on the calibration data
• Verification measure: Equitable thread score (ETS)
Types of models:
• REG - Linear regression
y  a0  a1 x1  a2 x2  ...  aN xN
• REG3 - Localized linear regression models (derived for single boxes)
• LREG - Logistic regression
• RAT - Rational regression
log( y ) 
1
a0  a1 x1  a2 x2  ...  a N x N
a0  a1 x1  a 2 x2  ...  a N x N
y
1  b1 x1  b2 x2  ...  bN x N
• NN - Neural network
(perceptron type, 1 hidden layer)
Predictand: 40x40 km, Precipitation  5mm (1%-3%)
a) Yes/No Threshold = 0.5
Model
REG
LREG
RAT
NN
REGG3
REGALL_5
00-03
0.04
0.12
0.07
0.06
0.04
0.06
04-06
0.09
0.08
0.10
0.11
0.11
0.12
07-09
0.09
0.11
0.10
0.10
0.09
0.08
10-12
0.04
0.09
0.11
0.09
0.07
0.09
13-15
0.02
0.03
0.04
0.03
0.05
0.06
16-18
0.10
0.13
0.13
0.11
0.12
0.11
19-21
0.07
0.17
0.13
0.10
0.07
0.07
22-00
0.14
0.18
0.17
0.16
0.14
0.14
Mean
0.073
0.115
0.107
0.096
0.087
0.090
10-12
0.20
0.23
0.22
0.25
0. 19
0.24
13-15
0.17
0.19
0.18
0.15
0.14
0.20
16-18
0.25
0.23
0.25
0.27
0.23
0.28
19-21
0.27
0.29
0.28
0.27
0.21
0.28
22-00
0.28
0.30
0.30
0.29
0.23
0.29
Mean
0.229
0.239
0.235
0.238
0.199
0.252
b) Optimum Yes/No Threshold
Model
REG
LREG
RAT
NN
REGG3
REGALL_5
00-03
0.20
0.23
0.20
0.22
0.20
0.22
04-06
0.25
0.23
0.22
0.24
0.21
0.26
07-09
0.21
0.23
0.23
0.24
0.18
0.24
Distribution of Forecast Probabilities
Example of forecasts by REG and LREG
predictand maximum 4x4km precipitation
a) T=12.5 mm, REG
10
b) T=12.5 mm, LREG
10
0.9
Probability Forecasts for
12.5 mm
0.8
8
0.9
0.8
8
0.7
0.6
6
0.7
0.6
6
0.5
4
0.4
0.5
4
0.4
0.3
2
0.2
2
6
8
10
12
14
0.2
0.1
2
c) T=25.4 mm, REG
10
Probability Forecasts for
25.4 mm
4
0.3
2
4
6
8
10
12
14
d) T=25.4 mm, LREG
10
0.45
0.45
0.4
8
0.4
8
0.35
0.35
0.3
6
0.3
6
0.25
0.25
4
0.2
4
0.2
0.15
0.15
2
0.1
2
6
8
10
12
14
2
0.05
e) 2000071514 - observation
10
Verifying precipitation amount
(left) and antecedent amount
(right)
4
0.1
2
10
4
6
8
10
12
14
25
30
25
8
20
20
15
6
15
6
10
10
4
5
4
5
2.5
2.5
2
1
2
4
6
8
10
12
14
0.1
0.05
f) 2000071511 - previous observation
30
8
0.1
2
1
2
4
6
8
10
12
14
0.1
Conclusions
• The localized approach REGG3 did not improve the
forecasts for the northeastern U.S.
• For the 0.5 yes/no threshold REG results are worse
than results of other methods. It is valid for higher
precipitation thresholds.
• If optimum threshold (maximizing ETS) is used then
resultant ETS of all the methods are similar.
• REG yields smoother probability fields than other
methods; LREG yields smaller areas of nonzero
probabilities but higher values within those areas.
• In general the best results were obtained by LREG
and NN methods.
• Our experience shows that NN method should use
only a limited number (10-30) a priori selected
predictors, otherwise the results are worse.