Tutorial
Other post-processing approaches …
1) Bayesian Model Averaging (BMA) –
Raftery et al (1997)
2) Analogue approaches –
Hopson and Webster, J. Hydromet (2010)
3) Kalman Filter with analogues –
Delle Monache et al (2010)
4) Quantile regression –
Hopson and Hacker, MWR (under review)
5) quantile-to-quantile (quantile matching) approach –
Hopson and Webster J. Hydromet (2010)
… many other approaches …
Significance of Weather Forecast Uncertainty
Bangladesh Discharge Forecasts
Precipitation Forecasts
1 day
7 day
4 day
10 day
Transformed
into
Discharge Forecasts
1 day
3 day
7 day
4 day
4 day
10 day
Calculating the rank histogram …
Get data from shared disk: AfricaColloqData.txt
Place in working directory (desktop will work) and read in spreadsheet
obs
First occurrence
Forecast
PDF
=> Count in bin 6
Flow rate [m3/s]
obs
Second occurrence
Forecast
PDF
=> Count in bin 3
Flow rate [m3/s]
… Generate histogram plot using spreadsheet
Next, redo in R
Next steps … do both in spreadsheet and R
-- Next, approximately determine 75 quantile
From the July observation data (let me know
Brier Score
your answer before proceeding further)
th
n
-- Then calculate the Brier score for probability
of occurrence below this threshold.
-- Then, using the climatology mean (using the
July obs data – what is the value?), calculate
The Brier skill score for this reference forecast
(what’s your answer?). Is there a better
reference forecast? Try this. Answer?
-- Finally, using this same threshold, calculate
The reliability diagram values (4 probabilities)
And plot using spreadsheet (what are the
Probabilities?)
Note: for this exercise, we will use a more
“discrete” form for the ensemble probability,
in which you just count the # of
ensemble members above or below
the threshold to calculate the probability
(arguably less exact then interpolating)
1
2
BS   yi  oi 
n i 1
y = forecasted event occurence
o = observed occurrence (0 or 1)
i = sample # of total n samples
=> Note similarity to MSE
Brier skill score
BS  BS ref
BS
BSS 
 1
0  BS ref
BS ref
Reliability Diagram
From Eumetcal
module on
forecast
verification