Statistical Weather Forecasting
Independent Study
Daria Kluver
From Statistical Methods in the Atmospheric Sciences
by Daniel Wilks
Perfect Prog and MOS
Classical statistical forecasts for projections
over a few days are not used. Current
dynamical NWP models are more accurate.
 2 types of classical statistical wx forecasting
are used to improve aspects of NWP
forecasts. (by post-processing the NWP
data)
 Both methods used large multiple regression
equations.

3 reasons why statistical reinterpretation of dynamical
NWP output is useful for practical weather forecasting:
1. NWP models simplify
and homogenize sfc
conditions.
Statistical relationships can
be developed btwn NWP
output and desired forecast
quantities.
2. NWP model forecasts are
subject to error. To the
extent that these errors
are systematic, statistical
forecasts based on NWP
info can correct forecast
biases.
3. NWP models are
deterministic. Using NWP
info in conjunction with
statistical methods allows
quantification of the
uncertainty associated with
different forecast situations.
1st Statistical approach for dealing
with forecasts from NWP

Perfect Prog (Klein et al. 1959)
◦ Takes NWP model forecasts for future
atmosphere assuming them to be perfect
◦ Perfect prog regression equations are similar to
classical regression equations except they do not
incorporate any time lag.
Example: equations specifying tomorrows predictand are
developed using tomorrow’s predictor values.
If the NWP forecasts for tomorrow’s predictors really are perfect,
the perfect-prog regression equations should provide very good
forecasts.
2nd Statistical approach

Model Output Statistics (MOS)
◦ Preferred because it can include directly in the
regression eqns the influences of specific
characteristics of different NWP models at different
projections into the future.
Example: predictand is tomorrows 1000-800mb
thickness as forecast today by a certain NWP model.
◦ To get MOS forecast eqns you need a developmental
data set with historical records of predictand, and
records of the forecasts by NWP model.
◦ Separate MOS forecast equations must by made for
different forecast projections.
Advantages and Disadvantages of
Perfect-Prog and MOS:
Perfect Prog

Advantages:





Large developmental sample (fit using historical climate data)
Equations developed without NWP info, so changes to NWP
models don’t require changes in regression equations
Improving NWP models will improve forecasts
Same equations can be used with any NWP models
Disadvantages:
◦ Potential predictors must be well forecast by the NWP model
MOS
 Advantages:
◦ Model-calculated, but un-observed quantities can be
predictors
◦ Systematic errors in the NWP model are accounted for
◦ Different MOS equations required for different projection
times
◦ Method of choice when practical
 Disadvantages:
◦ Requires archived records (several years) of forecast from
NWP model to develop, and models regularly undergo
changes.
◦ Different MOS equations required for different NWP
models
Operational MOS Forecasts


Example: FOUS14
MOS equations underlying the FOUS14 forecasts are
seasonally stratified: warm season (Apr- Sep) and cool
season (Oct-Mar).
◦ A finer stratification is preferable with sufficient
developmental data
Forecast equations (except t, td and winds) are
regionalized.
 Some MOS equations contain predictors representing
local climatological values
 Some equations are developed simultaneously to
enhance consistency (example: a td higher than t
doesn’t make sense)

Ensemble Forecasting
First, lets talk about the birth of
Chaos…
Lorenz
 Royal McBee
 Sensitive dependence on initial conditions

Lorenz’s data
What does Chaos have to do with
NWP?
The atmosphere can never be completely
observed
 A NWP model will always begin calculating
forecasts from a state slightly different from
the real atmosphere.
 These models have the property of
sensitive dependence on initial conditions.

Stochastic Dynamical Systems in
Phase Space

Stochastic dynamic prediction- physical laws
are deterministic, but the equations that
describe these laws must operate on initial
values not known with certainty
◦ can be described by a joint probability
distribution.

This process yields, as forecasts, probability
distributions describing uncertainty about
the future state of the atmos.
Phase space
Phase space is used to visualize the initial
and forecast probability distributions
 Phase space – geometrical representation
of the hypothetically possible states of a
dynamical system, where each of the
coordinate axes defining this geometry
pertains to one of the forecast variables
of the system.

Pendulum Animation
Lorenz attractor and the butterfly

Phase space of a atmospheric model has
many more dimensions.
◦ Simple model has 8 dimensions!
Operational NWP models have a million
dimensions
 More Complications:

◦ The trajectory is not attracted to a single
point like the pendulum
◦ Pendulum did not have sensitive dependence
to initial conditions.


Uncertainty about initial state of the
atmosphere can be conceived of as a
probability distribution in phase space.
The shape of the initial distribution is
stretched and distorted at longer forecast
projections.
◦ Also, remember there is no single attractor.
A single point in phase space is a unique
weather situation.
 The collection of possible points that equal
the attractor can be interpreted as the
climate of the NWP model.

Ensemble Forecasts





The ensemble forecast procedure begins by
drawing a finite sample from the probability
distribution describing the uncertainty of the
initial state of the atmos.
Members of the point cloud surrounding the
mean estimated atmospheric state are picked
randomly
These are the ensemble of initial conditions
The movement of the initial-state probability
distribution through phase space is approximated
by this sample’s trajectories
Each point provides the initial conditions for a
separate run of the NWP model.
Ensemble Average and Ensemble
Dispersion





To obtain a forecast more accurate than 1 model run
with the best estimate of the initial state of the
atmosphere, members of the ensemble are averaged.
The atmospheric state corresponding to the center
of the ensemble in phase space will approximate the
center of the stochastic dynamic probability
distribution at the future time.
Doing this with weather models averages out
elements of disagreement and emphasizes shared
features.
Over long time periods, the average smoothes out
and looks like climatology.
We get an idea of the uncertainty
◦ More confidence if the dispersion is small
◦ Formally calculated by ensemble standard deviation
Graphical Display of Ensemble
Forecast Information

Current practice includes 3 general types
of graphics:
◦ displays of raw ensemble output,
◦ displays of statistics summarizing the
ensemble distribution, and
◦ displays of ensemble relative frequencies for
selected predictands.
Effects of Model Errors

2 types of model errors:
◦ 1. models operate at a lower resolution than reality
◦ 2. certain physical processes- predominantly those operating at
scales smaller than the model resolution- are represented
incorrectly.
The parameterization (smooth curve) does not fully capture the range of
behaviors
To represent
residuals of
fig 6.31,
numbers
can(scatter
be
for the the
parameterized
process
thatrandom
are actually
possible
of added
points) to the parameterization function. Called “stochastic physics”
and used at ECMRF
Statistical Postprocessing: Ensemble
MOS

You can do MOS post processing on the
ensemble mean:
◦ There is still research on how best to do this.
◦ Multiple ways, which involve probability
distributions, which will not be discussed
here.
Subjective Probability Forecasts

The nature of subjective forecasts:
◦ Subjective integration and interpretation of
objective forecast info forecast guidance.
◦ Includes deterministic forecast info from NWP,
MOS, current obs, radar, sat, persistence info,
climate data, individual previous experiences.
◦ Subjective forecasting – the distillation by a
human forecaster of disparate and sometimes
conflicting info.
 A subjective forecast- one formulated on the basis of
the judgment of 1 or more individuals. Good one will
have some measure of uncertainty.
Assessing Discrete Probabilities

Tricks forecasters can use, like spinning
wheels or playing betting games.
Chapter 7: Forecast Verification

Purposes of Forecast Verification
◦ Forecast verification- the process of assessing the
quality of forecasts.
◦ Any given verification data set consists of a
collection of forecast/observation pairs whose
joint behavior can be characterized in terms of
the relative frequencies of the possible
combinations of forecast/observation outcomes.
◦ This is an empirical joint distribution
◦ It is important to do verification to improve
methods, evaluate forecasters, estimate error
characteristics.
The Joint Distribution of Forecasts
and Observations




Forecast =
Observation =
The joint distribution of the forecasts and
observations is denoted
This is a discrete bivariate probability
distribution function associating a
probability with each of the IxJ possible
combinations of forecast and observation.

The joint distribution can be factored in two
ways, the one used in a forecasting setting is:
Called calibration-refinement factorization
If y has occurred,
is the
The
unconditional
distribution,
 The this
refinement
of
a
set
of
forecasts
refers to
probability of o happening.
which specifies the relative
the
dispersion
of
the
distribution
p(yi)
Specifies how often each
frequencies
of use of each
of

i
j
possible weather event
occurred on those occasions
when the single forecast yi was
issued, or how well each
forecast is calibrated.
the forecast values yi
sometimes called the
refinement of a forecast.
Scalar Attributes of Forecast
Performance
Accuracy
Bias

Average correspondence
between
individual forecasts
and the events they
Partial
list
of
scalar
aspects,
or
attributes,
predict.
ofTheforecast
quality
correspondence between the average forecast and the average
observed value of the predictand.
Reliability
Pertains to the relationship of the forecast to the average observation, for
specific values of the forecast.
Resolution
The degree to which the forecasts sort the observed events into groups
that are different from each other.
Discrimination
Converse of resolution, pertains to differences between the conditional
averages of the forecasts for different values of the observation.
Sharpness
Characterize the unconditional distribution (relative frequencies of use) of
the forecasts.
Forecast Skill

Forecast skill- the relative accuracy of a
set of forecasts, wrt some set of standard
control, or reference, forecast (like
climatological average, persistence forecasts, random
forecasts based on climatological relative frequencies)

Skill score- a percentage improvement
over reference forecast.
Accuracy of
reference
accuracy
Accuracy that would be achieved by
a perfect forecast.
Next time

Continue to talk about forecast verification
◦ Looking at some forecast data
 NWS vs weather.com vs climatology
◦
◦
◦
◦
◦
◦
◦
2x2 contingency tables
Conversion from probabilistic to nonprobabilistic
Quantile plots
Probability forecasts of discrete predicands
Probability forecasts for continuous predictands
Accuracy measures, Skill scores, Brier Score, MSE
Multi-category events