Representing model uncertainty in weather
and climate: stochastic versa multi-physics
representations
Judith Berner, NCAR
Judith Berner: Representing Model Error by Stochastic Parameterizations
Key Points
 There is model error in weather and climate models
 from the need to parameterize subgrid-scale fluctuations
 This model error leads to overconfident uncertainty estimates
and possibly model bias
 We need a model error representation
 Hierarchy of simulations where statistical output from one
level is used to inform the next (e.g., stochastic kinetic
energy backscatter)
 Reliability of ensemble systems with stochastic
parameterizations start to become comparable to that of
ensembles systems with multi-physics
Judith Berner: Representing Model Error by Stochastic Parameterizations
“Domino Parameterization strategy”
 Higher-resolution model inform output of lower-resolution model
 Stochastic kinetic energy backscatter scheme provides such a
framework
 … But there are others, e.g. Cloud-resolving convective
parameterization or super-parameterization
Multiple scales of motion
1mm
10 m
Microphysics
100 m
Turbulence
1 km
10 km
100 km
1000
km
10000
km
Cumulus Cumulonimbus Mesoscale Extratropical Planetary
clouds
clouds
Convective Cyclones
waves
systems
Large Eddy Simulation (LES) Model
Cloud System Resolving Model (CSRM)
Numerical Weather Prediction (NWP) Model
Global Climate Model
Judith Berner: Representing Model Error by Stochastic Parameterizations
The spectral gap …
(Stull)
Atmospheric
Scientists
Nastrom and Gage, 1985
Judith Berner: Representing Model Error by Stochastic Parameterizations
TO W A RD SEA M LESS PRED IC T IO N
C alibr at ion of C lim at e C hange Pr oject ions U sing Seaso nal
For ecasts
BY
T. N . PA LM ER, F. J. D
O BL AS -R EY ES ,
A .W
EI SH EIM ER, AN D
M . J. R O D W ELL
In a seam l ess p r ed i cti o n syste m , t he r el i ab il it y o f co up l ed cl im at e m o d el fo r ec asts
mad e o n se aso n al ti me scal es c an p r o vi d e u sefu l q u an ti t ati ve c o nst r ai nt s fo r
im p r o vi ng t h e t r ust w o r t h in ess o f r e gio n al cl im at e c ha nge p ro j e ct io n s.
.... The link between
climate forcing and
climate impact
involves processes
acting on different
timescales …
F I G . 1 . A s c h e m a tic fig u re illu s tr a tin g th a t th e lin k b e tw e e n c lim a t e fo rc in g a n d c lim a te im p a c t in vo lv e s p ro c e ss e s a c tin g o n d iffe r e n t tim e sc a le s . T h e w h o le c h a in is a s s tro n g a s it s w e a k e st lin k . T h e u s e o f a se a m le ss
p r e d ict io n sy s te m
a llo w s p r o b a b ilis tic p ro je c tio n s o f c lim a te c h a n g e to b e c o n s tr a in e d b y v a lid a t io n s o n w e a th e r
o r s e a so n a l fo r e c a s t t im e s c a le s.
mad e o n se aso n al ti me scal es c an p r o vi d e u sefu l q u an ti t ati ve c o nst r ai nt s fo r
im p r o vi ng t h e t r ust w o r t h in ess o f r e gio n al cl im at e c ha nge p ro j e ct io n s.
Cloud resolving
Cloud resolving
model
model
Large Eddy
simulation
NPW
model
Climate model
Resolved
microphysics
Attempt to capture
Multi-scale nature of
atmospheric motion
F I G . 1 . A s c h e m a tic fig u re illu s tr a tin g th a t th e lin k b e tw e e n c lim a t e fo rc in g a n d c lim a te im p a c t in vo lv e s p ro c e ss e s a c tin g o n d iffe r e n t tim e sc a le s . T h e w h o le c h a in is a s s tro n g a s it s w e a k e st lin k . T h e u s e o f a se a m le ss
p r e d ict io n sy s te m
a llo w s p r o b a b ilis tic p ro je c tio n s o f c lim a te c h a n g e to b e c o n s tr a in e d b y v a lid a t io n s o n w e a th e r
mad e o n se aso n al ti me scal es c an p r o vi d e u sefu l q u an ti t ati ve c o nst r ai nt s fo r
im p r o vi ng t h e t r ust w o r t h in ess o f r e gio n al cl im at e c ha nge p ro j e ct io n s.
Hierarchical Parameterization Strategy
Cloud resolving
model
NPW
model
Climate model
Large Eddy
simulation
Resolved
microphysics
Related: Grabowski 1999, Shutts
and Palmer, 2007
F I G . 1 . ARepresenting
s c h e m a tic fiModel
g u re illu
s tr a tin
th a t th e li
n k b e tw e e n c lim a t e
Judith Berner:
Error
bygStochastic
Parameterizations
fo rc in g a n d c lim a te im p a c t in vo lv e s p ro -
c e ss e s a c tin g o n d iffe r e n t tim e sc a le s . T h e w h o le c h a in is a s s tro n g a s it s w e a k e st lin k . T h e u s e o f a se a m le ss
Validity of spectral gap …
Judith Berner: Representing Model Error by Stochastic Parameterizations
The spectral gap …
Mathematicians
Atmospheric
Scientists
The spectral gap …
M pathematicians
Atmospheric
Scientists
Spectral gap not necessary for stochastic
parameterizations
Judith Berner: Representing Model Error by Stochastic Parameterizations
Kinetic energy spectra in 500hPa
Rotational part
Rotational part
Kinetic energy spectrum is closer to that of T799 analysis !
Judith Berner: Representing Model Error by Stochastic Parameterizations
Limited vs unlimited predictability
Rotunno and Snyder, 2008
Lorenz 1969;
Judith Berner: Representing Model Error by Stochastic Parameterizations
Stochastic parameterizations have the potential
to reduce model error
Potential
Weak noise
Strong noise
Stochastic parameterizations
can change the mean and
variance of a PDF
Impacts variability of model
(e.g. internal variability of the
atmosphere)
PDF
Impacts systematic error (e.g.
blocking, precipitation error)
Unimodal
Multi-modal
Judith Berner: Representing Model Error by Stochastic Parameterizations
Outline
 Parameterizations in numerical weather prediction models and
climate models
 A stochastic kinetic energy backscatter scheme
 Impact on synoptic probabilistic weather forecasting
(short/medium-range)
 Impact on systematic model error
(seasonal to climatic time-scales)
Acknowledgements
Aime Fournier, So-young Ha, Josh Hacker, Thomas Jung, Tim
Palmer, Paco Doblas-Reyes, Glenn Shutts, Chris Snyder,
Antje Weisheimer
Judith Berner: Representing Model Error by Stochastic Parameterizations
Sensitivity to initial perturbations
Judith Berner: Representing Model Error by Stochastic Parameterizations
Representing initial state uncertainty by an
ensemble of states
RMS error
spread
ensemble mean
analysis
t0
t1
t2
 Represent initial uncertainty by ensemble of states
 Flow-dependence:
 Predictable states should have small ensemble spread
 Unpredictable states should have large ensemble spread
 Ensemble spread should grow like RMS error
 True atmospheric state should be indistinguishable from ensemble
system
Systems
Underdispersion of the ensemble system
------- spread around ensemble mean
RMS error of ensemble mean
The RMS error grows faster than
the spread
Ensemble is underdispersive
Ensemble forecast is
overconfident
Underdispersion is a form of
model error
Forecast error = initial error +
model error + boundary error
Buizza et al., 2004
Judith Berner: Representing Model Error by Stochastic Parameterizations
Manifestations of model error
In medium-range:
 Underdispersion of ensemble system (Overconfidence)
 Can “extreme” weather events be captured?
On seasonal to climatic scales:
 Systematic Biases
 Not enough internal variability
 To which degree do e.g. climate sensitivity depend on a
correct estimate of internal variability?
 Shortcomings in representation of physical processes:
 Underestimation of the frequency of blocking
 Tropical variability, e.g. MJO, wave propagation
Judith Berner: Representing Model Error by Stochastic Parameterizations
Representing model error in ensemble
systems
The multi-parameterization approach: each ensemble
member uses a different set of parameterizations (e.g.
for cumulus convection, planetary boundary layer,
microphysics, short-wave/long-wave radiation, land
use, land surface)
The multi-parameter approach: each ensemble member
uses the control pysics, but the parameters are varied
from one ensemble member to the next
Stochastic parameterizations: each ensemble member is
perturbed by a stochastic forcing term that represents
the statistical fluctuations in the subgrid-scale fluxes
(stochastic diabatic tendencies) as well as altogether
unrepresented interactions between the resolved an
unresolved scale (stochastic kinetic energy backscatter)
Judith Berner: Representing Model Error by Stochastic Parameterizations
Recent attempts at remedying model error in
NWP
 Using conventional
parameterizations
 Stochastic parameterizations
(Buizza et al, 1999, Lin and
Neelin, 2000)
 Multi-parameterization
approaches (Houtekamer, 1996,
Berner et al. 2010)
 Multi-parameter approaches
(e.g. Murphy et al,, 2004;
Stainforth et al, 2004)
 Multi-models (e.g. DEMETER,
ENSEMBLES, TIGGE,
Krishnamurti et. al 1999)
 Outside conventional
parameterizations
 Cloud-resolving convective
parameterization (CRCP) or superparameterization (Grabowski and
Smolarkiewicz 1999, Khairoutdinov
and Randall 2001)
 Nonlocal parameterizations, e.g.,
cellular automata pattern generator
(Palmer, 1997, 2001)
 Stochastic kinetic energy
backscatter in NWP (Shutts 2005,
Berner et al. 2008,2009,…)
Judith Berner: Representing Model Error by Stochastic Parameterizations
Stochastic kinetic energy backscatter
schemes
 Stochastic kinetic energy backscatter LES
Mason and Thompon, 1992, Weinbrecht and Mason, 2008
 Stochastic kinetic energy backscatter in simplified models
Frederiksen and Keupert 2004
 Stochastic kinetic energy backscatter in NWP
 IFS ensemble system, ECMWF:
Shutts and Palmer 2003, Shutts 2005, Berner et al. 2009a,b,
Steinheimer
 MOGREPS, MetOffice
Bowler et al 2008, 2009; Tennant et al 2010
 Canadian Ensemble System
Li et al 2008, Charron et al. 2010
 AFWA mesoscale ensemble system, NCAR
Berner et al. 2010
Judith Berner: Representing Model Error by Stochastic Parameterizations
Forcing streamfunction spectra by coarsegraining CRMs
from Glenn Shutts
Judith Berner: Representing Model Error by Stochastic Parameterizations
“Domino Parameterization strategy”
 Higher-resolution model inform output of lower-resolution model
 Stochastic kinetic energy backscatter scheme provides such a
framework
 … But there are others, e.g. Cloud-resolving convective
parameterization or super-parameterization
Judith Berner: Representing Model Error by Stochastic Parameterizations
 Model error in weather forecasting and climate models
 A stochastic kinetic energy backscatter scheme (SPBS)
 Impact of SPBS on probabilistic weather forecasting
(medium-range) ->
 Impact of SPBS on systematic model error
 Impact in a mesoscale model and comparison to a multi-physics
scheme
Judith Berner: Representing Model Error by Stochastic Parameterizations
Forecast error growth
For perfect ensemble system:
the true atmospheric state should be
indistinguishable from a perturbed
ensemble member
 forecast error and model uncertainty
(=spread) should be the same
Since IPs are reduced, forecast
error is reduced for small forecast
times
More kinetic energy in small
scales
 Model error in weather forecasting and climate models
 A stochastic kinetic energy backscatter scheme: SPectral
Backscatter Scheme
 Impact of SPBS on probabilistic weather forecasting (mediumrange)
 Impact of SPBS on systematic model error
 Impact in a mesoscale model and comparison to a multiphysics scheme
Judith Berner: Representing Model Error by Stochastic Parameterizations
Experimental Setup for Seasonal Runs
“Seasonal runs: Atmosphere only”
 Atmosphere only, observed SSTs
 40 start dates between 1962 – 2001 (Nov 1)
 5-month integrations
 One set of integrations with stochastic
backscatter, one without
 Model runs are compared to ERA40 reanalysis
(“truth”)
Judith Berner: Representing Model Error by Stochastic Parameterizations
Reduction of systematic error of z500 over
North Pacific and North Atlantic
No StochasticBackscatter
Stochastic Backscatter
Increase in occurrence of Atlantic and
Pacific blocking
ERA40 + confidence
interval
Stochastic Backscatter
No StochasticBackscatter
Judith Berner: Representing Model Error by Stochastic Parameterizations
Wavenumber-Frequency Spectrum
Symmetric part, background removed
(after Wheeler and Kiladis, 1999)
Observations (NOAA)
No Stochastic Backscatter
Improvement in Wavenumber-Frequency
Spectrum
Observations (NOAA)
Stochastic Backscatter
 Backscatter scheme reduces erroneous westward propagating modes
 Model error in weather forecasting and climate models
 A stochastic kinetic energy backscatter scheme: SPectral
Backscatter Scheme
 Impact of SPBS on probabilistic weather forecasting (mediumrange)
 Impact of SPBS on systematic model error
 Impact in a mesoscale model and comparison to a multiphysics scheme
Judith Berner: Representing Model Error by Stochastic Parameterizations
Experiment setup
 Ensemble A/B: 10 member ensemble with and without SPBS
 Ensemble C: 10 member multi-physics suite
 Weather Research and Forecast Model
 30 cases between Nov 2008 and Feb 2009
 40km horizontal resolution and 40 vertical levels
 Limited area model: Continuous United States (CONUS)
 Started from GFS initial condition (downscaled from NCEPs
Global Forecast System)
Multiple Physics packages
Judith Berner: Representing Model Error by Stochastic Parameterizations
WRF short-range ensemble: 60h-forecast for
Oct 13, 2006: SLP and surface wind
Control
Physics
Ensemble
Judith Berner: Representing Model Error by Stochastic Parameterizations
WRF short-range ensemble: 60h-forecast for
Oct 13, 2006: SLP and surface wind
 Stochastic
Backscatter
Ensemble
Judith Berner: Representing Model Error by Stochastic Parameterizations
Spread-Error Relationship
Control
Backscatter
Multi-Physics
Judith Berner: Representing Model Error by Stochastic Parameterizations
Brier Score, U
Control
Backscatter
Multi-Physics
Judith Berner: Representing Model Error by Stochastic Parameterizations
Scatterplots of
verification
scores
Both, Stochastic backscatter
and Multi-
physics are better than control
Stochastic backscatter is better than
Multi-physics is better
Their combination is even better
Judith Berner: Representing Model Error by Stochastic Parameterizations
Multiple Physics packages
Judith Berner: Representing Model Error by Stochastic Parameterizations
Brier Score
Control
Multi-Physics
Backscatter
Judith Berner: Representing Model Error by Stochastic Parameterizations
Spread-Error Relationship
Control
Backscatter
Multi-Physics
Judith Berner: Representing Model Error by Stochastic Parameterizations
Seasonal Predication
Uncalibrated
Calibrated
Stochastic
Ensemble
Multi-model
Curtosy: TimPalmer
Judith Berner: Representing Model Error by Stochastic Parameterizations
Summary and conclusion
 Stochastic parameterization have the potential to reduce
model error by changing the mean state and internal
variability.
 It was shown that the new stochastic kinetic energy
backscatter scheme (SPBS) produced a more skilful
ensemble and reduced certain aspects of systematic model
error
 Increases predictability across the scales (from
mesoscale over synoptic scale to climatic scales)
 Stochastic Backscatter outperforms Multi-physics Ens.
 Stochastic backscatter scheme provides a framework for
hierarchical parameterization strategy, where stochastic
parameterization for the lower resolution model is informed
by higher resolution model
Future Work
Understand the nature of model error better
Inform more parameters from coarsegrained high-resolution output
Impact on climate sensitivity
Consequences for error growth and
predictability
Challenges
How can we incorporate the “structural
uncertainty” estimated by multi-models into
stochastic parameterizations?
Judith Berner: Representing Model Error by Stochastic Parameterizations
Bibliography
 Berner, J., 2005: Linking Nonlinearity and non-Gaussianity by the
Fokker-Planck equation and the associated nonlinear stochastic model,
J. Atmos. Sci., 62, pp. 2098-2117
 Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in
ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 612, 30793102
 Berner, J., F. J. Doblas-Reyes, T. N. Palmer, G. Shutts, and A.
Weisheimer, 2008: Impact of a quasi-stochastic cellular automaton
backscatter scheme on the systematic error and seasonal predicition
skill of a global climate model, Phil. Trans. R. Soc A, 366, pp. 25612579, DOI: 10.1098/rsta.2008.0031.
 Berner J., G. Shutts, M. Leutbecher, and T.N. Palmer, 2009: A Spectral
Stochastic Kinetic Energy Backscatter Scheme and its Impact on Flowdependent Pre- dictability in the ECMWF Ensemble Prediction System,
J. Atmos. Sci.,66,pp.603-626
 T.N. Palmer, F.J. Doblas-Reyes, A. Weisheimer, G.J. Shutts, J.
Berner, J.M. Murphy, 2008: Towards the Probabilistic Earth-System
Model, J.Clim., in preparation
Judith Berner: Representing Model Error by Stochastic Parameterizations