Advanced Data Assimilation Methods
WP2.1 Perform (ensemble) experiments
to quantify model errors.
Stefano Migliorini, Ross Bannister,
National Centre for Earth Observation, University of
Reading
Ali Rudd, Laura Baker
Department of Meteorology, University of Reading
11 December
ESA DA Projects Progress MeetingUniversity
2
of Reading
Motivation
• Satellite observations form the vast majority of the total
number of observations assimilated in NWP models.
• To exploit information from satellite (as well as in-situ)
instruments, prior knowledge from NWP forecasts is needed
(Bayesian approach).
• Climatology of forecast errors at larger scales reflects wellknown balance relationships of atmospheric flow.
• Structure of high-res forecast errors are much more uncertain.
• Aim of this work is to provide reliable estimates of forecast
errors at convective scale (new generation models) to improve
assimilation of in-situ, radar and satellite data.
Aims of the project
• Investigate sources of uncertainty in high-res forecasts:
– Initial and boundary condition errors.
– Model errors due to the parameterisation of subgrid-scale
processes.
• Use a convective-scale ensemble prediction system (EPS).
• Evaluate the effects of these errors on the forecast error
covariances.
• Check reliability of errors using observations.
• Improve our knowledge of high-res forecast errors and of
their balance relationships for better high-res DA.
Case Study: 20 September 2011
• DIAMET IOP2 – flight campaign
case.
• Frontal wave structure.
• SW-NE flow across southern UK.
1200UTC analysis
• Interesting banded structure in
radar not captured in the
1800UTC analysis
operational 1.5km forecast or our
control forecast.
Ensemble system
UK Met Office operational ensemble
systems
MOGREPS: Met Office Global and
Regional Ensemble Prediction System:
MOGREPS-G
•MOGREPS-G: 60 km grid spacing, 70
vertical levels.
•MOGREPS-R: 18 km grid spacing, 70
vertical levels.
MOGREPS-R
•23 perturbed members and one
control member.
•(MOGREPS-UK: 2.2km grid spacing, 12member ensemble).
Figure source: J.F. Caron
MOGREPS-G
1.5 km domain
MOGREPS-R
07
08 09 10
11 12
13 14
15
16 17 18
6hr forecast
•
Domain over southern UK (360 x 288 grid points).
•
Control member from 3D-Var analysis.
•
23 perturbed members: initial condition perturbations and LBCs from MOGREPS-R.
•
Hourly-cycling ETKF for the first 6 hours.
•
6 hour forecast from 12Z.
Figure source: J.F. Caron
Simulating model error:
the Random Parameter scheme





Has been used operationally in MOGREPS.
Not used previously in a convective-scale EPS.
RP treats a set of parameters in various parametrization schemes as stochastic variables.
Applies different random perturbations to these parameters for each ensemble member.
Based on first-order auto-regression model (Pt is the parameter value at time t):
Pt = μ + r (Pt-1 – μ) + ε






μ is the default value of the parameter.
r = 0.95 is the auto-correlation coefficient of P.
ε is the stochastic shock term (random value in range ± (Pmax – Pmin) / 3).
Pmax, Pmin for each parameter are estimated by experts.
Have studied forecast sensitivities to each parameter.
Options to how RP can be applied:

CTL:
Parameters set the same between members (ics only).

RP-60:
Update every 60 minutes.

RP-30:
Update every 30 minutes.

RP-fix:
Parameters set at t = 0 only.

onlyRP:
RP-60\30\fix without ic.
Sensitivity to perturbed parameters
RMS difference between
perturbed and control
forecasts at T+3 (1500 UTC)
1.5m
temperature
10m uwind
Ensemble experiments
Ensemble name
Description
Model error
variability
IC and LBC
variability
Inflation
CTL
Control
No
Yes
Yes
IC+BC+RPfix
RP scheme with
fixed params
Yes
Yes
Yes
IC+BC+RP30
RP scheme with 30
minute update
Yes
Yes
Yes
IC+BC+RP60
RP scheme with 60
minute update
Yes
Yes
Yes
RPfix
ME only: RP
scheme with fixed
params
Yes
No
Yes
RP30
ME only: RP
scheme with 30
minute update
Yes
No
Yes
RP60
ME only: RP
scheme with 60
minute update
Yes
No
Yes
How does model error affect the spread?
1.5m temperature
10m wind speed
___ control ensemble
___ ensemble with fixed perturbed
parameters
Domain-averaged ensemble spread:
n  points
1
variancei

n-points i 1
Hourly rainfall accumulation
How does model error affect the spread?
1.5m temperature
10m wind speed
___ control ensemble
___ ensemble with fixed perturbed parameters
___ ensemble with periodic update (30 min)
___ ensemble with periodic update (60 min)
Domain-averaged ensemble spread:
n  points
1
variancei

n-points i 1
Hourly rainfall accumulation
How does model error affect the spread?
1.5m temperature
10m wind speed
Hourly rainfall accumulation
___ control ensemble
___ ensemble with fixed perturbed parameters
___ ensemble with periodic update (30 min)
___ ensemble with periodic update (60 min)
- - - model error only (fixed parameters)
- - - model error only (30 min)
- - - model error only (60 min)
How does model error affect the
forecast skill?
___ control ensemble
___ ensemble with fixed perturbed
parameters
___ ensemble with periodic update
(30 min)
___ ensemble with periodic update
(60 min)
Better skill
- Continous ranked
probability score.
- Comparison of
CDF of forecast
and obs.
rain accumulation
u-wind component
v-wind component
Better skill
CRPS:
surface temperature
How does model error affect the
forecast skill?
Precipitation skill score for hourly rainfall accumulation
Threshold of 0.2 mm
Threshold of 1.0 mm
Better skill
Threshold of 0 mm
___ control ensemble
___ ensemble with fixed perturbed parameters
Precipitation skill score:
BSens
PSSens  1 
BScontrol
How does model error affect the
forecast skill?
Precipitation skill score for hourly rainfall accumulation
Threshold of 0 mm
Threshold of 0.2 mm
___ control ensemble
___ ensemble with fixed perturbed parameters
___ ensemble with periodic update (30 min)
___ ensemble with periodic update (60 min)
Threshold of 1.0 mm
Precipitation skill score:
BSens
PSSens  1 
BScontrol
Summary
• Developing a method of representing model error in a convective
scale ensemble:
– Random parameters scheme.
• How does the additional representation of model error affect the
spread of the ensemble?
– Temperature and wind speed – applying the RP scheme increases the
spread.
– Rainfall rate – the RP scheme has an undesirable peaks in the spread –
this is reduced by keeping parameters fixed.
• How does model error affect the forecast skill?
– Small effect on forecast skill.
– Skill in rain rate and accumulation is reduced – probably due to
reduction in total rain rate.
Forecast errors in data assimilation
• q-w correlations > 0.
• The more buoyant the parcel the wetter.
• Descent leads to warmer and drier parcels.
Conclusions
• Including model error variability increases ensemble spread in most
quantities, over and above that found from ensembles that include
only initial condition and lateral boundary condition variability.
• Including model error variability is not guaranteed to increase
ensemble spread in all quantities, e.g. we have found that the spread
of rainfall forecasts is actually reduced, although it is not clear why this
is so.
• Ensemble forecasts can inform data assimilation studies of the correct
structure of forecast error statistics and the balances that are obeyed.
• Future work includes investigations on covariance length scales, study
of forecast errors using observations and effects of sampling errors.
Extra slides
Do any of the ensemble members
capture the banding in the rain?
“stamp plot”
Ensemble: RP scheme on - with model error
1500 UTC
Parameters in our modified RP scheme
•
•
•
Parameters above
the black line are in
the existing scheme;
those below are new
We vary some
parameters together
where appropriate
(eg. ei and eic; x1r,
x1i and x1ic) – i.e. we
use the same random
seed for ei and eic so
that they vary
together rather than
independently
We have found that
the particle size
distribution
parameters (x1r, etc.)
have a larger effect
than any others –
possibly too large