Current developments in data
assimilation for numerical weather
prediction in Canada:
EnVar, EnKF and 4D-Var
Mark Buehner1, Josée Morneau2 and Cecilien Charette1
1Data
Assimilation and Satellite Meteorology Research Section
2Data Assimilation and Quality Control Development Section
January, 2013
Contents
• Background and recent changes to the Canadian NWP
systems
• The ensemble-variational (EnVar) data assimilation
approach
• Recent results from using EnVar compared with standard
3D-Var and 4D-Var (but NO comparisons with 4D-VarBens or EnKF)
• Conclusions and next steps
Page 2 – April-7-15
Background
• Environment Canada currently has 2 relatively independent state-ofthe-art global data assimilation systems
• 4D-Var (Gauthier et al 2007) and EnKF (Houtekamer et al 2009):
– both operational since 2005
– both use GEM forecast model and assimilate similar set of
observations
– current effort towards unifying code of the two systems
• 4D-Var is used to initialize medium range global deterministic
forecasts (GDPS)
• EnKF is used to initialize global ensemble forecasts (GEPS)
• Intercomparison of approaches and various hybrid configurations was
performed in carefully controlled context: similar medium-range
forecast quality from EnKF and 4D-Var analyses, 4D-Var-Ben best
• Results presented at WMO workshop on intercomparison of 4D-Var
and EnKF, Buenos Aires, 2008 (Buehner et al 2010, MWR)
Page 3 – April-7-15
Summary of recent changes
Major improvements
to DA of global and
regional systems
IBM Power7
New 10km/4DVar
Regional Deterministic
Prediction System…
Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct
2011
2012
Nov
Improvements to global &
regional DA systems
(Nov 2011)
New Satellite Data
- 62 infrared channels from the IASI instrument on
board the METOP satellite.
- 7 microwave channels from the SSM/IS
instrument on board the DMSP F16 satellite.
- 1 water vapor channel from the GOES-W,
METSAT-1R and both METEOSAT satellites.
- Horizontal thinning of all satellite radiance
data, previously done at 250 km (except 200 km
for SSM/I) was reduced to 150 km, therefore
adding much more satellite data to the systems.
Amount of data assimilated in the
global system
- increase from ~1.9 million to ~4.2 million
pieces of information per day.
Fig: GZ-500hPa anomaly correlations over the S. Hemisphere during
the parallel run (Jan-Jun 2011): old vs new global system.
Other Assimilation Changes
- Moisture observations measured from
properly equipped aircraft (AMDAR) are
assimilated.
- Unified satellite data bias correction
scheme. Reduction of the time period to
compute the bias corrections from 15 to 7
days. The same code is used for all
radiance data.
- Updated version of the RTTOV radiative
transfer code for satellite radiance data.
- New sea surface temperature analysis
on a 0.2° grid.
New IBM Power7
• 2 clusters; 8192 cores each
• ~ 1/2 PFlops peak total
• on average, performance gain per
CPU about 2.7x compared to P5 CPU
• System was installed off-site
• Migration of operational jobs
took a year to complete
• Fully operational since early
May 2012
Major upgrade of the
Regional Deterministic
Prediction System (RDPS)
(Oct 2012)
Upgrade included:
• increase in resolution to 10 km
from the previous 15 km
• 4D-Var data assimilation system
replacing the previous 3D-Var
• important changes in the physics
Fig: The red line indicates the domain covered by the RDPS.
GZ
T
Significant improvements in
forecasts with most metrics
throughout most of the
atmosphere, especially:
• for the winter season
• for the lower portion of the
atmosphere and at the surface
Fig: Vertical profiles of bias (dashed lines) and error standard deviation
(solid lines) of 48-h forecasts against radiosonde data, over Canada
during the winter 2011, from the old (15km) versus new (10km) RDPS.
by the end of 2013
by Jan 2013
Upcoming NWP operational implementations
Global Deterministic
- resolution: from 33 km to 25 km
- new vertical coordinates / grid + improved physics
- 4D-Var analysis increments: from T108 to T180, ~100 km
Global EPS
- resolution: from 100 km to 66 km
- analysis (EnKF): multi-scale (3x more assimilated data)
Regional EPS
- resolution: from 33 km to 15 km
Global Deterministic
- resolution: from 25 km to 15 km
- from lat-lon to Yin-Yang grid
- from 4D-Var to EnVar: main focus of this presentation
- resolution of increments: from 100 km to 50 km
- new sea-ice and land-surface analyses
Global EPS
- resolution: from 66 km to 50 km
High-resolution System
- single grid (2.5-km resolution) covering most of Canada
GZ
T
Major upgrade of the
Global Deterministic
Prediction System (GDPS)
(expected to become operational in early 2012)
DA (4Dvar)
forecast model
Fig: Vertical profiles of bias (dashed lines) and error standard deviation
(solid lines) of 120-h forecasts against radiosonde data, over the S.
Hemisphere, during the summer 2011, from the old versus GDPS.
horizontal resolution
- from 33km to 25km
- improvements seen in analysis cycle
dynamics
- new vertical coordinate
- new vertical grid (from regular
to Charney-Phillips)
- reduction of errors in stratosphere
- noise reduction; improved numerical
stability and conservation properties
orographic
blocking
- amplification of bulk drag
coefficient
- significant reduction of tropospheric errors
in winter hemisphere
boundary
layer
- turbulent hysteresis effect
- reduction of errors associated with frontal
inversions; improvement of upper-air scores
physics
outer loop
inner
loop
- from 33km to 25km
TL / AD
- from 160km to 100km
- t: from 45min (9 bins) to 18min (21 bins)
background
error statistics
- from T108 to T180
minimization:# of iterations
- from 55 (30+25) to 65 (35+30)
- more data (AMSU-A and
Aircraft) due to increase of
# bins
- all changes contributed
to forecast improvements,
roughly doubling the gain
due to model changes
Upgrade of Global & Regional
Ensemble Prediction Systems
(expected to become operational in early 2012)
Changes to the Regional EPS
• Model component
- horiz. resolution: from 33 to 15km
- vertical levels from 28 to 40
- improved treatment of stochastic physical
tendency perturbations to avoid unrealistic
precipitation rates
- improved boundary layer parameterization
• Assimilation
component
- no changes
(i.e same initial
conditions as the
global EPS)
Fig: Global verification (CRPS error) of temperature at 500hPa against
radiosondes, ov OLD versus NEW GEPS, showing a gain in predictability of 12h
and plus.
Changes to the Global EnKF
and EPS
• multi-scale algorithm (horizontal
localization varies vertically)
• time-step: from 30 to 20min
• horiz. resolution: from 100 to 66km
• vertical levels: from 58 to 74
• topography filter
• reduced thinning of observations (2.7 X
number of assimilated radiances)
• improved dynamics and physics
Model Dynamics:
New approaches for global grids
Proposed Upgrades and Improvements to the MSC
Data Processing for Radiosonde and Aircraft Data
• Increased volume of data: selection of observations
according to model levels
• Revised observation error statistics
• Revised rejection criteria for radiosonde data based on
those used at ECMWF
• Horizontal drift of radiosonde balloon taken into account
in both data assimilation and verification systems
• Bias correction scheme for aircraft temperature reports
wind speed
temperature
operational
12h
48h
Fig: Verification scores against radiosondes over the N.
Hemisphere, Jan-Feb 2009 (dash = bias; solid = stde)
proposed for both
radiosonde & aircraft
Impact of proposed changes
• General short-range forecast
improvements above 500 hPa in both wind
and temperature fields
• The temperature forecast biases are
significantly improved due to the bias
correction scheme for aircraft below 200
hPa and to the new rejection criteria for
radiosonde humidity data above
2013-2017: Toward a Reorganization of
the NWP Suites at Environment Canada
Role of Ensembles…
Perturbed
members of
the global
prediction
system (GPS)
Global
EnKF
Regional
EnKF
Background
error
covariances
Regional
En-Var ?
Background
error
covariances
Global
En-Var
Control
member of the
global
prediction
system (GPS)
global system
High-res
En-Var
Page 14 – April-7-15
Perturbed
members of
the regional
prediction
system (RPS)
Control
member of the
regional
prediction
system (RPS)
High-resolution
deterministic
prediction
system
(HRDPS)
regional system
Ensemble-Variational assimilation: EnVar
• EnVar approach is currently being tested in the context of replacing
4D-Var in the operational Global Deterministic Prediction System
• EnVar uses a variational assimilation approach in combination
with the already available 4D ensemble covariances from the EnKF
• By making use of the 4D ensembles, EnVar performs a 4D analysis
without the need of the tangent-linear and adjoint of forecast model
• Consequently, it is more computationally efficient and easier to
maintain/adapt than 4D-Var
• Hybrid covariances can be used in EnVar by averaging the ensemble
covariances with the static NMC-method covariances
• Like 4D-Var, EnVar uses an incremental approach with:
– analysis increment at the horizontal/temporal resolution of EnKF
ensembles
– background state and analysis at the horizontal/temporal resolution of
the higher-resolution deterministic forecast model
Page 15 – April-7-15
EnVar formulation
•
In 4D-Var the 3D analysis increment is evolved in time using the
TL/AD forecast model (here included in H4D):
J (x ) 
1
2
•
T
1
1
x B x
T
2
In EnVar the background-error covariances and analysed state are
explicitly 4-dimensional, resulting in cost function:
J (  x 4D ) 
•
1
( H 4D [ x b ]  H 4D  x  y ) R ( H 4D [ x b ]  H 4D  x  y ) 
1
2
1
( H 4D [ x b ]  H  x 4D  y ) R ( H 4D [ x b ]  H  x 4D  y ) 
T
1
2
T
1
 x 4D B 4D  x 4D
Computations involving ensemble-based B4D can be more
expensive than with Bnmc depending on ensemble size and spatial/
temporal resolution, but significant parallelization is possible
Page 16 – April-7-15
4D error covariances
Temporal covariance evolution (explicit vs. implicit evolution)
EnKF and EnVar (4D B matrix):
192 NLM integrations provide 4D
background-error covariances
4D-Var:
TL/AD inner loop integrations,
2 outer loop iterations,
then 3h NLM forecast
-3h
0h
Page 17 – April-7-15
“analysis
time”
+3h
EnVar: a possible replacement of 4D-Var
Overall, EnVar analysis ~6X faster than 4D-Var on half as
many cpus, even though higher resolution increments
Wall-clock time of 4D-Var already close to allowable time
limit; increasing number of processors has negligible impact
To progress with 4D-Var, significant work would be required
to improve scalability of TL/AD versions of forecast model at
resolutions and grid configuration used in 4D-Var
Current focus for model is on development of higherresolution global Yin-Yang configuration that scales well
Decision made to try to replace 4D-Var with more efficient
EnVar  if EnVar is at least as good as current 4D-Var
Page 18 – April-7-15
Dependencies between global systems
•
GDPS:
Current system (1-way dependence):
xb
Bgcheck+BC
xb, obs
4D-Var
xa
GEM (9h fcst)
xb
obs
GEPS:
•
xb
EnKF
xa
GEM (9h fcst)
xb
GEPS relies on GDPS to perform quality control (background check)
for all observations and bias correction for satellite radiance
observations
Page 19 – April-7-15
Dependencies between global systems
•
GDPS:
Current system (1-way dependence):
xb
Bgcheck+BC
xb, obs
4D-Var
xa
GEM (9h fcst)
xb
obs
GEPS:
•
GDPS:
GEPS:
•
xb
EnKF
xa
GEM (9h fcst)
xb
With EnVar (2-way dependence):
xb
xb
Bgcheck+BC
xb
xb, obs
EnVar
obs
EnKF
xa
xa
GEM (9h fcst)
GEM (9h fcst)
xb
xb
2-way dependence (EnVar uses EnKF ensemble of background
states) increases complexity of overall system  2 systems have to
be run simultaneously
Page 20 – April-7-15
Dependencies between global systems
•
GDPS:
Current system (1-way dependence):
xb
Bgcheck+BC
xb, obs
4D-Var
xa
GEM (9h fcst)
xb
obs
GEPS:
•
GDPS:
GEPS:
•
GDPS:
xb
EnKF
xa
GEM (9h fcst)
xb
With EnVar (2-way dependence):
xb
xb
Bgcheck+BC
xb, obs
xb
EnVar
obs
EnKF
xa
xa
GEM (9h fcst)
GEM (9h fcst)
xb
xb
Another possibility with EnVar (1-way dependence):
xb
Bgcheck+BC
xb, obs
EnVar
xa
GEM (9h fcst)
xb
xb
GEPS:
xb
Bgcheck+BC
xb, obs
EnKF
Page 21 – April-7-15
xa
GEM (9h fcst)
xb
EnVar formulation: Preconditioning
•
Preconditioned cost function formulation at Environment Canada:
J (ξ ) 
1
ξ ξ
T
2
•
1
2
( H 4D  x b   H  x ξ   y ) R
T
( H 4D  x b   H  x ξ   y )
In EnVar with hybrid covariances, the control vector (x) is
composed of 2 vectors:
 ξ 1ens 
 ξ nmc 
ξ   

ξ
 ens 
•
1


 
 ξ N en s 
 ens 
ξ ens   

The analysis increment is computed as (ek is k’th ensemble
perturbation divided by sqrt(Nens-1) ):
 x ξ    nmc B nmc ξ nmc   ens
1/ 2
1/2
1/ 2
N ens


1/2
k
e k  L ξ ens


B   nmc B nmc   ens
N en s
 e
T

e L
k k
k 1
k 1
•
Better preconditioned than original “alpha control vector”
formulation (with L-1 and 1/β in background term of J)
•
Most, but maybe not all, applications of the approach use the better
preconditioned formulation
Page 22 – April-7-15
Single observation experiments
Difference in temporal covariance evolution
EnVar
• radiosonde
•
•
•
temperature
observation at
500hPa
observation at
beginning of
assimilation
window (-3h)
with same B,
increments very
similar from
4D-Var, EnKF
contours are
500hPa GZ
background
state at 0h
(ci=10m)
+
+
contour plots at 500 hPa
+
+
Page 23 – April-7-15
Single observation experiments
Difference in temporal covariance evolution
EnVar
• radiosonde
•
•
•
temperature
observation at
500hPa
observation at
middle of
assimilation
window (+0h)
with same B,
increments very
similar from
4D-Var, EnKF
contours are
500hPa GZ
background
state at 0h
(ci=10m)
+
+
contour plots at 500 hPa
+
+
Page 24 – April-7-15
Single observation experiments
Difference in temporal covariance evolution
EnVar
• radiosonde
•
•
•
temperature
observation at
500hPa
observation at
end of
assimilation
window (+3h)
with same B,
increments very
similar from
4D-Var, EnKF
contours are
500hPa GZ
background
state at 0h
(ci=10m)
+
+
contour plots at 500 hPa
+
+
Page 25 – April-7-15
Experimental results:
Configuration
EnVar tested in comparison with new version of forecast
system currently being implemented in operations:
• model top at 0.1hPa, 80 levels
• model has ~25km grid spacing
• 4D-Var analysis increments with ~100km grid spacing
EnVar experiments use ensemble members from new
configuration of EnKF:
• 192 members every 60min in 6-hour window
• model top at 2hPa, 75 levels
• model ~66km grid spacing  EnVar increments ~66km
Page 26 – April-7-15
EnVar uses Hybrid Covariance Matrix
Model top of EnKF is lower than GDPS
Therefore, EnVar not
expected to be better than
3D-Var above ~10-20hPa
Also tested 75% Bens and
25% Bnmc in troposphere,
but results slightly worse
pressure
Bens and Bnmc are averaged in troposphere ½ & ½,
tapering to 100% Bnmc at and above 6hPa (EnKF model
top at 2hPa)
Also did preliminary tests
with a full outer loop, but
degraded the results
Page 27 – April-7-15
Bens scale factor
Bnmc scale factor
scale factor
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Radiosonde verification scores – 6 weeks, Feb/Mar 2011
U
GZ
T-Td
|U|
T
EnVar vs. 3D-Var
120h forecast
North extra-tropics
Page 28 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Radiosonde verification scores – 6 weeks, Feb/Mar 2011
U
GZ
T-Td
|U|
T
EnVar vs. 3D-Var
120h forecast
North extra-tropics
Page 29 – April-7-15
U
GZ
T-Td
|U|
T
EnVar vs. 4D-Var
120h forecast
North extra-tropics
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Radiosonde verification scores – 6 weeks, Feb/Mar 2011
U
GZ
T-Td
|U|
T
EnVar vs. 3D-Var
120h forecast
South extra-tropics
Page 30 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Radiosonde verification scores – 6 weeks, Feb/Mar 2011
U
GZ
T-Td
|U|
T
EnVar vs. 3D-Var
120h forecast
South extra-tropics
Page 31 – April-7-15
U
GZ
T-Td
|U|
T
EnVar vs. 4D-Var
120h forecast
South extra-tropics
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Radiosonde verification scores – 6 weeks, Feb/Mar 2011
U
GZ
T-Td
|U|
T
EnVar vs. 3D-Var
24h forecast
Tropics
Page 32 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Radiosonde verification scores – 6 weeks, Feb/Mar 2011
U
GZ
T-Td
|U|
T
EnVar vs. 3D-Var
24h forecast
Tropics
Page 33 – April-7-15
U
GZ
T-Td
|U|
T
EnVar vs. 4D-Var
24h forecast
Tropics
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
EnVar vs. 3D-Var
48h forecast, global domain
no EnKF
covariances
transition
zone
½ EnKF and
½ NMC
covariances
U
RH
GZ
T
Page 34 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
EnVar vs. 3D-Var
48h forecast, global domain
no EnKF
covariances
transition
zone
no EnKF
covariances
transition
zone
½ EnKF and
½ NMC
covariances
½ EnKF and
½ NMC
covariances
EnVar vs. 4D-Var
U
RH
U
RH
GZ
T
GZ
T
Page 35 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
EnVar vs. 3D-Var
120h forecast, global domain
no EnKF
covariances
transition
zone
½ EnKF and
½ NMC
covariances
U
RH
GZ
T
Page 36 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
EnVar vs. 3D-Var
120h forecast, global domain
no EnKF
covariances
transition
zone
no EnKF
covariances
transition
zone
½ EnKF and
½ NMC
covariances
½ EnKF and
½ NMC
covariances
EnVar vs. 4D-Var
U
RH
U
RH
GZ
T
GZ
T
Page 37 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
North extra-tropics
500hPa GZ correlation anomaly
EnVar vs. 3D-Var
EnVar vs. 4D-Var
Page 38 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
South extra-tropics
500hPa GZ correlation anomaly
EnVar vs. 3D-Var
EnVar vs. 4D-Var
This is the only significant
degradation seen vs. 4D-Var in
troposphere;
Not in radiosonde scores
because it originates from
south of 45°S (see next slide)
Page 39 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
120h forecast of 500hPa GZ - STDDEV
120h forecast of 500hPa GZ
STDDEV - South extra-tropics
Page 40 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, Feb/Mar 2011
Tropics
250hPa U-wind STDDEV
EnVar vs. 3D-Var
EnVar vs. 4D-Var
Page 41 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, July-Aug 2011
North extra-tropics
500hPa GZ correlation anomaly
EnVar vs. 3D-Var
EnVar vs. 4D-Var
Page 42 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, July-Aug 2011
South extra-tropics
500hPa GZ correlation anomaly
EnVar vs. 3D-Var
EnVar vs. 4D-Var
Page 43 – April-7-15
Forecast Results: EnVar vs. 3D-Var and 4D-Var
Verification against ERA-Interim analyses – 6 weeks, July-Aug 2011
Tropics
250hPa U-wind STDDEV
EnVar vs. 3D-Var
EnVar vs. 4D-Var
Page 44 – April-7-15
Experimental results:
4D-EnVar vs. 3D-EnVar
3D version of EnVar also tested: only uses EnKF flowdependent ensembles valid at the centre of the 6h
assimilation window, instead of every 60 minutes
throughout the window
3D-EnVar compared with:
• 4D-EnVar: impact of 4D vs 3D covariances, and
• 3D-Var: impact of flow dependent vs stationary (NMC)
covariances (both 3D)
Page 45 – April-7-15
Forecast Results: 4D-EnVar vs. 3D-EnVar
Verification against ERA-Interim analyses – 4 weeks, Feb 2011
North extra-tropics
500hPa GZ correlation anomaly
4D-EnVar vs. 3D-EnVar
3D-EnVar vs. 3D-Var
Page 46 – April-7-15
Forecast Results: 4D-EnVar vs. 3D-EnVar
Verification against ERA-Interim analyses – 4 weeks, Feb 2011
South extra-tropics
500hPa GZ correlation anomaly
4D-EnVar vs. 3D-EnVar
3D-EnVar vs. 3D-Var
Page 47 – April-7-15
Forecast Results: 4D-EnVar vs. 3D-En-Var
Verification against ERA-Interim analyses – 4 weeks, Feb 2011
Tropics
250hPa U-wind STDDEV
4D-EnVar vs. 3D-EnVar
3D-EnVar vs. 3D-Var
Page 48 – April-7-15
Conclusions
• Comparison of EnVar with 3D-Var and 4D-Var:
– EnVar produces similar quality forecasts as 4D-Var below
~20hPa in extra-tropics, significantly improved in tropics
– above ~20hPa, scores similar to 3D-Var, worse than 4D-Var;
potential benefit from raising EnKF model top to 0.1hPa
• EnVar is an attractive alternative to 4D-Var:
– like EnKF, uses full nonlinear model dynamics/physics to evolve
covariances; no need to maintain TL/AD version of model
– instead, makes use of already available 4D ensembles
– more computationally efficient and easier to parallelize than 4DVar for high spatial resolution and large data volumes
– computational saving allows increase in analysis resolution and
volume of assimilated observations; more computational
resources for EnKF and forecasts
Page 49 – April-7-15
Next Steps
• Finalize testing EnVar with goal of replacing 4D-Var in
operational global prediction system during 2013 in
combination with other changes:
– GEM global model on 15 km Yin-Yang grid
– CALDAS: new surface analysis system based on an EnKF
– modified satellite radiance bias correction scheme that gives
conventional observations more influence on correction
– improved use of radiosonde and aircraft data
– additional AIRS/IASI channels and modified observation error
variances for all radiances
– increased resolution of EnKF 66km  50km
• Testing of EnVar in regional prediction system as
possible replacement of 4D-Var already started
Page 50 – April-7-15