Hybrid variational-ensemble data assimilation
(HVEDAS)
Daryl T. Kleist1,3
Dave Parrish1, John Derber1,Jeff Whitaker2, Kayo Ide3, & Xuguang Wang4
1NOAA/NCEP/EMC, 2NOAA/ESRL, 3University
of Maryland-College Park, 4University of Oklahoma
High Impact Weather Working Group Workshop
24 February, 2011 – Norman, OK
1
Variational Data Assimilation
J Var
   B   
1 '
x  x
2
'
T
1
Var
1 '
x  y o  Hx '
2
'
 R y
T
1
'
o

 Hx '  J c
J : Penalty (Fit to background + Fit to observations + Constraints)
x’ : Analysis increment (xa – xb) ; where xb is a background
BVar : Background error covariance
H : Observations (forward) operator (eg. CRTM)
R : Observation error covariance (Instrument +
representativeness)
yo’ : Observation innovations
Jc : Constraints (physical quantities, balance/noise, etc.)
B is typically static and estimated a-priori/offline
2
Motivation
• Current background error covariance (applied
operationally) in VAR
– Isotropic recursive filters
– Poor handle on cross-variable covariance
– Minimal flow-dependence added
• Implicit flow-dependence through linearization in
normal mode constraint (Kleist et al. 2009)
• Flow-dependent variances (only for wind, temperature,
and pressure) based on background tendencies
– Tuned NMC-based estimate (lagged forecast pairs)
3
Hybrid Variational-Ensemble
• Incorporate ensemble perturbations directly into
variational cost function through extended control
variable
– Lorenc (2003), Buehner (2005), Wang et. al. (2007), etc.


J x 'f ,   f
 
 



T
1 ' T 1 '
1 T
1
x f B x f   e   L1    y 'o  Hx 't R 1 y 'o  Hx 't
2
2
2

K
x  x    k  x ek
'
t
'
f
1
f


k 1

1
e
1
f & e: weighting coefficients for fixed and ensemble covariance respectively
xt: (total increment) sum of increment from fixed/static B (xf) and ensemble B
e
k: extended control variable; xk :ensemble perturbation
L: correlation matrix [localization on ensemble perturbations]
4
Why Hybrid?
VAR
EnKF Hybrid References
(3D, 4D)
Benefit from use of flow
dependent ensemble
covariance instead of static B
x
Hamill and Snyder 2000;
Wang et al. 2007b,2008ab,
2009b, Wang 2011; Buehner
et al. 2010ab
Robust for small ensemble
x
Wang et al. 2007b, 2009b;
Buehner et al. 2010b
Better localization for
integrated measure, e.g.
satellite radiance
x
Campbell et al. 2009
x
Easy framework to add
various constraints
x
x
Framework to treat nonGaussianity
x
x
Use of various existing
capabilities in VAR
x
x
Acknowledgement: Xuguang Wang
5
Hybrid with (global) GSI
• Control variable has been implemented into GSI 3DVAR*
– Full B preconditioning (Wang 2011)
• Working on extensions to B1/2 preconditioned minimization options
– Spectral filter for horizontal part of A
• Eventually replace with (anisotropic) recursive filters
– Recursive filter used for vertical
– Dual resolution capability
• Ensemble can be from different resolution than background/analysis
(vertical levels are the exception)
– Various localization options for A
• Grid units or scale height
• Level dependent (plans to expand)
– Option to apply TLNMC (Kleist et al. 2009) to analysis increment
 ' K

x  C xf   k  x ek 
k 1


'


6
*Acknowledgement: Dave Parrish for original implementation of extended control variable
Single Observation
7
Single 850mb Tv observation (1K O-F, 1K error)
Single Observation
8
Single ps observation (-2mb O-F, 1mb error) near center of Hurricane Ike
Single Observation
9
Single 850mb zonal wind observation (3 m/s O-F, 1m/s error) in Hurricane Ike circulation
Dual-Res Coupled Hybrid
recenter analysis ensemble
member 1
forecast
member 2
forecast
EnKF
member update
member 3
forecast
high res
forecast
Previous Cycle
GSI
Hybrid Ens/Var
high res
analysis
Current Update Cycle
member 1
analysis
member 2
analysis
member 3
analysis
Hybrid Var-EnKF GFS experiment
•
Model
– GFS deterministic (T574L64; post July 2010 version – current operational version)
– GFS ensemble (T254L64)
• 80 ensemble members, EnKF update, GSI for observation operators
•
Observations
– All operationally available observations (including radiances)
– Includes early (GFS) and late (GDAS/cycled) cycles as in production
•
Dual-resolution/Coupled
• High resolution control/deterministic component
–
Includes TC Relocation on guess
• Ensemble is recentered every cycle about hybrid analysis
–
•
Discard ensemble mean analysis
Satellite bias corrections
– Coefficients come from GSI/VAR
•
Parameter settings
• 1/3 static B, 2/3 ensemble
• Fixed localization: 800km & 1.5 scale heights
•
Test Period
– 15 July 2010 – 15 October 2010 (first two weeks ignored for “spin-up”)
11
500 hPa Anom.Corr.
Northern Hemisphere
Southern Hemisphere
12
Forecast Fits to Obs (Tropical Winds)
Forecasts from hybrid analyses fit observation much better.
13
Tropical Wind Errors (72hr)
Verification against ‘consensus analysis’ [ECMWF, NCEP, UKMET]
14
Hybrid versus 3DVAR Track Error
15
Hybrid versus 3DVAR Intensity Bias
16
HVEDAS (3D) for GDAS/GFS
• Prototype dual-resolution, two-way coupled hybrid Var/EnKF
system outperforms standard 3DVAR in GFS experiments
– 2010 Hurricane Season (August 15 through October 31 2010) run offsite
– Emphasis on AC, RMSE, TC Tracks
• Plan underway to implement into GDAS/GFS operationally
– Target: Spring 2012 (subject to many potential issues)
• Porting of codes/scripts back to IBM P6
• Cost analysis (will everything fit in production suite?)
• More thorough (pre-implementation) testing and evaluation
– More test periods (including NH winter)
– Other/more verification metrics
• Potential moratorium associated with move to new NCEP facility
• Issues
– Weighting between ensemble and static B
– Localization
– How should EnKF be used within ensemble forecasting paradigm?
17
HVEDAS Extension to other Applications
• Expand hybrid to 4D
– Hybrid within ‘traditional 4DVAR’ (with adjoint)
– Pure ensemble 4DVAR (non-adjoint)
– Ensemble 4DVAR with static B supplement (non-adjoint)
• Non-GFS applications in development
–
–
–
–
–
NASA GEOS-5 (GMAO)
NAM (Dave Parrish, others)
Hurricanes/HWRF (Mingjing Tong, HFIP, many collaborators)
Storm-scale initialization (Jacob Carley, collaborators)
RR (Xuguang Wang, Ming Xue, Stan Benjamin, Jeff Whitaker, Steve
Weygandt, others)
• NCEP strives to have single DA system to develop, maintain,
and run operationally (global, mesoscale, severe weather,
hurricanes, etc.)
– GSI (including hybrid development) is community code supported through
DTC
– EnKF used for GFS-based hybrid being expanded for use with other
applications
18
Backup Slides
19
Applications of hybrid DA
•
•
•
•
WRFVAR based hybrid (Wang et al 2008ab, MWR)
Application for hurricane track forecasts (Wang 2011, WAF)
For radar data assimilation for hurricanes, Li et al.
FAA application with GSI-based hybrid, Zhu et al.
Li et al. 2011
20
GSI-based ‘ensemble 4DVAR’
•
•
•
A natural extension of 3DVAR-based hybrid with no static covariance B.
Temporal evolution of the increment is obtained through evolved ensemble
perturbations in the assimilation window.
ens4dvar is a 4DVAR with no need for tangent linear and adjoint of the
forecast model (Liu et al. 2009).
Lei, Wang et al. 2011
21
Temporal evolution of the error covariance
Temp.
t-3h
t
t+3h
Height
t-3h
Downstream
impact
t
t+3h
Upstream
impact
22