Adjoint-based observation impact monitoring
at NRL-Monterey
1
Rolf H. Langland
Naval Research Laboratory – Monterey, Ca. USA
Gary G. Love, Nancy L. Baker
Fourth Workshop on Observing System Impact in NWP
WMO, Geneva, 19-21 May 2008
Outline of Talk
2
1. Methodology
2. Observation impact examples
3. On-line observation monitoring system
Goal for observation impact monitoring system
3
Develop a method to estimate the impacts of
all assimilated observations on a measure of
short-range forecast error in an operational
NWP system
Must be computationally efficient – run in nearreal-time for routine observation monitoring
Forecast Model and Analysis Procedure
4
Analysis procedure:
NAVDAS: NRL Atmospheric 3d-Variational Data
Assimilation System (0.5o lat-lon, 60 levels)
• Adjoint provides sensitivity to observations,
including moisture data
Forecast model:
NOGAPS: Navy Operational Global Atmospheric
Prediction System (T239L30)
• Adjoint run at T239L30, includes simplified vertical
mixing, large-scale precipitation
Observation Impact Concept
5
Observations move the forecast from the background
trajectory to the trajectory starting from the new analysis
In this context, “OBSERVATION
IMPACT” is the effect of
observations on the difference in
forecast error norms (ef - eg)
xg
eg
xf
ef
xt
xb
xa
t= -6 hrs
t=0
6 hr assimilation window
Langland and Baker (Tellus, 2004)
t= 24 hrs
Forecast error norms and differences
6
Global
forecast error
total energy
norm (J kg-1)
Forecasts from 0600 and 1800 UTC
have larger errors
e30
Forecast errors on
background-trajectories
e24
Forecast errors on
analysis-trajectories
e24 – e30 (adjoint)
e24 – e30 (nonlinear)
Observation Impact Equation
1
K  [ HP H  R ] HP
T
T
b
 e gf
b
 e f eg 
  y  Hx b  , K 



x

x
b
 a
T
•
Dry or moist total energy forecast error norm, f = 24h, g = 30hr
•
Forecasts are made with NOGAPS-NAVDAS.
• Adjoint versions of NOGAPS and NAVDAS are used to calculate
the observation impact
•
The impact of observation subsets (separate channels, or
separate satellites) can be easily quantified
7
Observation impact interpretation
8
For any observation / innovation … using this error measure
e
30
24
< 0.0
the observation is BENEFICIAL
the effect of the observation is to make the error of
the forecast started from xa less than the error of the
forecast started from xb, e.g. forecast error decrease
e
30
24
> 0.0
the observation is NON-BENEFICIAL
e.g., forecast error increase
Total impact by instrument type – Jan2007
9
Dry TE Norm (150mb-sfc)
Dry TE Norm (150mb-sfc)
Impacts per-observation by instrument type
10
10e-5 J kg-1
10e-5 J kg-1
Percent of observations that produce
forecast error reduction (e24 – e30 < 0)
11
Impact for AMSU-A channels - NAVDAS-NOGAPS
12
Units of impact = J kg-1
1 – 31 Jan 2007, 00,06,12,18 UTC
Ch. peak near
5
0
4_15
11:
20mb
Channel
4
5
6
7
10:
50mb
4_16
8
9
10
11
-5
Beneficial
-10
-15
NOAA 15
NOAA 16
NOAA 18
-20
-25
-30
9:
90mb
4_18
8: 150mb
4_19
7: 250mb
6:5_15
350mb
5:5_16
600mb
4: surface
5_18
5_19
6_15
6_16
6_18
6_19
Why do some “good data” have non-beneficial impact ?
13
• Observation and background error statistics for data
assimilation cannot be precisely specified
• This implies a statistical distribution of beneficial and nonbeneficial observation impacts
• Assimilating the global set of observations improves the
analysis and forecast, even though 40-50% of observation
data are non-beneficial in any selected assimilation
Information about the impact of individual
observations and subsets of observations can
be used to improve the data assimilation and
observation selection procedures
Impact of AMSU-A radiance data
14
Non-beneficial
Sum = - 0.906 J kg-1
Beneficial
86,308 observations
Observations assimilated at 0000 UTC 4 May 2008
Interpretation of observation impact
15
• Non-beneficial impacts: look for data QC issues,
instrument accuracy, specification of observation and
background errors, bias correction, or model
(background) problems …
• Beneficial impacts: associated with heavily
weighted observations in sensitive regions; “good”, but
extreme impacts indicate need for greater observation
density …
Best strategy: many observations which produce
small to moderate impacts, not few observations
which produce large impacts …
Example 1: AMV impact problem
16
Date: Jan-Feb 2006
Issue: Non-beneficial
impact from MTSAT
AMVs at edge of
coverage area
Action Taken: Data
provider identified
problem with wind
processing algorithm.
Restricting SSEC MTSAT Winds
500 mb Height Anomaly Correlation
17
Southern Hemisphere
Restricted Winds
Control
February 16 – March 27, 2006
Example 2: Ship data problem
18
Date: Jan-Feb 2006
Issue: Some ship data
having non-beneficial impact
Actions Taken: Ship ID
blacklist implemented;
increase wind observation
error for ship data
(previously was equal to
radiosonde surface wind
error)
SEA ARCTICA – one of
the “problem” ships
Example 3: AMSU-A over
land surface
19
Date: Jan-Feb 2006
Issue: Some AMSU-A
channels over-land
surfaces produce nonbeneficial impact
Action Taken:
Investigate bias
correction dependence
on land surface
temperature
AMMA RAOB Temperature Ob Impacts
May-Oct 2006
20
BANAKO:61291 SUM= -0.5755 J kg-1
TAMANASET:60680 SUM= -0.2791 J kg-1
AMMA RAOB Summary Ob Impacts
Aug 2006 SOP
21
Largest Fcst Error Reductions
< -0.10 J kg-1
Fcst Degradations
Current Uncertainty in Analyzed 500mb
Temperature – Operational Systems
22
RMSD
Current Radiosonde Distribution
23
Applications of observation impact information
24
• Adjoint-based observation impact information is a valuable
supplement to “conventional” data impact studies (OSEs, OSSEs)
• Provides quantitative information about every observation that is
assimilated and spatial patterns in observation impact
• Identifies possible problems with NAVDAS (observation and
background error, bias correction, etc.)
• Information is relevant to QC issues and daily monitoring of
observations in operational data assimilation
On-line observation Impact monitor
www.nrlmry.navy.mil/ob_sens/
25
Time-series of observation impact
www.nrlmry.navy.mil/ob_sens/
26
Menu for upper-air satellite wind plots
www.nrlmry.navy.mil/ob_sens/
27
MTSAT: 300-500 hPa wind obs
www.nrlmry.navy.mil/ob_sens/
28
30-day cumulative impact
30-day mean innovation
MTSAT: 300-500 hPa wind obs
www.nrlmry.navy.mil/ob_sens/
29
30-day cumulative impact
30-day mean wind speed
MDCRS Level-Flight: wind obs
www.nrlmry.navy.mil/ob_sens/
30
30-day cumulative impact
NAVDAS-AR
8 Apr - 7 May 2008
00UTC observations
31
4d-VAR
0
-0.5
-1
Impact perobservation
(10-5 J kg-1)
-1.5
-2
-2.5
-3
-3.5
hours before
Analysis Time
hours after
-4
-2.5
-1.5
-0.5
In-Situ
0
0.5
Satellite
1.5
2.5
Summary
32
• An adjoint-based system has been developed for daily
(currently for 00UTC) monitoring of all observations used in data
assimilation (3d-VAR and 4d-VAR) at NRL-FNMOC
• Computational cost is slightly less than the regular data
assimilation and (24h) nonlinear forecast
• Information can be used for observation quality-control and
improvement of the data assimilation procedure – valuable
supplement to data-denial or data-addition experiments
ObSens Monitor Design
33
• Pre-Processing
– Bin statistics into 2.5 degree grid
– Sort data combinations, totals and groups
• Web-Processing
– Provide top-level overviews and time lines
– Present comprehensive menus of choices
– Render on-demand maps, charts and time lines
• Archiving
– Zip 90-day old data, unzip as needed
ObSens Pre-Processing
34
•
•
•
•
•
Ingest obsens_52.$dtg
Calculate stats for $dtg, and 30-day and 1-yr stats
Calculate impact average and sum by category
Create category bar chart and time bar chart
Create 2.5 degree binned grids for $dtg
By data category, channel, variable type as appropriate
For seven hPa pressure levels:
sfc-901, 900-801, 800-701, 700-501, 500-301, 300-101, 100-10
obsens_52.$dtg
rd_obsens
(C-code)
AWK
script
stats
grids
GrADS
script
ObSens Web-Processing
35
•
Display top page with bar and time charts
•
Show other bar and time charts on mouse roll-over
•
Present menus for Observation Category
•
For selected Observation Category, present:
Combinations of platform, pressure, channel, variable, etc.
parameter: counts, ob value, innovation, impact, sensitivity
geo area: global, northern hemisphere, southern hemisphere
Totals for All platforms, pressures, channels, variables, etc.
Groups for classes satellite/aircraft types: All GOES, SSEC, Ascending, etc.
•
On Demand
–
–
Calculate 30-day/1-year grid stats
Create map plots and time lines
Menu page
html
Javascript
Tcl cgi
GrADS script
grids
Display page
html
ObSens Archiving
36
•
•
•
•
•
Compress data grids over 90-days old
Sparse grids compress 50:1
Uncompress data on-demand: ~ 2 sec/grid
Leave on-demand data uncompressed
Assuming future interest in uncompressed data
grids
GrADS script
Zip
Unzip and
open file
grids
Data Assimilation Equation
37
ANALYSIS
BACKGROUND
(6h) FORECAST
K
1
xa  xb  Pb H [HPb H  R] (y  Hxb )
T
Temperature
Moisture
Winds
Pressure
T
OBSERVATIONS
Adjoint of Assimilation Equation
38
Sensitivity to Observations:
J
J
T
1
 [HPb H  R ] HPb
y
xa
KT
Adjoint of forecast
model produces
sensitivity to x
a
Sensitivity to Background:
J
J
T J

H
x b xa
y
Baker and Daley 2000 (QJRMS)
Example 3: Isolated aircraft tracks
39
Date: First noticed Jan 05,
ongoing in several regions
Issue: aircraft flies in jet max
eastbound, outside of jet max
westbound: observation error
representativeness problem ?
Action Taken: Possible
change to observation error
AMDAR Level Flight
Hong Kong - LAX
Non-beneficial
observations
Example 4: QC for
land observing
stations
KQ-MIL
Stations
AK-METAR
Stations
40
Conventional
Land Stations
Date: Jan-Feb 2005
Issue: Land station observation problems linked to high
elevation and cold surface temperatures (METAR), also
problems with station elevation metadata (MIL, conventional)
Actions Taken: Selected stations blacklisted, data flagged if
stations above 740m, or above 300m and background
temperature below -15°C