World Meteorological Organization
Working together in weather, climate and water
WMO
Report from the Fifth WMO Workshop on
the Impact of Various Observing Systems
on NWP
Sedona, May 22-25 2012
Lars Peter Riishojgaard
Chair, OPAG-IOS, WMO Commission for Basic Systems
Lidar Working Group, Boulder, Oct 16-17 2012
Overview
• WMO context
– RRR, WMO Impact Workshops, role of
NWP Impact Studies
• Sample results from Fifth WMO Impact
Workshop
• Select key conclusions from Sedona
October 16-17, 2012
Lidar Working Group, Boulder CO
2
RRR in WMO
• Rolling Review of Requirements is the process
for developing WMO guidance on performance
versus requirements for observing systems
• Within the WMO Structure, CBS/OPAG-IOS has
overall responsibility of the RRR
– ET-EGOS
• Application Area (12) Points of Contact
• Rapporteur on Scientific Evaluation of Impact Studies by
NWP Centers
• Ad hoc interfaces to other TC’s and their Expert Teams
– Other OPAG-IOS Expert Teams
– WMO Impact Workshops
October 16-17, 2012
Lidar Working Group, Boulder CO
3
WMO Workshops on the Impact of Various
Observing Systems on NWP
Five Workshops held so far:
• 1st - Geneva, 1997
• 2nd – Toulouse, 2000
• 3rd – Alpbach, 2004
• 4th – Geneva, 2008
- Workshop Report available on
http://www.wmo.int/pages/prog/www/OSY/Reports/NWP-4_Geneva2008_index.html
• 5th – Sedona (AZ, USA), May 22-25 2012
-Workshop Report will be available late Oct 2012, presentations are on
http://www.wmo.int/pages/prog/www/OSY/Reports/NWP-5_Sedona2012.html
Workshops aim to bring together major NWP centers and
representatives from the research community to discuss the contribution
to forecast skill of various elements of the global observing system;
guidance to participants provided well in advance of Workshop itself.
October 16-17, 2012
Lidar Working Group, Boulder CO
4
Fifth WMO NWP Impact
Workshop, Sedona, May 22-25
• CBS Ext (10) in Namibia: US extended an invitation to host the 5th
WMO Impact Workshop in 2012
– Erik Andersson (ECMWF), R-SEIS, SOC
– Lars Peter Riishojgaard (JCSDA), CBS OPAG-IOS Chair, LOC
October 16-17, 2012
Lidar Working Group, Boulder CO
5
Scientific Organizing
Committee
•
•
•
•
•
•
•
•
Erik Andersson, ECMWF, (Chair), (R-SEIS)
Carla Cardinali, ECMWF (THORPEX)
John Eyre, Met Office UK (CBS/ET-EGOS)
Ron Gelaro, NASA/GSFC (THORPEX)
Miroslav Ondras, WMO Secretariat
Florence Rabier, Meteo-France (THORPEX)
Lars Peter Riishojgaard, JCSDA (CBS/OPAG-IOS)
Yoshiaki Sato, JMA (R-SEIS)
• Workshop funding provided by WMO (OBS and
THORPEX), NASA and NOAA (GOES-R Program Office)
Sedona in brief
• The largest WMO Impact Workshop so far:
– 3½ days
– 59 participants from 13 countries
– 40 presentations distributed in three sessions
• Ample discussion time during and after the sessions
• Very broad attendance from NWP community
• Space agencies and other NWP and observing
system managers also represented
– They are keenly aware of the power of NWP diagnostics as
aids for decision making
October 16-17, 2012
Lidar Working Group, Boulder CO
7
Role of NWP diagnostics
• NWP is a foundational activity for most weather and
climate applications
• Objective, quantitative metrics:
– NWP poses a well-defined prediction problem with
a “right” answer
• (and an infinity of wrong ones)
– Well-defined measures for quality of output
– Well-established methodologies for assigning
merit (or blame) to individual observing systems
October 16-17, 2012
Lidar Working Group, Boulder CO
8
NWP Diagnostics (I), OSE
•
•
•
•
OSEs (Observing System
Experiments) are based on data
denial (or addition):
– 1. Run a control with
operational data
– 2. Add (or subtract) data to
be tested
– 3. Compare
Impact focuses on the medium
to long range
Results show the impact of
withdrawing (or adding) certain
data
Absolute metric (score of perfect
forecast has a known value)
October 16-17, 2012
Jung et al., WMO Impact Workshop in Sedona, May 2012
Lidar Working Group, Boulder CO
9
NWP Diagnostics (II), FSO
• FSO (Forecast Sensitivity to
Observations) are based on the
adjoint of the model/analysis
system or an ensemble approach
• Approach focuses exclusively on
the short (quasi-linear) range
• Results show the impact of
observations in the presence of
all other observations
• Relative metric (numerical value
not meaningful by itself)
October 16-17, 2012
Gelaro et al, Fifth WMO Impact Workshop, Sedona 2012
Lidar Working Group, Boulder CO
10
1. Satellite data
October 16-17, 2012
Lidar Working Group, Boulder CO
11
Lidar Working Group, Boulder CO
12
1
3
Monthly number of observations used in the global model
IASI
SSMI/S
SEVIRI
GPS-RO
AIRS
SCAT
SSMI
ATOVS
2002
2004
WMO Workshop on the Impact of Various Observing Systems on
2006
2007
2008
Sedona – 22-25 May 2012
2010
2011
Forecast impact experiment from Dec. 2010 to Jan. 2011
Impact
Impact / Obs. number
AMSU-A
WMO Workshop on the Impact of Various Observing Systems on NWP
Sedona – 22-25 May 2012
FNMOC and GMAO Observation Impact Monitoring
Current Operations
Gelaro et al. , Sedona May 2012
GMAO:
AMSU-A
http://www.nrlmry.navy.mil/obsens/fnmoc/obsens
_main_od.html
http://gmao.gsfc.nasa.gov/products/forecasts/syste
ms/fp/obs_impact/
much larger relative impact of AMVs in Navy system
October 16-17, 2012
Lidar Working Group, Boulder CO
15
No Satellite / No Conventional Data
(JCSDA w/ NCEP GFS)
Northern Hemisphere
Southern Hemisphere
Strong impact of satellite data
Jung, 5th WMO Impact Workshop, Sedona 2012
overall in both hemispheres
October 16-17, 2012
Lidar Working Group, Boulder CO
16
No AMSU-A / No MHS
Northern
Hemisphere
Southern
Hemisphere
Jung, 5th WMO Impact Workshop, Sedona 2012
October 16-17, 2012
Lidar Working Group, Boulder CO
17
No RAOBs / No Aircraft
Northern
Hemisphere
Southern
Hemisphere
Strongest signal of any
individual GOS component!
October 16-17, 2012
Jung, 5th WMO Impact Workshop, Sedona 2012
Lidar Working Group, Boulder CO
18
2. Reemergence of the
importance of conventional
observations
October 16-17, 2012
Lidar Working Group, Boulder CO
19
October 16-17, 2012
Lidar Working Group, Boulder CO
20
AMDAR coverage (6-hour period)
October 16-17, 2012
Lidar Working Group, Boulder CO
21
Impact summary
Forecast error
reduction
Winds
Total impacts
Winds
Impacts per 1 obs
All observation types have positive impacts on average.
For the total impact, 1: aircraft, 2: AMSU-A, 3: radiosonde, 4: IASI, 5: GPSRO
For impact per 1 obs., 1: radiosonde, 2: GPSRO, 3: aircraft, 4: Scatterometer wind, 5:
marine surface observation
Ensemble-based FSO diagnostics, NCEP GFS, Ota et al., WMO, Sedona, May 2012
5/22/2012
5th WMO Workshop on the Impact of Various
Observing Systems on NWP (Sedona, AZ, USA)
22
Radiosonde and aircraft
5/22/2012
125-250 hPa
600-800 hPa
125-250
hPa
5th WMO Workshop on the Impact of Various
600-800 hPa
Observing Systems on NWP (Sedona, AZ, USA)
23
Radiosonde and aircraft
Radiosonde
Aircraft
Radiosonde observations on mid- to lower troposphere have larger impacts compared to
Aircraft: Upper troposphere;
the aircraft observations.
this is where we have data!
RAOBs: Mid- to lower troposphere; as
expected based on sensitive structures
5/22/2012
5th WMO Workshop on the Impact of Various
Observing Systems on NWP (Sedona, AZ, USA)
24
Radiosonde impacts
Total impacts of
radiosonde (12UTC
October 21 to
06UTC October 28)
Most observations
have positive
impacts on
average
Relatively large
impacts for East
Asia, Western US,
Canada, and South
America.
5/22/2012
5th WMO Workshop on the Impact of Various
Observing Systems on NWP (Sedona, AZ, USA)
25
Aircraft impacts
Total impacts, moist total energy
Impacts per 1 obs (250-125 hPa)
Aircraft observations over US, Europe and East Asia have large positive impacts.
The impact of aircraft observations is extremely large over US, however impact per 1
observation is small.
5/22/2012
5th WMO Workshop on the Impact of Various
Observing Systems on NWP (Sedona, AZ, USA)
26
300mb Wind Speed (2010) GFS / ECMWF
Langland, Sedona 2012
Root-Mean Square of Analysis Differences: 300mb Wind Speed
27
Langland and
Maue 2011
2010
Note the very significant effect of in-situ wind observations:
Radiosondes and Commercial Aircraft
2009
2011
2008
October 16-17, 2012
Lidar Working Group, Boulder CO
ms-1
3. Some conclusions (with
LWG implications)
October 16-17, 2012
Lidar Working Group, Boulder CO
28
Conclusions
(from presentation by John Eyre)
• Good progress on assimilation of IR sounder radiances
• cloud-affected and over land
• more progress expected
• implications for balance of observing system over land
• Dangers of “observation-free zones” in data assimilation
• examples: mesospheric temperature, upper tropospheric humidity
• GPS-ZTD - significant impact on forecasts of surface variables
• AMV and scatterometer impacts:
Don’t forget the high latitudes!
• Positive impact from all AMV types
• Polar AMV have more impact per ob, but fewer of them
• FSO statistics show which obs compensate when AMV/scatt denied
© Crown copyright 2007
Observation impact in global NWP
(slide shown by Erik Andersson, Workshop Chair, at EMS, September 2012)
 The highest ranked contributors for the forecast error reductions are:
 AMSU-A, AIRS/IASI, radiosonde, aircraft, AMVs
 GPS-RO also has substantial impact, but the data volume is declining
approaching the end of COSMIC lifetime.
 Several satellite sensors contribute to forecast skill. There is not a
single, dominating one

These are (or include)
More complementarity is seen, compared to
previous
years.
wind
measurements!
 The GOS has become more resilient, but this resilience is threatened by
expected decline of the operational polar orbiting satellites
 When one observation type is missing or removed the contribution of
other systems tend to increase without fully compensating
Slide 30
ECMWF
Workshop Recommendations
(slide shown by Erik Andersson, Workshop Chair, at EMS, September 2012)
 Augment the profiling network e.g. by extending coverage
of ascending and descending aircraft observations to
regional airports
 There is a need to invest in enhanced wind observations
in the tropics and over the oceans especially.
 Study observation impact that is more closely related to
high-impact weather (including TCs) and service delivery
to customers and forecast users
 Encouraged studies of impact per observing system or
per observation linked to their cost
 Define appropriate impact metrics for
 humidity and
 regional NWP including precipitation and other surface
weather elements
Slide 31
ECMWF
General Conclusions from the Fifth WMO
Impact Workshop in Sedona, May 2012
• Modern, 4-dimensional data assimilation methods (4D-VAR,
ENKF) have led to greatly improved use of data, especially of
– Asynoptic data (e.g. aircraft, satellite observations)
– Observations with complex relationships between measured
and model variables (satellite radiances, GPSRO, radar,…)
• Broad consensus about highest-ranking contributors to forecast
skill, but not necessarily about their ranking order:
– AMSU-A (microwave temperature sounder)
– AIRS/IASI (hyper-spectral infrared sounders)
– Radiosondes
– Aircraft observations
– Atmospheric motion vectors (feature tracking satellite winds)
• Investment in additional winds observations is a high priority
October 16-17, 2012
Lidar Working Group, Boulder CO
32