Observing System Simulation
Experiments at CIMSS
By
CIMSS/OSSE Team :
Bob Aune ; Paul Menzel ; Jonathan Thom
Gail Bayler ; Chris Velden ; Tim Olander
and Allen Huang
Cooperative Institute for Meteorological Satellite
Studies
University of Wisconsin
7 June, 1999
MESOSCALE OBSERVING SYSTEM
SIMULATION EXPERIMENTS
(OSSE)
GOAL
To assess the contribution of environmental
observing systems to operational mesoscale
numerical weather forecasts in a controlled
software environment.
Future observing systems can be tested using
projected instrument characteristics.
Current Research Plan
Initial Impact Study: Geostationary Interferometer
Simulated Products from “Nature” Atmosphere:
Soundings (T, Td)
Winds (Cloud drift / Water Vapor)
Soundings plus Winds
Soundings, Winds, Conventional Data
Radiances
Derived Products from Simulated Radiances
Retrievals (T, Td, derived products)
Winds (automated wind algorithm)
Direct radiance assimilation
PROCEDURES
Observations are synthesized from forecasts
generated by a numerical prediction model that
has a known history calibrated against reality.
These forecasts represent truth and are referred to
as the "nature" atmosphere.
Synthesized observations must mimic, as close as
possible, observations from the real observing
system that is being evaluated.
Synthesized observations are assimilated into an
assimilation system that is independent of the
"nature" model.
Pilot Experiment
HYPOTHESIS:
Information from a geostationary-based
interferometer will significantly improve the
accuracy of numerical weather forecasts over the
current geostationary radiometer.
Temperature and moisture retrievals are
simulated by superimposing estimated
observation errors on the "true" profiles
generated by the "nature" run.
OSSE Design
An OSSE can be subdivided into four basic steps:
1) Generate a "nature" atmosphere
2) Compute synthetic observations
3) Assimilate the synthetic observations
4) Assess the impact on the resulting forecast
Each step is performed with the goal of minimizing any
external influences, which may compromise the value of the
synthesized observations, the assimilation process, or the
results of the numerical forecasts.
This OSSE is being conducted over a limited area
domain. The influence of pre-specified lateral boundaries
must be minimized.
1. Generate a "nature" atmosphere
 University of Wisconsin, Nonhydrostatic Modeling System
 The horizontal domain is chosen to be as large as practical
to isolate the influence of the pre-defined lateral boundary
conditions. Horizontal resolution = 60 km.
 Boundary conditions: NCEP Eta forecast model, NCEP 104
grid.
 Ideally, the "nature" atmosphere should be two to four times
the resolution of the simulated observing system.
 The model vertical resolution is chosen to be a minimum of
two-times the resolution of the observing system to be
simulated. Vertical levels = 38.
 A 12hr forecast was generated to allow the model to "spin
up".
UW-NMS Domain in the Eta 104 grid
“Nature” Calibration
2. Simulate observations
 Temperature and moisture profiles from the "true"
atmosphere are modified using realistic observation
errors.
 Profiles of temperature and moisture are generated at
hourly intervals over the 12-hour analysis period.
 A cloud mask is used to simulate gaps in the coverage.
NMS Grid Locations with Cloud Mask
RAOBS
ACARS
Surface
Profilers
Observation Errors
Ob type
Count
RAOB Temperature
98*
RAOB Height
98*
RAOB Dewpoint
98*
RAOB Wind
98*
SFC Temperature
~600
SFC Dewpoint
~600
SCF Wind
~600
ACARS Temperature ~3000
ACARS Wind
~3000
Profiler Wind
31*
GEO-R Temperature ~3500*
GEO-I Temperature ~4000*
GEO-R Mixing ratio
~3500*
GEO-I Mixing ratio
~4000*
(* indicates a profile)
RMS Error
0.3 C
8-32 m
0.5 C
.8 - 1.3 m/s
0.3 C
0.5 C
0.4 m/s
1.0 C
1.0m/s
1.0 m/s
1.9 - 2.1 C
~1.0 C
~1.0 g/Kg
~0.5 g/Kg
BIAS
.27 C
0.1 C
.053 g/Kg
.02 g/Kg
Satellite Wind Errors
Ob type
Winds, clear
Level
GEO-R
Count ~7000
200mb
na
300mb
5.0 m/s
400mb
4.5 m/s
500mb
4.0 m/s
700mb
na
GEO-I
~10000
3.5 m/s
3.2 m/s
3.0 m/s
2.6 m/s
2.0 m/s
Count ~2000
4.5 m/s
4.0 m/s
3.5 m/s
3.5 m/s
3.0 m/s
2.5 m/s
~4000
3.0 m/s
2.6 m/s
2.3 m/s
2.3 m/s
2.0 m/s
2.0 m/s
Winds, cloudy
200mb
300mb
400mb
500mb
700mb
850mb
Simulated Error for Temperature
GEO-I
GEO-R
0
100
Pressure hPa
200
300
400
500
600
700
800
900
1000
1100
0.5
0.75
1
1.25
1.5
1.75
Degrees C
2
2.25
2.5
3. Assimilate the synthesized observations
 The operational 40km Rapid Update Cycle (RUC) was
used to assimilate the observations at hourly intervals.
 Boundary conditions: NCEP Eta model, projected onto
the AWIPS 211 grid (80km resolution).
Four assimilation experiments were performed:
1) Conventional observations only (CONV)
2) Geostationary radiometer (GEO-R); assimilate profiles
adjusted to emulate a GOES-type system
3) Geostationary interferometer (GEO-I): assimilate profiles
from a proposed geostationary interferometer
4) Perfect observation (BEST); assimilate "true" profiles
extracted directly from the "nature" run
Note: The CONV and BEST experiments represent the range of
performance that can be expected from the RUC system.
4. Assess the impact of the observations on the
resulting forecasts
 The impact of the observations will be assessed by
objectively measuring the ability of each observing system to
steer the resulting 12-hour forecasts toward the “true”
atmosphere
Sensitivity of RUC analysis to retrieval density
GEO-R versus GEO-I
Retrieved T and Td
GEO-R versus GEO-I
Retrieved T and Td
GEO-R versus GEO-I
Retrieved T and Td
 GEO-I results are significantly improved
over those from the GEO-R.
 500 hPa temperature errors are reduced
by 0.2 C root mean square (rms) over the
extended CONUS (contiguous United
States) and 700 hPa relative humidity
errors are reduced by 2%.
To compare the impact of the geostationary
interferometer against the geostationary radiometer a
relative score was computed using the No Observation
run (NO) and the Perfect Observation run (PO) to
normalize the verification statistics.
The RMS errors for temperature and relative humidity
were summed over four layers (700hPa, 500hPa, 400hPa,
300hPa) and normalized between the RMS error sums
from the NO run and the PO run. A score of 10 matches
the PO run.
Soundings + Winds 700hPa RH Validation
GEO-R versus GEO-I
14
RMSE (%)
12
Retrieved T and Td
Sat Winds
CONV
10
GEO-R
8
GEO-I
6
BEST
4
Conventional
2
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
2
1
0
-1
-2
-3
-4
-5
-6
Soundings + Winds 700hPa RH Validation
50
CONV
GEO-R
GEO-I
BEST
S1 Score
Bias (%)
Soundings + Winds 700hPa RH Validation
45
CONV
40
GEO-R
GEO-I
35
30
0
2
4
6
8
10 12 14 16 18 20 22 24
Hour
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
Soundings + Winds 850hPa RH Validation
GEO-R versus GEO-I
20
RMSE (%)
Retrieved T and Td
Sat Winds
15
CONV
GEO-R
10
GEO-I
BEST
5
Conventional
0
0
2
4
6
8
10 12 14 16 18 20 22 24
Hour
Soundings + Winds 850hPa RH Validation
2
55
0
50
CONV
-2
GEO-R
-4
GEO-I
BEST
-6
S1 Score
Bias (%)
Soundings + Winds 850hPa RH Validation
45
CONV
40
GEO-R
35
GEO-I
30
-8
25
0
2
4
6
8
10 12 14 16 18 20 22 24
Hour
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
Soundings + Winds 500hPa T Validation
GEO-R versus GEO-I
RMSE (deg C)
1.2
Retrieved T and Td
Sat Winds
1
CONV
0.8
GEO-R
0.6
GEO-I
0.4
BEST
0.2
Conventional
0
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
Soundings + Winds 500hPa T Validation
Soundings + Winds 500hPa T Validation
60
0.2
CONV
GEO-R
0
GEO-I
BEST
-0.2
-0.4
55
S1 Score
Bias (deg C0
0.4
CONV
50
GEO-R
45
GEO-I
40
35
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
Soundings + Winds 300hPa T Validation
GEO-R versus GEO-I
RMSE (deg C)
1.2
Retrieved T and Td
Sat Winds
1
CONV
0.8
GEO-R
0.6
GEO-I
0.4
BEST
0.2
0
Conventional
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
Soundings + Winds 300hPa T Validation
Soundings + Winds 300hPa T Validation
60
0.2
CONV
0
GEO-R
-0.2
GEO-I
-0.4
BEST
55
S1 Score
Bias (deg C)
0.4
GEO-R
45
-0.6
40
-0.8
35
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
CONV
50
GEO-I
0
2
4
6
8
10 12 14 16 18 20 22 24
Hour
Soundings + Winds 300hPa U
4
RMSE (m/s)
GEO-R versus GEO-I
Retrieved T and Td
Sat Winds
3.5
CONV
3
GEO-R
2.5
GEO-I
2
BEST
1.5
1
0
Conventional
2
4
8 10 12 14 16 18 20 22 24
Hour
Soundings + Winds 300hPa U
Soundings + Winds 300hPa U
60
1.4
1.2
1
0.8
0.6
0.4
0.2
0
-0.2
55
CONV
GEO-R
GEO-I
BEST
S1 Score
Bias (m/s)
6
50
CONV
45
GEO-R
40
GEO-I
35
30
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
Soundings + Winds 300hPa V
4
GEO-R versus GEO-I
RMSE (m/s)
3.5
Retrieved T and Td
Sat Winds
CONV
3
GEO-R
2.5
GEO-I
2
BEST
1.5
1
0
Conventional
2
4
10 12 14 16 18 20 22 24
Soundings + Winds 300hPa V
60
55
CONV
GEO-R
GEO-I
BEST
S1 Score
Bias (m/s)
8
Hour
Soundings + Winds 300hPa V
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
6
50
CONV
45
GEO-R
40
GEO-I
35
30
0
2
4
6
8 10 12 14 16 18 20 22 24
Hour
0
2
4
6
8
10 12 14 16 18 20 22 24
Hour
Plans for the Future
*
*
*
*
*
*
*
Simulate winds from radiances
Assimilate retrievals from radiances
Assimilate radiances with 3DVar
14 day test periods (winter and spring)
Resolve boundary condition issues
Low Earth Orbit (LEO) OSSE
Test other observing systems
Wind Experiment Using
Simulated Radiances
1) Simulate radiances from GOES and from
a geostationary interferometer using
forward radiative transfer.
2) Put simulated radiances into the
automated wind algorithm and generate
cloud drift and water vapor winds
Hurricane Bonnie Wind and Cloud Fields
Wind Vectors :
Red - 1 km level
Green - 14 km
level
Clouds :
Light gray Ice Cloud
Dark Gray Water Cloud
GOES Radiances Simulation Verification
Wind Tracking Verification
Wind Tracking Verification - Continued
Tracking Interferometer Radiances
Tracking Mixing Ratio from Model