Mesoscale Probabilistic
Prediction over the Northwest:
An Overview
Cliff Mass
University of Washington
Mesoscale Probabilistic Prediction
• By the late 1990’s, we had a good idea of the
benefits of high resolution.
• It was clear that initial condition and physics
uncertainty was large.
• We were also sitting on an unusual asset due to
our work evaluating major NWP centers: realtime initializations and forecasts from NWP
centers around the world.
• Also, inexpensive UNIX clusters became
available.
“Native” Models/Analyses Available
Resolution (~ @ 45 N )
Abbreviation/Model/Source
Type
avn, Global Forecast System (GFS),
Spectral T254 / L64
~55 km
National Centers for Environmental Prediction
cmcg, Global Environmental Multi-scale (GEM),
Computational
Distributed
1.0 / L14
~80 km
Objective
Analysis
SSI
3D Var
Finite
Diff
0.90.9/L28 1.25 / L11
~70 km
~100 km
3D Var
Finite
Diff.
32 km / L45
90 km / L37
SSI
3D Var
Spectral T239 / L29
~60 km
1.0 / L11
~80 km
3D Var
Spectral T106 / L21
~135 km
1.25 / L13
~100 km
OI
Spectral T239 / L30
Fleet Numerical Meteorological & Oceanographic Cntr.
~60 km
1.0 / L14
~80 km
OI
tcwb, Global Forecast System,
1.0 / L11
~80 km
OI
Canadian Meteorological Centre
eta, limited-area mesoscale model,
National Centers for Environmental Prediction
gasp, Global AnalysiS and Prediction model,
Australian Bureau of Meteorology
jma, Global Spectral Model (GSM),
Japan Meteorological Agency
ngps, Navy Operational Global Atmos. Pred. System,
Taiwan Central Weather Bureau
ukmo, Unified Model,
United Kingdom Meteorological Office
Spectral T79 / L18
~180 km
Finite
Diff.
5/65/9/L30 same / L12
~60 km
3D Var
“Ensemblers”
Eric Grimit (r )
and
Tony Eckel (l)
are besides
themselves over
the acquisition
of the new 20
processor
athelon cluster
UWME
– Core : 8 members, 00 and 12Z
• Each uses different synoptic
scale initial and boundary
conditions
• All use same physics
– Physics : 8 members, 00Z only
• Each uses different synoptic
scale initial and boundary
conditions
• Each uses different physics
• Each uses different SST
perturbations
• Each uses different land surface
characteristic perturbations
– Centroid, 00 and 12Z
• Average of 8 core members used
for initial and boundary
conditions
Ensemble-Based Probabilistic Products
The MURI Project
• In 2000, Statistic Professor Adrian Raftery
came to me with a wild idea: submit a
proposal to bring together a strong
interdisciplinary team to deal with
mesoscale probabilistic prediction.
• Include atmospheric sciences,
psychologists, statisticians, web display and
human factors experts.
The Muri
I didn’t think it had a chance.
I was wrong. It was funded and very successful.
The MURI
• Over five years substantial progress was
made:
– Successful development of Bayesian Model
Averaging (BMA) postprocessing for
temperature and precipitation
– Development of both global and local BMA
– Development of grid-based bias correction
– Completion of several studies on how people use
probabilistic information
– Development of new probabilistic icons.
Raw 12-h Forecast
Bias-Corrected Forecast
B
Skill for
Probability of T2 < 0°C
0.2
*ACMEcore
*UW Basic Ensemble with bias correction
ACMEcore
UW Basic Ensemble, no bias correction
*ACMEcore+
*UW Enhanced Ensemble with bias cor.
ACMEcore+
UW Enhanced Ensemble without bias cor
Uncertainty
0.1
0.0
-0.1
00
0.6
03
06
09
12
15
18
21
24
42
45
48
0.5
BSS
0.4
0.3
0.2
0.1
0.0
-0.1
00
03
06
09
12
15
18
21
24
27
30
33
36
BSS: Brier Skill Score
39
27
30
33
36
39
42
Calibration Example-Max 2-m Tempeature
(all stations in 12 km domain)
The Thanksgiving Forecast 2001
42h forecast (valid Thu 10AM)
SLP and winds
1: cent
Verification
- Reveals high uncertainty in storm track and intensity
- Indicates low probability of Puget Sound wind event
2: eta
5: ngps
8: eta*
11: ngps*
3: ukmo
6: cmcg
9: ukmo*
12: cmcg*
4: tcwb
7: avn
10: tcwb*
13: avn*
Ensemble-Based Probabilistic Products
Probability Density Functio
at one point
Ensemble-Based
Probabilistic Products
Providing forecast uncertainty information is good….
But you can have too much of a good thing…
MURI
• Improvements and extensions of UWME
ensembles to multi-physics
• Development of BMA and probcast web
sites for communication of probabilistic
information.
• Extensive verification and publication of a
large collection of papers.
• And plenty more…
Before Probcast: The BMA Site
PROBCAST
ENSEMBLES
AHEAD
The JEFS Phase
• Joint AF and Navy project (at least it was
supposed to be this way). UW and NCAR main
contractors.
• Provided support to continue development of basic
parameters.
• Joint project with NCAR to build a complete
mesoscale forecasting system for the Air Force.
• For the first few years was centered on North
Korea, then SW Asia, and now the U.S.
JEFS Highlights
• Under JEFS the post-processed BMA fields
has been extended to wind speed and
direction. Local BMA for precipitation.
• Development of EMOS, a regression-based
approach that produces results nearly as
good as BMA.
• Next steps: derived parameters (e.g.,
ceiling, visibility)
NSF Project
• Currently supporting extensive series of
human-subjects studies to determine how
people interpret uncertainty information.
• Further work on icons
• Further work on probcast.
Ensemble Kalman Filter Project
• Much more this afternoon.
• 80-member synoptic ensemble (36 km-12
km or 36 km)
• Uses WRF model
• Six-hour assimilation steps.
• Experimenting with 12 and 4 km to
determine value for mesoscale data
assimilation-AOR in 3D.
Big Picture
• The U.S. is not where it should be regarding
probabilistic prediction on the mesoscale.
• Current NCEP SREF is inadequate and
uncalibrated.
• Substantial challenges in data poor areas for
calibration and for fields like visibility that the
models don’t simulate at all or simulate poorly.
• A nationally organized effort to push rapidly to 4D probabilistic capabilities is required.
Opinion
• Creating sharp, reliable PDFs is only half
the battle.
• The hardest part is the human side, making
the output accessible, useful, and
compelling. We NEED the social scientists.
• Probabilistic forecast information has the
potential for great societal economic
benefit.
The END
Brief History
• Local high-resolution mesoscale NWP in
the Northwest began in the mid-1990s after
a period of experimentation showed the
substantial potential of small grid spacing
(12 to 4 km) over terrain.
• At that time NCEP was running 32-48km
grid spacing and the Eta model clearly had
difficulties in terrain.
The Northwest Environmental Prediction System
•Beginning in 1995, a team at the
University of Washington, with the help of
colleagues at Washington State University
and others have built the most extensive
regional weather/environmental prediction
system in the U.S.
•It represents a different model of how
weather and environmental prediction can
be accomplished.
Pacific Northwest Regional Prediction: Major
Components
• Real-time, operational mesoscale environmental
prediction
–
–
–
–
MM5/WRF atmospheric model
DHSVM distributed hydrological model
Calgrid Air Quality Model
A variety of application models (e.g., road surface)
• Real-time collection and quality control of regional
observations.
Why Probcast?
• We are rapidly gaining the ability to produce
useful probabilistic guidance-- forecasts that are
reasonably reliable and sharp.
• Believe it or not, this is the easy part of the
problem.
• What we have not done is to design interfaces that
allow users to make effective user of probabilistic
output … or even convince users that they should
“go probabilistic”.
• The recent NRC Report on Probabilistic
Prediction highlights this issue.
UW Uncertainty MURI
• The DOD-sponsored UW Uncertainty
MURI was designed to consider both
sides of the problem:
– Generation of probabilistic information-ensembles and post-processing
– Display and human interface issues.
• Includes UW Atmospheric Sciences,
Statistics, Psychology, and Applied
Physics Lab
UW Probabilistic Prediction
• UW Ensemble System is based on using
varying initialization and boundary conditions
from differing operational analyses.
• Also includes varying model physics and
surface properties (e.g., SST).
• Have developed sophisticated post-processing:
grid-based bias correction and Bayesian
Model Averaging (BMA)--both global and
local
Before Probcast: The BMA Site
UW MURI
• Considerable work by Susan Joslyn and
others in psychology and APL to examine
how forecasters and others process forecast
information and particularly probabilistic
information.
• One example has been their study of the
interpretation of weather forecast icons.
The Winner
WRF Domains: 36-12-4km
Putting it All Together
• During the last year, the MURI project has
attempted to create a user interface that
accesses the voluminous and detailed
probabilistic information produced by our
system in a way that is highly accessible to
lay and even professional users.
• The result: PROBCAST
• Available operationally at
www.probcast.com
PROBCAST
Probcast Features
•Local BMA for temperature
•Currently non-local BMA for precipitation
(working on it)
•Attempt to use probabilistic information in
a way that is accessible to deterministicoriented individuals.
•Can click anywhere on map to get
probabilistic information for specific
locations.
PROBCAST
Just the beginning
•Will expand probcast to more
parameters
•Will improve interface from
public feedback
•Will be testing more icon ideas
this year.
•Unfortunately, MURI ends in 8
months and no replacement.
Bayesian model averaging
is the deterministic forecast from member k,
is the weight associated with member k, and
is the estimated density function for y given memb
0.0
0.5
density
1.0
1.5
where
0
1
2
3
Observed speed
4
5
61
The END
Ensemble Forecasting
48-hour forecasts for maximum wind speeds on 7 August 2003
63
Update on the UW EnKF
Mesoscale Analysis and
Prediction System
Brian Ancell, Clifford F. Mass, and Gregory J. Hakim
The University of Washington
Outline
1) Brief review of motivation for a highresolution EnKF
2) Summary of 36km/12km EnKF results
3) Upgrades to EnKF system
4) Computational costs for more frequent
data assimilation cycles at higher
resolution
Motivation for High-resolution EnKF
• Key benefit is using flow-dependent, smallscale structure during assimilation
36-km vs. 12-km EnKF
3612km SLP, 925-mb temperature, surface
km
winds
36-km vs. 12-km EnKF
3612km SLP, 925-mb temperature, surface
km
winds
36-km vs. 12-km EnKF
3612km SLP, 925-mb temperature, surface
km
winds
Motivation for High-resolution EnKF
• Key benefit is using flow-dependent, smallscale structure during assimilation
• Analysis and forecast uncertainty is a major
advantage
EnKF Configuration
D3 (4km)
D2 (12km)
D1 (36km)
EnKF Configuration
•
•
•
•
•
WRF model
38 vertical levels
80 ensemble members
6-hour update cycle
Observations:
• Surface temperature, wind, altimeter
• ACARS aircraft winds, temperature
• Cloud-track winds
• Radiosonde wind, temperature, relative
humidity
Half of surface obs used for assimilation, other half for
EnKF 12km Surface
Observations
EnKF 36km vs. 12km
Wind
Analys
is
Foreca
10
%
10
Temperatur
e
Improvement of 12-km
EnKF
13
%
10
Reduction of Observation Variance in
12km EnKF
Wind
Analys
Temperatur
e
Improvement using reduced observation
uncertainty
5%
10
Further Improvement After Variance
Inflation, Bias Removal
Wind
Analys
Temperatur
e
Improvement using inflation and bias
removal
9%
3%
EnKF 12-km vs. GFS, NAM, RUC
Wind
GF
NA
S
RU
M
E
CnKF
2.38
2.30
m/s m/s
2.13 m/s
1.85 m/s
Temperatur
e
RMS analysis
errors
2.28 K
2.54 K
2.35 K
1.67 K
EnKF System Upgrades
• 4km domain
EnKF System Upgrades
36k
m
12k
m
SLP, 925-mb temperature, surface
winds
4km
EnKF System Upgrades
• 4km domain
• WRF V2 to WRF V3
EnKF System Upgrades
• 4km domain
• WRF V2 to WRF V3
• Physics matched with real time GFS-WRF
(changes bias and surface variance
characteristics)
EnKF System Upgrades
• 4km domain
• WRF V2 to WRF V3
• Physics matched with real time GFS-WRF
(changes bias and surface variance
characteristics)
• Additional altimeter observations
Surface observations
12km: ~10,000
~5,000 (50%)
4km: ~5,000
~4,000 (80%)
Terrain check
EnKF System Timing
6-hr cycle, 80 processors
• 36km:
• 4km
assimilation:
12 minutes
model integration: 20 minutes
overhead:
5 minutes
assimilation:
15 minutes
model integration: 240 minutes
nestdown:
10 minutes
overhead:
10 minutes
Total:
5 hours
EnKF System Timing
3-hr cycle, 80 processors
• 36km:
• 4km
assimilation:
12 minutes
model integration: 10 minutes
overhead:
5 minutes
assimilation:
15 minutes
model integration: 120 minutes
nestdown:
10 minutes
overhead:
10 minutes
Total:
3 hours
EnKF System Timing
2-hr cycle, 80 processors
• 36km:
• 4km
assimilation:
12 minutes
model integration: 7 minutes
overhead:
5 minutes
assimilation:
15 minutes
model integration: 80 minutes
nestdown:
10 minutes
overhead:
10 minutes
Total:
2.3 hours
EnKF System Timing
2-hr cycle, 160 processors
• 36km:
• 4km
assimilation:
12 minutes
model integration: 4 minutes
overhead:
5 minutes
assimilation:
15 minutes
model integration: 50 minutes
nestdown:
5 minutes
overhead:
10 minutes
Total:
1.7 hours