HWRF Model Diagnostics
Vijay Tallapragada
HWRF Team Lead, NCEP/EMC
HFIP Stream 1 Regional Hurricane Model Diagnostics Planning Meeting,
11/08/2010
1
Outline
Diagnostics for systematic evaluation of storm
track and intensity forecasts
 Complement the HFIP diagnostics team efforts

◦ Make use of preliminary diagnostic products to identify specific
model behavior
◦ Provide more in-depth analysis of important model deficiencies
and methods to improve model physics/dynamics/initialization

Additional requirements for operational support
◦ Diagnostics tailored for operational needs
◦ Specific focus on HWRF model improvement
2
End of the season forecast verification statistics provide first
order information on model performance (important measure of
forecast skill)
•Difficult to identify the source of errors
•Storm to storm variability
•Statistical properties of the forecast errors
•Inter-dependency of intensity and track errors
•Impact of initialization and model physics
3
Track forecasts are strongly linked to large-scale environment
•
•
•
•
•
•
•
Evolution of large-scale flow in HWRF domain
Location and strength of troughs and ridges
Steering currents, effects of shear
Influence of lateral boundary conditions
Sensitivity to physical parameterizations
Impact of data analysis
Non-physical issues (nest motion, interaction with topography, tracker
issues etc.)
4
How good is the evolution of large-scale
atmosphere in HWRF?
Issues with nest interaction
6
Operational HWRF
GFS Analysis
HWRF fails to simulate increased shear
in the
environment
1/11/2010
Persistent
northward bias,
but reversal of
zonal track bias
8
Intensity forecasts are dictated by several
factors….
• Impact of Vortex initialization and data assimilation
• Large-scale affects of shear, storm size, vertical structure of
winds, temperature and moisture, asymmetries, scale
interactions
• Sensitivity to microphysics, PBL, land surface, surface physics and
convection parameterizations
• Effects of horizontal diffusion, divergence damping, momentum
mixing, surface enthalpy coefficients etc.
• Effects of air-sea interaction – MLD, OHC, SST, loop current,
warm and cold core eddies
• Effects of resolution, nest feedback
9
Rapid growth of
intensity errors
Pronounced negative bias
10
Systematic nature of
HWRF Initialization
Positive bias for weaker storms,
negative for stronger storms
(spinup/spindown)
64 kts threshold for storm
size/intensity correction
(based on more than 1200 forecasts)
11
Vortex Initialization Issues/Challenges
H210 model produced
Fewer strong storms
Than observed
HWRF 2009 production
has too many strong storms
Insufficient vortex size correction
Broad initial vertical structure
12
t=00
t=06
Some of the maximum wind
speed values are not associated
with the storms.
t=12
Band 3 Loop Simulated
Band 3 Loop Real
Convection and moisture transport in HWRF –
systematic dryness
14
Outer Domain (27 km)
Nest (9 km)
Convective precipitation
Outer Domain (27 km)
Nest (9 km)
Total precipitation
Grid-scale Precipitation
dominates convective
precipitation at 9 km
resolution
15
Impact of horizontal
diffusion
Impact of surface
physics changes
HWRF Track forecasts are
somewhat less sensitive to
changes in changes in
horizontal diffusion and
surface physics
16
Impact of data assimilation
Analysis increments in the
outer domain
17
Heat Fluxes (for ocean coupling)
Adversely impacts SST fields in HyCOM
POM vs
HYCOM

IKE example
◦ Model SST minus
daily average GOES /
AVHRR data.
◦ POM on left, too
warm under
hurricane.
◦ HYCOM on right,
too cool in general.
Existing Tools for Hurricane Model Diagnostics at EMC (I)

Operational HWRF Model Output (archived):
 Track and intensity forecasts in ATCF format (6-hrly interval)
 6-hourly model output on native model grid (horizontal, rotated lat/lon, E-grid)
interpolated to standard pressure levels
 Model output in GRIB-1 and GRIB-2 format for selected 3-D variables on standard
pressure levels, and a suite of surface variables
 Hourly max. 10m wind and rainfall (ascii)
 HWRF input and analysis files (binary) for initial time
 6-hourly ocean model output for standard variables (T, S, U, V, W, MLD, OHC)

Real-time automated graphical display of selected model forecast variables
 Track and intensity forecasts, max wind and rainfall swaths
 Animations of
 850 hPa wind circulation (combined domain and nested domain)
 10m winds and MSLP (nested domain)
 Vertical cross-sections of zonal and meridional wind (along the nest center)
 Simulated GOES IR and WV imagery
 Simulated Radar Reflectivity
 Real-time statistics for track and intensity forecast errors and biases
20
Existing Tools for Hurricane Model Diagnostics at EMC (II)

HPLOT Diagnostics and visualization software
 GrADS/FORTRAN based software to plot model forecast variables in GRIB format
 Point and click vertical cross-sections, area-averaged quantities
 Model to model and model to analysis comparison (and difference plots)
 Variable level vertical wind shear and mean-layer wind
 Skew-T plots for thermodynamic structure
 Preliminary statistics for RMSE, ACC and Bias

TC Vortex Diagnostics
 Generates azimutially averaged structures of the TC vortex
 Computes vortex center for each level separately (based on geopotential height)
 Determines departure from gradient wind balance for each instance

BINPLOT software for visualization of model output on native grid

User developed MATLAB tools for ocean model evaluation
21
Diagnostic Efforts at EMC – Vortex initialization
Description Theory/Hypothesis
Methodology
Initialization
issues
HWRF model is quite sensitive to the
initialization procedure, especially w.r.t.
specification of storm size, cycling,
surface pressure calculation,
composite vortex, filter strength and
vertical structure.
Describe the problems with HWRF
initialization.
Sensitivity to GFDL filtering changes
Consider changing the filter strength
following GFDL approach
Sensitivity to storm size specification
Use of second parameter (34-kt wind
radius and or radius of storm circulation
Evaluate non-linear balance issues
Explicit formulation for post-balance
Gradient wind and hydrostatic balance
Describe if there is a systematic bias in the
gradient wind balance by examining
different storms and different initial
conditions. If this is a persistent feature of
HWRF ICs, identify methods to correct this
and impose dynamic and thermodynamic
balance consistent with the model. Test
the sensitivity of different approaches.
Sensitivity to Composite Vortex
Reconstruct axi-symmetric composite
vortex using current operational HWRF
configuration
22
Diagnostic Efforts at EMC – Vortex Evolution
Description Theory/Hypothesis
Methodology
Vortex scale
diagnostics
Intensity forecast skill is closely linked
to HWRF's ability to properly describe
the evolution of vortex through
organization of convection, primary
and secondary circulations, distribution
of heating and vertical motions, air-sea
exchange of fluxes for heat and
momentum and radiation, BL
processes etc.
1. Identify cases with the following
scenarios: a) HWRF tends to intensify the
storm while in reality the storm weakens
and vice-versa; b) reasonably good
intensity forecast (2-3 day and 4-5 day)
2. Identify available datasets that can be
used to evaluate model forecasts for these
selected cases.
Sensitivity to Microphysics
Identify coding errors (if any) in the Ferrier
Scheme and test their impacts on HWRF
forecasts; Analyze cloud-radiation
feedbacks (large-scale as well as storm
scale);
Sensitivity to BL parameterization
1. Sensitivity to specification of Ri; 2.
Describe the BL processes through budget
analysis of heat, momentum and moisture
Sensitivity to initialization changes
Analyze changes to vortex size and
structure with reference to initialization
experiments
Sensitivity to changes in the ocean
components
Identify areas in HWRF/HYCOM/POM that
have potential impact on improving
(reducing) negative bias of HWRF intensity
forecasts.
Diagnostics Efforts at EMC - Track forecasts and
sensitivity to initialization
Description Theory/Hypothesis
Methodology
Effect of
storm
relocation
and vortex
initialization
on track
forecasts
HWRF model is quite sensitive
to the initialization procedure,
especially w.r.t. specification of
storm size, cycling, surface
pressure calculation,
composite vortex, filter
strength and vertical structure.
1. Storm size correction using
RMW and 34-kt wind radius
2. Post-balance method for masswind balance
Sensitivity to GFDL filtering
changes
Consider changing the filter
strength following GFDL approach
Gradient wind and hydrostatic
balance
Impact of post-balanced vortex on
track forecasts
Sensitivity to Composite Vortex
Reconstruct axi-symmetric
composite vortex using current
operational HWRF configuration
Diagnostic Efforts at EMC – Large Scale
Description
Theory/Hypothesis
Methodology
Large scale
Diagnostics
It has been widely
observed that evolution of
large-scale features in
HWRF tend to deviate from
GFS analysis/forecast.
Particularly the strengths of
troughs and ridges and
their location tend to show
considerable differences.
1. Systematic bias in HWRF
output for the following fields:
(a) 850/700/500/200 hPa
heights
(b) wind fields at 850 hPa, 500
hPa and 200 hPa
(c) Mean-layer winds (steering
currents)
(d) Vertical wind shear
(e) Mean-layer thickness
(f) Stream function at 850 hPa
(g) Velocity Potential at 200
hPa
Track bias
•
Systematic biases in
HWRF model for track
forecasts (phase and
directional)
25
Diagnostic Efforts at EMC – Large Scale evolution
Description Theory/Hypothesis
Methodology
Large scale
Diagnostics
Sensitivity to radiation
parameterization
•
Replace GFDL radiation with
RRTM (or simple Dudhia
scheme) and analyze largescale features
Sensitivity to LSM
parameterization
•
NOAH LSM replacing slab
model
Sensitivity to convection •
parameterization
•
Effects of momentum mixing on
track forecasts
Evaluation of heating rates and
precipitation
•
Sensitivity to specification of Ri
and mixing length
Describe the BL processes
through budget analysis of heat,
momentum and moisture
Sensitivity to PBL
parameterization
•
Requirements for Stream-1 Model Diagnostics (I)
What is required to assess the model performance and provide input to
improve model formulation:
•
Define specific evaluation metrics along with suggestions for associated observed
datasets.
•
Provide guidance on specific case studies that will be used in extended model
diagnostics.
•
Identification of systematic errors and model biases
•
Development of objective techniques for comparing model fields (temperature,
moisture etc) to various well-validated satellite-derived fields
•
PDFs and structure functions comparing statistical properties of model output
with those from satellite derived observations, soundings from dropsondes,
aircraft data and radar observations (ground based and aircraft based).
•
Derived 3-D wind and temperature fields from these remotely sensed and insitu
observations for model evaluation.
27
Requirements for Stream-1 Model Diagnostics (II)
•
Computational methods for partitioning of forecast errors arising due to various
components of model dynamics and physics
•
Heat/energy budget studies (to include contributions from diabatic heating terms,
PV budget, moisture transport etc.)
•
Large scale track and intensity predictors, interactions among large scale
(synoptic) and vortex scale (mesoscale) circulations
•
Off-line programs to study the impact of changes to physics and initialization
(idealized simulations)
•
Event specific diagnostic methods to estimate the impacts of interaction with
topography, impact of various features of the ocean where applicable
•
Multi-dimensional visualization tools
•
Automated model diagnostics tools to assist real-time and post-season evaluation
of model performance
•
Diagnostics tailored to assist pre-implementation T&E
•
Develop capability to conduct sensitivity experiments in a common framework
where all models can be initialized with the same initial vortex and have the same
background environmental flow.
28
Requirements for Stream-1 Model Diagnostics (III)
Vortex Scale Observations
•Observations that describe details of vortex evolution (3-D structure) in
different environments (thermodynamic/kinematic)
Particularly in development stages
What is hurricane scale circulation?
Do we have comprehensive data sets that describe the intensity change
process??
Microphysics – missing data above melting level, e.g. hydrometeor fields –
nucleation is not well understood
Large Scale:
•GFS Analysis, gridded satellite derived datasets (atmosphere and ocean),
composite 3-D analysis of the hurricane environment using aircraft
dropsonde/radar etc.
Requirements for Stream-1 Model Diagnostics (IV)
Diagnostic Methods:
•Compute derived variables using a common methodology using scientifically established
procedures.
•Identify standard as well as derived variables needed for diagnostics
•Develop set of programs that can be used to compute extended diagnostic variables
from respective model forecast and observational datasets.
•Establish a code repository with a collection of such software
Visualization Tools:
•Common software that can incorporate forecast datasets from different models in
assisting model inter-comparisons and evaluation with observations.
•Integration of the capabilities of HPLOT, DIAPOST and TC Diagnostics
•Incorporating MATLAB, BINPLOT and other applications
•TC Diagnostics file and SHIPS predictors (Mark DeMaria); Initial Vortex Comparison
(Wallace Hogsett); and Post-Balance evaluation (In-Hyuk Kwon)
HWRF Annual Upgrade T&E and Process Automation for diagnostics:
•HWRF team at EMC has a very ambitious and aggressive T&E for annual upgrade
implementations. Diagnostics efforts should be geared to fit in the timeline. Need to
develop a framework for process automation of standard as well as advanced diagnostics.
•Bring the experience of experts in the community in analyzing these forecasts and provide
feedback and suggestions for model improvement.