WRF-Ensemble Kalman Filter
Lightning Assimilation
Project
December 18th, 2008
Phil Regulski, Cliff Mass and Greg Hakim
Overview
• Experiment review
• Case study results
– December 2002 (review)
– October 2004
– November 2006
• Statistical analysis
– Lightning flash rates and model output
variables
• Conclusions
– Implications of real-time assimilation of
lightning
– Modeling techniques
Experiment Review
• Control run
– No lightning observations were assimilated
– Radiosondes, surface stations (ASOS, ship, buoy),
ACARS and cloud drift wind (no radiances) observations
assimilated
• Experimental runs
– Lightning “observations” are assimilated along with
observations used in the control run
• Experiment 1 (LTNG1): Non-thinned lightning
– Lightning strike observations are converted into 30 min. lightning flash rates from
nearby lightning points
– Lightning rate is converted into an “observation” of convective rain using the
Pessi/Businger convective rain rate/lightning rate relationship
– Convective rain is assimilated into the WRF-EnKF
• Experiment 2 (LTNG2): Thinned lightning
– Same as previous experiment except that any lightning strikes used in a previous
density calculation are no longer allowed to be an assimilation point for the filter,
resulting in a thinning out of the lightning “observations” (although strikes can be
reused to calculate nearby densities)
Case Studies
December 16-21, 2002
Case Studies
December 16-21, 2002
Thinned Exp.
Summary:
• Assimilating lightning has
an impact on the analysis
and 12 (not shown) / 24
hour forecasts
• Are the impacts
improvements?
24-hr Forecast of SLP: Lightning assimilation (black lines) versus
control (red lines) with differences between the two shaded. Lightning
drawn as black dots with observations in blue.
Case Studies
December 16-21, 2002
12-hr Forecast SLP RMS Errors over domain (% Improvement
compared to control)
Dates
Verified versus GFS Analysis
LTNG1
LTNG2 6hr
1mm
LTNG2 1hr
5mm
12/16/2002 12 – 12/21/2002 00
0.0 %
0.9 %
2.1 %
12/18/2002 00 – 12/20/2002 12
6.0 %
9.0 %
11.2 %
24-hr Forecast SLP RMS Errors over domain (% Improvement
compared to control)
Dates
Verified versus GFS Analysis
LTNG1
LTNG2 6hr
1mm
LTNG2 1hr
5mm
12/17/2002 00 – 12/21/2002 12
1.1 %
-1.4 %
0.3 %
12/18/2002 00 – 12/20/2002 12
4.3 %
4.7 %
6.5 %
Summary:
• 12- and 24-hr
forecasts show
improvement when
lightning is assimilated
into the WRF-EnKF
Case Studies
December 16-21, 2002
Summary:
• Dec. 2002 test case has
abnormally high lightning flash
rates
• Are results repeatable with
typical winter extra-tropical
cyclones?
Case Studies
October 1-14, 2004
Case Studies
October 1-14, 2004
Non-thinned
Thinned
Example of 24-hr Fcst. of SLP: Lightning assimilation (black lines)
versus control (blue lines) with differences between the two shaded.
Summary: Smaller amounts of lightning having less impact on the 24-hr
forecasts.
Are the smaller impacts, at least, improving the forecasts?
Case Studies
October 1-14, 2004
Control
Thinned
Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines)
versus control (blue) and experiment (red) with differences between
experiment and the “truth” (WRF-EnKF Analysis) shaded.
Summary: Positive and negative impacts to the forecasts balance out
with no net improvements.
Case Studies
October 1-14, 2004
Control
Thinned
Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines)
versus control (blue) and experiment (red) with differences between
experiment and the “truth” (WRF-EnKF Analysis) shaded.
Summary: There are positive and negative impacts to the experimental
forecasts. No net improvement over the control.
Case Studies
November 6-20, 2006
Case Studies
November 6-20, 2006
Control
Non-thinned
Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines)
versus control (blue) and experiment (red) with differences between
experiment and the “truth” (WRF-EnKF Analysis) shaded.
Summary: No major improvements to forecasts with assimilation of
lightning.
Case Studies
November 6-20, 2006
Control
Thinned
Example of 24-hr Fcst. SLP Error Maps: WRF-EnKF Analysis (black lines)
versus control (blue) and experiment (red) with differences between
experiment and the “truth” (WRF-EnKF Analysis) shaded.
Summary: No major improvements to forecasts with assimilation of
lightning.
Statistical Analysis
• Why are we seeing less forecast
improvements, if any, in Case 2 and
Case 3?
– Variance analysis
– Statistical analysis of
predictor/predictand relationships
Statistical Analysis
During intense lighting
events the model
forecasted convective
rain rate exhibits good
agreement with lightning
observation locations. As
a result model variance
values are nicely spread
over wide range of model
output values.
Model forecasted 6-hr
accumulated convective
rain with 1-hr lighting
values.
Statistical Analysis
When lightning is
scattered in less dense
packets, such as for cold
advection cumulus,
model values trend to
zero causing a much
smaller range in
variance.
Model forecasted 6-hr
accumulated convective
rain with 1-hr lighting
values.
Statistical Analysis
Reducing the period of
model accumulation of
precipitation from 6- to 1hr may position the
lighting better but the
lightning now has higher
probability of sitting on
zero valued model
gridpoints reducing
variance.
Model forecasted 1-hr
accumulated convective
rain with 1-hr lighting
values.
Statistical Analysis
Variance analysis: Subtleties of Lightning
Assimilation Technique
– If the background forecast variance is too low the filter will
assume the forecast is correct and lightning will have little to
no impact.
• Cases 2 and 3 exhibit this problem consistently due to less intensive
precipitation events.
– Reducing the convective rain rate totals from 6- to 1-hr more
accurately positioned the dynamical features (i.e., fronts and
convective cells) with the lightning but 1-hr precipitation
totals reduced the values of the model field which meant that
many more lightning assimilation points had greatly reduced
variances.
Statistical Analysis
October 2004 and November 2006
• If lightning is not impacting the forecasts enough can we change
the filter sensitivity to the observations?
– Altered observation errors
• Changing the error associated with the observation proxy (convective rain)
changes the “weight” the Ensemble Kalman Filter gives to the assimilated
convective rain value versus the background 6-hour forecast value
• No major changes in results with new observation error values in
areas with low lightning density
– You can change the observation error values all you want but if the
model background field is zero it wont matter.
• The problem with assimilating a field, such as convective rain
rate, which often is zero, is that when it is zero, no matter what
information the observation provides at the same location, the
error covariance matrices will be useless. Therefore no additional
information from the lightning observations will be ingested into
the filter.
• All lightning observations that assimilated convective rain rate have this
problem
Statistical Analysis
•
Re-examine the lightning flash rate to convective rain rate relationship with
data from case studies.
•
Is there a better relationship with a field that is continuous?
– Existing Pessi/Businger relationship used as first attempt to relate model field
with lightning rate for WRF-EnKF.
– Can we find a model output variable parameter that always has a non-zero value
thereby making full use of Kalman filter?
• Convective rain values that are zero handicap the WRF-EnKF due to its use of error
covariance matrices.
•
Examine relationship of lightning flash rate to other model output variables
– Screening Linear Regressions
•
•
•
•
•
Precipitation fields: Convective, grid scale and total rain
Moisture fields: relative humidity
Wind fields (u, v)
Vertical winds (w)
Lapse rates, vorticity advection, Lifted Index, etc.
Statistical Analysis
Used the Pessi/Businger lightning
rate/convective rain rate relationship as the
conversion from lightning to ingestible
model information to the WRF-EnKF.
• Already studied and ready to use.
• Works well for the Ensemble Kalman filter
when there are very large amounts of
lightning.
• Convective rain is a problem for the
Ensemble Kalman filter when the
background field is zero.
• Does our model output have a similar
relationship during typical winter storms
traversing the eastern Pacific?
Statistical Analysis
• Relationship between convective rain rate and lightning rate breaks down when Case 2 and Case 3 are
added to data analysis.
• Many more lightning observations over small conv. rain model values (increase in bottom left corner of right graph)
• Mean precipitation value goes down from .83 to .23 when data from Case 2 and Case 3 are added
• Large amount of low flash rates and high rain rates also contaminate the relationship (upper left of right graph).
• Where there is convective rain there are only small lightning densities which hurt relationship.
Case 1
Case 1, 2 and 3
Flash frequency versus convective rain rate
Statistical Analysis
•
Testing the convective rain rate/lightning rate relationship
–
•
The convective rain rate/lightning rate relationship using the WRF-Ensemble Kalman Filter
model output is less robust during less lightning active events in the eastern Pacific using our
small sample.
Are we being fooled by the test cases?
–
Larger sample needed.
•
Examined Vaisala’s Real-time LR unfiltered lightning data from entire month of December 2007 with
Real-time WRF-EnKF fields from same time period.
–
–
Also examined…
•
•
•
•
9200 unique lightning points within domain
Size of box to determine grid density
Lightning over ocean vs. land vs. both
Background model value (interpolation, grid averages, single grid)
Examine relationship of lightning flash rate to other model output variables
–
Screening Linear Regressions
•
•
•
•
•
Precipitation fields: Convective, grid scale and total rain
Moisture fields: relative humidity
Wind fields (u, v)
Vertical winds (w)
Lapse rates, vorticity advection, Lifted Index, etc.
Statistical Analysis
• Larger sample?
– Examined Vaisala’s Real-time LR unfiltered lightning data from entire
month of December 2007 with Real-time WRF-EnKF fields from same
time period
• Size of box to determine grid density
– Optimal box size is approx 1-1.5 degrees
• Lightning over land vs. ocean vs. both
– Using land lightning hinders relationships with model metrics due to terrain and
other complications (land v. ocean effects, orographic induced dynamics, diurnal
cycles, etc.)
• Background model value (interpolation, grid averages, single grid)
– Little difference in results to how model value at lightning observation point is
reached.
– Relationships still not robust
• Still see areas where lightning occurs but WRF model does not have the
corresponding synoptic pattern
• Can we force relationship using a convective rain filter?
– Only use lighting where there are positive convective rain rate values in the model
for the statistical analysis (model shows signs of correct synoptic pattern if there is
precipitation there)
Statistical Analysis
Best predictors of lightning using 1.5 degree density box
Lapse rate (850-500) * RH850
R^2 = 0.172
Grid scale rain
R^2 = 0.115
RH850
R^2 = 0.131
Total rain
R^2 = 0.092
(T2-SST)*RH850
W850
R^2 = 0.120
R^2 = 0.113
Can we find better relationships in longer timer periods (6-,12 and 24-hr) accumulations?
Statistical Analysis
6-hr ltng and 6-hr Total rain
1-hr ltng and 6-hr Total rain
12-hr ltng and 12-hr Total rain
Relationship degrades
using longer
assimilation time
periods.
Does a robust
precipitation
rate/lightning rate
relationship exist?...
Statistical Analysis
• Best model metric is Lapse Rate (850-500 hPa) * RH850
• Not very convincing metric/lightning rate relationships using UW’s Real-Time
WRF-EnKF model output fields
• What might be impairing finding a relationship between metric and lightning
rate hurting the statistical analysis?
• Model’s inability to correctly predict the synoptic pattern (most likely)
• Unfiltered data: Streak contamination, bad data
• Detection efficiency: Missing data over non-zero model output
• Higher model resolution, more detailed microphysics… (graupel relationship)?
• Same fundamental problem of model incorrectly forecasting synoptics
Conclusions
• Analysis and forecast improvements during the
abnormally high flash rate cyclone of December
2002 can be attributed to
– Large non-zero values of convective rain throughout
domain
– Strong convective rain rate/lightning flash rate
relationship during December 2002 storm
• Poor performance of forecasts during Cases 2 and
3
– Poor convective rain rate/lightning flash rate relationship
– Low lightning rates
– Small model forecast variances in predictor (convective
rain rate)
Conclusions
• Lightning observations will not help improve forecasts in
the WRF-Ensemble Kalman Filter, IF the model does not
correctly predict the synoptic pattern
– Without correct placement of dynamical features there will
never be a statistically significant relationship between
lightning rate and a model metric for the Ensemble Kalman
Filter
• Improvements to real time WRF-EnKF will only be useful
when there are extreme amounts of lightning
• Nudging may prove a better method than WRF-EnKF since
you are altering the model profiles rather than depending
on background forecasts
– Adding lightning to the real-time system will only improve
forecasts IF there are extreme amounts of lighting
– Nudging doesn’t ask anything from the model only requires
observations
• WRF-EnKF/nudging hybrid?