Post-Processing of Numerical Ozone Model
Forecasts:
The Land-Sea Problem
Bill Ryan
Department of Meteorology
The Pennsylvania State University
wfr1@psu.edu
2010 International Workshop on Air Quality Forecasting Research
Quebec City, Quebec
The Problem: False Alarms of High O3
• Operational air quality forecasters along the I-95 Corridor in
the northeastern US are increasingly reliant upon photochemical models for O3 forecast guidance.
– Significant recent changes in regional emissions have reduced
skill of guidance methods that require training period.
• Overall model performance in this region is good but
suffers from frequent “false alarms” of unhealthy O3 levels.
• The false alarm rate is primarily a function of strong
forecast O3 gradients along land-sea boundaries.
• Numerical guidance output (1200 UTC initialization) arrives
close (within minutes in some cases) to forecast deadlines.
Forecasters require a simple, quick method to deal with this
issue.
Example of the Land-Sea O3 Gradient
NOAA-EPA Numerical Ozone Forecast Model (www.weather.gov/aq)
Pattern Recurs in Certain Locations
Not an Issue with PM2.5 Forecasts
Daily maximum 8-Hour Average
Ozone (ppbv), NOAA-EPA Model
24-hour (midnight-midnight)
average PM2.5 (µg/m3)
Complicating the Issue: Sea Breezes Can Drive
Steep Coastal O3 Gradients
MODIS/Aqua
~ 1730 UTC
In this case (August 11, 2010) sea breeze circulations developed north of a
frontal boundary and re-circulated the previous day’s polluted air mass.
Simplest Application: Philadelphia Forecast Area
The Philadelphia Metro Forecast Area, roughly enclosed in rectangle, is peripherally
affected by modeled sea breeze-O3 effects, primarily in southern NJ
How Do US Operational Forecasters Receive
Local Scale Model Forecast Information?
Millville
Model forecast output is automatically
generated by NOAA-EPA by extracting
peak O3 concentrations over land
areas within designated warning areas
(using zipcodes) that are used for
email/Web notification. This is posted in
timely fashion at AirNow Tech
(http://www.airnowtech.org/).
Millville, NJ monitor location shown at left.
Interstates and zip code boundaries shown.
Forecast Peak O3 Frequently Skirts Coastal Areas
Simplest post-processing method to
remove land-sea effects is
to extract O3 forecast at locations
near monitors and/or away from the
boundary. This can be quickly
done using point data extraction from the
NOAA forecast web site.
Results of Simple Method (2010)
• Removing coastal zip codes from the Millville
forecast reduces bias at that location from 17
ppbv to 9 ppbv.
• Because this location is often the location of peak
modeled O3 in the Philadelphia metro area, Code
Orange model forecast frequency in PHL is
reduced from 44 to 20 days (Threshold for Code
Orange - Air Quality Alert is 76 ppbv 8-hour
average) .
• Of the 20 forecast Code Orange cases, 16
observed Code Orange in PHL in 2010.
Effect on Metropolitan Scale Forecasts (2010)
• When this simple post-processing is applied to the PHL metro domain,
numerical model skill equals, or exceeds, expert forecast skill and is a
large improvement on earlier forecast methods (not shown).
1
0.9
0.8
0.7
0.6
Forecast
0.5
Model
0.4
0.3
0.2
0.1
0
Hit
False Alarm
Threat
Simple Approach Less Useful When Monitors are
Located Immediately Along Sea-Land Boundary
Southern Delaware forecast
area includes monitor along
coast at Lewes.
Bias Correction Post-Processing
• Use of Sonoma Tech Air Quality MOS (AQMOS)
– http://aqmos.sonomatech.com/index.cfm
• Not a true MOS, method is long term bias
removal with additional correction in higher O3
cases (hierarchal bias correction method).
• Applied to southern Delaware in 2010, reduces
overall forecast bias by 4.6 ppbv.
• Also, reduces number of false alarms of Code
Orange O3 (76 ppbv) from 19 to 5.
• At cost of 4 additional missed Code Orange cases
although “close” forecasts (71,74,75 ppbv).
More Difficult Application: Baltimore
Metropolitan Area
In 2010, the monitor at Edgewood,
located northeast of Baltimore,
reached the Code Orange range
on 75% of all Code Orange days
in the Baltimore region.
On half of the Code Orange days,
the three bayside monitors
were the only locations reaching
the alert threshold.
An Example of the Bay Breeze Effect on O3 in
the Baltimore Metro Area
2200 UTC hourly O3 observations and wind barbs,
June 25, 2009.
Note on Determining Air Quality Forecast Skill in
the mid-Atlantic
• False alarms can be due
to factors other than
water-land effects.
• A frequent problem is
forecasting the timing and
extent of convection.
• At right, a late day
thunderstorm on June 3
limited O3 to 74 ppbv.
• A set of 7 convection
cases excluded in further
analysis of 2010 results.
Model Forecast O3 in Baltimore
• The problem is solving
the “false alarm” issue
in non-convective cases.
• Using raw model
output, 26 false alarm
cases in 2010.
• Using the simple
method of excluding
near-water zipcode
locations reduces false
alarms to 19.
Further Post-Processing Approaches
• Due to the close proximity of emissions sources to the
sea-land boundary, improvement by removing bay side
grid points (simple method) is useful but limited.
• Skill decreases quickly if further exclude near-bayside
locations.
• Analysis of 2010 Cases (a very active O3 season) gives
some hints:
– On average, observed Code Orange cases feature a
stronger modeled gradient from bay side to inland, but too
much variability to use operationally (CART method).
– Most useful method combines the magnitude of modeled
bayside O3 and the mean domain modeled O3.
Forecast Results for Code Orange Threshold
(Air Quality Alert, 8-hour O3 ≥ 76 ppbv, 2010)
1.40
NAQC
POST
PERS
1.20
1.00
0.80
0.60
0.40
0.20
0.00
Bias
False Alarm
Hit
Cnull
Blue: Model Forecast (NAQC)
Red: Post-Processed Model (POST)
Green: Persistence (PERS)
Accuracy
Heidke
Threat
Summary of Baltimore Results
• Pro: Removal of bias and reduction of false
alarm rate.
• Hit rate also reduced but, at 75%,
approximately equal to, or better than,
historical expert forecast accuracy.
• Small (~5%) improvement in skill scores.
• Half of the misses and false alarms occur in
range [74, 78] ppbv (close calls).
Application to Connecticut Ozone
The majority of high O3 events occur at coastal
monitors (62%). Only 5 of 24 observed Code Orange
days did not have a coastal monitor above the
threshold (76 ppbv, 8-hour average).
Background on Model Performance in CT
• The NOAA-EPA model, on a statewide domain
basis, did a reasonable job in 2010. Overprediction bias of 5 ppbv (~10%).
• Bias correction, on the seasonal or synoptic scale,
slightly improves forecasts but has little impact at
the Code Orange (Air Quality Alert) threshold.
• Hit rate of 84% but high false alarm rate (34%).
High bias for warning forecasts.
• Expert forecast, typical of most locations along I95, has fewer false alarms but many more
“misses”. Low bias for warning forecasts.
Behavior of Model Forecasts in Long Island
Sound Similar to Chesapeake Bay/Baltimore
• Used NOAA-EPA model
forecasts for locations of
all CT monitors.
• As in Baltimore,
exploratory analysis with
CART suggests domain
wide average peak O3
([65-70 ppbv])and/or
land-sea gradient ([15-20
ppbv]) are useful
thresholds for analysis.
Results of Post Processed Forecasts (Statewide)
0.9
0.8
0.7
0.6
0.5
Hit
0.4
False Alarm
0.3
Threat
0.2
0.1
0
NAQC
Mean
Mean+Grad
Forecast
Model forecasts (NAQC) have a high hit and false alarm rate (high bias) while
the expert forecast (Forecast) is the reverse. The two post-processed forecasts
Mean and Mean plus Gradient straddle the bias threshold but provide
better skill scores (Threat Score, or CSI shown above in green.
Summary and Conclusions
• In populated NE US Corridor, numerical model O3 forecast
guidance is useful but has high false alarm rate due to
modeled steep O3 gradients along land-sea boundaries.
• A variety of simple post-processing methods, available
within operational time constraints, can limit false alarm
frequency.
• Efficacy of methods vary from location to location and
include:
– Removal of coastal locations
– Bias correction (of various lengths)
– Domain mean peak concentrations and/or strength of gradient
Acknowledgements
• This research is supported by ongoing grants
from the Delaware Valley Regional Planning
Commission (including the State of Pennsylvania
Department of Environmental Protection) and
the State of Delaware Department of Natural
Resources and Environmental Control.
• Assistance for this presentation also provided by
Sonoma Technology (Dianne Miller and Jessica
Johnson) and Michael Geigert of the Connecticut
Department of Environmental Protection.
Additional Slides
Preliminary Comments
• This presentation is from the operational
forecaster’s perspective.
• Air quality forecasting in the US is a state/local
government responsibility.
• The air quality forecasting community in the
US is therefore non-centralized and diverse in
terms of expertise, experience and resources.
Sea Breeze Fronts Can Lead to High O3
NOAA-EPA Model O3 forecast for
August 11, 2010 (1200 UTC run)
HPC Surface Analysis, 1800 UTC,
August 11, 2010
Hi-Res NAM Forecast:
Sea Breeze Front Forming Behind Frontal Boundary
Sea Breeze is Observed
Sea Breeze Verified
(left) 3-hour temperature change
and winds,
1900 UTC, August 11
Figure Courtesy:
SPC Mesoscale Analysis
http://www.spc.noaa.gov/exper/mesoanalysis/
Example of Mean Domain Model O3 and
Observations
Scatter plot of observed peak
O3 and mean model
Air Quality Index (AQI) for
in CT.
CT Domain Wide Code Orange Threshold
Forecasts: Model and Expert
1.4
1.28
1.2
NAQC
Forecast
1
0.84
0.8
0.71
0.58
0.6
0.4
0.34
0.18
0.2
0
Bias
Hit
False Alarm
Bias at Code Orange Threshold
1.2
1.1
1
0.9
0.8
0.7
0.6
Mean
Mean+Grad
Connecticut Miss and False Alarm Cases
• Missed cases (4) were marginal Code Orange
cases (76-78 ppbv) with only one hour
exceedance in each case.
• Three of seven False Alarms were “close calls”
(≥72 ppbv) and remaining four were False Alarms
under any of the post-processing methods.
• False Alarms: 6/20 (lingering clouds), 7/20
(clouds/frontal boundary), 8/1 (SE winds),
8/31(?).