An Extended Approach to Calculate the Ozone Relative Response Factors used in the
Attainment Demonstration for the National Ambient Air Quality Standards
Sarika Kulkarni1,*, Ajith P. Kaduwela1, 2, Jeremy C. Avise1, 3, John A. DaMassa1,
and Daniel Chau1
1
Air Quality Planning and Science Division, Air Resources Board,
California Environmental Protection Agency, Sacramento, CA 95814, USA
2
Air Quality Research Center, University of California, Davis, CA 95816, USA
3
Department of Civil and Environmental Engineering, Washington State University, Pullman, WA 99164, USA
Key Words: Relative Response Factors; RRF; 1-hour ozone, 8-hour ozone; State Implementation
Plans; National Ambient Air Quality Standards; Band-RRFs.
*
Please address correspondences to: Dr. Sarika Kulkarni, Air Quality Planning and Science
Division, Air Resources Board, California Environmental Protection Agency, 1001 I Street,
Sacramento, CA 95814, USA; Tel: +1-916-324-0594; Fax: +1-916-327-8524; E-Mail:
skulkarn@arb.ca.gov
SUPPLEMENTAL MATERIALS
EVALUATION OF MODEL PERFORMANCE
The U.S. EPA modeling guidance (U.S. EPA, 2007) recommends calculation of model
performance statistics both with and without a pre-defined threshold for 8-hour ozone (O3) as the
high/low ozone values reflect the different processes for formation/destruction and transport of
O3. A recent review paper by Simon et al., (2012) has summarized the model performance of air
quality model simulations for O3, PM2.5, and deposition parameters and found that 8-hr O3 is
overestimated when using all data and underestimated when using high ozone days (> 60 and 75
ppb cutoff). The O3 bias/error approach statistics can be skewed by data points taken at low
ambient concentrations. The study by Simon et al., (2012) recommends using a minimum
1
threshold of 60 ppb for calculation of 8 –hour O3 model performance statistics particularly for
analysis focused on elevated O3 levels. The study also suggests using a binned approach based
on observed O3 for model performance evaluation.
It is important to note that the current RRF concept was developed by the U.S. EPA for the 8hour O3 and PM2.5 NAAQS (U.S. EPA, 2007) and there is no comparable concept for 1-hour O3.
To assess the model prediction skills in simulating O3, a model performance analysis was carried
out for the 2007 base case by including simulated O3 values above 60 ppb threshold and using
statistical metrics recommended by the U.S. EPA for 1-hour O3 (U.S. EPA, 1991). The
statistical metrics include peak (unpaired) prediction accuracy (±15-20 percent), overall bias (±515 percent), and gross error statistics (30-35 percent). The acceptable ranges indicated in the
parenthesis are approximate and were recommended by Tesche at al. (1990).
The model performance analysis was conducted for sub regions of the modeling domain (Figure
1 in main manuscript) and results are summarized in Table S1. SJVAPCD stands for the San
Joaquin Valley Air Pollution Control District in Table S1. The northern region consists of San
Joaquin and Stanislaus counties. We define the central region as the portion of Merced,
Mariposa, Fresno, Madera, King, and Tulare counties that are below 3000 feet elevation. The
rest of these counties are labeled “SJVAPCD above 3000 ft.” Only the western portion of Kern
County, which lies in the jurisdiction of the SJVAPCD, is included in the Kern sub region. The
boundaries of these four sub regions are shown in Figure 1 of the main manuscript.
Table S1 shows the total number of days in each month that meet the model performance criteria
by region. The SJVAPCD north region has the least number of days that meet the model
performance criteria for both 1-hour and 8-hour O3 among all regions. This is due to a combined
effect of a large number of days that had simulated values below the 60 ppb threshold,
overestimation of simulated ozone in early summer events (May through mid-July),
and the availability of a relatively fewer number of monitoring sites (Modesto, Turlock and
Stockton as seen in Figure 1) for the model performance evaluation.
2
We have applied the model performance criteria from a relatively older 1-hour O3 U.S. EPA
guideline (U.S. EPA, 1991) to assess the model prediction skills under specified 1-hour O3
guidelines. However, it is quite possible that large biases may exist even if the O3 simulations
represent the fundamental physical and chemical processes. There are also instances when low
model biases are seen when the model fails to capture the essential processes impacting the
predicted ozone values. The U.S. EPA modeling guidance (U.S. EPA, 2007) recognizes this fact
and hence, deemphasized the use of model performance criteria explicitly in model attainment
demonstrations.
REFERENCES
Simon et al. (2012) Compilation and interpretation of photochemical model performance
statistics published between 2006 and 2012. Atmos. Environ. 61:124-139
Tesche, T.W., P. Georgopoulos, J.H. Seinfeld, G. Cass, F.L. Lurmann, and P.M. Roth. 1990.
Improvement of Procedures for Evaluating Photochemical Models. Draft Final Report.
California Air Resources Board, Sacramento, CA. Found at
http://gate1.baaqmd.gov/pdf/0122_Improvement_Procedures_Evaluating_Photochemical_Model
s_1990.pdf
U.S. EPA. 1991. Guideline for Regulatory Application of the Urban Airshed Model. EPA450/4-91-013. Found at http://www.epa.gov/ttn/scram/guidance_sip.htm
U.S. EPA. 2007. Guidance on the Use of Models and Other Analyses for Demonstrating
Attainment of Air Quality Goals for Ozone, PM2.5, and Regional Haze. EPA-454/B07-002.
Found at http://www.epa.gov/scram001/guidance/guide/final-03-pm-rh-guidance.pdf
3
LIST OF TABLES:
Table S1: Model performance shown as the number of days meeting standard statistical metrics
for both 1-hour and 8-hour O3 by month in sub regions of modeling domain. (See Figure 1 for
details of the sub region boundaries used in this analysis).
Simulation Period (May through September 2007)
1-hour O3
Region
May
Jun
Jul
Aug
Sep
All Months
SJVAPCD North
7
5
1
2
5
20
SJVAPCD Central
20
23
12
16
17
88
SJVAPCD above 3000 ft.
12
12
15
6
10
55
Kern
21
24
17
11
12
85
All Region
60
64
45
35
44
248
8-hour O3
Region
May
Jun
Jul
Aug
Sep
All Months
SJVAPCD North
6
2
2
3
3
16
SJVAPCD Central
20
22
12
16
12
82
SJVAPCD above 3000 ft.
13
6
22
8
8
57
Kern
21
25
24
13
12
95
All Region
60
55
60
40
35
250
SJVAPCD stands for the San Joaquin Valley Air Pollution Control District
4