Assessment of Uncertainty in Urban Air Toxics
Simulations: Monte Carlo Study of the Houston Ship
Control # 1160
David Heinold, Robert Paine and Elizabeth Kintigh
ENSR International, 2 Technology Park Drive, Westford, MA 01886
Steven Hanna
Steven R. Hanna, Hanna Consultants, 7 Crescent Ave., Kennebunkport, ME 04046
Dan Baker
Shell Refining, Houston Texas
Richard Karp
American Petroleum Institute, 1220 L Street NW, Washington, DC 20005
ABSTRACT
Because of the interest in accurately predicting air toxics exposure in urban areas, a
Monte Carlo (MC) probabilistic uncertainty study has been conducted by the American
Petroleum Institute (API). The API study, based on a 15 km by 15 km receptor domain
centered on the Houston Ship Channel, complements a MC uncertainty study which was
carried out by the EPA for the entire Houston metropolitan area. The focus of the API
study is on uncertainties in ISCST3 and AERMOD predictions of annual averaged
concentrations of benzene and 1,3-butadiene, due to uncertainties in emissions and
meteorological inputs. The uncertainties in emissions components were estimated based
on observed data variability supplemented by guidance from an API-EPA workshop held
on this topic (typical emissions uncertainties are about +/- a factor of three (i.e., covering
the 95 % range) for 21 benzene emissions categories and 13 1,3-butadiene emissions
categories). The uncertainties in meteorological inputs (such as wind speed, wind
direction, mixing height, cloud cover and temperature gradient) were also determined
from analysis of the field data plus consultation with experts. Uncertainty in the
dispersion coefficients applied in the models were also evaluated. ISC3ST and AERMOD
were each run 100 times in MC mode, using random and independent perturbations of all
inputs in order to estimate 1) the total uncertainty of the annual averaged concentrations,
and 2) the inputs with uncertainties that are most strongly correlated with uncertainties in
predicted concentrations. The results of the MC runs with ISCST3 and AERMOD in
terms of overall uncertainty are discussed and the primary sources of uncertainty are
identified. The implication of these findings to urban air toxics modeling is also
discussed.
1
STUDY OBJECTIVES
The U. S. Environmental Protection Agency (EPA) Office of Air Quality Planning and
Standards (OAQPS) published guidance for modeling air toxics in urban areas as part of
EPA's Integrated Urban Air Toxics Strategy (IUATS). Because the UATS represents
EPA OAQPS guidance for modeling air toxics in urban areas, the American Petroleum
Institute (API) is interested in subjecting the methods employed to a thorough sensitivity
analysis. A previous studies sponsored by API examined the sensitivity of these modeling
techniques to modeling parameters.1 The current paper involves a quantitative Monte
Carlo evaluation of the level of uncertainty involving a wide range of parameters. The
study area for both the sensitivity study and the Monte Carlo analysis is a 15 km by 15
km portion of EPA's Houston air toxics example application, centered on the Houston
Ship Channel.
The UATS Houston case study is of particular interest because the emissions and
modeled the impacts of mobile and industrial sources are comparable and the
contribution of petrochemical-related activities to hydrocarbon emissions is pronounced.
The study evaluates the modeling uncertainty of two hazardous air pollutants (HAPs)
benzene and 1,3-butadiene. Two dispersion models are used in the analysis: ISCST3,
which is the basis of current EPA guidance, and AERMOD (with PRIME), which will
soon replace ISCST3 in EPA's Guideline on Air Quality Models2.
In the previous sensitivity study, the various models and parameters are varied in a oneat-a-time procedure, allowing the sensitivity to each particular parameter to be clearly
determined. In the Monte Carlo uncertainty study the modeling parameters are
independently varied. In the Monte Carlo probabilistic uncertainty methodology, the
modeling system is run 100 times for random choices of variations in the input
parameters and the responses of the key model output parameters are analyzed.
The objective of the study is to gain insight into the following questions:
Question 1: What is the total uncertainty in the annual average (among all
receptors and maximum at any receptor) and which input variables and model
parameters have the most influence on this total uncertainty?
Question 2: What is the relative uncertainty between the emissions and the
transport and dispersion model?
Question 3: How do the total uncertainties and correlations differ for different
source categories and could these differences impact conclusions regarding source
apportionment?
Question 4: Are conclusions concerning uncertainty depended on model?
HOUSTON STUDY AREA
The Houston Ship Channel, which contains a wide variety of source types as well as
transportation routes and heavily populated areas. The study area is shown in Figure 1.
The inner portion of the study area is a 15 km x15 km area where model receptors are
located and a 30 km x 30 km outer portion where additional emission sources are
modeled. The larger emissions area is shifted slightly to the east in relation to the
2
receptor area to capture major point sources. This nested design provides a buffer area
that helps to reduce "edge effects". Most of the study area is comprised of “urban”
sources as specified by EPA, based on the land use within a 1 km grid.1. The previous
sensitivity analysis identified the specification of rural sources within a highly urbanized
area as a source of substantial uncertainty2. Therefore, for the Monte Carlo analysis
ISCST3 was applied with mix of rural and urban sources and all urban sources.
AERMOD was run using the EPA rural/urban designations.
Figure 2 shows the forty-three model receptor used by EPA1.2 to represent population
centroids of census tracts, plus three ambient air quality monitors located within the study
area. The focus is on annual average concentrations, since that model output is used by
EPA1 as the basis for estimating health effects to the population. The modeled
concentration averaged over this set of centroid receptors is representative of the ambient
air to which the population is exposed. In addition, the Monte Carlo analysis evaluated
the maximum annual concentration among all receptors.
MONTE CARLO PROCEDURE
The Monte Carlo analysis consisted of 100 separate runs using the 1996 meteorological
data from Busch International Airport. Two types of uncertainty were evaluated, the
uncertainty associated with each run (“site-to-site” uncertainty) and, for some parameters,
hour-to-hour uncertainty. Depending on the parameter either a log-normal or normal
distribution of the uncertainty was evaluated. Log-normal distributions were applied to
parameters that are by nature positive and unbounded (such as wind speed) and normal
distributions are applied to bounded variables (such a cloud cover).
Table 1 lists the parameters that were varied on a site-to-site basis (100 variations) and on
an hourly basis (8784 variations for 1996, a leap year). Random numbers were generated
for each modeling parameter listed in Table 1 and for individual benzene and 1,3butadiene emissions categories. Each random distribution was checked to verify that the
desired statistical attributes were approximated and that no value deviated from the mean
by more than 5 times the standard deviation.
Table 1a Log-Normally Distributed Modeling Parameters
Parameter
Emissions
Wind Speed
Mixing Height
Surface Roughness
Bowen Ratio
dT/dz
σy and σz
Model
AERMOD
ISCST3
x
x
x
x
x
x
x
x
x
x
x
x
Variability
Site-to-site Hourly
x
x
x
x
x
x
x
x
x
x
x
x
95% Uncertainty
Factor
3
1.3
1.2
3
2
2
1.5
Geometric
Stand. Dev.
0.549
0.131
0.091
0.549
0.347
0.347
0.203
Arithmetic
Mean
Stand. Dev.
1.163
0.690
1.009
0.133
1.004
0.092
1.163
0.690
1.062
0.379
1.062
0.379
1.021
0.209
Table 1b Normally Distributed Modeling Parameters
Parameter
Wind Direction
Cloud Cover
AERMOD
x
x
ISCST3
x
x
Site-to-site
x
x
Hourly
x
x
Arithmetic SD
15
0.05
Range
zero to 360
Zero to 1
3
Figure 1. Facility-Related Sources in the API Houston Monte Carlo Analysis Study
4
Figure 2 Discrete Receptors (large dots) and Monitor Locations (triangles)
5
The these parameter uncertainty distributions were based on professional judgment and
were determined in collaboration with EPA, which had conducted a separate uncertainty
assessment, in parallel with the API-sponsored uncertainty study. The two groups (EPA
and API project managers and their contractors) collaborated by exchanging work plans,
data files and ideas, and by having conferences calls and meetings. For example,
Scientists from both groups participated in the emissions uncertainty workshop on 26-27
August 2003 from which the source categories and uncertainty levels were established.4
EMISSION CATEGORIES
The benzene and 1,3-butadiene emission categories are listed in Table 2 and 3,
respectively. The category assignments and numbering scheme was adapted from the
categories used in EPA’s air toxics uncertainty assessment. Due to their small
contributions, as noted, some categories were combined with categories with more
substantial emissions. Individual sets of random numbers were generated for each
emission category. The thousands of modeled sources include sources representing
individual facility emission points as well as gridded area sources which include
contributions of several emission categories. For individual sources the corresponding
emission category uncertainty factor was applied. For gridded sources the effective
uncertainty factor was computed according to the emission-weighted contribution of each
contributing emission category.
ANALYSIS OF MONTE CARLO SIMULATIONS
The100 AERMOD and 300 ISCST3 runs were executed to estimate annual average
concentrations of benzene and 1,3-butadiene. For ISCST3 two sets of concentrations
were developed, 1) mixed urban and rural sources and 2) all sources as urban. For
AERMOD one set of results was produced, representing mixed rural and urban sources.
Receptors were placed at 43 population centroids and three monitoring locations shown
in Figure 2. Individual results for all 46 receptors have been archived.
The following summary concentrations were calculated for each model configuration:

Spatial average concentration over the 436 population-based receptors and

Peak concentration among all 469 receptors and receptor location.
6
Table 2 Emission Categories for Benzene
Category
Description
1&2
Light Duty Gas Vehicles (LDGV), Light Gas Trucks (LDGT),
Combined Road Segments
TPY
% of
Total
474.972
28.5
3
Petroleum refineries
412.657
24.7
4
Nonroad 4-stroke gas engines, Internal Combustion Engines
145.766
8.7
5
Nonroad 2-stroke gas engines
34.271
2.1
6
Nonroad diesel (construction, farm, and industrial)
26.217
1.6
7
Oil and gas production
10.244
0.6
8
Natural gas transmission and marine transport
63.677
3.8
9
Heavy Duty Gas Vehicles (HDGV) (Assigned to 1&2)
0.000
0.0
10
Forest wildfires, Municipal Landfills
5.842
0.4
11
Solid waste disp (sewage treatment, aeration tanks)
59.182
3.5
12
Acetylene production (butylene, ethylene, propylene,olefin)
47.850
2.9
13
Fuel oil external combustion, External Combustion Boilers
37.861
2.3
14
Typical ethylene plant
16.967
1.0
15
Gas service stations stage 1
9.616
0.6
16
Petroleum industry fugitives
26.838
1.6
17
Managed burning, prescribed
0.654
0.04
18
Heavy Duty Diesel Vehicles (HDDV) (Assigned to 1&2)
0.000
0.0
19
Chemical manufacturing; fugitive emissions
16.718
1.0
20
Aircraft
6.533
0.4
21
Petroleum industry; fugitive emissions; misc., Evap. losses
121.826
7.3
22
Process vents in refinery production
14.954
0.9
23
Loading, ballasting, transit losses from marine vessels
Chemical manufacturing, general processes, fugitive leaks
Assigned to 19)
Industrial Processes
Total Emissions
21.596
1.3
24
25
0.000
0.0
113.347
6.8
1667.588 100.0%
7
Table 3 Emission Categories for 1,3-Butadiene
Category and Description
1
2
3
4
5
6
7
8
9
10
11
12
13
Fuel oil external combustion, petroleum and solventt
evaporation,organic solvent evaporation, fuel fired equipment,
natural gas, flares, industrial processes, petroleum industry,
process gas
Styrene-butadiene rubber and latex production, nitrile butadiene
rubber production
Chemical manufacturing fugitive emissions, industrial
processes, general processes, fabricated metal products
fugitive emissions, plastics production
Industrial processes, chemical manufacturing, butadiene
fugitive emissions
Ethylene plant, inidustrial processes chemical manufacturing
butylenes. Ethylene propylene, olefin production fugitives
emissions
Loading, ballasting, transit losses from marine vehicles
Industrial processes, petroleum industry cooling towers and
fugitive emissions from flanges and all streams
Aircraft
Unknown
Road Segments
On-road Gridded
Non-road
Non-point
Total Emissions
TPY
% of
Total
271.774 40.1
105.800 15.6
118.848 17.5
17.099
2.5
26.257
10.660
3.9
1.6
13.886
5.120
6.420
42.438
29.975
17.523
12.647
678.447
2.0
0.8
0.9
6.3
4.4
2.6
1.9
100.0
8
The following statistical analyses were conducted for each of the summary
concentrations:

Cumulative frequency distribution plots

5th, 10th, 25th 75th, 90thand 95th percentile values

Rank correlation coefficient for spatial average concentrations.

Rank correlation coefficient for peak receptor concentration.

Scatter plots of the modeled concentration versus perturbed variables
In addition, the rank correlation coefficient were calculated for each for each of the 469
receptors. The goal of these statistical analyses is to identify the parameters that
contribute most to modeling uncertainty. Five parameters with the largest correlation
coefficients have been included in a multiple linear regression analysis.
DISCUSSION and CONCLUSIONS
[Authors’ note: The Monte Carlo analysis is in its final stages and
results are still being generated at the January 24th draft manuscript
deadline. We will replace this draft manuscript with a complete
manuscript suitable for review in early February. The author’s
appreciate the consideration of the reviewer’s in this matter. ]
REFERENCES
1. US EPA. Example Application of Modeling Toxic Air Pollutants in Urban Areas
U.S. EPA OAQPS (EPA-454/R-02-003). (2002).US EPA.
2. Heinold, D., B. Paine, and H. Feldman, 2003: Quantitative evaluation of the EPA
urban air toxics modeling strategy: Results of sensitivity studies. Paper number
69639, Proceedings of AWMA Annual Conference, San Diego, June
3. 68 FR 18440 ENVIRONMENTAL PROTECTION AGENCY 40 CFR Part 51,
Revision to the Guideline on Air Quality Models: Adoption of a Preferred Long
Range Transport Model and Other Revisions: Final rule (April 15, 2003).
4. Hanna, S.R., 2003: Summary of 26-27 August 2003 Houston Emissions
Uncertainty Workshop on Benzene and 1,3-Butadiene. Prepared for the
9
American Petroleum Institute by Hanna Consultants, 7 Crescent Ave.,
Kennebunkport, ME 04046, 31 pages
KEYWORDS
Air toxics
Urban dispersion
Modeling sensitivity
10