Interagency Monitoring of
Protected Visual Environments
(IMPROVE) Data Precision
Nicole P. Hyslop and Warren H. White
IMPROVE Collocated (Duplicate)
Measurements
• Established for the first time in 2004.
• Seven collocated modules of each type.
• Two objectives
– Evaluate precision of measurements and
– Determine if uncertainty estimates are accurate.
• Captures uncertainty resulting from sample
•
collection, sample analysis, and data processing.
Does not capture uncertainty resulting from
temperature uncertainty or calibration
uncertainty.
Module
Type
A
PM2.5
mass &
elements
B
PM2.5
anions
Site
Code
Site Name
Stat
e
Start
Date
MEVE
Mesa Verde NP
CO
8/13/03
PMRF
Proctor Maple
Research Facility
VT
9/3/03
OLYM
Olympic NP
WA
11/8/03
PHOE
Phoenix
AZ
3/30/04
SAFO
Sac and Fox Nation
KS
11/20/0
3
TRCR
Trapper Creek
AK
6/22/04
SAMA
Saint Marks
FL
11/18/0
4
LAVO
Lassen Volcanic NP
CA
4/18/03
MACA
Mammoth Cave NP
KY
5/12/03
BIBE
Big Bend NP
TX
8/30/03
GAMO
Gates of the
Mountains WA
MT
9/23/03
FRRE
Frostburg Reservoir
MD
4/15/04
BLMO
Blue Mounds State
Park
MN
9/16/04
PHOE
Phoenix
AZ
3/30/04
Module
Type
C
PM2.5
organic
&
elemental
carbon
D
PM10
mass
Site
Code
Site Name
Stat
e
Start
Date
EVER
Everglades NP
FL
7/11/03
SENE
Seney NWR
MI
8/10/03
HOOV
Hoover WA
CA
8/13/03
MELA
Medicine Lake NWR
MT
9/25/03
PHOE
Phoenix
AZ
3/30/04
SAWE
Saguaro NP West
AZ
3/25/04
HEGL
Hercules-Glade WA
MO
9/15/04
JOSH
Joshua Tree NP
CA
8/7/03
QURE
Quabbin Reservoir
MA
9/4/03
HOUS
Houston
TX
4/4/03
JARB
Jarbridge WA
NV
6/30/04
PHOE
Phoenix
AZ
3/30/04
WICA
Wind Cave NP
SD
9/17/04
SWAN
Swanquarter WA
NC
11/9/04
Uncertainty and Precision in this
Presentation
• Uncertainty refers to the expected
variance in outcomes of measurements
repeated under the same conditions.
• Collocated precision refers to the variance
of relative differences between the
duplicate measurements.
Collocated Sulfur Data
Sulfur
1.5
SITES
0.2
MDL
MEVE
OLYM
PHOE
PMRF
SAFO
TRCR
0.0
-0.2
1.0
Relative Difference
Arithmetic Difference (ug m-3)
0.4
0.5
0.0
-0.5
-1.0
-0.4
-1.5
0.0
0
01
0
10
0
.
0
0
00
1
.
0
00
0
1.0
Average Concentration (ug m-3)
10
0
0.0
1
0.0
00
00
0
0.1
0
1.0
00
Average Concentration (ug m-3)
Collocated Selenium Data
Selenium
1.5
SITES
0.00015
MDL
MEVE
OLYM
PHOE
PMRF
SAFO
TRCR
0.00000
-0.00015
1.0
Relative Difference
Arithmetic Difference (ug m-3)
0.00030
0.5
0.0
-0.5
-1.0
-0.00030
0.00001
0.00010
0.00100
AverageConcentration(ug m-3)
-1.5
0.00001
0.00010
0.00100
AverageConcentration(ug m-3)
Collocated Precision Calculations
1  Yi    X i  / 2 

 *100%
Collocated Precision 

n i 1  Yi    X i  / 2 
n
2
Yi  denotes routine concentrat ion,
Yi  denotes collocated concentrat ion,
n is number of sample pairs
• U.S. EPA designated method (FRM, 1997)
• Only use pairs where the average
concentration is greater than 3 * MDL.
Estimated Uncertainty for each
Concentration
• An uncertainty estimate is reported with every
species concentration.
– Estimates are based on individual sources of
measurement uncertainty and propagation of errors.
UncC  
C   unc
2
2
analytic
 unc
2
volume
 Unc
constant
where [C ]  concentrat ion
uncanalytic  proportion al analytical uncertaint y
uncvolume  proportion al volume uncertaint y
Uncconstant  constant analytical uncertaint y

2
Why don’t the collocated precisions
meet the expectations?
• Analytical issues
– Analysis area
– Minimum detectable limits (mdl’s)
– Concentrations always close to mdl
• Sampling discrepancies
– Cyclone cutpoint differences
– Non-uniform deposits
– Deposit area
Copper-anode XRF Element Metrics
Species
Na
Mg
Al
Si
P
S
Cl
K
Ca
Ti
V
Cr
Mn
Fe
† 577 valid pairs
Count >
Estimated
3*MDL† Uncertainty
15
25%
9
n < 10
257
14%
533
11%
21
18%
577
5%
105
12%
577
6%
577
6%
528
11%
457
18%
108
28%
529
12%
577
5%
Collocated
Precision
50%
n < 10
69%
41%
131%
8%
68%
13%
19%
30%
21%
56%
24%
18%
Collocated/
Estimated
2.0
5.0
3.7
7.3
1.6
5.6
2.3
3.0
2.6
1.2
2.0
2.0
3.3
Sulfur by XRF and Sulfate by IC
Sulfur
Sulfate
1.5
1.5
MDL
MEVE
OLYM
PHOE
PMRF
SAFO
TRCR
0.5
0.0
-0.5
-1.0
0.5
0.0
-0.5
-1.0
-1.5
0
AVG_MDL
BIBE1
BLMO1
FRRE1
GAMO1
LAVO1
MACA1
PHOE1
1.0
Relative Difference
Relative Difference
1.0
1
.00
SITES
SITES
0
0
0
.01
0
0
0
.10
0
1
0
.00
0
-1.5
0.01
Average Concentration (ug m-3)
Species
Count >
3*MDL
Estimated
Uncertainty
0.10
1.00
10.00
Average Concentration (ug m-3)
Collocated
Precision
Collocated/
Estimated
Sulfur
577/577
5.3%
8.2%
1.56
Sulfate
801/823
4.8%
4.1%
0.86
Soil-derived Elements
Iron
Calcium
1.5
1.5
SITES
MDL
MEVE
OLYM
PHOE
PMRF
SAFO
TRCR
0.5
0.0
-0.5
Relative Difference
Relative Difference
1.0
S
1.0
0.5
0.0
-0.5
-1.0
-1.0
-1.5
-1.5
0.0001
0.0010
0.0100
0.1000
Average Concentration (ug m-3)
Species
Ca
Fe
Count >
3*MDL
577/577
577/577
0
0
.00
10
0
1
.00
00
0
0
.01
00
0
0
.10
00
Average Concentration (ug m-3)
Estimated
Uncertainty
6.2%
5.5%
Collocated
Precision
18.8%
17.9%
Collocated/
Estimated
3.0
3.3
collo Ca / routine Ca, Mo XRF
2
Phoenix
1.5
2004
1
2005
0.5
0
0
0.5
1
1.5
2
collo Ca / routine Ca, Cu XRF
2.5
collo PM2.5 / routine PM2.5, grav
1.5
Phoenix
1.25
2004
1
2005
0.75
0.5
0
0.5
1
1.5
collo Ca / routine Ca, Mo XRF
2
Minimum Detectable Limits (mdl)
Silicon
Chlorine
1.5
1.5
SITES
MDL
MEVE
OLYM
PHOE
PMRF
SAFO
TRCR
0.5
0.0
-0.5
-1.0
0.5
0.0
-0.5
-1.0
-1.5
0.0
S
1.0
Relative Difference
Relative Difference
1.0
0
01
0.0
0
10
0.1
0
00
1.0
0
00
Average Concentration (ug m-3)
Species
Si
Cl
Count >
3*MDL
533/577
105/577
Estimated
Uncertainty
10.9%
12.1%
-1.5
0.0001
0.1000
0.0100
0.0010
Average Concentration (ug m-3)
Collocated
Precision
40.5%
68.4%
Collocated/
Estimated
3.7
5.7
Entire network
January - April
1
2005
fraction detected
2004
2003
0.5
0
0.1
1
10
[Si val x (Al/Si)crust] / Al mdl
100
Original Criteria
Species
No.
Pairs
Both Detected
Collo /
Estimated
No.
Pairs
Original Criteria
Collo /
Estimated
Species
No.
Pairs
Both Detected
Collo /
Estimated
No.
Pairs
Collo /
Estimated
Na
15
2.0
71
1.1
PM2.5
539
1.0
577
0.6
Mg
9
-
45
1.0
H
577
1.3
577
1.3
Al
257
5.0
212
1.7
Ni
222
2.8
359
1.6
Si
533
3.7
500
2.0
Cu
499
2.5
512
1.8
P
21
7.3
3
-
Zn
575
4.0
571
3.7
S
577
1.6
577
1.6
As
60
1.1
275
0.9
Cl
105
5.6
95
1.4
Se
406
1.1
500
1.1
K
577
2.3
577
2.3
Br
576
1.2
577
1.2
Ca
577
3.0
576
2.9
Rb
64
1.2
256
0.8
Ti
528
2.6
520
1.8
Sr
266
2.1
469
1.3
V
457
1.2
485
1.0
Zr
5
-
21
1.2
Cr
108
2.0
195
1.1
Pb
541
1.8
545
1.5
Mn
529
2.0
546
1.8
Fe
577
3.3
577
3.3
BADL1
even
odd
0.1
Cu, ug/m
3
0.01
0.001
0.0001
NON 0.00001
DETECT
Jan-02
Jan-03
Jan-04
Jan-05
YNA
YMG
YAL
YSI
YP
YS
YCL
YK
YCA
YTI
YV
YCR
YMN
YFE
percent of 1/04 samples throughout network yielding 'quantitative' values (ERR/VAL < 10%)
0%
0%
13%
74%
1% 100%
10%
99%
98%
67%
5.8%
5.1%
3.7%
1.5%
0.7%
0.1%
0.2%
7.4%
3.6%
1.0%
0.3%
2.0%
0.6%
15%
1%
49%
99%
percent of quantitative values below field blank (FB) levels
95%ile 1/04 FB
90%ile 1/04 FB
75%ile 1/04 FB
50%ile 1/04 FB
2.9%
percent of quantitative values below acceptance test (AT) levels
95%ile 10/04 AT
90%ile 10/04 AT
75%ile 10/04 AT
0.3%
FE
NI
CU
0.7%
0.2%
ZN
AS
PB
SE
BR
RB
SR
Y
ZR
CD
percent of 1/04 samples throughout network yielding 'quantitative' values (ERR/VAL < 10%)
99%
9%
34%
99%
3%
37%
percent of quantitative values below field blank (FB) levels
95%ile 1/04 FB
90%ile 1/04 FB
75%ile 1/04 FB
50%ile 1/04 FB
5.7%
0.9%
35.4%
20.1%
3.7%
0.1%
percent of quantitative values below acceptance test (AT) levels
95%ile 10/04 AT
90%ile 10/04 AT
75%ile 10/04 AT
0.2%
26.0%
3.2%
26%
91%
0%
9%
0%
0%
0%
Carbon
Fraction
OC1
OC2
OC3
OC4
OP
EC1
EC2
EC3
OC
EC
Analytical
Uncertainty,
uanalytic
27%
16%
11%
13%
27%
13%
26%
40%
-
2004
Replicate
Precision
41%
17%
19%
20%
50%
24%
31%
66%
11%
21%
Precision/
Uncertainty
1.5
1.1
1.7
1.5
1.9
1.9
1.2
1.7
-
Conclusions and Future Work
• Estimated uncertainties are not accurate
for most species
– Additional sampling-related sources of
uncertainty must be considered
– Sample deposit uniformity must be measured
– Cyclone collection efficiency must be modeled
• Minimum detectable limits (mdl’s) are too
low for several species
– Collocated data will be used to evaluate mdl’s
Species
Species
with Decent
Agreement
between
Precision
and
Uncertainty
PM2.5
PM10
Na
S
V
Mn
As
Pb
Se
Br
Rb
H
Nitrate
Sulfate
OC1
OC2
OC3
OC4
OP
EC1
EC2
OC
EC
TC
Estimated
Uncertainty
7%
5%
25%
5%
18%
12%
14%
12%
14%
7%
24%
6%
8%
5%
33%
22%
17%
17%
29%
16%
31%
12%
12%
11%
Collocated Collocated/
Precision
Estimated
6%
8%
50%
8%
21%
24%
15%
22%
15%
9%
28%
8%
10%
4%
44%
18%
24%
26%
45%
24%
37%
17%
22%
17%
1.0
1.6
2.0
1.6
1.2
2.0
1.1
1.8
1.1
1.2
1.2
1.3
1.3
0.9
1.3
0.8
1.5
1.5
1.5
1.5
1.2
1.5
1.8
1.5