Quality Assurance/
Quality Control
Nate Herbst
Southern Ute Indian Tribe
Intro to QA/QC
• Getting good data requires many different
steps
– Data quality objectives (DQOs) developed
• DQOs for ozone being developed
• Measurement quality objectives (MQOs) for ozone exist
– Analysis begun (after correct calibration)
– QC checks performed
– QA conducted
2
Data Quality Objectives (7 steps)
• State problem
– Define why monitoring is needed
– Create team and purpose
• Identify decision
– What decision will be made with data?
• Identify decision inputs
– What data necessary to make decision?
• Define boundaries
– What are study area boundaries?
3
Data Quality Objectives (cont.)
• Develop decision rule
– What conditions will require action (action level)?
• Specify decision error limits
– What margin of error is allowable
• Optimize monitoring design
– Develop most cost-effective method of reaching
DQOs
• EPA hasn’t yet defined DQOs for ozone
analysis
4
DQOs
(cont.)
Diagram of
DQO steps
(Diagram courtesy of
U.S. Department of
Energy – DQO
homepage)
5
Measurement Quality
Objectives (MQOs)
Thanks to
Melinda RoncaBatista (ITEP)
• EPA has ozone analysis MQOs
• Use these in element 7 of your QAPP
• MQOs in a nutshell
–
–
–
–
Shelter temperature kept between 20-30oC ± 2oC
Analyzer must be reference or equivalent method
Lower detectable limit 0.01ppm, noise 0.005ppm
Data completeness 75% of hourly values between
9:01am and 9:00pm (for the ozone season)
– Transfer standard certification ±4% or 4ppb
(whichever is greater)
6
MQOs (cont.)
– Transfer standard re-certification to primary
std.dev 1.5%
– Local primary standard certification ±5% of
reference
– EPA reference photometer regression slope 1.00
± 0.01
– Zero air free of O3 and anything that might react
with O3
7
MQOs (cont.)
– Ozone analyzer calibration
• Z/S check zero ±10ppb, span ± 15%
• 5pt calibration linearity error ±5%
– Performance (NPAP) mean absolute
difference ± 15%
– Precision (quarterly) 95% CI < ±15%
– Audits (annually) 95% CI < ±20%
8
Quality Assurance Project Plans
(QAPPs)
• Contain 24 “elements”
– Element 7 is where MQOs go
– Cut and paste from red book
•
•
•
•
Ensure data quality
Required by EPA
Developed by program approved by EPA
They must be followed!
– No good if not followed
9
Documentation
• Document everything!!!
• Documentation in
–
–
–
–
Logbooks
Site folders
QA/QC field forms
Anywhere else you think is appropriate
• QA/QC – document standard values and
response
10
Documentation (cont.)
• Document repairs, checks, fine tunes
• Document site conditions
• Document everything that could ever be
important
• Write only in pen (black if possible)
• Cross out errors with a single line
11
Linearity
• Slope = rise over
run
• m = slope
• b = intercept (where
Y axis
y=x
6
5
4
y=x
R2 = 1
3
2
1
0
b
0
2
m
4
6
X axis
• r2 close to 1 shows
correlation
Y axis
y = mx+b
14
12
10
8
6
4
2
0
y = 2.0829x + 1.9095
R2 = 0.997
0
2
X axis
the trend-line crosses the Y
axis)
4
6
12
Instrument Calibration
• Measurements require point of reference
• Measurement without standard is impossible
• Calibration involves setting instrument to
known level
• Calibrations performed fairly regularly
– When monitoring is begun
– When repairs or maintenance are performed
– When precision checks or audits show need
• Calibrations must be done correctly
13
Calibrations (cont.)
• Calibration = setting analyzer to standard
– Data only good within linear range (~0-0.400ppm)
• Calibration followed by a 5-pt check
• Analyzer must agree with standard at all 5 pts
– Linearity error < 5%
– See next slide on linearity
• Monitoring begins after calibration
Note: Never initiate
monitoring without calibration
14
Pre-Calibration Check
5pt check
0.500
Analyzer
0.400
y = 1.1368x + 0.0203
R2 = 0.98
0.300
0.200
0.100
0.000
0.000
0.100
0.200
0.300
0.400
0.500
Standard
Standard
Analyzer
% dif.
0.000
0.000
0.0
0.080
0.100
-25.0
0.150
0.220
-46.7
0.250
0.340
-36.0
0.350
0.400
-14.3
0.400
0.460
-15.0
• Not always
necessary
• Can do 5-pt check
• Analyzer must be
calibrated
• The r2 value and %
differences for each
point are
unacceptable
15
Instrument Calibration
• Calibrate instrument to the standard
• Use calibration point near URL
– Setting low produces large error at URL
•
•
•
•
Set standard to ~0.400 ppm
Let analyzer stabilize
Calibrate analyzer
Do new 5-pt check
16
Post-Calibration 5-pt Check
Calibration
Analyzer
0.600
0.400
y = 1.0145x - 0.0006
R2 = 0.9997
0.200
0.000
0.000
0.100
0.200 0.300
Standard
0.400
0.500
Standard
Analyzer
% dif.
0.000
0.000
0.0
0.080
0.081
-1.3
0.150
0.153
-2.0
0.250
0.248
0.8
0.350
0.354
-1.1
0.400
0.408
-2.0
• Is analyzer
response within 7%
at each point?
• Would you put this
analyzer online?
17
Quality Control (QC)
• QC involves “in-house” verifications
• Also referred to as precision checks
• Verifications are comparisons between
transfer standard and analyzer
– Relative % difference within allowable margin?
• Verifications determine monitoring
repeatability
– Standard deviation
• Different types of verifications
18
QC (cont.)
• Level 1: 40 CFR, Pt. 58, App. A, Table A-1
defines ozone verification requirements (for
SLAMS)
– Biweekly response check between 0.08 and 0.1
ppm
• Comparison between analyzer and standard
– Determines repeatability
• Level 2: “extra” precision checks
– Weekly “span level” (~80% URL) checks
– Quarterly 5-pt checks
– Determines analyzer performance trends
19
Quality Assurance (QA)
• QA involves “external” checks
• Referred to as “audits”
• Audits involve comparison between transfer
standard and analyzer
– Accuracy levels must be within ±10%
• Audits determine how close monitoring gets
to actual values
• Different types of audits
20
QA (cont.)
• 40 CFR, Pt. 58, App. A, Table A-1 defines
ozone audit requirements (for SLAMS)
– Annual (and other) response checks at multiple
points
• 0.03-0.08 ppm
• 0.15-0.2 ppm
• 0.35-0.45 ppm
– Comparison between analyzer and external
standard
– Audits should include zero check
21
QA (cont.)
• Different types of audits
– By reporting organization (RO) certified by
RO
– By RO certified by other than RO
– By other than RO certified by other than
RO
22
Precision & Accuracy (P&A) Data
• Precision data come from biweekly precision
checks
• Accuracy data come from annual and other
audits
• P&A data validate ambient data
• P&A data must be included in AQS data
submittals
23
Siting Criteria
• Data quality depends on correct siting of all
instrumentation
• Specific instrument siting guidelines
• Following guidelines is vital part of quality
assurance and control
• We’ll learn more about these guidelines in the
next presentation
24
Summary
• Establish DQOs
• Develop QAPP
– Get it approved by EPA
• Follow your QAPP
• Conduct bi-weekly precision checks
– Conduct level 2 checks to follow monitor trends
• Participate in annual audits and others if
possible
• Data quality will be guaranteed
25