An End User’s Perspective
Aleta Kennard & Janice Lam Snyder
PQAO Training, Sacramento/Diamond Bar
May 28-29, 2014
June 11-12, 2014
Key Questions:
Why is it so important?
What if I don’t document?
What do we mean by document?
What are the requirements of documentation?
How can I fulfill the requirements?
Resources
The Process
Data
Regulatory
Collection
Requirement
&
DQOs
Quality
Control
Quality
Assurance
Data Analysis &
Studies
Decisions/
Regulatory
Action
Improve
Public Health
Data Assessment
The data assessment process includes both internal and external quality
control assessments of accuracy, precision, data completeness, and
criteria that are specified in associated QMP, QAPPs, SOPs, and R&R
Documents.
 Internal Assessments –independent multi-level data review
conducted by the monitoring organization
 External Assessments – activities such as data, lab, performance,
and technical system audits conducted by ARB, EPA or other
PQAO.
 The data assessment process will be covered in more detail in Session 2
of the PQAO Training (Fall 2014)
Data Assessment
ARB is committed to effective review, verification,
validation, reporting, certification and assessment of all
data generated by, on behalf of (such as contractors), or
utilized within its PQAO (including secondary data)
ARB and air monitoring organizations within its PQAO
define and formalize these activities in a Roles and
Responsibilities (R&R) Document
-Patrick Rainey, Manager, Quality Management Section
Why is it so Important?
 Documentation is “proof” or “evidence”
 That all required procedures were carried out and
carried out at the correct time
 What occurred during that time
 To ensure equipment is working properly
 Helps reconstruct the study (Legal Defensibility)
 Data is used for Regulatory Purposes
 Certify data is correct and accurate
 Creates confidence in the data
Who are the End-Users?
Quality
Assurance
Personnel
Auditors
Site
Operators
Data
Public
Decision
Makers
Scientists
Analyst
What happens if I don’t document?
 “If it wasn’t documented, then it didn’t happen”
 Data may be lost
 Compromises integrity and accuracy (counts when it really





shouldn’t)
A lot of wasted time for the end user
Incomplete documentation may lead to invalidating an
entire study
Unable to replicate (document so they know what you did)
Regulatory Action
Difficulty in justifying that procedures were followed
correctly and completed when it was supposed to
WHAT IS DOCUMENTATION?
 A description of what activities occurred at what time
 Who? What? Where? How?
 Activities that might influence data
 Procedures (Manuals, SOPs, & Safety Information)
 Results of measurements, QC checks, observations,
troubleshooting, and activities associated with the activity
and instrument
 Certifications (Calibrators and Gasses)
 Examples: Laboratory Notebook, Logbook, Forms, paper
printouts, electronic records, databases, and photos
Benefits of Documentation
 Does not rely on memory
 Improves communication between operators and end-
users
 Consistency
 Helps identify issues early or prevents bigger issues
 Proactive vs. Reactive




Piece of the Puzzle
Aids in Exceptional Event Requests, and Planning
Helps Auditors
Helps with Future Troubleshooting (instrument logs)
Regulatory Requirements
 40 CFR 58 Appendix A: Quality Assurance
requirements
 Quality Systems Requirements
 Measurement Quality Check Requirements
 Table A-2 of Appendix A to Part 58-Minimum Data
Assessment Requirements for SLAMS Sites
 EPA QA Handbook Vol II, Appendix D
 Will be discussed in the key documents group
Requirements of Documentation
Sign and
Date
Promptness
Legible
Identifiable
Accuracy
Fullfill Requirements by:
 Make a single line through the error
 Record the correct entry above or next to the original
entry with the reason for correction
 Sign/Initial and Date
198
200
198
200
Error in writing
Mistake
JLS 03-14-14
March-14
Type of Documentation
Automated
Manual
Check sheets
Auto Calibrations
Logbooks
Data loggers
Chain of Custody Forms
Instrument Diagnostics
Calibrations
Troubleshooting
Corrective Action Notification
(CANS)
On Site Documentation
 Station Logbook-required
 Instrument Log Books w/ monthly/bi-weekly checks
 Residence times posted
 Certifications
 SOPs
 Manuals
 Safety Information
Document Troubleshooting
 Document
 What happened, or what you suspected happened
 When? For how long?
 How did you find out?
 What you did to resolve the problem? Did you change
out parts? Talk to the manufacturer?
 Saves time in the long run
 Resource for you and others
 Helps in budgets – maybe its time to buy a new one!
Paper vs. Electronic Logs
Paper Logs
Electronic Logs
Uses a paper notebook
Uses a Computer
Conventional (“Tried and True”)
Quick Remote Access
Requires storage space
Requires extra security
measures/backup version control (Files
could be corrupted)
More convenient for quick notes
Need to fall back on paper logs (if no
access to electronic logs)
End user access after the month
Share information with end-users faster
& searchable database
“More Green”
Analysis on information
Summary-Recap
 Documentation is important because
 Its required
 Saves time
 Creates confidence
 Resource
 Improves communication
 Good documentation contains the following elements:
 Legible, Prompt, Sign/initial and date, Identifiable, and
Accurate
Resources
 CARB PQAO Repository
http://www.arb.ca.gov/aaqm/qa/pqao/repository/qm_docs.htm
 EPA Quality Assurance Handbook for Air Pollution Measurement Systems,
Volume II
http://www.epa.gov/ttn/amtic/files/ambient/pm25/qa/QA-Handbook-Vol-II.pdf
 CARB’s Quality Assurance website: http://www.arb.ca.gov/aaqm/qa/qa.htm
 Documents and Records, Section 5.