School of Health Systems and Public Health
Monitoring & Evaluation of HIV and AIDS Programs
Data Quality
Wednesday March 2, 2011
Win Brown USAID/South Africa
Slide 1 of 18
Objectives of the Session
• To Review and Discuss:
– A Data Quality approach to M&E
– Six important elements of data quality
– Practical applications
Slide 2 of 18
Why Data Quality?
•
Program is “evidence-based”
•
Data quality  Data use
•
Accountability
Slide 3 of 18
Data Quality
Real World
Data Management System
In the real world, activities are
implemented in the field. These
activities are designed to produce
results that are quantifiable.
An information system represents
these activities by collecting the results
that were produced and mapping them
to a recording system.
Data Quality: How well the DMS represents the real world
Real
World
?
Data
Management
System
Slide 4 of 18
? = Dimensions of Data Quality
Validity
Reliability
Valid data are considered accurate: They measure
what they are intended to measure.
The data are measured and collected consistently; definitions
and methodologies are the same over time.
Completely inclusive: the DMS represents the complete list of
Completeness eligible names and not a fraction of the list.
Precision
The data have sufficient detail; in this case the “accuracy” of
the data refers to the fineness of measurement units.
Timeliness
Data are up-to-date (current), and information is available on
time; the DMS produces reports under deadline.
Integrity
The data are protected from deliberate bias or manipulation for
political or personal reasons.
Slide 5 of 18
Good Data are Valid and Reliable
≠ Valid
≠ Reliable
X
X X
XX
X
X
X
X
X
≠ Valid
 Reliable
XXX
XXXX
XXX
 Valid
 Reliable
XXX
XXXX
XXX
Slide 6 of 18
What are:
– Valid data?
– Reliable data?
– Complete data?
– Precise data?
– Timely data?
– Data with integrity?
Slide 7 of 18
Framework for Enhancing
Data Quality
Data Quality System
Data Management
Processes /
Procedures
Data Quality
Processes /
Procedures
Source
Validity
Collection
Reliability
Collation
Completeness
Analysis
Reporting
Use
Precision
Timeliness
Integrity
Auditable
System
Document!
Paper Trail
that allows
verification of
the entire
DMS and the
data
produced
within it
Risk Verification
Data Management System
Slide 8 of 18
The South Africa Approach
•
Data Quality Assessment
•
Training
•
Data Warehouse
•
SASI Manual
•
Standard M&E plan  DQ Plan
Slide 9 of 18
Data Quality Assessment
•
PMTCT Data; District focus
•
Trace and Verify
•
Routine Data Quality
Assessment Tool (RDQA)
Slide 10 of 18
M&E Training?
M
E
•Routine data collection
•Internal validity
•Data quality
•Operations research
•Results reporting
•Instrument design
•Strategic planning
•Survey sampling
•Data analysis for data use
•Local training partners
•Participant follow-up
•User’s groups/networks
Slide 11 of 18
PEPFAR Reporting Issues
• Are PEPFAR’s results valid & reliable?
• How do you know?
• Are your patient numbers valid & reliable?
• How do you know?
Slide 12 of 18
Data and Statistics are Empowering
100
# random
samples
drawn
50
1
0%
10%
20%
30%
40%
50%
Percent reporting: “I understand statistics.”
Slide 13 of 18
Data Warehouse
•
Online results reporting system
•
Standardized data capture
•
Control of data quality
•
Customized reporting tool
•
Online indicator guidance
Slide 14 of 18
South Africa Strategic Information
Manual (SASI Manual)
•
Operational manual
•
Standard definitions for
PARTNERS
•
Addresses common data quality
problems
Slide 15 of 18
Try Making a Data Quality Plan
•
Component of the M&E plan
•
Strategically think about data
quality
Slide 16 of 18
Measurement
With monitoring of progress in a clinic or in a
community, always try to hit the bull’s eye.
Paper Trail
Always document progress.
Data Use
Who is using the data?
Slide 17 of 18
Thank you
Slide 18 of 18