Collecting High-Quality Data
Part of the M&E Plan
Data
Collection
Data Quality
Indicators
Framework
Data Use
and
Reporting
Evaluation
Strategy
M&E
Plan
Budget
What is Data Quality?
Actual
Results
?
Reported
Results
Data Quality:
How well our M&E data “tell the true story.”
Adapted from: http://www.cpc.unc.edu/measure presentation by Win Brown, USAID/South Africa, School of Health
Systems and Public Health, Monitoring and evaluation of HIV/AIDS Programs, Data Quality; March 2, 2011.
Elements of Data Quality
Validity
Reliability
Completeness
Precision
Timeliness
Integrity
Data measure what they are supposed to measure.
Everyone defines, measures, and collects data the same way—all
the time.
Data include all of the values needed to calculate indicators.
No variables are missing.
Data have sufficient detail. Units of measurement are very clear.
Data are up to date. Information is available on time.
Data are true. The values are safe from deliberate bias and have
not been changed for political or personal reasons.
Adapted from: http://www.cpc.unc.edu/measure presentation by Win Brown, USAID/South Africa, School of Health Systems and Public Health,
Monitoring and evaluation of HIV/AIDS Programs, Data Quality; March 2, 2011.
Validity and Reliability: Hitting the Target
NOT Valid
NOT Reliable
X X X
XX
X
X
X
X
X
Reliable but
NOT valid
XXX
XXXX
XXX
Reliable AND
Valid!!!
XXX
XXXX
XXX
Adapted from: http://www.cpc.unc.edu/measure presentation by Win Brown, USAID/South Africa, School of Health
Systems and Public Health, Monitoring and evaluation of HIV/AIDS Programs, Data Quality; March 2, 2011.
Precision
Which indicator description will yield the most precise result?
Indicator
Description
Treatment success rate
Cure PLUS completed treatment
Indicator
Description
Treatment success rate
All patients in the cohort:
- with smear conversion and - who completed full course
negative smear at 5 months of treatment but do not
PLUS
meet cure definition
DIVIDED BY:
Total number of smear-positive patients in the treatment
cohort
MULTIPLIED BY: 100
Completeness
NGO
partner
Number of members
participating in social
mobilization
Comment
Not available
Participant log not maintained
Friends of TB
TB Matters
10
TB Helpline
Not available
Stop TB NOW!
Unclear how to determine
who actively participated
12
TOTAL ???
Often related to:
• ease of collecting and reporting data
• data sources
• training
Timeliness
1. Are we meeting internal and external deadlines?
• Communicate expectations clearly.
• Offer support to collect/analyze where needed
(budget?).
2. Are we analyzing results often enough to be useful
for program management?
• The sooner we know about a problem, the sooner
we can fix it!
Integrity
• Often difficult and sensitive topic.
• Routine verification from the start can help
avoid bias of any kind.
• A partner submits perfect reports every month on
time and meets or exceeds targets.
• A partner submits reports with a few errors every
month, sometimes 1-2 days late; usually meets or
comes close to targets.
 Which data would you verify and why?
Data Quality Plans
How can we
ensure:
Validity
Reliability
Completeness
Precision
Timeliness
Integrity
Strategy
Resources
Helpful Resources
MEASURE Evaluation Project
Data Quality
http://www.cpc.unc.edu/measure/tools/mo
nitoring-evaluation-systems/data-qualityassurance-tools
Data Use
http://www.cpc.unc.edu/measure/ourwork/data-demand-and-use