Facilitating Use of
Strategic Information
Outline
• Creating Data Demand and Use
• Analyzing and reporting results according to
the target audience
• Data-utilization strategies
• Tools for facilitating use of M&E results for
policy and program development
Information for Decision Making
• Information must be based on quality data in
order to be useful
• Information must be communicated
effectively in order to be useful
• Information must be used to be good
M&E
Information Needs
• Different for different stakeholders
– view activities from different perspectives
– have different degrees of understanding of the program
– need different information
– want different information
– need/want information at different levels of complexity
– have different intensities of interest
Rationale for Data Demand and Utilization
Central to achieving evidence-based decisions are:
•
Increasing user demand for quality information
•
Increasing availability of data and analysis
methods
•
Increasing use of data and analysis methods
Components of Strategic Information
Identifying
Needs
Designing &
Implementing
Intervention
and Impact
Routine HMIS
Facility Survey
Surveillance
Population
Surveys
Programs
Basic/Academic
Research
Measuring Outcome
Monitoring,
Analyzing and
Triangulating Core
Indicators
Program Evaluation
Operations
Research
Targeted Evaluation
(Including, outcome
evaluations, rapid
ethnographies, economic
evaluations, case studies)
National Level
Response
Coverage Surveys
National Level
Impact/Collective
Effectiveness
Modeling
Surveillance/Impact
M&E
The Big Picture – improved health outcomes
Demand for Information
Systems
Approach
Technical
Approach
Individual
Behavior
Data Demand and Use
Technical
Approach
Organizational
Approach
Individual
Behavior
Political, Economic, Cultural, Social Environment
Use of data, analyses, and other information for
decision making occurs within a political context
Information for Decision Making
What
information?
Decision
makers
Decisions
How?
Systems Approach to DDU
Look at:
• Current practices
• Organizational incentives and culture
• Individual attitudes and motivation
Develop a Data Demand and Use Profile:
• specify decision makers at various levels
• data needed to make decisions
• timing of data requirements
• the format for data presentation
Technical Approach
Technical resources for transforming data into
information
– Data sources
– Tools for data collection
– Methodologies
Analyzing M & E Data
Focus on the main information
the M&E data need to provide?
• Process:
– Was the program carried out as planned?
– How well was it carried out?
• Results:
– Did the expected change occur?
– How much change occurred?
• Impact:
– Is the change attributable to the program?
– Does the change mean program “success”?
Increase availability of data
• Explain indicators
– design
– interpretation
– significance
– meaning of changes in value
• Explain data
– collection processes
– calculation processes
– quality
Explain Changes
• Meaning of changes in indicator
values – what can explain the
change
• What other information should M&E
communicate to help use this indicator to
make program decisions?
Explaining Contradictory
Data and Results
• Results from various data sources – are the
sources providing complimentary
information?
– immunization rates for DPT-3 in children are very
different between official government HMIS figures
versus survey with verification of children’s
immunization cards
• Results at various times – are there
unexpected changes over time?
Clarify Analysis Steps
• Data calculation processes
– Percentage of population reporting any risky sexual
behavior in last six months
– Denominator could be (a) all; (b) all sexually active; (c) all
within certain age groups; (d) all within defined “high risk”
groups
• Which calculation would you recommend? Why? What
are key factors to consider?
• How would you explain the significance and meaning
of the different options to the program manager(s)?
Data Quality
• Data quality
– a child-immunization survey, verifying with
cards, provides data for several Child Survival
indicators
– you find out that the non-response rate on many
key questions is 40 percent
• What are the implications for data quality of this
information?
• How would you explain this to the program manager(s)?
• What options could be recommended?
Triangulation
• “Short-hand term” for data synthesis and
integrated data analysis for decision making
• Originally coined by social scientists: basis for
validity/truth/credibility
• Process in which multiple data sources
(quantitative or qualitative) are brought together
and analyzed to better understand the same public
health phenomenon
Triangulation for HIV Programs Evaluation
Core Consensus Indicators for all Initiatives Based on United Nations General Assembly
Special Session (UNGASS) and Recent Meetings
Input
Process
Output
Programme Monitoring
Outcome
Impact
Household Surveys, e.g. BSS
Facility Surveys
Coverage Surveys
HIV/STI
Surveillance
Patients’ tracking/ARV resistance surveillance
Second-Generation HIV
Surveillance
HMIS in PEPFAR, CDC, MAP, GFATM
Reproductive Health Survey/ CDC,
MICS/ UNICEF, MACRO/ DHS and
AIDS Indicators Surveys,
Service Availability Mapping (SAM),
Health Facility Surveys (HSPA),
Coverage Surveys
Data Use
and Communication
Improve Data Use & Communication
To increase use M&E Data must:
– be manageable and timely
– be presented according to the audience’s
• interest
• capacity to understand, analyze
• time, competing demands on time
– have transparent quality
– focus on activities, results of interest
– focus on meaning and direction for action
Communicating M&E information
Often hard to communicate, because:
• Technical issues are important but boring
• Decision makers are busy and distracted
• Program managers have tunnel vision
• Program implementers are too busy
• Data experts/wizards have few communication skills
• Communication experts/wizards have few data skills
It’s
complex!
Support Services
Information
systems
Resources
Quality
assurance
Pharmacy
Staff
Care
Process
Supervision
Triage
Community
input
History
Diagnostic
services
Counseling Equipment
Client
and
supplies
Examination
Treatment
Training
and skills
maintenance
Diagnosis
Financing
Transport and
communication
Leadership
and management
Infrastructure
Logistics
Effective communication
• Identify your audience and their needs.
• Focus on what your audience needs to know, not what
you know.
• Consider your audience’s technical knowledge,
motivations, and interests. Tailor the points
accordingly.
• Decide what you hope the audience will do as a result
of the presentation.
• Anticipate possible questions.
• Make sure your message source is someone your
audience trusts and respects.
Desired Characteristics of Information
• Needs to be simple
• Needs to be timely
• Needs to be compelling
• Needs to be consistent with stakeholder
values
Tools for Facilitating Use of
M&E Results for Policy and
Program Development
Types of Tools and Resources
• Tools
– printed documents
– computer programs (often on CDs)
– Internet-based applications
• Methods/techniques resources include
– analytical techniques
– case studies and best practices
– data resources
Types of Tools
• Policy Development
• Program Development and Management
• Technical Resources
• Best Practices and Case Studies
• HIV/AIDS Programs
• Data Sources
Simple Tool for Analysis
Policy Development
• The Policy Circle
– The Policy Circle is a framework through which the
dynamic components of policy development and
implementation can be better understood and
analyzed.
– The Policy Circle is used to analyze different levels of
policy, including national and local policies, and
sectoral and operational policies.
Program Development and
Management
• The Health Manager’s Toolkit
The Health Manager's Toolkit is an electronic
compendium of tools designed to assist health
professionals at all levels of an organization to provide
accessible high-quality and sustainable health services.
The tools have been developed by organizations working
throughout the world to improve the delivery of health
services.
http://erc.msh.org
Technical Resources
• REPROLINE: Reproductive Health Online
Reproductive Health Online is an Internet resource for
health-care professionals, providing technical
information and training materials on the full range of
reproductive-health topics, including family planning
(contraceptive methods and other family planning
topics), maternal and neonatal health, cervical cancer,
and STDs.
www.reproline.jhu.edu
Technical Resources
• IGWG: Gender Publications and Tools
The Interagency Gender Working Group (IGWG)
Gender Publications and Tools. The IGWG promotes
gender equity within population, health, and nutrition
programs with the goal of improving reproductive
health/HIV/AIDS outcomes and fostering sustainable
development.
www.igwg.org/pubstools.htm
Best Practices and Case Studies
• Advance Africa: Compendium of Best
Practices
The Compendium of Best Practices is a searchable online database
of best practices in reproductive health and family planning service
delivery. It can be used by program managers who have identified
gaps, needs, and opportunities in their programs. By reviewing the
lessons learned from other practices, program managers can apply
their own practices with greater confidence that they will lead to
improved program performance.
www.advanceafrica.org/compendium
Best Practices and Case Studies
• UNAIDS: Best Practice Collection
The UNAIDS Best Practice Collection – hundreds of
UNAIDS publications documenting successful
approaches to particular HIV/AIDS problems and issues.
www.unaids.org
HIV/AIDS Programs
• Synergy Project: APDIME Toolkit
The Synergy Project APDIME Toolkit is a resource to
support program designers and managers in HIV/AIDS
prevention, care and support programming. Tools for
Assessment, Planning, Design, Implementation
Monitoring, and Evaluation are included.
www.synergyaids.com
HIV/AIDS Programs
• HEARD: AIDS Toolkits
The AIDS Toolkits developed by the Health Economics and
HIV/AIDS Research Division, University of KwaZulu-Natal
encompass a broad range of interventions for HIV/AIDS
including assessment of the impact of AIDS on different
sectors and ministries.
www.nu.ac.za/heard/toolkits
HIV/AIDS Programs
• UNAIDS: M&E Toolkit for National AIDS
Programs
The Guide to Monitoring and Evaluation for National
AIDS Programs was developed by UNAIDS in
collaboration with MEASURE Evaluation and many other
partner organizations. It includes a toolkit providing
guidelines and instruments for the development and use
of indicators for all components of a national program.
www.cpc.unc.edu/measure/publications/unaids-00.17e
Data Sources
• DHS: Statcompiler
The Demographic and Health Surveys STATcompiler
builds customized tables from hundreds of DHS+ surveys
and indicators. The HIV/AIDS Survey Indicators Database
includes data from DHS+, MICS, BSS, and other surveys.
www.measuredhs.com
Data Sources
• U.S. Census Bureau
International Data Base: The International Data
Base (IDB) is a computerized data bank containing
statistical tables of demographic and socioeconomic data for 227 countries and areas of the
world.
www.census.gov/ipc/www/idbnew.html
Data Sources
• U.S. Census Bureau
HIV/AIDS Surveillance Data Base:
The HIV/AIDS Surveillance Data Base is a
compilation of information from those studies
appearing in the medical and scientific literature,
presented at international conferences, and
appearing in the press.
www.census.gov/ipc/www/hivaidsd.html
Data Sources
About DOLPHN
The Data Online for Population, Health and Nutrition
(DOLPHN) system is an online statistical data resource
containing selected current and historical country-level
demographic and health-indicator data. The DOLPHN
system is designed to provide users with quick and easy
access to frequently-used statistics and can be helpful as
both a reference and analytical tool.
www.phnip.com/dolphn
Remember!!
We learn
• 1% through taste
• 2% through touch
• 3% through smell
• 11% through hearing
• 83% through seeing
We remember
• 10% of what we read
• 20% of what we hear
• 30% of what we see
• 50% of what we see and hear