Monitoring and Evaluation:
Child Health Programs
Session Objectives
By the end of this session, participants will be able to:
• Identify interventions and approaches to improve child
health
• Describe M&E challenges of specific child health program
areas
• Design an M&E framework for an identified intervention
• Choose core output and outcome indicators for a specified
intervention & recognize their strengths & limitations
Current Situation
• Each year nearly 11 million children die before the age of
five, 30,000 every day,largely from preventable causes.
• 33% of all child deaths occur in the first month of life.
• 41% of these deaths occur in Africa, which has only 10 %
of the world’s under-five population.
• 50% of these deaths occur in only six countries; 90% of
these deaths occur in 42 of 192 countries.
Global Targets
• Millennium Development Goal (2002):
• To ensure a two-thirds reduction in under-five mortality
by 2015 from the base year 1990
• Related targets
– Target 2: Halve between 1990 and 2015, the proportion of people who
suffer from hunger
– Target 6: Reduce by three-quarters, between 1990 and 2015, the
maternal mortality ratio
– Target 7: Have halted by 2015 and begun to reverse the spread of
HIV/AIDS
– Target 8: Have halted by 2015 and begun to reverse the incidence of
malaria and other major diseases
– Target 10: Halve, by 2015, the proportion of people without sustainable
access to safe drinking water and basic sanitation
Child Health Programs
Immunization
Control of diarrheal diseases & acute
respiratory infections (ARI)
Prevention of HIV in infants and young
children
Roll Back Malaria (RBM)
Integrated Management of Childhood Illness
(IMCI)
Newborn health
Nutrition
Newborn Intervention Packages
•
•
•
•
•
Pre-pregnancy health
Care during pregnancy
Care during delivery
Postpartum care of the mother
Newborn care
– Essential newborn care
– Extra newborn care
– Emergency newborn care
IMCI Strategy: Components
Improving Health System
Improving family
and community
practices
Improving health
workers skills
Key Family & Community Practices I
Growth Promotion
and Development
Disease Prevention
• Exclusive
breastfeeding
• Appropriate
complementary
feeding
• Micronutrient intake
• Providing stimulating
environment
• Immunization
• Handwashing and
hygiene
• Insecticide-treated nets
• HIV prevention and care
Key Family & Community Practices II
Appropriate Home
Care
• Increased fluids &
continued feeding
• Appropriate home
treatment
• Prevention/management
of injuries/accidents
• Prevention of child
abuse & neglect
• Male involvement in care
provision
Care Seeking
• Recognition of dangers
signs
• Appropriate providers
• Treatment compliance
• Accepting referral
• Follow-up
• Adequate antenatal and
delivery care
M&E Challenges
M&E Challenges: Immunization
• Age groups differ among data sources
Routine data
• Estimation of denominators
• Accuracy, completeness & timeliness of reporting
Survey data
• Card availability & reliability of recall
• Sampling errors & sub-national estimates
• Survey timing in relation to campaigns
• Use of data for timely program action
Class Activity: Funny Numbers for
Immunization Coverage
• Example 1: For 6 months in a row, one health center
recorded coverage over 100 percent.
• Example 2: Last year, 51 out of 100 districts in the country
reported higher figures for DTP3 than DTP1 coverage.
• Example 3: As of May, most health facilities in the country
were reporting 30% to 40% measles coverage, which is
approximately what one would expect. One health center,
however, reported 100% measles coverage.
M&E Challenges: Diarrhea, ARI,
Fever
• Seasonality of disease
• Selection bias in health facility data
• Difficulty of standardizing & comparing across
different malaria transmission settings
• Changing definitions (ARI)
• Nonstandard indicators & survey questions on
water and sanitation
• Validation
Class Activity: Management of
Childhood Diarrhea
Time
Indicator/definitions
Before mid 1980s
ORS Use
Mid 1980s
ORT Use: treatment with
ORS or sugar/salt solution
Mid to late 1980s
% of children < 5
years with diarrhea in
past 2 weeks
Ghana 1988
34.8%
ORT use: treatment with
ORS or recommended home
fluids
Ghana 1993
37.1%
1990s
Increased fluid intake
Ghana 1998
56.1%
Currently
Increased fluids and
continued feeding during
illness
Ghana 2003
Ghana 2008*
10.5%
11.0%
M&E Challenges: PMTCT (I)
• Wide range of interventions
– Degree of integration
•
•
•
•
Non-standard output index
Replacement feeding M&E is complex
Impact indicators difficult to obtain
Follow-up of mother-infant pairs
M&E Challenges: PMTCT (II)
•
•
•
•
Estimating the population at risk
Political and ideological factors
Rapid scale-up of interventions
Ethical issues
M&E Challenges: Newborn
Interventions (I)
• Definitions of births, deaths, and newborn period
• Data quality
–
–
–
–
Omission of still birth & early neonatal death
Heaping on age at death
Sampling errors on survey estimates
Sensitivity of perinatal (late fetal plus early neonatal)
mortality rate to changes in data quality
M&E Challenges: Newborn
Interventions (II)
• Measurement of neonatal morbidity
• Lack of standardized output indicators
– Facility-level and outreach monitoring
• Outcomes needed for both mother and baby
• Attribution is difficult due to packaged services
Class Activity: Neonatal Health
In Area A, there are 5000 live births, 100
neonatal deaths, and 20 of the deaths are
due to neonatal asphyxia. In Area B,
there also 100 neonatal deaths, and five
deaths are due to neonatal asphyxia. In
Area B, there are 2000 live births. What
conclusions can you draw from these
data?
M&E Challenges: Child Nutrition
• Complexity of child nutrition programs
• Accurate measurement of child’s age
• Need for large sample sizes to detect change in
breastfeeding
• More complex survey questions
– Interviewer or respondent error
M&E Challenges: Facility IMCI
• Complexity of indicator measurement
• Quality of health facility data
– Observation bias
– Courtesy bias
– Selection bias
M&E Challenges: Disease
Surveillance
• Use of non-standard case definitions
• Accuracy, completeness and timeliness of
reporting
• Representation
• Errors in descriptive information about reported
case
• Lack of laboratory support for outbreak
confirmation or patient management
• Infrastructure and communication constraints
Class Activity: Disease Surveillance
in Mirriah District
• What might account for the increase in the number
of new cases observed during the two most recent
years?
• How might you explain the discrepancy between
the hospital cases and reported cases?
• What is the effect of including the children without
fever status recorded on the chart in the case
definition?
Adapted from CDC Case Study No. 891-903
M&E Challenges: Mortality
•
•
•
•
Reliability of various data sources
Measuring short-term changes
Data quality
Inconsistent definitions of newborn, infant and
child death
• Misclassification of cause of death
• Demonstrating mortality impact
Class Activity: Mortality
During the year 1997, data from birth registers showed that 8776 live births were
recorded in the municipality of Mamfi, out of which almost 7744 were children of
mothers residing in the area. One hundred and thirty-nine (139) fetal deaths were
recorded also from mothers living in the area. Two hundred and seven (207) deaths
of babies under one year of age were registered to residents in the area of Mamfi,
with the age distribution shown on your handout.
(1) Calculate the infant mortality rate (IMR)
(2) Calculate the early neonatal mortality rate (ENMR)
An investigation conducted in hospitals and maternity wards revealed that
approximately 65% of births (alive and dead) took place in hospitals. Every one of
such events was scrutinized, consulting the mothers’ medical charts and notes
taken by the physician and nurses inside the labor room. Findings showed that
thirty live births have been mistakenly declared as fetal deaths and were recorded
as such. Twenty-two fetal deaths were erroneously declared as live births.
(3) Calculate the accurate IMR & ENMR
(4) How were the rates affected by errors in the definition of live births and fetal
deaths?
(5) To what extent are the corrected rates representative of the municipality of
Mamfi?
ILLUSTRATIVE EXAMPLE
IMCI
Logic Model for Improved Health
Worker Skills
FUNCTIONAL
OUTPUTS
•No. of health workers
trained in IMCI
PROCESS
Train first-level
health workers
& district
supervisors in
IMCI
•No. of district
supervisors trained in
IMCI
OUTCOMES
SERVICE OUTPUTS
•Improved HW skills
•Improved HW
competence
•Reduced missed
opportunities for vacc.
& treatment of
childhood illness
•Increased client
satisfaction
•Improved caretaker
knowledge & practices
•Early case
management
•Appropriate care
seeking
•Compliance with
treatment
IMPACT
•Reduced
infant & child
mortality
•Improved
infant & child
health &
nutrition
WHO Priority Indicators of Health
Worker Skills (I)
• Assessment
– Child checked for three danger signs
– Child checked for the presence of cough, diarrhea, and
fever
– Child’s weight checked against a growth chart
– Child’s vaccination status checked
– Index of integrated assessment
– Child under two years of age assessed for feeding
practices
WHO Priority Indicators of Health
Worker Skills (II)
• Correct treatment and counseling
– Child needing oral antibiotic and/or antimalarial is prescribed the
drug(s) correctly
– Child not needing antibiotic leaves the facility without antibiotic
– Caretaker of sick child is advised to give extra fluids and
continued feeding
– Child needing vaccinations leaves facility with all needed
vaccinations
– Caretaker of child who is prescribed ORS and/or oral antibiotic
and/or oral antimalarial know how to give the treatment
• Correct management of severely ill children
– Child needing referral is referred
Sources of Data: M&E of IMCI
Training (I)
• Health facility surveys (e.g. Service Provision
Assessment
–
–
–
–
Inventory of resources and support services
Provider interview
Observation of services provided
Client exit interview
Sources of Data: M&E of IMCI
Training (II)
•
•
•
•
•
•
Supervisory checklists
Review of national, district, and facility records
Demographic surveillance
Mortality survey
Vital registration (if 90+% complete)
Qualitative studies
Case Study
Despite substantial efforts to improve infant and child
health and nutrition in Lakkha District in the past decade,
about one out of five babies born in a given year die before
they reach their fifth birthday, many during the first year of
life. In 2002, the Government recognized that improving
the quality of care for sick children at the health facility
alone would have a limited impact on reducing child
mortality. Lakkha District was chosen, therefore, as a pilot
site for the development and implementation of a
household and community-based approach to promote key
household practices for child survival, growth and
development . Please refer to the handout for further
instructions.
References
•
•
•
•
•
•
•
Arimond, Mary and Marie T. Ruel. 2003. Generating indicators of Appropriate Feeding of
Children 6 through 23 Months from the KPC 2000+. Washington, D.C.: FANTA.
Bos, E. and A. Batson. 2000. Using Immunization Coverage Rates for Monitoring Health
Sector Performance: Measurement and Interpretation Issues. HNP Discussion Paper.
Washington, D.C.: World Bank.
Gage, Anastasia J., Disha Ali, and Chiho Suzuki. (Forthcoming). A Guide for Monitoring
and Evaluating Child Health Programs. MEASURE Evaluation. Chapel Hill, NC: Carolina
Population Center, University of North Carolina.
United Nations. 1983. Manual X: Indirect Techniques for Demographic Estimation. New
York: United Nations.
WHO. 2001. Indicators for IMCI at First-level Facilities and Households. Geneva: WHO.
WHO. 2005. National Guide for Monitoring and Evaluating Programmes for the
Prevention of HIV in Infants and Young Children. Geneva: WHO.
WHO and UNICEF. 2005. World Malaria Report 2005. Geneva: WHO.