Anna Herforth -
June 13, 2013 - LCIRAH

Anna Herforth and Terri Ballard
FAO consultants
Funded by the EU-FAO Improved Global Governance for
Hunger Reduction Programme (2012-2015)

• Aim: to understand how agriculture projects will measure impact on nutrition: which indicators are being used, how, and why
• Why?
– Researchers may be able to connect if desired; informal technical support possible
– Discussions about the evidence base can be informed about the current generation of new evidence
• How:
– Follow up on DFID-funded LCIRAH mapping study
• 50% of studies identified were applicable
• Excluded secondary data analysis, unspecified research, and research with no agricultural intervention

• Key principles:
– Incorporate explicit nutrition objectives and indicators into their design
– Maintain or improve the natural resource base
– Empower women
– Facilitate production diversification, and increase production of nutritious foods
– Expand markets and market access for vulnerable groups
– Incorporate nutrition promotion and education to enhance the impact of production and income
– Collaborate and coordinate with other sectors
See: http://unscn.org/files/Agriculture-Nutrition-CoP/Agriculture-Nutrition_Key_recommendations.pdf

• 77 project PIs contacted
• 68 responded (88%)
– 3 of these excluded
– 2 did not complete
 82% with complete data
Respondents' Affiliations
NGO
28%
University
35%
CGIAR
31%
UN
2%
Foundation
4%

Type of measure % measuring Notes
Food consumption or diet almost all Many measuring HDDS, WDDS, and
IDDS for kids; intake of specific foods of these, specific varieties?
half (of above) Biofortified varieties; natural varieties
Food security most
Knowledge or behaviors most half
HFIAS, HHS, seasonality, coping strategies
Specific to project
Women’s empowerment or labor
Economic outcomes
Nutritional status many many
Indicators somewhat unclear; a couple using/testing WEAI of these, 2/3 disaggregating by gender stunting, underweight, BMI, anemia, indicators of VAS
Link with water, health, or sanitation
Natural resource management many some Few indictors described; e.g. use of soil and water conservation practices

• Majority are measuring in a comparison population
• Majority are collecting qualitative data
• Sample sizes range from 120 to 4000 (one
9000)

• Focus is on nutrition impact among producers
• Total shift from previous generation of research regarding measurement of diet quality
• Newly developed indicators get used
• High number of studies measure nutritional status, but available sample sizes suggest low power
• Seems to be attention to program impact pathways
– Indicators chosen mostly because important to project goals, or evaluates a specific aspect of project
– Interest in support for: “Adapting indicators to fit your particular study activities and aims”

• This sample describes projects that set out to affect nutrition
• It does not describe larger agriculture programs or investments where nutrition is not necessarily the primary goal

F. James Levinson and Anna Herforth
Ag2Nut Community of Practice http://knowledge-gateway.org/ag2nut

Monitoring and evaluation of ag projects seeking to address food security and nutrition
(in addition to their generally primary production objectives) would be highly desirable:
• there is, to date, so little empirical data documenting successes and failures; and
• possible adverse effects need to be identified and addressed rapidly within programs

• Impact on food security on nutrition not necessarily first priority of agricultural managers
• Not familiar with how to measure
• M&E in general is often not very strong
– FAO/World Bank study (2010) of M&E in agriculture projects in Asia, Africa and Latin America

• M&E is often perceived as an externally imposed obligation with findings seldom integrated into management and action systems
• Ag managers complain of unmanageable data collection and reporting demands
• What M&E is carried out gives primary attention to physical achievements to the neglect of project outcomes
• Inadequate institutional capacity consistently limit
M&E
(All of these, of course, are problems common to M&E in development projects more generally.)

• Roles
– identify sensible indicators to measure nutritionrelevant impact based on the type of activities in the program
– carry out the key M&E necessary for tracking progress
– feed back to the program management
(monitoring)
– Understand reasons for impact or lack thereof
(eval)
– could support nutrition-sensitive program design or adjustment

Geographically representative sentinel sites within the overall project area are one approach to M&E in ag projects with Ag2Nut interests
Baseline data would be followed by the collection of quantitative and qualitative data at
6 month intervals from these sites, and from comparable sites in non-project areas.

• Extent to which households have been reached/affected by the project
• Relevant outputs specific to project (e.g. certain crops/foods consumed)
• Household food insecurity levels
• Dietary quality
• Women’s empowerment
• If it makes sense, nutritional status among young children and women
All complemented by qualitative data to better understand the dynamics of project effects

• Employment levels have remained static or deteriorated;
• Small producers have been excluded;
• Household food insecurity has deteriorated (overall or seasonally);
• Intra-household equity of income has declined;
• The labor burden of women has increased;
• Debt burden has increased;
• In irrigation/water use projects, changes in waterborne diseases;
• In livestock projects, changes in zoonotic disease
• Harm to natural resources (particularly water, soil, biodiversity)

Among possible indicators:
• Access, use and satisfaction with services provided under the project
• Changes in farmer assets
• Percentage of households considering themselves better off now than 12 months ago
• Percentage of the labor force underemployed or unemployed

Such sentinel site data collection is likely to be useful if:
• Good quality data can be sensibly aggregated and presented to project management in timely fashion
– Challenge to collect data that are meaningful enough to be useful, and brief enough to be usable
• Information indicating harmful effects or shortcomings in project implementation – will be addressed by project management

Adequate staff and funding:
• There is a need to identify Ag-Nut M&E teams capable of participating actively in an initial stream of nutrition-sensitive agriculture projects.
• External funds probably necessary

Through the creative use of separately managed sentinel site-based M&E, it should be possible to:
• Generate cooperative efforts
• Generate much needed data
• Document successes in nutrition-sensitive agriculture
• Develop prototypes, training modules and TA mechanisms for subsequent use

• Need more examples of successful programs
– Not just nutrition outcomes, but win-wins with other goals
• Ultimately, what is it that we want to scale up?
– Probably not individual programs
– Rather, principles that explain how individual programs have positive impact
– Evaluations need to offer generalizable lessons learned

Ag2Nut Community of Practice http://knowledge-gateway.org/ag2nut