Committee on World Food Security
Monitoring Food Security
Comments on Session 4:
Measuring Hunger: what is required for policy-making ?
Paulo Jannuzzi
Secretariat of Evaluation and Information Management of
Ministry of Social Development and the Fight against Hunger
(SAGI/MDS)
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
My comments aims also to contribute to the
questions raised in the first panel
• Developing a suite of indicators
– Dimensions to be monitored
– Which indicators
– How to select them
• Platform for commom dissemination
– An Interactive Tool to present and analyze the indicators
– Suite vs. Composite Index
• International standards on food measurement
– International cooperation on designing Surveys and Registers,
organizing existing data on monitoring panels and using them as tools
for policy and program improvements.
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
• Evidence based Policy-makers and program managers need a
suite of indicators that reflect key dimensions of a normative or
logical framework of Food Policy on food
production/distribution, food access/consumption and food
diversity, quantitative and qualitative intake, to allow the
improvement of public initiatives - as programs and legislationand the achievement of goals on one or more of the policy
objectives as alleviate of hunger, guarantee adequate
nourishment and promote food and nutrition security.
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
• Evidence based Policy-makers and program managers do not
need an exhaustive suite of indicators, but mainly relevant,
valid, consistent and easy-to-understand indicators , organized
by the operational aspect, geographical and decision-level they
are involved, in a timely and costly basis, build by means of
information available on different sources– national census and
household surveys, administrative registers, data produced by
programs, contracted specific surveys etc- that can answer the
questions they have over the Policy-Program Cycle (indicators
for Diagnosing the problem, Planning the programs, Monitoring
the public actions, Evaluating the overall effort).
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
A Possible Logical Framework on Food Policy-Making
Policy instruments: programs and legislation
Policy objectives/impacts
Food Production
Food Avaliability
Alleviate Hunger
Food Intern. Trade
Food equipaments
Family income
Food Access
Food consumption
Garantee adequate
nourishment
Food and Nutrition
Security
External factors not or not totally controled by Food Policy-Makers
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
Key indicators chain based on Logical Framework on Food Policy-Making
Policy, Program and Context indicators
Environment ind
Crop production ind
Impacts indicators
Commodities Stos
ind
Food stocks ind
Extreme poverty ind
Ethanol internal
demand ind
China´s food
demand ind
Govern Cash
Tranfers ind
Food prices ind
Anthropometric ind
Food sales ind
Nutrition surveys ind
Food purchases
Unemployment ind
SAGI - Secretaria de Avaliação e Gestão da Informação
The impact measure
depends on the nature
of public intervention
Brazil´s experience on Indicators for Food Related Policy-Making
Institutional framework
– National Policy on Food and Nutrition
Security Law
– Food Security National Information System
– National Council composed by civil society
and governmental institutions
representatives, supported by Presidency
Technical aspects
– 7 dimensions: Production, Avaliability,
Income, Access, Health, Education, Public
Policy Efforts
– More than 50 indicators
– National, state and local indicators
– More than 20 sources of information
(Statistical Agency, Public Registers, Program
data etc)
www4.planalto.gov.br/consea/publicacoes
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
Measuring Hunger: what is required for policy-making ?
Evidence based Policy-Makers and Program Managers do not need a
exhaustive set of indicators, but set of indicators on context, input,
resource, process, intermediate, output and impact dimensions, to follow
the Public Budget and resources being transformed into social well-being.
Budget and Resource
indicators refleting
the public effort on
food programs
Process and Output
indicators refleting
the products and
deliveries of food
programs
Effect and Impact
indicators
Context indicators that may affect Food programs achievements
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
Evidence based Policy-Makers and Program Managers do not need a
exhaustive set of indicators, but a set of indicators organized in panels
of level of relevance for decision making and explicative power.
Key indicators
for action
2nd level Indicators for
first understanding
3rd level Indic
for explication the
whole picture
Key indicators,
sensitive and timely
avaliable
Could be the
FAO/CFS´s
priority
indicators
A broader set of indicators, to
help to understand the
evolution of the key ones
An even broader set of indicators to
contextualize, diagnose and analyze the
previous, computed in more detailed geography
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
Evidence based Policy-Makers and Program Managers need easy-tounderstand indicators, that can be very complex to build but catches/points
what they are suppose to mean, and which limitations are clearly known.
Suite of good
Indicators
Reality of Food
Related
Problems
SAGI - Secretaria de Avaliação e Gestão da Informação
Composite
Indicators
Measuring Hunger: what is required for policy-making ?
Evidence based Policy-Makers and Program Managers need a different set of
indicators- with specific properties- depending on their step on Policy cycle
Public agenda
A broad set of
indicators to
quantify, qualify
and localize the
food problem
Food
related
problems
Formulation
Ad hoc Indicators to
help program design
Sistema
indicators
de
M&A
needs
Decision making
Multicriteria or composite
indicators to help choosing
alternatives
Evaluation
Specific
measures built
by surveys
designed to
evaluate the
program impact
and its
improvements
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
Implementation
Timely indicators to
monitor implementation
process
Measuring Hunger: what is required for policy-making ?
Evidence based Policy-Makers and Program Managers may need indicators
from different sources, because of costs and timeliness, but also because
they need to get a better comprehension of the food issues phenomena.
Measure Triangulation should not be a problem, if they know the potential
and limitations of each indicator.
FAO undernourishment
indicator
Extreme poverty and
consumption ind
Hunger
Household Food Security Scale ind
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
• Evidence based Policy-makers and program managers do not
blame for too many indicators or a single composite one, but
ask for :
– a normative or logical framework to understand the whole picture
– indicators of some key dimensions of this framework
– detailed information for diagnosing population vulnerability and for
policy/program formulation, which takes time, large budget and less frequent
data collection
– monitoring indicators, timely, based on low-costly official records, quick data
collecting strategies or simplified surveys
– evaluation indicators, carried on a reasonable period of time, including specific
population and geographical areas, balancing internal/external validity and costs
– cost-efectiveness measures, as food programs get maturity, to support budget
allocation discussion
– analytical tools to investigate relations not just to check goals
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
• Some of the Brazilian institutions, research centers and agencies that
produce data, information and studies on Food Policy Agenda
Institution
Food Related Themes/Surveys
IBGE –
Brazilian
Statistical
Agency
•Crop and Cattle annual surveys (Food production)
•Food products quaterly surveys (Food avaliability)
•Crops Prospective estimates (Food production in near future)
•Price Consumer Index (Food costs, monthly basis for 11 major areas)
•Price Producer Index (Commodities costs, planned for next years)
•Market and sales annual and monthly surveys (Food sales)
•Agriculture-Cattle-Farm Census (Food production, area used, 10/10-year)
•Decennial Demographic Census (Water,Sewage, Poverty/Income in very small areas )
•HHold surveys (Water, Sewage/Pov/Income, state-lev, on 3-month basis (2011 on)
•Expenditure and Income Survey (Food acquisition and intake, poverty subjective
indicators, anthropometric and food security ind, 5/5-year, state, urban, rural)
•Students Health National Survey (Anthopometric indicators, 2009 )
www.ibge.gov.br
Information for
mainly Diagnosis
and Monitoring,
feed Logical
Framework at
macro-level,
evaluate overall
Policy (not
programs)
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
• Some of the Brazilian institutions, research centers and agencies that
produce data, information and studies on Food Policy Agenda
Institution
Food Related Themes/Surveys
Ministry of
Social
Delopment
and the Fight
against
Hunger
•Bolsa Familia Longitudinal Impact Evaluation Study (Food acquisition, access to health
services, subjective poverty, sewage, anthropometric indicators, national, 3-macroareas, 2005, 2009 and planned 2013)
•National HH Supplement questionnaire on Food Security done with IBGE ( Food
Security Scale, state, urban, rural areas, 2004 and 2009)
•Nutrition Surveys (Anthropometric indicators, live conditions, access to programs)
www.mds.gov.br
Information for
mainly evaluation
of deliveries and
impact of specific
MDS programs
– Semiárido (deprived region in Northern ) : 2005 , planned 2012
– Quilombola (originated by refugee slaves in last centuries): 2006/07, current 2011
– Urban Amazonas State: 2006/07
– Ribeirinhos (familes living over Amazonia rivers): planned 2012/13
• Milk Acquisition/Distribution Program Evaluation ( Food production, beneficiares´s
profile and satisfaction)
•Food Equipament Evaluations (Food quantity and diversity offering, user´s profile and
satisfaction)
SAGI - Secretaria de Avaliação e Gestão da Informação
Evaluation studies on Social Programs and Food Security Agenda
MDS Experience on Evaluation Studies
– Since 2004 till september 2011, 88
evaluation studies done
– Mainly done by external contracted
studies, as IBGE, universities, research
centers, NGOs, consultancy firms
– Tight supervision on sample planning,
questionnaire design, field operations,
final report, microdata provision
– Mix-methods approach: quantitative,
qualitative, quasi-experimental
techniques
– Developing Tools for Information
Gathering and Program Monitoring
www.mds.gov.br/gestaodainformacao/disseminacao/2010/no-13-sintese-das-pesquisas-de-avaliacao-de-programas-sociais-do-mds-versao-atualizadae-revisada-2006-2010/no-13-sintese-das-pesquisas-de-avaliacao-de-programas-sociais-do-mds-versao-atualizada-e-revisada-2006-2010
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
Measuring Hunger: what is required for policy-making ?
• Some of the Brazilian institutions, research centers and agencies that
produce data, information and studies on Food Policy Agenda
Institution
Ministry of
Health
www.saude.gov.br
www.datasus.gov.
br
Information for
mainly monitoring
health programs
Food Related Themes/Surveys/Registers
• National Demographic and Health Surveys ( Anthropometric ind, Women and
Children Health Status, HH Food Security Scale, national, 5 macro-areas, 1996/2006)
•National Health Survey, partnership with IBGE (Health indicators, feeding habbits,
anthropometry, morbidity indicators, state level, each 5 years)
•Health National Registers with information organized in a user-friendly web plataform
to gather micro-local-data
– Basic attendance made by Household-attedance-teams (Children malnutrition ind, locallevel , monthly basis )
–Hospitalization attendance (Malnutrition-caused hospitalization, local-level , monthly basis
)
–Vital Statistics Registers (7-day, 28-day, 1-year Child mortality measures, local level, yearly
basis)
SAGI - Secretaria de Avaliação e Gestão da Informação
Measuring Hunger: what is required for policy-making ?
• Figures on Food Security in Brazil: data from Food Security Scale Supplement
www.ibge.gov.br/home/estatistica/populacao/seguranca_alimentar_2004_2009/default.shtm
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
Measuring Hunger: what is required for policy-making ?
on Food Security in Brazil: data from Consumer Expenditure Survey
www.ibge.gov.br/home/estatistica/populacao/condicaodevida/pof/2008_2009_encaa/default.shtm
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
Measuring Hunger: what is required for policy-making ?
n Food Security in Brazil: data from National Students Heath Survey
www.ibge.gov.br/home/estatistica/populacao/pense_avaliacao_nutricional_2009/pense_avaliacao_nutricional_2009.pdf
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
Measuring Hunger: what is required for policy-making ?
• Figures on Food Security in Brazil: data from HHold Socioeconomic Survey
www.ipea.gov.br
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management
Thank you !!
www.mds.gov.br
paulo.jannuzzi@mds.gov.br
SAGI–-Secretariat
Secretaria de
Avaliação eand
Gestão
da Informação
SAGI
of Evaluation
Information
Management