CARE Uganda
Program Impact
Evaluation Process
Module 1 - Overview
Companion to:
Module 2 - M&E Tool Box
By:
Tom Barton, CRC
September 1998
Contents
Introduction ........................................................................................................................... 2
CARE Uganda and program impact evaluation ..................................................................... 2
CARE International and HLS................................................................................................. 7
Household well-being and the CARE Uganda HLS Model .................................................... 9
The CARE Uganda HLS Model ........................................................................................... 11
Indicators: measuring and evaluating HLS and well-being .................................................. 13
Utility of the HLS concepts .................................................................................................. 29
References ......................................................................................................................... 32
Acknowledgements
This document is a result of team work, including, but not limited to the following:
The CARE Uganda M&E Task Force – for creative ideas, patient reading and deep
thinking
Abeja,
Fred,
James,
Philemon,
Adam,
Geoffrey,
Lorna,
Philip,
Barbara,
Gervase,
Moses,
Polly,
Daniel,
Imelda,
Nick,
and
David,
Jackson,
Patience,
Sandy
-
-
Nick Ritchie and Geoffrey Chege – a special thank you for creating the opportunity and
their own compelling discussions and stimulating leadership of the team
-
Tim Frankenberger (in absentia) for all the hard work on the CARE International
concept of household livelihood security
Abbreviations:
ARI
HH
HLS
LRSP
M&E
NGO
PIE
PRA
PLA
RLSA
..... ................................................... Acute Respiratory Infection
..... ................................................... Household
..... ................................................... Household livelihood security
..... ................................................... Long range strategic plan
..... ................................................... Monitoring and evaluation
..... ................................................... Non-governmental organization
..... ................................................... Program impact evaluation
..... ................................................... Participatory Rapid/Rural Appraisal
..... ................................................... Participatory Learning Approach
..... ................................................... Rapid Livelihood Security Assessment
Program Impact Evaluation Module 1: Overview
CARE Uganda
1
Program Impact Evaluation
Introduction
There has been growing interest within CARE Uganda about evaluating the impact
of its development program on poverty and livelihood security in households and
communities, and how to do so in a way that recognizes key constraints:

A relatively small amount of resources available for evaluation (feasibility)

A lack of indicators and tools relevant to ‘program’ impact versus project
impact (technical)

No clear system for program impact evaluation versus the set cycle of
baseline, end-of-project and ex post evaluations for projects (management)
This first module outlines the process of grappling with these issues internally in
CARE Uganda, the links between CARE Uganda’s concerns and related efforts by
CARE International, and the current outcomes for CARE Uganda of drawing from
both streams of thought. It includes both graphic and text presentations of these
ideas, and then develops a carefully identified set of indicators that the CARE
Uganda team believes to be adequate and appropriate to the task of demonstrating
programmatic aspects of household livelihood security in this setting.
A second module (the “M&E Tool Box”) presents a selected but relatively
comprehensive set of methods and tools, specifically linked to CARE Uganda’s
program impact indicators as identified in Module 1.
The third module is still in development (the “Program Impact Evaluation Guideline”);
this unit will present a strategic guideline for how a program impact evaluation
exercise can be carried out. The details of this process are still being refined, and
will then be tested in a pilot study in early 1999 before finalization of the module.
CARE Uganda and program
impact evaluation
As noted in the foreword to CARE Uganda’s M&E guidelines (‘How Are We Doing’,
1997), the Country Office faces a big challenge in assessing and understanding the
ways in which it is making a difference, beneficial or otherwise, in people’s lives and
the institutions serving them. Those initial broad M&E guidelines, which were
developed in 1996/97, focused on effects (intermediate goal level) and deferred
attention to impacts (final goal level) and their assessment until a second phase.
This second impact-oriented phase began in early 1998 with the compilation of a
“Technical/Conceptual Reader”, and then two “Impact Assessment” workshops held
at CARE Uganda headquarters 4-6 March, and 5-8 May.
Program Impact Evaluation Module 1: Overview
CARE Uganda
2
Program Impact Evaluation
CARE Uganda – 1st impact assessment workshop
During the first workshop (March 1998), CARE staff reviewed the conceptual and
technical reader in detail and then discussed its content: reasons for impact
assessment, concepts of poverty and household security, poverty interventions,
indicators and variables, design issues, data collection strategies and methods, and
impact analysis issues. The purpose of this review was to establish a common
knowledge base on poverty, the HLS conceptual framework, and the potential for
impact evaluation in the CARE program. Drawing on the literature of poverty and
development included in the conceptual reader, the aspects of poverty shown in the
box below were deemed especially important to the CARE Uganda staff.
Perspectives on poverty
 The poor are those persons deprived of basic human needs. The most fundamental
needs relate to survival (summarized as food, water and shelter, but incorporating
needs such as fuel wood for cooking or warmth). Aware of the situation in Uganda, the
participants would add personal safety to these most fundamental needs. These are
followed by needs related to ability to function and competence (health and literacy) and
finally to needs for empowerment (covering the less tangible, socio-politicalpsychological needs of choice, status and self-esteem).
Adapted from Greeley, 1992
 Poverty is deprivation of essential requirements for minimally acceptable fulfillment of
human needs, including food. This concept of deprivation goes well beyond the lack of
private income; it includes the need for basic health and education and other services
that have to be provided by the community to prevent people from falling into poverty. It
also recognizes the need for employment and participation.
Adapted from UNDP, 1997
 Poverty can be identified with the deprivation of basic needs, or with the lack of the
means to escape this deprivation, e.g., the lack of resources or capabilities to meet the
basic needs. Phrased in this way, the deprivation model of poverty is also able to
incorporate the idea of insufficient or inappropriate behavior to fulfil the basic needs,
e.g., misusing income for alcohol and other leisure items.

Very often, there are substantial intra-household differences in basic needs on the basis
of age, gender and life-cycle status, with the consequence that members of poor
households are not all equally poor. Indeed, there is some evidence that poverty may
aggravate intra-household inequity, e.g., when girl children are the first to be withdrawn
from school if there is little money for fees. A household framework recognizes that
individuals do not make decisions in isolation and that frequently decisions pertaining to
the allocation of resources, such as labor, capital and physical assets are made at the
household level.
Adapted from Sebstad 1995
 Individuals and households belong to social networks; membership in these networks
involves implicit mutual recognition of both an obligation to share resources and a claim
on resources in times of need. Examples include charity, collective work parties,
shared meals, patron-client relationships, rules of access to common property
resources, and community taxes. These mechanisms add to the security systems of
the poor, but if the only way an individual or household is regularly able to meet basic
survival needs is through debts that lead to subservient labor or dependency, then they
have not escaped from poverty.
Adapted from Greeley 1992
Program Impact Evaluation Module 1: Overview
CARE Uganda
3
Program Impact Evaluation
Impacts: Project versus program levels
One of the key issues to emerge from discussion of the reader was consensus to
use the broad definition for impact that the CARE Uganda M&E Task Force had
developed in the 1997 M&E guidelines. This definition was considered more
applicable and useful than a narrower definition focused only on outcomes limited to
households.
As the next activity in that first workshop, small sectoral teams reviewed a very wide
selection of indicators from sectoral documents and HLS materials. Their challenge
was to extract and/or develop a focused set of indicators and tools for impact
evaluation in Uganda relevant to the HLS concept. Their task concentrated on
showing what the selected indicators could demonstrate about poverty and/or HLS,
the value of the indicators for showing attributable change, feasibility of assessing
the indicators, and what methods/tools could be suitable for collection of data
relevant to the indicator.
All of the groups went through two rounds of identification, prioritizing discussions,
answering the critical questions, and then presenting their recommendations and
analyses to the plenary. Discussions were very active and constructive, everyone
pitching in to help the other groups.
One fundamental issue that emerged in the process of these discussions was an
acknowledgement that PROGRAM IMPACT is different than PROJECT IMPACT.
Programs refer to the total sum of interventions (i.e., projects and activities) affecting
a target group or geographic area. Use of the word ‘program’ in this sense does not
automatically imply ownership, as the interventions may be carried out by multiple
parties and partners, not CARE alone.
The discussion helped to clarify that program impact is, therefore, the result of efforts
at many levels and in many sectors – and by many actors. Program impact is more
than the sum of the separate ‘effects’ level changes brought about by the individual
projects. The indicators for Program Impact need to show the totality of impacts on
household livelihoods and community well-being.
Project impact, on the other hand, is tied to more limited objectives. Project Impact
indicators (and all the other output and effects indicators for a project) tend to focus
on sector-specific measures. Many, if not most, projects do not collect data on
household level impacts that are not directly linked to the sectoral efforts of the
project. For example, agricultural projects do not usually collect much, if any,
information on health impacts. At the same time, however, single sector projects can
have multiple direct and indirect impacts on the satisfaction of basic needs.
In this regard, the participants agreed that Program Impact indicators are not
intended to replace the sectoral indicators of Projects, but the two kinds of indicators
can complement each other. The Program Impact indicators are able to show more
about the context and constraints for the household as a whole, and the Project
Impact indicators are able to be much more specific and detailed about the
attributable changes due to a given intervention.
Program Impact Evaluation Module 1: Overview
CARE Uganda
4
Program Impact Evaluation
IMPACT: Project or Program
As used in CARE project terminology, ‘impact’ refers to fundamental and sustainable
changes in the human well-being of target populations, reflecting the satisfaction of
basic needs. Basic needs include adequate nutrition, good health, favorable habitat
(shelter, sanitation, water and immediate environment), life skills, and safety.
To qualify as project impacts, changes in these basic needs should be observable at
household level, and be attributable to project interventions. Because of the time
needed to attain household level impacts, they may be difficult to identify within the
lifetime of a project. Project impact indicators are usually associated with the final
goal level in a project’s logical framework.
At the program level, not all impacts on household and individual well-being will be
directly attributable to local interventions. Some might be, e.g., improvements in
health or access to natural resources, but others may come about through synergistic
effects and/or indirect program activities at other levels, e.g., advocacy, policy
development, national information campaigns, etc. In addition, at the program level,
CARE is particularly concerned about the total well-being of the household and its
members, not focussing on aspects of a single sector as projects tend to do.
Program impact indicators must, therefore, be plausibly associated with changes in
the overall well-being of the household.
A word of warning: as with ‘effects’, impacts may be intended or unintended and
beneficial or harmful.
(adapted from: “How Are We Doing”, 1997)
CARE Uganda – 2nd impact workshop
At the start of the second impact workshop, CARE staff reviewed a draft version of
this overview module, plus the M&E Tool Box (now the second module in the series).
The discussions that grew out of these reviews were very stimulating and fruitful.
Among the outcomes were a number of suggestions about additional ways to revise
the impact definition statement (see the box on this page).
The team also identified many aspects of the CARE International HLS framework
that needed adaptation in order to suit both the CARE Uganda program and the
existing situation in Uganda. The results of these discussions form the core of this
module, and are presented as the CARE Uganda HLS Model after a summary of the
CARE International HLS framework (see graphic on next page).
Next in the May workshop, the participants ‘went back to the drawing board’ with the
program impact indicators they had selected during the previous workshop and
rethought their selections. On this occasion, the small groups focused on careful,
clear definitions for the indicators, explanation of what each indicator could show,
data required to produce the indicator, format and/or units for the results, and
important considerations about each indicator (how selected, limitations,
assumptions, etc.).
Program Impact Evaluation Module 1: Overview
CARE Uganda
5
Program Impact Evaluation
The bottom line in this effort was aiming for a minimum set of indicators for program
evaluation in Uganda. While trying to keep the list to a minimum for practical
reasons, the other criteria guiding this selection were to choose a set of indicators
that would be:
 Feasible to collect simultaneously,
 Comprehensible for analysis, and
 Capable of reflecting the totality of well-being in the household.
 Able to show change occurring in a period comparable to the life-span of many
projects and/or program cycles, i.e., two to five years.
This exercise was quite challenging and the team responded with very energetic and
participatory discussions. As each of the groups was tasked with both sectoral and
cross-cutting and community level indicators, people were stimulated to think about
multi-sectorality and deepening their analysis of the HLS and well-being model.
The latter part of the workshop was devoted to reading and discussing a draft
implementation plan for program impact evaluation. The group considered technical
issues, as well as issues of timing, application for the exercise, and what are the
differences between the CARE Uganda program impact evaluation process versus
the Livelihood Security Assessments of CARE international. The results of this
session are presented in the latter part of this module under the heading, Utility of
the Uganda HLS conceptual model.
CARE’s Household Livelihood Security
Model
Household Livelihood Security (HLS) is an
organizing principle and integrating framework for
CARE’s work globally in relief, rehabilitation and
development. In this framework, households are
the ultimate social and economic units within which
resources are organized and allocated to meet the
basic needs of its individual members.
HLS is defined, in general terms, as “adequate and
sustainable access to income and resources to meet basic
needs.” Within this framework, CARE International defines
the six basic needs as: Food, Shelter, Education, Health,
Water, and Community Participation. The HLS model says
that the means by which household members meet the
basic needs include: income, financial resources, productive
assets, know-how, time, and natural resources. The model
also says that certain social, political, economic and
cultural factors influence the distribution and/or access to
productive resources or assets that households and their
various members have to assist them in meeting their basic
needs.
(Adapted from HLS documents, CARE International; 1995-7)
Program Impact Evaluation Module 1: Overview
CARE Uganda
6
Household Livelihood Security Models
CARE International and HLS
The concept of ‘security’
The notion of "security" has moved
to a central place in poverty
analyses, including those of CARE
International, in order to cover the
risk factors that constitute
vulnerability. Vulnerability has
special significance because the
sudden collapse of livelihoods
caused by shock or stressful
events necessitates more
immediate and costly interventions
for development programs to
address than it does for stable
poverty.
Risks that threaten the well-being
or livelihood security of a
population may be external or
internal. At a very broad level of
generalization, external risks or
shocks, including natural disasters,
war, or epidemics may constitute
vulnerability even in the absence of
poverty. Meanwhile, internal risks,
referring to stresses or shocks to
the household's access to or
control of basic means (and
therefore, coping capacity) may be
particularly associated with poverty
and scarce resources. It can be
assumed in the real world that all
people who are poor are to some
degree vulnerable, as they are
likely to have limited coping
capacities.
Adapted from Bart et al 1996
Component definitions within
the HLS framework
In order to more adequately explain and
monitor HLS, CARE International has
developed a set of components or sectoral
‘securities’ within the HLS framework that
expand the concepts for ease in defining
indicators.
The concept of “sustainability” appears
several times in this set of definitions for
component ‘securities’; and it is used with
more than one meaning. On the one hand,
sustainability is used in a functional sense to
refer to meeting basic needs over time, e.g.,
throughout the seasons, years and
generations. On the other hand, it is also
used in a more structural sense to refer to
maintaining the means of meeting basic
needs, e.g., assets, resources for production,
social networks, etc.
The term “access” is used frequently in the
list; in CARE terminology, this refers to
‘capacity to procure’, whether by production,
exchange or their claims on social networks
(e.g., reciprocal borrowing) or external forms
of support (e.g., remittances). Another
recurrent concept in these definitions is
“adequacy”. As used in this list, adequacy
refers to both qualitative and quantitative
aspects of sufficiency.
The following list covers the set of CARE
securities and their definitions that have been
disseminated to date, plus some additional
clarifying comments that have emerged in the
process of CARE Uganda’s work with this
list.
Household Livelihood Security – Adequate and sustainable access to income and
resources to meet the basic needs of the members of the household. The
Program Impact Evaluation Module 1: Overview
CARE Uganda
7
Household Livelihood Security Models
CARE Uganda HLS model inserts behavior into this equation, i.e., the
household’s utilization of available resources and income to cater for basic
needs. The well-being aspect of the model also brings in the idea of sufficient
know-how (skills and knowledge) to make use of the means.
Nutritional security – Sustainable access to a quantitatively adequate and
qualitatively appropriate (nutritionally sufficient and culturally acceptable)
dietary intake, personal care and the proper health and sanitation to maintain
the biological condition of the members of the household. Although nutritional
security implies food security (see below), it is also able to consider how food
choices, food preparation and feeding behavior affect nutritional status within
the household.
Food security – Sustainable access to a quantitatively (in calories) and qualitatively
adequate and stable (across seasons and years) food basket that is culturally
acceptable to the members of the household. Due to the difficulties of
assessing feeding behaviors, food security is often used as a proxy for
nutritional security.
Health security – Individuals belonging to a household are able to identify, prevent
and manage risks posed to their health, supported by appropriate health
technologies, community norms and systems, institutions and public policies.
Health security also includes aspects of availability, access and utilization of
basic preventive and curative health services. Note that the concept of
‘appropriate’ does not automatically exclude alternative, non-health center
forms of health management, e.g., traditional medicine.
Habitat security – Adequate access to shelter and related habitat resources, such
as fuel, lighting, and water, a healthy and sanitary environment and protection
from harm (personal and domestic violence, aggression) and other
detrimental elements and risks. This ‘security’ is closely linked with water and
sanitation and environmental security (see below).
Water and sanitation security – Adequate access to, and proper utilization, of safe
drinking water and sanitation services to the household.
Educational security – The availability, access and feasible utilization of the basic
educational services towards skill and personal development to enable the
household to affect its economic, political and social situation.
Economic security – Adequate and sustainable access to income derived from
production, labor and/or other sources to enable households to meet their
basic needs.
Environmental security – no operational definition from CARE (Atlanta) yet; though
the literature within CARE so far makes reference to springs and water
sources, and areas with conservation practices
Civil society security - no operational definition from CARE (Atlanta) yet, though
the literature within CARE includes issues of participation in community
Program Impact Evaluation Module 1: Overview
CARE Uganda
8
Household Livelihood Security Models
organizations, level of democratization of community organizations, impact of
community organizations, community organizations’ ability to influence local
or regional government. These networks, social groups, shared values,
norms and trust that make up the social fabric of a society are being referred
to as “social capital”, and are now seen to be important in supporting a
society’s productive potential. This element could also incorporate protection
from harm at the community level (collective action or delegation of action to
maintain social norms of peace and relative harmony).
Household livelihoods can be seen to consist of a range of “on-farm” and “off-farm”
activities, which together provide a variety of procurement strategies for food, cash,
and other basic needs. Livelihoods are secure when households have ownership of,
or access to, the resources and income-earning activities – including reserves and
assets – to offset risks, ease shocks, and meet contingencies.
The CARE Uganda staff in the Impact workshop accepted the above point, noting,
however, that some households may have adequate assets (e.g., land and/or
livestock), but lack essential knowledge in order to translate these assets into viable
production for the household. In addition, there are some relatively common types of
vulnerable households, e.g., those ‘headed’ by alcoholics.
A key limitation of the CARE International HLS framework, with its focus on ends and
means, is that it does not confront inequities or imbalances within the household (or
the community). For example, some households with enough food for all the family
members do not allocate sufficient food to certain vulnerable groups, like very young
children, girls, or pregnant women. In other households, there may be sufficient
land, but some members are restricted from access to it or to a share of the income
generated by their labor, such as adolescents. In still others, the family members
are not given a chance to participate in deciding about the use of income they have
earned, e.g., the husband who takes the wife’s earnings and spends them on
alcohol. All of these issues relate to the concepts of behavior and equity, i.e., what
happens when people have to apply the ‘means’ to reach the ‘ends’?
Household well-being and the
CARE Uganda HLS Model
Adapting the basic needs and HLS concepts
This text is designed to be read together with the accompanying graphic illustrating a
“CARE Uganda HLS Model” (see page 11). This model adapts the HLS framework
of CARE International to the conditions of Uganda. It moves from an outer ring of
‘context’, through ‘basic means’, to ‘behaviors’ and thence to individual and
household well-being attained by sustainable and adequate satisfaction of basic
needs.
Program Impact Evaluation Module 1: Overview
CARE Uganda
9
Household Livelihood Security Models
In its long range and strategic planning, CARE believes its development efforts
should ultimately have a positive influence on the community and household levels,
and that is where their impact should be evaluated. This CARE Uganda HLS model
is quite robust in the way it incorporates both structure and function at community
and household levels, thus allowing a rich analysis of livelihood and household
security.
ENABLING CONTEXT – the human and natural
environment
The surrounding context that enables (or disables) the functioning of households is
comprised of many elements, some of the main ones being the entirety of the natural
environment and the human environment, especially the societal, cultural, political,
and economic aspects.
As a variable or set of variables, the context is essentially independent of the
individual household – it influences each and every household but it is unlikely to be
influenced by any particular household alone.
BASIC MEANS – sustainable and adequate access to
crucial resources and opportunities
Every household requires sufficient availability and access to certain fundamental
resources, including services and opportunities, in order to meet its basic needs.
These include:

Natural resources – raw materials for food, water, fuel, lighting, and shelter

Household and/or communal assets – for production and for storing value
(including the ‘human capital’ of skills and labor capacity)

Income - production of value that can be exchanged for supplying basic
needs, whether monetary, labor/time, barter, etc.

Social structures for participation – effective community committees and fora
for decision-making and implementing the decisions related to communal
well-being.

Services – primary education, preventive and curative health care, sanitation
in public places, collective protection of human rights and life.
The “basic means” constitute an outer level of intermediate variables, i.e., they are
prerequisite opportunities that are partially, but not completely within the control of
any given household. Many, if not most, of the basic means are shared with other
households and/or controlled collectively in a community.
Program Impact Evaluation Module 1: Overview
CARE Uganda
10
Household Livelihood Security Models
The CARE Uganda HLS Model
Context
Basic means
Behaviors
Basic ends
Program Impact Evaluation Module 1: Overview
CARE Uganda
11
Household Livelihood Security Models
BEHAVIOR – sustained application of appropriate
preventive, productive and coping actions
Access and availability of means are insufficient in themselves to be transformed into
satisfaction of basic needs for all members of a household; that step requires
conscious and appropriate behaviors that equitably balance the resources and
needs in the best interests of all household (and community) members. These
behaviors include:

Feeding - Appropriately and equitably providing for the different nutritional
needs of all household members (infants, pregnant women, elderly, etc.)

Labor – Responding appropriately and equitably to the differences in capacity
for physical labor of all household members (children, pregnant women,
disabled, etc.)

Time – Appropriately and equitably providing for the needs of all household
members to rest and have time to apply their skills for personal and
household well-being (by age and gender, including activity outside the home
like community participation)

Protecting – Ensuring the security of all persons against abuse, violence, and
personal harm, in the community and within the household (domestic
protection)

Health care – Responding appropriately and equitably to the preventive and
curative health care needs of all household members (women, disabled,
elderly, etc.)

Educating – Equitably supporting the opportunity for all household members
(regardless of gender, disability, etc.) to obtain at least a minimum standard of
education (literacy, numeracy, life skills, social values)

Housing and hygiene – providing and maintaining clean and safe
surroundings in and around the dwelling for all ages and genders in the
household.

Resource use – Ensuring the sustainability and quality of natural resources
used by the household. Some people refer to this concept as ‘primary
environmental care’ at the village and household level.
The “basic behaviors” are the second level of intermediate or intervening variables,
i.e., they represent the process of decisions or choices that the household makes
internally in distributing resources and coping with the vulnerabilities and needs of its
members. These behaviors are within the control of the household, but strongly
influenced by cultural and social values, education, and the nature of the needs in
the family. The behaviors are also dynamic, varying over time and with the differing
situations to which people are exposed.
Program Impact Evaluation Module 1: Overview
CARE Uganda
12
Indicators of HLS
BASIC, DESIRED ENDS – adequate, sustained, and
sustainable satisfaction of the basic needs
The concept of basic or desired ‘ends’ is the “full cup” phrase referring to the
satisfaction of the “empty cup” aspect of ‘basic needs’. Satisfaction of these needs
can be expressed, and assessed, both qualitatively (personal perceptions) and
quantitatively (i.e., with some standard measures of status). The basic or desired
‘ends’ include the following:

Nutrition – sufficient for growth, health, and performance of roles/tasks

Health – positive physical and mental state, not just absence of disease

Safety – from personal harm/violence

Habitat – safe and hygienic shelter and surroundings; adequate basic
resources, including safe water and fuel for cooking, heat, lighting

Life skills – problem solving capacity, know-how, literacy/numeracy, personal
hygiene, etc.
These “basic ends” are essentially ‘outcome’ variables, dependent on the behaviors
of households in using or accessing the basic means that are in turn dependent on
the context for their availability and appropriateness.
Indicators: measuring and
evaluating HLS and well-being
General caveats
Using either the CARE Uganda HLS Model or the CARE International Livelihood
Security Framework for impact evaluation calls for a cross-sectoral assessment and
analysis, which generally puts it at levels higher than individual sectoral projects. As
such, it has been a slow process within CARE to identify suitable indicators of
program impact for either the HLS framework as a whole, or for its component parts.
A difficult point to resolve has been whether to evaluate “ends” (outcomes) or
“means” (e.g., capacities). Much discussion has centered on whether “success”
(impact) has been achieved if capacities are established but outcomes are not yet
demonstrable. The satisfaction of basic needs (i.e., achieving ‘basic ends’) takes
much longer than improving capacities, necessitating a longer range perspective in
assessment and evaluation.
In addition, there is the issue of attribution – it is easier to isolate and demonstrate
sectoral outcomes (especially effects) than the more general HLS or well-being
impacts at program level. Well-being changes can be influenced by many other
Program Impact Evaluation Module 1: Overview
CARE Uganda
13
Indicators of HLS
concurrent factors in the context of the households (e.g., social, political, economic,
cultural, environmental; plus other agencies and their interventions).
CARE Uganda’s minimum set of indicators for evaluating
Program Impact
CARE Uganda’s minimum cluster of indicators for evaluating program impact is
shown in the box below. In the planning workshops, sectoral groups started with
project level indicators and worked up to program indicators able to be integrated for
a broad picture of the HLS. They also started with household level indicators, but
found they needed to add some community level indicators related to the means and
behavior rings in the CARE Uganda HLS model.
Indicators for Program HLS Impact Assessment, CARE Uganda
 Food security and nutritional coping – access to adequate quantity, quality,
intake of food on a regular basis (and coping strategies to achieve this)
 Growth stunting – proportion of children 12-36 months who are below 2SD of the
international height for age standard
 Child spacing – intervals between end of the last two pregnancies, regardless of
outcome, for women with at least one pregnancy ending within past two years.
 Recent illness patterns – new illness occurring in past two weeks to any member
of household, serious enough to require treatment
 Morbidity from water-borne diseases – frequency of diarrheal diseases in children
6-24 months assessed during 2 week illness recall
 Economic assets – household assets (land, shelter, animals, tools); business
assets (shop, inventory, equipment); monetary assets (savings, other inflows)
 Economic coping – sources of funds (or other means) to pay for common
economic shocks and stresses to the household
 Educational attainment – formal educational system achievements of children age
13-15 at the time of survey
 Household relationship to key natural resources – procedures and behaviors at
household level to ensure sustainable access and use of locally important natural
resources on public and private land
 Social structures and patterns of social interactions – patterns and prevalence
of formal/organized social groups in sampled community, and their leadership
 Existence and use of physical infrastructure in the community – availability and
quality of key physical infrastructure in the community
 Community participation in management of ‘common goods’ – extent of
participation of local people in management of ‘common goods’: health, water,
sanitation, security, publicly owned environment (forest, lake, wetland, river, grazing
land); including participation of various vulnerable groups and constraints to
participation
The list is relatively short (only 12 indicators), since feasibility of data collection and
analysis by a busy country office was one of the strictest of the team’s filters.
Program Impact Evaluation Module 1: Overview
CARE Uganda
14
Indicators of HLS
Nonetheless, the list is also quite robust in its capacity to demonstrate change in
most of the crucial elements of the HLS.
The following three sets of tables illustrate just how robust and comprehensive this
‘minimum’ set of indicators is in covering these issues.
The first table illustrates how all of the CARE International’s HLS component
securities are addressed by CARE Uganda’s selection of HLS (and community)
indicators. Moreover, both of the inner rings in the CARE International HLS
Framework (see page 6) and all three of the inner rings of the CARE Uganda HLS
Model (see page 11) are dealt with in a comprehensive way.
There are two conceptual areas which are not currently addressed in detail in the
defined indicators: time utilization by family members and personal or family
protection and safety. The team recognizes that these two areas are important, but
did not specify exact indicators for either of them at this time. Some proxy
information about time utilization can be obtained from the data on educational
attainment (by gender), household relationship to natural resources (who is involved
at household level), social group interaction and community participation. Safety
issues may be covered by economic coping, social structures, and community
participation. In addition, using historical mapping to generate a trend profile for the
community can also help to encourage such issues to emerge.
The second set of tables portrays key issues that affected the selection of these
indicators: operational definitions, explanations (what the indicator shows, what
influences the indicator, and any existing literature or international standards), uses
and applications, and critical considerations about the indicator (constraints or
limitations and how they can be addressed).
This table and the next one also give a picture of how the indicators correlate with
the existing sectors in CARE. The full list is somewhat heavier on ANR and Health
sectors, followed by Civil Society, Water and Education – a balance that CARE
Uganda believes is reflective of both their country situation and their program
emphasis.
The third set of tables focuses on how the indicators can be implemented: data
requirements for generating or analyzing the indicators, possible methods and tools
for data collection (linked to the accompanying second module – the M&E Tool Box),
and the anticipated format of the results from the indicator findings.
Program Impact Evaluation Module 1: Overview
CARE Uganda
15
Indicators of HLS
CARE Uganda’s Program Impact Evaluation Indicators
Indicators for HLS
Impact Evaluation
Food security and
nutritional coping
Growth stunting
Child spacing
Recent illness
patterns
Morbidity from waterborne diseases
Economic assets
Economic coping
Educational
attainment
Household relation to
key natural resources
Social structures and
interactions
Existence and use of
physical infrastructure
in the community
Community
participation in
management of
‘common goods’
How do CARE Uganda’s indicators relate to
CARE International’s HLS “securities”?
Food – sustainability of access to adequate food
How do the CARE Uganda indicators relate to CARE Uganda’s HLS
Model?
Ends – nutrition Behavior – feeding
Means – natural resources, assets
Nutritional – adequacy of sustained/chronic
nutritional intake
Health – intentional prevention of health risks
Ends – nutrition, health
Behavior – feeding, health care
Means – natural resources, assets
Ends – health, life skills
Behavior – health care, education
Means – services (health)
Ends – health, life skills, habitat, nutrition
Behavior – health care,
sanitation/water, feeding, housing Means – services, assets
Health – quality of home diagnosis and
management; can include usual prevention
Water and sanitation – adequacy of utilization of
clean water and good sanitation
Economic – adequate and sustainable access to
income
Economic – access to income to meet basic needs
Health – manage risks, service use in emergency
Education – utilization of basic services
Educational – utilization of basic educational
services
Habitat (Environment) – access to habitat resources
Civil society – capacity to influence setting
Civil society – participation in community
organizations, democratization
Education – access to services
Health – appropriate technology, system, institutions
Water – access to water and sanitation services
Ends – habitat, life skills, health Behavior – sanitation/water, health care
Means – services, assets, participation
Ends – habitat, health, nutrition
Behavior – housing, labor and time
Means – assets, income
Ends – safety, health, nutrition
Behavior – housing, health care, education, protection
Means – assets, income, participation
Ends – life skills
Behavior – education, labor and time
Means – services
Ends – habitat, nutrition Behavior – labor/time, sanitation/water, feeding,
education
Means – natural resources, participation
Ends – safety
Behavior – protection
Means – participation
Ends – safety, health, life skills
Behavior – health care, education, protection
Means – services, participation
Ends – safety, habitat
Behavior – protection, labor and time, sanitation/water, health care
Means – participation, natural resources
Civil society – community capacity to influence local
government, participation, impact of local
organizations; equity, democratization
Economic security
(See also following set of tables with details on the definition of the indicators and their means of assessment)
Program Impact Evaluation Module 1: Overview
CARE Uganda
16
Indicators of HLS
Indicators – definitions, explanations, applications and
considerations
The next table presents explanatory information that will help the reader to
understand more about why these particular indicators were selected for Program
Impact Evaluation by the CARE Uganda M&E Task Force.
The first column lists the indicators, followed by a notation about relevant CARE
sectors or components. As described in more detail in the following table (Indicators
– data requirements, tools, results), this notation of sector may be relevant for
deciding on additional emphasis when the Program Impact Evaluation is being done
in an area that has had a particular sector-oriented project or set of projects.
The second column, ‘Definition’, gives an operational definition or specification for
the indicator, often presented in rather concise terms. The indicators include a
balance of qualitative and quantitative information; they also balance outcomes and
behaviors, household and community levels.
The third column, ‘Explanation’ gives a more detailed illustration of what the indicator
is analytically capable of demonstrating. The content in the cells of the table for this
column is divided into three subsections:
 “Shows” presents what the indicator can demonstrate;
 “Influenced by” discusses factors affecting the findings of indicator
 “Literature” gives some idea of whether any existing literature or international
standards are available for comparison with field based findings (based on
experience in Uganda).
The fourth column, ‘Use/Application’, suggests how the analysis of each indicator
can be interpreted and applied in program impact evaluation. The individual entries
in this column are focused only on the individual indicators; naturally, the analysis of
the full set of indicators has considerably more potential for seeing the interlinking
influences, as per the conceptual model shown earlier (see page 11).
The fifth column, ‘Considerations’, discusses some of the leading limitations of each
indicator and strategies to help minimize these constraints.
Program Impact Evaluation Module 1: Overview
CARE Uganda
17
Indicators of HLS
Indicators – definitions, explanations, applications and considerations
Indicator
1. Food
security and
nutritional
coping
strategies
Definition
Access to adequate:
- quantity
- quality
- Intake of food on
regular basis
(ANR,
nutrition)
2. Stunting
(nutrition,
health)
Proportion of children
12-36 months who
are below 2Z (2 SDs)
of the international
height for age
standard
The pre-12 months
period assesses
maternal nutrition;
above 36 months
makes the duration of
assessment too long
Explanation
Shows – Can show sustainability
of access to adequate food
supplies
Influenced by – routine season and
droughts or floods, culture, soil
fertility, access to land, family
size, assets, personal security
Literature – some older
documents exist on food security
for various regions
Shows – capacity and behavior
toward feeding and nourishing
young children
Influenced by – economic and
environmental conditions,
culture, behavior; HIV, education
of mother, birth interval
Literature – existing data for
comparison (especially UDHS),
and there is an international
standard
Program Impact Evaluation Module 1: Overview
CARE Uganda
Use/Application
- Portray changes in
patterns of food security
- Portray patterns of
response to nutritional
crises
Consideration
- Seasonal influences are very strong;
repetitions of this indicator should be done
at the same season
- Based on recall data; best information
from women
- May require extra questions to assess
intra-family equity of food distribution (e.g.,
to pregnant women, infants, etc.)
- Prevalence of stunting by
type of household, age and
gender of child
- May be able to see relation
between # of young
children in HH and stunting
- Shows part of the picture
of nutritional security of the
household
- Primary collection allows
HH level correlation with
others socio-economic
status indicators
- Stunting is the major indicator for
nutritional status in Uganda (better than
wasting)
- Is a long term indicator – and the changes
are cumulative, so the age interval of
children needs to be limited
- Use length of child under 24 months, and
standing height thereafter, which is
associated with an automatic drop in
height on the growth chart
- Does not obtain exact patterns of feeding
18
Indicators of HLS
Indicator
3. Child
spacing
(health,
population)
4. Recent
Illness
patterns
(health)
Definition
Interval between
end of two
recent
pregnancies,
irrespective of
outcome
Explanation
Shows – reproductive health
behavior. Can also give
indication of infant mortality
and pre-natal mortality
Influenced by – age of mother,
education, use of birth control,
marital status, death of
previous child or loss of
pregnancy, access to services
Literature – existing literature for
comparison; and an
international standard for safe
interval
New (fresh)
illness occurring
in the past 2
weeks, for any
member of the
family; serious
enough to
require
treatment
- Shows common illness
occurrence (and management
choices) by age and gender
Influenced by – season,
knowledge, access to service,
exposure and behavior
Literature – existing literature for
comparison – but no
international standard
Program Impact Evaluation Module 1: Overview
CARE Uganda
Use/Application
- Patterns of intentional child
spacing, by household
characteristics and
characteristics of the mothers
- Rates of pregnancy wastage and
death of infants and young
children
- Patterns of adolescent
childbearing
- Some picture of reproductive
health security and decisions
- Proportion and characteristics of
HHs with birth intervals that are
too short, too early, too late
- Ability of HHs to recognize,
prevent and manage risk to their
health
- Can give patterns of use of
household resources (cash,
caregivers, etc)
- Shows use patterns for health
resources: drug shops, clinics,
home treatment, healers, etc.
Consideration
- Intentional abortion is a sensitive issue and it
may be difficult to obtain accurate information
without special questions and/or training of the
interviewers
- Contraception use data applies only to women
who have had a recent pregnancy
- Will miss maternal mortality because the
questions focus on living women
- Is based on recall data; likely to have poor
data if one tries to go back further than two
pregnancies
- Need information about local diagnostic terms
and perceptions (e.g., omusuja for fever and
malaria), including severity (unable to work?)
- Is recall data; will be best quality if it comes
from the main caregiver in the family (often the
mother)
- Seasonal influences may be strong enough to
recommend that repetitions of this indicator be
done at approximately the same season
19
Indicators of HLS
Indicator
5. Recent
morbidity
due to
water-borne
disease
(water,
health)
6. Assets
(SEAD,
ANR)
Definition
Frequency of
diarrheal disease in
children 6-24
months, assessed
by 2 week recall
(obtain as part of
recent illness recall,
No.1 above)
- Assets owned or
managed by HHs;
- HH assets: land,
shelter, animals,
tools, savings (other
inflows,
remittances, etc.)
- Business assets
are: shops,
inventory,
equipment, etc.
Explanation
Shows – susceptibility to
diseases caused by
inadequate water/sanitation
(access, use)
Influenced by – season,
knowledge, behavior,
exposure, access to water and
sanitation facilities, HIV/AIDS
- Is some existing literature for
comparison, and some
standards, e.g., for latrines
Shows – ability of HH to offset
risk by storing value
Influenced by – culture, genderbased access to (and control
of) key assets
Literature – some existing
literature (World Bank –
Integrated SES surveys)
Program Impact Evaluation Module 1: Overview
CARE Uganda
Use/Application
Reflects part of the pattern
of habitat security
Consideration
- Seasonal influences may be strong, repetitions of
this indicator be done at same season
- Need to obtain local perceptions and criteria for
diagnosing diarrhea since it has context-specific
definitions (e.g., watery, bloody, smelly, etc.)
- Need special questions on treatment of diarrhea to
find out use of simple remedies that might not be
called ‘serious treatment’, e.g., fluids, ORS, etc.
- Assessment of economic
security of HHs; is key
indicator of economic
well-being
- Can create an index of
financial value for
standard assets in local
communities
- Can do gender analysis
for variations in access
to, or control of assets
- Can be difficulty in valuing quality (worth) of
assets, e.g., because of variations in age,
condition, market opportunities for resale
- May be different gender-based perceptions about
access to and control of assets
- business assets may be difficult to assess,
because they can be less visible
- does not show debts (liabilities), e.g., loans
- People can be shy to reveal financial data
- Pace of change is slow in most rural areas; may
not give sufficient link to changes occurring during
a specific time period
20
Indicators of HLS
Indicator
7. HH coping
in response
to economic
shocks and
stresses
(SEAD,
health,
education,
ANR)
8.
Educational
achievement
of young
adolescents
(education)
Definition
Means of paying (sources
of funds, barter) for
unplanned economic
shocks to the HH and
large economic stresses
‘Unplanned’ shocks:
urgent health care,
funeral costs, losses
from fire or theft,
displacement, etc.
- ‘Planned’ stresses
include: marriage
dowry, asset purchase,
education costs,
repaying loans, etc.
- Proportion of children
aged 13-15 with a
minimum of P7
education by gender
and age
Explanation
Shows – coping capacity and
strategies of households, including
whether they have to use
productive assets to cope with
emergencies
Influenced by – exposure, assets
available, season, markets,
education, gender and age of HoH,
family size (exposure to risks),
culture, group membership and
opportunity for joint savings or
reciprocal loans (borrowing
networks)
Literature – some literature in certain
sectors, e.g., health
Shows – P7 level is proxy for
problem solving skills, social
values, basic numeracy and
literacy. Age 13-15 is the usual
age for completion of P7, and it is
at the end of the pre-employment
and pre-childbearing age
Influenced by – parents’ education,
affluence, national policy, security,
education system, culture, behavior
of teachers and peers
Literature – some statistics (Min of
Education), some info on life skills,
but little for comparison
Program Impact Evaluation Module 1: Overview
CARE Uganda
Use/Application
- Assessment of
economic
security of
households
- Assessment of
community
support
systems
Consideration
- The nature of shocks may influence the choice of
coping strategy
- Certain shocks may be more overwhelming, e.g.,
displacement
- Pace of change is unpredictable, depends on
events
- Reason of selection
- Is more feasible and culturally acceptable than
trying to collect large amounts of information on
income or expenditures; recall of shocks is easy
for most people
- Addresses livelihood security of HH
- Life stage of family may be important in their
pattern of asset acquisition/investment
- Assess whether
basic
educational
needs are being
met
- Assess capacity
of HH to
support
education
- Indicator does not say anything about quality of
education or school environment (bullying, stigma,
lack of privacy at toilets, sexual harassment by
teachers and fellow students, etc.)
- Does not assess performance on P7 exams
- Does not assess current enrolment in school
21
Indicators of HLS
Indicator
9.
Household
relationship
to key
natural
resources
(ANR, water)
10. Social
structures
and patterns
of social
interaction
(civil society)
Definition
- Procedures
(including by-laws,
rules), knowledge
and behaviors in
place at household
level to ensure
sustainable access
and use of locally
most important
natural resources
on public and
private land
Explanation
Shows – ease of current access
and can demonstrate changes
over time. Also shows
understanding and actions to
maintain access
Influenced by – available
resources, affluence,
population density, education,
seasons, gender of head of
household and of user in HH
Literature – no international
standards; little literature
except with water
Use/Application
- Prevalence of environmentally
protective knowledge and
behaviors
- Patterns and
prevalence of
formal, organized
social groups in
the community
Shows – some information
about coping strategies,
safety nets, and opportunities
in HHs, communities. It is
linked with access to services
and resources
Influenced by – culture,
affluence, education,
proximity of HHs, stability of
community, HH/community
stresses and shocks
Literature – some literature, but
no international standards
- Clarifying social “capital”
(resources)
- Type of groups, their activities,
proportions of belonging by
age, gender, education, socioeconomic status
- Behavioral characteristics, e.g.,
recreational behavior, social
status through type of network
involved in
- Gender analysis of leadership
and membership patterns
Program Impact Evaluation Module 1: Overview
CARE Uganda
Consideration
- Strongly influenced by seasons; do
repetitions in same season
- Need qualitative preparatory work to identify
key natural resources in the locale and
relevant behaviors
- Links with indicator on management of
common good resources (community level)
- Need for trained person to verify subjective
assessments of the effectiveness of the
measures in use, i.e., that rate of use does
not exceed rate of recovery
- Illegal use may be difficult to identify
- Can use qualitative group approaches to find
out trends
- Community and HH can require group and
survey data collection
- Still learning about influence of socioeconomic status, etc. on patterns of
membership and groups; interpretation of
results will therefore require skills and time
- CARE is already using social groups as
resource for interventions.
22
Indicators of HLS
Indicator
11. Existence
and use of
physical
infrastructure
in the
community
(civil society,
health, water,
education)
12.
Participation
of local
people in the
management
of “common
goods”
(civil society,
health, water,
education,
ANR)
Definition
Availability and quality of
selected (key) types of
services and
infrastructure
Includes: schools, health
facilities, safe water
points, roads
Use refers to persons
getting the designated
services of the
infrastructure (health
care, schooling, etc.)
- Participation includes
nature and extent of
community members’
roles in decision-making
and monitoring
- Key ‘common goods’
include: Health, Water,
Sanitation, Security,
and Environment
(public owned, forests,
lakes, rives, grazing
land, wetlands)
Explanation
Shows – state of development
and essential service provision
of community
Influenced by – policy, culture,
community affluence, value for
‘common good’, population
density, security and
displacement (stability of the
community)
Literature – some literature but
little for comparison; there are
some international standards
(health units, water, schools)
Shows – can show participation
as contribution to
discussion/debate;
representation of stakeholders
on management committees;
involvement in key decisions,
implementing and enforcement
Influenced by – policy, literacy,
population density, culture,
season
Literature – some general
literature, no standards
Program Impact Evaluation Module 1: Overview
CARE Uganda
Use/Application
- Key indicator of well
being beyond the HH
level, i.e., community
well-being
- Can link with health
care utilization,
educational
achievements and
attendance, waterborne diseases
Consideration
- Definitions of quality and adequacy may be
difficult
- May be easier to assess use than non-use
- Reasons for non-use may vary (from dislike,
or poor access, to cost, or preferences for
other sources of services, etc.)
- Pace of change is gradual
- Portrayal of stakeholder
involvement in
leadership and
management of their
own community
- Involvement of and
perceptions about
participation by
marginalized groups
- Can be used to identify
prevalent perceptions
about ‘protected areas’
- Interpretation of ‘participation’ is variable;
have to be careful to distinguish ‘attendance’
and ‘participation’
- Barriers to participation may be extensive,
and they can be subtle (not so easy for
people to be aware of or to admit)
- Sustainability of changes extent of
participation may be difficult, especially if it
starts with external support
- Need to collect information for two year span
23
Indicators of HLS
Indicators – data requirements, tools, results
The next table presents another detailed view related to implementation of the
program impact evaluation indicators selected by the CARE Uganda M&E Task
Force. As in the previous table, the first column lists the indicators, followed by a
notation about relevant CARE sectors or components. As noted below, this may be
relevant for deciding on additional emphasis when the Program Impact Evaluation is
being done in an area that has had a particular sector-oriented project or set of
projects.
The second column in this table focuses on the practical issue of data required from
the field (or secondary literature) in order to describe or calculate each indicator.
The individual indicators often require more than one piece of information for
adequate analysis and presentation.
The third column shows a set of methods and tools that can be used to generate the
required data. These methods and tools are described in detail in the
accompanying module: The M&E Tool Box. The survey method is principally
quantitative and the focus groups are mostly qualitative, although each of these can
be used for both kinds of data. An additional, optional set of methods and tools is
listed for each indicator; these approaches can be used as supplementary
explanatory tools if time and resources are adequate. The optional methods can
also be applied for specific indicators when they relate to the sectoral emphasis of
projects in a given program area.
The fourth column indicates the expected format for the results of each indicator.
The optional material in italics shows the expected results if the optional methods
and tools are applied from the previous column.
Other general information that will be important to obtain:
In addition to the information listed in column two of the following table, there are
some additional data that will need to be collected in order to adequately calculate
and analyze the indicators. The collection of this data, which is mostly public and
non-sensitive information, can also serve a dual purpose in being part of the opening
sequence of questions for the survey part of the evaluation.

Head of household – age, gender

Size and composition of household – dependency ratio, and the presence of
any special groups (disabled, orphan, chronic illness, etc.)

Literacy – of HoH and/or main caregiver and decision-maker; if so, what
language(s)
Besides the additional quantitative information, it will be desirable to also obtain
some overall qualitative information, especially about perceptions of trends in the
community/region being evaluated. The development trends of interest will include
major household livelihood and well-being problems, constraints, resources and
supporting agencies, and unmet needs. Two useful tools for gathering this data will
be historical mapping for the program cycle (approximately five years) and a SWOL
analysis of major interventions in the region, including those of CARE.
Program Impact Evaluation Module 1: Overview
CARE Uganda
24
Indicators of HLS
Indicators – data requirements, tools, results
Indicator
1. Food
security and
nutritional
coping
strategies
(nutrition,
ANR)
2. Stunting
(nutrition,
health)
3. Child
spacing
(health,
population)
4. Recent
Illness
patterns
Data Required
- Coping strategies from food security index
- Duration of hunger period (months of
shortage of standard meals with dietary
balance)
- Family size, composition, nature of head of
household
- Age, gender, length or height (depending on
age of the child being measured)
- Need to know if this a biological child of the
adult caregivers in the household (or if it is
an orphan or fostered child)
- Age of mother, number of children in 12-36
month age bracket in household
- Age and marital status of mother
- Base information on women who have had
at least one pregnancy ending within the
past two years
- Dates and outcomes of the last pregnancy
and the one before that, if any; source of
delivery care
- Number of living children, intentional birth
spacing & method, current pregnancy or not
- Persons ill by age, gender; nature of illness
and sources of treatment.
- Marker diseases to be extracted at analysis:
measles, ARI, diarrhea for those under 5
years and malaria and ARI for all ages
(health)
Program Impact Evaluation Module 1: Overview
(For details about the recommended tools, refer to the M&E Tool Box)
Possible Methods/Tools
Unit of Results
Household survey
- Proportion of households using different coping
- Nutritional coping strategies
strategies during the lean months, by categories of
Focus groups
households (age, gender, education of head of
- Seasonal calendar
household, family size, presence of orphans)
Optional:
- Perceptions about seasonal periods of food shortage
- Rating and ranking
Option: Perceptions about trends, risks and
- Time line for trends
effectiveness of strategies
Secondary data
- Number and proportion of children per household
Household survey
who are ‘stunted’, by household characteristics
- Anthropometry
(socio-economic, numbers of young children, age of
Optional:
mother, etc.) and characteristics of the child
- Focus groups
- Number and proportion of stunted children in a
- Rating and ranking
population, by degree of stunting
Option: Perceptions about household and community
equity, contributing factors
Household survey
- Average (median) interval between end of
- Pregnancy and birth interval
pregnancies for women in reproductive age group
matrix
- Proportion of women with recent pregnancies who
are now using contraception
Optional
- Recent pregnancies by age of mother
- Focus groups
- Proportion of recent pregnancies not being carried to
- Rating and ranking
term: abortions and miscarriages
Option: perceptions about maternal and reproductive
health, risks, trends
Household survey
- Number of episodes per household per 2 week period
- Recent illness pattern matrix
- Number of persons by age and gender ill per
Focus groups
household per 2 wk period
- Rapid profile of significant
- Most common sources of treatment for marker
diseases (includes brainstorm,
diseases by household characteristics
free listing and ranking)
- Perceptions about most significant illnesses, their
Optional
usual management and prevention in this community
- Risk mapping for health in the
Option: perceptions about high risk areas in the
community; Time lines
community and trends in health
CARE Uganda
25
Indicators of HLS
Indicator
5. Recent
morbidity
due to
water-borne
disease
Data Required
- Age of young children, in
months; gender
- source of water for household
- availability and use of
latrine/toilet
- local definition or perceptions
of ‘diarrhea’
Possible Methods/Tools
Household survey
- Recent illness pattern matrix
Focus groups
- Free list terms for diarrhea
- Map water point location and quality; and
homes most affected by diarrhea
Optional:
- Rapid profile of significant diseases
- Community (male & female)
agreement on types of assets.
- Presence (and quality?) of
items on a short list of common
assets
- Characteristics of HH (male,
female, or child head;
education of HoH, etc.)
Focus groups
- Brainstorm, free listing, ranking to identify
locally important assets, develop local
asset value index
Household survey
- Assets and livelihood profile
Optional
- Pocket chart on local patterns of
selection and control of assets
- Inspection and observation of assets
- Wealth ranking of self and neighbors
Focus groups
- Brainstorm, free listing and ranking to
identify locally important shocks and
stresses for index
Household survey: Economic coping
profile (adapt nutrition coping index)
Optional
- Time line
- Historical map
- Group discussion of borrowing/lending
patterns, and equity
(water,
health)
6. Assets
(SEAD,
ANR)
7. HH
coping in
response to
economic
shocks and
stresses
(SEAD,
health,
education,
ANR)
- List of common sources of
cash (or barter) used locally to
meet shocks and stresses
- Recall period of 2 years.
- Type of HH: female headed,
age of head of household,
presence of orphans,
ownership of assets, etc.
Program Impact Evaluation Module 1: Overview
CARE Uganda
Unit of Results
- Proportions of children having diarrhea by age, socioeconomic status, mother’s education.
- Proportions of households having (and using) latrines or
toilets
- Locally relevant terminology for diarrhea
- Location of water points
- Perceptions about water-related health risks in the
community
Option: management of diarrheal episodes, age of children
being allowed to use latrines, personal hygiene
- Local patterns of asset acquisition and ownership
- Perceptions of important assets in this community
- Index of values for locally important assets
- Proportion of HHs owning, using, or controlling specified
assets by type of asset and type of household
Option: Patterns of control of assets, selection by age and
stage of family, quality of assets
- Perception on ability to cope with more stresses or shocks.
- Index of locally relevant economic shocks and stresses
- Proportion of HH using coping strategies correlated with
other wellbeing indicators (assets, family size, main
sources of livelihood, education , etc)
- Proportion of HHs using higher risk coping mechanism
(sale or barter of production assets)
Option: trends in the community coping capacities and
equity; local explanations of cause and effect with shocks
and stresses
26
Indicators of HLS
Indicator
8.
Educational
achievement
of young
adolescents
(education)
9.
Household
relationship
to key
natural
resources
(ANR, water)
10. Social
structures
and patterns
of social
interaction
Data Required
- Educational achievements by age, gender
and class level completed for children in 1315 year age group only
- List of key local natural resources used at
household level (consumption, collection,
production and sales) and persons using
- Reporting by household or group of
behaviors affecting sustainability of the
resources
- HH characteristics – age, gender, assets, etc.
- Quality of resource and availability through
the year, including distance to obtain
sufficient quality and quantity; also whether
resource is on public or private land
1. For HHs/individuals - Gender and age of
HH members belonging to groups, by type of
group; Leadership position in group, if any
2. For groups – number, age and gender of
members, admission rules, major activities,
life span of the group
(civil society)
Program Impact Evaluation Module 1: Overview
Possible Methods/Tools
Household survey
Optional
- Focus groups to discuss quality of
educational system, trends in
attendance and performance
- Find out current attendance patterns
Focus groups
- Resource map with commentary on
location and quality
Household survey
- Resource profile and household
behaviors
Optional
- Inspection, observation of resources
and any evidence of the household
behaviors with a checklist
Unit of Results
Proportion children aged 13-15 years who have
completed P7
Focus groups
- Social group assessment
Household survey
- See question in social group
assessment tool
Optional
- Venn/chapati diagrams on
relationships of various groups and
influence in the community, SWOL
- Types of groups, activities of groups in
community
- Proportion persons belonging to groups by
HH type, age, gender, education, occupation,
socio-economic status link by type and groups
- Proportion of HHs with group linkages, by type
of HH
Option: perceptions of lines of status and
influence
CARE Uganda
Option: community perceptions about education
system, trends, contributing factors
- Patterns of protective and extractive behaviors
in the community, by category of household
and resource; possibly disaggregated by user
(age, gender)
- Explanations or rationales for the behaviors
- Proportion of population using a given
resource and the prevalence of types of uses
- Opinions about protected areas
Option: confirm perceptions about resources
and reports about behaviors
27
Indicators of HLS
Indicator
11. Existence and
use of physical
infrastructure in
the community
(civil society,
health, water,
education)
Data Required
- List of key physical resources
- Reported (and observed) quality for each
resource.
- Perceptions about access to and use of
service and infrastructure resources by
community
Possible Methods/Tools
Focus groups
- Free list of services and
infrastructure
- Mobility, service map with
comments on quality, access, use
Optional
- Inspection, observation of
infrastructure and services
12. Participation
of local people in
the management
of “common
goods”
(civil society,
health, water,
education, ANR)
- Management structures available in
communities dealing with common good
items
- Membership of management
groups/committees by category: age,
gender, and other special criteria like
disability, etc.
- Types of decisions being made, extent of
implementation or monitoring
- Any groups excluded or less likely to
participate
Focus groups
- Free listing of locally important
common good items
- Pocket chart for decision making
- Venn/chapati diagram
Optional:
- Inspection, observation of the
common good items
- Trend assessment of changes in
management over time, reasons
Program Impact Evaluation Module 1: Overview
CARE Uganda
Unit of Results
- Presence and location of infrastructure by type
of items or facilities,
- Perceptions about quality and standards of
facilities and services
- Perceptions about use of/access to resources by
category of people (age, gender, marginalized,
etc.)
Option: confirm perceptions and reports of
community (compared to checklist or qualitative
observations)
- Nature and extent of community participation in
decisions (also implementation and monitoring)
of selected types of common good items: by
type of decision, type of common good, type of
community, and categories of persons in the
community (age, gender, disability, etc.)
- Perceptions of population about participation
Option: confirmation of community perceptions;
can include perceptions about ‘protected areas’
28
Utility of the HLS concepts
Utility of the HLS concepts
Program impact
The Project Evaluation Cycle
Information is gathered for or about
projects at several standard points in
the lifetime of a project:
 Before project – identification of
problems, contributing factors, and
extent of needs. Such studies are
often called pre-appraisal or needs
assessments.
 Project start-up – establishing
the pre-intervention status for
sectoral indicators that are
expected to change during the life
of the project, i.e., a baseline
study.
CARE International encourages the
development of relatively independent
country office programs to address the local
needs of the relief-rehabilitation-development
continuum.
These programs are comprised of projects
with clearly defined lifetimes, set by their
proposals and funders. Within CARE,
projects usually run for 3-5 year spans,
though individual projects may go through
more than one funding cycle.
 Mid-term – this is the time for
reassessing project strategies,
management systems, linkages
with partners, and starting to look
for effects level changes. These
exercises are formative, in the
sense that the information is used
to improve the project during the
latter part of its lifetime.
Country Programs, however, are less clearly
defined. CARE has already been operating
for more than two decades in some
countries. Increasingly, CARE is promoting
the value of periodic reviews of program
achievements and direction, potentially
including collection of new data, reflection
and analysis, as part of the Long-Range
Strategic Planning process.
 End-of-project – this is the point
when project interventions (and
funds) are terminated, and an
assessment is made of project
achievements during the period of
support. Such studies are called
final or summative evaluations.
Program information needs are therefore
linked with, but also distinct from, the
information needs of projects. More
specifically, programs require information for
the following three purposes:
 After project – at this time, well
after the withdrawal of project
support, it is possible to see if
benefits are sustained and can
therefore be truly called ‘impacts’.
These ‘ex-post’ studies rarely
happen, simply because the
funding has already ended, and
most donors are not willing to go
back for a second look.
Adapted from ‘How Are We Doing?’,
1997
Program impact
evaluation/assessment
Programs need impact information for their
planning cycles (e.g., annual and periodic
long-range strategic planning). In addition,
programs exist across project cycles and
thus have the opportunity to capture ex-post
project information long after the key actors
in a project have moved on to other activities.
A Program Impact Evaluation based on the
Program Impact Evaluation Module 1: Overview
CARE Uganda
29
Utility of the HLS concepts
HLS model can encourage country offices to do holistic problem analysis toward
understanding the context and totality of well-being for households and communities.
From a programmatic point of view, using the same set of indicators can permit
greater capacity for comparisons within a country program, e.g., between regions
with multiple projects in different sectors versus areas with a single project/single
sector intervention. Analyzing a common set of indicators can facilitate comparison
of the value of certain strategies in different settings, or between strategies in similar
settings. It can thereby help in the explanation of factors contributing to the success
and failure of projects. At the same time, however, it will be important to recognize
and acknowledge the contribution of many other partners and implementers in these
areas, from the community itself, up to local and central government and other
agencies.
Periodic assessment and holistic analysis of household livelihood security and wellbeing , possibly every 4-5 years, in conjunction with the LRSP cycle, could help in
defining collateral needs in CARE’s target regions. Such issues include situations of
need for which CARE could develop projects, or it could broker with other partners
for co-operative effort and thereby aim at synergistic outcomes. Issues in need of
advocacy could be highlighted by the Program Impact Evaluation process and then
be addressed through national as well as regional channels. In addition, the
Program Impact Evaluation process offers the opportunity for considerable crosssectoral sharing of ideas and experiences, leading to institutional strengthening
within the CARE program.
Targeting
From time to time, the country office needs to determine which groups or regions are
currently vulnerable or expected to be so in the near future. Vulnerability in this
sense implies a level of poverty where the poor are not meeting their basic needs,
either seasonally or year-round. This may be due to lack of means (e.g., poor
income generation, lack of essential services, etc.) or various constraints affecting
their use of the existing means. The constraints can be internal, e.g., insufficient or
inadequate behaviors (e.g., intra-community or intra-household inequity). There can
also be external constraints, e.g., inappropriate policies, prolonged drought,
insecurity, etc.
A targeting exercise is essentially a rapid situation analysis to select regions for
concentration, plus priorities for emphasis. Targeting where no CARE program or
projects exist necessitates gathering enough information to understand the local
potential for development, plus potential risks. To ensure that development
assistance projects in some sectors are viable (e.g., agricultural interventions,
income generation development), areas that are targeted must have a minimal level
of resources with which to work. In addition, different program interventions may be
applied in different regions, depending on the risks and opportunities of the target
group. The targeting exercise can help to identify key leverage points for bringing
about a bigger return on the investment of scarce resources.
The information needs for regional targeting are broad, and can often be answered
with a review of secondary data together with some qualitative field work for
confirmation, e.g., some focus groups with PRA/PLA activities and some key
Program Impact Evaluation Module 1: Overview
CARE Uganda
30
Utility of the HLS concepts
informant interviews, or CARE’s RLSA approach. Factors limiting the amount of
data gathering are budget (funding for this exploratory work does not come from any
specific project or donor), and time (the personnel involved in this work usually have
to be drawn away from other posts or projects within the program).
The indicators for targeting may be focused principally on needs, as illustrated by
standard indicators in many frequently used data sets, e.g., latrine coverage,
population density, AIDS cases per million, etc. In addition, it is important in the
targeting to identify resources, e.g., existing governmental and NGO programs and
projects in the region by nature of interventions, categories of beneficiaries and
distribution of activity. The qualitative data collection can be especially useful in
identifying the major constraints facing vulnerable households and communities.
Although the targeting exercise does not require using the minimum household wellbeing data set, keeping the CARE Uganda HLS model in mind (or the CARE
International HLS framework) will help in evolving multi-sector regional strategies
instead of single sector approaches.
Focusing interventions
This activity tends to be project- and sector-specific, as interventions are almost
entirely carried out within sectorally-oriented projects. Focusing interventions
includes the pre-appraisal stage of projects, where specific interventions are
selected, and mid-term formative appraisals where they are sharpened and refined.
In addition, some projects carry out operational research during their implementation
with a focus on project improvement. These three kinds of studies are generally
carried out within the M&E plans of the projects.
Some or all of the indicators developed for the CARE Uganda HLS model could
potentially be applied in this process, if time and resources were available. This
would permit a more careful exploration of constraints from the household and
community perspective, as well as enabling the program and project to look for ripple
effects in other sectors and components of the basic means, behaviors and needs.
It can also facilitate mobilizing partner agencies who can target interventions toward
key problem areas in the HLS model that the CARE project(s) in that region is/are
not funded to work with. Realistically, however, it is more likely that only the
sectorally-relevant indicators from the minimum data set may be selected for use in
these studies.
Additional information related to the purpose of understanding and refining
intervention approaches will come from the baselines and final evaluation studies of
projects. Such information is of particular use in building up the country office database about the strengths and weaknesses of particular program strategies in this
national setting.
Program Impact Evaluation Module 1: Overview
CARE Uganda
31
References
References
Bart, C.; Coelho, K; Fernicola, K.; Halmo, D.; and Langworthy, M. (1996) Cross-country
comparisons of food and livelihood security: a synthesis of RFSAs conducted by
CARE; (draft) University of Arizona
Barton, T. (1997) How Are We Doing? Guidelines to Monitoring and Evaluation; CARE
Uganda
Barton, T. (1998) CARE-Uganda Impact Assessment Project: A Technical/Conceptual
Reader; CARE
CARE (1996) LRSP Handbook (selected annexes); CARE
CARE Uganda (1998) Impact Assessment Workshop, Held at CARE Uganda Headquarters,
Kampala, 4th – 6th March 1998 (Proceedings); CARE
Edgcomb, E.L. (1996) PVO/NGO tools for household and business impact assessment:
report of a planning meeting; AIMS, USAID
Food Security Unit; (1996) CARE’ Approach to Food and Livelihood Security Profile:
Methodology and Analysis; CARE
Frankenberger, T.R. (1996?) Key issues related to household livelihood security; CARE
Frankenberger, T. R. (1997) HLS Co-ordination Meeting: Summary of major issues;
September 22-26, 1997, Atlanta; CARE
Greeley, M.; Kabeer, N.; Davies, S.; and Hussein, K. (1992) Measuring the poverty reduction
impact of development interventions: research proposal; IDS, Univ. of Sussex
Household Livelihood Security Working Group (1995) Household Livelihood Security: a
unifying conceptual framework for CARE Programming; CARE
Sebstad, J.; Neill, C.; Barnes, C.; and Chen, G. (1995) A framework for assessing the impact
of micro-enterprise interventions at the level of the household, the enterprise, the
individual and the community; USAID, PRISM
UNDP (1997) Human Development Report 1997; UNDP, NY
Program Impact Evaluation Module 1: Overview
CARE Uganda
32