Indicators -- definitions and distinctions

advertisement
INDICATORS: DEFINITIONS AND DISTINCTIONS
An indicator is a measure that is used to demonstrate change in a situation, or the progress in, or results of,
an activity, project, or programme.
TYPES OF INDICATORS
Indicators are essential instruments for monitoring and evaluation, and exist in many different forms.

Direct
Indirect/proxy


Qualitative

Quantitative
Direct indicators correspond precisely to results at any performance
level. For example, counts of children vaccinated are direct measures of
output; instances of change in hand-washing behaviour are a direct
measure of project/programme outcome; and a decrease in infant
mortality is a direct measure of project/programme impact.
Indirect or "proxy" indicators are used to demonstrate the change or
results where direct measures are not feasible. Some objectives,
particularly impact objectives, are difficult to monitor. It is often
necessary to select indirect or proxy indicators that may be easier for
evaluators to measure. For example, the effectiveness of a child health
programme may best be measured by mortality rates. These rates are
difficult to determine over short periods of time. For this reason, a proxy
indicator, such as the percentage of births that are attended by trained
health personnel, and the availability and frequency of use of health
facilities may be used.
Indicators are usually quantitative measures, i.e. a percentage or share,
rate (e.g. birth rate), or ratio (e.g. inhabitants/doctors).
Qualitative indicators can be defined as people's judgements and
perceptions about a subject.1 For example: the number of people
owning sewing machines in a village is a quantitative indicator, the
confidence those people have in sewing machines as instruments of
financial independence is a qualitative indicator. Note that "quality of life"
indicators, such as those that measure changes in a population's health,
education or employment, are often confused with qualitative indicators,
as they both refer to "quality". In fact, either qualitative or quantitative
indicators can measure health, education or any other subject.
.
Global
standardised
Locally
developed
Impact



Performance indicators "show results relative to what was planned"
(OECD-DAC) at each level of the "results chain" (see next page) —
inputs, processes, outputs, outcomes, impacts.

They are tools for performance-based decisions about programme
strategies and activities. They can also be used later to evaluate
project/programme success.
Outcome
Output
Input
In UNICEF’s work to date, health and child survival programmes can
most often be monitored using standardised global indicators
(comparable in all settings). For example, the procedures for measuring
nutritional status (e.g., weight for age) and immunization coverage have
been standardised by WHO.
The indicators for measuring change in areas such as child protection
have not been standardised as yet. Indicators for many such areas tend
to be context-specific and must be developed locally.
1 CIDA, 1997, “Guide to Gender Sensitive Indicators”.
UNICEF M&E Training Resource
Indicators 1/3
PERFORMANCE INDICATORS
Definitions
Example of hierarchy of
indicators for a health
education campaign
Impact indicators: Long-term results
Measure the quality and quantity of long-term results
generated by programme outputs (e.g. measurable change
in quality of life, reduced incidence of diseases, increased
income for women, reduced mortality).
Reduction in morbidity and mortality
*In the absence of more definite impact indicators, early pointers of impact
may be used during project implementation to indicate progress toward
achieving project objectives. For example, in many cases, output indicators
and indicators of risk factors can serve as suitable intermediate or leading
indicators of impact.
Outcome indicators: Medium-term results
Measure the intermediate results generated by programme
outputs. They often correspond to any change in people’s
behaviour as a result of programme, e.g. the use of oral rehydration therapy.
Proportion of target population that meet
prescribed standards of behaviour
Number whose behaviour change
Output indicators: Short-term results
Number whose opinions change
Measure the quantity, quality, and timeliness of the products
— goods or services — that are the result of an activity/
project/programme.
Number who learn the facts
Process indicators:
Measure the progress of activities in a programme/project
and the way these are carried out (for example, referring to
the degree of participation).
Number who read the literature
Number of people who receive the
literature
Input indicators:
Measure the quantity, quality, and timeliness of resources —
human, financial and material, technological and information
— provided for an activity/project/programme.
Amount of literature distributed
Number of pieces of literature available
for distribution
Source: adapted from Patton 1997: 220.
Limitations of the results chain
The "results chain" is the expected sequence of results to achieve the desired programme objectives,
beginning with inputs, moving through activities and outputs, and culminating in outcomes, impacts and
follow-up (OECD-DAC, 2001). It is generally just a reference for developing performance indicators, as it
obviously a simplification of reality. In practice, it is sometimes difficult to distinguish between a programme’s
outputs and outcomes or between outcomes and impacts. An outcome indicator for a small project may
represent an output for a larger programme. In addition, there are often multiple intermediate results, both for
outputs and outcomes.
When defining indicators, it is important to think of the types of indicators on a continuum in logical relation to
the project's/programme’s strategic objective. Where there are multiple intermediate results, you may end up
with two sequential so-called "output indicators" or "outcome indicators". In other words, it doesn't matter so
much whether you label something an output or an outcome indicator as long as it fits with the logical
sequence of results of the programme in question and fits the characteristics of a good indicator. (See
content sheet "Selecting indicators — Criteria".)
UNICEF M&E Training Resource
Indicators 2/3
Challenges posed by impact indicators
Usually it is easier to define and measure simple output indicators, as impact indicators can be complex,
costly, and difficult to measure. The time lag between programme implementation and impact is significant.
As we move toward impact, we see that numerous contextual factors can influence results. Before-and-after
household and community surveys are excellent tools for collecting comprehensive impact data for ex-post
evaluation, but the time and cost they require make them impractical for regular programme monitoring or
evaluation during programme implementation.
We, therefore, often seek proxy measures that can inform us early on about a programme’s likely impact.
Monitoring performance indicators during implementation can provide this information. Leading process and
intermediate indicators can serve as proxies for impact indicators. Beneficiary assessments, rapid rural
appraisals, and focus group interviews are useful ways of collecting qualitative impact data.
Impact and outcome indicators must be defined and interpreted with care. Demonstrating that a project
caused a particular impact is usually difficult since many factors outside the project influence the results
(DAC). See Core Content Sheet "Evaluating impact" (Module 4).
INDICATORS, TARGETS, BENCHMARKS
An indicator is the measure; for example, the proportion of
under-fives who fall below minus 3 standard deviations from
median weight-for-height of NCHS/WHO reference population.
A target is the explicit statement of desired results for a
specific indicator over a specified period of time; the
planned performance standard against which actual
performance may be subsequently compared and
measured. It is usually specified in terms of quantity or
quality, target area or target group (intended primary
stakeholders), and time. For example: Reduction of
severe and moderate malnutrition by 20 percent by 2002.
A benchmark is also a reference point or standard
against which performance or achievements can be
assessed. The distinction is that a benchmark refers to
the performance that has been achieved in the recent
past by other comparable organizations, or what can
reasonably be inferred to have been achieved in the
circumstances (OECD-DAC, 2001). In the context of
situation monitoring, a benchmark represents a
reference to a standard of what is achievable. For
example, the median weight-for-height of NCHS/WHO
reference population is a benchmark in nutritional status.
Note that some organisations or sources use these terms interchangeably, which can cause confusion.
UNICEF M&E Training Resource
Indicators 3/3
Download