The Use and Abuse of Indicators
Joachim Nahem
UNDP Oslo Governance Centre
World Forum on Statistics, Knowledge and Policy:
Istanbul 28-6-2007
"Economists have come to feel what can’t be
measured, isn’t real. The truth is always an
amount. Count numbers, only numbers count.“
“Not everything that can be counted counts,
and not everything that counts can be
counted.”
-Albert Einstein
Who Uses Governance Indicators and
for What Purposes?
Plethora of governance indicators used by:
–
–
–
–
–
–
governments
development agencies
civil society
media
academic institutions
private sector
Indicators used for:
–
–
–
–
–
policy decisions
aid allocation
advocacy
research
business investment
Governance Indicators
A Users’ Guide (2nd Edition 2006)
• Provides guidance on governance
indicators; What are they? What are they
used for? How to use and not to use
indicators
• Intended for non-specialist users
• Contains 35 sources of inter-country
governance indicators
Users Guide:
‘The Normative Assumption’
• Every indicator will have an underlying
normative assumption.
• Ex. Number of persons detained without
charge – The assumption is that a lower
number is better.
• Ex. Perceived state of corruption-The
assumption is that less corruption is better.
Governance Indicators
A Users’ Guide
Golden Rule 1: Use a range of indicators
• The single governance indicator which captures the subtleties
and intricacies of national situations, in a manner which
enables global, non-value laden comparison does not exist.
• Using just one indicator could very easily produce perverse
assessments of any country and will rarely reflect the full
situation.
• …at the other extreme, having too many indicators results in a
different range of problems, including a lack of focus and
burdensome data collection and analysis. The key is a
balanced set with sufficient but not superfluous information.
Golden Rule 1: Use a Range of
Indicators
Example : Voter turnout used as an indicator of democracy
• Voter turnout figure – 85%
• Clarify the definition: voter turnout is the number of those
casting votes as a proportion of those eligible. Who are
eligible?
• Who were the 15%
• Who is not eligible?
• Who did vote?
• Was there a choice not to vote?
• What are the barriers to eligibility?
• Are there any elements of compulsion?
Golden Rule 2: Use an Indicator as a first
question-not a last
• Often indicators can be developed step by step as more
information becomes available. In using an indicator one
might go through the following steps.
• Yes/No – Asking the basic question, does a problem exist?
• Number – After determining that a problem exists,
determining the size of the problem.
• Percentage – Put the problem into context.
• Significance – Use a significance test to examine whether the
problem is evenly spread or certain groups face more of a
problem.
Golden Rule 3: Understand an Indicator
Before You Use it
• By using an indicator you can be considered to be implicitly
endorsing it, including its methodology and underlying values
(normative assumptions)
Example: Voter Turnout - how is this defined?
• It could either be votes cast as a percentage of voters
registered, or votes cast as a percentage of voting age
population. Are there any other assessments which are
included in the data – for example does the election have to be
considered ‘free and fair’ before the data is included in the
publication. How, who and where is the data compiled?
UNDP Project
Inter-country sources inadequate for governance and
poverty reduction at national level
•
•
•
•
Need for governance indicators that are:
poverty and gender focused
policy-action oriented
nationally owned and participatory
methodologically sound