What makes
indicators
successful?
Lessons from
practitioners
Funded by:
The Project
BRAINPOoL (Bringing alternative indicators into policy) is an
EU-funded project aimed at identifying and overcoming the
barriers to ‘Beyond GDP’ indicators being used in policy.
During the project we are carrying out research and
interviews, conducting workshops and knowledge-brokerage
seminars and carrying out various action research case
studies to explore ways to improve uptake of Beyond GDP
indicators.
The Project (work package
structure)
This
presentation
Understanding the supply of
Beyond GDP indicators
• Work Package 1 outcomes:
This
presentation
– Catalogue of over 100 Beyond GDP indicator initiatives
– Understanding of intentions of indicator
producers/promoters
– Documenting of impact, including media impact
– Fact sheets for 16 indicator initiatives
– Understanding of success factors for Beyond GDP
indicators.
Report available at: www.brainpoolproject.eu/research
The initiatives studied
Domestic Material
Consumption
Happy Life Years
OECD Handbook of
Subjective Well-Being
UN Commission for
Sustainable Development
Success factors
Policy / Context
factors
Indicator
User
Relationship
factors
Indicator factors
Salience
Legitimacy
Credibility
User
factors
From the perspectives of:
The public
Policy-makers
Scientific community
The factors that determine
the extent to which an
indicator is used by users
cluster into four categories
(the light blue spheres).
Of these, indicator factors
need to be understand from
three different perspectives.
Indicator Factors –
salience for policy
makers
aenimation
Indicator Factors – salience for
policy makers
• Fit with a vision or organisational strategy – this is
particularly relevant for those initiatives promoting new
indicators so as to shift priorities or assess progress
differently.
• Measure things that can be influenced by policy – this can
be problematic for alternative indicators seeking to measure
overarching concepts such as progress or well-being.
• Low-cost or money-saving – e.g. minimal expensive data
collection, providing clues for low cost policies or ways to
save money.
Indicator Factors – salience for
policy makers
• Links with other outcomes– links between what the indicator
measures and other outcomes (e.g. subjective well-being
being related to reducing staff turnover).
• Reaching multiple audiences – this can ensure indicators
do not sit within particular silos and can achieve crosscutting outcomes.
• Perceived need – this is particularly important where
initiatives are bringing together data rather than creating
new measures.
Indicator Factors – salience for
public/broad audience
Knivesout
Indicator Factors – salience for
public/broad audience
• Simplicity – initiatives are effective when they allow one to
produce a simple and attractive message.
• Ease of understanding – while what they measure may be
complicated, successful indicators manage to illustrate a
complex reality using understandable concepts.
• Engagement with communications experts – close
collaboration, rather than simply handing over data, can
ensure that both communicability and accuracy are
maintained.
• Avoiding taboo words – in the UK ‘happiness’ is considered
‘woolly’ or ‘unscientific’, while in the USA, practitioners have
avoided the mention of ‘climate change’ instead referring to
‘air quality’.
Indicator Factors – Credibility
GuySie
Indicator Factors – Credibility
• Data quality – a particular concern was whether subjective
well-being data changes over time.
• Concerns regarding composite indicators – concerns over
methodology and the weighting of different components of
composite indicators can elicit strong resistance. The
OECD’s Better Life Initiative has managed to secure
acceptance of a composite indicator by allowing users to
decide for themselves how to weight the different
dimensions of the measure
Indicator Factors – Legitimacy
United Nations Photo
Indicator Factors – Legitimacy
• Being (or appearing) neutral – some indicator initiatives
work within a framework of simply providing ‘neutral’
information, while others are clearly connected to political
agendas, such as social cohesion or respecting
environmental limits. Mechanisms used to ensure neutrality
included monitoring funding mix and barring staff
involvement in political parties.
• Institutional power – governmental bodies or supragovernmental bodies like the Council of Europe and the
OECD often carry greater legitimacy than NGOs.
Relationship
Relationshipand
andprocess
processfactors
factors
Rabanito
Relationship and process factors
• Engage one’s audience from the start – fundamental to the
success of local initiatives, it was also seen in terms of
getting policy-makers involved in large-scale initiatives.
• Direct contact with audiences – while not all initiatives can
or want to engage their audience from the beginning, all the
most successful initiatives had direct contact with the
people they were trying to influence.
• Small is beautiful – to date, local initiatives have been able
to achieve more impact than larger/national ones, with local
bodies tending to be more ‘flexible’ and responsive.
Relationship and process factors
• Partnership working – aside from allowing a greater network
to be reached and a greater skill base to be marshalled,
partnerships allow different organisations to take on
different roles. This can ensure an initiative is not too
associated with a particular agenda.
• Picking one’s audience – On the one hand, some initiatives
worked with individuals within organisations who could be
seen as ‘allies’, or organisations who are overall supportive.
On the other hand, several initiatives highlighted the need
to reach those bodies potentially least sympathetic to their
initiative – ministries of finance, treasuries or economic
departments.
User factors
AEN Foto
User factors
• Users’ capacity to use social and environmental indicators –
Beyond GDP initiatives typically involve a rebalancing
towards social and environmental indicators and away from
economic ones. This is not just a matter of calculating
different things but of grasping different disciplines, and
valuing different academic perspectives.
• The OECD’s approach of using economic techniques with
subjective well-being may be a fruitful technique for
convincing economists, by using their own language.
Policy and context factors
Ryan Fitton
Policy and context factors
• The Stiglitz/Sen/Fitoussi Commision – Seen as the biggest
positive factor.
• The economic crisis - On the contrary, the financial crisis is
seen by many as hindering Beyond GDP efforts, leading
people to view well-being as a distraction. Others, however,
see an opportunity in highlighting the role of the fixation on
GDP in causing the crisis.
Policy and context factors
• Ideology as a barrier – For example, subjective well-being
has been criticised from a libertarian perspective as not
being something government should influence.
• Vested interests – might Beyond GDP efforts have a
negative effect for certain groups?
• Public pressure – support for the idea of alternative
indicators required from the bottom up.
• Indicator initiatives take time – a last sobering lesson is that
it can take generations for an indicator to become
sufficiently embedded in the system to maximise its impact.
For the full report, visit:
www.brainpoolproject.eu/research
For more information please contact:
Alistair Whitby, World Future Council
alistair.whitby@worldfuturecouncil.org
Saamah Abdallah, nef (the new economics foundation)
saamah.abdallah@neweconomics.org
Tomas Hak, Charles University Environment Centre
tomas.hak@czp.cuni.cz
James Jordan