Policy Implications of POINT – Policy Influence of
Indicators, a pan-European research project
March 2010
POINT POLICY BRIEFING 2
REPORTING OF COMPOSITE INDICATORS IN
THE UK MEDIA
Composite Indicators
Composite indicators (CIs) combine information from a number of
separate indicators into one. Designing such indicators entails
unavoidable trade-offs between technical excellence and simplicity
needed to convey the ideas represented by the indicator. CIs are
meant to be picked-up and promoted by non-specialists, including
the mass media. Newspapers in particular have been shown to be
influential with regard to environmental issues such as global
warming, but much of their impact may depend upon the extent as
well as the continuity of reporting. Furthermore, little is known about
the extent to which CIs have been reported in newspapers. As part
of the work package 5 (WP5) of the POINT project, an analysis was
conducted on the extent to which such indicators are indeed pickedup by the popular press in the UK, and the ‘use’ that the press
makes of the indicators.
The analysis focussed on three CIs
environmental and business domains:
spanning
the social,
Social: Human Development Index (HDI) published by the United
Nations Development Programme
Environmental: Ecological Footprint (EF) published by a number of
groups at international, national and local spatial scales
Business: Corruption Perception Index
Transparency International based in Berlin
(CPI)
published
by
2
EUROPEAN POLICY BRIEF
The definition of ‘national’ newspapers is somewhat vague and
equates to a newspaper that is sold throughout a nation, even if
coverage may be patchy. The NEWS UK database was used to
search the electronic editions of a total of 24 newspapers from
January 1st 1991 to the end of October 2009. Some of the
preliminary findings from the analysis are presented below. The first
part of the analysis measured press coverage as the number of
times that an indicator was cited in newspaper articles (figure 1).
Figure 1. The number of articles which mention the indicator at least once
Asterisks represent the level of statistical significance of the regression line: **** represents a
probability of < 0.001 and ** represents a probability of < 0.01.
For both CPI and HDI there is a linear increase in the number of
articles mentioning the indicator. Yet the frequency at which the
indicator is mentioned is higher for HDI than for CPI. The EF has the
largest number of article counts of any of the three indicators,
showing an exponential growth in the number of article counts in the
early 2000s, followed by a steep decline since 2006.
The second phase of analysis went a step further, seeking to
establish the degree of diversity in coverage amongst newspapers.
Diversity was estimated using the Shannon-Wiener Index, with
higher values of the indicator indicating that a broad range of
newspapers report on the indicator (figure 2)1.
1
Further details on the Shannon-Wiener index are available at en.wikipedia.org/wiki/Shannon_index.
Maximum value of the diversity index is reached when all newspapers included in the search mention the
indicator to the same degree while the minimum value occurs when the indicator is mentioned by only one
newspaper.
2
3
EUROPEAN POLICY BRIEF
Figure 2. The diversity of coverage in the newspapers included in the
survey.
The highest diversity of coverage is found with EF and the lowest
with CPI. In fact, the diversity index for CPI remains relatively
constant and results from a concentration of coverage within just
three newspapers; ‘The Times’, ‘The Irish Times’ and the ‘Financial
Times’. For both the HDI and EF there is a statistically significant
increase in diversity of coverage between 1991 and 2009. Thus it
would appear that the coverage of the HDI and EF has widened
amongst the newspapers over time.
Preliminary findings:
messages to EU indicator
developers
All three indicators have been successful in reaching their intended
objective of high and increasing coverage in the UK national press.
However, the pattern of reporting varies among the indicators. The
HDI is reported across a wider range of newspapers and there is
evidence of a steady increase with time. The evidence concerning
the reporting on the HDI, published annually by UNDP as part of its
Human Development Reports, suggests that both the degree of
reporting and diversity amongst titles has risen steadily between
1991 and 2009.
The EF has the most interesting trend, since the steady increase in
the number of articles in the early 2000s turned into a decline in
2006. Why might the interest in the indicators have fallen off? One
answer may rest with the timing of the EF reports. The release of a
major report in 2005 by the organisation ‘Redefining Progress’ would
certainly explain the surge of reporting in 2006, which gradually
tailed off during the two subsequent years. It is likely that the highprofile release of the EF report in 2005 heightened awareness of the
indicator amongst journalists and contributors which continued to
echo (albeit fading) through 2007 and 2008. The CPI is an example
of a more specialist indicator which is reported by relatively few
newspapers – essentially those with a strong ‘business’ focus such
as the ‘Financial Times’, ‘The Times’ and the ‘Irish Times’.
The HDI and
represent two
and CPI could
while the EF
CPI on the one hand, and the EF on the other,
quite different models of indicator release. The HDI
be described as examples of a ‘Steady Impact’ model
reflects more of a ‘Big Bang’ model. The findings
3
4
EUROPEAN POLICY BRIEF
concerning the EF have some resonance with the findings by
Sampei and Aoyagi-Usui (2009), who observed that reporting on
global warming in Japan was often linked to a major conference or
policy change, and followed a transitory pattern rather than being
sustained over time. However, it is perhaps all too easy to assume
that the ‘steady impact’ model is the better of the two, notably
because at its peak (2006 and 2007) the EF was mentioned in many
more articles than the HDI and CPI. Work in WP5 is ongoing to
explore this pattern in more detail, notably by analysing the content
of the articles. Interviews are also underway with senior
policymakers in the European Commission, newspaper journalists,
and representatives of pressure groups to determine how these
individuals are influenced by and indeed influence CIs.
Reference
Sampei Y and Aoyagi-Usui M (2009). Mass-media coverage, its
influence on public awareness of climate-change issues, and
implications for Japan's national campaign to reduce greenhouse
gas emissions. Global Environmental Change-Human and Policy
Dimensions 19(2): 203-212.
Some useful website links
Human Development Index
Corruption Perception Index
Redefining Progress
News UK
http://hdr.undp.org/en
http://www.transparency.org
http://www.rprogress.org/index.htm
http://www.newsuk.co.uk/password
GENERAL POINT PROGRESS
POINT is progressing as
planned..
The POINT project reached its mid-point with the end of the first
reporting period on 30th September 2009. The period was
characterised by mainly theoretical and methodological work, which
resulted in the finalisation of an analytical framework for POINT.
Empirical data collection was initiated within the different WPs
focusing on sustainability issues, sector integration and composite
indicators. Four interactive stakeholder workshops were held, with
the first POINT Policy Briefing reporting some of the findings.
A recent project meeting, held in Malta on 19-20 November 2009,
showed good progress in the WPs. The empirical studies in the work
packages will continue until the late spring 2010, when the emphasis
will shift towards comparing results across different case studies,
and the possible modifications that these results would imply for the
POINT analytical framework. These topics will be discussed in a
meeting with the Advisory Panel in September. POINT partners have
presented their work at several international conferences and
workshops, and attended a number of international meetings.
Conference attendances included the 1st ISA Forum (International
Sociological Association) in Barcelona, 5-8 September 2008, the 9th
Nordic Environmental Social Sciences conference (NESS) in
London, 10-12 June 2009, and the 3rd OECD World Forum on
“Measuring the Progress of Societies” in Busan, Korea, 27-30
4
5
EUROPEAN POLICY BRIEF
October 2009.
Planned meetings:
16 March 2010: WP3 meeting in Paris
10-11 June, 5th project meeting, in Copenhagen
14-15 September: 2nd meeting with the advisory panel, followed by
project meeting
ABOUT POINT
The Project..
POINT is an EU FP7-funded research project, which examines
the role of indicators in policymaking. It does this by examining
whether, how and under which circumstances indicators
influence policymaking, and by whom they are used and for
which purposes. It designs a coherent framework of analysis
and tests this in a number of case studies in three areas:
composite sustainability indicators, indicators of sectoral
integration, and indicators to monitor the implementation of
national and EU sustainable development strategies. The case
studies seek to identify the multiple direct and indirect,
intended and unintended impacts of indicators, as well as to
identify the factors conditioning indicator influence. The
ultimate aim is to recommend ways to enhance the likelihood
that indicators indeed foster sustainable development.
5