Michaela Saisana
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
“Reflections on the Human
Development Index”
(paper by J. Foster)
Additional Considerations
Michaela Saisana
michaela.saisana@jrc.ec.europa.eu
European Commission
Joint Research Centre
Econometrics and Applied Statistics Unit
1
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
250,000
21,300
~5-fold increase
since18,600
2000
20,000
20,000
21,300
200,000
18,600
17,800
16,200
15,000
15,000
12,600
115,000
10,000
7,730
7,730
5,000
1985
“Yet the dimensions of the HDI do not
easily meld into one. And without a
systematic method […prices…] the index
could prove difficult to explain and defend”
(J. Foster, 2013)
100,000
77,100
4,460
5,000
0
150,000
12,600
10,000
50,000
4,460
0
1985
180
1990
180
1990
1995
1995
2000
2000
2005
Year 2010
2005
Scholar Google hits on "Gross
domestic product"
Achievements
The challenge
The measure
Popularity
Scholar Google hits on
"Human development index"
•
•
•
•
Scholar Google hits on
"Human development index"
Introduction
25,00025,000
2010
0
2015
2015
Year
It is exactly the “unobserved” nature of
composite indicators that is their main
limitation and their raison d'être.
2
Michaela Saisana
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Main points
•
•
•
•
Calibration
Goalposts
Gaterories
Cobb-Douglas HDI
3
Michaela Saisana
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Main points
•
•
•
•
•
Calibration
Goalposts
Categories
Aggregation
New HDI
“Frequent recalibration gives the strong suggestion that HDI values are
contingent and temporary and depend importantly on arbitrary constructs”
Foster’s suggestion:
1) ~ 10 year recalibration (as for poverty)
2) Crossover between calibration periods:
process outlined explicitly and transparently
Source: Global Innovation Index
4
Michaela Saisana
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Source: Global Innovation Index
5
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
Main points
•
•
•
•
“The HDI is typically cast and interpreted as a multidimensional measure of
size and hence is seen to be an absolute measure. […] Yet in actual
implementation, this is not necessarily the way the HDI behaves.”
Calibration
Goalposts
Categories
Cobb-Douglas HDI
Source: Wikipedia
Life expectancy at birth
Bounds in the HDI
After 2010: 20y – observed (83.2 y, JN)
Before : 25y – 85y
6
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
Main points
•
•
•
•
Calibration
Goalposts
Categories
Cobb-Douglas HDI
Life expectancy at birth
Suggestion: Fixed bounds
30y (Early 20th Century) – 87 years
Life expectancy at birth (years)
Minimum and Maximum across 194 countries
90
85
80
75
70
65
60
55
50
45
40
35
30
1970
76.7
82.3
79.0
83.4
85.6
47.8
44.0
39.8
34.6
1980
32.8
1990
2000
2010
2020
Similarly for the other indicators
7
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
Categories of Human Development
Main points
•
•
•
•
Calibration
Goalposts
Categories
Cobb-Douglas HDI
Relative (since 2010) versus Absolute (before 2010)
+ progress against other countries, rather than arbitrary numerical
cutoffs whose meaning may vary with each new calibration.
- fuzzy incentives, less practical value for the country
- many factors enter into the determination of progress (e.g. different
calibrations, performance of other countries, policies of the country,
or inclusion of new countries).
- a country can not set a meaningful numerical target to achieve over
time.
Foster’s suggestion:
1) A staggered recalibration schedule &
2) Fixed numerical cutoffs for the four HD categories
(e.g. WB grouping by income)
8
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
Further recommendation:
Main points
•
•
•
•
Calibration
Goalposts
Categories
Cobb-Douglas HDI
HDI
To present the fixed cutoffs for the HDI
with respect to the raw data (assuming an
even performance)
Life
Mean
Expected
GNI per
expectancy at
years of
years of
capita
birth (years)
schooling
schooling
(PPP$)
0.6
58.2
7.9
10.8
6,487
0.8
…
…
9
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
Main points
“[…] attempt to view the HDI more as a social evaluation function that
aggregates across dimensional variables directly”
•Calibration
•Goalposts
•Categories
• Cobb-Douglas HDI
W  L E
1/ 3
1/ 3
Y
1/ 3
L= life expectancy - 20 years
E =1/2 (mean years of schooling + expected years of schooling)
Y= ln (GNI per capita) – ln (100)
H  W /W
*
W*= target social evaluation level
10
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
More on the geometric mean in the case of the HDI…
HDI
HDI 2011
Liberia’s
(arithmetic)
(geometric)
improvement
Life
Edu
GNI stdev
Mali
.496
.270
.346
.115
.371 (176)
.359 (175)
Liberia
.580
.439
.140
.225
.386 (175)
.329 (182)
Option A
.680
.439
.140
.419
.347
5.5%
Option B
.580
.439
.240
.419
.394
19.8%
Advantages of the geometric mean versus the arithmetic mean for the HDI
1) implies only partial compensability, i.e. poor performance in one HD dimension cannot be fully
compensated by good performance in another,
2) rewards balance by penalizing uneven performance between dimensions,
3) encourages improvements in the weak dimensions, i.e. the lower the performance in a particular
HD dimension, the more urgent it becomes to improve in that dimension.
11
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
More on the “quality” of the HDI… (Implicit Weights)
We suggest to use as a measure of importance
of a variable in an index what is known as:
‐ Pearson’s correlation ratio
HDI
Life Expectancy
‐ First order effect
‐ Top marginal variance
- Main effect
…
𝑉𝑋𝑖 (𝐸𝑋~𝑖 𝑌 𝑋𝑖 )
𝑆𝑖 =
𝑉(𝑌)
Using these points we can compute a statistics that tells us:
 How much (on average) would the variance of the HDI scores be reduced if one could
fix “Life expectancy”?
Source: Paruolo, Saisana, Saltelli, 2013, J.Royal Stat. Society A
12
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
More on the “quality” of the HDI… (Implicit Weights)
𝑉𝑋𝑖 (𝐸𝑋~𝑖 𝑌 𝑋𝑖 )
𝑆𝑖 =
𝑉(𝑌)
HDI
HDI 2011
Life Expectancy
Nominal
Implicit
Weights (wi)
Weights (Si)
Life expectancy
.333
.83 [.81 .85]
Education
.333
.88 [.83 .87]
GNI
.333
.90 [.88 .91]
We could reduce the variation of the
HDI scores by 83% by fixing ‘Life
expectancy”.
Quality check:
The HDI is balanced in its three underlying dimensions (Si values are very similar)
13
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
More on the “quality” of the HDI… (Marginal weights)
Marginal Weights=
Recommendation: To plot life expectancy
instead to evidence that countries with low
life expectancy are more encouraged to
improve
14
Michaela Saisana
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Some recent criticism…
Tradeoffs = marginal rate of
substitution, i.e. how much of one
dimension must be given up for an
extra unit of another, keeping the
index constant.
Previous HDI
The new HDI has devalued
longevity, especially in poor
countries.
Source: M. Ravallion (2012) Troubling
tradeoffs in the HDI, J. Dev. Economics,
99:201-209
15
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Michaela Saisana
Final considerations
 Simply take the log of GNI just once (now logged twice)
 Take the arithmetic average the two education indicators (now
geometric)
 Use two indicators per dimension (now only in case of education)
 Use the generalized mean of the three dimensions (a compromise
solution between arithmetic-geometric averaging)


 1/ 
HDI  ( L  E  Y )
  0.5
  0, geometricmean
  1, arithmeticmean
16
Michaela Saisana
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
Assess any new calibration formula in terms of:
 Implicit weights (reduction in the HDI variance by fixing one
dimension at a time)
 Marginal weights (impact on HDI of 1% increase in one of the
dimensions)
 Marginal rate of substitution (how much of one component must
be given up for an extra unit of another, keeping the index
constant)
More reading at:
http://composite-indicators.jrc.ec.europa.eu
(first Google hit on “composite indicators” over the last 10 years!)
17
Michaela Saisana
Second Conference on Measuring Human Progress
New York, 4-5 March 2013
References and Related Reading
1.
Paruolo P., Saisana M., Saltelli A., 2013, Ratings and Rankings: voodoo or science?. J
Royal Statistical Society A 176(2).
2.
Saisana M., Saltelli A., 2012, JRC audit on the 2012 WJP Rule of Law Index, In Agrast, M.,
Botero, J., Martinez, J., Ponce, A., & Pratt, C. WJP Rule of Law Index® 2012.
Washington, D.C.: The World Justice Project.
3.
Saisana M., Philippas D., 2012, Sustainable Society Index (SSI): Taking societies’ pulse along
social, environmental and economic issues, EUR 25578, Joint Research Centre, Publications
Office of the European Union, Italy.
4.
Saisana M., D’Hombres B., Saltelli A., 2011, Rickety Numbers: Volatility of university
rankings and policy implications. Research Policy 40, 165–177.
5.
Saisana M., Saltelli A., Tarantola S., 2005, Uncertainty and sensitivity analysis
techniques as tools for the analysis and validation of composite indicators. J Royal
Statistical Society A 168(2), 307-323.
6.
OECD/JRC, 2008, Handbook on Constructing Composite Indicators. Methodology and user
Guide, OECD Publishing, ISBN 978-92-64-04345-9.
18