The performance
of the public sector
Pierre Pestieau
CREPP, University of Liège,
CORE, PSE and CEPR
Outline
1. Introduction
2. The performance approach and the
concept of best practice
3. Measuring productive efficiency
4. The performance of social protection
5. Conclusion
2
1. Introduction

Measuring and ranking: a must




Important distinction between the public sector as a whole and
its components



People do it anyway but badly
Transparency and governance
Yardstick competition – Open Method of Coordination (OMC)
Problem of aggregation
Technical link between outcomes (outputs) and resources (inputs)
The performance is to be measured by the extent to which the
preassigned objectives are achieved.
3
2. The performance approach and the
concept of best practice




The public sector is a set of more or less aggregated production
units (social security administration, railways, health care,
education, national defence, social protection,…)
Each unit is supposed to use a number of resources, within a
particular setting, to produce a number of outputs
Those outputs are related to the objectives that have been
assigned to the production unit by the principal, the authority in
charge
Approach used here: productive efficiency and to measure it, the
efficiency frontier technique is going to be used
4

Productive efficiency is just a part of an overall
performance analysis. It has two advantages:



It can be measured
It is a necessary condition for any other type of objectives
Main drawback: it is relative


Based on a comparison among a number of rather similar
production units
Its quality depends on the quality of the observation units.
5
Illustration with one input/one output


Set of observations
Best practice frontier



Non parametric method: DEA (data envelopment
analysis)
Parametric method
Comparative advantage
6
Set of comparable observations
output
input
Figure 1
7
Parametric
output
input
Figure 2
8
Non Parametric
output
input
Figure 3
9
t+1
output
b
g
b
c
t
a
a
A
B
input
Figure 4
10
Technical progress: ga
Efficiency in t:
aA
aA
bB
in t + 1:
bB
Change in efficiency: ca - ga
11
 Motivation of efficiency study: performance
improvement
 Factors of inefficiency:
 Exogenous (location)
 Endogenous (low effort)
 Policy related (ownership, competition)
12
3. Measuring productive efficiency.
Conceptual and data problems
Two problems.


Weak link between the inputs used and the expected
outcomes
Confusion between lack of data and conceptual
difficulties
Research strategy. Two areas quite typical of public
spending: education and railways transports; how
performance should be measured if data availability were
not a constraint?
More precisely, when listing the outputs and the inputs,
assume that the best evidence one can dream of is
available.
13
3.1. The best evidence
Inter-country comparison.
Importance of institutional, political and
geographical factors.
14
Railways
Ideal data
Outputs
Passenger kilometres
Comfort and punctuality
Freight tons and kilometers
- bulk
- containers
- others
Delivery quality and punctuality
Equity of access
Passengers per seat
Inputs
Labor (disaggregated)
Equipment (disaggregated by type and by
quality)
Tracks (length and quality)
Energy (sources)
Environment Geography, stage length
Autonomy
Competition or contestability
Price discrimination
Community service obligation
Observations Very large number of years and countries
15
High schools
Ideal data
Output
Acquired skills (of sample of 18y. old
individuals)
- math, science, reading
- foreign languages
Direct employability
Indirect employability (through college)
Happiness
Contribution to R and D
Input
Teachers (level and quality)
Staff
Building, equipment
Spatial distribution of schools
Skills at the end of the primary education level
Environment
Competition between networks
Competition with private schools
Role of the family
Unemployment rate, economic growth
Pedagogical technique
Observations Large number of countries and years
16
3.2. Actual studies
Most qualitative variables are missing.
Difference between developed and less developed
countries.
Focus on financial variables.
17
Railways
Ideal data
Data used
in recent studies
Outputs
Passenger kilometres
Comfort and punctuality
Freight tons and kilometers
- bulk
- containers
- others
Delivery quality and punctuality
Equity of access
Passengers per seat
v
~
v
~
~
~
~
–
~
Inputs
Labor (disaggregated)
Equipment (disaggregated by type and by
quality)
Tracks (length and quality)
Energy (sources)
v
v
~
~
Geography, stage length
Autonomy
Competition or contestability
Price discrimination
Community service obligation
~
~
~
~
~
Environment
Observations Very large number of years and countries
Too small
Note: v = OK; ~ = more or less; – = unavailable
18
Education
Ideal data
Output
Recent studies
Acquired skills (of sample of 18y. old
individuals)
- math, science, reading
- foreign languages
Direct employability
Indirect employability (through college)
Happiness
Contribution to R and D
v
–
–
~
–
~
Input
Teachers (level and quality)
Staff
Building, equipment
Spatial distribution of schools
Skills at the end of the primary education level
~
~
v
–
–
Environment
Competition between networks
Competition with private schools
Role of the family
Unemployment rate, economic growth
Pedagogical technique
~
–
~
~
Large number of countries and years
~
Observations
Note: v = OK; ~ = more or less; – = unavailable
19
Productive efficiency comparative studies of public and private firms
Sector and
authors
Number of
outputs and
inputs
Method
Mean
efficiency
degrees
Number of units
Type and
period of data
Remarks and other
findings
64 public and
57 private
universities in US
Panel annual
1971, 1974,
1981
5 outputs
5 inputs
Non
parametric
About 88% a
year
- Private universities have
slightly hither efficiency
scores, for everyyear
considered
21 railways
companies
Annual data
1 output
1978-1988
Parametric
1 each year
- Limited evidence has been
found for a relationship
between the share of state
in capital and cost efficiency
- Positive correlation
appears between cost
efficiency and the
importance of the cantons’s
participation in the deficit of
firms
57 railways under
mixed ownership
Annual data
1985-1988
1output
3 inputs
+2 network
characteristics
Non
parametric
81%
- Tendered services have
higher efficiency scores that
non-tendered ones.
Education
Rhodes and
South-wick
(1988)
Railways
Oum & Yu
(1991)
Filippini &
Maggi (1991)
20
Is it worth the amount of time?
Yes, but with caution
 Technical efficiency is just one aspect of
efficiency.
 Lack of quantitative variables may distort the
results.
 For education importance of employability.
21
4. Measuring the performance of
the public sector as a whole
 Ideally:

Data on happiness (average and distribution) with
and without social protection or at least on how
the welfare state fulfils its objectives: health,
education, employment, poverty alleviation,
inequality reduction;

Data on inputs.
22
 Actually:
 Data
on indicators of social inclusion (or exclusion);
 Data on social spending.
23
 Three issues:
 Aggregation:
 Scaling:
 Use
DEA or SPI,
(0,1) or average or goalposts,
of inputs: performance versus inefficiency.
24
Table 1: Indicators of exclusion. Definition and correlations
Definition
POV
At-risk-of-poverty rate
INE
Inequality
UNE
Long term unemployed
EDU
Early school leavers
EXP
Life expectancy
Correlation
POV
INE
UNE
EDU
POV
1.000
INE
0.912
1.000
UNE
0.420
0.409
1.000
EDU
0.668
0.782
0.252
1.000
EXP
-0.069
-0.098
0.084
-0.203
Source: The five indicators are taken from the Eurostat database on Laeken indicators (2007).
EXP
1.000
25
Table 2: HDI normalization and SPI1 - 2004
AT
BE
DE
DK
ES
FI
FR
GR
IE
IT
LU
NL
PT
SE
UK
Mean
POV
0.80
0.60
0.50
1.00
0.10
1.00
0.70
0.10
0.00
0.20
1.00
0.90
0.00
1.00
0.30
0.55
INE
0.87
0.82
0.72
0.97
0.54
0.95
0.77
0.31
0.56
0.41
0.90
0.82
0.00
1.00
0.49
0.68
UNE
0.93
0.33
0.04
0.96
0.48
0.76
0.37
0.00
0.87
0.35
0.98
0.87
0.57
0.96
1.00
0.63
EDU
0.99
0.89
0.88
1.00
0.25
0.99
0.82
0.79
0.86
0.55
0.86
0.82
0.00
1.00
0.79
0.77
EXP
0.57
0.53
0.58
0.07
0.91
0.00
0.87
0.51
0.35
1.00
0.35
0.54
0.00
0.90
0.47
0.51
SPI1
0.83
0.63
0.54
0.80
0.46
0.74
0.70
0.34
0.53
0.50
0.82
0.79
0.11
0.97
0.61
0.63
Rank
2
8
10
4
13
6
7
14
11
12
3
5
15
1
9
26
SPI1 and SPI2
Difference in shadow prices
SPI1
SPI2
POV
-0.02
-0.03
INE
-0.05
-0.04
UNE
-0.04
-0.05
EDU
-0.006
-0.010
EXP
0.06
-0.003
Correlation: 0.9
Dependent on irrelevant alternatives.
27
DEA with same input:
- DEA1: 0.921
- DEA2: 0.990
DEA is not invariant to non linear transformation.
- DEA3: 0.992
28
Figure 1: DEA1 frontier
q1
A
D*
B
E*
D
E
F*
F
0
C
q2
29
Table 3: DEA efficiency scores. 2004
AT
BE
DE
DK
ES
FI
FR
GR
IE
IT
LU
NL
PT
SE
UK
DEA1
Scores
0.995
0.892
0.886
1.000
0.939
1.000
0.937
0.795
0.900
1.000
1.000
0.900
0.565
1.000
1.000
Mean
0.921
rank
7
12
13
1
8
1
9
14
10
1
1
10
15
1
1
DEA2
Scores
0.988
0.983
0.984
1.000
0.997
1.000
0.997
0.981
0.976
1.000
1.000
0.984
0.959
1.000
1.000
0.990
rank
9
12
10
1
7
1
7
13
14
1
1
10
15
1
1
DEA3
Scores
0.999
0.972
0.975
1.000
0.996
1.000
0.995
0.969
0.995
1.000
1.000
0.995
0.980
1.000
1.000
rank
7
14
13
1
8
1
9
15
10
1
1
10
12
1
1
0.992
Note: DEA1, DEA2 and DEA3 results correspond to HDI, Afonso et al. and “goalspot” normalization data respectively.
30
Table 4: Correlations between indexes
31
Measuring performance or efficiency


Problem: weak link between social spending
and education, health, unemployment.
Ranking modified
32
Table 5
DEA efficiency scores without and with social expenditures as input. 2004
AT
BE
DE
DK
ES
FI
FR
GR
IE
IT
LU
NL
PT
SE
UK
DEA1
Scores
0.995
0.892
0.886
1.000
0.939
1.000
0.937
0.795
0.900
1.000
1.000
0.900
0.565
1.000
1.000
Mean
0.921
rank
7
12
13
1
8
1
9
14
10
1
1
10
15
1
1
AT
BE
DE
DK
ES
FI
FR
GR
IE
IT
LU
NL
PT
SE
UK
Mean
DEA1
Scores rank
0.917
8
0.809
12
0.769
13
0.824
11
1.000
1
0.943
6
0.924
7
0.752
14
1.000
1
0.988
5
1.000
1
0.864
9
0.444
15
1.000
1
0.825
10
0.871
33
Race to the bottom?


Test of convergence
SPI1 and Malmquist decomposition
34
Figure 6: Convergence of SPI1
9%
y = -1.2741x + 1.0326
Growth rate of SPI1 (1995-2004)
8%
R 2 = 0.8024
ES
7%
PT
IE
IT
6%
UK
5%
4%
GR
3%
LU
BE
FR
2%
DE
AT
NL
1%
DK
FI
SE
0%
-1%
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1,0
SPI1 - 1995
35
Figure 7: Convergence of DEA1 according to “technical efficiency” change
5%
y = -0.0862x + 0.0853
Average Effciciency change 1995-2004
IT
ES
R 2 = 0.9468
4%
3%
GR
UK
2%
BE
FR
1%
DE
NL
IE
0%
FI
SE
PT
LU
AT
DK
-1%
0,4
0,5
0,6
0,7
0,8
0,9
1,0
1,1
DEA1 1995
36
5. Conclusion

Yes for efficiency measures when the
production technology is well understood.

Caution when the technology is unclear and
environmental variables are missing.

For the welfare states, ranking performance is
preferable.

DEA is to be preferred over SPI.

No clear guidelines on the choice of scaling.
37