Measuring Progress and Efficiency
in Public Administration
(Government at a Glance)
Oscar Huerta-Melchor
OECD - Public Governance and Territorial
Development Directorate (GOV)
Tukums, Latvia, 28 August 2008
Presentation outline
1. Describing Government at a Glance (GaaG)
2. Measuring Efficiency in the Public Sector
3. Building an HRM Indicator – delegation of authority
2
1. Describing Government at a Glance (GaaG)
The OECD GaaG project aims to provide comparative data in
order to:
a) enable countries to better understand their own practices;
b) benchmark their own achievements through international
comparisons;
c) learn from the experiences of other countries facing similar
challenges.
3
…. and in the longer term
 Contribute to OECD-wide learning about public sector
efficiency and institutional effectiveness.
 Explore possible causes of performance differences
among governments.
 Begin to identify the impact of public sector reforms.
4
GaaG will contain data on:
Antecedents
Revenue
Inputs
Process
What is the
context in
which
government
operates?
How much
money does
the public
sector
collect?
How much
and what
kind of
resources
does the
public
sector use?
What does
the public
sector do
and how
does it do
it?
Outputs
Outcomes
What are
the goods
and services
which the
public
sector
produces?
What is the
resulting
impact on
the
country?
Framework for understanding and measuring government’s activities
5
a) Antecedents
The political institutions and administrative structures provide the context
that dictates public sector efficiency and effectiveness.
b) Revenue data
It reflects the redistributive power of government and the degree of
‘excludability’.
c) Input data
- Expenditures:
Expenditure structure
- Employment data:
Labour as input for the production of public
goods and services.
- Compensation costs:
The costs of public employment in order to
gain proper understanding of production costs.
- Production costs:
Composition of employees, direct production
cost, indirect production cost. 6
d) Data on processes / institutional arrangements
- Budget practices:
An annual budget law conditions
implementation of political decisions.
- Performance
measurement:
Governments are under pressure to improve
efficiency and effectiveness while controlling
public expenditure.
- HRM practices:
Efficiency in the management of human
resources largely conditions the effectiveness
of governmental policies.
- Regulatory quality
management:
It underpins government’s ability to ensure
that regulations are efficient, effective and of
good quality.
7
the
- Integrity framework:
Integrity is critical for achieving good
governance. However, existing indicators are
mainly based on perceptions and do little to
track changes over time.
- e-Government:
Public sector transformation and egovernment are increasingly seen as closely
linked policy areas.
e) Output data
Outputs and outcomes of public administration are concerned with the
machinery of government. Defining outputs for measurement is a complex
matter as what is an output for one sector is an input for other one.
Therefore it is necessary to ask the question: outputs for whom?
f) Outcome data
Executive governance indicators: trust in government, citizens satisfaction,
equity and fiscal/economic stability.
8
The outcome at one level is an input at a higher level
9
Outcome as the starting point
10
2. Measuring Efficiency in the Public Sector
a) Why measure government?
 The size and economic significance of the public sector make it a
major contributor to growth and social welfare.
 The government is a major economic actor in modern society. It
provides goods and services in kind and it redistributes income
through taxation and transfers.
 The government also provides non-economic and regulatory services
(less tangible outcomes).
 The way of measuring the output of government strongly affects the
growth performance of the national economy.
11
b) What are productivity and efficiency?
 Productivity is the ratio of a volume measure of output to a volume
measure of input (OECD).
 Efficiency has two dimensions:
 Technical (or operational) efficiency.
 Allocative efficiency.
Technical
efficiency
Allocative
efficiency
Economic (or cost)
efficiency
 Two key objectives of efficiency measurement in the public sector:
1) To trace technical inefficiencies
2) To identify inefficiencies in the mix of production factors.
12
c) How to measure them?
 Measuring productivity and efficiency requires quantitative
information on inputs (costs) and outputs (volume) of public service
provision.
BUT
 Public services are complex activities and therefore defining the
relevant input and output variables for efficiency or productivity
analysis is not straightforward.
Output measurement – some challenges
 “Activities where non-market producers dominate pose specific problems
of productivity measurement, due to the difficulty or impossibility of
observing and/or defining market prices or output” (OECD).
 The input method is based on the assumption that output equals inputs
for non-market services, then productivity is considered constant over
time and across countries.
13
 It is necessary to carry out quality adjustments.
 Outputs can be measured at various levels: – Micro level
– Intermediate level
– Macro level
BUT
– Only final outputs should be counted since intermediate outputs
may lead to double counting .
– Aggregation requires weight .
 The output mix of many public sector organisations includes intangibles.
 The challenge of measuring outputs for collective services since it is
difficult to define the output.
 The creation of incentives to ‘game’ or cheat.
14
Input measurement
 Efficiency analysis requires cost accounting.
 For costs to be imputed to outputs it is necessary to separate costs and
outputs.
 Input information is not always available in monetary terms.
Difficulties in measuring efficiency
 Weak data.
 Poor-quality measures of output with perception-based quality
indicators
 More importantly, the definition of the output and the methodology to
measure it.
15
The impact of institutional arrangements on efficiency
 Understanding institutional arrangements is essential to understand
efficiency of public sector production processes.
 Evidence on the impact of institutional drivers on efficiency in the public
sector is rather limited.
However:
 Efficiency gains may be obtained by increasing the scale of
operations.
 Efficiency may increase via functional or political decentralisation to
sub-national governments.
 Human Resource Management practices largely influence efficiency.
 Findings are inconclusive on the impact of ownership, competition and
agencification over efficiency.
16
d) Exemplifying institutional drivers of efficiency – workforce
issues
a.
Workforce size
Are larger organisations more efficient than smaller ones?
b. Workforce composition
What is the impact of workforce diversity and representation on efficiency?
c.
Extent and nature of unionisation composition
What is the impact of unions on public sector efficiency?
d. Attractiveness of the public sector
How to recruit and retain highly skilled staff?
17
3. Building an HRM Indicator – delegation of
authority
a) Why delegate?
 Good management is essential for improving performance and efficiency
and improving services to citizens.
 Hence, OECD governments have enable and empowered public managers
to adapt their HR systems to the business needs of their organisations.
 Delegation of authority to managers makes public organisations attractive
employers.
 The speed and the extent of the reforms vary across countries due to
differences in political, cultural and historical context.
18
b) What can be delegated?
1.
Delegation of manpower planning.
2.
Delegation of recruitment functions.
3.
Delegation of staff training and development.
4.
Careers systems and planning.
5.
Organisations have to be able to motivate their staff.
6.
Working arrangements.
7.
Retirement benefits.
A good indicator of freedom to manage is the degree to which
departments control the personnel budget.
19
c) OECD criteria for building composite indicators
a) A clear theoretical framework.
b) Indicators must be selected on the basis of their quality and
relevance.
c) Explain and justify the selection of weights and aggregation
methods.
d) Select a method for handling missing values.
e) Normalise indicators to render them comparable.
f) Explicit assessment on the robustness of the composite indicator.
g) Presentation should clarify not mislead.
h) The composite indicator should be easily replicated.
20
d) Delegation composite – the process
• Scoring and selection of each variables done within an expert group
• Variables aggregated and weighted according to an underlying
theoretical framework (Alesina & all.)
• Implementation of statistical checkings:
1) Questions not statistically relevant were removed for building the
composite (i.e. Factor analysis)
2) Good correlation among the set of variables (i.e. Cronbach alpha)
3) Index robustness (i.e. Sensitivity analysis)
• The Delegation Index sticks to the steps identified in the OECD
Handbook
21
e) The variables used for the HRM composite on delegation
Existence of a central HRM body (question 20)
Delegation of establishment (question 21)
Delegation of compensation levels (question 24)
Delegation of position classification, recruitment and dismissals (question 27)
Delegation of conditions of employment (question 30)
The arrangements for delegation (question 35)
Impact of delegation for pay/terms and conditions of employment across
government organisations.
22
f) Example of scoring
Q. 20 Is there a central agency/department in charge of human resources
at central/national/federal government?
•Yes: 0.000
•No. 1.000
•Not responsible, but a central agency/department aims to coordinate
the HR policies across departments: 0.5
23
Q. 21 Delegation of establishment
Central HRM
body.
Central HRM
body but with
some latitude for
ministries/
departments/
agencies in
applying the
general
principles
Ministries/depar
tments/agencies
Unit/team level
Other variable,
depends largely
on departmental
functions
Numbers and
types of posts
within
organisations
0.250
0.500
0.750
1.000
0.625
Allocation of
budget
envelope
between payroll
and other
expenses
0.250
0.500
0.750
1.000
0.625
Primarily
determined by
24
Delegation in the public service in OECD countries
+
High
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Composite Index
Non weighted average OECD
Source: OECD – 2006 HRM Survey Questions 20, 21, 24, 27, 30, 35, 36
Notes:
- Index comprised between 0 (no delegation) and 1 (high level of delegation)
- Cronbach alpha : 0.876 (computed with SPSS). A Cronbach 's alpha above 0.6 indicates a high degree of correlation
among a set of variables.
- Data missing for Greece
25
Turkey
France
Italy
Portugal
Luxemburg
Slovak Republic
Ireland
Mexico
Japan
Austria
Hungary
Germany
Netherlands
Denmark
Belgium
Norway
Poland
Czech Republic
Switzerland
Iceland
Spain
Korea
United States
United Kingdom
Finland
Canada
Australia
New Zealand
-
Sweden
0.0
Low
g) Pros and cons of composite indicators
Pros
 Can summarise complex, multidimensional realities to support
decision-making.
 Are easier to interpret than
many separate indicators.
 Can assess progress of countries
over time.
 Condense information and thus
make possible to include more
information.
 Place issues of country
performance and progress at
the centre of the policy arena.
 Help to construct narratives.
 Enable users to compare
complex dimensions effectively.
Cons
 May send misleading policy
messages if poorly constructed
or misinterpreted.
 May invite simplistic policy
conclusions.
 May be misused.
 The selection of indicators and
weights could be subject of
political dispute.
 May disguise serious failings in
some dimensions.
 May lead to inappropriate
policies if dimensions of
performance that are difficult to
measure are ignored.
26
Further reading:
•
OECD (2007), “Towards Better Measurement of Government”,
OECD Working Papers on Public Governance, 2007/1, OECD
Publishing.
doi: 10.1787/301575636734
•
Lonti, Z. and M. Woods (2008), “Towards Government at a Glance:
Identification of Core Data and Issues related to Public Sector
Efficiency”, OECD Working Papers on Public Governance, No.7
OECD Publishing.
doi: 10.1787/245570167540
•
OECD (2008), “Handbook on constructing Composite Indicators:
methodology and user guide”, OECD JRC European Commission.
OECD Statistics Working Paper.
•
European Commission (2005), “Input to handbook of good
practices for composite indicator’s development”, Knowledge
Economic Indicators; Workpackage 5 .
27
OECD GOV Technical Papers:
•
Technical Paper 1: ‘How and why should government activity be
measured in ‘Government at a Glance’.
•
Technical Paper 2: ‘Issues in output measurement for ‘Government at a
Glance’.
•
Technical Paper 3: ‘Issues in Outcome Measurement for ‘Government at
a Glance’.
•
Technical Paper 4: Institutional Drivers of Efficiency in the Public Sector’.
Information available on line: www.oecd.org
28
Oscar Huerta-Melchor
Policy Analyst
OECD GOV Budgeting and Public Expenditures Division
oscar.huertamelchor@oecd.org
For further information on the OECD Government at a Glance project contact:
Zsuzsanna Lonti
Project Manager
zsuzsanna.lonti@oecd.org
Laurent Nahmias
Statistician
laurent.nahmias@oecd.org
29