Indicators based on representative
surveys of firms, households and service
providers
Javier Herrera (IRD - DIAL)
Mireille Razafindrakoto (IRD - DIAL), and François Roubaud (IRD - DIAL)
Seminar on “The Empirics Of Governance“
May 1 – 2, 2008, Washington DC
1
PRESENTATION SCHEME
I.
Uses of household surveys
II.
Taking into account sources of
bias
III. Objective vs subjective corruption
indicators
IV. Conclusions & Perspectives
DIAL
Développement,
Institutions et
Analyse de Long terme
2
I. Uses of household surveys
• Household survey as a « voicing »
instrument; public awareness on
corruption (particularly in of
authoritarian political contexts).
• Complex links with policy
3
Historical perspective
 First estimation of the extent of corruption in 1995 in
Madagascar (with Household survey)
 Headlines in the press: « Outcry against corruption! »
 It then became impossible to ignore the problem
4
MEASURE
1995  1998 
1995
It then became impossible to ignore the problem
 the Ministry of Justice took steps to introduce a
DEFINITION
OF POLICY
Interactions
QUALITY OF
INSTITUTIONS
/ Political
Regime
Positive
Impact
system of sanctions.
 a draft law on the fight against corruption
“ … It is important to bear in mind that, on the basis of the statistical survey conducted in May 1995
by the MADIO project, co-financed by the French Ministry for Cooperation and the European
Union: ‘the issue of corruption seems to be a recurring problem which haunts the capital’s
inhabitants. 96% of them consider that it is a major problem in Madagascar’. ‘Over 40% of people
aged over 18 in the capital had had to pay a corrupt civil servant during the previous year’.
Whatever the credibility of this survey and the interpretations that have been drawn from it, there is
no doubt that corruption is a social problem in Madagascar … and it is necessary to combat the
practice of corruption as energetically as possible”.
BUT Draft law rejected by the Government Council
in 1999
 Importance of context & characteristics of institution
(~ authoritarian regime and problem of governance)
2002
Institutional change  Stress put on transparency
 2003 Creation of an independent council (CSLCC)
 2004 Independent anti-corruption office (BIANCO)
5
 Headlines in the press in may 2005: « More confidence & less corruption »
after the public conference presenting the first results of the survey on « governance,
democracy and fight against poverty »
6
GOVERNANCE
An improvement of civil servant wages
or an active anti-corruption policy
 a sharp drop in the incidence of corruption
180
44
40
36
32
28
24
20
16
12
8
4
0
170
160
150
Active anticorruption policy
Political
crisis
130
120
110
100
90
1995
1996
1997
1998
Level of corruption
Sources :
140
Base 100=1995
%
Civil servant salaries and corruption levels in Madagascar 1995-2004
1999
2000
2001
2002
2004
Civil servants real wages (right scale)
Razafindrakoto, Roubaud (2002) and 1-2-3 Survey 2002-2004, INSTAT, DIAL, authors calculations.
7
II. Taking into account sources of
bias
• Who answers the question matters
(random sample selection within the
household)
• But also it matters to know more about
who don’t answer (non respondent profile)
• Taking into account the working of
corruption (middlemen, social norms, etc.)
• Robustness needs to be assessed
• Informal production units needs adapted
survey approach (the 123 surveys)
8
III. Taking into account sources of
bias
• The wording and sequence of question
matters (but not only when measuring
corruption, ie. labor surveys). Pilot
surveys are important.
• Non response to governance questions in
HH surveys are lower than in standard
questions (ie. Income)
• Unobserved heterogeneity (« over
optimistic »/ »under optimistic », use
panel data -ongoing research)
9
The surveys
Governance and Democracy Modules grafted onto official household
survey conducted by National Statistical Institutes
Survey support (stratified multi-stage sample surveys: area/HH) for the Modules (Individuals):
• In Africa: 1-2-3 Survey (LFS, informal sector, consumption and poverty)
- Madagascar 1995-2004, Capital + urban areas (2000, 2001), rural, entreprises surveys (time series)
- West Africa 2001/2003, in 7 WAEMU capital cities (Abidjan, Bamako, Cotonou, Dakar, Lomé,
Ouagadougou and Niamey)
35 594 persons interviewed
• In Latin America: Standard National Household surveys (ECH, Bolivia; SIE-ENEMDU, Ecuador;
ENAHO, Peru). National and regional inference for Peru and Ecuador.
- Peru 2002-, national level (18 000 HH sample in 2002; continuous survey from 2003 to date).
- Ecuador 2004, (20 000 HH)
- Bolivia 2004, national (1 700 HH).
-The survey is taking place in Colombia during the 2nd term of 2005.
More than 50 000 persons interviewed
10
Strong points / Basic principles:
Conducted by
National Statistical
Institute
Systematic
Presentation /
Publication of the
result  wider public
Ownership
 Integration in the
National Statistical
System
Validation
Democratic debate
 demand
Reliability
Relevance
Bottom-up approach
Investment in capacity
building
Light, flexible tool
Reconductible
 time-series
Sustainability
Marginal cost
HH survey « Voicing »
 empowerment, accountability
Supply side
Interactions
Demand side
11
Non response
There is (often) more willingness to answer questions on governance and
democray than usual economic questions
Non response rate to selected questions
West Africa
Mada
%
Coton
ou
Ouag
adougo
u
Abidj
an
Bama
ko
Niame
y
Dakar
Lomé
Antan
anarivo
Opinion on governement
functioning
0
3.6
0.9
1.1
2.7
5.2
3.5
0.1
2.2
Opinion on democracy
0
2.8
0.5
0.9
1.9
3.1
0.3
0.2
1.3
Income level (continuous)
40.2
54.3
40.9
43.3
51.6
56.3
37.2
47.3
Income level (continuous or
discrete)
2.3
6.5
3.2
6.5
14.1
9.2
1.6
0.1
Total
Sources : 1-2-3 Surveys, Phase 1, Governance and Democracy module, 2001/2003, National Statistical Institutes, AFRISTAT,
DIAL, authors’ calculations.
12
Robustness
Probabilistic surveys allows a quantitative assessement of indicators’ precision
(also relevant for group and intertemporal comparaisons)
In your opinion, corruption since last year?
has increased
95% confidence interval
has decreased
95% confidence interval
is the same
95% confidence interval
don't know
95% confidence interval
Total
rural
urban
Total
31,0
39,1
36,6
[29,0 - 33,0]
[37,4 - 40,9]
[35,3 - 38,0]
8,43
10,47
9,85
[7,5 - 9,5]
[9,5 - 11,6]
[9,1 - 10,7]
49,1
48,8
48,9
[47,5 - 14,0]
[46,9 - 50,6]
[47,4 - 50,3]
11,5
1,6
4,7
[10,3 - 12,9]
[1,3 - 2,1]
[4,2 - 5,2]
100
100
100
Source: our estimation based on ENAHO july 2003-june 2004, INEI, Peru.
13
Taking into account the working of corruption:
middlemen; failed corruption; “accepted”
corruption, amounts paid, access to services;
links with inefficiency…
Source: ENAHO 2004 HH survey Peru
14
Corruption incidence taking into account access to public
services
 Incidence (percentage of households victims) has decreased
 Total amount paid has decreased in real terms
 The poor as well as the rich had benefited from this improvement
Incidence and cost of corruption (2002/2003 and 2004 in Antananarivo)
En 2002/2003
Total
En 2004
Income per capita
Total
1st quartile 4th quartile
Income per capita
1st quartile 4th quartile
16,3
40,4
27,4
11,9
46,8
22,4
20,1
31,5
29,3
8,4
29,9
12,0
6,0
34,2
9,1
10,3
22,4
13,3
Mean (HH victims of corruption)
381
219
682
300
77
484
Median (HH victims)
25
3,3
20
7,8
50
3,1
30
1,2
14
1,8
50
1,2
Incidence (all households)
No contact with civil service
Incidence (households victims)
Total annual amount (1 000 Fmg)
% of income (HH victims)
Source : Enquête 1-2-3, modules qualitatifs, 2003 et 2004, INSTAT, authors calculations.
15
“Don’t know” answer is not random: it
concerns mainly the poor, the rural, the
less educated, the socially excluded
Importance of democracy
Do you think that having a
democratic government is
important?:
Yes
More or less
Not much
No
Does not know
Total
Rural
%
Urban
%
Not poor
%
Poor
%
Total
%
34.8
38.0
13.7
4.6
8.9
100.0
55.9
30.6
8.4
3.4
1.7
100.0
56.3
29.7
8.7
3.3
2.1
100.0
41.5
36.6
11.6
4.3
6.0
100.0
49.4
32.9
10.0
3.7
3.9
100.0
Authors’ own calculations from ENAHO 2002-IV.
16
Corruption incidence for the informal sector
production units can be measured
Cotonou
%Incidence of
Informal
Units that
had a
problem
with civil
servants
last year
Ouagadougou
Abidjan
Bamako
Niamey
Dakar
Lome
Total
4.7
5.0
7.0
3.5
6.2
8.5
6.2
6.2
By paying a
fine
75.5
76.7
42.9
38.5
40.4
49.2
44.5
47.3
By giving a
“gift”
13.9
5.7
50.2
32.6
24.7
38.1
45.1
41.3
other
10.5
17.6
6.9
28.9
34.9
12.7
10.4
11.4
% that settled
the
problem:
Source: Alain Brilleau et al. « Le secteur informel : Performances, insertion, perspectives, enquête
1-2-3 phase 2 », STATECO n°99, 2005 p.82.
17
III. Objective vs subjective
corruption indicators
• Expert’s surveys vs. HH surveys
• Do they measure the same
phenomena? (petty vs. big corruption)
• Minding the gap between :

-perceptions and objective
indicators
 -aspirations and perceived outcomes
18
The mirror survey
To round out the collection of surveys on Governance and
Democracy in seven West African capitals and in Madagascar
(Antananarivo),
an additional survey to get the opinions of a certain number of
Southern and Northern experts (researchers, development
practitioners, decision-makers, senior civil servants, politicians,
etc.).
The aim --> to compare answers from the population surveyed in
each country with the specialists’ point of view.
19
The mirror survey (…)
Two sets of questions:
- what the experts think the respondents answered on average.
- their own answer to these same questions (“What is your personal
opinion?”)
Questions :
- Specificity of the answers of the population / experts, specialists
- Knowledge of Northern or Southern experts on what happens and
on people’s thinking in the South
-Relevance and reliability of indicators based only on appreciation of
panel of experts
DIAL
Développement,
Institutions et
Analyse de Long terme
Sample size : 250 experts (30 per country in average)
20
The need for complementary approaches / indicators
How far can we trust the experts’ opinion on corruption?
Discrepancies between real extent of corruption and experts’
perception in Francophone Africa
 On the level of corruption / To what extent corruption acceptable
 On the relative positions (rank) of the different countries
Population who believes that making bribe is
acceptable
Mirror Survey
40
35
30
25
20
HH Survey
15
10
5
er
ag
e
Av
Lo
m
é
ak
ar
D
Ta
na
Ba
m
ak
o
N
ia
m
ey
C
ot
on
ou
O
ua
ga
Ab
id
ja
n
0
General population * (% from Household (HH) survey)
Expert panel (what they think could be the % of population who
believes that making a bribe is acceptable)
Sources: General Household Survey (35,594 persons interviewed; 4500 for each country in average); Expert
panel survey or Mirror survey (246 persons surveyed; 30 experts for each country in average). * In Madagascar,
results are drawn from the 2003 survey. Authors calculations.
21
DEMOCRACY
Measuring the gap between aspirations and effectiveness as
regards democratic principles
Perception of the main democratic principles by the population
 Are they fundamental? Are they respected in the country?
100
90
80
70
60
% 50
40
30
20
10
0
Cotonou
Ouaga
Abidjan
Bamako
Niamey
Dakar
Lomé
Tana
The six principles of democracy are:
Fundamental
Respected
Sources : Enquêtes 1-2-3, module Démocratie, 2001/2003, Instituts Nationaux de la Statistique, AFRISTAT, DIAL
authors calculations.
22
DEMOCRACY
Measuring the gap between aspirations and effectiveness as
regards democratic principles
Lomé
Religious
freedom
Free,
transparent
elections
100
Abidjan
Political
freedom (right
to choose party)
50
Religious
freedom
0
Freedom of
expression, of
the press
Equality before
the law
Fundamental
Respected
Free, trans parent
elections
Fundamental
Respected
80
60
40
P olitical freedo m
(right to cho os e party)
Religious
freedom
20
0
Freedo m of
expres s ion, of the
pres s
Freedo m to travel
Equality before the law
Free,
transparent
elections
100
50
0
Equality before
the law
Bamako
Free,
transparent
elections
100
50
Political
freedom (right
to choose party)
Antananarivo
Political
freedom (right
to choose
party)
Religious
freedom
Free,
transparent
elections
100
50
Freedom to
travel
Respected
Religious
freedom
50
0
Freedom of
expression, of
the press
0
Fundamental
Free,
transparent
elections
100
0
FundamentalEquality before
the law
Respected
Freedom of
expression, of
the press
Freedom to
travel
Fundamental
Respected
Political
freedom (right Religious
to choose freedom
party)
Equality
before the
law
Fundamental
Respected
Niamey
Freedom of
expression, Freedom to
of the press travel
Freedom to
travel
Cotonou
10 0
Religious freedo m
50
0
Freedom to
travel
Dakar
Free,
transparent
elections
100
Po
freedo
to c
p
Free
expre
the
Freedom to
travel
Fundamental Equality
Respected before the law
Ouagadougou
Free,
transparent
elections
100
Political
freedom (right
to choose
party)
Religious
freedom
Freedom of
expression, of
the press
Freedom to
travel
50
0
P
fre
(r
choo
Free
expr
of th
Fundamental
Equality before
the law
Respected
Equality
before the
law
Sources : Enquêtes 1-2-3, module Démocratie, 2001/2003, Instituts Nationaux de la Statistique, AFRISTAT, DIAL
authors calculations.
23
The need for complementary approaches / indicators
No correlation between
 Objective measure (incidence of corruption)
 Subjective perception of the working of civil service
% of negative opinion on the
working of the civil service
90%
Cor=-0,48
80%
Lomé
70%
60%
Cotonou
Ensemble
Niamey Bamako
50%
40%
Ouaga
Abidjan
Dakar
30%
Tana
20%
10%
0%
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
Incidence of corruption
24
The global lessons
• Advantages of household sample surveys
Transparency of measurement procedures
Representativeness
Quantification,
Comparability of indicators over time.
• In-depth policy-oriented analyses
More appropriate than international indicators and aggregates.
•(perception,
Both objective (behaviour, actual experiences) and subjective information
satisfaction)
Monitoring and relating the two fundamental dimensions of these phenomena.
• Socio-economic disaggregation
These two dimensions can be combined with traditional variables related to the
socio-economic characteristics of individuals and households (income,
occupation, sex, age, ethnic group, etc.).
Possibility to disaggregate information between different population categories
(gender, poverty, ethnic groups, discriminated people, etc.
• International comparability
25
IV. Conclusions & Perspectives
• Measure different dimensions of governance
• A detailed understanding of different forms of
corruption is necessary to analyse its
determinants and consequences
• Different sources of potential bias should be
considered in designing & analysing surveys
• Both, objective and subjective, governance
indicators need to be collected to better
understand aspiration/outcome gaps and why
corruption may become or not a political issue.
26
END
For more works on Governance, Democracy and Poverty see our web site
http://www.dial.prd.fr
Our article:
“Governance, Democracy and Poverty Reduction: Lessons drawn
from household Surveys”, International Statistical Review (2007),
vol. 75, issue 1, pp.70-95
is available upon request
27