For Peace and Goodwill?
Using an Experimental Game to Analyse the Effect of the Desarollo y Paz
Programmes in Colombia.
David PHILLIPS
Institute for Fiscal Studies, London
(also Orazio Attanasio and Luca Pellerano)
LACEA Annual Conference
Buenos Aires, 1st October 2009
Introduction
•
Increasing emphasis on evaluating programmes with socio-cultural aims:
–
–
•
values, aptitudes, perceptions, motivations, capabilities, empowerment
relationships, networks, social capital
Rigorous impact evaluation in this domain is extremely challenging as:
–
–
–
•
Expected socio-cultural changes are often not clearly specified (Theory of Change)
They are difficult to measure
As the analysis moves in the domain of the “intangibles”, the evaluation must be driven
my a multi method and multidisciplinary approach.
Attanasio, Pellerano and Phillips (2009):
–
We evaluate the effect of a Social Fund type intervention in Colombia on social capital
and conflict management using a Public Goods game.
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
The Setting. Conflict in Colombia.
•
Colombia has been affected by a complex civil conflict along the whole course of
its recent history.
•
Two main illegal factions: Guerrilla(s) and Paramilitary groups (+ new emerging
groups after process of desmovilización)
•
Despite progress, violence still affects many rural areas of the country in a very
clustered way.
•
Strong association between the presence of illegal groups and violence levels, local
political instability, inequality (Sanchez et al, 2003)
The Peace and Development Regional
Programmes (PRDPs)
•
Community-driven development / peace-building initiative
•
Networks of grassroots organizations formed from existing civil society (labour
unions, the church, private foundations) in poor and violent regions
•
Claim peace is bottom up and based on development and community action.
•
Support a wide range of development processes informed by the principles of
protecting life (“Primero la vida”), equity, solidarity and participatory democracy
•
The PRDPs work as an “umbrella fund”, financing a series of projects along three
main axes:
1. Productive Development
2. Institutional Development and Governance
3. Human Rights and “Culture of Peace”
The Peace and Development Regional
Programmes
• Started in Magdalena Medio in the early 1990s (Jesuits)
• 6 PRDPs have been supported by the UNDP, the World Bank (LIL1 and LIL2 grants,
1998-2003; Paz y Desarrollo Loan, 2004-2008) and the European Commission
(Laboratorios de Paz I, II and III, 2002-2010)
• Total investment of more than 180 million USD
The Peace and Development Regional
Programmes
6 Regions
9 Departments
125 Municipalities
1363 projects
800 grassroots
organizations
180,000 estimated
beneficiaries
On average:
300 beneficiaries
per project
2 years project
duration
120,000 USD project
value
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
Experimental Games
Social capital
•
•
Competing definitions and competing measuring tools
Standard survey techniques
–
–
•
•
Structural approach (membership) versus motivational/value based approach
(trust, reciprocity)
People tend to respond to value based survey questions according to social
expectations and social norms (Glaeser et al., 2000).
Our definition: social cooperation (act collectively for mutual gain)
Our measure: a behavioural measure of contribution to a local public
good in a field experimental setting
Experimental Games
•
Based on the theoretical framework of the experimental economics:
players’ decisions should reveal their preferences/motivations
•
The “controlled” experiments are designed in such a way that decisions
only have monetary consequences (no reputation effects)
•
As the decisions imply a real monetary payoff (on average the equivalent
of 3 USD) participants are expected to reveal their preferences more
similarly to how they would behave in a real life situation
Experimental Games
Voluntary Contribution Mechanism – VCM (Marwell and Ames, 1979)
•
•
•
The experiment is a simple public good game in which subjects decide to
invest in a ‘public’ or a ‘private’ account
Played in group (40 participants on average)
Every token invested in the public account yields benefits for the whole
group, while the token invested in the private account only yields benefits to
the owner of the account
n
Si  2000(1  G)i  100 (G j )
j
•
•
Investing in the public pot is socially optimal
However an individual is always better off by investing in the private account
(Dominant Strategy)
The game is played in Two Rounds:
First Round. Anonymous decision
Second Round. 10 minutes communication
contribution to local public good
communication
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
The Data
Two main data sources
1. (Late) Baseline for the evaluation of PyD and LP (Nov 2006 – Feb 2007).
Treated Locations. Beneficiary and Non-Beneficiary Households
18 months of exposure to the activities of the PRDPs on average
High variability in exposure because of the roll out of the PRDPs both within and
across municipalities
.3
0
.1
.2
Fraction
.4
.5
•
•
•
0
50
100
150
Months
2. Second follow up for the evaluation of Familias en Acción (Nov 2005 – Apr 2006)
•
•
•
Control Locations
Sample for the evaluation of the famous CCT programme.
Targeted to the poor (SISBEN1), mainly women
The Data
FA Data
FA Municipalities (67)
Sample of Control
Locations
PRDP Data
PRDP Municipalities (37)
Sample of Treated
Locations
Members of Other Civic
Organizations
PRDP Project Beneficiaries
Sample of 2472
904 Non-Beneficiaries
782 Beneficiaries
(Control Group 1)
(Control Group 2)
(Treatment Group)
Beneficiaries with
higher exposure
Beneficiaries with
lower exposure
The Data. Experimental Game
Control
Treatment
Locations
Locations
Total
2242
1518
3760
Number of sessions
67
45
112
Size of the group for the smaller session
21
11
11
Size of the group for the biggest session
40
90
90
Average session size
36.90
35.96
36.52
Average proportion of women in the session
0.54
0.88
0.74
Average proportion of beneficiaries in the session
0
0.45
0.18
SD of the proportion of beneficiaries in the session
0
0.11
0.23
Number of participants
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
Descriptive Evidence
Municipality Characteristics
Altitude
Municipality development index
Coca crops extension
Murder rate (per 1000 inh.)
1285.766
37.243
56.472
0.926
Total population
Proportion urban population
Outgoing displaced population (per 1000 inh.)
Voting rate (local elections)
Voting rate (presidential elections)
Voting majority (local election)
Per capita public expenditure
Per capita public investment
Primary school students (per 1000 inh.)
Secundary school students (per 1000 inh.)
Judiciary processes initiated (per 1000 inh.)
Judiciary processes terminated (per 1000 inh.)
92067.281
0.543
27.98
0.485
0.768
0.408
0.308
0.246
148.43
243.632
17.962
18.723
Descriptive Evidence
Individual Characteristics
Female
Age (years)
Less than primary
Full primary
Some secondary
Full secondary +
0.528
39.85
0.297
0.207
0.159
0.253
Membership in Juntas de Acción Comunal
Leadership role in grassroots organizations
Voted in last elections (0-6)
One can trust the majority of people in the comm.
0.367
0.421
3.956
0.348
0.204
1.568
0.495
0.652
2.973
0.711
0.476
0.449
0.485
0.228
0.118
0.313
239
0.298
0.422
Monthly income (COP)
Monthly food expenditure (COP)
Value of self produced food (COP)
Any savings during the last year
Any member sufferd HHRR violation (prev.year)
368165
253129
73619
0.111
0.101
Household Characteristics
Female head of the household
Family members younger than 18
Urban
Own house
Number of rooms
Water by pipe
Sewage system
Rubbish recollection
Gas by pipe
Phone (landline)
Top distribution assets
Displaced (self declared)
Months living in the neighborhood
Other support
Laboratorios de Paz
•
High degree of heterogeneity in beneficiary characteristics according to project
types
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
Evaluation Strategy
• No random assignment -> Potential endogenous programme placement
(selection and self selection)
– Across locations
– Within locations
• No pre-programme information (apart from municipality characteristics)
• Identification strategy
a) within locations (beneficiaries vs. non-beneficiaries in treated locations)
b) across locations (treated vs. control matched municipalities)
c) within treated households (based on the intensity of exposure)
• May provide insights on the presence and magnitude of spill-over effects
• Multivariate regression models. All models are based on the “selection on
observables” assumption
Evaluation Strategy. Across Municipalities
•
We match treatment and control locations on a complete set of preprogramme municipality characteristics
0
.2
.6
.4
Propensity Score
Untreated: Off support
Treated: On support
•
•
•
.8
1
Untreated: On support
Treated: Off support
2 control locations dropped because of missing information
20 control locations and 16 treated locations are dropped because they fall outside
the common support
2,427 households (58% of the original sample) are on the common support)
Evaluation Strategy. Across Municipalities
Matched
Matched
Control
Treatment
Locations
Locations
0.545***
40.592
0.557
0.728
0.504***
0.480**
0.505***
0.252***
0.306***
0.198***
0.163
0.250***
0.837
41.542
0.624
0.65
0.307
0.375
0.102
0.078
0.446
0.133
0.155
0.053
0.538***
41.575
0.518
0.706
0.438*
0.412
0.532***
0.238***
0.305***
0.200***
0.159
0.238***
817.422
34.962
8.193
0.599
1285.766***
37.243
56.472
0.926*
1028.791
34.589
12.618
0.660
1146.092
34.883
6.955
0.648
27496.389
0.504
15.799
0.516
0.861
0.410
0.305
0.243
161.537
85.490
13.106
13.117
92067.281***
0.543
27.98
0.485
0.768
0.408
0.308
0.246
148.430**
243.632
17.962
18.723*
32226.170
0.520
20.959
0.503
0.821
0.413
0.301
0.236
156.607
87.676
14.033
15.082
33416.668
0.504
18.246
0.526
0.833
0.432
0.315
0.259
156.547
75.788**
13.067
14.489
Control
Treatment
Locations
Locations
Household Characteristics
Female
Age (years)
Urban
Water by pipe
Sewage system
Rubbish recollection
Gas by pipe
Phone (landline)
Less than primary
Full primary
Some secondary
Full secondary +
0.839
41.567
0.580
0.658
0.290
0.350
0.097
0.079
0.451
0.135
0.146
0.053
Municipality Characteristics
Altitude
Municipality development index
Coca crops extension
Murder rate (per 1000 inh.)
Total population
Proportion urban population
Outgoing displaced population (per 1000 inh.)
Voting rate (local elections)
Voting rate (presidential elections)
Voting majority (local election)
Per capita public expenditure
Per capita public investment
Primary school students (per 1000 inh.)
Secundary school students (per 1000 inh.)
Judiciary processes initiated (per 1000 inh.)
Judiciary processes terminated (per 1000 inh.)
Evaluation Strategy. Within Treated Municipalities
Treatment Locations
Control
Descriptive statistics
Treated
Households Households
Below
Above
Median
Median
Exposure
Exposure
Household Characteristics
Female
Age (years)
Urban
Water by pipe
Sewage system
Rubbish recollection
Gas by pipe
Phone (landline)
Less than primary
Full primary
Some secondary
Full secondary +
Female head of the household
Family members younger than 18
Displaced (self declared)
Months living in the neighborhood
Owns house
Number of rooms
Top distribution assets
Other support
Laboratorios de Paz
0.56
41.233
0.611
0.742
0.529
0.508
0.523
0.272
0.314
0.189
0.166
0.247
0.229
1.4
0.228
246.238
0.685
2.883
0.143
0.271
-
0.528
39.85
0.495**
0.711
0.476
0.449
0.485
0.228
0.297
0.207
0.159
0.253
0.204
1.568*
0.313**
239.297
0.652
2.973
0.118
0.298
-
0.535
38.817
0.47
0.72
0.459
0.433
0.49
0.195
0.309
0.234
0.152
0.23
0.187
1.557
0.319
236.096
0.64
2.949
0.096
0.301
0.4
0.517
41.712**
0.542
0.697
0.509
0.48
0.472
0.292**
0.277
0.159**
0.166
0.295*
0.236
1.583
0.306
243.727
0.672
3.018
0.159*
0.295
0.465
Municipality Characteristics
Altitude
Municipality development index
% Coca crops
Murder rate (1000 inhabitants)
1278.455
37.336
51.79
0.94
1294.217
37.135
61.884
0.911
1352.762
36.978
43.925
0.949
1185.251
37.382
96.055
0.839
* difference significant at 10%; ** significant at 5%; *** significant at 1%
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
Results. Experimental Game
Effects within treated municipalities
Round 1
Treated Household
Round2
-0.004
-0.003
-0.012
0.030
0.023
0.022
[0.037]
[0.037]
[0.036]
[0.043]
[0.046]
[0.046]
YES
YES
YES
Round 1 Decision
Location Controls
YES
YES
YES
YES
YES
YES
Household Controls
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
Session Composition Controls
Potentially Endogenous Vars.
Observations
YES
1485
1485
1485
YES
1485
1485
1485
Marginal Probit. Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%
•
There is no significant difference in the contribution rates between treated and
control households within treated locations.
Results. Experimental Game
Effects across municipalities
Round 1
0.256***
[0.059]
Matched
Locations
0.221***
[0.071]
Location Controls
Household Controls
Session Composition Controls
Potentially Endogenous Vars.
YES
YES
Observations
3782
All Locations
Treated Location
0.150*
[0.083]
Matched
Locations
0.185*
[0.102]
YES
YES
YES
YES
YES
2243
3782
All Locations
0.135*
[0.082]
Matched
Locations
0.141
[0.098]
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
2162
3778
2162
All Locations
Marginal Probit. Cluster at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%
Round 2
0.332***
[0.065]
Matched
Locations
0.313***
[0.075]
Round 1 Decision
Location Controls
Household Controls
Session Composition Controls
Potentially Endogenous Vars.
YES
YES
YES
Observations
3782
All Locations
Treated Location
0.305***
[0.099]
Matched
Locations
0.288***
[0.110]
YES
YES
YES
YES
YES
YES
YES
2243
3674
All Locations
0.300***
[0.100]
Matched
Locations
0.263**
[0.104]
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
YES
2162
3674
2162
All Locations
Marginal Probit. Cluster at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%
•
•
The level of contribution to the public good is much higher in municipalities
where the PRDPs operate, than in other municipalities in the country with
comparable characteristics.
Results hold if CS is imposed based on matching on individual characteristics
Results. Experimental Game
Exposure effects
Round 1
Exposure (7-14 months)
Exposure (>14 months)
Months of Exposure
Months of Exposure
(Municipality Max)
Round 2
0.045
0.010
-0.047
-0.058
[0.048]
[0.047]
[0.047]
[0.043]
0.085**
0.064
-0.014
-0.032
[0.043]
[0.042]
[0.048]
[0.043]
0.003
0.002
-0.001
-0.001
[0.003]
[0.003]
[0.003]
[0.003]
0.003**
0.002*
-0.001
-0.001
[0.001]
[0.001]
[0.001]
[0.001]
Round 1 Decision
YES
YES
Location Controls
YES
YES
YES
YES
Household Controls
YES
YES
YES
YES
Session Composition Controls
Observations
YES
684
684
YES
684
684
Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%
•
We find some light evidence that, ceteris paribus, increased exposure to the PRDPs
lead to higher contribution to the public good, particularly in the long run.
Results. Experimental Game
•
The previous findings suggest that there might be some contamination effect in
social capital formation at the community level between treated and control
households within the same municipality.
Examining the case for Spill Over effects.
Round 1
Knows PRDP
Treated Household
Round 2
0.078**
0.094***
0.001
0.013
[0.033]
[0.034]
[0.031]
[0.032]
-0.044
-0.048
0.03
0.016
[0.038]
[0.037]
[0.046]
[0.048]
YES
YES
Round 1 Decision
Location Controls
YES
YES
YES
YES
Household Controls
YES
YES
YES
YES
Session Composition Controls
Observations
YES
1472
1472
YES
1472
1472
Marginal Probit. Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%
•
•
Exploiting knowledge networks, social capital effects seem to propagate within
treated municipality through a spill-over mechanism.
The fact of knowing the PRDP could however be endogenous to social capital.
Results. Experimental Game
Critical Mass or Conditional Cooperation?
Round 1
Proportion of Treated in the Session
Treated Household
Interacted (proportion * treated)
Round 2
0.521**
0.614***
0.584**
0.634**
[0.214]
[0.223]
[0.264]
[0.256]
0.025
0.060
0.248*
0.265**
[0.145]
[0.153]
[0.129]
[0.124]
-0.082
-0.143
-0.518*
-0.564*
[0.316]
[0.328]
[0.314]
[0.301]
YES
YES
Decision in Round 1
Location Controls
YES
YES
YES
YES
Household Controls
YES
YES
YES
YES
Session Composition Controls
Observations
YES
1485
1485
YES
1485
1485
Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%
•
•
•
•
Because of the sampling procedures, the proportion of beneficiaries in the
session doesn’t reflect coverage rates in the municipality
In the second round, the group effect seems to be operative for control
households only
This suggests that it is mainly driven by reputation factors
Complementary evidence suggests that PRPD beneficiaries are actually
considered more “trustworthy” than the rest of the population
Results. Experimental Game
.85
.65
.7
.75
.8
lowess r_b2a mean_tipo_pers
6
2
Density
4
.7
.65
.6
.55
0
0
.5
2
Density
4
lowess r_b1a mean_tipo_pers
6
.75
Exploring the “critical mass” hypothesis
.2
.3
Density
.4
.5
mean_tipo_pers
.6
lowess r_b1a mean_tipo_pers
.7
.2
.3
Density
.4
.5
mean_tipo_pers
.6
lowess r_b2a mean_tipo_pers
.7
1.
2.
3.
4.
5.
6.
7.
The Setting and the Programme
The Experimental Games
The Data
Descriptive Statistics
The Evaluation Strategy
Results
Conclusions
Conclusions
•
PRDPs are grassroots-driven programmes designed to improve social capital
and hence entrench peace and drive development.
•
Survey-based evaluations has problems so an experimental Public Goods
game is used instead: contribution as measure of social capital
•
Difficult to identify an effect at individual level.
•
Much more contribution in treatment versus control municipalities.
•
Increased duration of exposure, knowledge of programme increase
contribution.
•
Proportion of beneficiaries in game increase contribution
•
Suggest reputation is important and critical mass and intensity and
coverage are key
For Peace and Goodwill?
Using an Experimental Game and a Social Dilemma to Analyse the Effect
of the Desarollo y Paz Programmes in Colombia.
more information on the PRDPs can be found at:
http://www.redprodepaz.org/
http://www.accionsocial.gov.co/
THANKS
LACEA Conference
Buenos Aires, 1st October 2009