Measuring economic development
and evaluating impact
Sonia Laszlo
Associate Professor, Economics
Associate Director, ISID
Outline
 Motivation: why measure?
 Measuring economic development:
• The Big Picture
• The Micro picture
 Measuring impact of development policies
• Standard toolbox
• National policies
• Smaller projects
 Conclusions
What is Economic Development?
"Development can be seen (...) as a process of expanding
the real freedoms that people enjoy.
Focusing on human freedoms contrasts with narrower views
of development, such as identifying development with the
growth of gross national product, or with the rise in personal
incomes, or with industrialization, or with technological
advance, or with social modernization.
Growth of GNP or of individual incomes can, of course, be
very important as means to expanding the freedoms enjoyed
by the members of the society.
But freedoms depend also on other determinants, such as
social and economic arrangements (for example, facilities for
education and health care) as well as political and civil rights
(for example, the liberty to participate in public discussion and
scrutiny).”
From Amartya Sen, Development as Freedom, Anchor Books, 1999 (p.3)
Why measure economic development?
 Evaluate “health” of an





economy
Evaluate changes in the
“health” of an economy
Set goals accordingly
Evaluate development policies
and projects
To compare
To better understand
Why measure economic Development?
Example
 Millennium Development
Goals (MDG)
• Goal 1: halve the absolute
poverty rate between 19902015
• Reached in 2010
• But only thanks to China
 Post-2015 MDG
• Now what?
The Big Picture
Macroeconomic indicators
 Gross National Product
(GNP), per capita
 Gross Domestic Product
(GDP), per capita
 Savings rates
 Human Development Index
• Combines GNP, Education
and longevity
 Happiness levels
 A subjective measure of
wellbeing?
 …
Socio-economic indicators
 Poverty rate (e.g. head
count index)
 Educational attainment
 Absence of illness
 Life expectancy
 Labour force participation
 Inequality (of income, land,
assets…)
 …
The Big Picture
 Aggregate income
measures (GDP, GNI):
• Aggregate
• Unidimensional
• Measurement issues
• Relation to welfare and
economic wellbeing?
 However, they perform
remarkably well in
predicting other, social,
outcomes:
• http://hdr.undp.org/en/dat
a/explorer/
The big picture
 Can money buy you
happiness?
• What does life
satisfaction mean?
• Is this causal?
Chart by: Angus Deaton
Micro measures
 We may want to look at individual based measures
instead
 Maybe even focus on the poor
 Why?
• Rawlsian Maximin & Veil of Ignorance argument
• Altruism
 Examples:
• Income, educational attainment, absence of ill health…
• Ability to meet basic needs
Micro measures
 Poverty:
 Total poverty gap (TPG):
• Amount of money required
• H= # poor, N= Population
to bring all the poor up to
• Poverty line? Yp
the poverty line
• Head Count Index = H/N
• Policy relevant
• Extent versus depth of poverty?
Measuring impact of development
policy and projects
 Suppose a project aims
for a particular target:
• Reduce poverty rate
• Increase educational
attainment
• Improve maternal and
child health
• Provide access to credit
via micro-finance
Policy prescription
Measuring impact of development
policy and projects
 How do you go about to evaluate whether it has
achieved success?
 Naïve approach: compare before and after
 Problems of identification:
• Reverse causation
• Self-selection bias
• Omitted variable
 Or, lack of counter-factual
Causation versus correlation
 “Country X invested in education
and has high income”
• Education affects income
• Income affects education
• Both determined by other factor (e.g.
institutions)
 “Job training programs have high
success rates”
• Or did participants self-select into
job training based on unobservable
factors, such as ability
Standard tool box - Quantitative methods
 Macro economic time series (growth, trends,
cross-country)
• But: identification without counterfactual?
• e.g: Vietnam with and without liberalization, holding all
else constant?
 Micro data sets
• Censuses (population, agriculture, health/schools)
• Household or labour force surveys (e.g. Demographic
and Health Surveys, WB’s Living Standards
Measurement Surveys, etc…)
• Still, need to find plausible sources of exogenous
variation
Example 1: Education as a development
policy
 Education is important for economic growth and
wellbeing
 MDG2: “Achieve universal primary education”
 MDG3: “Promote gender equality and empower
women”
• Target 3.A: “Eliminate gender disparity in primary and
secondary education”
 Canada’s foreign policy (DFATD) “Canada is
participating in the global educational effort”
Example 1: Education as a development
policy
 Theory:
• Education raises marginal productivity of labour
(increases skills) and so increases earnings
• Education serves as a signaling device for high ability,
and so increases earnings
• Higher education leads to better health & fewer children
(but invest more in children)
• Higher education associated with higher civic
participation
Example 1: Education as a development
policy
 Evidence:
Example 1: Education as a development
policy
 Some policy options:
• Build more schools
• Make them better quality
• Provide incentives to attend school (Conditional Cash
Transfers)
 Let’s look at some real world examples of these
policies and how they can be evaluated
Example 1.A.: Build more schools!
(Duflo, 2001)
 Duflo (2001): Effect of massive investment of new
schools in Indonesia
 Q: Infrastructure  Education
 Q: Education  Earnings
 Problems:
• omitted variables – family background,
community characteristics: third factors
• Endogeneity of schooling
Example 1.A.: Build more schools!
(Duflo, 2001)
 Natural experiment – Indonesia’s “Massive school
construction project” (INPRES)
 built approximately 2 new schools per 1,000
children in 1973-1974 to 1978-1979
• targeted kids who had low educational
attainment
• Primary school construction, hire and train new
teachers
• Financed from oil revenues.
Example 1.A.: Build more schools!
(Duflo, 2001)
 Effect of program
should be:
• zero for those
aged 13 and
above in 1974
• increasing for
younger children.
Example 1.A.: Build more schools!
(Duflo, 2001)
 Results – INPRES program:
• increased quantity, not quality
• increased educational attainment by 0.25 to 0.40
years
• increased probability of completing primary
school by 12%
• increased wages by approximately 3 to 5.4%
Example 1.B.: Progresa/Oportunidades
 Conditional Cash Transfer Program in Mexico
 Designed as a Randomized Controlled Trial (RCT)
experiment (“Pilot”: Progresa)
 Objective: increase educational outcomes
• Poverty is a binding constraint
• So, relax that constraint
 Provide cash transfers to poor households
conditional on children’s school attendance (&
visits to school nurse)
• Why conditional?
Example 1.B.: Progresa/Oportunidades
 RCT design:
Treatment
Control
Before
YTo
YCo
After
YT1
YC1
 Simple estimate of impact of program:
• Difference-in-difference: (YT1-YC1)-(YT0-YC0)
 Measurable outcomes:
• School enrolment
• Test scores
• Child labour
• Child health
• Poverty
Example 1.B.: Progresa/Oportunidades
 IFPRI & Mexican government:
• 24,000 households in 506 localities in randomly assigned PROGRESA
and non-PROGRESA areas.
• Formal surveys, interviews, focus groups, and workshops
 Their results showed huge success:
•
•
•
•
Increased primary enrolment by 1.07 ppts (boys) & 1.45 ppts (girls)
Increased secondary enrolment by 9.3 ppts (girls) and 5.8 ppts (boys)
Reductions in child labour by up to 25%
Reduced child stunting (12-36 month olds): increase of 16% mean
growth per year
 Scaled up: Oportunidades
Example 2: Sexual and reproductive health
 Both health and human capital implications
• Excess fertility (actual > desired # of children)
• Sexually transmitted infections
 MDG 5: Improve maternal health
 MDG 6: Combat HIV/AIDS, Malaria and other
diseases
Example 2: Excess fertility in Zambia
(Ashraf et al. 2014)
 Excess fertility: desired number of children less
than actual number of children
• High rate of unwanted births
• Yet, low rate of contraceptive use
 Outcome of bargaining process within the
household over contraceptive control?
 Discordant preferences between men and women?
Note some women hide contraceptive use from
husbands
Example 2: Excess fertility in Zambia
 Ashraf, Field and Lee (2014)
 The research problem: can we test for whether
spousal discordance can explain excess fertility?
 Outcomes of interest: implications of spousal
discordance in fertility decisions
• Women demand more birth control than men
• Women would prefer concealable forms of birth control in
the presence of spousal discordance
• These combined should lead to lower fertility rates
Example 2: Excess fertility in Zambia
 Treatment - voucher for immediate and free
access to contraceptives
 Random assignment to 2 different treatment
groups:
• Tindividual=1: voucher given to women alone/in private
• Tcouple=1: voucher given to women in the presence of
their husbands
 Baseline in 2007, follow-up in 2009
From Ashraf et al. 2014
Example 2: Excess fertility in Zambia
 Frequent shortages in contraception in local
clinics...
 Spousal consent required by law until 2005.
Nevertheless, practice still continues
 Problem in the study: injectable scare
• Dec 2007 - March 2008
• Misinformation in the press: Injectables tested HIV
positive
• Will thus be difficult to say much about effect on fertility
within this 2 year period
Example 2: Excess fertility in Zambia
(Ashraf et al. 2014)
 Results - Table 2:
voucher take-up
• give people free stuff,
they take it
• but effect not 100%
• effect stronger in
Tindividual
 Take up stronger in
Tindividual, but only for
women who do not want a
child in the next 2 years
Example 2: Excess fertility in Zambia
(Ashraf et al 2014)
 Concerns:
• treatment assignment to couple might have increased
discussion about family planning that wouldn't have
occurred in the absence of study
• selection bias in the take-up of vouchers in the couples
treatment (may exclude couples with extreme
discordance)
• involving men in family planning might not be optimal
Conclusions
 A large set of tools in the measurement toolbox
 Increasingly, researchers and policy makers
turning towards RCTs
• But: not without some caveats
• Internal vs external validity?
• Need to look at theoretical concerns
 Other tools:
• Qualitative methods
• Behavioural & lab experiments….
 Nevertheless, lots of innovations to measurement
and impact assessment