Randomized Control Trials for
Agriculture
Pace Phillips,
Innovations for Poverty Action
www.poverty-action.org
Outline
•
•
•
•
•
•
What we do
IPA Ghana
IPA’s Mission and Approach
Why evaluate?
How to evaluate impact?
Examples of evaluations
What does IPA do?
• IPA was created in 2002 to determine what
works and what does not work in poverty
alleviation through rigorous evaluation.
• Estimate magnitude of effects
– Most cost-effective solutions; can be surprising
• Learn how to improve programs through
testing operational questions
• Determine where to spend limited resources
IPA Ghana
• In Ghana since 2005
• 16 projects nationwide
• Permanent offices in Accra and
Tamale
• Current Agric Partners:
– Ghana Insurers Association
– GIZ
– IFPRI
– Presbyterian Agricultural Service
– MoFA
– SARI
– ISSER
• Trainings /Conferences
Why focus on impact evaluation?
• Surprisingly little hard evidence on what works
• Can do more with given budget with better evidence
• If people knew money was going to programs that
worked, could help increase pot for anti-poverty
programs
• Instead of asking “do aid/development programs
work?” should be asking:
– Which work best, why and when?
– How can we scale up what works?
6
Process and impact evaluation
Inputs
•Process
evaluations
•Monitoring
Outputs
In the control
of the
organization
Outcomes
Impact
Monitoring/Process evaluation: tracking development and outputs
of a program as compared to stated objectives, targets and timelines
Impact evaluation: how, and how much, does the organization affect
the welfare of beneficiaries (and non beneficiaries)
How do you evaluate a program?
• In Northern Ghana, a development
organization undertakes a program to
promote fertilizer use to improve yields. How
do we know if the program was successful?
Impact: What is it?
Crop Yields
Intervention
Impact
Time
What is the impact?
Crop Yields
Intervention
Impact
Time
What is the impact?
Impact
Crop Yields
Intervention
Time
Counterfactual
• Counterfactual: What would have happened
in the absence of the program?
• Problem: Counterfactual is not observable
 the key goal of all impact evaluation methods
is to construct or “mimic” the counterfactual.
• Solution: Counterfactual is often constructed
by selecting a group not affected by the
program.
Why randomize?
• Without Random Selection, the control group will
have intrinsic differences that can bias the study.
• Because members of the groups (treatment and
control) do not differ systematically at the outset
of the experiment
• Any difference that subsequently arises between
them can be attributed to the program rather
than to other factors
• If properly designed and conducted, randomized
experiments provide the most credible method to
estimate the impact of a program
13
Random sampling and random assignment
Randomly
sample
from area of
interest
Random sampling and random assignment
Randomly
sample
from area of
interest
Randomly
assign
to treatment
and control
Randomly
sample
from both
treatment and
control
Which method we use matters!
Method
Impact Estimate
(1) Pre-post
26.42*
(2) Simple Difference
-5.05*
(3) Difference-in-Difference
6.82*
(4) Regression
1.92
(5)Randomized Experiment
5.87*
*: Statistically significant at the 5% level
Underinvestment in Agriculture
• A study in Northern Ghana to understand if farmers do not invest in
their farms because either capital constraint or risk aversion.
Year 2: 1377 farmers in 74 villages
Year 2 Randomization
Insurance: 729 farmers
Rainfall index
insurance offered at
varying prices
Capital: 363 farmers
GHS 350 provided
Control: 177 farmers
Both: 108 farmers
Rainfall index
insurance offered at
varying prices, AND
GHS 350 provided
Results
• Capital only farmers
– Used more inorganic fertilizer only
• Capital and insurance farmers
– Increased farm investment by 20 percent
• Insurance farmers
– Increased total farm expenditure by 13 percent
•
•
•
•
•
Increased inorganic fertilizer use by 25 percent
Increased cultivation area by 8 percent
Increased expenditures on land preparation by 12 percent
Increased total labor use on plots by 13 percent
Increased harvest output by 8 percent
• Increase in investment but no increase in
profitability
Demand for Agriculture Insurance
Uptake
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0
1
4
8
9.5
Price per Acre (GH Cedis)
(Act. Fair = 9.5)
12
14
A Well Timed Nudge
• A study in Western Kenya tested whether farmers purchases of
fertilizer was dependent on the time of the year it was offered.
Baseline: 1,125 Farmers in Suri Kenya
Randomization
Fertilizer offered
AFTER HARVEST
with free delivery
CHOICE of when
they want the
fertilizer offer with
free delivery.
Fertilizer offered
just BEFORE
GROWING SEASON
with free delivery
Control – No Offer
50% Subsidy
Fertilizer offered
just BEFORE
GROWING SEASON
with free delivery
Results
• Farmers had high demand and ability to
purchase fertilizer in advance.
• Impact of the After Harvest offers were
comparable to that of a 50% subsidy at fertilizer
application time.
• Increase in Fertilizer Use
•
•
•
•
•
Control Group 28%
After Harvest 39%
Choice of Time 47%
Before Growing Season 33%
50% Subsidy Before Growing Season 41%
• Once the program stopped, fertilizer usage went
back to what it had been.
Thank You
www.poverty-action.org
IPA Ghana
• Why do farmers under-invest in farms?
• What is the impact of financial savings programs
in schools?
• What is the willingness to pay for clean water?
• What is the impact of remedial education
programs, smaller class size, teacher training and
tracking on literacy and numeracy?
• What is the impact of health insurance education
on enrollment and reenrollment in NHIS?
• Are mobile reminders an effective way to
improve adherence to ACT regimens?
Problems Selecting a Control Group
Without Random Selection, the control group will have intrinsic
differences that can bias the study.
Difference between
Groups
Selection Bias
Intrinsic Differences
Observables
Non Observables
Impact of the program
Our approach
We generate insights on what works and what does not through randomized evaluations,
and ensure that those findings will be useful to, and used by practitioners and policy
makers
Innovate
- Understand
market failures
-Develop
innovative
solutions to
poverty
- Use frontier
knowledge
from
economics,
and psychology
Evaluate
Randomized
Controlled
Trials
-Impact
evaluations
- Comparing
variations of an
intervention
- Experiment
with product
designs
Replicate
Replicate
evaluations in
various settings
to :
- Generalize
research
findings
- Tell
practitioners
what works
(and not), when
Communicate
Effectively
communicate
to practitioners:
-Conferences
-Workshops
with policy
makers and
practitioners
-Policy memos
and focus notes
Scale
Facilitate scaleup of effective
solutions :
- Active policy
outreach
-Practitioners’
toolkits
- Hands-on
technical
assistance