The MCC Effect: Quantifying Incentives for Policy Change in an Ex-Post Reward System
Prepared for the Millennium Challenge Corporation by Ingrid Aune, Yanyan Chen, Christina Miller, and Joshua Williams
Robert M. La Follette School of Public Affairs, University of Wisconsin-Madison
Executive Summary
The Millennium Challenge Corporation (MCC) is
a U.S. Government foreign aid agency using an
ex-post rewards system to determine funding
eligibility. When countries change policies to
meet MCC eligibility requirements, this is the
“MCC effect.” The performance of ex-post
rewards for good governance has important
policy implications beyond the agency itself for
global development aid. The only two existing
quantitative studies of the MCC effect are
unpersuasive: one finds an effect, but the study
is too early, the evidence sparse and
inconsistent, and the finding statistically weak;
the other analyzes the effect of only one
eligibility criterion. Our report identifies and
focuses on five unique, plausible “treatment
groups”—countries we believe are most likely
to exhibit an MCC effect. Using an original panel
dataset, we conduct preliminary analyses to
test our hypotheses and models. We then use
three difference-in-differences regression
models to analyze results by funding eligibility
indicators, treatment groups, country income
ranges, and time periods. We find no
compelling evidence of an overall MCC
incentive effect, but we do find signs of limited
MCC effects for some eligibility indicators,
country groups, and time periods. We conclude
our report with a road map for future analyses
of the MCC effect and recommendations about
how to stimulate a stronger MCC effect.
Indicator Mobility
MCC Ex-post Aid Criteria
(2004-2010)
1. Countries must perform above
the median score of their income
peer group on at least half of all
eligibility indicators.
2. Countries must perform above
the median on the Control of
Corruption indicator.
Indicator mobility refers to the relative
speed of eligibility indicator change. We
conjecture that policy can bring about
visible change along some dimensions
more quickly than along others; this has
implications for responsiveness to MCC
incentives. As each indicator has its own
scale, we standardize “speed” by ranking
indicator scores for the countries for
each year and show how quickly the
rank of each changes in a given period.
Light yellow represents relatively fast
changes, dark red represents slower
changes.
Research Questions
Ruling
Justly
Civil Liberty
0.19
0.12
0.65
0.14
0.33
0.45
Corruption
0.29
0.13
0.21
0.24
0.35
0.68
Government
Effectiveness
0.13
0.38
0.1
0.93
0.38
0.29
Political
Rights
0.01
0.28
0.53
0.8
0.73
0.41
Rule of Law
0.26
0.01
0.01
0.11
0.01
0.14
Voice and
Accountability
Cost to Start
Business
Data
Drawing directly from the third-party sources the
MCC itself uses, we compiled an original, single
panel database with 20 funding eligibility
indicators for 98 countries from 2002 through
2010. We chose this labor-intensive effort, rather
than simply analyze data in the MCC dataset, to
strengthen data accuracy: the third-party sources
revise their historical data as new information
becomes available.
Treatment
Close to Passing or Failing
Control of Corruption Standard
Threshold Partnership
Quartile of Government
Expenditure
MCC Quanlitative Evidence
Covariates
Sd
-0.09
(0.14)
0.49***
(0.14)
0.72***
(0.16)
0.54***
(0.14)
Standard errors in parentheses ，*** p<0.01, ** p<0.05, * p<0.1
1. Qualitative Evidence
Countries for which the MCC has anecdotal evidence
of the MCC effect should also exhibit quantifiable
evidence of the MCC effect.
2. Close to Passing or Failing the Indicator Threshold
Countries close to passing or failing the MCC eligibility
threshold have relatively bigger incentives to change
policies to obtain MCC funding.
3. Control of Corruption Standard
Countries that have not (and perhaps cannot) pass the
corruption benchmark are not incentivized to change
policies to attempt to pass other standards.
4. Threshold Partnership
Countries in the MCC Threshold Partnership program,
which provides economic incentives and direct
support to improve policies, are relatively more likely
to change policies to obtain MCC funding.
5. Government Expenditure
Poor governments with large expenditures are
relatively more incentivized to change policies to
obtain MCC funding.
Difference-in-Differences Models
Correlation
among Indicators
1. Does rigorous quantitative analysis
support MCC anecdotal evidence of an MCC
effect?
2. What analytical and policy prescriptions do
quantitative findings suggest for the agency?
Test for Conjectures of
Treatment Groups
To test if there is a quantitative basis
for our treatment group conjectures,
we compare each group to the group
with qualitative evidence. We find
such a basis.
Treatment Groups:
Conjectures about Incentives for
Policy Change
0.094
Investing in
People
0
Time to Start
Business
0
0.095
0.61
0
0.15
0
0
0.15
0
0
0.1
Inflation
0.14
0.11
Time to
Register
Property
0.25
0.4
Regulation
Fiscal Policy
0
0
0
0
0
0.46
0.43
0
0
0.43
0.22
0.12
0.16
High correlations among eligibility
indicators suggest redundancy in
MCC eligibility requirements.
Difference-in-differences models are relatively easy to implement, control for fixed effects, and are
widely used to estimate causal relations. We specify three models:
(1) Rate of Reform: We estimate the MCC effect on the rate of reform for each of 20 indicators for
the five treatment groups.
(2) Likelihood of Improvement: We use a probit regression model to estimate the probability of
policy improvement for each of the 20 indicators for the five treatment groups.
(3) Proportion of Indicators Improved: We calculate the proportion of the number of improved
indicators as a fraction of all indicators in a given period. This allows us to see the aggregate MCC
effect for all 20 indicators across all five treatment groups.
Examples of Results
All Countries
The table shows the
number of indicators
with a statistically
significant MCC
effect over 645
regressions by
model, indicator
cluster, and
treatment group.
Model 1
Rate of Reform
Treatment
Economic
Freedom
Investing in
People
Model 2
Likelihood of Improvement
Ruling Justly
Economic
Freedom
Investing in
People
Model 3
Proportion of Indicators Improved
Ruling Justly
Economic
Freedom
Investing in
People
Ruling Justly
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
1: Qualitative Evidence
2
1
1
1
2
3
2
0
3
0
2
3
0
0
0
0
0
1
2: Close to Passing or Failing
the Indicator Threshold
1
0
0
0
2
0
0
0
0
0
3
0
0
0
0
0
0
0
3: Control of Corruption
Standard
2
3
0
0
2
0
1
2
0
0
0
0
0
0
0
0
0
0
4: Threshold Partnership
0
2
1
3
2
1
0
0
0
2
2
1
0
1
0
1
0
0
5: Countries in the Middle
Range of Government
Expenditure
3
2
2
1
0
2
1
0
3
0
0
1
0
0
1
0
0
0
Totals of Positive and
Negative Results
8
8
4
5
8
6
4
2
6
2
7
5
0
1
1
1
0
1
Economic
Freedom
Breakdowns
Ruling
The bar graphs
Justly
present visual
statistical
summaries of
results for separate
regressions when
broken downs by Investing in
People
funding eligibility
indicator, country
income range, and
time period.
Time
Effect
Formula: (Yt+2 −Yt )ig = α + β ∗ period + γ ∗ treatment + c ∗ Xigt + δ ∗ treatment ∗ period
Main Findings
1. We find no compelling evidence of an overall MCC
effect.
2. We do find signs of limited MCC effects for some
eligibility indicators, country groups, and time periods:
for example, a strong effect in the lowest-income
countries for democratic governance indicators and a
steadily declining effect for these indicators in slightly
less impoverished countries.
Main Recommendations
Improving Future Analyses
1. Run randomized samples in treatment groups to get an empirical distribution to check for false positives.
2. Reduce endogeneity by employing instruments or combining a regression discontinuity design with the difference-indifferences models.
Policy: Stimulating a Stronger MCC Effect
1. Inspire countries to pursue MCC funding.
2. Provide more information about MCC criteria to developing countries: publish country rankings for each eligibility
indicator, reduce the number of indicators, and maintain consistency in indicators over longer periods of time.
3. Set more distinct goals for countries based on income levels.
This image cannot currently be displayed.