Natural Experiments and Firms
Rocco Macchiavello
CAGE Summer School
July, 14th 2010
E-mail: r.macchiavello@warwick.ac.uk
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Plan
1. Introduction: review of Difference in Difference
»
basic idea
»
further issues
- further remarks on:
»
event studies, falsification tests, examples
»
regression discontinuity
2. Examples from the literature:
- a micro ‘experiment’
- another micro ‘experiment’
[regulation and credit]
[entry and spillover]
Goal: give you an informal (i.e., non technical), practice oriented,
list of boxes to check when you are planning to use to natural
experiment in a particular setting ...
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID & RD
Let me jump straight into the problem, and leave for the end of the
class a discussion of methods in the study of firms in LDC
Suppose we want to evaluate the effect of a program. I use the word
program in a very broad way:
- a policy,
- giving money, redeeming debts, giving a subsidy,
- the entry of a competitor, etc ...
- a shock (of intrinsic interest, or that you use to identify a
response)
Let us assume that the program was not randomly offered to (let
alone taken up by) a subset of the observational units.
In other words: you do not have an experiment (and, when thinking
about larger firms, you typically won’t !)
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID & RD
Suppose you only have data post treatment, i.e., after the program
has been implemented.
Then just give up !! Even if you have a “control” group that has not
received the program, you won’t be able to show it is a good control
group since you can’t show the two groups looked very similar before
the program.
So, at the very minimum, you want to use pre- and post- treatment
data. Suppose you just have pre- and post- treatment data, but no
control group (i.e., all the units you observe were affected by the
program). Then, again, just give up!!
There are too many other things that might have changed in
between before and after and you can’t show the effects are due to
the program.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID
So, at the very very very minimum, you need
1. data from before and after,
2. a control group
This is the idea beyond differences in differences (DID)
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID Outcome
Treatment
Problem is that between the
two periods many other
things might have changed
Y1T
Y0T
0
Macchiavello
1
Natural Experiments and Firms
Time
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID Outcome
Treatment
Solution:
Y1T
find a control group that is
unaffected by the program but
otherwise behaves exactly the
same ...
Y0T
Wait a minute?
How do I know it behaves the
same?
Y1C
Y0C
0
Macchiavello
1
Natural Experiments and Firms
Time
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID Outcome
Treatment
How do you make sure that
the control is a valid group?
Usually check for equality in
pre-existing trends
!! Not levels !!
0
Macchiavello
1
Natural Experiments and Firms
Time
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID Outcome
Treatment
This is the difference
in difference estimator
0
Macchiavello
1
Natural Experiments and Firms
Time
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID
You can do this in a regression format:
Yit  1 2 Ti 3 Post t 4 Ti Post t 
it
Pre-treatment outcome
in C group
Macchiavello
Natural Experiments and Firms
July 2010
Plan
DID
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Outcome
Treatment
β₁
Time
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID
You can do this in a regression format:
Yit  1 2 Ti 3 Post t 4 Ti Post t 
it
Pre-treatment outcome
in T group
Macchiavello
Natural Experiments and Firms
July 2010
Plan
DID
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Outcome
Treatment
β₂
β₁
Time
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID
You can do this in a regression format:
Yit  1 2 Ti 3 Post t 4 Ti Post t 
it
Post-treatment outcome
in C group
Macchiavello
Natural Experiments and Firms
July 2010
Plan
DID
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Outcome
Treatment
β₃
β₃
β₁
Time
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID
You can do this in a regression format:
Yit  1 2 Ti 3 Post t 4 Ti Post t 
it
Post-treatment outcome
in T group
Macchiavello
Natural Experiments and Firms
July 2010
Plan
DID
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Outcome
Treatment
β₄
β₃
β₂
β₁
Time
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID
You can do this in a regression format:
Yit  1 2 Ti 3 Post t 4 Ti Post t 
it
DID 
Y T1 Y T0 
Y C1 Y C0 
1 2 3 4 1 2 1 3 1 


3 4 3 

 4
Macchiavello
Natural Experiments and Firms
July 2010
Plan
DID
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Outcome
Treatment
β₄
β₃
β₂
β₃
β₁
Time
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID in regression format has several advantages
Easy to calculate standard errors
In principle you can do this by OLS. In practice, you have to take into
account several issues:
- autocorrelation over time,
- (potentially) spatial correlation across units
Two ways of dealing with this:
- if you have very strong prior you can explicitly model st. err.
(Conley (2007))
- if you do not, then cluster (i.e., allow arbitrary correlation
patterns within cluster (see, e.g., Bertrand et al. (QJE 2002),
Cameron, Guelbach, Miller (2009))
What is the cost of clustering then?
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID in regression format has several advantages
Easy to calculate standard errors
In principle you can do this by OLS. In practice, you have to take into
account several issues:
- autocorrelation over time,
- (potentially) spatial correlation across units
Two ways of dealing with this:
- if you have very strong prior you can explicitly model st. err.
(Conley (2007))
- if you do not, then cluster (i.e., allow arbitrary correlation
patterns within cluster (see, e.g., Bertrand et al. (QJE 2002),
Cameron, Guelbach, Miller (2009))
What is the cost of clustering then? Loss of efficiency!
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID in regression format has several advantages
Easy to include multiple periods
- you want multiple pre-periods to validate identification
- you want multiple post-period to assess ST vs LT effect
Control for other variables (e.g., trends)
- If Identification strategy is valid, this should only affect the
residual variance, i.e., standard errors, not coefficient
Study treatments with different intensities
- the treatment variable can be, e.g., amount of a loan,
subsidy or tax rate
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID in regression format has several advantages
Remark
The idea that interaction terms can be used to identified channels is
broader. An example: Rajan and Zingales (1998)
Suppose you want to know whether financial development leads to
growth. Hard problem if you just use cross-country data.
Intuition: if FD → growth, it should do so relatively more in industries
that require a lot of finance
Interact FD in the country with exogenous component of demand for
finance in the industry (e.g., proxy for a technological characteristic)
Which further falsification test?
- is it really FD?
- is it really demand for finance?
Same logic can be applied in the context of DiD to get DiDiD
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
DID: some other issues ...
So far we have referred to a generic “program”. Depending on what
you are trying to “evaluate”, think about some of the following:
- endogenous selection (e.g., a program was offered but not every eligible
took it up – intention to treat) => this can lead to bias
- anticipation effects (e.g., a major policy change was approved, but after
a lengthy discussion, ...) => this can lead to bias
* this is problematic because you’d be tempted to learn from
unanticipated shocks, but then it might be hard to extrapolate
- short-term vs. long term effects (if you have many periods, always
check graphically non-parametrically)
- harvesting effect (e.g., effect of heat waves on elderly)
- spillover effects (e.g., industry equilibrium)
- heterogeneous effects (not a problem per se, but the parameter you are
identifying might not be relevant)
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Falsification Tests
This is a cheap slide. Often, soft-sceptics of your work (including
yourself!) can be convinced by a placebo test
What is the idea of a placebo test?
The idea is to show that some outcome variable that should have not
changed did not change. Stated in this form, it is not very clever.
Consider examples:
- Regulations to sectors, infrastructures to regions, etc.
- Margins that could not adjust (e.g., in the short run, product
Xs as given)
- Equality of trends before the treatment
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Event Studies (a remark)
The idea is very similar to a DiD, though it is implemented differently
across literature.
In Finance, for instance, people look at the effects of news or events
on stock market returns
Two issues:
- you need to construct a comparable portfolio, i.e., a control
group
- the effect should appear “immediately”, so that you need to
show results around a “window”.
This is clearly related to evaluation of (short-run) shocks using
synthetic control groups (Abadie et al. (various))
Fisman (AER 2004), Guidolin and La Ferrara (AER 2007) are good
examples.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
An Example (for the graphs ...)
Guns N’ Roses: the effects of ethnic violence on the Kenya Flower
Industry
A paper of mine, written with two co-authors. I present few graphs /
tables simply to illustrate the checks I have suggested.
Basically the paper look at the effect of a shock (ethnic violence) on
firms in Kenya.
Divide the country into two: a set of treated firms, a set of control
firms.
A short-run shock to the supply function: data on before, during, and
after
So, what do we need?
Macchiavello
Natural Experiments and Firms
July 2010
Variable
Observations
Mean in
SE
No-Conflict No-conflict
Mean in
Conflict
SE
Conflict
p-value
FIRMS ≠ ACROSS REGIONS
Firm Size
Export, Jan+Feb 2007, in Kg '000
114
90.60
11.20
104.67
15.65
0.48
Number of Workers Jan 2008
79
480.83
103.82
456.45
45.18
0.81
Land (Ha)
79
44.93
9.10
98.61
63.74
0.47
Year Firm Created
79
1997
1.03
1998
0.81
0.66
Foreign Owner
114
0.34
0.06
0.42
0.06
0.37
Indian Owner
114
0.22
0.06
0.21
0.05
0.87
Kenyan Owner
114
0.36
0.06
0.32
0.06
0.61
Politically Connected Firm
114
0.26
0.06
0.20
0.05
0.42
% of Female Workers
79
61.28
2.10
62.53
2.63
0.73
% of Temporary Workers
79
15.86
4.11
20.66
4.12
0.43
% of Workers with Prim. Educ.
79
36.73
5.43
49.31
5.54
0.11
% of Workers with Sec. Educ.
79
52.08
4.99
41.08
4.89
0.12
% of Workers Housed
79
11.20
3.57
11.21
3.14
1.00
KFC Member
79
0.63
0.09
0.52
0.08
0.35
Fair Trade Certification
79
0.30
0.09
0.32
0.07
0.87
Max Havelaar Certification
79
0.20
0.07
0.18
0.06
0.85
MPS Certification
79
0.40
0.09
0.50
0.08
0.40
% Exports to Auctions
114
49.95
4.65
50.74
4.50
0.90
=1 if Exports to Direct Buyers Only
114
0.20
0.06
0.22
0.05
0.87
% Production in Roses
114
0.67
0.06
0.61
0.06
0.41
Number of Insulated Trucks
79
1.40
0.22
1.11
0.25
0.39
Firm History & Ownership
Firm Labor Force
Firm Certification & Standards
Firm Products & Marketing
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
-6
-4
-2
0
2
Plan
-10
-5
0
5
Weeks from Beginning of Conflict
Conflict
Difference
Macchiavello
10
No Conflict
Natural Experiments and Firms
July 2010
VIOLENCE EFFECTS – NO CATCH UP
Panel A: Expanding Window
Panel B: Rolling Window
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
An Example (for the graphs ...)
1. validate identification assumption
- done: but trends, not levels!
- can you do “over-identification” tests?
2. rule out harvesting effect / catch-up
3. no anticipation
survey + detailed knowledge of production process /
institutional setting in the industry
4. spillover effects?
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
RD
An alternative (though the basic idea is very similar) way to go is
through a regression discontinuity.
RD comes in two styles:
sharp: treatment status is a deterministic and discontinuous
function of one covariate
fuzzy: probability of treatment is discontinuous at a certain
point of one covariate => use discontinuity as an IV
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
RD
In its simplest form you just need to add an indicator variable at the
discontinuity.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
RD
But what about this?
Well, if the trend relation is non-linear, you can still do RD by fitting a
polynomial in X (and allowing the polynomial to differ on both sides)
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
RD
But pay attention! It is easy to get a ...
To avoid this, just look at data in the neighbourhood of the
discontinuity.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
RD
... but the ultimate challenge to a successful RD design is
sorting.
That is, you can’t do RD if firms sort around the discontinuity
Can you think of an example?
Yes! Many countries have regulations of the form “pay taxes /
costs if employ more than Z employees”
Can you look around Z to understand the effects of the
regulation?
No! Firms sort! How do we know it? Well, typically the
distribution of firm size is discontinuous at that point.
Urquiola & Verooghen (AER 2009) very nice paper on this.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Examples
Banerjee and Duflo (2008)
=> use changes in regulation to identify credit
constraints
Greenstone, Hornbeck and Moretti
=> use entry of large plant to identify spillovers
NOTE:
Macchiavello
- both are DiD papers (not RD)
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
Are firms credit constrained? Knowing the answer to this question is
important for many reasons (e.g., understanding aggregate
differences in TFP, give policy recommendations, ...)
When is a firm credit constrained? If MPK > r (the interest rate paid
on the marginal unit borrowed)
So, to answer the question we just need to get an estimate of the
MPK. This is more easily said than done. In fact, it is an extremely
difficult problem. Not least because inputs (e.g., capital) are
endogenously chosen by firms that know more than the
econometrician about their productivity => error term is correlated
with dependent variables.
One solution is to hand out money randomly. De Mel, McKenzie &
Woodruff (2008) randomly allocated capital (about $200) to
microenterprises and find about 5% returns per month
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
Another sector where it is relatively easy to infer the importance of
credit constraints is agriculture (use random variation in whether)
Great. But still, it is important to know whether larger firms are
credit constrained – if anything because the bulk of capital is
invested there (but also for other reasons).
Banerjee and Duflo (2008) look at this issue exploiting a natural
experiment. The experiment is given by changes in law directing
(subsidized) credit to priority sector in India. The priority sector is
defined w.r.t. to the capital invested in the firm.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
This is an interesting paper on many different levels:
1. clever design to tackle an otherwise difficult problem,
2. simple theory that does 2 things:
- derive the test for credit constraints (intuitive)
- provide some guidance on how to interpret results
3. both reduced form and IV results: nice example of when a
natural experiment can be used to identify a structural
parameter
4. effectively they have 2 experiments (expansion and
contraction in the rule) which allows them to test the validity
of the identification assumption
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
This is an interesting paper on many different levels:
1. clever design to tackle an otherwise difficult problem,
2. simple theory that does 2 things:
- derive the test for credit constraints (intuitive)
- provide some guidance on how to interpret results
3. both reduced form and IV results: nice example of when a
natural experiment can be used to identify a structural
parameter
4. effectively they have 2 experiments (expansion and
contraction in the rule) which allows them to test the validity
of the identification assumption
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
All banks (public and private) are required to lend at least 40% of their net
credit to the “priority sector", which includes agriculture, agricultural
processing, transport industry, and small scale industry (SSI).
If banks do not satisfy the priority sector target, they are required to lend
money to specific government agencies at very low rates of interest (i.e., policy
is binding)
Change definition of SSI sector:
January 1998: investment in plant and machinery < Rs. 6.5 to < Rs. 30 million,
January 2000:
to < Rs. 10 million
Reform should lead to an increase in lending to the larger firms newly included
possibly at the expense of the smaller firms, and viceversa for the second
change.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
Focus on demand for subsidized bank credit. Consider a firm with limited access
to cheap bank credit that can also borrow from the market at a higher rate.
How increased access to cheap bank credit affects the market borrowing,
revenues and profits of the firm?
R = f(k) rupees of revenue after a suitable period, where k is working capital,
f(k) is increasing and concave.
Definition: Firm is credit constrained if there is no interest rate such that the
amount that the firm wants to borrow at that rate is equal to an amount that
all the lenders taken together are willing to lend at that rate.
Bank rate r, market rate i, r>i
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
Policy means that, at the same rate, the bank now offers more. If firms accepted the
additional credit then they are credit rationed with the bank. But this does not imply
they are credit constrained. They might have not borrowed more at the market interest
rate.
Since i<r, a non-constrained firm has MPK = i and, therefore, all new cheap credit from
the bank goes to pay down other debt. No increase in invested capital, production,
sales, etc. Firms profit increase because of the subsidy.
Output could increase only if the priority sector credit fully substitutes for market
borrowing.
Under credit constraints, instead, the firm output increase but the firm still borrow from
the market.
Note: logic of the test fails if the firm cannot pay down debt in the market and/or the
choice is not at the margin (i.e., subsidy allows firm to survive => this can be checked)
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008)
Describe the specification [triple difference]
Note: quite nicely, because of the two experiments, the identification
strategy can be tested.
If results were purely driven by changes in trends across the groups,
the trends should have been increasing for a group, and then
decreasing for a subset of this group.
Further restrictions on estimated parameters are imposed by the fact
that an increase and a decrease in capital should mirror each other
(under some additional assumptions?)
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Banerjee & Duflo (2008): Results
Bank lending and firm revenues went up (down) for the newly targeted
(dismissed) firms in the year of the reform, relative to firms that were
already included (remained included).
No evidence of substitution of bank credit for borrowing from the
market and no evidence that revenue growth was driven by firms that
had fully substituted bank credit for market borrowing.
Overidentification test: is the effect of credit the same in the two cases?
We also use this data to estimate parameters of the production
function: evidence is consistent with IRC
Further results on the allocation of credit [not discussed here]
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
Production of traded good is geographically concentrated (sometimes in locations, e.g.,
London) where costs are extremely high. Why?
Agglomeration externalities could be one answer. E.g., input and labour market
thickness advantages, direct productivity spillovers, etc.
The issue is very important from an industrial policy point of view
•
There are two primary approaches in testing for spillovers:
1. tests for an unequal geographic distribution of firms: are firms spread
unevenly? Are co-agglomeration rates higher between industries that are
economically similar?
The approach does not provide a direct measure of spillovers.
2. is a firm TFP higher when similar firms are located nearby?
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
The challenge for both approaches is that firms base their location decisions on where
their profits will be highest, and this could be due to spillovers, natural advantages, or
other cost shifters.
A causal estimate of the magnitude of spillovers requires a solution to this problem of
identification
GHM (2008) propose one. Take a very large firm, e.g., Toyota, deciding where to locate a
huge new factory. To do that, Toyota chooses among a list of potential sites (i.e.,
counties). Typically, it will start with a very long list.
Eventually, the list boils down to 2: the winner county and the runner up. Sure – the
winner is not randomly chosen. But the runner up should give a much better control
group than the average county.
cfr. with Synthetic Control Methods [A. Abadie 2003/07]
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
Example: how BMW picked the location for one of its plants?
Worldwide competition considering 250 potential sites => announced in 1991 that
list narrowed to 20 U.S. Candidates => 6 months later BMW announced two
finalists (Greenville-Spartanburg, South Carolina, and Omaha, Nebraska) =>
1992, BMW announced Greenville-Spartanburg won
BMW received a package of incentives worth approximately $115 million funded
by the state and local governments.
Why did BMW choose Greenville-Spartanburg?
1. BMW’s expected future costs of production in GS: according to BMW, the
characteristics that made GS were: low union density; supply of qualified workers;
numerous global firms in the area, including 58 German companies; high quality
transportation infrastructure, including air, rail, highway, and port, access to key
local services.
2. subsidy [Not random at all!] => this could be a big issue, no?
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
METHODOLOGICAL NOTE:
information comes from a journal (Site Selection). General point here: there is often a lot
(really a lot!) of information in business directories, specialized trade journals, etc. Often
this information is in non-anonymous format (i.e., with names) and – though hard to
collect – can potentially be matched with administrative records.
THIS IS SOMETHING THAT DEVELOPMENT ECONOMISTS INTERESTED IN FIRMS
AND INDUSTRIAL DEVELOPMENT OUGHT TO EXPLOIT MORE !
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
The paper does three things:
1. test for spillover. Compare productivity of incumbent plants in winner and loser
county before and after the opening of the MDP. Essentially a diff-in-diff
- how is TFP estimated? Easy. More on that tomorrow.
- they find large effects (12-15% after 5 years)
- but these are highly heterogeneous across counties (some get <0)
2. test for mechanisms: are spillovers larger in related industries?
3. are productivity gains reflected in higher input prices?
Paper develops a very simple theoretical framework – we skip it in the interest of
time. But it is quite helpful in highlighting a few issues.
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
Back to the paper: do you believe the identification strategy?
Note: the winner and loser county should look similar after controlling for many
things that can be controlled in a regression, i.e., plant fixed effects, industryyear fixed effects.
Note: focus on pre-existing plants [overestimate?]
Note: some (e.g., delays) are excluded. This gives 47 usable MDP.
So, what about inference? St. Err. always clustered at the county level [what
does that mean?]
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
bla bla bla
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
bla bla bla
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
Specification comes from a simple model
Note: pure did has no change in trends ...
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Greenstone, Hornbeck and Moretti (2008)
Couple of issues that would be nice to see discussed:
- focus on impact of very large plants. What does it say about
cluster of SMEs?
- what about the subsidy? Think about the incentives of the
local gvt. to give a subsidy: which implications does this have on
the interpretation of the effects?
» unfortunately, these are unobserved.
- can you think of other?
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Conclusions:
1. There are various types of shocks that you might want to
study for a variety of reasons. The key issues to be addressed
depend to on the nature of the shock (SR vs LR, permanent vs
temporary, anticipate or not) and on what you want to study
2. There are plenty of policies that are routinely done in
developing countries that cry to be evaluated:
- does the policy work?
- can we use policy changes to identify parameters of interest?
3. Often data to evaluate these policies already exist and are
collected by government agencies, business associations,
industry publications, often precisely because of the policy
4. Eventually, lot of detailed institutional knowledge is required
(names must be matched, characteristics of product,
environment, industry etc.) => this requires “case studies”
But, ! problem with case studies !
Macchiavello
Natural Experiments and Firms
July 2010
Plan
Natural Experiments & Review
Examples 1,2 & 3
Further Ideas
Conclusions
Ok –
This was in large part sloppy and rushed. I hope you take away
a list of things to think about before you start project / data
collection.
The whole point is that even w/out experimental data a lot of
the work has to be done at the research design stage even if
you do Industrial Organization.
Necessary (but not sufficient) conditions for good paper/presentation:
What is the question (& Why should I care)?
How do I answer it?
What is the answer?
Macchiavello
Natural Experiments and Firms
July 2010