Financial Markets and Human Capital
Unit 02: Empirical Methods
Ernst Maug
University of Mannheim
http://cf.bwl.uni-mannheim.de
ernst.maug@uni-mannheim.de
Secretary: Angelika Wolf-Tobaben
cf.secretary@uni-mannheim.de
Tel: +49 (621) 181-1951
Difference-in-differences analyses
→What is a diff-in-diff analysis?
- A statistical method to analyze the causal effect of a treatment on an outcome variable.
→Idea: Emulate a randomized control trial (RCT) as in medicine, psychology, etc.
- For example, test a new drug against a certain type of cancer
- Give the drug to 100 patients (“treated” group)
- Give a placebo to another 100 patients (“control” group) e.x.: standard medicine for cancer
- Nobody knows in which group they are, nobody can select
- Both groups are as similar as possible in terms of gender, age, history of diseases, etc. (hence, “random”)
- Assumption: if the “treated” group were to receive the placebo, they would develop exactly like the control group
© 2022 Ernst Maug
Financial Markets and Human Capital: Internal Labor Markets
2
Quasi-random experiments
→How would we do this in business, finance, economics
- Have planning authority assign firms to “treatment” and “control” group
- Make sure treated and control group are the same, no self-selection!
- Prescribe treatment (e.g., merger, divestiture, investment,…)
- It simply can’t be done…
→Solution: Exploit quasi-experiments, events with differential impact on otherwise comparable samples:
▪ Labor protection legislation in some states but not in the others.
▪ Cash windfalls to some firms but not to the others.
▪ Failure of main banks for some firms but not for the others
→Measure difference between “treated” group (subject to event) and “control” group (not subject to event)
© 2022 Ernst Maug
Financial Markets and Human Capital: Internal Labor Markets
3
Difference-in-differences: recall study on AWS shock from unit 1
→Compare the treated-control difference
- before the event to
- after the event
- hence, difference (after minus before) of differences (treated minus control)
→Study on impact of AWS shock on capital requirements of start-ups
- Treated group: firms with high computing requirements (proxied by membership in 8 industries)
- Control group: firms with lower computing requirements (proxied by membership in 18 industries)
- Before = before 2006 (year Amazon Web Services became available to start-ups)
- After = after 2006
→Compute:
𝑇𝑟𝑒𝑎𝑡𝑒𝑑
𝐶𝑜𝑛𝑡𝑟𝑜𝑙
- Difference in start-up capital before 2006: ∆𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐵𝑒𝑓𝑜𝑟𝑒 = 𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐵𝑒𝑓𝑜𝑟𝑒
− 𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐵𝑒𝑓𝑜𝑟𝑒
𝑇𝑟𝑒𝑎𝑡𝑒𝑑
𝐶𝑜𝑛𝑡𝑟𝑜𝑙
- Difference in start-up capital after 2006: ∆𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐴𝑓𝑡𝑒𝑟 = 𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐴𝑓𝑡𝑒𝑟
− 𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐴𝑓𝑡𝑒𝑟
- Difference in differences:
© 2022 Ernst Maug
DD = ∆𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐴𝑓𝑡𝑒𝑟 − ∆𝐶𝑎𝑝𝑖𝑡𝑎𝑙𝐵𝑒𝑓𝑜𝑟𝑒
Financial Markets and Human Capital: Internal Labor Markets
4
Diff-in-diff in a graph
→Hypothesis: if the AWS shock had not occurred,
start-up capital requirements in the treated group
would have developed exactly as those in the
control group (remember RCT!)
→This does not mean they would have been the
same.
→Only that their developments would have
continued on the same paths as before
→Assumption: deviations from pre-AWS path of
treated group can be attributed to AWS shock
Outcome
variable:
Start-up
capital
AWS becomes available
Control group
Treatment
effect
Treatment group
Timeline
before 2006
© 2022 Ernst Maug
Financial Markets and Human Capital: Internal Labor Markets
after 2006
5
Regression analysis
→This can be analyzed using standard OLS regressions
→Let
𝐶
- 𝐶𝑎𝑝𝐵𝑒𝑓
: capital requirements of control (C) group before (t<2006) AWS shock
𝑇
- 𝐶𝑎𝑝𝐵𝑒𝑓
: capital requirements of treated (T) group before (t<2006) AWS shock
𝐶
- 𝐶𝑎𝑝𝐴𝑓𝑡
: capital requirements of control (C) group after (t≥2006) AWS shock
𝑇
- 𝐶𝑎𝑝𝐴𝑓𝑡
: capital requirements of treated (T) group after (t ≥ 2006) AWS shock
→Then
- 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡 = ∆𝐶𝑎𝑝𝑇 − ∆𝐶𝑎𝑝𝐶
𝑇
𝑇
𝐶
𝐶
= 𝐶𝑎𝑝𝐴𝑓𝑡
− 𝐶𝑎𝑝𝐵𝑒𝑓
− 𝐶𝑎𝑝𝐴𝑓𝑡
− 𝐶𝑎𝑝𝐵𝑒𝑓
© 2022 Ernst Maug
Financial Markets and Human Capital: Internal Labor Markets
6
Diff-in-diff regressions
E-C= beta difference
= conterfactual change
B-A
= actual change in control
D-E = delta
treatment effect ( or change of treated- change of control)
D-C = beta+ delta
= change in the treated
in step 4 after =1 and treatment = 1 but after*treatment = 0
> pretend that the shock had no impact bay setting condition 2
> hypothetical
→Run regression:
𝐶𝑎𝑝𝑡𝑖 = 𝛼 + 𝛽 × 𝐴𝑓𝑡𝑒𝑟𝑡𝑖 +𝛾 × 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑡𝑖 +𝛿 × 𝐴𝑓𝑡𝑒𝑟𝑡𝑖 × 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑡𝑖 +𝑢𝑡𝑖
where:
dummy variables (0,1)
𝐴𝑓𝑡𝑒𝑟𝑡𝑖 = 1 if t ≥ 2006, zero otherwise
𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑡𝑖 = 1 if 𝑖 = high internet exposure firm,
zero otherwise
AWS becomes available
B
→Then the coefficient estimates can be interpreted as follows:
𝛼: average of control group before AWS shock
𝛽: average change in control group (placebo effect)
𝛾: average difference treated minus controls before AWS shock
𝛿: treatment effect
→ Advantage: flexibility
- can add further control variables and fixed effects to regression
E
𝛽
Control group
A
𝛾
C
D
Treatment group
before 2006
© 2022 Ernst Maug
Financial Markets and Human Capital: Internal Labor Markets
𝛿
𝛼
after 2006
7
Diff-in-diff regressions: What the coefficients mean (supplemental material)
𝐶𝑎𝑝𝑡𝑖 = 𝛼 + 𝛽 × 𝐴𝑓𝑡𝑒𝑟𝑡𝑖 +𝛾 × 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑡𝑖 +𝛿 × 𝐴𝑓𝑡𝑒𝑟𝑡𝑖 × 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑡𝑖 +𝑢𝑡𝑖
→Subsamples
- Subsample A: Control group before the shock (After = 0, Treatment = 0)
▪ 𝐶𝑎𝑝𝑡𝑖 = 𝛼 + 0 + 0 + 0 + 𝑢𝑡𝑖
- Subsample B: Control group after the shock (After = 1, Treatment = 0)
▪ 𝐶𝑎𝑝𝑡𝑖 = 𝛼 + 𝛽 + 0 + 0 + 𝑢𝑡𝑖
- Subtract A from B: 𝐁 − 𝐀 = 𝛼 + 𝛽 − 𝛼 = 𝛽 = Impact of After on controls.
- Subsample C: Treated group before the shock (After = 0, Treatment = 1)
▪ 𝐶𝑎𝑝𝑡𝑖 = 𝛼 + 0 + γ + 0 + 𝑢𝑡𝑖
- Subtract A from C: C − A = 𝛼 + 𝛾 − 𝛼 = 𝛾 = Difference treated minus controls before
- Subsample D: Control group after the shock (After = 1, Treatment = 1)
▪ 𝐶𝑎𝑝𝑡𝑖 = 𝛼 + 𝛽 + 𝛾 + 𝛿 + 𝑢𝑡𝑖
- Subtract C from D: 𝐃 − 𝐂 = 𝛼 + 𝛽 + 𝛾 + 𝛿 − 𝛼 + γ = 𝛽 + 𝛿 = Impact of After on the treated.
- Treatment effect = (D – C) – (B – A): 𝛽 + 𝛿 − 𝛽 = 𝛿 =Impact of After on treated minus Impact of
After on controls
© 2022 Ernst Maug
Financial Markets and Human Capital: Internal Labor Markets
8
Regression results
All VCs
Treatment × After
𝛿
Treatment
𝛾
Syndicate size
Startup based in CA
Startup based in MA
Startup based in NY
After
𝛽
(1)
-0.294***
(0.0663)
-0.0293
(0.0504)
0.323***
(0.0200)
0.438***
(0.0773)
0.465***
(0.0797)
0.276***
(0.0743)
0.0741*
(0.0435)
(2)
-0.269***
(0.0673)
-0.0338
(0.0504)
0.325***
(0.0197)
0.424***
(0.0766)
0.453***
(0.0792)
0.290***
(0.0740)
0.179***
(0.0665)
AWS becomes available
β = 0.179
Control group
γ = -0.338
Treatment group
before 2006
δ = -0.269
α
after 2006
/ ** / *** = p values smaller than 10% / 5% / 1%
(2) includes more control variables than (1)
numbers refer to regression (2)
two/three stars, depends on the number of beta hat* #standard deviation away from beta
Standard deviations are in parentheses, as defined in the event study part. is the 0 inside or outside of the confidence interval
From Ewens, Nanda, and Rhodes-Kopf (2017), Table III
© 2022 Ernst Maug
if we are in > we cannot reject H0 (more or less equal to 0)
if we are out > we can reject H0 ("significantly different" so not equal to 0)
the greater the number of set > teh smaller p > the more precise (smaller confidence interval )
Financial Markets and Human Capital: Internal Labor Markets
9
Postscript and summary
importance of good estimates and good research design
during the assignement
→Event studies are a special type of diff-in-diff analysis
- Outcome variable = stock return
- Exogenous shock = event
- Treated units = events
- Control units = market index
→Event studies and diff-in-diff analyses allow us to evaluate observational data as quasi-random experiments
- Event / shock needs to be exogenous (no self selection)
- Treated and controls have to be similar except for the event / shock
→Powerful tools, but need to be handled with care!
© 2022 Ernst Maug
Financial Markets and Human Capital: Internal Labor Markets
10