Endogenous Regressors and Instrumental Variables Estimation Adapted from Vera Tabakova, East Carolina University 10.1 Linear Regression with Random x’s 10.2 Cases in which x and e are Correlated 10.3 Estimators Based on the Method of Moments 10.4 Specification Tests Show Kennedy;s graphs and explain “regression” Principles of Econometrics, 3rd Edition Slide 10-2 The assumptions of the simple linear regression are: SR1. yi 1 2 xi ei i 1, , N SR2. E (ei ) 0 SR3. var(ei ) 2 SR4. cov(ei , e j ) 0 SR5. The variable xi is not random, and it must take at least two different values. SR6. (optional) ei ~ N (0, 2 ) Principles of Econometrics, 3rd Edition Slide 10-3 The purpose of this chapter is to discuss regression models in which xi is random and correlated with the error term ei. We will: Discuss the conditions under which having a random x is not a problem, and how to test whether our data satisfies these conditions. Present cases in which the randomness of x causes the least squares estimator to fail. Provide estimators that have good properties even when xi is random and correlated with the error ei. Principles of Econometrics, 3rd Edition Slide 10-4 A10.1 yi 1 2 xi ei correctly describes the relationship between yi and xi in the population, where β1 and β2 are unknown (fixed) parameters and ei is an unobservable random error term. A10.2 The data pairs xi , yi i 1, , N , are obtained by random sampling. That is, the data pairs are collected from the same population, by a process in which each pair is independent of every other pair. Such data are said to be independent and identically distributed. Principles of Econometrics, 3rd Edition Slide 10-5 A10.3 E ei | xi 0. The expected value of the error term ei, conditional on the value of xi, is zero. This assumption implies that we have (i) omitted no important variables, (ii) used the correct functional form, and (iii) there exist no factors that cause the error term ei to be correlated with xi. If E ei | xi 0 , then we can show that it is also true that xi and ei are uncorrelated, and that cov xi , ei 0 . Conversely, if xi and ei are correlated, then show that E ei | xi 0 . Principles of Econometrics, 3rd Edition cov xi , ei 0 and we can Slide 10-6 A10.4 In the sample, xi must take at least two different values. A10.5 var ei | xi 2 . The variance of the error term, conditional on xi is a constant σ2. A10.6 ei | xi ~ N 0, 2 . The distribution of the error term, conditional on xi, is normal. Principles of Econometrics, 3rd Edition Slide 10-7 Under assumptions A10.1-A10.4 the least squares estimator is unbiased. Under assumptions A10.1-A10.5 the least squares estimator is the best linear unbiased estimator of the regression parameters, conditional on the x’s, and the usual estimator of σ2 is unbiased. Principles of Econometrics, 3rd Edition Slide 10-8 Under assumptions A10.1-A10.6 the distributions of the least squares estimators, conditional upon the x’s, are normal, and their variances are estimated in the usual way. Consequently the usual interval estimation and hypothesis testing procedures are valid. Principles of Econometrics, 3rd Edition Slide 10-9 Figure 10.1 An illustration of consistency Principles of Econometrics, 3rd Edition Slide 10-10 Remark: Consistency is a “large sample” or “asymptotic” property. We have stated another large sample property of the least squares estimators in Chapter 2.6. We found that even when the random errors in a regression model are not normally distributed, the least squares estimators still have approximate normal distributions if the sample size N is large enough. How large must the sample size be for these large sample properties to be valid approximations of reality? In a simple regression 50 observations might be enough. In multiple regression models the number might be much higher, depending on the quality of the data. Principles of Econometrics, 3rd Edition Slide 10-11 A10.3* E ei 0 and cov xi , ei 0 E ei | xi 0 cov xi , ei 0 E ei | xi 0 E ei 0 Principles of Econometrics, 3rd Edition Slide 10-12 Under assumption A10.3* the least squares estimators are consistent. That is, they converge to the true parameter values as N. Under assumptions A10.1, A10.2, A10.3*, A10.4 and A10.5, the least squares estimators have approximate normal distributions in large samples, whether the errors are normally distributed or not. Furthermore our usual interval estimators and test statistics are valid, if the sample is large. Principles of Econometrics, 3rd Edition Slide 10-13 If assumption A10.3* is not true, and in particular if cov xi , ei 0 so that xi and ei are correlated, then the least squares estimators are inconsistent. They do not converge to the true parameter values even in very large samples. Furthermore, none of our usual hypothesis testing or interval estimation procedures are valid. Principles of Econometrics, 3rd Edition Slide 10-14 Figure 10.2 Plot of correlated x and e Principles of Econometrics, 3rd Edition Slide 10-15 y E y e 1 2 x e 1 1 x e True model, but it gets estimated as: yˆ b1 b2 x .9789 1.7034 x …so there is s substantial bias … Principles of Econometrics, 3rd Edition Slide 10-16 Example Wages = f(intelligence) In the case of a positive correlation between x and the error Figure 10.3 Plot of data, true and fitted regressions Principles of Econometrics, 3rd Edition Slide 10-17 When an explanatory variable and the error term are correlated the explanatory variable is said to be endogenous and means “determined within the system.” Principles of Econometrics, 3rd Edition Slide 10-18 yi 1 2 xi* vi xi xi* ui X Y u v Principles of Econometrics, 3rd Edition (10.1) (10.2) e Slide 10-19 yi 1 2 xi* vi 1 2 xi ui vi 1 2 xi vi 2ui (10.3) 1 2 xi ei Principles of Econometrics, 3rd Edition Slide 10-20 cov xi , ei E xi ei E xi* ui vi 2ui E 2ui2 2u2 0 Principles of Econometrics, 3rd Edition (10.4) Slide 10-21 We will focus on this case WAGEi 1 2 EDUCi ei (10.5) Omitted factors: experience, ability and motivation. Therefore, we expect that cov EDUCi , ei 0. X Y W Principles of Econometrics, 3rd Edition Slide 10-22 We will focus on this case Other examples: Y = crime, X = marriage, W = “marriageability” Y = divorce, X = “shacking up”, W = “good match” Y = crime, X = watching a lot of TV, W = “parental involvement” Y = sexual violence, X = watching porn, W = any unobserved anything that would affect both W and Y The list is endless! X . Y W Principles of Econometrics, 3rd Edition Slide 10-23 Qi 1 2 Pi ei (10.6) There is a feedback relationship between Pi and Qi. Because of this feedback, which results because price and quantity are jointly, or simultaneously, determined, we can show that cov( Pi , ei ) 0. The resulting bias (and inconsistency) is called the simultaneous equations bias. Principles of Econometrics, 3rd Edition X Y Slide 10-24 yt 1 2 yt 1 3 xt et AR(1) process: et et 1 vt If 0 there will be correlation between yt 1 and et . In this case the least squares estimator applied to the lagged dependent variable model will be biased and inconsistent. Principles of Econometrics, 3rd Edition Slide 10-25 When all the usual assumptions of the linear model hold, the method of moments leads us to the least squares estimator. If x is random and correlated with the error term, the method of moments leads us to an alternative, called instrumental variables estimation, or two-stage least squares estimation, that will work in large samples. Principles of Econometrics, 3rd Edition Slide 10-26 Suppose that there is another variable, z, such that z does not have a direct effect on y, and thus it does not belong on the right-hand side of the model as an explanatory variable ONCE X IS IN THE MODEL! zi should not be simultaneously affected by y either, of course zi is not correlated with the regression error term ei. Variables with this property are said to be exogenous z is strongly [or at least not weakly] correlated with x, the endogenous explanatory variable A variable z with these properties is called an instrumental variable. Slide 10-27 Principles of Econometrics, 3rd Edition An instrument is a variable z correlated with x but not with the error e In addition, the instrument does not directly affect y and thus does not belong in the actual model as a separate regressor (of course it should affect it through the instrumented regressor x) It is common to have more than one instrument for x (just not good ones!) These instruments, z1; z2; : : : ; zs, must be correlated with x, but not with e Consistent estimation is obtained through the instrumental variables or two-stage least squares (2SLS) estimator, rather than the usual OLS estimator Principles of Econometrics, 3rd Edition Slide 10-28 Using Z, an “instrumental variable” for X is one solution to the problem of omitted variables bias Z, to be a valid instrument for X must be: Z X W Y e – Relevant = Correlated with X – Exogenous = Not correlated with Y except through its correlation with X E ziei 0 E zi yi 1 2 xi 0 (10.16) 1 yi ˆ 1 ˆ 2 xi 0 N (10.17) 1 zi yi ˆ 1 ˆ 2 xi 0 N Based on the method of moments… Principles of Econometrics, 3rd Edition Slide 10-30 Solving the previous system, we obtain a new estimator that cleanses the endogeneity of X and exploits only the component of the variation of X that is not correlated with e instead: ˆ N zi yi zi yi 2 N zi xi zi xi zi z yi y zi z xi x (10.18) ˆ 1 y ˆ 2 x We do that by ensuring that we only use the predicted value of X from its regression on Z in the main regression Principles of Econometrics, 3rd Edition Slide 10-31 These new estimators have the following properties: They are consistent, if E zi ei 0. In large samples the instrumental variable estimators have approximate normal distributions. In the simple regression model 2 ˆ 2 ~ N 2 , 2 2 r x x zx i Principles of Econometrics, 3rd Edition (10.19) Slide 10-32 The error variance is estimated using the estimator ˆ 2IV yi ˆ 1 ˆ 2 xi 2 N 2 For: 2 ˆ 2 ~ N 2 , 2 2 r x x zx i (10.19) The stronger the correlation between the instrument and X the better! Principles of Econometrics, 3rd Edition Slide 10-33 var ˆ 2 2 2 zx r xi x 2 var b2 rzx2 Using the instrumental variables is less efficient than OLS (because you must throw away some information on the variation of X) so it leads to wider confidence intervals and less precise inference. The bottom line is that when instruments are weak, instrumental variables estimation is not reliable: you throw away too much information when trying to avoid the endogeneity bias Principles of Econometrics, 3rd Edition Slide 10-34 Not all of the available variation in X is used Only that component of X that is “explained” by Z is used to explain Y X Y Z X = Endogenous variable Y = Response variable Z = Instrumental variable X Y Best-case scenario: A lot of X is explained by Z, and most of the overlap between X and Y is accounted for Y Realistic scenario: Very little of X is explained by Z, and/or what is explained does not overlap much with Y Z X Z The IV estimator is BIASED E(bIV) ≠ β (finite-sample bias) but consistent: E(b) → β as N → ∞ So IV studies must often have very large samples But with endogeneity, E(bLS) ≠ β and plim(bLS) ≠ β anyway… Asymptotic behavior of IV: plim(bIV) = β + Cov(Z,e) / Cov(Z,X) If Z is truly exogenous, then Cov(Z,e) = 0 Three different models to be familiar with First stage: EDUC = α0 + α1Z + ω (‘reduced form equation’) Structural model: WAGES = β0 + β1EDUC + ε Also: WAGES = δ0 + δ1Z + ξ ω An interesting equality: δ1 = α1 × β1 Z α1 X ε β1 Y so… β1 = δ1 / α1 ξ Z δ1 Y xˆ .1947 .5700 z1 .2068 z2 (se) (.079) (.089) (.077) yˆ IV _ z1 , z2 1.1376 1.0399 x (se) Principles of Econometrics, 3rd Edition (.116) (.194) (10.26) (10.27) Slide 10-39 yˆOLS .9789 1.7034 x yˆ IV _ z1 1.1011 1.1924 x (se) (.088) (.090) (se) yˆ IV _ z2 1.3451 .1724 x yˆ IV _ z3 .9640 1.7657 x (se) (se) (.256) (.797) Principles of Econometrics, 3rd Edition (.109) (.195) (.095) (.172) Slide 10-40 ln WAGE 1 2 EDUC 3EXPER 4 EXPER2 e ln WAGE = .5220 .1075 EDUC .0416 EXPER .0008 EXPER 2 (se) (.1986) (.0141) Principles of Econometrics, 3rd Edition (.0132) (.0004) Slide 10-41 EDUC 9.7751 .0489 EXPER .0013 EXPER 2 .2677 MOTHEREDUC (se) (.4249) (.0417) (.0012) (.0311) We hope for a high t ratio here ln WAGE .1982 .0493 EDUC .0449 EXPER .0009 EXPER 2 (se) (.4729) (.0374) (.0136) (.0004) Check it out: as expected much lower than from OLS!!! Principles of Econometrics, 3rd Edition Slide 10-42 open "@gretldir\data\poe\mroz.gdt" logs wage square exper list x = const educ exper sq_exper list z = const exper sq_exper mothereduc # least squares and IV estimation of wage eq ols l_wage x tsls l_wage x ; z # tsls--manually smpl wage>0 --restrict ols educ z series educ_hat = $yhat ols l_wage const educ_hat exper sq_exper Include in z all G exogenous variables and the instruments available Exogenous variables Instrument themselves! But check that the latter will give you wrong s.e. so not reccommended Principles of Econometrics, 3rd Edition 43 In econometrics, two-stage least squares (TSLS or 2SLS) and instrumental variables (IV) estimation are often used interchangeably The `two-stage' terminology comes from the time when the easiest way to estimate the model was to actually use two separate least squares regressions With better software, the computation is done in a single step to ensure the other model statistics are computed correctly Unfortunately GRETL outputs R2 for the IV regression; you should ignore it! It is based on the correlation of y and yhat, so it does not have the usual interpretation… Principles of Econometrics, 3rd Edition 44 A 2-step process. Regress x on a constant term, z and all other exogenous variables G, and obtain the predicted values xˆ . Use xˆ as an instrumental variable for x. Principles of Econometrics, 3rd Edition Slide 10-45 Two-stage least squares (2SLS) estimator: Stage 1 is the regression of x on a constant term, z and all other exogenous variables G, to obtain the predicted values xˆ . This first stage is called the reduced form model estimation. Stage 2 is ordinary least squares estimation of the simple linear regression yi 1 2 xˆi errori Principles of Econometrics, 3rd Edition (10.23) Slide 10-46 2 var ˆ 2 2 ˆ xi x ˆ 2IV yi ˆ 1 ˆ 2 xi Principles of Econometrics, 3rd Edition 2 N 2 2 ˆ IV var ˆ 2 2 xˆi x (10.24) (10.25) Slide 10-47 If we regress education on all the exogenous variables and the TWO instruments: Rule of thumb These look promising for strong instruments: F>10 In fact: F test says Null hypothesis: the regression parameters are zero for the variables mothereduc, fathereduc Test statistic: F(2, 423) = 55.4003, p-value 4.26891e-022 Principles of Econometrics, 3rd Edition Slide 10-48 ln WAGE = .0481 .0614EDUC .0442 EXPER .0009 EXPER2 (se) (.4003) (.0314) (.0134) (.0004) With the additional instrument, we achieve a significant result in this case Principles of Econometrics, 3rd Edition Slide 10-49 Is a bit more complex, but the idea is that you need at least as many Instruments as you have endogenous variables You cannot use the F test we just saw to test for the weakness of the instruments, but see Appendix for a test based on the notion of partial correlations For example, run this experiment with the mroz.gdt data: # partial correlations--the FWL result ols educ const exper sq_exper series e1 = $uhat ols mothereduc const exper sq_exper series e2 = $uhat ols e1 e2 corr e1 e2 Principles of Econometrics, 3rd Edition Slide 10-50 it is the independent correlation between the instrument and the endogenous regressor that is important. We can partial out the correlation in X that is due to the exogenous regressors. Whatever common variation that remains will measure the independent correlation between X and the instrument Z: we ask ourselves how much of the independent variation of educ is captured by the independent variation in mothereduc? Partial correlations For example, run this experiment with the mroz.gdt data: # partial correlations--the FWL result ols educ const exper sq_exper series e1 = $uhat ols mothereduc const exper sq_exper series e2 = $uhat ols e1 e2 corr e1 e2 And compare your results with ols educ const mothereduc exper sq_exper Principles of Econometrics, 3rd Edition Slide 10-51 When testing the null hypothesis H 0 : k c use of the test statistic t ˆ c se ˆ is valid in large samples. It is common, but not k k universal, practice to use critical values, and p-values, based on the Student-t distribution rather than the more strictly appropriate N(0,1) distribution. The reason is that tests based on the t-distribution tend to work better in samples of data that are not large. Principles of Econometrics, 3rd Edition Slide 10-52 When testing a joint hypothesis, such as H 0 : 2 c2 , 3 c3 , the test may be based on the chi-square distribution with the number of degrees of freedom equal to the number of hypotheses (J) being tested. The test itself may be called a “Wald” test, or a likelihood ratio (LR) test, or a Lagrange multiplier (LM) test. These testing procedures are all asymptotically equivalent Another complication: you might need to use tests that are robust to heteroskedasticity and/or autocorrelation… Principles of Econometrics, 3rd Edition Slide 10-53 y 1 2 x e eˆ y ˆ ˆ x 1 2 R 1 eˆ 2 2 i yi y 2 Unfortunately R2 can be negative when based on IV estimates. Therefore the use of measures like R2 outside the context of the least squares estimation should be avoided (even if GRETL produces one!) Principles of Econometrics, 3rd Edition Slide 10-54 Can we test for whether x is correlated with the error term? This might give us a guide of when to use least squares and when to use IV estimators. TEST OF EXOGENEITY: (Durbin-Wu) HAUSMAN TEST Can we test whether our instrument is sufficiently strong to avoid the problems associated with “weak” instruments? TESTS FOR WEAK INSTRUMENTS Can we test if our instrument is valid, and uncorrelated with the regression error, as required? SOMETIMES ONLY WITH A SARGAN TEST Principles of Econometrics, 3rd Edition Slide 10-55 H0 : cov xi , ei 0 H1 : cov xi , ei 0 If the null hypothesis is true, both OLS and the IV estimator are consistent. If the null hypothesis holds, use the more efficient estimator, OLS. If the null hypothesis is false, OLS is not consistent, and the IV estimator is consistent, so use the IV estimator. Principles of Econometrics, 3rd Edition Slide 10-56 yi 1 2 xi ei Let z1 and z2 be instrumental variables for x. 1. Estimate the model xi 1 1zi1 2 zi 2 vi by least squares, and obtain the residuals vˆ x ˆ ˆ z ˆ z . If there are more than i i 1 1 i1 2 i2 one explanatory variables that are being tested for endogeneity, repeat this estimation for each one, using all available instrumental variables in each regression. Principles of Econometrics, 3rd Edition Slide 10-57 2. Include the residuals computed in step 1 as an explanatory variable in the original regression,yi 1 2 xi vˆi ei . Estimate this "artificial regression" by least squares, and employ the usual t-test for the hypothesis of significance H 0 : 0 no correlation between xi and ei H1 : 0 correlation between xi and ei Principles of Econometrics, 3rd Edition Slide 10-58 3. If more than one variable is being tested for endogeneity, the test will be an F-test of joint significance of the coefficients on the included residuals. Note: This is a test for the exogeneity of the regressors xi and not for the exogeneity of the instruments zi. If the instruments are not valid, the Hausman test is not valid either. Principles of Econometrics, 3rd Edition Slide 10-59 # GRETL Hausman test (manually) ols educ z --quiet series ehat = $uhat ols l_wage x ehat Note: whenever you use 2SLS GRETLautomatically produces the test statistic for the Hausman test. There are several different ways of computing it, so don't worry if it differs from the one you compute manually using the above script Principles of Econometrics, 3rd Edition Slide 10-60 y 1 2 x2 G xG G1xG1 e xG1 1 2 x2 G xG 1z1 v If we have L > 1 instruments available then the reduced form equation is xG1 1 2 x2 Principles of Econometrics, 3rd Edition G xG 1z1 L zL v Slide 10-61 y 1 2 x2 G xG G1xG1 e xG1 1 2 x2 G xG 1z1 v Then test with an F test whether the instruments help to determine the value of the endogenous variable. Rule of thumb F > 10 for one endogenous regressor xG1 1 2 x2 Principles of Econometrics, 3rd Edition G xG 1z1 L zL v Slide 10-62 As we saw before: open "@gretldir\data\poe\mroz.gdt" smpl wage>0 --restrict logs wage square exper list x = const educ exper sq_exper list z2 = const exper sq_exper mothereduc fathereduc ols educ z2 omit mothereduc fathereduc Principles of Econometrics, 3rd Edition Slide 10-63 We want to check that the instrument is itself exogenous…but you can only do it for surplus instruments if you have them (overidentified equation)… 1. ˆ using all available instruments, including the Compute the IV estimates k G variables x1=1, x2, …, xG that are presumed to be exogenous, and the L instruments z1, …, zL. 2. Obtain the residuals Principles of Econometrics, 3rd Edition eˆ y ˆ 1 ˆ 2 x2 ˆ K xK . Slide 10-64 3. Regress eˆ on all the available instruments described in step 1. 4. Compute NR2 from this regression, where N is the sample size and R2 is the usual goodness-of-fit measure. 5. If all of the surplus moment conditions are valid, then NR2 ~ (2LB ) . If the value of the test statistic exceeds the 100(1−α)-percentile from 2 the ( L B ) distribution, then we conclude that at least one of the surplus moment conditions restrictions is not valid. Principles of Econometrics, 3rd Edition Slide 10-65 # Sargan test of overidentification tsls l_wage x; z2 series uhat2 = $uhat ols uhat2 z2 scalar test = $trsq pvalue X 1 test tsls l_wage x ; z2 You should learn about the Cragg-Uhler test if you end up having more than one endogenous variable Principles of Econometrics, 3rd Edition Slide 10-66 asymptotic properties conditional expectation endogenous variables errors-in-variables exogenous variables finite sample properties Hausman test instrumental variable instrumental variable estimator just identified equations large sample properties over identified equations population moments random sampling reduced form equation Principles of Econometrics, 3rd Edition sample moments simultaneous equations bias test of surplus moment conditions two-stage least squares estimation weak instruments Slide 10-67