SIMULTANEOUS EQUATIONS REGRESSIONS MODEL
INTRODUCTION
The classical linear regression model, general linear regression model, and seemingly unrelated
regressions model make the following assumption:
The error term is uncorrelated with each explanatory variable.
If an explanatory variable is uncorrelated with the error term it is called an exogenous variable.
If an explanatory variable is correlated with the error term it is called an endogenous variable.
The 3 most important sources that produce a correlation between the error term and an
explanatory variable are the following. 1) Omission of an important explanatory variable. 2)
Measurement error in an explanatory variable. 3) Reverse causation. We will focus on reverse
causation. One or more of the explanatory variables will be endogenous if the data are
generated by a process that is described by a simultaneous equations system. This occurs when
a change in a right-hand side variable causes a change in the left-hand side variable, and a
change in the left-hand side variable causes a change in the right-hand side variable.
Example
Consider the following simple Keynesian model of income determination comprised of two
equations: a consumption function, and equilibrium condition
C=a+bY+
Y=C+I
Where C is aggregate consumption; Y is aggregate income; I is exogenous investment; a and b
are parameters; and  is an error term that summarizes all factors other than Y that influence C
(e.g., wealth, interest rate). Now, suppose that  increases. This will directly increase C in the
consumption function. However, the equilibrium condition tells us that the increase in C will
increase Y. Therefore,  and Y are positively correlated.
INTRODUCTION TO THE SIMULTANEOUS EQUATONS REGRESSION MODEL
When a single equation is embedded in a system of simultaneous equations, at least one of the
right-hand side variables will be endogenous, and therefore the error term will be correlated
with at least one of the right-hand side variables. In this case, the true data generation process
is not described by the classical linear regression model, general linear regression model, or
seemingly unrelated regression model; rather, it is described by a simultaneous equations
regression model. If you use the OLS estimator, FGLS estimator, SUR estimator, or ISUR
estimator, you will get biased and inconsistent estimates of the population parameters.
Definitions and Basic Concepts
Endogenous variable – a variable whose value is determined within an equation system. The
values of
the endogenous variables are the solution of the equation system.
Exogenous variable – a variable whose value is determined outside an equation system.
Structural equation – an equation that has one or more endogenous right-hand side variables.
Reduced form equation – an equation for which all right-hand side variables are exogenous.
Structural parameters – the parameters of a structural equation.
Reduced form parameters – the parameters of a reduced form equation.
Specifying a Simultaneous Equation System
A simultaneous equation system is one of 4 important types of equation systems that are used
to specify statistical models in economics. The others are the seemingly unrelated equations
system, recursive equations system, and block recursive equation system. It is important to
know the difference between these 4 types of equation systems when specifying statistical
models of data generation processes.
The Identification Problem
Before you estimate a structural equation that is part of a simultaneous equation system, you
must first determine whether the equation is identified. If the equation is not identified, then
estimating its parameters is meaningless. This is because the estimates you obtain will have no
interpretation, and therefore will not provide any useful information.
Classifying Structural Equations
Every structural equation can be placed in one of the following three categories.
1. Unidentified equation – The parameters of an unidentified equation have no interpretation,
because you do not have enough information to obtain meaningful estimates.
2. Exactly identified equation – The parameters of an exactly identified equation have an
interpretation, because you have just enough information to obtain meaningful estimates.
3. Overidentified equation – The parameters of an overidentified equation have an
interpretation, because you have more than enough information to obtain meaningful
estimates.
Exclusion Restrictions
The most often used way to identify a structural equation is to use prior information provided
by economic theory to exclude certain variables from an equation that appear in a model. This
is called obtaining identification through exclusion restrictions. To exclude a variable from a
structural equation, you restrict the value of its coefficient to zero. This type of zero fixed value
restriction is called an exclusion restriction because it has the effect of omitting a variable from
the equation to obtain identification.
Rank and Order Condition for Identification
Exclusion restrictions are most often used to identify a structural equation in a simultaneous
equations model. When using exclusion restrictions, you can use two general rules to check if
identification is achieved. These are the rank condition and the order condition. The order
condition is a necessary but not sufficient condition for identification. The rank condition is
both a necessary and sufficient condition for identification. Because the rank condition is more
difficult to apply, many economists only check the order condition and gamble that the rank
condition is satisfied. This is usually, but not always the case.
Order Condition
The order condition is a simple counting rule that you can use to determine if one structural
equation in a system of linear simultaneous equations is identified. Define the following:
G = total number of endogenous variables in the model (i.e., in all equations that
comprise the
model).
K = total number of variables (endogenous and exogenous) excluded in the equation
being
checked for identification.
The order condition is as follows.
If
If
If
K=G–1
K>G–1
K<G–1
the equation is exactly identified
the equation is overidentified
the equation is unidentified
Rank Condition
The rank condition tells you whether the structural equation you are checking for identification
can be distinguished from a linear combination of all structural equations in the simultaneous
equation system. The procedure is as follows.
1. Construct a matrix for which each row represents one equation and each column represents
one variable in the simultaneous equations model.
2. If a variable occurs in an equation, mark it with an X. If a variable does not occur in an
equation, market it with a 0.
3. Delete the row for the equation you are checking for identification.
4. Form a new matrix from the columns that correspond to the elements that have zeros in
the row that you deleted.
5. For this new matrix, if you can find at least (G – 1) rows and columns that are not all zeros,
then the equation is identified. If you cannot, the equation is unidentified.
SPECIFICATION
A simultaneous equation regression model has two alternative specifications:
1. Reduced form
2. Structural form
The reduced-form specification is comprised of M reduced-form equations and a set of
assumptions about the error terms in the reduced form equations. The reduced-form
specification of the model is usually not estimated, because it provides limited information
about the economic process in which you are interested. The structural-form specification is
comprised of M structural equations and a set of assumptions about the error terms in the
structural equations. The structural-form specification of the model is the specification most
often estimated. This is because it provides more information about the economic process in
which you are interested.
Specification of the Structural Form of the Model
A set of assumptions defines the specification of the structural form of a simultaneous
equations regression model. The key assumption is that the error term is correlated with one or
more explanatory variables. There are several alternative specifications of the structural form
of the model depending on the remaining assumptions we make about the error term. For
example, if we assume that the error term has non-constant variance, then we have a
simultaneous equation regression model with heteroscedasticity. If we assume the errors in
one or more equations are correlated, then we have a simultaneous equation regression model
with autocorrelation. We will assume that the error term has constant variance, and the errors
not correlated within equations. However, we will allow errors to be contemporaneously
correlated across equations.
ESTIMATION
Single Equation Vs System Estimation
Two alternative approaches can be used to estimate a simultaneous equation regression
model.
1. Single equation estimation
2. System estimation
Single Equation Estimation
Single equation estimation involves estimating either one equation in the model, or two or
more equations in the model separately. For example, suppose you have a simultaneous
equation regression model that consists of two equations: a demand equation and a supply
equation. Suppose your objective is to obtain an estimate of the price elasticity of demand. In
this case, you might estimate the demand equation only. Suppose your objective is to obtain
estimates of price elasticity of demand and price elasticity of supply. In this case, you might
estimate the demand equation by itself and the supply equation by itself.
System Estimation
System estimation involves estimating two or more equations in the model jointly. For
instance, in the above example you might estimate the demand and supply equations together.
You might do this even if your objective is to obtain an estimate of the price elasticity of
demand only.
Advantages and Disadvantages of the Two Approaches
The major advantage of system estimation is that it uses more information, and therefore
results in more precise parameter estimates. The major disadvantages are that it requires
more data and is sensitive to model specification errors. The opposite is true for single
equation estimation.
Single Equation Estimators
We will consider 3 single equation estimators.
1. Ordinary least squares (OLS) estimator
2. Instrumental variables (IV) estimator
3. Two-stage least squares (2SLS) estimator
Each of these estimators is biased in small samples. Therefore, if the sample data are
generated by a simultaneous equation regression model you cannot find an estimator that has
desirable small sample properties. This means that you must look for an estimator that has
desirable asymptotic properties.
Ordinary Least Squares (OLS) estimator
The OLS estimator is given by the rule:
^ = (XTX)-1XTy
Properties of the OLS Estimator
If the sample data are generated by a simultaneous equation regression model, then the OLS
estimator is biased in small samples, and inconsistent in large samples. It does not produced
maximum likelihood estimates. Thus, it has undesirable small and large sample properties.
Role of OLS Estimator
The OLS estimator should be used as a preliminary estimator. You should initially estimate the
equation using the OLS estimator. You should then estimate the equation using a consistent
estimator. You can then compare the OLS estimate and consistent estimate of a parameter to
determine the direction of the bias.
Instrumental Variables (IV) Estimator
The IV estimator is given by the following two step procedure.
1. Find one instrumental variable for each right-hand side variable in the equation to be
estimated. A valid instrumental variable has two properties:
1. Instrument relevance. It is correlated with the variable for which it is to serve as an
instrument.
2. Instrument exogeneity. It is not correlated with the error term in the equation to be
estimated.
2. Apply the following formula to the sample data: IV^ = (ZTX)-1ZTy . Where X is the TxK data
matrix for the original right-hand side variables; Z is the TxK data matrix for the
instrumental variables; y is the Tx1 column vector of observations on the dependent
variable in the equation to be estimated.
Comments
Each exogenous right-hand side variable in the equation to be estimated can serve as its own
instrumental variable. This is because an exogenous variable is perfectly correlated with itself
and is not correlated with the error term in any equation by assumption of exogeneity. The best
candidates to be an instrumental variable for an endogenous right-hand side variable in the
equation to be estimated are exogenous variables that appear in other equations in the model.
This is because they are correlated with the endogenous variables in the model via the reducedform equations, but they are not correlated with the error term in any equation.
Often times there will exist more than one exogenous variable that can serve as an
instrumental variable for an endogenous variable. In this case, you can do one of two things. 1)
Use as your instrumental variable the exogenous variable that is most highly correlated with
the endogenous variable. 2) Use as your instrumental variable the linear combination of
candidate exogenous variables most highly correlated with the endogenous variable.
Relationship Between the IV Estimator and Identification
The following relationship exists between the IV estimator and identification.
1. If the equation to be estimated is exactly identified, then there are exactly enough
exogenous variables excluded from the equation to serve as instrumental variables for the
endogenous right-hand side variable(s).
2. If the equation to be estimated is overidentified, then there are more than enough
exogenous variables excluded from the equation to serve as instrumental variables for the
endogenous right-hand side variable(s).
3. If the equation to be estimated is unidentified, then there are not enough exogenous
variables excluded from the equation to serve as instrumental variables for the endogenous
right-hand side variable(s). In this case, the IV estimator cannot be used.
Properties of the IV Estimator
If the sample data are generated by a simultaneous equation regression model, then the IV
estimator is biased in small samples, but consistent in large samples. Thus, it has desirable
large sample properties. However, in the class of single equation estimators, the IV estimator is
not necessarily asymptotically efficient. This is because a given endogenous right-hand side
variable can have a number of possible instrumental variables, and each instrumental variable
results in a different IV estimator. While all such IV estimators are consistent, not all are
asymptotically efficient. The greater the correlation between the endogenous right-hand side
variable and its instrumental variable, the more efficient the IV estimator. Note that the IV
estimator does not produce maximum likelihood estimates.
Two-Stage Least Squares (2SLS) Estimator
The 2SLS estimator is a special type of IV estimator. It involves two successive applications of
the OLS estimator, and is given by the following two stage procedure.
1. Regress each right-hand side endogenous variable in the equation to be estimated on all
exogenous variables in the simultaneous equation model using the OLS estimator. Calculate
the fitted values for each of these endogenous variables.
2. In the equation to be estimated, replace each endogenous right-hand side variable by its
fitted value variable. Estimate the equation using the OLS estimator.
Comments
1. Stage 1 is identical to estimating the reduced-form equation for each endogenous righthand side variable in the equation to be estimated.
2. The estimated standard errors obtained from the stage 2 regression are incorrect and must
be corrected. Statistical programs that have a 2SLS procedure make this correction
automatically and report the correct standard errors.
3. The 2SLS estimator is the most popular single equation estimator, and one of the most
often used estimators in economics.
Properties of the 2SLS Estimator
If the error term is correlated with one or more explanatory variables, then the 2SLS estimator
is biased in small samples. However it is consistent and in the class of single equation
estimators asymptotically efficient. Thus, it has desirable large sample properties. Note that
the 2SLS estimator does not produce maximum likelihood estimates.
Monte Carlo studies Also suggest that under most conditions the 2SLS estimator has
better small sample properties than alternative single equation estimators.
Checking the Validity of the Instruments
If the instrumental variable(s) are uncorrelated with the endogenous explanatory variable, then
they are not relevant. If the instrumental variable(s) have a relatively low correlation with the
endogenous variable, then they are said to be weak instruments. If the instruments are
irrelevant or weak, then 2SLS will be inconsistent in large samples. Also, it will not have an
asymptotic normal distribution so that hypothesis tests will not be valid. If any instrumental
variable is correlated with the error term, then it is not exogenous. If any instrument variable is
not exogenous, then 2SLS will be inconsistent in large samples. If 2SLS is inconsistent, then it
will not produce an estimate that is close to the true value of the population parameter, even if
the sample size is large. Therefore, it is important to check the validity of your instrumental
variable(s).
Checking Instrument Relevance
To check for instrument relevance (strength of instruments), you calculate the F-statistic for the
null hypothesis that the coefficients of the variables used as instruments are all zero in the firststage regression. The bigger (smaller) the F-statistic, the stronger (weaker) the instrument(s).
An often used rule-of-thumb is that an F-statistic of less than 10 indicates possible weak
instruments. Why? It can be shown that the mean of the sampling distribution of the 2SLS
estimator in large samples is approximately:
(βOLS – β)
^2SLS
E(β
) = β + ————
[E(F) – 1]
where βOLS is the OLS estimator, (βOLS – β) is the bias in the OLS estimator, and E(F) is the
expected value of the F-statistic. Note that the expression 1 / [E(F) – 1] is the asymptotic bias in
β^2SLS relative to βOLS. The larger (smaller) the F-statistic, the smaller (larger) the bias in β^2SLS
relative to the βOLS. For example, if F =2 then, 1 / [E(F) – 1] = 1 / (2 – 1) = 1. In this case, the bias
in β^2SLS is the same as the bias in βOLS. If F = 3 then, 1 / [E(F) – 1] = 1 / (3 – 1) = ½. In this case,
the bias in β^2SLS is one-half (50%) of the bias in βOLS. If F = 10, then 1 / [E(F) – 1] = 1 / (10 – 1) =
1/9. In this case, the bias in β^2SLS is one-ninth (just over 10%) of the bias in βOLS. Many
econometricians believe that a bias of about 10% or less is small enough to be acceptable in
most applications.
Checking Instrument Exogeneity
To test whether one or more instruments is correlated with the error term, you can test the
overidentifying restrictions (if the equation is overidentified). This test is discussed below.
System Estimators
A system estimator can be use to estimate two or more identified equations in a simultaneous
equation model together. Thus, a system estimator uses more information than a single
equation estimator (e.g., contemporaneous correlation among the error terms across
equations, cross-equation restrictions, etc.), and therefore will produce more precise estimates.
We will consider 2 system estimators.
1. Three-stage least squares (3SLS) estimator
2. Iterated three-stage least squares (I3SLS) estimator
Three-Stage Least Squares (3SLS) Estimator
The I3SLS estimator involves the following 3 stage procedure.
1. Same as 2SLS.
2. Same as 2SLS.
3. Apply the SUR estimator.
Properties of the 3SLS Estimator
If the error term is correlated with one or more explanatory variables, then the 3SLS estimator
is biased in small samples. However it is consistent and asymptotically more efficient than
single equation estimators. Thus, it has desirable large sample properties.
Iterated Three-Stage Least Squares (I3SLS) Estimator
The I3SLS estimator involves the following 3 stage procedure.
1. Same as 2SLS.
2. Same as 2SLS.
3. Apply the SUR estimator.
Properties of the I3SLS Estimator
The I3SLS estimator has the same asymptotic properties as the 3SLS estimator. However, there
is an ongoing debate about whether I3SLS or 3SLS produces better estimates when using small
samples. A major advantage of the I3SLS estimator is that for a singular equation system it
produces parameter estimates that are invariant to the equation dropped. (See material on the
ISUR estimator). Econometricians seem to prefer the I3SLS estimator.
SPECIFICATON TESTING
Two important specification tests for simultaneous equations regression models are:
1. Test of Exogeneity
2. Test of overidentifying restrictions
We will implement these tests using a single equation estimation procedure.
Test of Exogeneity
If you believe one or more right-hand side variables appearing in an equation may or may not
be exogenous, then you can perform a formal test of exogeneity.
Notation
Designate the equation to be estimated and the identifying instruments as
Y = a + bY1 + cX + 
Z = identifying instruments
Where Y is the dependent (left-hand side) variable; Y1 is a vector of one or more right-hand side
variables that you believe may or may not be endogenous; X is a vector of right-hand side
variables you believe are exogenous; a is the intercept; b and c are vectors of slope coefficients
attached to the variables in Y1 and X, respectively; and Z is a vector of exogenous variables that
are excluded from this equation, and therefore used as identifying instruments for the
endogenous variable(s) in Y1; and  is the error term.
Hausman Test
The most often used test of exogeneity is the Hausman test. The Hausman test is based on the
following methodology. Let Y1 be interpreted more generally as a vector that contains one or
more variables that you believe may be correlated with the error term . The null and
alternative hypotheses are as follows:
H0: Y1 and  are not correlated
H1: Y1 and  are correlated
To test the null hypothesis that Y1 and  are not correlated, you proceed as follows.
1. Compare two estimators. One estimator should be a consistent estimator if the null
hypothesis is true but an inconsistent estimator if the null hypothesis is false (e.g., OLS
estimator). The second estimator should be a consistent estimator regardless of whether
the null hypothesis is true or false (e.g., 2SLS estimator).
2. If the null hypothesis is true, then both estimators should produce similar estimates of the
parameters of the equation. If the null hypothesis is false, then the two estimators should
produce significantly different estimates of the parameters of the equation. Thus, to test
the null hypothesis you test the equality of the estimates produced by the two estimators.
3. If the parameter estimates produced by the two estimators are significantly different, then
you reject the null hypothesis and conclude that the sample provides evidence that Y1 is
correlated with  in the population. If the parameter estimates produced by the two
estimators are not significantly different, then you accept the null hypothesis and conclude
that Y1 is not correlated with  in the population.
If you are testing whether the variable(s) in Y1 are endogenous, then you interpret the null and
alternative hypotheses as follows.
H0: Y1 is exogenous
(Y1 and  are not correlated)
H1: Y1 is endogenous
(Y1 and  are correlated)
If the vector Y1 contains one variable, then you are testing whether a single right-hand side
variable is exogenous. If the vector Y1 contains two or more variables, then you are testing
whether two or more right-hand side variables are jointly exogenous.
Interpretation of Hausman Test
If you reject the null hypothesis, then you have found evidence that Y1 is correlated with . You
interpret this as evidence that Y1 is endogenous. However, there are other reasons why Y1
might be correlated with  (e.g., Y1 is measured with error). Thus, you cannot conclude with
certainty what causes the correlation between X and . However, your interpretation is that
you have found evidence that Y1 is endogenous.
Implementation of the Hausman Test
Implementing the Hausman test involves the following steps.
1. Regress each variable in Y1 on all variables in X and Z (all exogenous variables in the model)
using the OLS estimator.
2. Save the residuals from each of these regressions. Denote this vector of residuals ^. The
residuals from each regression in step #1 is a “residual variable”.
3. Estimate the following regression equation using the OLS estimator:
Y = a + bY1 + cX + d^ + v
Where d denotes the vector of coefficients attached to the residual variables.
4. Test the following null and alternative hypotheses:
H0: d = 0
H1: d  0
(Y1 is exogenous)
(Y1 is endogenous)
5. If there is one variable in Y1, and therefore one residual variable in ^ and one coefficient in
d, then this hypothesis can be tested using a t-test. If there is more than one variable in Y1,
and therefore more than one residual variable in ^ and more than one coefficient in d, then
this hypothesis can be tested using a F-test.
Test of Overidentifying Restrictions
It is possible to test the overidentifying restrictions for a single equation in a system of
equations. Strictly speaking it is not possible to test the identifying restrictions for an equation.
This is because the equation must be identified for estimation of its parameters to be
meaningful. When you test the overidentifying restrictions for an equation you are testing
whether the variables that you have excluded from the equation to overidentify it can be validly
excluded, or whether at least one of these variables should be included in the equation.
Therefore, you are testing the following null and alternative hypotheses,
H0: The variables excluded to overidentify the equation do not belong in the equation
H1: At least one variable excluded to overidentify the equation does belong in the
equation.
Notation
Designate the equation to be estimated before the identifying instruments are excluded as
Y = a + bY1 + cX + dZ + 
Where all variables and parameters have been defined previously. Note that this is the
equation before it is identified, and therefore the variables in the vector Z have not been
excluded. The null and alternative hypotheses can be expressed as follows.
H0: d = 0
equation)
H1: d  0
equation)
(The variables excluded to overidentify the equation do not belong in the
(At least one variable excluded to overidentify the equation belongs in the
Lagrange Multiplier Test
The easiest way to test the null of hypothesis that the overidentifying restrictions are correct is
to use a Lagrange multiplier test. The test statistic and sampling distribution for this test are
LM = TR2 ~ 2(Z – Y1)
Where T is the sample size; R2 is the unadjusted R-squared statistic from the second of two
auxiliary regressions; 2 is the chi square distribution with Z – Y1 degrees of freedom, where Z is
the number of variables excluded from the equation and Y1 is the number of endogenous righthand side variables in the equation (this difference is equal to the number of overidentifying
restrictions).
Calculating the LM Test Statistic
To calculate the LM test statistic, you need to obtain the R2 statistic from the second of two
auxiliary regressions. This involves the following steps.
1. Estimate the following model using the 2SLS estimator
Y = a + bY1 + cX + 
Use as instruments for Y1 all variables in the vectors X and Z.
2. Save the residuals from this regression. Denote the residual variable as ^.
3. Regress the residual variable, ^, on all the variables in X and Z using the OLS estimator; that
is, estimate the following equation using the OLS estimator
^ =  + X + Z + v
4. Use the unadjusted R2 statistic from this regression to calculate the LM test statistic.
Notes about the Test of Overidentifying Restrictions
1. If you reject the null hypothesis, then you are rejecting the overidentifying restrictions. This
casts doubt on the identifying restrictions. This is because the overidentifying restrictions
cannot be separated from the identifying restrictions.
2. If you reject the overidentifying restrictions, the test gives you no guidance about what to
do next. A test does not exist that allows you do determine which variable or variables in Z
should not be excluded from the equation being estimated.
HYPOTHESIS TESTING
The small sample t-test and F-test cannot be used in the context of a simultaneous equation
regression model. This is because if the error term is correlated with one or more explanatory
variables, we don’t know the sampling distributions of the t-statistic and F-statistic. The
following large sample tests can be used:
1.
2.
3.
4.
Asymptotic t-test
Approximate F-test
Wald test
Lagrange multiplier test
Note that because the IV, 2SLS, 3SLS, and I3SLS estimators do not produce maximum likelihood
estimates, the likelihood ratio test cannot be used to test hypotheses.