k
k
k
k
k
Principles of
Econometrics
Fifth Edition
k
R. CARTER HILL
Louisiana State University
WILLIAM E. GRIFFITHS
University of Melbourne
GUAY C. LIM
University of Melbourne
k
k
k
EDITORIAL DIRECTOR
EXECUTIVE EDITOR
CHANNEL MARKETING MANAGER
CONTENT ENABLEMENT SENIOR MANAGER
CONTENT MANAGEMENT DIRECTOR
CONTENT MANAGER
SENIOR CONTENT SPECIALIST
PRODUCTION EDITOR
COVER PHOTO CREDIT
Michael McDonald
Lise Johnson
Michele Szczesniak
Leah Michael
Lisa Wojcik
Nichole Urban
Nicole Repasky
Abidha Sulaiman
© liuzishan/iStockphoto
This book was set in STIX-Regular 10/12pt by SPi Global and printed and bound by Strategic Content Imaging.
This book is printed on acid free paper. ∞
Founded in 1807, John Wiley & Sons, Inc. has been a valued source of knowledge and understanding for more than 200 years,
helping people around the world meet their needs and fulfill their aspirations. Our company is built on a foundation of
principles that include responsibility to the communities we serve and where we live and work. In 2008, we launched a
Corporate Citizenship Initiative, a global effort to address the environmental, social, economic, and ethical challenges we face
in our business. Among the issues we are addressing are carbon impact, paper specifications and procurement, ethical conduct
within our business and among our vendors, and community and charitable support. For more information, please visit our
website: www.wiley.com/go/citizenship.
Copyright © 2018, 2011 and 2007 John Wiley & Sons, Inc. All rights reserved. No part of this publication may be reproduced,
stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording,
scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either
the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the
Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923 (Web site: www.copyright.com). Requests to the
Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street,
Hoboken, NJ 07030-5774, (201) 748-6011, fax (201) 748-6008, or online at: www.wiley.com/go/permissions.
k
Evaluation copies are provided to qualified academics and professionals for review purposes only, for use in their courses
during the next academic year. These copies are licensed and may not be sold or transferred to a third party. Upon completion
of the review period, please return the evaluation copy to Wiley. Return instructions and a free of charge return shipping label
are available at: www.wiley.com/go/returnlabel. If you have chosen to adopt this textbook for use in your course, please accept
this book as your complimentary desk copy. Outside of the United States, please contact your local sales representative.
ISBN: 9781118452271 (PBK)
ISBN: 9781119320951 (EVALC)
Library of Congress Cataloging-in-Publication Data
Names: Hill, R. Carter, author. | Griffiths, William E., author. | Lim, G. C.
(Guay C.), author.
Title: Principles of econometrics / R. Carter Hill, Louisiana State
University, William E. Griffiths, University of Melbourne, Guay C. Lim,
University of Melbourn.
Description: Fifth Edition. | Hoboken : Wiley, 2017. | Revised edition of the
authors’ Principles of econometrics, c2011. | Includes bibliographical
references and index. |
Identifiers: LCCN 2017043094 (print) | LCCN 2017056927 (ebook) | ISBN
9781119342854 (pdf) | ISBN 9781119320944 (epub) | ISBN 9781118452271
(paperback) | ISBN 9781119320951 (eval copy)
Subjects: LCSH: Econometrics. | BISAC: BUSINESS & ECONOMICS / Econometrics.
Classification: LCC HB139 (ebook) | LCC HB139 .H548 2018 (print) | DDC
330.01/5195—dc23
LC record available at https://lccn.loc.gov/2017043094
The inside back cover will contain printing identification and country of origin if omitted from this page. In addition, if the ISBN
on the back cover differs from the ISBN on this page, the one on the back cover is correct.
10 9 8 7 6 5 4 3 2 1
k
k
k
Carter Hill dedicates this work to his wife, Melissa Waters
Bill Griffiths dedicates this work to Jill, David, and Wendy Griffiths
Guay Lim dedicates this work to Tony Meagher
k
k
k
k
Brief Contents
PREFACE
v
LIST OF EXAMPLES
1
An Introduction to Econometrics
Probability Primer
2
3
4
10
Endogenous Regressors and
Moment-Based Estimation 481
11
Simultaneous Equations Models 531
12
Regression with Time-Series Data:
Nonstationary Variables 563
13
Vector Error Correction and Vector
Autoregressive Models 597
14
Time-Varying Volatility and ARCH
Models 614
15
Panel Data Models 634
16
Qualitative and Limited Dependent
Variable Models 681
xxi
15
The Simple Linear Regression
Model 46
Interval Estimation and Hypothesis
Testing 112
Prediction, Goodness-of-Fit, and
Modeling Issues 152
1
5
The Multiple Regression Model 196
6
Further Inference in the Multiple
Regression Model 260
APPENDIX A Mathematical Tools
7
Using Indicator Variables 317
APPENDIXB Probability Concepts
8
Heteroskedasticity 368
APPENDIXC Review of Statistical Inference
9
Regression with Time-Series Data:
Stationary Variables 417
k
APPENDIXD Statistical Tables
INDEX
iv
k
869
k
748
862
768
812
k
Preface
Principles of Econometrics, Fifth Edition, is an introductory book for undergraduate students in
economics and finance, as well as first-year graduate students in economics, finance, accounting,
agricultural economics, marketing, public policy, sociology, law, forestry, and political science.
We assume that students have taken courses in the principles of economics and elementary statistics. Matrix algebra is not used, and we introduce and develop calculus concepts in an Appendix.
The title Principles of Econometrics emphasizes our belief that econometrics should be part of
the economics curriculum, in the same way as the principles of microeconomics and the principles of macroeconomics. Those who have been studying and teaching econometrics as long
as we have will remember that Principles of Econometrics was the title that Henri Theil used
for his 1971 classic, which was also published by John Wiley & Sons. Our choice of the same
title is not intended to signal that our book is similar in level and content. Theil’s work was, and
remains, a unique treatise on advanced graduate level econometrics. Our book is an introductory
level econometrics text.
Book Objectives
Principles of Econometrics is designed to give students an understanding of why econometrics is
necessary and to provide them with a working knowledge of basic econometric tools so that
k
i. They can apply these tools to modeling, estimation, inference, and forecasting in the context
of real-world economic problems.
ii. They can evaluate critically the results and conclusions from others who use basic econometric tools.
iii. They have a foundation and understanding for further study of econometrics.
iv. They have an appreciation of the range of more advanced techniques that exist and that may
be covered in later econometric courses.
k
The book is neither an econometrics cookbook nor is it in a theorem-proof format. It emphasizes
motivation, understanding, and implementation. Motivation is achieved by introducing very simple economic models and asking economic questions that the student can answer. Understanding
is aided by lucid description of techniques, clear interpretation, and appropriate applications.
Learning is reinforced by doing, with clear worked examples in the text and exercises at the end
of each chapter.
Overview of Contents
This fifth edition is a major revision in format and content. The chapters contain core material and
exercises, while appendices contain more advanced material. Chapter examples are now identified and separated from other content so that they may be easily referenced. From the beginning,
we recognize the observational nature of most economic data and modify modeling assumptions accordingly. Chapter 1 introduces econometrics and gives general guidelines for writing an
empirical research paper and locating economic data sources. The Probability Primer preceding
Chapter 2 summarizes essential properties of random variables and their probability distributions
and reviews summation notation. The simple linear regression model is covered in Chapters 2–4,
while the multiple regression model is treated in Chapters 5–7. Chapters 8 and 9 introduce econometric problems that are unique to cross-sectional data (heteroskedasticity) and time-series data
k
v
k
vi
Preface
(dynamic models), respectively. Chapters 10 and 11 deal with endogenous regressors, the failure
of least squares when a regressor is endogenous, and instrumental variables estimation, first in
the general case, and then in the simultaneous equations model. In Chapter 12, the analysis of
time-series data is extended to discussions of nonstationarity and cointegration. Chapter 13 introduces econometric issues specific to two special time-series models, the vector error correction,
and vector autoregressive models, while Chapter 14 considers the analysis of volatility in data
and the ARCH model. In Chapters 15 and 16, we introduce microeconometric models for panel
data and qualitative and limited dependent variables. In appendices A, B, and C, we introduce
math, probability, and statistical inference concepts that are used in the book.
Summary of Changes and New Material
This edition includes a great deal of new material, including new examples and exercises using
real data and some significant reorganizations. In this edition, we number examples for easy reference and offer 25–30 new exercises in each chapter. Important new features include
k
• Chapter 1 includes a discussion of data types and sources of economic data on the Internet.
Tips on writing a research paper are given “up front” so that students can form ideas for a
paper as the course develops.
• A Probability Primer precedes Chapter 2. This Primer reviews the concepts of random variables and how probabilities are calculated from probability density functions. Mathematical
expectation and rules of expected values are summarized for discrete random variables.
These rules are applied to develop the concepts of variance and covariance. Calculations of
probabilities using the normal distribution are illustrated. New material includes sections on
conditional expectation, conditional variance, iterated expectations, and the bivariate normal
distribution.
• Chapter 2 now starts with a discussion of causality. We define the population regression
function and discuss exogeneity in considerable detail. The properties of the ordinary least
squares (OLS) estimator are examined within the framework of the new assumptions. New
appendices have been added on the independent variable, covering the various assumptions
that might be made about the sampling process, derivations of the properties of the OLS
estimator, and Monte Carlo experiments to numerically illustrate estimator properties.
• In Chapter 3, we note that hypothesis test mechanics remain the same under the revised
assumptions because test statistics are “pivotal,” meaning that their distributions under the
null hypothesis do not depend on the data. In appendices, we add an extended discussion of
test behavior under the alternative, introduce the noncentral t-distribution, and illustrate test
power. We also include new Monte Carlo experiments illustrating test properties when the
explanatory variable is random.
• Chapter 4 discusses in detail nonlinear relationships such as the log-log, log-linear, linear-log,
and polynomial models. We have expanded the discussion of diagnostic residual plots and
added sections on identifying influential observations. The familiar concepts of compound
interest are used to motivate several log-linear models. We add an appendix on the concept
of mean squared error and the minimum mean squared error predictor.
• Chapter 5 introduces multiple regression in the random-x framework. The Frisch–Waugh–
Lovell (FWL) theorem is introduced as a way to help understand interpretation of the multiple regression model and used throughout the remainder of the book. Discussions of the
properties of the OLS estimator, and interval estimates and t-tests, are updated. The large
sample properties of the OLS estimator, and the delta method, are now introduced within
the chapter rather than an appendix. Appendices provide further discussion and Monte Carlo
k
k
k
Preface
vii
properties to illustrate the delta method. We provide a new appendix on bootstrapping and
its uses.
• Chapter 6 adds a new section on large sample tests. We explain the use of control variables and the difference between causal and predictive models. We revise the discussion
of collinearity and include a discussion of influential observations. We introduce nonlinear regression models and nonlinear least squares algorithms are discussed. Appendices are
added to discuss the statistical power of F-tests and further uses of the Frisch–Waugh–Lovell
theorem.
• Chapter 7 now includes an extensive section on treatment effects and causal modeling in
Rubin’s potential outcomes framework. We explain and illustrate the interesting regression discontinuity design. An appendix includes a discussion of the important “overlap”
assumption.
k
• Chapter 8 has been reorganized so that the heteroskedasticity robust variance of the OLS
estimator appears before testing. We add a section on how model specification can ameliorate
heteroskedasticity in some applications. We add appendices to explain the properties of the
OLS residuals and another to explain alternative robust sandwich variance estimators. We
present Monte Carlo experiments to illustrate the differences.
• Chapter 9 has been reorganized and streamlined. The initial section introduces the different ways that dynamic elements can be added to the regression model. These include using
finite lag models, infinite lag models, and autoregressive errors. We carefully discuss autocorrelations, including testing for autocorrelation and representing autocorrelations using a
correlogram. After introducing the concepts of stationarity and weak dependence, we discuss the general notions of forecasting and forecast intervals in the context of autoregressive
distributed lag (ARDL) models. Following these introductory concepts, there are details of
estimating and using alternative models, covering such topics as choosing lag lengths, testing
for Granger causality, the Lagrange multiplier test for serial correlation, and using models for
policy analysis. We provide very specific sets of assumptions for time-series regression models and outline how heteroskedastic and autocorrelation consistent, robust, standard errors
are used. We discuss generalized least squares estimation of a time-series regression model
and its relation to nonlinear least squares regression. A detailed discussion of the infinite lag
model and how to use multiplier analysis is provided. An appendix contains details of the
Durbin–Watson test.
• Chapter 10 on endogeneity problems has been streamlined because the concept of random
explanatory variables is now introduced much earlier in the book. We provide further analysis of weak instruments and how weak instruments adversely affect the precision of IV
estimation. The details of the Hausman test are now included in the chapter.
• Chapter 11 now adds Klein’s Model I as an example.
• Chapter 12 includes more details of deterministic trends and unit roots. The section on unit
root testing has been restructured so that each Dickey–Fuller test is more fully explained
and illustrated with an example. Numerical examples of ARDL models with nonstationary
variables that are, and are not, cointegrated have been added.
• The data in Chapter 13 have been updated and new exercises added.
• Chapter 14 mentions further extensions of ARCH volatility models.
• Chapter 15 has been restructured to give priority to how panel data can be used to cope with
the endogeneity caused by unobserved heterogeneity. We introduce the advantages of having
panel data using the first difference estimator, and then discuss the within/fixed effects estimator. We provide an extended discussion of cluster robust standard errors in both the OLS
and fixed effects model. We discuss the Mundlak version of the Hausman test for endogeneity.
We give brief mention to how to extend the use of panel data in several ways.
k
k
k
viii
Preface
• The Chapter 16 discussion of binary choice models is reorganized and expanded. It now
includes brief discussions of advanced topics such as binary choice models with endogenous
explanatory variables and binary choice models with panel data. We add new appendices on
random utility models and latent variable models.
• Appendix A includes new sections on second derivatives and finding maxima and minima
of univariate and bivariate functions.
• Appendix B includes new material on conditional expectations and conditional variances,
including several useful decompositions. We include new sections on truncated random variables, including the truncated normal and Poisson distributions. To facilitate discussions of
test power, we have new sections on the noncentral t-distribution, the noncentral Chi-square
distribution, and the noncentral F-distribution. We have included an expanded new section
on the log-normal distribution.
• Appendix C content does not change a great deal, but 20 new exercises are included.
• Statistical Tables for the Standard Normal cumulative distribution function, the t-distribution
and Chi-square distribution critical values for selected percentiles, the F-distribution critical
values for the 95th and 99th percentiles, and the Standard Normal density function values
appear in Appendix D.
• A useful “cheat sheet” of essential formulas is provided at the authors’ website,
www.principlesofeconometrics.com, rather than inside the covers as in the previous
edition.
k
For Instructors: Suggested Course Plans
Principles of Econometrics, Fifth Edition is suitable for one or two semester courses at the undergraduate or first year graduate level. Some suitable plans for alternative courses are as follows:
• One-semester survey course: Sections P.1–P.6.2 and P.7; Sections 2.1–2.9; Chapters 3 and
4; Sections 5.1–5.6; Sections 6.1–6.5; Sections 7.1–7.3; Sections 8.1–8.4 and 8.6; Sections
9.1–9.4.2 and 9.5–9.5.1.
• One-semester survey course enhancements for Master’s or Ph.D.: Include Appendices for
Chapters 2–9.
• Two-semester survey second course, cross-section emphasis: Section P.6; Section 2.10;
Section 5.7; Section 6.6; Sections 7.4–7.6; Sections 8.5 and 8.6.3–8.6.5; Sections 10.1–10.4;
Sections 15.1–15.4; Sections 16.1–16.2 and 16.6;
• Two-semester survey second course, time series emphasis: Section P.6; Section 2.10;
Section 5.7; Section 6.6; Sections 7.4–7.6; Sections 8.5 and 8.6.3–8.6.5; Section 9.5;
Sections 10.1–10.4; Sections 12.1–12.5; Sections 13.1–13.5; Sections 14.1–14.4;
• Two-semester survey course enhancements for Master’s or Ph.D.: Include Appendices from
Chapters 10, Chapter 11, Appendices 15A–15B, Sections 16.3–16.5 and 16.7, Appendices
16A–16D, Book Appendices B and C.
Computer Supplement Books
There are several computer supplements to Principles of Econometrics, Fifth Edition. The supplements are not versions of the text and cannot substitute for the text. They use the examples in
the text as a vehicle for learning the software. We show how to use the software to get the answers
for each example in the text.
k
k
k
Preface
k
• Using EViews for the Principles of Econometrics, Fifth Edition, by William E. Griffiths,
R. Carter Hill, and Guay C. Lim [ISBN 9781118469842]. This supplementary book presents
the EViews 10 [www.eviews.com] software commands required for the examples in Principles of Econometrics in a clear and concise way. It includes many illustrations that are student
friendly. It is useful not only for students and instructors who will be using this software as
part of their econometrics course but also for those who wish to learn how to use EViews.
• Using Stata for the Principles of Econometrics, Fifth Edition, by Lee C. Adkins and
R. Carter Hill [ISBN 9781118469873]. This supplementary book presents the Stata 15.0
[www.stata.com] software commands required for the examples in Principles of Econometrics. It is useful not only for students and instructors who will be using this software as part
of their econometrics course but also for those who wish to learn how to use Stata. Screen
shots illustrate the use of Stata’s drop-down menus. Stata commands are explained and the
use of “do-files” illustrated.
• Using SAS for the Principles of Econometrics, Fifth Edition, by Randall C. Campbell and
R. Carter Hill [ISBN 9781118469880]. This supplementary book gives SAS 9.4 [www.sas
.com] software commands for econometric tasks, following the general outline of Principles
of Econometrics, Fifth Edition. It includes enough background material on econometrics so
that instructors using any textbook can easily use this book as a supplement. The volume
spans several levels of econometrics. It is suitable for undergraduate students who will use
“canned” SAS statistical procedures, and for graduate students who will use advanced procedures as well as direct programming in SAS’s matrix language; the latter is discussed in
chapter appendices.
• Using Excel for Principles of Econometrics, Fifth Edition, by Genevieve Briand and R. Carter
Hill [ISBN 9781118469835]. This supplement explains how to use Excel to reproduce most
of the examples in Principles of Econometrics. Detailed instructions and screen shots are
provided explaining both the computations and clarifying the operations of Excel. Templates
are developed for common tasks.
• Using GRETL for Principles of Econometrics, Fifth Edition, by Lee C. Adkins. This free
supplement, readable using Adobe Acrobat, explains how to use the freely available statistical
software GRETL (download from http://gretl.sourceforge.net). Professor Adkins explains in
detail, and using screen shots, how to use GRETL to replicate the examples in Principles of
Econometrics. The manual is freely available at www.learneconometrics.com/gretl.html.
• Using R for Principles of Econometrics, Fifth Edition, by Constantin Colonescu and
R. Carter Hill. This free supplement, readable using Adobe Acrobat, explains how to
use the freely available statistical software R (download from https://www.r-project.org/).
The supplement explains in detail, and using screen shots, how to use R to replicate the
examples in Principles of Econometrics, Fifth Edition. The manual is freely available at
https://bookdown.org/ccolonescu/RPOE5/.
Data Files
Data files for the book are provided in a variety of formats at the book website www.wiley.com
/college/hill. These include
• ASCII format (*.dat). These are text files containing only data.
• Definition files (*.def). These are text files describing the data file contents, with a listing of
variable names, variable definitions, and summary statistics.
• EViews (*.wf1) workfiles for each data file.
• Excel (*.xls) workbooks for each data file, including variable names in the first row.
k
ix
k
k
x
Preface
•
•
•
•
Comma separated values (*.csv) files that can be read into almost all software.
Stata (*.dta) data files.
SAS (*.sas7bdat) data files.
GRETL (*.gdt) data files.
• R (*.rdata) data files.
The author website www.principlesofeconometrics.com includes a complete list of the data files
and where they are used in the book.
Additional Resources
The book website www.principlesofeconometrics.com includes
• Individual data files in each format as well as ZIP files containing data in compressed format.
• Book errata.
• Brief answers to odd number problems. These answers are also provided on the book website at www.wiley.com/college/hill.
• Additional examples with solutions. Some extra examples come with complete solutions so
that students will know what a good answer looks like.
• Tips on writing research papers.
k
k
Resources for Instructors
For instructors, also available at the website www.wiley.com/college/hill are
• Complete solutions, in both Microsoft Word and *.pdf formats, to all exercises in the text.
• PowerPoint slides and PowerPoint Viewer.
k
k
Preface
xi
Acknowledgments
Authors Hill and Griffiths want to acknowledge the gifts given to them over the past 40 years by
mentor, friend, and colleague George Judge. Neither this book nor any of the other books we have
shared in the writing of would have ever seen the light of day without his vision and inspiration.
We also wish to give thanks to the many students and faculty who have commented on
the fourth edition of the text and contributed to the fifth edition. This list includes Alejandra
Breve Ferrari, Alex James, Alyssa Wans, August Saibeni, Barry Rafferty, Bill Rising, Bob
Martin, Brad Lewis, Bronson Fong, David Harris, David Iseral, Deborah Williams, Deokrye
Baek, Diana Whistler, Emma Powers, Ercan Saridogan, Erdogan Cevher, Erika Haguette, Ethan
Luedecke, Gareth Thomas, Gawon Yoon, Genevieve Briand, German Altgelt, Glenn Sueyoshi,
Henry McCool, James Railton, Jana Ruimerman, Jeffery Parker, Joe Goss, John Jackson, Julie
Leiby, Katharina Hauck, Katherine Ramirez, Kelley Pace, Lee Adkins, Matias Cattaneo, Max
O’Krepki, Meagan McCollum, Micah West, Michelle Savolainen, Oystein Myrland, Patrick
Scholten, Randy Campbell, Regina Riphahn, Sandamali Kankanamge, Sergio Pastorello,
Shahrokh Towfighi, Tom Fomby, Tong Zeng, Victoria Pryor, Yann Nicolas, and Yuanbo Zhang.
In the book errata we acknowledge those who have pointed out our errors.
R. Carter Hill
William E. Griffiths
Guay C. Lim
k
k
k
k
Table of Contents
PREFACE
v
LIST OF EXAMPLES
P.5.6
xxi
P.6
1 An Introduction to
Econometrics
1.1
1.2
Why Study Econometrics?
1.3
k
2
4
P.8
5
2.1
7
Time-Series Data
7
Cross-Section Data
8
Panel or Longitudinal Data
2.2
Writing an Empirical Research Paper
9
13
Probability Distributions
P.3
Joint, Marginal, and Conditional
Probabilities
20
17
2.4
22
Properties of Probability Distributions
Expected Value of a Random Variable
Conditional Expectation
25
Rules for Expected Values
25
Variance of a Random Variable
26
Expected Values of Several Random
Variables
27
23
49
Data Generating Process
51
The Random Error and Strict
Exogeneity
52
The Regression Function
53
Random Error Variation
54
Variation in x
56
Error Normality
56
Generalizing the Exogeneity
Assumption
56
Error Correlation
57
Summarizing the Assumptions
k
58
The Gauss–Markov Theorem
2.6
The Probability Distributions of the Least
Squares Estimators
73
2.7
2.7.2
k
72
Estimating the Variance of the Error Term
2.7.1
66
The Estimator b2
67
68
The Expected Values of b1 and b2
Sampling Variation
69
The Variances and Covariance of b1
and b2
69
2.5
24
59
The Least Squares Principle
61
Other Economic Models
65
Assessing the Least Squares Estimators
2.4.1
2.4.2
2.4.3
2.4.4
Marginal Distributions
20
Conditional Probability
21
Statistical Independence
21
A Digression: Summation Notation
P.5.1
P.5.2
P.5.3
P.5.4
P.5.5
xii
16
47
Estimating the Regression Parameters
2.3.1
2.3.2
P.2
P.5
2.2.8
2.2.9
2.3
Random Variables
P.4
13
15
P.1
P.3.1
P.3.2
P.3.3
11
Links to Economic Data on the Internet
Interpreting Economic Data
14
Obtaining the Data
14
Probability Primer
46
An Econometric Model
2.2.3
2.2.4
2.2.5
2.2.6
2.2.7
11
Writing a Research Proposal
11
A Format for Writing a Research Report
Sources of Economic Data
39
An Economic Model
2.2.1
2.2.2
9
1.7
1.8.1
1.8.2
1.8.3
37
7
The Research Process
1.8
Exercises
Model
6
1.6
1.7.1
1.7.2
34
The Bivariate Normal Distribution
2 The Simple Linear Regression
5
Experimental Data
6
Quasi-Experimental Data
Nonexperimental Data
29
Conditional Expectation
30
Conditional Variance
31
Iterated Expectations
32
Variance Decomposition
33
Covariance Decomposition
34
The Normal Distribution
P.7.1
Causality and Prediction
Economic Data Types
1.5.1
1.5.2
1.5.3
P.7
3
How Are Data Generated?
1.4.1
1.4.2
1.4.3
1.5
Some Examples
The Econometric Model
1.3.1
1.4
1
What Is Econometrics About?
1.2.1
Conditioning
P.6.1
P.6.2
P.6.3
P.6.4
P.6.5
1
Covariance Between Two Random
Variables
27
74
Estimating the Variances and Covariance of
the Least Squares Estimators
74
Interpreting the Standard Errors
76
k
Table of Contents
2.8
Estimating Nonlinear Relationships
2.8.1
2.8.2
2.8.3
2.8.4
2.8.5
2.9
77
Regression with Indicator Variables
2.10 The Independent Variable
2.11 Exercises
84
86
3.3.3
Derivation of the Least Squares
Estimates
98
Appendix 2B
Deviation from the Mean Form
99
of b2
Appendix 2C
b2 Is a Linear Estimator
Appendix 2D
Derivation of Theoretical Expression
for b2 100
Appendix 2E
Deriving the Conditional Variance
of b2 100
Proof of the Gauss–Markov
Theorem 102
Appendix 2G
Proofs of Results Introduced in
Section 2.10 103
Appendix 2H
2H.1
2H.2
2H.3
2H.4
2H.5
2H.6
2H.7
The p-Value
3.6
Linear Combinations of Parameters
Interval Estimation
126
129
133
Problems 133
Computer Exercises
139
Appendix 3A
Derivation of the t-Distribution
Appendix 3B
Distribution of the t-Statistic under
H1 145
Appendix 3C
3C.1
3C.2
3C.3
3C.4
Monte Carlo Simulation
144
147
Sampling Properties of Interval
Estimators 148
Sampling Properties of Hypothesis
Tests 149
Choosing the Number of Monte Carlo
Samples 149
Random-x Monte Carlo Results 150
106
4 Prediction, Goodness-of-Fit,
and Modeling Issues
4.1
4.2
Least Squares Prediction
Measuring Goodness-of-Fit
4.2.1
4.2.2
4.3
113
115
k
152
153
156
Correlation Analysis 158
Correlation Analysis and R2
Modeling Issues
4.3.1
4.3.2
4.3.3
4.3.4
4.3.5
112
The t-Distribution 113
Obtaining Interval Estimates
The Sampling Context 116
123
Testing a Linear Combination of
Parameters 131
Exercises
3.7.1
3.7.2
The Regression Function 106
The Random Error 107
Theoretically True Values 107
Creating a Sample of Data 108
Monte Carlo Objectives 109
Monte Carlo Results 109
Random-x Monte Carlo Results 110
Hypothesis Testing
3.1.1
3.1.2
3.1.3
3.5
3.7
3 Interval Estimation and
3.1
Examples of Hypothesis Tests
The Implications of Strict Exogeneity 103
The Random and Independent x Case 103
The Random and Strictly Exogenous x
Case 105
Random Sampling 106
Monte Carlo Simulation
One-Tail Tests with Alternative
‘‘Greater Than’’ (>) 120
One-Tail Tests with Alternative
‘‘Less Than’’ (<) 121
Two-Tail Tests with Alternative
‘‘Not Equal To’’ (≠) 122
3.4
3.6.1
100
Appendix 2F
2G.4
3.3.2
93
118
The Null Hypothesis 118
The Alternative Hypothesis 118
The Test Statistic 119
The Rejection Region 119
A Conclusion 120
Rejection Regions for Specific
Alternatives 120
3.3.1
Appendix 2A
2G.1
2G.2
2G.3
3.3
82
89
2.11.1 Problems
89
2.11.2 Computer Exercises
Hypothesis Tests
3.2.1
3.2.2
3.2.3
3.2.4
3.2.5
2.10.1 Random and Independent x
84
2.10.2 Random and Strictly Exogenous x
2.10.3 Random Sampling
87
k
3.2
Quadratic Functions
77
Using a Quadratic Model
77
A Log-Linear Function
79
Using a Log-Linear Model
80
Choosing a Functional Form
82
xiii
158
160
The Effects of Scaling the Data 160
Choosing a Functional Form 161
A Linear-Log Food Expenditure Model
Using Diagnostic Residual Plots 165
Are the Regression Errors Normally
Distributed? 167
163
k
k
xiv
Table of Contents
4.3.6
4.4
Polynomial Models
4.4.1
4.5
4.7
Exercises
5.5.2
5.5.3
171
173
Prediction in the Log-Linear Model 175
A Generalized R2 Measure 176
Prediction Intervals in the Log-Linear
Model 177
Log-Log Models
4.7.1
4.7.2
169
171
Quadratic and Cubic Equations
Log-Linear Models
4.5.1
4.5.2
4.5.3
4.6
Identifying Influential Observations
5.6
Nonlinear Relationships
5.7
Large Sample Properties of the Least Squares
Estimator 227
5.7.1
5.7.2
5.7.3
5.7.4
177
179
Problems 179
Computer Exercises
185
5.8
Development of a Prediction
Interval 192
Appendix 4B
The Sum of Squares
Decomposition 193
Appendix 5A
Appendix 4C
Mean Squared Error: Estimation and
Prediction 193
Appendix 5B
5.8.1
5.8.2
5B.1
5 The Multiple Regression
5B.2
Appendix 5C
196
5C.1
k
5.1
Introduction
5.1.1
5.1.2
5.1.3
5.1.4
5.2
5.3.2
Appendix 5D
5D.1
5D.2
5D.3
5D.4
5D.5
5.4.2
Hypothesis Testing
5.5.1
240
Derivation of Least Squares
Estimators 247
The Delta Method
248
Nonlinear Function of a Single
Parameter 248
Nonlinear Function of Two Parameters
Monte Carlo Simulation
249
250
Least Squares Estimation with Chi-Square
Errors 250
Monte Carlo Simulation of the Delta
Method 252
Bootstrapping
254
Resampling 255
Bootstrap Bias Estimate 256
Bootstrap Standard Error 256
Bootstrap Percentile Interval Estimate
Asymptotic Refinement 258
Regression Model
6.1
257
6.1.3
6.1.4
6.1.5
6.2
6.3
Testing the Significance of a Single
Coefficient 219
k
261
Testing the Significance of the Model
The Relationship Between t- and
F-Tests 265
More General F-Tests 267
Using Computer Software 268
Large Sample Tests 269
The Use of Nonsample Information
Model Specification
6.3.1
6.3.2
6.3.3
218
260
Testing Joint Hypotheses: The F-test
6.1.1
6.1.2
216
Interval Estimation for a Single
Coefficient 216
Interval Estimation for a Linear Combination
of Coefficients 217
234
Problems 234
Computer Exercises
6 Further Inference in the Multiple
The Variances and Covariances of the Least
Squares Estimators 212
The Distribution of the Least Squares
Estimators 214
Interval Estimation
5.4.1
5.5
Least Squares Estimation Procedure 205
Estimating the Error Variance σ2 207
Measuring Goodness-of-Fit 208
Frisch–Waugh–Lovell (FWL) Theorem 209
Finite Sample Properties of the Least Squares
Estimator 211
5.3.1
5.4
The Economic Model 197
The Econometric Model 198
The General Model 202
Assumptions of the Multiple Regression
Model 203
5C.2
Estimating the Parameters of the Multiple
Regression Model 205
5.2.1
5.2.2
5.2.3
5.2.4
5.3
197
222
Consistency 227
Asymptotic Normality 229
Relaxing Assumptions 230
Inference for a Nonlinear Function of
Coefficients 232
Exercises
Appendix 4A
Model
One-Tail Hypothesis Testing for a Single
Coefficient 220
Hypothesis Testing for a Linear Combination
of Coefficients 221
271
273
Causality versus Prediction
Omitted Variables 275
Irrelevant Variables 277
273
264
k
k
Table of Contents
6.3.4
6.3.5
6.3.6
6.4
Prediction
6.4.1
6.5
Control Variables
Choosing a Model
RESET 281
7.7
278
280
282
Predictive Model Selection
Criteria 285
The Consequences of Collinearity 289
Identifying and Mitigating Collinearity 290
Investigating Influential Observations 293
6.6
Nonlinear Least Squares
6.7
Exercises
6.7.1
6.7.2
294
303
Appendix 6A
The Statistical Power of F-Tests
Appendix 6B
Further Results from the FWL
Theorem 315
7 Using Indicator Variables
Indicator Variables
7.1.1
7.1.2
k
7.2
7.2.2
7.2.3
7.2.4
7.3
7.5
7.6
Details of Log-Linear Model
Interpretation 366
Appendix 7B
Derivation of the
Differences-in-Differences
Estimator 366
Appendix 7C
The Overlap Assumption:
Details 367
317
Treatment Effects
8.2
Heteroskedasticity in the Multiple Regression
Model 370
8.4
Generalized Least Squares: Known Form
of Variance 375
8.4.1
8.4.2
8.5
8.6
8.6.4
8.6.5
332
8.7
8.8
342
k
381
383
Residual Plots 384
The Goldfeld–Quandt Test 384
A General Test for Conditional
Heteroskedasticity 385
The White Test 387
Model Specification and
Heteroskedasticity 388
Heteroskedasticity in the Linear Probability
Model 390
Exercises
8.8.1
8.8.2
The Nature of Causal Effects 342
Treatment Effect Models 343
Decomposing the Treatment Effect 344
Introducing Control Variables 345
The Overlap Assumption 347
Regression Discontinuity Designs 347
Estimating the Multiplicative Model
Detecting Heteroskedasticity
8.6.1
8.6.2
8.6.3
331
k
Transforming the Model: Proportional
Heteroskedasticity 375
Weighted Least Squares: Proportional
Heteroskedasticity 377
Generalized Least Squares: Unknown Form
of Variance 379
8.5.1
334
The Heteroskedastic Regression Model 371
Heteroskedasticity Consequences for the OLS
Estimator 373
Heteroskedasticity Robust Variance
Estimator 374
330
330
The Difference Estimator 334
Analysis of the Difference Estimator
The Differences-in-Differences
Estimator 338
369
8.3
323
Treatment Effects and Causal Modeling
7.6.1
7.6.2
7.6.3
7.6.4
7.6.5
7.6.6
The Nature of Heteroskedasticity
329
A Rough Calculation
An Exact Calculation
368
8.1
8.2.1
8.2.2
318
The Linear Probability Model
7.5.1
7.5.2
7.5.3
311
Interactions Between Qualitative
Factors 323
Qualitative Factors with Several
Categories 324
Testing the Equivalence of Two
Regressions 326
Controlling for Time 328
Log-Linear Models
7.3.1
7.3.2
7.4
Appendix 7A
Intercept Indicator Variables 318
Slope-Indicator Variables 320
Applying Indicator Variables
7.2.1
358
8 Heteroskedasticity
297
Problems 297
Computer Exercises
351
Problems 351
Computer Exercises
Poor Data, Collinearity, and
Insignificance 288
6.5.1
6.5.2
6.5.3
7.1
Exercises
7.7.1
7.7.2
xv
391
Problems 391
Computer Exercises
401
Appendix 8A
Properties of the Least Squares
Estimator 407
Appendix 8B
Lagrange Multiplier Tests for
Heteroskedasticity 408
k
xvi
Table of Contents
8C.1
10.1.2 Why Least Squares Estimation
Fails 484
10.1.3 Proving the Inconsistency
of OLS 486
Properties of the Least Squares
Residuals 410
Appendix 8C
Details of Multiplicative Heteroskedasticity
Model 411
10.2 Cases in Which x and e are Contemporaneously
Correlated 487
Appendix 8D
Alternative Robust Sandwich
Estimators 411
Appendix 8E
Monte Carlo Evidence: OLS, GLS, and
FGLS 414
10.2.1 Measurement Error 487
10.2.2 Simultaneous Equations Bias 488
10.2.3 Lagged-Dependent Variable Models with
Serial Correlation 489
10.2.4 Omitted Variables 489
9 Regression with Time-Series
Data: Stationary Variables
9.1
Introduction
9.1.1
9.1.2
9.2
9.3
9.4
9.4.2
9.4.3
9.5
427
430
Forecast Intervals and Standard Errors
Assumptions for Forecasting 435
Selecting Lag Lengths 436
Testing for Granger Causality 437
433
438
Checking the Correlogram of the Least
Squares Residuals 439
Lagrange Multiplier Test 440
Durbin–Watson Test 443
Time-Series Regressions for Policy
Analysis 443
9.5.1
9.5.2
9.5.3
9.5.4
9.6
420
Testing for Serially Correlated Errors
9.4.1
k
Modeling Dynamic Relationships
Autocorrelations 424
Forecasting
10.3.1 Method of Moments Estimation of a
Population Mean and Variance 490
10.3.2 Method of Moments Estimation in the Simple
Regression Model 491
10.3.3 Instrumental Variables Estimation in the
Simple Regression Model 492
10.3.4 The Importance of Using Strong
Instruments 493
10.3.5 Proving the Consistency of the IV
Estimator 494
10.3.6 IV Estimation Using Two-Stage Least Squares
(2SLS) 495
10.3.7 Using Surplus Moment Conditions 496
10.3.8 Instrumental Variables Estimation in the
Multiple Regression Model 498
10.3.9 Assessing Instrument Strength Using the
First-Stage Model 500
10.3.10 Instrumental Variables Estimation in a
General Model 502
10.3.11 Additional Issues When Using IV
Estimation 504
418
Stationarity and Weak Dependence
9.3.1
9.3.2
9.3.3
9.3.4
10.3 Estimators Based on the Method
of Moments 490
417
Finite Distributed Lags 445
HAC Standard Errors 448
Estimation with AR(1) Errors 452
Infinite Distributed Lags 456
Exercises
9.6.1
9.6.2
Appendix 9A
9A.1
Appendix 9B
10.4 Specification Tests
463
Problems 463
Computer Exercises
468
The Durbin–Watson Test
505
10.4.1 The Hausman Test for Endogeneity 505
10.4.2 The Logic of the Hausman Test 507
10.4.3 Testing Instrument Validity 508
476
The Durbin–Watson Bounds Test
Properties of an AR(1) Error
478
516
Appendix 10A Testing for Weak Instruments
10A.1
10A.2
481
10.1 Least Squares Estimation with Endogenous
Regressors 482
510
10.5.1 Problems 510
10.5.2 Computer Exercises
479
10 Endogenous Regressors and
Moment-Based Estimation
10.5 Exercises
Appendix 10B Monte Carlo Simulation
10B.1
10B.2
10.1.1 Large Sample Properties of the OLS
Estimator 483
k
520
A Test for Weak Identification 521
Testing for Weak Identification:
Conclusions 525
525
Illustrations Using Simulated Data
The Sampling Properties of
IV/2SLS 528
526
k
k
Table of Contents
11 Simultaneous Equations
Models
12.7 Exercises
11.1 A Supply and Demand Model
532
11.2 The Reduced-Form Equations
11.3 The Failure of Least Squares Estimation
11.4 The Identification Problem
Vector Autoregressive
Models 597
535
535
536
13.1 VEC and VAR Models
11.5 Two-Stage Least Squares Estimation
538
k
13.4.1 Impulse Response Functions
13.4.2 Forecast Error Variance
Decompositions 605
557
The k-Class of Estimators 557
The LIML Estimator 558
Monte Carlo Simulation Results
13.5 Exercises
562
ARCH Models
564
12.1.1 Trend Stationary Variables 567
12.1.2 The First-Order Autoregressive Model
12.1.3 Random Walk Models 572
570
14.1 The ARCH Model
615
616
14.3 Testing, Estimating, and Forecasting
14.4 Extensions
574
620
622
14.4.1 The GARCH Model—Generalized
ARCH 622
14.4.2 Allowing for an Asymmetric Effect 623
14.4.3 GARCH-in-Mean and Time-Varying Risk
Premium 624
14.4.4 Other Developments 625
576
14.5 Exercises
626
14.5.1 Problems 626
14.5.2 Computer Exercises
15 Panel Data Models
582
627
634
15.1 The Panel Data Regression Function
584
636
15.1.1 Further Discussion of Unobserved
Heterogeneity 638
15.1.2 The Panel Data Regression Exogeneity
Assumption 639
12.5 Regression When There Is No
Cointegration 585
12.6 Summary
614
14.2 Time-Varying Volatility
12.3.1 Unit Roots 576
12.3.2 Dickey–Fuller Tests 577
12.3.3 Dickey–Fuller Test with Intercept and No
Trend 577
12.3.4 Dickey–Fuller Test with Intercept and
Trend 579
12.3.5 Dickey–Fuller Test with No Intercept and No
Trend 580
12.3.6 Order of Integration 581
12.3.7 Other Unit Root Tests 582
12.4.1 The Error Correction Model
612
14 Time-Varying Volatility and
12.1 Stationary and Nonstationary Variables
12.4 Cointegration
608
Appendix 13A The Identification Problem
Data: Nonstationary
Variables 563
12.2 Consequences of Stochastic Trends
603
607
13.5.1 Problems 607
13.5.2 Computer Exercises
12 Regression with Time-Series
12.3 Unit Root Tests for Stationarity
601
13.4 Impulse Responses and Variance
Decompositions 603
551
Appendix 11A 2SLS Alternatives
11A.1
11A.2
11A.3
13.3 Estimating a VAR Model
545
11.6.1 Problems 545
11.6.2 Computer Exercises
598
13.2 Estimating a Vector Error Correction
Model 600
11.5.1 The General Two-Stage Least Squares
Estimation Procedure 539
11.5.2 The Properties of the Two-Stage Least
Squares Estimator 540
11.6 Exercises
592
13 Vector Error Correction and
534
11.3.1 Proving the Failure of OLS
588
12.7.1 Problems 588
12.7.2 Computer Exercises
531
xvii
587
k
k
k
xviii
Table of Contents
15.1.3 Using OLS to Estimate the Panel Data
Regression 639
15.2 The Fixed Effects Estimator
15.2.1
15.2.2
15.2.3
15.2.4
16.2.9 Binary Choice Models with a Binary
Endogenous Variable 699
16.2.10 Binary Endogenous Explanatory
Variables 700
16.2.11 Binary Choice Models and Panel
Data 701
640
The Difference Estimator: T = 2 640
The Within Estimator: T = 2 642
The Within Estimator: T > 2 643
The Least Squares Dummy Variable
Model 644
16.3 Multinomial Logit
15.3 Panel Data Regression Error Assumptions
646
15.3.1 OLS Estimation with Cluster-Robust
Standard Errors 648
15.3.2 Fixed Effects Estimation with Cluster-Robust
Standard Errors 650
15.4 The Random Effects Estimator
15.5 Exercises
k
16.4 Conditional Logit
16.5 Ordered Choice Models
16.6 Models for Count Data
Appendix 15A Cluster-Robust Standard Errors:
Some Details 677
681
16.1 Introducing Models with Binary Dependent
Variables 682
16.8 Exercises
685
16.2.1 The Probit Model for Binary
Choice 686
16.2.2 Interpreting the Probit Model 687
16.2.3 Maximum Likelihood Estimation of the Probit
Model 690
16.2.4 The Logit Model for Binary Choices 693
16.2.5 Wald Hypothesis Tests 695
16.2.6 Likelihood Ratio Hypothesis Tests 696
16.2.7 Robust Inference in Probit and Logit
Models 698
16.2.8 Binary Choice Models with a Continuous
Endogenous Variable 698
718
725
16.8.1 Problems 725
16.8.2 Computer Exercises
683
733
Appendix 16A Probit Marginal Effects: Details
16A.1
16A.2
741
Binary Choice Model 741
Probit or Logit? 742
Appendix 16C Using Latent Variables
16C.1
16C.2
743
Tobit (Tobit Type I) 743
Heckit (Tobit Type II) 744
Appendix 16D A Tobit Monte Carlo Experiment
k
739
Standard Error of Marginal Effect at a Given
Point 739
Standard Error of Average Marginal
Effect 740
Appendix 16B Random Utility Models
16B.1
16B.2
k
717
16.7.1 Maximum Likelihood Estimation
of the Simple Linear Regression
Model 717
16.7.2 Truncated Regression 718
16.7.3 Censored Samples and Regression
16.7.4 Tobit Model Interpretation 720
16.7.5 Sample Selection 723
16 Qualitative and Limited
16.2 Modeling Binary Choices
713
16.7 Limited Dependent Variables
Appendix 15B Estimation of Error
Components 679
16.1.1 The Linear Probability Model
710
16.6.1 Maximum Likelihood Estimation of the
Poisson Regression Model 713
16.6.2 Interpreting the Poisson Regression
Model 714
670
Dependent Variable Models
709
16.5.1 Ordinal Probit Choice Probabilities
16.5.2 Ordered Probit Estimation and
Interpretation 711
663
15.5.1 Problems 663
15.5.2 Computer Exercises
707
16.4.1 Conditional Logit Choice
Probabilities 707
16.4.2 Conditional Logit Postestimation
Analysis 708
651
15.4.1 Testing for Random Effects 653
15.4.2 A Hausman Test for Endogeneity in the
Random Effects Model 654
15.4.3 A Regression-Based Hausman Test 656
15.4.4 The Hausman–Taylor Estimator 658
15.4.5 Summarizing Panel Data Assumptions 660
15.4.6 Summarizing and Extending Panel Data
Model Estimation 661
702
16.3.1 Multinomial Logit Choice
Probabilities 703
16.3.2 Maximum Likelihood Estimation 703
16.3.3 Multinomial Logit Postestimation
Analysis 704
745
k
Table of Contents
A Mathematical Tools
A.1
Some Basics
A.1.1
A.1.2
A.1.3
A.1.4
A.1.5
A.1.6
A.2
A.3
k
A.5
753
753
Rules for Derivatives 754
Elasticity of a Nonlinear Relationship
Second Derivatives 757
Maxima and Minima 758
Partial Derivatives 759
Maxima and Minima of Bivariate
Functions 760
B.1.2
B.1.3
B.1.4
B.1.5
B.1.6
B.1.7
B.1.8
B.1.9
B.2
B.2.3
B.2.4
805
806
A Sample of Data
C.2
An Econometric Model
C.3
Estimating the Mean of a Population
C.3.1
C.3.2
C.3.3
C.3.4
C.3.5
769
Expected Value of a Discrete Random
Variable 769
Variance of a Discrete Random
Variable 770
Joint, Marginal, and Conditional
Distributions 771
Expectations Involving Several Random
Variables 772
Covariance and Correlation 773
Conditional Expectations 774
Iterated Expectations 774
Variance Decomposition 774
Covariance Decomposition 777
C.4
C.4.2
C.5
815
Estimating the Population
Variance 821
Estimating Higher Moments
821
822
Interval Estimation: σ2 Known 822
Interval Estimation: σ2 Unknown 825
Hypothesis Tests About a Population
Mean 826
C.6.1
C.6.2
k
814
The Expected Value of Y 816
The Variance of Y 817
The Sampling Distribution of Y 817
The Central Limit Theorem 818
Best Linear Unbiased Estimation 820
Interval Estimation
C.5.1
C.5.2
C.6
813
Estimating the Population Variance and Other
Moments 820
C.4.1
Probability Calculations 779
Properties of Continuous Random
Variables 780
Joint, Marginal, and Conditional Probability
Distributions 781
Using Iterated Expectations with Continuous
Random Variables 785
812
C.1
768
Working with Continuous Random
Variables 778
B.2.1
B.2.2
Exercises
Inference
764
Discrete Random Variables
800
Uniform Random Numbers
C Review of Statistical
Computing the Area Under a
Curve 762
Exercises
B.1.1
B.5
The Bernoulli Distribution 790
The Binomial Distribution 790
The Poisson Distribution 791
The Uniform Distribution 792
The Normal Distribution 793
The Chi-Square Distribution 794
The t-Distribution 796
The F-Distribution 797
The Log-Normal Distribution 799
Random Numbers
B.4.1
762
B Probability Concepts
B.1
B.4
757
Distributions of Functions of Random
Variables 787
Truncated Random Variables 789
Some Important Probability
Distributions 789
B.3.1
B.3.2
B.3.3
B.3.4
B.3.5
B.3.6
B.3.7
B.3.8
B.3.9
752
Slopes and Derivatives
Elasticity 753
Integrals
A.4.1
B.3
Numbers 749
Exponents 749
Scientific Notation 749
Logarithms and the Number e 750
Decimals and Percentages 751
Logarithms and Percentages 751
Nonlinear Relationships
A.3.1
A.3.2
A.3.3
A.3.4
A.3.5
A.3.6
A.4
B.2.6
749
Linear Relationships
A.2.1
A.2.2
B.2.5
748
xix
C.6.3
C.6.4
C.6.5
C.6.6
Components of Hypothesis Tests 826
One-Tail Tests with Alternative ‘‘Greater
Than’’ (>) 828
One-Tail Tests with Alternative ‘‘Less
Than’’ (<) 829
Two-Tail Tests with Alternative ‘‘Not Equal
To’’ (≠) 829
The p-Value 831
A Comment on Stating Null and Alternative
Hypotheses 832
k
k
xx
Table of Contents
C.6.7
C.6.8
C.7
Some Other Useful Tests
C.7.1
C.7.2
C.7.3
C.7.4
C.8
C.8.2
C.8.3
C.8.4
Inference with Maximum Likelihood
Estimators 840
The Variance of the Maximum Likelihood
Estimator 841
The Distribution of the Sample
Proportion 842
Asymptotic Test Procedures 843
Algebraic Supplements
C.9.1
C.9.2
C.10 Kernel Density Estimator
C.11 Exercises
848
851
854
C.11.1 Problems 854
C.11.2 Computer Exercises
857
D Statistical Tables
862
834
Testing the Population Variance 834
Testing the Equality of Two Population
Means 834
Testing the Ratio of Two Population
Variances 835
Testing the Normality of a Population 836
Introduction to Maximum Likelihood
Estimation 837
C.8.1
C.9
Type I and Type II Errors 833
A Relationship Between Hypothesis Testing
and Confidence Intervals 833
TableD.1 Cumulative Probabilities for the Standard
Normal Distribution 𝚽(z) = P(Z ≤ z) 862
TableD.2 Percentiles of the t-distribution
863
TableD.3 Percentiles of the Chi-square
Distribution 864
TableD.4 95th Percentile for the F-distribution
865
TableD.5 99th Percentile for the F-distribution
866
TableD.6 Standard Normal pdf Values 𝛟(z)
INDEX
867
869
Derivation of Least Squares Estimator 848
Best Linear Unbiased Estimation 849
k
k
k
k
List of Examples
Example P.1
Using a cdf
Example P.2
Calculating a Conditional
Probability
21
19
Example 3.4 (continued) p-Value for a Left-Tail
Test 128
Example P.3
Calculating an Expected Value
Example P.4
Calculating a Conditional
Expectation
25
Example P.5
Calculating a Variance
Example P.6
Calculating a Correlation
28
Example P.7
Conditional Expectation
30
Example P.8
Conditional Variance
31
Example P.9
Iterated Expectation
32
Example 3.5 (continued) p-Value for a Two-Tail
Test 129
24
Example 3.6 (continued) p-Value for a Two-Tail Test
of Significance 129
26
Example P.10 Covariance Decomposition
34
Example P.11 Normal Distribution Probability
Calculation
36
k
Example 3.7
Estimating Expected Food
Expenditure 130
Example 3.8
An Interval Estimate of Expected
Food Expenditure 131
Example 3.9
Testing Expected Food
Expenditure 132
Example 4.1
Prediction in the Food Expenditure
Model 156
Example 4.2
Goodness-of-Fit in the Food
Expenditure Model 159
Example 2.1
A Failure of the Exogeneity
Assumption
53
Example 4.3
Reporting Regression Results
Example 2.2
Strict Exogeneity in the Household
Food Expenditure Model
54
Example 4.4
Using the Linear-Log Model for Food
Expenditure 164
Example 2.3
Food Expenditure Model Data
59
Example 4.5
Example 2.4a Estimates for the Food Expenditure
Function
63
Heteroskedasticity in the Food
Expenditure Model 167
Example 4.6
Testing Normality in the Food
Expenditure Model 168
Example 4.7
Influential Observations in the Food
Expenditure Data 171
Example 4.8
An Empirical Example of a Cubic
Equation 172
Example 4.9
A Growth Model
Example 2.4b Using the Estimates
64
Example 2.5
Calculations for the Food
Expenditure Data
75
Example 2.6
Baton Rouge House Data
Example 2.7
Baton Rouge House Data, Log-Linear
Model
81
Example 3.1
78
Interval Estimate for Food
Expenditure Data 116
Example 4.10 A Wage Equation
159
174
175
Example 4.11 Prediction in a Log-Linear
Model 176
Example 3.2
Right-Tail Test of Significance
Example 3.3
Right-Tail Test of an Economic
Hypothesis 124
Example 4.12 Prediction Intervals for a Log-Linear
Model 177
Example 3.4
Left-Tail Test of an Economic
Hypothesis 125
Example 4.13 A Log-Log Poultry Demand
Equation 178
Example 3.5
Two-Tail Test of an Economic
Hypothesis 125
Example 5.1
Data for Hamburger Chain
Example 5.2
Example 3.6
Two-Tail Test of Significance
OLS Estimates for Hamburger Chain
Data 206
Example 5.3
Error Variance Estimate for
Hamburger Chain Data 208
123
126
Example 3.3 (continued) p-Value for a Right-Tail
Test 127
k
200
xxi
k
k
xxii
List of Examples
Example 5.4
R2 for Hamburger Chain Data
Example 5.5
Variances, Covariances, and
Standard Errors for Hamburger Chain
Data 214
209
Example 5.6
Interval Estimates for Coefficients in
Hamburger Sales Equation 216
Example 5.7
Interval Estimate for a Change in
Sales 218
Example 5.8
Testing the Significance of
Price 219
Example 5.9
Testing the Significance of
Advertising Expenditure 220
Example 5.10 Testing for Elastic Demand
270
Example 6.8
A Nonlinear Hypothesis
Example 6.9
Restricted Least Squares
272
Example 6.10 Family Income Equation
275
Example 6.11 Adding Children Aged Less Than
6 Years 277
277
Example 6.13 A Control Variable for Ability
279
Example 6.14 Applying RESET to Family Income
Equation 282
Example 6.15 Forecasting SALES for the Burger
Barn 284
220
Example 6.16 Predicting House Prices
Example 5.12 Testing the Effect of Changes in Price
and Advertising 222
Example 5.13 Cost and Product Curves
270
Example 6.12 Adding Irrelevant Variables
Example 5.11 Testing Advertising
Effectiveness 221
223
Example 5.14 Extending the Model for Burger Barn
Sales 224
k
Examples6.2and6.5 Revisited
Example 5.16 A Log-Quadratic Wage Equation
Example 6.18 Influential Observations in the House
Price Equation 293
Example 6.19 Nonlinear Least Squares Estimates
for Simple Model 295
Example 5.18 How Much Experience Maximizes
Wages? 233
Example 5.19 An Interval Estimate for
(
)
exp 𝛃2 ∕10 249
250
Example 5.21 Bootstrapping for Nonlinear
( )
(
)
Functions g1 𝛃2 = exp 𝛃2 ∕10 and
(
)
g2 𝛃1 , 𝛃2 = 𝛃1 ∕𝛃2 258
296
Example 7.1
The University Effect on House
Prices 321
Example 7.2
The Effects of Race and Sex on
Wage 323
Example 7.3
A Wage Equation with Regional
Indicators 325
Example 7.4
Testing the Equivalence of Two
Regressions: The Chow Test 327
Example 7.5
Indicator Variables in a Log-Linear
Model: The Rough
Approximation 330
Example 7.6
Indicator Variables in a Log-Linear
Model: An Exact Calculation 330
Example 7.7
The Linear Probability Model: An
Example from Marketing 332
226
Example 5.17 The Optimal Level of
Advertising 232
Example 5.20 An Interval Estimate for 𝛃1 ∕𝛃2
Example 6.17 Collinearity in a Rice Production
Function 291
Example 6.20 A Logistic Growth Curve
Example 5.15 An Interaction Variable in a Wage
Equation 225
287
Example 6.1
Testing the Effect of Advertising
Example 6.2
The F-Test Procedure
Example 6.3
Overall Significance of Burger Barns
Equation 265
Example 7.8
An Application of Difference
Estimation: Project STAR 335
Example 6.4
When are t- and F-tests
equivalent? 266
Example 7.9
The Difference Estimator with
Additional Controls 336
Example 6.5
Testing Optimal Advertising
Example 6.6
A One-Tail Test
Example 6.7
Two (J = 2) Complex
Hypotheses 268
268
262
263
267
Example 7.10 The Difference Estimator with Fixed
Effects 337
Example 7.11 Linear Probability Model Check of
Random Assignment 338
k
k
k
List of Examples
xxiii
Example 7.12 Estimating the Effect of a Minimum
Wage Change: The DID
Estimator 340
Example 9.10 Checking the Residual Correlogram
for the ARDL(2, 1) Unemployment
Equation 439
Example 7.13 Estimating the Effect of a Minimum
Wage Change: Using Panel Data 341
Example 9.11 Checking the Residual Correlogram
for an ARDL(1, 1) Unemployment
Equation 440
Example 8.1
Example 8.2
Heteroskedasticity in the Food
Expenditure Model 372
Robust Standard Errors in the Food
Expenditure Model 374
Example 8.3
Applying GLS/WLS to the Food
Expenditure Data 378
Example 8.4
Multiplicative Heteroskedasticity in
the Food Expenditure Model 382
Example 8.5
A Heteroskedastic Partition
Example 8.6
The Goldfeld–Quandt Test with
Partitioned Data 384
Example 8.7
The Goldfeld–Quandt Test in the
Food Expenditure Model 385
Example 8.8
Variance Stabilizing
Log-transformation 389
Example 8.9
The Marketing Example
Revisited 391
k
Example 9.12 LM Test for Serial Correlation in the
ARDL Unemployment Equation 443
Example 9.13 Okun’s Law
Example 9.14 A Phillips Curve
450
Example 9.15 The Phillips Curve with AR(1)
Errors 455
Example 9.16 A Consumption Function
383
458
Example 9.17 Deriving Multipliers for an Infinite Lag
Okun’s Law Model 459
Example 9.18 Computing the Multiplier Estimates
for the Infinite Lag Okun’s Law
Model 460
Example 9.19 Testing for Consistency of Least
Squares Estimation of Okun’s
Law 462
Example 8.10 Alternative Robust Standard Errors in
the Food Expenditure Model 413
Example 9.1
446
Plotting the Unemployment Rate and
the GDP Growth Rate for the United
States 419
k
Example 9.20 Durbin–Watson Bounds Test for
Phillips Curve 479
Example 10.1 Least Squares Estimation of a Wage
Equation 489
Example 10.2 IV Estimation of a Simple Wage
Equation 495
Example 9.2
Sample Autocorrelations for
Unemployment 426
Example 10.3 2SLS Estimation of a Simple Wage
Equation 496
Example 9.3
Sample Autocorrelations for GDP
Growth Rate 427
Example 10.4 Using Surplus Instruments in the
Simple Wage Equation 497
Example 9.4
Are the Unemployment and Growth
Rate Series Stationary and Weakly
Dependent? 428
Example 10.5 IV/2SLS Estimation in the Wage
Equation 499
Example 9.5
Forecasting Unemployment with an
AR(2) Model 431
Example 9.6
Forecast Intervals for Unemployment
from the AR(2) Model 434
Example 9.7
Forecasting Unemployment with an
ARDL(2, 1) Model 434
Example 9.8
Choosing Lag Lengths in an
ARDL(p, q) Unemployment
Equation 437
Example 11.2 Supply and Demand at the Fulton
Fish Market 542
Does the Growth Rate Granger Cause
Unemployment? 438
Example 11.4 Testing for Weak Instruments Using
LIML 560
Example 9.9
Example 10.6 Checking Instrument Strength in the
Wage Equation 502
Example 10.7 Specification Tests for the Wage
Equation 509
Example 10.8 Testing for Weak Instruments
523
Example 11.1 Supply and Demand for Truffles
Example 11.3 Klein’s Model I
k
541
544
k
xxiv
List of Examples
Example 11.5 Testing for Weak Instruments with
Fuller-Modified LIML 561
Example 15.3 Using T = 2 Differenced Observations
for a Wage Equation 641
Example 12.1 Plots of Some U.S. Economic Time
Series 564
Example 15.4 Using the Within Transformation with
T = 2 Observations for a Production
Function 642
Example 12.2 A Deterministic Trend for Wheat
Yield 569
Example 12.3 A Regression with Two Random
Walks 575
Example 12.4 Checking the Two Interest Rate
Series for Stationarity 579
Example 12.5 Is GDP Trend Stationary?
580
Example 12.7 The Order of Integration of the Two
Interest Rate Series 581
Example 12.8 Are the Federal Funds Rate and Bond
Rate Cointegrated? 583
Example 12.9 An Error Correction Model for the
Bond and Federal Funds Rates 585
Example12.10 A Consumption Function in First
Differences 586
Example 13.1 VEC Model for GDP
600
Example 13.2 VAR Model for Consumption and
Income 602
Example 14.1 Characteristics of Financial
Variables 617
Example 14.3 Testing for ARCH in BrightenYourDay
(BYD) Lighting 620
Example 14.4 ARCH Model Estimates for
BrightenYourDay (BYD)
Lighting 621
Example 14.6 A GARCH Model for
BrightenYourDay 623
624
Example 14.8 A GARCH-in-Mean Model for
BYD 625
Example 15.1 A Microeconometric Panel
Example 15.9 Random Effects Estimation of a
Production Function 652
Example15.10 Random Effects Estimation of a Wage
Equation 652
Example15.11 Testing for Random Effects in a
Production Function 654
Example15.12 Testing for Endogenous Random
Effects in a Production
Function 656
Example15.14 The Mundlak Approach for a
Production Function 657
Example15.15 The Mundlak Approach for a Wage
Equation 658
Example15.16 The Hausman–Taylor Estimator for a
Wage Equation 659
Example 14.5 Forecasting BrightenYourDay (BYD)
Volatility 621
Example 14.7 A T-GARCH Model for BYD
Example 15.8 Using Fixed Effects and
Cluster-Robust Standard Errors for a
Production Function 651
Example15.13 Testing for Endogenous Random
Effects in a Wage Equation 656
Example 14.2 Simulating Time-Varying
Volatility 618
Example 15.1 Revisited
Example 15.6 Using the Fixed Effects Estimator
with T = 3 Observations for a
Production Function 646
Example 15.7 Using Pooled OLS with
Cluster-Robust Standard Errors for a
Production Function 650
Example 12.6 Is Wheat Yield Trend
Stationary? 580
k
Example 15.5 Using the Within Transformation with
T = 3 Observations for a Production
Function 644
Example 16.1 A Transportation Problem
683
Example 16.2 A Transportation Problem: The Linear
Probability Model 684
Example 16.3 Probit Maximum Likelihood: A Small
Example 690
Example 16.4 The Transportation Data:
Probit 691
636
637
Example 15.2 Using T = 2 Differenced Observations
for a Production Function 641
Example 16.5 The Transportation Data: More
Postestimation Analysis 692
Example 16.6 An Empirical Example from
Marketing 694
k
k
k
List of Examples
Example 16.7 Coke Choice Model: Wald Hypothesis
Tests 696
Computing a Joint Probability
783
Example B.7
Example 16.8 Coke Choice Model: Likelihood Ratio
Hypothesis Tests 697
Another Joint Probability
Calculation 783
Example B.8
Example 16.9 Estimating the Effect of Education on
Labor Force Participation 699
Finding and Using a Marginal
pdf 784
Example B.9
Finding and Using a Conditional
pdf 784
Example16.10 Women’s Labor Force Participation
and Having More Than Two
Children 700
Example B.10 Computing a Correlation
786
Example B.12 Change of Variable: Continuous
Case 788
Example16.12 Postsecondary Education
Multinomial Choice 705
Example B.13 Change of Variable: Continuous
Case 789
Example16.13 Conditional Logit Soft Drink
Choice 708
Example B.14 An Inverse Transformation
Example B.16 Linear Congruential Generator
Example 805
Example16.15 A Count Model for the Number of
Doctor Visits 715
Example16.16 Tobit Model of Hours Worked
801
Example B.15 The Inversion Method: An
Example 802
Example16.14 Postsecondary Education Ordered
Choice Model 712
Example16.17 Heckit Model of Wages
785
Example B.11 Using Iterated Expectation
Example16.11 Effect of War Veteran Status on
Wages 701
k
Example B.6
xxv
721
724
Example C.1
Histogram of Hip Width Data
Example C.2
Sample Mean of Hip Width Data
Example C.3
Sampling Variation of Sample Means
of Hip Width Data 816
Example C.4
The Effect of Sample Size on Sample
Mean Precision 818
814
Example A.1
Slope of a Linear Function
Example A.2
Slope of a Quadratic Function
Example A.3
Taylor Series Approximation
Example A.4
Second Derivative of a Linear
Function 758
Example C.5
Illustrating the Central Limit
Theorem 819
Example A.5
Second Derivative of a Quadratic
Function 758
Example C.6
Sample Moments of the Hip
Data 822
Example A.6
Finding the Minimum of a Quadratic
Function 759
Example C.7
Using the Hip Data Estimates
Example C.8
Example A.7
Maximizing a Profit Function
Simulating the Hip Data: Interval
Estimates 824
Example A.8
Minimizing a Sum of Squared
Differences 761
Example C.9
Simulating the Hip Data:
Continued 825
Example A.9
Area Under a Curve
Example B.1
Variance Decomposition: Numerical
Example 776
Example C.10 Interval Estimation Using the Hip
Data 826
Example C.11 One-tail Test Using the Hip Data
Example B.2
Probability Calculation Using
Geometry 779
Example C.12 Two-tail Test Using the Hip
Data 830
Example B.3
Probability Calculation Using
Integration 780
Example C.13 One-tail Test p-value: The Hip
Data 831
Example B.4
Expected Value of a Continuous
Random Variable 780
Example C.14 Two-Tail Test p-Value: The Hip
Data 832
Example B.5
Variance of a Continuous Random
Variable 781
Example C.15 Testing the Normality of the Hip
Data 836
755
755
756
761
763
k
815
822
830
k
k
xxvi
List of Examples
Example C.16 The ‘‘Wheel of Fortune’’ Game:
p = 1∕4 or 3/4 837
Example C.21 Likelihood Ratio Test of the
Population Proportion 845
Example C.17 The ‘‘Wheel of Fortune’’ Game:
0 < p < 1 838
Example C.22 Wald Test of the Population
Proportion 846
Example C.18 The ‘‘Wheel of Fortune’’ Game:
Maximizing the Log-likelihood 839
Example C.23 Lagrange Multiplier Test of the
Population Proportion 848
Example C.19 Estimating a Population
Proportion 840
Example C.24 Hip Data: Minimizing the Sum of
Squares Function 849
Example C.20 Testing a Population
Proportion 843
k
k
k
k
CHAPTER 1
An Introduction to
Econometrics
1.1
k
Why Study Econometrics?
Econometrics is fundamental for economic measurement. However, its importance extends far
beyond the discipline of economics. Econometrics is a set of research tools also employed in
the business disciplines of accounting, finance, marketing, and management. It is used by social
scientists, specifically researchers in history, political science, and sociology. Econometrics plays
an important role in such diverse fields as forestry and agricultural economics. This breadth of
interest in econometrics arises in part because economics is the foundation of business analysis
and is the core social science. Thus, research methods employed by economists, which includes
the field of econometrics, are useful to a broad spectrum of individuals.
Econometrics plays a special role in the training of economists. As a student of economics,
you are learning to “think like an economist.” You are learning economic concepts such as opportunity cost, scarcity, and comparative advantage. You are working with economic models of
supply and demand, macroeconomic behavior, and international trade. Through this training you
become a person who better understands the world in which we live; you become someone who
understands how markets work, and the way in which government policies affect the marketplace.
If economics is your major or minor field of study, a wide range of opportunities is open to
you upon graduation. If you wish to enter the business world, your employer will want to know
the answer to the question, “What can you do for me?” Students taking a traditional economics
curriculum answer, “I can think like an economist.” While we may view such a response to be
powerful, it is not very specific and may not be very satisfying to an employer who does not
understand economics.
The problem is that a gap exists between what you have learned as an economics student
and what economists actually do. Very few economists make their livings by studying economic
theory alone, and those who do are usually employed by universities. Most economists, whether
they work in the business world or for the government, or teach in universities, engage in economic
analysis that is in part “empirical.” By this we mean that they use economic data to estimate
economic relationships, test economic hypotheses, and predict economic outcomes.
Studying econometrics fills the gap between being “a student of economics” and being
“a practicing economist.” With the econometric skills you will learn from this book, including
how to work with econometric software, you will be able to elaborate on your answer to the
employer’s question above by saying “I can predict the sales of your product.” “I can estimate the
effect on your sales if your competition lowers its price by $1 per unit.” “I can test whether your
new ad campaign is actually increasing your sales.” These answers are music to an employer’s
ears, because they reflect your ability to think like an economist and to analyze economic data.
k
1
k
k
2
CHAPTER 1
An Introduction to Econometrics
Such pieces of information are keys to good business decisions. Being able to provide your
employer with useful information will make you a valuable employee and increase your odds of
getting a desirable job.
On the other hand, if you plan to continue your education by enrolling in graduate school or
law school, you will find that this introduction to econometrics is invaluable. If your goal is to earn
a master’s or Ph.D. degree in economics, finance, data analytics, data science, accounting, marketing, agricultural economics, sociology, political science, or forestry, you will encounter more
econometrics in your future. The graduate courses tend to be quite technical and mathematical,
and the forest often gets lost in studying the trees. By taking this introduction to econometrics
you will gain an overview of what econometrics is about and develop some “intuition” about how
things work before entering a technically oriented course.
1.2
What Is Econometrics About?
At this point we need to describe the nature of econometrics. It all begins with a theory from your
field of study—whether it is accounting, sociology, or economics—about how important variables are related to one another. In economics we express our ideas about relationships between
economic variables using the mathematical concept of a function. For example, to express a relationship between income and consumption, we may write
CONSUMPTION = 𝑓 (INCOME)
which says that the level of consumption is some function, f (•), of income.
The demand for an individual commodity—say, the Honda Accord—might be expressed as
k
Qd = 𝑓 (P, Ps , Pc , INC)
which says that the quantity of Honda Accords demanded, Qd , is a function f (P, Ps , Pc , INC) of
the price of Honda Accords P, the price of cars that are substitutes Ps , the price of items that are
complements Pc (like gasoline), and the level of income INC.
The supply of an agricultural commodity such as beef might be written as
)
(
Qs = 𝑓 P, Pc , P𝑓
where Qs is the quantity supplied, P is the price of beef, Pc is the price of competitive products
in production (e.g., the price of hogs), and Pf is the price of factors or inputs (e.g., the price of
corn) used in the production process.
Each of the above equations is a general economic model that describes how we visualize
the way in which economic variables are interrelated. Economic models of this type guide our
economic analysis.
Econometrics allows us to go further than knowing that certain economic variables are interrelated, or even the direction of a relationship. Econometrics allows us to assign magnitudes to
questions about the interrelationships between variables. One aspect of econometrics is prediction or forecasting. If we know the value of INCOME, what will be the magnitude of CONSUMPTION? If we have values for the prices of Honda Accords, their substitutes and complements, and
income, how many Honda Accords will be sold? Similarly, we could ask how much beef would
be supplied given values of the variables on which its supply depends.
A second contribution of econometrics is to enable us to say how much a change in one
variable affects another. If the price for Honda Accords is increased, by how much will quantity
demanded decline? If the price of beef goes up, by how much will quantity supplied increase?
Finally, econometrics contributes to our understanding of the interrelationships between variables
by giving us the ability to test the validity of hypothesized relationships.
k
k
k
1.2 What Is Econometrics About?
3
Econometrics is about how we can use theory and data from economics, business, and the
social sciences, along with tools from statistics, to predict outcomes, answer “how much”
type questions, and test hypotheses.
1.2.1
k
Some Examples
Consider the problem faced by decision makers in a central bank. In the United States, the Federal
Reserve System and, in particular, the Chair of the Board of Governors of the FRB must make
decisions about interest rates. When prices are observed to rise, suggesting an increase in the
inflation rate, the FRB must make a decision about whether to dampen the rate of growth of the
economy. It can do so by raising the interest rate it charges its member banks when they borrow
money (the discount rate) or the rate on overnight loans between banks (the federal funds rate).
Increasing these rates sends a ripple effect through the economy, causing increases in other interest
rates, such as those faced by would-be investors, who may be firms seeking funds for capital
expansion or individuals who wish to buy consumer durables like automobiles and refrigerators.
This has the economic effect of increasing costs, and consumers react by reducing the quantity of
the durable goods demanded. Overall, aggregate demand falls, which slows the rate of inflation.
These relationships are suggested by economic theory.
The real question facing the Chair is “How much should we increase the discount rate to
slow inflation and yet maintain a stable and growing economy?” The answer will depend on
the responsiveness of firms and individuals to increases in the interest rates and to the effects
of reduced investment on gross national product (GNP). The key elasticities and multipliers are
called parameters. The values of economic parameters are unknown and must be estimated using
a sample of economic data when formulating economic policies.
Econometrics is about how to best estimate economic parameters given the data we have.
“Good” econometrics is important since errors in the estimates used by policymakers such as the
FRB may lead to interest rate corrections that are too large or too small, which has consequences
for all of us.
Every day, decision-makers face “how much” questions similar to those facing the FRB Chair:
• A city council ponders the question of how much violent crime will be reduced if an additional million dollars is spent putting uniformed police on the street.
• The owner of a local Pizza Hut must decide how much advertising space to purchase in the
local newspaper and thus must estimate the relationship between advertising and sales.
• Louisiana State University must estimate how much enrollment will fall if tuition is raised
by $300 per semester and thus whether its revenue from tuition will rise or fall.
• The CEO of Proctor & Gamble must predict how much demand there will be in 10 years for
the detergent Tide and how much to invest in new plant and equipment.
• A real estate developer must predict by how much population and income will increase to the
south of Baton Rouge, Louisiana, over the next few years and whether it will be profitable to
begin construction of a gambling casino and golf course.
• You must decide how much of your savings will go into a stock fund and how much into the
money market. This requires you to make predictions of the level of economic activity, the
rate of inflation, and interest rates over your planning horizon.
• A public transportation council in Melbourne, Australia, must decide how an increase in fares
for public transportation (trams, trains, and buses) will affect the number of travelers who
switch to car or bike and the effect of this switch on revenue going to public transportation.
k
k
k
4
CHAPTER 1
An Introduction to Econometrics
To answer these questions of “how much,” decision-makers rely on information provided by
empirical economic research. In such research, an economist uses economic theory and reasoning
to construct relationships between the variables in question. Data on these variables are collected
and econometric methods are used to estimate the key underlying parameters and to make predictions. The decision-makers in the above examples obtain their “estimates” and “predictions”
in different ways. The FRB has a large staff of economists to carry out econometric analyses. The
CEO of Proctor & Gamble may hire econometric consultants to provide the firm with projections of sales. You may get advice about investing from a stock broker, who in turn is provided
with econometric projections made by economists working for the parent company. Whatever the
source of your information about “how much” questions, it is a good bet that there is an economist
involved who is using econometric methods to analyze data that yield the answers.
In the next section, we show how to introduce parameters into an economic model and how
to convert an economic model into an econometric model.
1.3
k
The Econometric Model
What is an econometric model, and where does it come from? We will give you a general
overview, and we may use terms that are unfamiliar to you. Be assured that before you are too
far into this book, all the terminology will be clearly defined. In an econometric model we must
first realize that economic relations are not exact. Economic theory does not claim to be able
to predict the specific behavior of any individual or firm, but rather describes the average or
systematic behavior of many individuals or firms. When studying car sales we recognize that the
actual number of Hondas sold is the sum of this systematic part and a random and unpredictable
component e that we will call a random error. Thus, an econometric model representing the
sales of Honda Accords is
Qd = 𝑓 (P, Ps, Pc, INC) + e
The random error e accounts for the many factors that affect sales that we have omitted from this
simple model, and it also reflects the intrinsic uncertainty in economic activity.
To complete the specification of the econometric model, we must also say something about
the form of the algebraic relationship among our economic variables. For example, in your first
economics courses quantity demanded was depicted as a linear function of price. We extend that
assumption to the other variables as well, making the systematic part of the demand relation
𝑓 (P, Ps, Pc, INC) = β1 + β2 P + β3 Ps + β4 Pc + β5 INC
The corresponding econometric model is
Qd = β1 + β2 P + β3 Ps + β4 Pc + β5 INC + e
The coefficients β1 , β2 ,…,β5 are unknown parameters of the model that we estimate using economic data and an econometric technique. The functional form represents a hypothesis about the
relationship between the variables. In any particular problem, one challenge is to determine a
functional form that is compatible with economic theory and the data.
In every econometric model, whether it is a demand equation, a supply equation, or a production function, there is a systematic portion and an unobservable random component. The
systematic portion is the part we obtain from economic theory, and includes an assumption about
the functional form. The random component represents a “noise” component, which obscures
our understanding of the relationship among variables, and which we represent using the random
variable e.
We use the econometric model as a basis for statistical inference. Using the econometric
model and a sample of data, we make inferences concerning the real world, learning something
in the process. The ways in which statistical inference are carried out include the following:
• Estimating economic parameters, such as elasticities, using econometric methods
k
k
k
1.4 How Are Data Generated?
5
• Predicting economic outcomes, such as the enrollment in two-year colleges in the United
States for the next 10 years
• Testing economic hypotheses, such as the question of whether newspaper advertising is better
than store displays for increasing sales
Econometrics includes all of these aspects of statistical inference. As we proceed through this
book, you will learn how to properly estimate, predict, and test, given the characteristics of the
data at hand.
1.3.1
Causality and Prediction
A question that often arises when specifying an econometric model is whether a relationship can
be viewed as both causal and predictive or only predictive. To appreciate the difference, consider
an equation where a student’s grade in Econometrics GRADE is related to the proportion of class
lectures that are skipped SKIP.
GRADE = β1 + β2 SKIP + e
k
We would expect β2 to be negative: the greater the proportion of lectures that are skipped, the
lower the grade. But, can we say that skipping lectures causes grades to be lower? If lectures are
captured by video, they could be viewed at another time. Perhaps a student is skipping lectures
because he or she has a demanding job, and the demanding job does not leave enough time for
study, and this is the underlying cause of a poor grade. Or, it might be that skipping lectures comes
from a general lack of commitment or motivation, and this is the cause of a poor grade. Under
these circumstances, what can we say about the equation that relates GRADE to SKIP? We can
still call it a predictive equation. GRADE and SKIP are (negatively) correlated and so information
about SKIP can be used to help predict GRADE. However, we cannot call it a causal relationship.
Skipping lectures does not cause a low grade. The parameter β2 does not convey the direct causal
effect of skipping lectures on grade. It also includes the effect of other variables that are omitted
from the equation and correlated with SKIP, such as hours spent studying or student motivation.
Economists are frequently interested in parameters that can be interpreted as causal. Honda
would like to know the direct effect of a price change on the sales of their Accords. When there is
technological improvement in the beef industry, the price elasticities of demand and supply have
important implications for changes in consumer and producer welfare. One of our tasks will be
to see what assumptions are necessary for an econometric model to be interpreted as causal and
to assess whether those assumptions hold.
An area where predictive relationships are important is in the use of “big data.” Advances
in computer technology have led to storage of massive amounts of information. Travel sites on
the Internet keep track of destinations you have been looking at. Google targets you with advertisements based on sites that you have been surfing. Through their loyalty cards, supermarkets
keep data on your purchases and identify sale items relevant for you. Data analysts use big data
to identify predictive relationships that help predict our behavior.
In general, the type of data we have impacts on the specification of an econometric model
and the assumptions that we make about it. We turn now to a discussion of different types of data
and where they can be found.
1.4
How Are Data Generated?
In order to carry out statistical inference we must have data. Where do data come from? What
type of real processes generate data? Economists and other social scientists work in a complex
world in which data on variables are “observed” and rarely obtained from a controlled experiment.
This makes the task of learning about economic parameters all the more difficult. Procedures for
using such data to answer questions of economic importance are the subject matter of this book.
k
k
k
6
CHAPTER 1
An Introduction to Econometrics
1.4.1
k
Experimental Data
One way to acquire information about the unknown parameters of economic relationships is to
conduct or observe the outcome of an experiment. In the physical sciences and agriculture, it is
easy to imagine controlled experiments. Scientists specify the values of key control variables and
then observe the outcome. We might plant similar plots of land with a particular variety of wheat,
and then vary the amounts of fertilizer and pesticide applied to each plot, observing at the end of
the growing season the bushels of wheat produced on each plot. Repeating the experiment on N
plots of land creates a sample of N observations. Such controlled experiments are rare in business
and the social sciences. A key aspect of experimental data is that the values of the explanatory
variables can be fixed at specific values in repeated trials of the experiment.
One business example comes from marketing research. Suppose we are interested in the
weekly sales of a particular item at a supermarket. As an item is sold it is passed over a scanning
unit to record the price and the amount that will appear on your grocery bill. But at the same
time, a data record is created, and at every point in time the price of the item and the prices of
all its competitors are known, as well as current store displays and coupon usage. The prices and
shopping environment are controlled by store management, so this “experiment” can be repeated
a number of days or weeks using the same values of the “control” variables.
There are some examples of planned experiments in the social sciences, but they are rare
because of the difficulties in organizing and funding them. A notable example of a planned
experiment is Tennessee’s Project Star.1 This experiment followed a single cohort of elementary
school children from kindergarten through the third grade, beginning in 1985 and ending in 1989.
In the experiment children and teachers were randomly assigned within schools into three types
of classes: small classes with 13–17 students, regular-sized classes with 22–25 students, and
regular-sized classes with a full-time teacher aide to assist the teacher. The objective was to determine the effect of small classes on student learning, as measured by student scores on achievement
tests. We will analyze the data in Chapter 7 and show that small classes significantly increase
performance. This finding will influence public policy toward education for years to come.
1.4.2
Quasi-Experimental Data
It is useful to distinguish between “pure” experimental data and “quasi”-experimental data. A
pure experiment is characterized by random assignment. In the example where varying amounts
of fertilizer and pesticides are applied to plots of land for growing wheat, the different applications
of fertilizer and pesticides are randomly assigned to different plots. In Tennessee’s Project Star,
students and teachers are randomly assigned to different sized classes with and without a teacher’s
aide. In general, if we have a control group and a treatment group, and we want to examine the
effect of a policy intervention or treatment, pure experimental data are such that individuals are
randomly assigned to the control and treatment groups.
Random assignment is not always possible however, particularly when dealing with human
subjects. With quasi-experimental data, allocation to the control and treatment groups is not random but based on another criterion. An example is a study by Card and Krueger that is studied
in more detail in Chapter 7. They examined the effect of an increase in New Jersey’s minimum
wage in 1992 on the number of people employed in fast-food restaurants. The treatment group
was fast-food restaurants in New Jersey. The control group was fast-food restaurants in eastern
Pennsylvania where there was no change in the minimum wage. Another example is the effect on
spending habits of a change in the income tax rate for individuals above a threshold income. The
treatment group is the group with incomes above the threshold. The control group is those with
incomes below the threshold. When dealing with quasi-experimental data, one must be aware that
the effect of the treatment may be confounded with the effect of the criterion for assignment.
............................................................................................................................................
1 See
https://dataverse.harvard.edu/dataset.xhtml?persistentId=hdl:1902.1/10766 for program description, public use
data, and extensive literature.
k
k
k
1.5 Economic Data Types
1.4.3
7
Nonexperimental Data
An example of nonexperimental data is survey data. The Public Policy Research Lab at Louisiana
State University (www.survey.lsu.edu) conducts telephone and mail surveys for clients. In a telephone survey, numbers are selected randomly and called. Responses to questions are recorded
and analyzed. In such an environment, data on all variables are collected simultaneously, and the
values are neither fixed nor repeatable. These are nonexperimental data.
Such surveys are carried out on a massive scale by national governments. For example, the
Current Population Survey (CPS)2 is a monthly survey of about 50,000 households conducted
by the U.S. Bureau of the Census. The survey has been conducted for more than 50 years. The
CPS website says “CPS data are used by government policymakers and legislators as important
indicators of our nation’s economic situation and for planning and evaluating many government programs. They are also used by the press, students, academics, and the general public.”
In Section 1.8 we describe some similar data sources.
1.5
Economic Data Types
Economic data comes in a variety of “flavors.” In this section we describe and give an example
of each. In each example, be aware of the different data characteristics, such as the following:
k
1. Data may be collected at various levels of aggregation:
⚬ micro—data collected on individual economic decision-making units such as individuals,
households, and firms.
⚬ macro—data resulting from a pooling or aggregating over individuals, households, or
firms at the local, state, or national levels.
2. Data may also represent a flow or a stock:
⚬ flow—outcome measures over a period of time, such as the consumption of gasoline during the last quarter of 2018.
⚬ stock—outcome measured at a particular point in time, such as the quantity of crude oil
held by ExxonMobil in its U.S. storage tanks on November 1, 2018, or the asset value of
the Wells Fargo Bank on July 1, 2018.
3. Data may be quantitative or qualitative:
⚬ quantitative—outcomes such as prices or income that may be expressed as numbers or
some transformation of them, such as real prices or per capita income.
⚬ qualitative—outcomes that are of an “either-or” situation. For example, a consumer either
did or did not make a purchase of a particular good, or a person either is or is not married.
1.5.1
Time-Series Data
A time-series is data collected over discrete intervals of time. Examples include the annual price
of wheat in the United States and the daily price of General Electric stock shares. Macroeconomic
data are usually reported in monthly, quarterly, or annual terms. Financial data, such as stock
prices, can be recorded daily, or at even higher frequencies. The key feature of time-series data is
that the same economic quantity is recorded at a regular time interval.
For example, the annual real gross domestic product (GDP) for the United States is depicted
in Figure 1.1. A few values are given in Table 1.1. For each year, we have the recorded value.
The data are annual, or yearly, and have been “deflated” by the Bureau of Economic Analysis to
billions of real 2009 dollars.
............................................................................................................................................
2 www.census.gov/cps/
k
k
k
8
CHAPTER 1
An Introduction to Econometrics
17,000
16,000
15,000
GDP
14,000
13,000
12,000
11,000
10,000
9,000
1992
1996
2000
2004
Year
2008
2012
2016
FIGURE 1.1 Real U.S. GDP, 1994–2014.3
U.S. Annual GDP (Billions
of Real 2009 Dollars)
T A B L E 1.1
k
1.5.2
Year
GDP
2006
2007
2008
2009
2010
2011
2012
2013
2014
14,613.8
14,873.7
14,830.4
14,418.7
14,783.8
15,020.6
15,354.6
15,583.3
15,961.7
Cross-Section Data
A cross-section of data is collected across sample units in a particular time period. Examples are
income by counties in California during 2016 or high school graduation rates by state in 2015. The
“sample units” are individual entities and may be firms, persons, households, states, or countries.
For example, the CPS reports results of personal interviews on a monthly basis, covering items
such as employment, unemployment, earnings, educational attainment, and income. In Table 1.2,
we report a few observations from the March 2013 survey on the variables RACE, EDUCATION,
SEX, and WAGE (hourly wage rate).4 There are many detailed questions asked of the respondents.
............................................................................................................................................
3 Source:
www.bea.gov/national/index.htm
4 In
the actual raw data, the variable descriptions are coded differently to the names in Table 1.2. We have used
shortened versions for convenience.
k
k
k
1.6 The Research Process
T A B L E 1.2
9
Cross-Section Data: CPS, March 2013
Variables
Individual
RACE
EDUCATION
SEX
WAGE
1
2
3
4
5
6
7
8
9
White
White
Black
White
White
White
White
Black
White
Assoc Degree
Master’s Degree
Bachelor’s Degree
High School Graduate
Master’s Degree
Master’s Degree
Master’s Degree
Assoc Degree
Some College, No Degree
Male
Male
Male
Female
Male
Female
Female
Female
Female
10.00
60.83
17.80
30.38
12.50
49.50
23.08
28.95
9.20
1.5.3
k
Panel or Longitudinal Data
A “panel” of data, also known as “longitudinal” data, has observations on individual micro-units
that are followed over time. For example, the Panel Study of Income Dynamics (PSID)5 describes
itself as “a nationally representative longitudinal study of nearly 9000 U.S. families. Following the
same families and individuals since 1969, the PSID collects data on economic, health, and social
behavior.” Other national panels exist, and many are described at “Resources for Economists,” at
www.rfe.org.
To illustrate, data from two rice farms6 are given in Table 1.3. The data are annual observations on rice farms (or firms) over the period 1990–1997.
The key aspect of panel data is that we observe each micro-unit, here a farm, for a number of
time periods. Here we have amount of rice produced, area planted, labor input, and fertilizer use.
If we have the same number of time period observations for each micro-unit, which is the case
here, we have a balanced panel. Usually the number of time-series observations is small relative
to the number of micro-units, but not always. The Penn World Table7 provides purchasing power
parity and national income accounts converted to international prices for 182 countries for some
or all of the years 1950–2014.
1.6
The Research Process
Econometrics is ultimately a research tool. Students of econometrics plan to do research or they
plan to read and evaluate the research of others, or both. This section provides a frame of reference
and guide for future work. In particular, we show you the role of econometrics in research.
Research is a process, and like many such activities, it flows according to an orderly pattern.
Research is an adventure, and can be fun! Searching for an answer to your question, seeking new
knowledge, is very addictive—for the more you seek, the more new questions you will find.
A research project is an opportunity to investigate a topic that is important to you. Choosing
a good research topic is essential if you are to complete a project successfully. A starting point
is the question “What are my interests?” Interest in a particular topic will add pleasure to the
............................................................................................................................................
5 http://psidonline.isr.umich.edu
6 These
data were used by O’Donnell, C.J. and W.E. Griffiths (2006), Estimating State-Contingent Production Frontiers,
American Journal of Agricultural Economics, 88(1), 249–266.
7 www.rug.nl/ggdc/productivity/pwt
k
k
k
10
CHAPTER 1
An Introduction to Econometrics
T A B L E 1.3
k
Panel Data from Two Rice Farms
FARM
YEAR
PROD
AREA
LABOR
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
1990
1991
1992
1993
1994
1995
1996
1997
1990
1991
1992
1993
1994
1995
1996
1997
7.87
7.18
8.92
7.31
7.54
4.51
4.37
7.27
10.35
10.21
13.29
18.58
17.07
16.61
12.28
14.20
2.50
2.50
2.50
2.50
2.50
2.50
2.25
2.15
3.80
3.80
3.80
3.80
3.80
4.25
4.25
3.75
160
138
140
127
145
123
123
87
184
151
185
262
174
244
159
133
FERT
207.5
295.5
362.5
338.0
337.5
207.2
345.0
222.8
303.5
206.0
374.5
421.0
595.7
234.8
479.0
170.0
research effort. Also, if you begin working on a topic, other questions will usually occur to you.
These new questions may put another light on the original topic or may represent new paths to
follow that are even more interesting to you. The idea may come after lengthy study of all that
has been written on a particular topic. You will find that “inspiration is 99% perspiration.” That
means that after you dig at a topic long enough, a new and interesting question will occur to you.
Alternatively, you may be led by your natural curiosity to an interesting question. Professor Hal
Varian8 suggests that you look for ideas outside academic journals—in newspapers, magazines,
etc. He relates a story about a research project that developed from his shopping for a new TV set.
By the time you have completed several semesters of economics classes, you will find yourself enjoying some areas more than others. For each of us, specialized areas such as health
economics, economic development, industrial organization, public finance, resource economics,
monetary economics, environmental economics, and international trade hold a different appeal.
If you find an area or topic in which you are interested, consult the Journal of Economic Literature (JEL) for a list of related journal articles. The JEL has a classification scheme that makes
isolating particular areas of study an easy task. Alternatively, type a few descriptive words into
your favorite search engine and see what pops up.
Once you have focused on a particular idea, begin the research process, which generally
follows steps like these:
1. Economic theory gives us a way of thinking about the problem. Which economic variables
are involved, and what is the possible direction of the relationship(s)? Every research project,
given the initial question, begins by building an economic model and listing the questions
(hypotheses) of interest. More questions will arise during the research project, but it is good
to list those that motivate you at the project’s beginning.
2. The working economic model leads to an econometric model. We must choose a functional
form and make some assumptions about the nature of the error term.
............................................................................................................................................
8 Varian,
H. How to Build an Economic Model in Your Spare Time, The American Economist, 41(2), Fall 1997,
pp. 3–10.
k
k
k
1.7 Writing an Empirical Research Paper
11
3. Sample data are obtained and a desirable method of statistical analysis chosen, based on
initial assumptions and an understanding of how the data were collected.
4. Estimates of the unknown parameters are obtained with the help of a statistical software
package, predictions are made, and hypothesis tests are performed.
5. Model diagnostics are performed to check the validity of assumptions. For example, were
all of the right-hand side explanatory variables relevant? Was an adequate functional form
used?
6. The economic consequences and the implications of the empirical results are analyzed and
evaluated. What economic resource allocation and distribution results are implied, and what
are their policy-choice implications? What remaining questions might be answered with
further study or with new and better data?
These steps provide some direction for what must be done. However, research always includes
some surprises that may send you back to an earlier point in your research plan or that may even
cause you to revise it completely. Research requires a sense of urgency, which keeps the project
moving forward, the patience not to rush beyond careful analysis, and the willingness to explore
new ideas.
1.7
k
Writing an Empirical Research Paper
Research rewards you with new knowledge, but it is incomplete until a research paper or report
is written. The process of writing forces the distillation of ideas. In no other way will your
depth of understanding be so clearly revealed. When you have difficulty explaining a concept
or thought, it may mean that your understanding is incomplete. Thus, writing is an integral part
of research. We provide this section as a building block for future writing assignments. Consult it as needed. You will find other tips on writing economics papers on the book website,
www.principlesofeconometrics.com.
1.7.1
Writing a Research Proposal
After you have selected a specific topic, it is a good idea to write up a brief project summary, or
proposal. Writing it will help to focus your thoughts about what you really want to do. Show it to
your colleagues or instructor for preliminary comments. The summary should be short, usually
no longer than 500 words, and should include the following:
1. A concise statement of the problem
2. Comments on the information that is available, with one or two key references
3. A description of the research design that includes
a. the economic model
b. the econometric estimation and inference methods
c. data sources
d. estimation, hypothesis testing, and prediction procedures, including the econometric
software and version used
4. The potential contribution of the research
1.7.2
A Format for Writing a Research Report
Economic research reports have a standard format in which the various steps of the research
project are discussed and the results interpreted. The following outline is typical.
k
k
k
12
k
CHAPTER 1
An Introduction to Econometrics
1. Statement of the Problem The place to start your report is with a summary of the questions
you wish to investigate as well as why they are important and who should be interested in the
results. This introductory section should be nontechnical and should motivate the reader
to continue reading the paper. It is also useful to map out the contents of the following
sections of the report. This is the first section to work on and also the last. In today’s busy
world, the reader’s attention must be garnered very quickly. A clear, concise, well-written
introduction is a must and is arguably the most important part of the paper.
2. Review of the Literature Briefly summarize the relevant literature in the research area
you have chosen and clarify how your work extends our knowledge. By all means, cite the
works of others who have motivated your research, but keep it brief. You do not have to
survey everything that has been written on the topic.
3. The Economic Model Specify the economic model that you used and define the economic
variables. State the model’s assumptions and identify hypotheses that you wish to test.
Economic models can get complicated. Your task is to explain the model clearly, but as
briefly and simply as possible. Don’t use unnecessary technical jargon. Use simple terms
instead of complicated ones when possible. Your objective is to display the quality of your
thinking, not the extent of your vocabulary.
4. The Econometric Model Discuss the econometric model that corresponds to the economic
model. Make sure you include a discussion of the variables in the model, the functional
form, the error assumptions, and any other assumptions that you make. Use notation that is
as simple as possible, and do not clutter the body of the paper with long proofs or derivations; these can go into a technical appendix.
5. The Data Describe the data you used, as well as the source of the data and any reservations
you have about their appropriateness.
6. The Estimation and Inference Procedures Describe the estimation methods you used and
why they were chosen. Explain hypothesis testing procedures and their usage. Indicate the
software used and the version, such as Stata 15 or EViews 10.
7. The Empirical Results and Conclusions Report the parameter estimates, their interpretation, and the values of test statistics. Comment on their statistical significance, their relation
to previous estimates, and their economic implications.
8. Possible Extensions and Limitations of the Study Your research will raise questions about
the economic model, data, and estimation techniques. What future research is suggested by
your findings, and how might you go about performing it?
9. Acknowledgments It is appropriate to recognize those who have commented on and contributed to your research. This may include your instructor, a librarian who helped you find
data, or a fellow student who read and commented on your paper.
10. References An alphabetical list of the literature you cite in your study, as well as references
to the data sources you used.
Once you’ve written the first draft, use your computer’s spell-check software to check for spelling
errors. Have a friend read the paper, make suggestions for clarifying the prose, and check your
logic and conclusions. Before you submit the paper, you should eliminate as many errors as possible. Your work should look good. Use a word processor, and be consistent with font sizes,
section headings, style of footnotes, references, and so on. Often software developers provide
templates for term papers and theses. A little searching for a good paper layout before beginning
is a good idea. Typos, missing references, and incorrect formulas can spell doom for an otherwise
excellent paper. Some do’s and don’ts are summarized nicely, and with good humor, by Deidre
N. McClosky in Economical Writing, 2nd edition (Prospect Heights, IL: Waveland Press, Inc.,
1999).
While it is not a pleasant topic to discuss, you should be aware of the rules of plagiarism.
You must not use someone else’s words as if they were your own. If you are unclear about what
you can and cannot use, check with the style manuals listed in the next paragraph, or consult
k
k
k
1.8 Sources of Economic Data
13
your instructor. Your university may provide a plagiarism-checking software, such as Turnitin
or iThenticate, that will compare your paper to millions of online sources and look for problem
areas. There are some free online versions as well. The paper should have clearly defined sections
and subsections. The pages, equations, tables, and figures should be numbered. References and
footnotes should be formatted in an acceptable fashion. A style guide is a good investment. Two
classics are the following:
• The Chicago Manual of Style, 16th edition, is available online and in other formats.
• A Manual for Writers of Research Papers, Theses, and Dissertations: Chicago Style for Students and Researchers, 8th edition, by Kate L. Turabian; revised by Wayne C. Booth, Gregory
G. Colomb, and Joseph M Williams (2013, University of Chicago Press).
1.8
Sources of Economic Data
Economic data are much easier to obtain since the development of the World Wide Web. In this
section we direct you to some places on the Internet where economic data are accessible. During
your study of econometrics, browse some of the sources listed to gain some familiarity with data
availability.
1.8.1
Links to Economic Data on the Internet
There are a number of fantastic sites on the World Wide Web for obtaining economic data.
k
Resources for Economists (RFE)
www.rfe.org is a primary gateway to resources on
the Internet for economists. This excellent site is the work of Bill Goffe. Here you will find links
to sites for economic data and sites of general interest to economists. The Data link has these
broad data categories:
• U.S. Macro and Regional Data Here you will find links to various data sources such as the
Bureau of Economic Analysis, Bureau of Labor Statistics, Economic Reports of the President, and the Federal Reserve Banks.
• Other U.S. Data Here you will find links to the U.S. Census Bureau, as well as links to
many panel and survey data sources. The gateway to U.S. government agencies is FedStats
(fedstats.sites.usa.gov). Once there, click on Agencies to see a complete list of U.S. government agencies and links to their homepages.
• World and Non-U.S. Data Here there are links to world data, such as at the CIA World
Factbook and the Penn World Tables, as well as international organizations such as the Asian
Development Bank, the International Monetary Fund, the World Bank, and so on. There are
also links to sites with data on specific countries and sectors of the world.
• Finance and Financial Markets Here are links to sources of U.S. and world financial data
on variables such as exchange rates, interest rates, and share prices.
• Journal Data and Program Archives Some economic journals post data used in articles.
Links to these journals are provided here. (Many of the articles in these journals will be
beyond the scope of undergraduate economics majors.)
National Bureau of Economic Research (NBER)
access to a great amount of data. There are headings for
• Macro Data
• Industry Productivity and Digitalization Data
• International Trade Data
k
www.nber.org/data provides
k
k
14
CHAPTER 1
An Introduction to Econometrics
•
•
•
•
•
Individual Data
Healthcare Data—Hospitals, Providers, Drugs, and Devices
Demographic and Vital Statistics
Patent and Scientific Papers Data
Other Data
Economagic Some websites make extracting data relatively easy. For example, Economagic (www.economagic.com) is an excellent and easy-to-use source of macro time series (some
100,000 series available). The data series are easily viewed in a copy and paste format, or graphed.
1.8.2
Interpreting Economic Data
In many cases it is easier to obtain economic data than it is to understand the meaning of the data.
It is essential when using macroeconomic or financial data that you understand the definitions
of the variables. Just what is the index of leading economic indicators? What is included in personal consumption expenditures? You may find the answers to some questions like these in your
textbooks. Another resource you might find useful is A Guide to Everyday Economic Statistics,
7th edition, by Gary E. Clayton and Martin Gerhard Giesbrecht, (Boston: Irwin/McGraw-Hill,
2009). This slender volume examines how economic statistics are constructed, and how they can
be used.
1.8.3
k
Obtaining the Data
Finding a data source is not the same as obtaining the data. Although there are a great many
easy-to-use websites, “easy-to-use” is a relative term. The data will come packaged in a variety
of formats. It is also true that there are many, many variables at each of these websites. A primary
challenge is identifying the specific variables that you want, and what exactly they measure. The
following examples are illustrative.
The Federal Reserve Bank of St. Louis 9 has a system called FRED (Federal Reserve Economic Data). Under “Categories” there are links to financial variables, population and labor variables, national accounts, and many others. Data on these variables can be downloaded in a number
of formats. For reading the data, you may need specific knowledge of your statistical software.
Accompanying Principles of Econometrics, 5e, are computer manuals for Excel, EViews, Stata,
SAS, R, and Gretl to aid this process. See the publisher website www.wiley.com/college/hill, or
the book website at www.principlesofeconometrics.com for a description of these aids.
The CPS (www.census.gov/cps) has a tool called DataFerrett. This tool will help you find
and download data series that are of particular interest to you. There are tutorials that guide you
through the process. Variable descriptions, as well as the specific survey questions, are provided
to aid in your selection. It is somewhat like an Internet shopping site. Desired series are “ticked”
and added to a “Shopping Basket.” Once you have filled your basket, you download the data to use
with specific software. Other Web-based data sources operate in this same manner. One example
is the PSID.10 The Penn World Tables11 offer data downloads in both Excel and Stata formats.
You can expect to find massive amounts of readily available data at the various sites we have
mentioned, but there is a learning curve. You should not expect to find, download, and process
the data without considerable work effort. Being skilled with Excel and statistical software is a
must if you plan to regularly use these data sources.
............................................................................................................................................
9 https://fred.stlouisfed.org
10 http://psidonline.isr.umich.edu
11 www.rug.nl/ggdc/productivity/pwt
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 15
Probability Primer
LEARNING OBJECTIVES
Remark
Learning Objectives and Keywords sections will appear at the beginning of each chapter. We
urge you to think about, and possibly write out answers to the questions, and make sure you
recognize and can define the keywords. If you are unsure about the questions or answers,
consult your instructor. When examples are requested in Learning Objectives sections, you
should think of examples not in the book.
k
Based on the material in this primer, you should be able to
1. Explain the difference between a random
variable and its values, and give an example.
9. Use the definition of expected value for a
discrete random variable to compute expectations, given a pdf f (x) and a function g(X) of X.
2. Explain the difference between discrete and
continuous random variables, and give
examples of each.
10. Define the variance of a discrete random
variable, and explain in what sense the values of
a random variable are more spread out if the
variance is larger.
3. State the characteristics of a probability density
function (pdf ) for a discrete random variable,
and give an example.
11. Use a joint pdf (table) for two discrete random
variables to compute probabilities of joint
events and to find the (marginal) pdf of each
individual random variable.
4. Compute probabilities of events, given a discrete
probability function.
5. Explain the meaning of the following statement:
‘‘The probability that the discrete random
variable takes the value 2 is 0.3.’’
12. Find the conditional pdf for one discrete random
variable given the value of another and their
joint pdf .
6. Explain how the pdf of a continuous random
variable is different from the pdf of a discrete
random variable.
13. Work with single and double summation
notation.
14. Give an intuitive explanation of statistical
independence of two random variables, and
state the conditions that must hold to prove
statistical independence. Give examples of two
independent random variables and two
dependent random variables.
7. Show, geometrically, how to compute
probabilities given a pdf for a continuous
random variable.
8. Explain, intuitively, the concept of the mean, or
expected value, of a random variable.
15
k
k
Trim Size: 8in x 10in
16
k
Page 16
Probability Primer
15. Define the covariance and correlation between
two random variables, and compute these
values given a joint probability function of two
discrete random variables.
17. Use Statistical Table 1, Cumulative Probabilities
for the Standard Normal Distribution, and your
computer software to compute probabilities
involving normal random variables.
16. Find the mean and variance of a sum of random
variables.
18. Use the Law of Iterated Expectations to find the
expected value of a random variable.
KEYWORDS
conditional expectation
conditional pdf
conditional probability
continuous random variable
correlation
covariance
cumulative distribution function
discrete random variable
expected value
k
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
experiment
indicator variable
iterated expectation
joint probability density function
marginal distribution
mean
normal distribution
population
probability
probability density function
random variable
standard deviation
standard normal distribution
statistical independence
summation operations
variance
We assume that you have had a basic probability and statistics course. In this primer, we review
some essential probability concepts. Section P.1 defines discrete and continuous random variables. Probability distributions are discussed in Section P.2. Section P.3 introduces joint probability distributions, defines conditional probability and statistical independence. In Section P.4, we
digress and discuss operations with summations. In Section P.5, we review the properties of probability distributions, paying particular attention to expected values and variances. In Section P.6,
we discuss the important concept of conditioning, and how knowing the value of one variable
might provide information about, or help us predict, another variable. Section P.7 summarizes
important facts about the normal probability distribution. In Appendix B, “Probability Concepts,”
are enhancements and additions to this material.
P.1
Random Variables
Benjamin Franklin is credited with the saying “The only things certain in life are death and taxes.”
While not the original intent, this bit of wisdom points out that almost everything we encounter
in life is uncertain. We do not know how many games our football team will win next season. You
do not know what score you will make on the next exam. We don’t know what the stock market
index will be tomorrow. These events, or outcomes, are uncertain, or random. Probability gives
us a way to talk about possible outcomes.
A random variable is a variable whose value is unknown until it is observed; in other words,
it is a variable that is not perfectly predictable. Each random variable has a set of possible values it
can take. If W is the number of games our football team wins next year, then W can take the values
0, 1, 2, …, 13, if there are a maximum of 13 games. This is a discrete random variable since
it can take only a limited, or countable, number of values. Other examples of discrete random
variables are the number of computers owned by a randomly selected household, and the number
of times you will visit your physician next year. A special case occurs when a random variable
can only be one of two possible values—for example, in a phone survey, if you are asked if you
are a college graduate or not, your answer can only be “yes” or “no.” Outcomes like this can
be characterized by an indicator variable taking the values one if yes or zero if no. Indicator
variables are discrete and are used to represent qualitative characteristics such as sex (male or
female) or race (white or nonwhite).
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.2 Probability Distributions
Page 17
17
The U.S. GDP is yet another example of a random variable, because its value is unknown
until it is observed. In the third quarter of 2014 it was calculated to be 16,164.1 billion dollars.
What the value will be in the second quarter of 2025 is unknown, and it cannot be predicted
perfectly. GDP is measured in dollars and it can be counted in whole dollars, but the value is so
large that counting individual dollars serves no purpose. For practical purposes, GDP can take
any value in the interval zero to infinity, and it is treated as a continuous random variable.
Other common macroeconomic variables, such as interest rates, investment, and consumption,
are also treated as continuous random variables. In finance, stock market indices, like the Dow
Jones Industrial Index, are also treated as continuous. The key attribute of these variables that
makes them continuous is that they can take any value in an interval.
P.2
k
Probability Distributions
Probability is usually defined in terms of experiments. Let us illustrate this in the context of a
simple experiment. Consider the objects in Table P.1 to be a population of interest. In statistics and
econometrics, the term population is an important one. A population is a group of objects, such
as people, farms, or business firms, having something in common. The population is a complete
set and is the focus of an analysis. In this case the population is the set of ten objects shown
in Table P.1. Using this population, we will discuss some probability concepts. In an empirical
analysis, a sample of observations is collected from the population of interest, and using the
sample observations we make inferences about the population.
If we were to select one cell from the table at random (imagine cutting the table into 10
equally sized pieces of paper, stirring them up, and drawing one of the slips without looking), that
would constitute a random experiment. Based on this random experiment, we can define several
random variables. For example, let the random variable X be the numerical value showing on a
slip that we draw. (We use uppercase letters like X to represent random variables in this primer).
The term random variable is a bit odd, as it is actually a rule for assigning numerical values to
experimental outcomes. In the context of Table P.1, the rule says, “Perform the experiment (stir
the slips, and draw one) and for the slip that you obtain assign X to be the number showing.”
The values that X can take are denoted by corresponding lowercase letters, x, and in this case the
values of X are x = 1, 2, 3, or 4.
For the experiment using the population in Table P.1,1 we can create a number of random
variables. Let Y be a discrete random variable designating the color of the slip, with Y = 1 denoting
T A B L E P.1
The Seussian Slips: A Population
1
2
3
4
4
2
3
3
4
4
............................................................................................................................................
1A
table suitable for classroom experiments can be obtained at www.principlesofeconometrics.com/poe5/extras/
table_p1. We thank Veronica Deschner McGregor for the suggestion of “One slip, two slip, white slip, blue slip” for this
experiment, inspired by Dr. Seuss’s “One Fish Two Fish Red Fish Blue Fish (I Can Read It All by Myself),” Random
House Books for Young Readers (1960).
k
k
Trim Size: 8in x 10in
18
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 18
Probability Primer
a shaded slip and Y = 0 denoting a slip with no shading. The numerical values that Y can take are
y = 0, 1.
Consider X, the numerical value on the slip. If the slips are equally likely to be chosen after
shuffling, then in a large number of experiments (i.e., shuffling and drawing one of the ten slips),
10% of the time we would observe X = 1, 20% of the time X = 2, 30% of the time X = 3, and
40% of the time X = 4. These are probabilities that the specific values will occur. We would say,
for example, P(X = 3) = 0.3. This interpretation is tied to the relative frequency of a particular
outcome’s occurring in a large number of experiment replications.
We summarize the probabilities of possible outcomes using a probability density function
(pdf ). The pdf for a discrete random variable indicates the probability of each possible value
occurring. For a discrete random variable X the value of the pdf f (x) is the probability that the
random variable X takes the value x, f (x) = P(X = x). Because f (x) is a probability, it must be true
that 0 ≤ f (x) ≤ 1 and, if X takes n possible values x1 ,…, xn , then the sum of their probabilities
must be one
( )
( )
( )
(P.1)
𝑓 x1 + 𝑓 x2 + · · · + 𝑓 xn = 1
For discrete random variables, the pdf might be presented as a table, such as in Table P.2.
As shown in Figure P.1, the pdf may also be represented as a bar graph, with the height of
the bar representing the probability with which the corresponding value occurs.
The cumulative distribution function (cdf ) is an alternative way to represent probabilities.
The cdf of the random variable X, denoted F(x), gives the probability that X is less than or equal
to a specific value x. That is,
F(x) = P(X ≤ x)
(P.2)
k
T A B L E P.2
Probability
Density
Function of X
x
f (x)
1
2
3
4
0.1
0.2
0.3
0.4
k
0.45
0.40
Probability
0.35
0.30
0.25
0.20
0.15
0.10
X value
0.05
1
2
3
4
FIGURE P.1 Probability density function for X.
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.2 Probability Distributions
E X A M P L E P.1
Page 19
19
Using a cdf
Using the probabilities in Table P.2, we find that F(1) =
P(X ≤ 1) = 0.1, F(2) = P(X ≤ 2) = 0.3, F(3) = P(X ≤ 3) =
0.6, and F(4) = P(X ≤ 4) = 1. For example, using the pdf
f (x) we compute the probability that X is less than or equal
to 2 as
F(2) = P(X ≤ 2) = P(X = 1) + P(X = 2) = 0.1 + 0.2 = 0.3
Since the sum of the probabilities P(X = 1) + P(X = 2) +
P(X = 3) + P(X = 4) = 1, we can compute the probability
that X is greater than 2 as
P(X > 2) = 1 − P(X ≤ 2) = 1 − F(2) = 1 − 0.3 = 0.7
An important difference between the pdf and cdf for X is
revealed by the question “Using the probability distribution
in Table P.2, what is the probability that X = 2.5?” This probability is zero because X cannot take this value. The question
“What is the probability that X is less than or equal to 2.5?”
does have an answer.
F(2.5) = P(X ≤ 2.5) = P(X = 1) + P(X = 2)
= 0.1 + 0.2 = 0.3
The cumulative probability can be calculated for any x
between −∞ and +∞.
Continuous random variables can take any value in an interval and have an uncountable number
of values. Consequently, the probability of any specific value is zero. For continuous random
variables, we talk about outcomes being in a certain range. Figure P.2 illustrates the pdf f (x) of a
continuous random variable X that takes values of x from 0 to infinity. The shape is representative
of the distribution for an economic variable such as an individual’s income or wage. Areas under
the curve represent probabilities that X falls in an interval. The cdf F(x) is defined as in (P.2). For
this distribution,
P(10 < X < 20) = F(20) − F(10) = 0.52236 − 0.17512
k
= 0.34724
(P.3)
How are these areas obtained? The integral from calculus gives the area under a curve. We will
not compute many integrals in this book.2 Instead, we will use the computer and compute cdf
values and probabilities using software commands.
0
f(x)
P(10 < X < 20)
0
10
20
x
30
40
50
FIGURE P.2 Probability density function for a continuous random
variable.
............................................................................................................................................
2 See
Appendix A.4 for a brief explanation of integrals, and illustrations using integrals to compute probabilities in
Appendix B.2.1. The calculations in (P.3) are explained in Appendix B.3.9.
k
k
Trim Size: 8in x 10in
20
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 20
Probability Primer
Joint, Marginal, and Conditional
Probabilities
P.3
Working with more than one random variable requires a joint probability density function. For
the population in Table P.1 we defined two random variables, X the numeric value of a randomly
drawn slip and the indicator variable Y that equals 1 if the selected slip is shaded, and 0 if it is
not shaded.
Using the joint pdf for X and Y we can say “The probability of selecting a shaded 2 is 0.10.”
This is a joint probability because we are talking about the probability of two events occurring
simultaneously; the selection takes the value X = 2 and the slip is shaded so that Y = 1. We can
write this as
P(X = 2 and Y = 1) = P(X = 2, Y = 1) = 𝑓 (x = 2, y = 1) = 0.1
The entries in Table P.3 are probabilities f (x, y) = P(X = x, Y = y) of joint outcomes. Like the pdf
of a single random variable, the sum of the joint probabilities is 1.
P.3.1
Marginal Distributions
Given a joint pdf , we can obtain the probability distributions of individual random variables,
which are also known as marginal distributions. In Table P.3, we see that a shaded slip, Y = 1,
can be obtained with the values x = 1, 2, 3, and 4. The probability that we select a shaded slip
is the sum of the probabilities that we obtain a shaded 1, a shaded 2, a shaded 3, and a shaded 4.
The probability that Y = 1 is
P(Y = 1) = 𝑓Y (1) = 0.1 + 0.1 + 0.1 + 0.1 = 0.4
k
k
This is the sum of the probabilities across the second row of the table. Similarly the probability of drawing a white slip is the sum of the probabilities across the first row of the table, and
P(Y = 0) = fY (0) = 0 + 0.1 + 0.2 + 0.3 = 0.6, where fY (y) denotes the pdf of the random variable
Y. The probabilities P(X = x) are computed similarly by summing down across the values of Y.
The joint and marginal distributions are often reported as in Table P.4.3
Joint Probability
Density Function
for X and Y
T A B L E P.3
x
T A B L E P.4
y
1
2
3
4
0
1
0
0.1
0.1
0.1
0.2
0.1
0.3
0.1
Joint and Marginal Probabilities
y/x
1
2
3
4
f (y)
0
1
f (x)
0
0.1
0.1
0.1
0.1
0.2
0.2
0.1
0.3
0.3
0.1
0.4
0.6
0.4
1.0
............................................................................................................................................
3 Similar
calculations for continuous random variables use integration. See Appendix B.2.3 for an illustration.
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.3 Joint, Marginal, and Conditional Probabilities
P.3.2
Page 21
21
Conditional Probability
What is the probability that a randomly chosen slip will take the value 2 given that it is shaded?
This question is about the conditional probability of the outcome X = 2 given that the outcome
Y = 1 has occurred. The effect of the conditioning is to reduce the set of possible outcomes.
Conditional on Y = 1 we only consider the four possible slips that are shaded. One of them is a
2, so the conditional probability of the outcome X = 2 given that Y = 1 is 0.25. There is a one
in four chance of selecting a 2 given only the shaded slips. Conditioning reduces the size of the
population under consideration, and conditional probabilities characterize the reduced population.
For discrete random variables the probability that the random variable X takes the value x given
that Y = y is written P(X = x|Y = y). This conditional probability is given by the conditional pdf
f (x|y)
P(X = x, Y = y) 𝑓 (x, y)
=
(P.4)
𝑓 (x|y) = P(X = x|Y = y) =
P(Y = y)
𝑓Y (y)
where fY (y) is the marginal pdf of Y.
E X A M P L E P.2
Calculating a Conditional Probability
𝑓 (x = 2|y = 1) = P(X = 2|Y = 1)
Using the marginal probability P(Y =1) = 0.4, the conditional
pdf of X given Y = 1 is obtained by using (P.4) for each value
of X. For example,
𝑓 (x = 2, y = 1)
P(X = 2, Y = 1)
=
P(Y = 1)
𝑓Y (1)
0.1
= 0.25
=
0.4
=
k
A key point to remember is that by conditioning we are considering only the subset of a population
for which the condition holds. Probability calculations are then based on the “new” population.
We can repeat this process for each value of X to obtain the complete conditional pdf given in
Table P.5.
P.3.3
Statistical Independence
When selecting a shaded slip from Table P.1, the probability of selecting each possible outcome,
x = 1, 2, 3, and 4 is 0.25. In the population of shaded slips the numeric values are equally likely.
The probability of randomly selecting X = 2 from the entire population, from the marginal pdf ,
is P(X = 2) = fX (2) = 0.2. This is different from the conditional probability. Knowing that the
slip is shaded tells us something about the probability of obtaining X = 2. Such random variables are dependent in a statistical sense. Two random variables are statistically independent,
or simply independent, if the conditional probability that X = x given that Y = y is the same
as the unconditional probability that X = x. This means, if X and Y are independent random
variables, then
P(X = x|Y = y) = P(X = x)
T A B L E P.5
x
𝑓 (x|y = 1)
(P.5)
Conditional Probability of X Given Y = 1
1
2
3
4
0.25
0.25
0.25
0.25
k
k
Trim Size: 8in x 10in
22
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 22
Probability Primer
Equivalently, if X and Y are independent, then the conditional pdf of X given Y = y is the same
as the unconditional, or marginal, pdf of X alone.
𝑓 (x|y) =
𝑓 (x, y)
= 𝑓X (x)
𝑓Y (y)
(P.6)
Solving (P.6) for the joint pdf , we can also say that X and Y are statistically independent if their
joint pdf factors into the product of their marginal pdf s
P(X = x, Y = y) = 𝑓 (x, y) = 𝑓X (x) 𝑓Y (y) = P(X = x) × P(Y = y)
(P.7)
If (P.5) or (P.7) is true for each and every pair of values x and y, then X and Y are statistically
independent. This result extends to more than two random variables. The rule allows us to
check the independence of random variables X and Y in Table P.4. If (P.7) is violated for any
pair of values, then X and Y are not statistically independent. Consider the pair of values X = 1
and Y = 1.
P(X = 1, Y = 1) = 𝑓 (1, 1) = 0.1 ≠ 𝑓X (1) 𝑓Y (1) = P(X = 1) × P(Y = 1) = 0.1 × 0.4 = 0.04
The joint probability is 0.1 and the product of the individual probabilities is 0.04. Since these
are not equal, we can conclude that X and Y are not statistically independent.
P.4
k
A Digression: Summation Notation
∑
Throughout this book we will use a summation sign, denoted by the symbol , to shorten algebraic expressions. Suppose the random variable X takes the values x1 , x2 , …, x15 . The sum of
these values is x1 + x2 + · · · + x15 . Rather than write this sum out each time we will represent
∑15
∑15
it as i=1 xi , so that i=1 xi = x1 + x2 + · · · + x15 . If we sum n terms, a general number, then the
∑n
summation will be i=1 xi = x1 + x2 + · · · + xn . In this notation
∑
• The symbol is the capital Greek letter sigma and means “the sum of.”
• The letter i is called the index of summation. This letter is arbitrary and may also appear
as t, j, or k.
∑n
• The expression i=1 xi is read “the sum of the terms xi , from i equals 1 to n.”
• The numbers 1 and n are the lower limit and upper limit of summation.
The following rules apply to the summation operation.
Sum 1. The sum of n values x1 , …, xn is
n
∑
i=1
xi = x1 + x2 + · · · + xn
Sum 2. If a is a constant, then
n
∑
i=1
axi = a
n
∑
xi
i=1
Sum 3. If a is a constant, then
n
∑
a = a + a + · · · + a = na
i=1
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.5 Properties of Probability Distributions
Page 23
23
Sum 4. If X and Y are two variables, then
n (
∑
n
n
) ∑
∑
xi + yi = xi + yi
i=1
i=1
i=1
Sum 5. If X and Y are two variables, then
n (
∑
n
n
)
∑
∑
axi + byi = a xi + b yi
i=1
i=1
i=1
Sum 6. The arithmetic mean (average) of n values of X is
n
∑
x=
i=1
n
xi
=
x1 + x2 + · · · + xn
n
Sum 7. A property of the arithmetic mean (average) is that
n (
∑
n
n
n
n
n
) ∑
∑
∑
∑
∑
xi − x = xi − x = xi − nx = xi − xi = 0
i=1
i=1
i=1
i=1
i=1
i=1
Sum 8. We often use an abbreviated form of the summation notation. For example, if f (x) is a
function of the values of X,
n
( )
( )
( )
( )
∑
𝑓 xi = 𝑓 x1 + 𝑓 x2 + · · · + 𝑓 xn
i=1
∑ ( )
= 𝑓 xi (“Sum over all values of the index i ”)
i
=
k
∑
𝑓 (x) (“Sum over all possible values of X”)
x
Sum 9. Several summation signs can be used in one expression. Suppose the variable Y takes
n values and X takes m values, and let f (x, y) = x + y. Then the double summation of
this function is
m n
m n (
) ∑
)
∑
∑ (
∑
xi + yj
𝑓 xi , yj =
i=1 j=1
i=1 j=1
To evaluate such expressions work from the innermost sum outward. First set i = 1 and sum over
all values of j, and so on. That is,
m n
m[ (
) ∑
)
(
)
(
)]
∑
∑ (
𝑓 xi , y1 + 𝑓 xi , y2 + · · · + 𝑓 xi , yn
𝑓 xi , yj =
i=1 j=1
i=1
The order of summation does not matter, so
m n
n m
) ∑
)
∑
∑ (
∑ (
𝑓 xi , yj =
𝑓 xi , yj
i=1 j=1
P.5
j=1 i=1
Properties of Probability Distributions
Figures P.1 and P.2 give us a picture of how frequently values of the random variables will occur.
Two key features of a probability distribution are its center (location) and width (dispersion).
A key measure of the center is the mean, or expected value. Measures of dispersion are variance,
and its square root, the standard deviation.
k
k
Trim Size: 8in x 10in
24
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 24
Probability Primer
P.5.1
Expected Value of a Random Variable
The mean of a random variable is given by its mathematical expectation. If X is a discrete
random variable taking the values x1 , … , xn , then the mathematical expectation, or expected
value, of X is
)
)
)
(
(
(
(P.8)
E(X) = x1 P X = x1 + x2 P X = x2 + · · · + xn P X = xn
The expected value, or mean, of X is a weighted average of its values, the weights being the
probabilities that the values occur. The uppercase letter “E” represents the expected value
operation. E(X) is read as “the expected value of X.” The expected value of X is also called
the mean of X. The mean is often symbolized by μ or μX . It is the average value of the random
variable in an infinite number of repetitions of the underlying experiment. The mean of a random variable is the population mean. We use Greek letters for population parameters because
later on we will use data to estimate these real world unknowns. In particular, keep separate the
population mean μ and the arithmetic (or sample) mean x that we introduced in Section P.4 as
Sum 6. This can be particularly confusing when a conversation includes the term “mean” without
the qualifying term “population” or “arithmetic.” Pay attention to the usage context.
E X A M P L E P.3
Calculating an Expected Value
For the population in Table P.1, the expected value of X is
E(X) = 1 × P(X = 1) + 2 × P(X = 2) + 3 × P(X = 3) + 4 × P(X = 4)
= (1 × 0.1) + (2 × 0.2) + (3 × 0.3) + (4 × 0.4) = 3
k
k
For a discrete random variable the probability that X takes the value x is given by its pdf f (x),
P(X = x) = f (x). The expected value in (P.8) can be written equivalently as
( )
( )
( )
μX = E(X) = x1 𝑓 x1 + x2 𝑓 x2 + · · · + xn 𝑓 xn
n
( ) ∑
∑
(P.9)
= xi 𝑓 xi = x𝑓 (x)
x
i=1
Using (P.9), the expected value of X, the numeric value on a randomly drawn slip from Table P.1 is
μX = E(X) =
4
∑
x𝑓 (x) = (1 × 0.1) + (2 × 0.2) + (3 × 0.3) + (4 × 0.4) = 3
x=1
What does this mean? Draw one “slip” at random from Table P.1, and observe its numerical
value X. This constitutes an experiment. If we repeat this experiment many times, the values
x = 1, 2, 3, and 4 will appear 10%, 20%, 30%, and 40% of the time, respectively. The arithmetic
average of all the numerical values will approach μX = 3, as the number of experiments becomes
large. The key point is that the expected value of the random variable is the average value
that occurs in many repeated trials of an experiment.
For continuous random variables, the interpretation of the expected value of X is
unchanged—it is the average value of X if many values are obtained by repeatedly performing
the underlying random experiment.4
............................................................................................................................................
4 Since
there are now an uncountable number of values to sum, mathematically we must replace the “summation over all
possible values” in (P.9) by the “integral over all possible values.” See Appendix B.2.2 for a brief discussion.
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.5 Properties of Probability Distributions
P.5.2
Page 25
25
Conditional Expectation
Many economic questions are formulated in terms of conditional expectation, or the conditional
mean. One example is “What is the mean (expected value) wage of a person who has 16 years
of education?” In expected value notation, what is E(WAGE|EDUCATION = 16)? For a discrete
random variable, the calculation of conditional expected value uses (P.9) with the conditional pdf
𝑓 (x|y) replacing f (x), so that
∑
μX|Y = E(X|Y = y) = x𝑓 (x|y)
x
E X A M P L E P.4
Calculating a Conditional Expectation
Using the population in Table P.1, what is the expected
numerical value of X given that Y = 1, the slip is shaded? The
conditional probabilities 𝑓 (x|y = 1) are given in Table P.5.
The conditional expectation of X is
E(X|Y = 1) =
4
∑
x𝑓 (x|1) = 1 × 𝑓 (1|1) + 2 × 𝑓 (2|1)
The average value of X in many repeated trials of the experiment of drawing from the shaded slips is 2.5. This example
makes a good point about expected values in general, namely
that the expected value of X does not have to be a value that
X can take. The expected value of X is not the value that you
expect to occur in any single experiment.
x=1
+ 3 × 𝑓 (3|1) + 4 × 𝑓 (4|1)
= 1(0.25) + 2(0.25) + 3(0.25) + 4(0.25) = 2.5
k
What is the conditional expectation of X given that Y = y if the random variables are statistically
independent? If X and Y are statistically independent the conditional pdf f (x|y) equals the pdf of
X alone, f (x), as shown in (P.6). The conditional expectation is then
∑
∑
E(X|Y = y) = x𝑓 (x|y) = x𝑓 (x) = E(X)
x
x
If X and Y are statistically independent, conditioning does not affect the expected value.
P.5.3
Rules for Expected Values
Functions of random variables are also random. If g(X) is a function of the random variable X,
such as g(X) = X 2 , then g(X) is also random. If X is a discrete random variable, then the expected
value of g(X) is obtained using calculations similar to those in (P.9).
] ∑
[
(P.10)
E g(X) = g(x) 𝑓 (x)
x
For example, if a is a constant, then g(X) = aX is a function of X, and
] ∑
[
E(aX) = E g(X) = g(x) 𝑓 (x)
x
∑
∑
= ax𝑓 (x) = a x𝑓 (x)
x
x
= aE(X)
Similarly, if a and b are constants, then we can show that
E(aX + b) = aE(X) + b
If g1 (X) and g2 (X) are functions of X, then
]
[
]
[
]
[
E g1 (X) + g2 (X) = E g1 (X) + E g2 (X)
(P.11)
(P.12)
This rule extends to any number of functions. Remember the phrase “the expected value of a
sum is the sum of the expected values.”
k
k
Trim Size: 8in x 10in
26
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 26
Probability Primer
P.5.4
Variance of a Random Variable
The variance of a discrete or continuous random variable X is the expected value of
]2
[
g(X) = X − E(X)
The variance of a random variable is important in characterizing the scale of measurement and the
spread of the probability distribution. We give it the symbol σ2 , or σ2X , read “sigma squared.”
The variance σ2 has a Greek symbol because it is a population parameter. Algebraically, letting
E(X) = μ, using the rules of expected values and the fact that E(X) = μ is not random, we have
var(X) = σ2X = E(X − μ)2
)
( )
(
= E X 2 − 2μX + μ2 = E X 2 − 2μE(X) + μ2
( )
= E X 2 − μ2
(P.13)
We use the letters “var” to represent variance, and var(X)
( is) read as “the variance of X,”
where X is a random variable. The calculation var(X) = E X2 − μ2 is usually simpler than
var(X) = E(X − μ)2 , but the solution is the same.
E X A M P L E P.5
Calculating a Variance
For the population in Table P.1, we have shown that E(X) =
μ = 3. Using (P.10), the expectation of the random variable
g(X) = X 2 is
k
Then, the variance of the random variable X is
( )
var(X) = σ2X = E X 2 − μ2 = 10 − 32 = 1
4
4
( ) ∑
∑
x2 𝑓 (x)
E X 2 = g(x) 𝑓 (x) =
x=1
k
x=1
] [
] [
] [
]
[
= 12 × 0.1 + 22 × 0.2 + 32 × 0.3 + 42 × 0.4 = 10
The square root of the variance is called the standard deviation; it is denoted by σ or
sometimes as σX if more than one random variable is being discussed. It also measures the spread
or dispersion of a probability distribution and has the advantage of being in the same units of
measure as the random variable.
A useful property of variances is the following. Let a and b be constants, then
var(aX + b) = a2 var(X)
(P.14)
An additive constant like b changes the mean (expected value) of a random variable, but it does
not affect its dispersion (variance). A multiplicative constant like a affects the mean, and it affects
the variance by the square of the constant.
To see this, let Y = aX + b. Using (P.11)
E(Y) = μY = aE(X) + b = aμX + b
Then
[(
)2 ]
[(
(
]
))2
= E aX + b − aμX + b
var(aX + b) = var(Y) = E Y − μY
[ (
[(
)2 ]
)2 ]
= E a2 X − μX
= E aX − aμX
[(
)2 ]
= a2 var(X)
= a2 E X − μX
The variance of a random variable is the average squared difference between the random variable
X and its mean value μX . The larger the variance of a random variable, the more “spread out” the
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.5 Properties of Probability Distributions
Page 27
27
0.8
0.7
0.6
f (x)
0.5
0.4
Smaller variance
Larger variance
0.3
0.2
0.1
0.0
–1
0
1
2
3
x
4
5
6
7
FIGURE P.3 Distributions with different variances.
values of the random variable are. Figure P.3 shows two pdf s for a continuous random variable,
both with mean μ = 3. The distribution with the smaller variance (the solid curve) is less spread
out about its mean.
P.5.5
Expected Values of Several Random Variables
Let X and Y be random variables. The rule “the expected value of the sum is the sum of the
expected values” applies. Then5
k
E(X + Y) = E(X) + E(Y)
(P.15)
E(aX + bY + c) = aE(X) + bE(Y) + c
(P.16)
Similarly
The product of random variables is not as easy. E(XY) = E(X)E(Y) if X and Y are independent.
These rules can be extended to more random variables.
P.5.6
Covariance Between Two Random Variables
The covariance between X and Y is a measure of linear association between them. Think about
two continuous variables, such as height and weight of children. We expect that there is an association between height and weight, with taller than average children tending to weigh more than
the average. The product of X minus its mean times Y minus its mean is
(
)(
)
X − μX Y − μY
(P.17)
In Figure P.4, we plot values (x and y) of X and Y that have been constructed so that
E(X) = E(Y) = 0.
The x and y values of X and Y fall predominately in quadrants I and III, so that the arithmetic
average of the values (x − μX )(y − μY ) is positive. We define the covariance between two random
variables as the expected (population average) value of the product in (P.17).
[(
)(
)]
cov(X, Y) = σXY = E X − μX Y − μY = E(XY) − μX μY
(P.18)
We use the letters “cov” to represent covariance, and cov(X, Y) is read as “the covariance
between X and Y,” where X and Y are random variables. The covariance σXY of the random
variables underlying Figure P.4 is positive, which tells us that when the values x are greater
............................................................................................................................................
5 These
results are proven in Appendix B.1.4.
k
k
Trim Size: 8in x 10in
Page 28
Probability Primer
II
I
III
IV
0
–4
y
4
28
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
–4
0
x
4
FIGURE P.4 Correlated data.
than μX , then the values y also tend to be greater than μY ; and when the values x are below μX , then
the values y also tend to (be less )(
than μY . )If the random variables values tend primarily to fall in
quadrants II and IV, then x − μX y − μY will tend to be negative and σXY will be negative. If the
random variables values are spread evenly across the four quadrants, and show neither positive
nor negative association, then the covariance is zero. The sign of σXY tells us whether the two
random variables X and Y are positively associated or negatively associated.
Interpreting the actual value of σXY is difficult because X and Y may have different units of
measurement. Scaling the covariance by the standard deviations of the variables eliminates the
units of measurement, and defines the correlation between X and Y
k
σ
cov(X, Y)
= XY
ρ= √
√
var(X) var(Y) σX σY
(P.19)
As with the covariance, the correlation ρ between two random variables measures the degree of
linear association between them. However, unlike the covariance, the correlation must lie between
–1 and 1. Thus, the correlation between X and Y is 1 or –1 if X is a perfect positive or negative
linear function of Y. If there is no linear association between X and Y, then cov(X, Y) = 0 and
ρ = 0. For other values of correlation the magnitude of the absolute value |ρ| indicates the
“strength” of the linear association between the values of the random variables. In Figure P.4,
the correlation between X and Y is ρ = 0.5.
E X A M P L E P.6
Calculating a Correlation
To illustrate the calculation, reconsider the population in
Table P.1 with joint pdf given in Table P.4. The expected
value of XY is
E(XY) =
1 4
∑
∑
xy𝑓 (x, y)
The random variable X has expected value E(X) = μX = 3 and
the random variable Y has expected value E(Y) = μY = 0.4.
Then the covariance between X and Y is
cov(X, Y) = σXY = E(XY) − μX μY = 1 − 3 × (0.4) = −0.2
y=0 x=1
= (1 × 0 × 0) + (2 × 0 × 0.1) + (3 × 0 × 0.2)
+ (4 × 0 × 0.3) + (1 × 1 × 0.1)
+ (2 × 1 × 0.1) + (3 × 1 × 0.1) + (4 × 1 × 0.1)
The correlation between X and Y is
cov(X, Y)
−0.2
= √
= −0.4082
ρ= √
√
√
var(X) var(Y)
1 × 0.24
= 0.1 + 0.2 + 0.3 + 0.4
=1
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.6 Conditioning
Page 29
29
If X and Y are independent random variables, then their covariance and correlation are zero.
The converse of this relationship is not true. Independent random variables X and Y have zero
covariance, indicating that there is no linear association between them. However, just because the
covariance or correlation between two random variables is zero does not mean that they are necessarily independent. There may be more complicated nonlinear associations such as X 2 + Y 2 = 1.
In (P.15) we obtain the expected value of a sum of random variables. There are similar rules
for variances. If a and b are constants, then
var(aX + bY) = a2 var(X) + b2 var(Y) + 2ab cov(X, Y)
(P.20)
A significant point to note is that the variance of a sum is not just the sum of the variances. There
is a covariance term present. Two special cases of (P.20) are
var(X + Y) = var(X) + var(Y) + 2cov(X, Y)
(P.21)
var(X − Y) = var(X) + var(Y) − 2cov(X, Y)
(P.22)
To show that (P.22) is true, let Z = X − Y. Using the rules of expected value
E(Z) = μZ = E(X) − E(Y) = μX − μY
The variance of Z = X − Y is obtained using the basic definition of variance, with some
substitution,
[(
]
[(
(
)2 ]
))2
= E X − Y − μX − μY
var(X − Y) = var(Z) = E Z − μZ
}
{[
(
) (
)]2
X − μX − Y − μY
=E
)2
)(
)}
(
(
+ Y − μY − 2 X − μX Y − μY
[(
[(
[(
)2 ]
)2 ]
)(
)]
+ E Y − μY
− 2E X − μX Y − μY
= E X − μX
k
=E
{(
X − μX
)2
k
= var(X) + var(Y) − 2cov(X, Y)
If X and Y are independent, or if cov(X, Y) = 0, then
var(aX + bY) = a2 var(X) + b2 var(Y)
(P.23)
var(X ± Y) = var(X) + var(Y)
(P.24)
These rules extend to more random variables.
P.6
Conditioning
In Table P.4, we summarized the joint and marginal probability functions for the random variables
X and Y defined on the population in Table P.1. In Table P.6 we make two modifications. First, the
probabilities are expressed as fractions. The many calculations below are simpler using arithmetic
Joint, Marginal, and Conditional Probabilities
T A B L E P.6
y/x
0
1
f (x)
f (x|y = 0)
f (x|y = 1)
1
2
3
4
0
1/10
1/10
0
1/4
1/10
1/10
2/10
1/6
1/4
2/10
1/10
3/10
2/6
1/4
3/10
1/10
4/10
3/6
1/4
f ( y) f ( y|x = 1) f ( y|x = 2) f ( y|x = 3) f ( y|x = 4)
6/10
4/10
0
1
1/2
1/2
k
2/3
1/3
3/4
1/4
Trim Size: 8in x 10in
30
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 30
Probability Primer
with fractions. Second, we added the conditional probability functions (P.4) for Y given each of the
values that X can take and the conditional probability functions for X given each of the values that
Y can take. Now would be a good time for you to review Section P.3.2 on conditional probability.
For example, what is the probability that Y = 0 given that X = 2? That is, if we only consider
population members with X = 2, what is the probability that Y = 0? There are only two population
elements with X = 2, one with Y = 0 and one with Y = 1. The probability of randomly selecting
Y = 0 is one-half. For discrete random variables, the conditional probability is calculated as the
joint probability divided by the probability of the conditioning event.
𝑓 (y = 0|x = 2) = P(Y = 0|X = 2) =
P(X = 2, Y = 0) 1∕10 1
=
=
P(X = 2)
2∕10 2
In the following sections, we discuss the concepts of conditional expectation and conditional
variance.
P.6.1
Conditional Expectation
Many economic questions are formulated in terms of a conditional expectation, or the
conditional mean. One example is “What is the mean wage of a person who has 16 years
of education?” In expected value notation, what is E(WAGE|EDUC = 16)? The effect of
conditioning on the value of EDUC is to reduce the population of interest to only individuals
with 16 years of education. The mean, or expected value, of wage for these individuals may be
quite different than the mean wage for all individuals regardless of years of education, E(WAGE),
which is the unconditional expectation or unconditional mean.
For discrete random variables,6 the calculation of a conditional expected value uses
equation (P.9) with the conditional pdf replacing the usual pdf , so that
∑
∑
E(X|Y = y) = x x𝑓 (x|y) = x xP(X = x|Y = y)
(P.25)
∑
∑
E(Y|X = x) = y y𝑓 (y|x) = y yP(Y = y|X = x)
k
E X A M P L E P.7
Conditional Expectation
Using the population in Table P.1, what is the expected
numerical value of X given that Y = 1? The conditional
probabilities P(X = x|Y = 1) = 𝑓 (x|y = 1) = 𝑓 (x|1) are
given in Table P.6. The conditional expectation of X is
∑4
E(X|Y = 1) = x=1 x𝑓 (x|1)
[
] [
] [
]
= 1 × 𝑓 (1|1) + 2 × 𝑓 (2|1) + 3 × 𝑓 (3|1)
[
]
+ 4 × 𝑓 (4|1)
= 1(1∕4) + 2(1∕4) + 3(1∕4) + 4(1∕4) = 10∕4
= 5∕2
The average value of X in many repeated trials of the experiment of drawing from the shaded slips (Y = 1) is 2.5. This
example makes a good point about expected values in general, namely that the expected value of X does not have to be a
value that X can take. The expected value of X is not the value
that you expect to occur in any single experiment. It is the
average value of X after many repetitions of the experiment.
What is the expected value of X given that we only consider
values where Y = 0? Confirm that E(X|Y = 0) = 10∕3.
For comparison purposes recall from Section P.5.1 that the
unconditional expectation of X is E(X) = 3.
Similarly, if we condition on the X values, the conditional expectations of Y are
∑
E(Y|X = 1) = y y𝑓 (y|1) = 0(0) + 1(1) = 1
∑
E(Y|X = 2) = y y𝑓 (y|2) = 0(1∕2) + 1(1∕2) = 1∕2
∑
E(Y|X = 3) = y y𝑓 (y|3) = 0(2∕3) + 1(1∕3) = 1∕3
∑
E(Y|X = 4) = y y𝑓 (y|4) = 0(3∕4) + 1(1∕4) = 1∕4
Note that E(Y|X) varies as X varies; it is a function of X. For
comparison, the unconditional expectation of Y, E(Y), is
E(Y) =
∑
y y𝑓 (y)
= 0(6∕10) + 1(4∕10) = 2∕5
............................................................................................................................................
6 For
continuous random variables the sums are replaced by integrals. See Appendix B.2.
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.6 Conditioning
P.6.2
Page 31
31
Conditional Variance
The unconditional variance of a discrete random variable X is
[(
)2 ] ∑(
)2
x − μX 𝑓 (x)
=
var(X) = σ2X = E X − μX
(P.26)
x
It measures how much variation there is in X around the unconditional mean of X, μX . For example,
the unconditional variance var(WAGE) measures the variation in WAGE around the unconditional
mean E(WAGE). In (P.13) we show that equivalently
( )
∑
var(X) = σ2X = E X 2 − μ2X = x2 𝑓 (x) − μ2X
(P.27)
x
In Section P.6.1 we discussed how to answer the question “What is the mean wage of a person who
has 16 years of education?” Now we ask “How much variation is there in wages for a person who
has 16 years of education?” The answer to this question is given by the conditional variance,
var (WAGE|EDUC = 16). The conditional variance measures the variation in WAGE around the
conditional mean E(WAGE|EDUC = 16) for individuals with 16 years of education. The conditional variance of WAGE for individuals with 16 years of education is the average squared
difference in the population between WAGE and the conditional mean of WAGE,
}
{[
]2
WAGE − E(WAGE | EDUC = 16) ||
var(WAGE | EDUC = 16) = E
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟ | EDUC = 16
|
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟
conditional mean
|
conditional variance
k
To obtain the conditional variance we modify the definitions of variance in equations (P.26) and
(P.27); replace the unconditional mean E(X) = μX with the conditional mean E(X|Y = y), and the
unconditional pdf f (x) with the conditional pdf f (x|y). Then
} ∑(
{[
)2
]2 |
x − E(X|Y = y) 𝑓 (x|y)
var(X|Y = y) = E X − E(X|Y = y) |Y = y =
(P.28)
|
x
or
]2 ∑
]2
(
) [
[
var(X|Y = y) = E X 2 |Y = y − E(X|Y = y) = x2 𝑓 (x|y) − E(X|Y = y)
(P.29)
x
E X A M P L E P.8
Conditional Variance
For the population in Table P.1, the unconditional variance of
X is var(X) = 1. What is the variance of X given that Y = 1?
To use (P.29) first compute
)
(
E X 2 |Y = 1
∑
= x2 𝑓 (x|Y = 1)
x
= 12 (1∕4) + 22 (1∕4) + 32 (1∕4) + 42 (1∕4) = 15∕2
Then
) ∑
(
E X 2 |Y = 0 = x2 𝑓 (x|Y = 0)
x
= 12 (0) + 22 (1∕6) + 32 (2∕6) + 42 (3∕6)
= 35∕3
Then
) [
]2
(
var(X|Y = 0) = E X 2 |Y = 0 − E(X|Y = 0)
= 35∕3 −(10∕3)2 = 5∕9
(
) [
]2
var(X|Y = 1) = E X 2 |Y = 1 − E(X|Y = 1)
= 15∕2 −(5∕2)2 = 5∕4
In this case, the conditional variance of X, given that Y = 1,
is larger than the unconditional variance of X, var(X) = 1.
To calculate the conditional variance of X given that
Y = 0, we first obtain
k
In this case, the conditional variance of X, given that Y = 0,
is smaller than the unconditional variance of X, var(X) = 1.
These examples have illustrated that in general the conditional variance can be larger or smaller than the unconditional variance. Try working out var(Y|X = 1), var(Y|X = 2),
var(Y|X = 3), and var(Y|X = 4).
k
Trim Size: 8in x 10in
32
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 32
Probability Primer
P.6.3
Iterated Expectations
The Law of Iterated Expectations says that we can find the expected value of Y in two steps.
First, find the conditional expectation E(Y|X). Second, find the expected value E(Y|X) treating X
as random.
] ∑
[
(P.30)
Law of Iterated Expectations∶ E(Y) = EX E(Y|X) = x E(Y|X = x) 𝑓X (x)
In this expression we put an “X” subscript in the expectation EX [E(Y|X)] and the probability
function fX (x) to emphasize that we are treating X as random. The Law of Iterated Expectations
is true for both discrete and continuous random variables.
E X A M P L E P.9
k
Iterated Expectation
[
]
[
] ∑
∑
EX E(Y|X) = EX g(X) = x g(x)𝑓X (x) = x E(Y|X = x) 𝑓X (x)
[
] [
]
= E(Y|X = 1) 𝑓X (1) + E(Y|X = 2) 𝑓X (2)
] [
]
[
+ E(Y|X = 3) 𝑓X (3) + E(Y|X = 4) 𝑓X (4)
Consider the conditional expectation E(X|Y = y) =
∑
x xf (x|y). As we computed in Section P.6.1, E(X|Y = 0)
= 10/3 and E(X|Y = 1) = 5/2. Similarly, the conditional
∑
expectation E(Y|X = x) = y yf (y|x). For the population
in Table P.1, these conditional expectations were calculated
in Section P.6.1 to be E(Y|X = 1) = 1, E(Y|X = 2) = 1/2,
E(Y|X = 3) = 1/3 and E(Y|X = 4) = 1/4. Note that E(Y|X = x)
changes when x changes. If X is allowed to vary randomly7
then the conditional expectation varies randomly. The
conditional expectation is a function of X, or E(Y|X) = g(X),
and is random when viewed this way. Using (P.10) we can
find the expected value of g(X).
= 1(1∕10) + (1∕2)(2∕10) + (1∕3)(3∕10)
+ (1∕4)(4∕10) = 2∕5
If we draw many values x from the population in Table P.1,
the average of E(Y|X) is 2/5. For comparison the “unconditional” expectation of Y is E(Y) = 2/5. EX [E(Y|X)] and E(Y)
are the same.
Proof of the Law of Iterated Expectations To prove the Law of Iterated Expectations we make use of relationships between joint, marginal, and conditional pdf s that we introduced in Section P.3. In Section P.3.1 we discussed marginal distributions. Given a joint pdf
f (x, y) we can obtain the marginal pdf of y alone fY (y) by summing, for each y, the joint pdf
f (x, y) across all values of the variable we wish to eliminate, in this case x. That is, for Y and X,
𝑓 (y) = 𝑓Y (y) =
𝑓 (x) = 𝑓X (x) =
∑
x 𝑓 (x, y)
∑
y 𝑓 (x, y)
(P.31)
Because f ( ) is used to represent pdf s in general, sometimes we will put a subscript, X or Y, to be
very clear about which variable is random.
Using equation (P.4) we can define the conditional pdf of y given X = x as
𝑓 (y|x) =
𝑓 (x, y)
𝑓X (x)
Rearrange this expression to obtain
𝑓 (x, y) = 𝑓 (y|x) 𝑓X (x)
(P.32)
A joint pdf is the product of the conditional pdf and the pdf of the conditioning variable.
............................................................................................................................................
7 Imagine
shuffling the population elements and randomly choosing one. This is an experiment and the resulting number
showing is a value of X. By doing this repeatedly X varies randomly.
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.6 Conditioning
Page 33
33
To show that the Law of Iterated Expectations is true8 we begin with the definition of the
expected value of Y, and operate with the summation.
[
]
]
[
∑
∑ ∑
substitute for 𝑓 (y)
E(Y) = y𝑓 (y) = y
𝑓 (x, y)
y
=
∑
[
y
y
∑
y
=
x
[
∑ ∑
x
∑
y
x
]
𝑓 (y|x) 𝑓X (x)
]
y𝑓 (y|x) 𝑓X (x)
E(Y|x)𝑓X (x)
x
]
[
= EX E(Y|X)
=
[
substitute for 𝑓 (x, y)
]
[
]
change order of summation
[
recognize the conditional expectation
]
While this result may seem an esoteric oddity it is very important and widely used in modern
econometrics.
P.6.4
Variance Decomposition
Just as we can break up the expected value using the Law of Iterated Expectations we can decompose the variance of a random variable into two parts.
]
]
[
[
(P.33)
Variance Decomposition∶ var(Y) = varX E(Y|X) + EX var(Y|X)
k
This “beautiful” result9 says that the variance of the random variable Y equals the sum of the
variance of the conditional mean of Y given X and the mean of the conditional variance of Y
given X. In this section we will discuss this result.10
Suppose that we are interested in the wages of the population consisting of working adults.
How much variation do wages display in the population? If WAGE is the wage of a randomly
drawn population member, then we are asking about the variance of WAGE, that is, var(WAGE).
The variance decomposition says
]
]
[
[
var(WAGE) = varEDUC E(WAGE|EDUC) + EEDUC var(WAGE|EDUC)
E(WAGE|EDUC) is the expected value of WAGE given a specific value of education, such as
EDUC = 12 or EDUC = 16. E(WAGE|EDUC = 12) is the average WAGE in the population,
given that we only consider workers who have 12 years of education. If EDUC changes then the
conditional mean E(WAGE|EDUC) changes, so that E(WAGE|EDUC = 16) is not the same as
E(WAGE|EDUC = 12), and in fact we expect E(WAGE|EDUC = 16) > E(WAGE|EDUC = 12);
more education means more “human capital” and thus
[ the average wage] should be higher. The first
component in the variance decomposition varEDUC E(WAGE|EDUC) measures the variation in
E(WAGE|EDUC) due to variation in education.
[
]
The second part of the variance decomposition is EEDUC var(WAGE|EDUC) . If we
restrict our attention to population members who have 12 years of education, the mean wage
is E(WAGE|EDUC = 12). Within the group of workers who have 12 years of education we
will observe wide ranges of wages. For example, using one sample of CPS data from 2013,11
wages for those with 12 years of education varied from $3.11/hour to $100.00/hour; for
those with 16 years of education wages varied from $2.75/hour to $221.10/hour. For workers
with 12 and 16 years of education that variation is measured by var(WAGE|EDUC = 12) and
............................................................................................................................................
8 The
proof for continuous variables is in Appendix B.2.4.
9 Tony
O’Hagan, “A Thing of Beauty,” Significance Magazine, Volume 9 Issue 3 (June 2012), 26–28.
10 The
proof of the variance decomposition is given in Appendix B.1.8 and Example B.1.
11 The
data file cps5.
k
k
Trim Size: 8in x 10in
34
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 34
Probability Primer
[
]
var(WAGE|EDUC = 16). The term EEDUC var(WAGE|EDUC) measures the average of
var(WAGE|EDUC) as education changes.
To summarize, the variation of WAGE in the population can be attributed to two sources:
variation in the conditional mean E(WAGE|EDUC) and variation due to changes in education in
the conditional variance of WAGE given education.
P.6.5
Covariance Decomposition
Recall
that the covariance
between two random variables Y and X is cov(X, Y) =
[(
)(
)]
E X − μX Y − μY . For discrete random variables this is
cov(X, Y) =
∑ ∑(
x
y
x − μX
)(
)
y − μY 𝑓 (x, y)
By using the relationships between marginal, conditional and joint pdf s we can show
)
∑(
x − μX E(Y|X = x) 𝑓 (x)
cov(X, Y) =
(P.34)
x
Recall that E(Y|X) = g(X) so this result says that the covariance between X and Y can be calculated
as the expected value of X, minus its mean, times a function of X, cov(X, Y) = EX [(X − μX )E(Y|X)].
An important special case is important in later chapters. When the conditional expectation
of Y given X is a constant, E(Y|X = x) = c, then
)
)
∑(
∑(
x − μX E(Y|X = x) 𝑓 (x) = c x − μX 𝑓 (x) = 0
cov(X, Y) =
x
k
x
A special case is E(Y|X = x) = 0, which by direct substitution implies cov(X, Y) = 0.
E X A M P L E P.10
Covariance Decomposition
To illustrate we compute cov(X, Y) for the population in
Table P.1 using the covariance decomposition. We have
computed that cov(X, Y) = −0.2 in Section P.5.6. The ingredients are the values of the random variable X, its mean
μX = 3, the probabilities P(X = x) = f (x) and conditional
expectations
Using the covariance decomposition we have
)
∑(
x − μX E(Y|X = x) 𝑓 (x)
cov(X, Y) =
x
= (1 − 3)(1)(1∕10) + (2 − 3)(1∕2)(2∕10)
+ (3 − 3)(1∕3)(3∕10) + (4 − 3)(1∕4)(4∕10)
= −2∕10 − 1∕10 + 1∕10 = −2∕10 = −0.2
E(Y|X = 1) = 1, E(Y|X = 2) = 1∕2,
E(Y|X = 3) = 1∕3 and E(Y|X = 4) = 1∕4
P.7
We see that the covariance decomposition yields the correct
result, and it is convenient in this example.
The Normal Distribution
In the previous sections we discussed random variables and their pdf s in a general way. In real
economic contexts, some specific pdf s have been found to be very useful. The most important is
the normal distribution. If X(is a normally
distributed random variable with mean μ and variance
)
σ2 , it is symbolized as X ∼ N μ, σ2 . The pdf of X is given by the impressive formula
[
]
−(x − μ)2
1
𝑓 (x) = √
exp
, −∞ < x < ∞
(P.35)
2σ2
2πσ2
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.7 The Normal Distribution
Page 35
35
0.4
f (x)
0.3
N(0, 1)
N(2, 1)
0.2
N(0, 4)
0.1
0
–4
–2
0
2
4
6
x
(
)
FIGURE P.5 Normal probability density functions N 𝛍, 𝛔2 .
k
where exp(a) denotes the exponential12 function ea . The mean μ and variance σ2 are the parameters of this distribution and determine its center and dispersion. The range of the continuous
normal random variable is from minus infinity to plus infinity. Pictures of the normal pdf s are
given in Figure P.5 for several values of the mean and variance. Note that the distribution is
symmetric and centered at μ.
Like all continuous random variables, probabilities involving normal random variables are
found as areas under the pdf . For calculating probabilities both computer software and statistical
tables values make use of the relation between a normal random variable and its “standardized”
equivalent. A standard
random variable is one that has a normal pdf with mean 0 and
)
( normal
variance 1. If X ∼ N μ, σ2 , then
Z=
X−μ
∼ N(0, 1)
σ
(P.36)
The standard normal random variable Z is so widely used that its pdf and cdf are given
their own special notation. The cdf is denoted Φ(z) = P(Z ≤ z). Computer programs, and
Statistical Table 1 in Appendix D give values of Φ(z). The pdf for the standard normal random
variable is
(
)
1
ϕ(z) = √ exp −z2 ∕2 , −∞ < z < ∞
2π
Values of the density function are given in Statistical Table 6 in Appendix D. To calculate normal
probabilities, remember that the distribution is symmetric, so that P(Z > a) = P(Z < −a), and
P(Z > a) = P(Z ≥ a), since the probability of any one point is zero for a continuous random
variable. If X ∼ N(μ, σ2 ) and a and b are constants, then
(
)
(a − μ)
(
X−μ a−μ
a − μ)
=Φ
(P.37)
P(X ≤ a) = P
≤
=P Z≤
σ
σ
σ
σ
(
)
(a − μ)
(
X−μ a−μ
a − μ)
=1−Φ
(P.38)
P(X > a) = P
>
=P Z>
σ
σ
σ
σ
(
)
(
)
(a − μ)
a−μ
b−μ
b−μ
(P.39)
P(a ≤ X ≤ b) = P
≤Z≤
=Φ
−Φ
σ
σ
σ
σ
............................................................................................................................................
12 See
Appendix A.1.2 for a review of exponents.
k
k
Trim Size: 8in x 10in
36
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 36
Probability Primer
E X A M P L E P.11
Normal Distribution Probability Calculation
For example, if X ∼ N(3, 9), then
P(4 ≤ X ≤ 6) = P(0.33 ≤ Z ≤ 1) = Φ(1) − Φ(0.33) = 0.8413 − 0.6293 = 0.2120
k
In addition to finding normal probabilities
) we sometimes must find a value zα of a standard nor(
zα is called the 100𝛂-percentile. For
mal random variable such that P Z ≤ zα (= α. The value
)
example, z0.975 is the value of Z such that P Z ≤ z0.975 = 0.975. This particular percentile can be
found using Statistical Table 1, Cumulative Probabilities for the Standard Normal Distribution.
The cumulative probability associated with the value z =1.96 is P(Z ≤ 1.96) = 0.975, so that the
97.5 percentile is z0.975 = 1.96. Using Statistical Table 1 we can only roughly obtain other percentiles. Using the cumulative probabilities P(Z ≤ 1.64) = 0.9495 and P(Z ≤ 1.65) = 0.9505 we
can say that the 95th percentile of the standard normal distribution is between 1.64 and 1.65,
and is about 1.645.
Luckily computer software makes these approximations
)unnecessary.
( ) The inverse normal
(
function finds percentiles zα given α. Formally, if P Z ≤ zα = Φ zα = α then zα = Φ−1 (α).
Econometric software, even spreadsheets, have the inverse normal function built in. Some
commonly used percentiles are shown in Table P.7. In the last column are the percentiles
rounded to fewer decimals. It would be useful for you to remember the numbers 2.58, 1.96,
and 1.645.
An interesting and useful fact about the normal distribution( is that) a weighted sum
)
( of normal
random variables has a normal distribution. That is, if X1 ∼ N μ1 , σ21 and X2 ∼ N μ2 , σ22 then
)
(
(P.40)
Y = a1 X1 + a2 X2 ∼ N μY = a1 μ1 + a2 μ2 , σ2Y = a21 σ21 + a22 σ22 + 2a1 a2 σ12
)
(
where σ12 = cov X1 , X2 . A number of important probability distributions are related to the normal distribution. The t-distribution, the chi-square distribution, and the F-distribution are discussed in Appendix B.
T A B L E P.7
Standard Normal Percentiles
𝛂
z𝛂 = 𝚽−1 (𝛂)
Rounded
2.57583
2.32635
1.95996
1.64485
1.28155
−1.28155
−1.64485
−1.95996
−2.32635
−2.57583
2.58
2.33
1.96
1.645
1.28
−1.28
−1.645
−1.96
−2.33
−2.58
0.995
0.990
0.975
0.950
0.900
0.100
0.050
0.025
0.010
0.005
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.7 The Normal Distribution
Page 37
37
The Bivariate Normal Distribution
P.7.1
Two continuous random variables, X and Y, have a joint normal, or bivariate normal, distribution if their joint pdf takes the form
{ [(
(
)
)(
)
x − μX 2
x − μX
y − μY
1
exp −
− 2ρ
𝑓 (x, y) =
√
σX
σX
σY
2πσX σY 1 − ρ2
}
(
) ]/
)
(
y − μY 2
2
+
2 1−ρ
σY
where −∞ < x < ∞, −∞ < y < ∞. The parameters μX and μY are the means of X and Y, σ2X and
σ2Y are the variances of X and Y, so that σX and σY are the standard deviations. The parameter ρ
is the correlation between X and Y. If cov(X, Y) = σXY then
σ
cov(X, Y)
= XY
ρ= √
√
var(X) var(Y) σX σY
4
2
–4
y
k
The complex equation for f (x, y) defines a surface in three-dimensional space. In Figure P.6a13
we depict the surface if μX = μY = 0, σX = σY = 1, and ρ = 0.7. The positive correlation means
there is a positive linear association between the values of X and Y, as described in Figure P.4.
Figure P.6b depicts the contours of the density, the result of slicing the density horizontally, at a
given height. The contours are more “cigar-shaped” the larger the absolute value of the correlation ρ. In Figure P.7a the correlation is ρ = 0. In this case the joint density is symmetrical and
the contours in Figure P.7b are circles. If X and Y are jointly normal then they are statistically
independent if, and only if, ρ = 0.
–2
0
x 0
–2
2
4
–4
–2
0
y
2
4
–4
–4
–2
(a)
0
x
2
4
(b)
FIGURE P.6 The bivariate normal distribution: 𝛍X = 𝛍Y = 0, 𝛔X = 𝛔Y = 1, and 𝛒 = 0.7.
............................................................................................................................................
13 “The
Bivariate Normal Distribution” from the Wolfram Demonstrations Project http://demonstrations.wolfram.com/.
Figures P.6, P.7, and P.8 represent the interactive graphics on the site as static graphics for the primer. The site permits
easy manipulation of distribution parameters. The joint density function figure can be rotated and viewed from different
angles.
k
k
Trim Size: 8in x 10in
38
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 38
Probability Primer
4
y
2
0
–4
–2
4
x
0
–2
2
0
2
–2
4 –4
y
–4
–4
–2
2
0
x
(a)
4
(b)
FIGURE P.7 The bivariate normal distribution: 𝛍X = 𝛍Y = 0, 𝛔X = 𝛔Y = 1, and 𝛒 = 0.
k
There are several relations between the normal, bivariate normal, and the conditional distributions that are used in statistics and econometrics. First, if X and Y have a bivariate normal
dis-)
(
2
,
σ
tribution then
the
marginal
distributions
of
X
and
Y
are
normal
distributions
too,
X
∼
N
μ
X
X
)
(
2
and Y ∼ N μY , σY .
Second, the conditional distribution for Y given X is normal, with conditional mean E(Y|X)
(
)=
2
2
2
1
−
ρ
.
−
βμ
and
β
=
σ
∕σ
,
and
conditional
variance
var(Y|X)
=
σ
α + βX, where
α
=
μ
Y ( X
XY
X
Y
[
)]
Or Y|X ∼ N α + βX, σ2Y 1 − ρ2 . Three noteworthy points about these results are (i) that the
conditional mean is a linear function of X, and is called a linear regression function; (ii) the conditional variance is constant and does not vary with X; and (iii) the conditional variance is smaller
than the unconditional variance if ρ ≠ 0. In Figure P.814 we display a joint normal density with
Conditional distribution of Y when X = x
Bivariate normal density
20
y
15
0
5
x
10
10
5
15
0
x
20
10
20
15
5
0
(a)
10
y
(b)
FIGURE P.8 (a) Bivariate normal distribution with 𝛍X = 𝛍Y = 5, 𝛔X = 𝛔Y = 3,
and 𝛒 = 0.7; (b) conditional distribution of Y given X = 10.
............................................................................................................................................
14 “The
Bivariate Normal and Conditional Distributions” from the Wolfram Demonstrations Project
http://demonstrations.wolfram.com/TheBivariateNormalAndConditionalDistributions/. Both the bivariate distribution
and conditional distributions can be rotated and viewed from different perspectives.
k
k
Trim Size: 8in x 10in
k
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
P.8 Exercises
Page 39
39
μX = μY = 5, σX = σY = 3, and ρ = 0.7. The covariance between X and Y is σXY = ρσX σY = 0.7
× 3 × 3 = 6.3 so that β = σXY ∕σ2X = 6.3∕9 = 0.7 and α = μY – βμX = 5 – 0.7 × 5 = 1.5. The
conditional mean of Y given X = 10 is E(Y|X = 10) =
( α + βX
) = 1.5
( + 0.7X )= 1.5 + 0.7 × 10 =
8.5. The conditional variance is var(Y|X = 10) = σ2Y 1 − ρ2 = 32 1 − 0.72 = 9(0.51) = 4.59.
That is, the conditional distribution is (Y|X = 10) ∼ N(8.5, 4.59).
P.8 Exercises
Answers to odd-numbered exercises are on the book website www.principlesofeconometrics.com/poe5.
P.1 Let x1 = 17, x2 = 1, x3 = 0; y1 = 5, y2 = 2, y3 = 8. Calculate the following:
∑2
a.
xi
∑i=1
3
b.
xy
t=1( t t
)
∑3
∕3 [Note: x is called the arithmetic average or arithmetic mean.]
c. x =
x
i
i=1
)
∑3 (
d.
x −x
i=1 i
)2
∑3 (
e.
x −x
i=1 i )
(∑
2
3
f.
x2 − 3x
i=1 i
)
(∑
)(
)
∑3 (
3
g.
y ∕3
x − x yi − y where y =
i=1 i
i=1 i
(∑
)
3
h.
− 3x y
x
y
j
j
j=1
k
P.2 Express each of the following sums in summation notation.
) (
) (
) (
)
(
a. x1 ∕y1 + x2 ∕y2 + x3 ∕y3 + x4 ∕y4
b. y2 + y3 + y4
c. x1 y1 + x2 y2 + x3 y3 + x4 y4
d. x3 y5 + x4 y6 + x5 y7
(
) (
)
e. x3 ∕y23 + x4 ∕y24
) (
) (
) (
)
(
f. x1 − y1 + x2 − y2 + x3 − y3 + x4 − y4
P.3 Write out each of the following sums and compute where possible.
)
∑3 (
a − bxi
a.
i=1
∑4 2
t
b.
)
∑t=1
2 ( 2
c.
2x + 3x + 1
x=0
∑4
𝑓 (x + 3)
d.
∑x=2
3
𝑓 (x, y)
e.
∑x=1
4 ∑2
f.
(x + 2y)
x=3
y=1
P.4 Show algebraically that
)2 (∑n 2 )
∑n (
2
a.
x − nx
x −x =
i=1 ( i
)(
)i=1 (i ∑n
)
∑n
b.
x y − nx y
xi − x yi − y =
i=1 i i
)
∑i=1
n (
c.
x −x =0
j=1 j
P.5 Let SALES denote the monthly sales at a bookstore. Assume SALES are normally distributed with a
mean of $50,000 and a standard deviation of $6000.
a. Compute the probability that the firm has a month with SALES greater than $60,000. Show a sketch.
b. Compute the probability that the firm has a month with SALES between $40,000 and $55,000.
Show a sketch.
c. Find the value of SALES that represents the 97th percentile of the distribution. That is, find the
value SALES0.97 such that P(SALES > SALES0.97 ) = 0.03.
d. The bookstore knows their PROFITS are 30% of SALES minus fixed costs of $12,000. Find the
probability of having a month in which PROFITS were zero or negative. Show a sketch. [Hint:
What is the distribution of PROFITS?]
k
k
Trim Size: 8in x 10in
40
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 40
Probability Primer
P.6 A venture capital company feels that the rate of return (X) on a proposed investment is approximately
normally distributed with a mean of 40% and a standard deviation of 10%.
a. Find the probability that the return X will exceed 55%.
b. The banking firm who will fund the venture sees the rate of return differently, claiming that venture
capitalists are always too optimistic. They perceive that the distribution of returns is V = 0.8X −
5%, where X is the rate of return expected by the venture capital company. If this is correct, find
the probability that the return V will exceed 55%.
P.7 At supermarkets sales of “Chicken of the Sea” canned tuna vary from week to week. Marketing
researchers have determined that there is a relationship between sales of canned tuna and the price
of canned tuna. Specifically, SALES = 50000 − 100 PRICE. SALES is measured as the number of
cans per week and PRICE is measured in cents per can. Suppose PRICE over the year can be considered (approximately) a normal random variable with mean μ = 248 cents and standard deviation
σ = 10 cents.
a. Find the expected value of SALES.
b. Find the variance of SALES.
c. Find the probability that more than 24,000 cans are sold in a week. Draw a sketch illustrating the
calculation.
d. Find the PRICE such that SALES is at its 95th percentile value. That is, let SALES0.95 be the 95th
percentile of SALES. Find the value PRICE0.95 such that P (SALES > SALES0.95 ) = 0.05.
P.8 The Shoulder and Knee Clinic knows that their expected monthly revenue from patients depends on
their level of advertising. They hire an econometric consultant who reports that their expected monthly
revenue, measured in $1000 units, is given by the following equation E(REVENUE|ADVERT) = 100
+ 20 ADVERT, where ADVERT is advertising expenditure in $1000 units. The econometric consultant
also claims that REVENUE is normally distributed with variance var(REVENUE|ADVERT) = 900.
k
a. Draw a sketch of the relationship between expected REVENUE and ADVERT as ADVERT varies
from 0 to 5.
b. Compute the probability that REVENUE is greater than 110 if ADVERT = 2. Draw a sketch to
illustrate your calculation.
c. Compute the probability that REVENUE is greater than 110 if ADVERT = 3.
d. Find the 2.5 and 97.5 percentiles of the distribution of REVENUE when ADVERT = 2. What is the
probability that REVENUE will fall in this range if ADVERT = 2?
e. Compute the level of ADVERT required to ensure that the probability of REVENUE being larger
than 110 is 0.95.
P.9 Consider the U.S. population of registered voters, who may be Democrats, Republicans or independents. When surveyed about the war with ISIS, they were asked if they strongly supported war efforts,
strongly opposed the war, or were neutral. Suppose that the proportion of voters in each category is
given in Table P.8:
T A B L E P.8
Table for Exercise P.9
Against
Political Party
Republican
Independent
Democrat
0.05
0.05
0.35
War Attitude
Neutral In Favor
0.15
0.05
0.05
0.25
0.05
0
a. Find the “marginal” probability distributions for war attitudes and political party affiliation.
b. What is the probability that a randomly selected person is a political independent given that they
are in favor of the war?
c. Are the attitudes about war with ISIS and political party affiliation statistically independent or not?
Why?
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.8 Exercises
Page 41
41
d. For the attitudes about the war assign the numerical values AGAINST = 1, NEUTRAL = 2, and IN
FAVOR = 3. Call this variable WAR. Find the expected value and variance of WAR.
e. The Republican party has determined that monthly fundraising depends on the value of WAR from
month to month. In particular the monthly contributions to the party are given by the relation (in
millions of dollars) CONTRIBUTIONS = 10 + 2 × WAR. Find the mean and standard deviation of
CONTRIBUTIONS using the rules of expectations and variance.
P.10 A firm wants to bid on a contract worth $80,000. If it spends $5000 on the proposal it has a 50–50
chance of getting the contract. If it spends $10,000 on the proposal it has a 60% chance of winning the
contract. Let X denote the net revenue from the contract when the $5000 proposal is used and let Y
denote the net revenue from the contract when the $10,000 proposal is used.
X
−5,000
75,000
a.
b.
c.
d.
f (x)
0.5
0.5
y
−10,000
70,000
f (y)
0.4
0.6
If the firm bases its choice solely on expected value, how much should it spend on the proposal?
Compute the variance of X. [Hint: Using scientific notation simplifies calculations.]
Compute the variance of Y.
How might the variance of the net revenue affect which proposal the firm chooses?
P.11 Prior to presidential elections citizens of voting age are surveyed. In the population, two characteristics
of voters are their registered party affiliation (republican, democrat, or independent) and for whom they
voted in the previous presidential election (republican or democrat). Let us draw a citizen at random,
defining these two variables.
k
⎧−1
⎪
PARTY = ⎨ 0
⎪ 1
⎩
{
−1
VOTE =
1
registered republican
independent or unregistered
registered democrat
voted republican in previous election
voted democratic in previous election
a. Suppose that the probability of drawing a person who voted republication in the last election is
0.466, and the probability of drawing a person who is registered republican is 0.32, and the probability that a randomly selected person votes republican given that they are a registered republican
is 0.97. Compute the joint probability Prob[PARTY = −1, VOTE = −1]. Show your work.
b. Are these random variables statistically independent? Explain.
P.12 Based on years of experience, an economics professor knows that on the first principles of economics
exam of the semester 13% of students will receive an A, 22% will receive a B, 35% will receive a C,
20% will receive a D, and the remainder will earn an F. Assume a 4 point grading scale (A = 4, B = 3,
C = 2, D = 1, and F = 0). Define the random variable GRADE = 4, 3, 2, 1, 0 to be the grade of a
randomly chosen student.
a. What is the probability distribution f (GRADE) for this random variable?
b. What is the expected value of GRADE? What is the variance of GRADE? Show your work.
c. The professor has 300 students in each class. Suppose that the grade of the ith student is GRADEi
and that the probability distribution of grades f (GRADEi ) is the same for all students. Define
∑300
CLASS_ AVG = i=1 GRADEi ∕300. Find the expected value and variance of CLASS_AVG.
d. The professor has estimated that the number of economics majors coming from the class is related
to the grade on the first exam. He believes the relationship to be MAJORS = 50 + 10CLASS_AVG.
Find the expected value and variance of MAJORS. Show your work.
P.13 The LSU Tigers baseball team will play the Alabama baseball team in a weekend series of two games.
Let W = 0, 1, or 2 equal the number of games LSU wins. Let the weekend’s weather be designated
as Cold or Not Cold. Let C = 1 if the weather is cold and C = 0 if the weather is not cold. The joint
probability function of these two random variables is given in Table P.9, along with space for the
marginal distributions.
k
k
Trim Size: 8in x 10in
42
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
Page 42
Probability Primer
T A B L E P.9
C=1
C=0
f (w)
Table for Exercise P.13
W=0
W=1
W=2
f (c)
(i)
0.07
(v)
0.12
0.14
(vi)
0.12
(iii)
0.61
(ii)
(iv)
a. Fill in the blanks, (i)–(vi).
b. Using the results of (a), find the conditional probability distribution of the number of wins, W,
conditional on the weather being warm, C = 0. Based on a comparison of the conditional probability
distribution f (w|C = 0) and the marginal distribution f (w), can you conclude that the number of
games LSU wins W is statistically independent of the weather conditions C, or not? Explain.
c. Find the expected value of the number of LSU wins, W. Also find the conditional expectation
E(W|C = 0). Show your work. What kind of weather is more favorable for the LSU Tigers baseball
team?
d. The revenue of vendors at the LSU Alex Box baseball stadium depends on the crowds, which in
turn depends on the weather. Suppose that food sales FOOD = $10,000 − 3000C. Use the rules
for expected value and variance to find the expected value and standard deviation of food sales.
k
P.14 A clinic specializes in shoulder injuries. A patient is randomly selected from the population of all clinic
clients. Let S be the number of doctor visits for shoulder problems in the past six months. Assume the
values of S are s = 1, 2, 3, or 4. Patients at the shoulder clinic are also asked about knee injuries. Let
K = the number of doctor visits for knee injuries during the past six months. Assume the values of
K are k = 0, 1 or 2. The joint probability distribution of the numbers of shoulder and knee injuries is
shown in Table P.10. Use the information in the joint probability distribution to answer the following
questions. Show brief calculations for each
T A B L E P.10
Table for Exercise P.14
Knee = K
Shoulder = S
1
2
3
4
f (k)
0
1
2
0.15
0.06
0.02
0.02
0.09
0.06
0.10
0.08
f (s)
0.2
0.10
0.33
a. What is the probability that a randomly chosen patient will have two doctor visits for shoulder
problems during the past six months?
b. What is the probability that a randomly chosen patient will have two doctor visits for shoulder
problems during the past six months given that they have had one doctor visit for a knee injury in
the past six months?
c. What is the probability that a randomly chosen patient will have had three doctor visits for shoulder
problems and two doctor visits for knee problems in the past six months?
d. Are the number of doctor visits for knee and shoulder injuries statistically independent? Explain.
e. What is the expected value of the number of doctor visits for shoulder injuries from this population?
f. What is the variance of the number of doctor visits for shoulder injuries from this population?
P.15 As you walk into your econometrics exam, a friend bets you $20 that she will outscore you on the
exam. Let X be a random variable denoting your winnings. X can take the values 20, 0 [if there is a
tie], or −20. You know that the probability distribution for X, f (x), depends on whether she studied for
k
k
Trim Size: 8in x 10in
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
k
P.8 Exercises
Page 43
43
the exam or not. Let Y = 0 if she studied and Y = 1 if she did not study. Consider the following joint
distribution Table P.11.
T A B L E P.11
Joint pdf for Exercise P.15
Y
X
a.
b.
c.
d.
e.
0
1
f (x)
−20
0
20
(i)
(iii)
0.10
0
0.15
(iv)
(ii)
0.25
(v)
f (y)
(vi)
0.60
Fill in the missing elements (i)–(vi) in the table.
Compute E(X). Should you take the bet?
What is the probability distribution of your winnings if you know that she did not study?
Find your expected winnings given that she did not study.
Use the Law of Iterated Expectations to find E(X).
P.16 Breast cancer prevalence in the United Kingdom can be summarized for the population (data are in
1000s) as in Table P.12.
T A B L E P.12
Table for Exercise P.16
k
k
Sex
Suffers from Breast Cancer
Not Suffering from Breast Cancer
Total
Female
Male
Total
550
30,868
31,418
3
30,371
30,374
553
61,239
61,792
Compute the probability that a randomly drawn person has breast cancer.
Compute the probability that a randomly drawn female has breast cancer.
Compute the probability that a person is female given that the person has breast cancer.
What is the conditional probability function for the prevalence of breast cancer given that the person
is female?
e. What is the conditional probability function for the prevalence of breast cancer given that the person
is male?
a.
b.
c.
d.
P.17 A continuous random variable Y has pdf
𝑓 (y) =
a.
b.
c.
d.
e.
f.
{
2y 0 < y < 1
0 otherwise
Sketch the pdf .
Find the cdf , F(y) = P(Y ≤ y) and sketch it. [Hint: Requires calculus.]
Use the pdf and a geometric argument to find the probability P(Y ≤ 1/2).
Use the cdf from part (b) to compute P(Y ≤ 1/2).
Using the pdf and a geometric argument find the probability P(1/4 ≤ Y ≤ 3/4).
Use the cdf from part (b) to compute P(1/4 ≤ Y ≤ 3/4).
P.18 Answer each of the following:
a. An internal revenue service auditor knows that 3% of all income tax forms contain errors. Returns
are assigned randomly to auditors for review. What is the probability that an auditor will have to
k
Trim Size: 8in x 10in
44
k
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
Page 44
Probability Primer
b.
c.
d.
e.
f.
view four tax returns until the first error is observed? That is, what is the probability of observing
three returns with no errors, and then observing an error in the fourth return?
Let Y be the number of independent trials of an experiment before a success is observed. That is,
it is the number of failures before the first success. Assume each trial has a probability of success
of p and a probability of failure of 1 − p. Is this a discrete or continuous random variable? What is
the set of possible values that Y can take? Can Y take the value zero? Can Y take the value 500?
Consider the pdf f (y) = P(Y = y) = p(1 − p)y . Using this pdf compute the probability in (a). Argue
that this probability function generally holds for the experiment described in (b).
Using the value p = 0.5, plot the pdf in (c) for y = 0, 1, 2, 3, 4.
∑∞
∑∞
Show that y=0 𝑓 (y) = y=0 p(1 − p)y = 1. [Hint: If |r| < 1 then 1 + r + r2 + r3 + · · · = 1/(1−r).]
Verify for y = 0, 1, 2, 3, 4 that the cdf P(Y ≤ y) = 1 − (1 − p)y+1 yields the correct values.
P.19 Let X and Y be random variables with expected values μ = μX = μY and variances σ2 = σ2X = σ2Y . Let
Z = (2X + Y)/2.
a.
b.
c.
d.
Find the expected value of Z.
Find the variance of Z assuming X and Y are statistically independent.
Find the variance of Z assuming that the correlation between X and Y is −0.5.
Let the correlation between X and Y be −0.5. Find the correlation between aX and bY, where a and
b are any nonzero constants.
P.20 Suppose the pdf of the continuous random variable X is f (x) = 1, for 0 < x < 1 and f (x) = 0 otherwise.
k
a. Draw a sketch of the pdf . Verify that the area under the pdf for 0 < x < 1 is 1.
b. Find the cdf of X. [Hint: Requires the use of calculus.]
c. Compute the probability that X falls in each of the intervals [0, 0.1], [0.5, 0.6], and [0.79, 0.89].
Indicate the probabilities on the sketch drawn in (a).
d. Find the expected value of X.
e. Show that the variance of X is 1/12.
f. Let Y be a discrete random variable taking the values 1 and 0 with conditional probabilities
P(Y = 1|X = x) = x and P(Y = 0|X = x) = 1 − x. Use the Law of Iterated Expectations to find
E(Y).
g. Use the variance decomposition to find var(Y).
P.21 A fair die is rolled. Let Y be the face value showing, 1, 2, 3, 4, 5, or 6 with each having the probability
1/6 of occurring. Let X be another random variable that is given by
{
Y if Y is even
X=
0 if Y is odd
a.
b.
c.
d.
e.
f.
( )
Find E(Y), E Y2 , and var(Y).
( )
What is the probability distribution for X? Find E(X), E X2 , and var(X).
Find the conditional probability distribution of Y given each X.
Find the conditional expected value of Y given each value of X, E(Y|X). ( )
Find the probability distribution of Z = XY. Show that E(Z) = E(XY) = E X2 .
Find cov(X, Y).
P.22 A large survey of married women asked “How many extramarital affairs did you have last year?” 77%
said they had none, 5% said they had one, 2% said two, 3% said three, and the rest said more than three.
Assume these women are representative of the entire population.
a. What is the probability that a randomly selected married woman will have had one affair in the past
year?
b. What is the probability that a randomly selected married woman will have had more than one affair
in the past year?
c. What is the probability that a randomly chosen married woman will have had less than three affairs
in the past year?
d. What is the probability that a randomly chosen married woman will have had one or two affairs in
the past year?
e. What is the probability that a randomly chosen married woman will have had one or two affairs in
the past year, given that they had at least one?
k
k
Trim Size: 8in x 10in
k
Hill5e c01a.tex V1 - 12/21/2017 2:43pm
P.8 Exercises
Page 45
45
P.23 Let NKIDS represent the number of children ever born to a woman.
/ The possible values of NKIDS
are nkids = 0, 1, 2, 3, 4, . . . . Suppose the pdf is f (nkids) = 2nkids (7.389nkids!), where ! denotes the
factorial operation.
a. Is NKIDS a discrete or continuous random variable?
b. Calculate the pdf for nkids = 0, 1, 2, 3, 4. Sketch it. [Note: It may be convenient to use a spreadsheet
or other software to carry out tedious calculations.]
c. Calculate the probabilities P[NKIDS ≤ nkids] for nkids = 0, 1, 2, 3, 4. Sketch the cumulative distribuiton function.
d. What is the probability that a woman will have more than one child.
e. What is the probability that a woman will have two or fewer children?
P.24 Five baseballs are thrown to a batter who attempts to hit the ball 350 feet or more. Let H denote the
number of successes, with the pd for having h successes being f (h) = 120 × 0.4h × 0.65−h ∕[h!(5−h)!],
where ! denotes the factorial operation.
a. Is H a discrete or continuous random variable? What values can it take?
b. Calculate the probabilities that the number of successes h = 0, 1, 2, 3, 4, and 5. [Note: It may be
convenient to use a spreadsheet or other software to carry out tedious calculations.] Sketch the pdf .
c. What is the probability of two or fewer successes?
d. Find the expected value of the random variable H. Show your work.
e. The prizes are $1000 for the first success, $2000 for the second success, $3000 for the third success,
and so on. What is the pdf for the random variable PRIZE, which is the total prize winnings?
f. Find the expected value of total prize winnings, PRIZE.
P.25 An author knows that a certain number of typographical errors (0, 1, 2, 3, …) are on each book page.
Define the random variable T equaling the number of errors per page. Suppose that T has a Poisson
distribution [Appendix B.3.3], with pdf , f (t) = μt exp(−μ)/t!, where ! denotes the factorial operation,
and μ = E(T) is the mean number of typographical errors per page.
k
a. If μ = 3, what is the probability that a page has one error? What is the probability that a page has
four errors?
b. An editor independently checks each word of every page and catches 90% of the errors, but misses
10%. Let Y denote the number of errors caught on a page. The values of y must be less than or
equal to the actual number t of errors on the page. Suppose that the number of errors caught on a
page with t errors has a binomial distribution [Appendix B.3.2].
g(y|t, p = 0.9) =
t!
0.9y 0.1t−y , y = 0, 1, … , t
y!(t − y)!
Compute the probability that the editor finds one error on a page given that the page actually has
four errors.
c. Find the joint probability P[Y = 3, T = 4].
d. It can be shown that the probability the editor will find Y errors on a page follows a Poisson distribution with mean E(Y) = 0.9μ. Use this information to find the conditional probability that there
are T = 4 errors on a page given that Y = 3 are found.
k
k
k
CHAPTER 2
The Simple Linear
Regression Model
LEARNING OBJECTIVES
Remark
Learning Objectives and Keywords sections will appear at the beginning of each chapter. We
urge you to think about and possibly write out answers to the questions, and make sure you
recognize and can define the keywords. If you are unsure about the questions or answers,
consult your instructor. When examples are requested in Learning Objectives sections, you
should think of examples not in the book.
k
Based on the material in this chapter you should be able to
1. Explain the difference between an estimator and
an estimate, and why the least squares
estimators are random variables, and why least
squares estimates are not.
2. Discuss the interpretation of the slope and intercept parameters of the simple regression model,
and sketch the graph of an estimated equation.
3. Explain the theoretical decomposition of an
observable variable y into its systematic and
random components, and show this
decomposition graphically.
4. Discuss and explain each of the assumptions of
the simple linear regression model.
5. Explain how the least squares principle is used
to fit a line through a scatter plot of data. Be able
to define the least squares residual and the least
squares fitted value of the dependent variable
and show them on a graph.
6. Define the elasticity of y with respect to x and
explain its computation in the simple linear
regression model when y and x are not
transformed in any way, and when y and/or x
have been transformed to model a nonlinear
relationship.
7. Explain the meaning of the statement ‘‘If
regression model assumptions SR1–SR5 hold,
then the least squares estimator b2 is
unbiased.’’ In particular, what exactly does
‘‘unbiased’’ mean? Why is b2 biased if an
important variable has been omitted from the
model?
8. Explain the meaning of the phrase ‘‘sampling
variability.’’
)2
∑(
9. Explain how the factors σ2 ,
xi − x , and N
affect the precision with which we can estimate
the unknown parameter β2 .
46
k
k
k
2.1 An Economic Model
12. Explain the difference between an explanatory
variable that is fixed in repeated samples and an
explanatory variable that is random.
10. State and explain the Gauss–Markov theorem.
11. Use the least squares estimator to estimate
nonlinear relationships and interpret the
results.
KEYWORDS
assumptions
asymptotic
biased estimator
BLUE
degrees of freedom
dependent variable
deviation from the mean form
econometric model
economic model
elasticity
exogenous variable
Gauss–Markov theorem
heteroskedastic
k
13. Explain the term ‘‘random sampling.’’
homoskedastic
independent variable
indicator variable
least squares estimates
least squares estimators
least squares principle
linear estimator
log-linear model
nonlinear relationship
prediction
quadratic model
random error term
random-x
regression model
regression parameters
repeated sampling
sampling precision
sampling properties
scatter diagram
simple linear regression analysis
simple linear regression function
specification error
strictly exogenous
unbiased estimator
Economic theory suggests many relationships between economic variables. In microeconomics,
you considered demand and supply models in which the quantities demanded and supplied of a
good depend on its price. You considered “production functions” and “total product curves” that
explained the amount of a good produced as a function of the amount of an input, such as labor,
that is used. In macroeconomics, you specified “investment functions” to explain that the amount
of aggregate investment in the economy depends on the interest rate and “consumption functions”
that related aggregate consumption to the level of disposable income.
Each of these models involves a relationship between economic variables. In this chapter, we
consider how to use a sample of economic data to quantify such relationships. As economists, we
are interested in questions such as the following: If one variable (e.g., the price of a good) changes
in a certain way, by how much will another variable (the quantity demanded or supplied) change?
Also, given that we know the value of one variable, can we forecast or predict the corresponding
value of another? We will answer these questions by using a regression model. Like all models,
the regression model is based on assumptions. In this chapter, we hope to be very clear about
these assumptions, as they are the conditions under which the analysis in subsequent chapters
is appropriate.
2.1
47
An Economic Model
In order to develop the ideas of regression models, we are going to use a simple, but important,
economic example. Suppose that we are interested in studying the relationship between household
income and expenditure on food. Consider the “experiment” of randomly selecting households
from a particular population. The population might consist of households within a particular city,
state, province, or country. For the present, suppose that we are interested only in households
with an income of $1000 per week. In this experiment, we randomly select a number of households from this population and interview them. We ask the question “How much did you spend
per person on food last week?” Weekly food expenditure, which we denote as y, is a random
variable since the value is unknown to us until a household is selected and the question is asked
and answered.
k
k
k
48
CHAPTER 2
The Simple Linear Regression Model
Remark
In the Probability Primer and Appendices B and C, we distinguished random variables from
their values by using uppercase (Y) letters for random variables and lowercase (y) letters for
their values. We will not make this distinction any longer because it leads to complicated
notation. We will use lowercase letters, like “y,” to denote random variables as well as their
values, and we will make the interpretation clear in the surrounding text.
The continuous random variable y has a probability density function (which we will abbreviate as
pdf ) that describes the probabilities of obtaining various food expenditure values. If you are rusty
or uncertain about probability concepts, see the Probability Primer and Appendix B at the end of
this book for a comprehensive review. The amount spent on food per person will vary from one
household to another for a variety of reasons: some households will be devoted to gourmet food,
some will contain teenagers, some will contain senior citizens, some will be vegetarian, and some
will eat at restaurants more frequently. All of these factors and many others, including random,
impulsive buying, will cause weekly expenditures on food to vary from one household to another,
even if they all have the same income. The pdf f(y) describes how expenditures are “distributed”
over the population and might look like Figure 2.1.
The pdf in Figure 2.1a is actually a conditional pdf since it is “conditional” upon household
income. If x = weekly household income = $1000, then the conditional pdf is f(y|x = $1000).
The conditional mean, or expected value, of y is E(y|x = $1000) = μy|x and is our population’s
mean weekly food expenditure per person.
k
k
Remark
The expected value of a random variable is called its “mean” value, which is really a contraction of population mean, the center of the probability distribution of the random variable.
This is not the same as the sample mean, which is the arithmetic average of numerical values.
Keep the distinction between these two usages of the term “mean” in mind.
The conditional variance of y is var(y|x = $1000) = σ2 , which measures the dispersion of household expenditures y about their mean μy|x . The parameters μy|x and σ2 , if they were known,
would give us some valuable information about the population we are considering. If we knew
these parameters, and if we knew that the conditional distribution f (y|x = $1000) was normal,
f(y|x = 1000)
f(y|x = 1000)
f(y|x)
μy|x
y
f(y|x = 1000)
f(y|x = 2000)
μy|2000
μy|1000
(b)
(a)
FIGURE 2.1 (a) Probability distribution f ( y|x = 1000) of food expenditure y given income
x = $1000. (b) Probability distributions of food expenditure y given incomes
x = $1000 and x = $2000.
k
y
k
2.2 An Econometric Model
49
)
(
N μy|x , σ2 , then we could calculate probabilities that y falls in specific intervals using properties
of the normal distribution. That is, we could compute the proportion of the household population
that spends between $50 and $75 per person on food, given $1000 per week income.
As economists, we are usually interested in studying relationships between variables, in this
case the relationship between y = weekly food expenditure per person and x = weekly household
income. Economic theory tells us that expenditure on economic goods depends on income. Consequently, we call y the “dependent variable” and x the “independent” or “explanatory” variable.
In econometrics, we recognize that real-world expenditures are random variables, and we want
to use data to learn about the relationship.
An econometric analysis of the expenditure relationship can provide answers to some important questions, such as: If weekly income goes up by $100, how much will average weekly food
expenditures rise? Or, could weekly food expenditures fall as income rises? How much would we
predict the weekly per person expenditure on food to be for a household with an income of $2000
per week? The answers to such questions provide valuable information for decision makers.
k
Using … per person food spending information … one can determine the similarities
and disparities in the spending habits of households of differing sizes, races, incomes,
geographic areas, and other socioeconomic and demographic features. This information
is valuable for assessing existing market conditions, product distribution patterns, consumer buying habits, and consumer living conditions. Combined with demographic and
income projections, this information may be used to anticipate consumption trends. The
information may also be used to develop typical market baskets of food for special population groups, such as the elderly. These market baskets may, in turn, be used to develop
price indices tailored to the consumption patterns of these population groups. [Blisard,
Noel, Food Spending in American Households, 1997–1998, Electronic Report from the
Economic Research Service, U.S. Department of Agriculture, Statistical Bulletin Number
972, June 2001]
From a business perspective, if we are managers of a supermarket chain (or restaurant, or health
food store, etc.), we must consider long-range plans. If economic forecasters are predicting that
local income will increase over the next few years, then we must decide whether, and how much,
to expand our facilities to serve our customers. Or, if we plan to open franchises in high-income
and low-income neighborhoods, then forecasts of expenditures on food per person, along with
neighborhood demographic information, give an indication of how large the stores in those areas
should be.
In order to investigate the relationship between expenditure and income, we must build an
economic model and then a corresponding econometric model that forms the basis for a quantitative or empirical economic analysis. In our food expenditure example, economic theory suggests
that average weekly per person household expenditure on food, represented mathematically by the
conditional mean E(y|x) = μy|x , depends on household income x. If we consider households with
different levels of income, we expect the average expenditure on food to change. In Figure 2.1b,
we show the pdfs of food expenditure for two different levels of weekly income, $1000 and $2000.
Each conditional pdf f (y|x) shows that expenditures will be distributed about a mean value μy|x ,
but the mean expenditure by households with higher income is larger than the mean expenditure
by lower income households.
In order to use data, we must now specify an econometric model that describes how the data
on household income and food expenditure are obtained and that guides the econometric analysis.
2.2
An Econometric Model
Given the economic reasoning in the previous section, and to quantify the relationship between
food expenditure and income, we must progress from the ideas in Figure 2.1, to an econometric
model. First, suppose a three-person household has an unwavering rule that each week they
k
k
k
50
k
CHAPTER 2
The Simple Linear Regression Model
spend $80 and then also spend 10 cents of each dollar of income received on food. Let y = weekly
household food expenditure ($) and let x = weekly household income ($). Algebraically their
rule is y = 80 + 0.10x. Knowing this relationship we calculate that in a week in which the
household income is $1000, the household will spend $180 on food. If weekly income increases
by $100 to $1100, then food expenditure increases to $190. These are predictions of food
expenditure given income. Predicting the value of one variable given the value of another, or
others, is one of the primary uses of regression analysis.
A second primary use of regression analysis is to attribute, or relate, changes in one variable
to changes in another variable. To that end, let “Δ” denote “change in” in the usual algebraic
way. A change in income of $100 means that Δx = 100. Because of the spending rule y = 80 +
0.10x the change in food expenditure is Δy = 0.10Δx = 0.10(100) = 10. An increase in income of
$100 leads to, or causes, a $10 increase in food expenditure. Geometrically, the rule is a line with
“y-intercept” 80 and slope Δy∕Δx = 0.10. An economist might say that the household “marginal
propensity to spend on food is 0.10,” which means that from each additional dollar of income
10 cents is spent on food. Alternatively, in a kind of economist shorthand, the “marginal effect of
income on food expenditure is 0.10.” Much of economic and econometric analysis is an attempt
to measure a causal relationship between two economic variables. Claiming causality here, that
is, changing income leads to a change in food expenditure, is quite clear given the household’s
expenditure rule. It is not always so straightforward.
In reality, many other factors may affect household expenditure on food; the ages and sexes
of the household members, their physical size, whether they do physical labor or have desk jobs,
whether there is a party following the big game, whether it is an urban or rural household, whether
household members are vegetarians or into a paleo-diet, as well as other taste and preference
factors (“I really like truffles”) and impulse shopping (“Wow those peaches look good!”). Lots of
factors. Let e = everything else affecting food expenditure other than income. Furthermore, even
if a household has a rule, strict or otherwise, we do not know it. To account for these realities, we
suppose that the household’s food expenditure decision is based on the equation
y = β1 + β2 x + e
(2.1)
In addition to y and x, equation (2.1) contains two unknown parameters, β1 and β2 , instead of
“80” and “0.10,” and an error term e, which represents all those other factors (everything else)
affecting weekly household food expenditure.
Imagine that we can perform an experiment on the household. Let’s increase the household’s
income by $100 per week and hold other things constant. Holding other things constant, or holding all else (everything else) equal, is the ceteris paribus assumption discussed extensively in
economic principles courses. Let Δx = 100 denote the change in household income. Assuming
everything else affecting household food expenditure, e, is held constant means that Δe = 0. The
effect of the change in income is Δy = β2 Δx + Δe = β2 Δx = β2 × 100. The change in weekly food
expenditure Δy = β2 × 100 is explained by, or caused by, the change in income. The unknown
parameter β2 , the marginal propensity to spend on food from income, tells us the proportion of
the increase in income used for food purchases; it answers the “how much” question “How much
will food expenditure change given a change in income, holding all else constant?”
The experiment in the previous paragraph is not feasible. We can give a household an
extra $100 income, but we cannot hold all else constant. The simple calculation of the marginal
effect of an increase in income on food expenditure Δy = β2 × 100 is not possible. However,
we can shed light on this “how much” question by using regression analysis to estimate β2 .
Regression analysis is a statistical method that uses data to explore relationships between
variables. A simple linear regression analysis examines the relationship between a y-variable
and one x-variable. It is said to be “simple” not because it is easy, but because there is only one
x-variable. The y-variable is called the dependent variable, the outcome variable, the explained
variable, the left-hand-side variable, or the regressand. In our example, the dependent variable is
k
k
k
2.2 An Econometric Model
51
y = weekly household expenditure on food. The variable x = weekly household income is called
the independent variable, the explanatory variable, the right-hand-side variable, or the regressor.
Equation (2.1) is the simple linear regression model.
All models are abstractions from reality and working with models requires assumptions. The
same is true for the regression model. The first assumption of the simple linear regression model is
that relationship (2.1) holds for the members of the population under consideration. For example,
define the population to be three-person households in a given geographic region, say southern
Australia. The unknowns β1 and β2 are called population parameters. We assert the behavioral
rule y = β1 + β2 x + e holds for all households in the population. Each week food expenditure
equals β1 , plus a proportion β2 of income, plus other factors, e.
The field of statistics was developed because, in general, populations are large, and it is
impossible (or impossibly costly) to examine every population member. The population of
three-person households in a given geographic region, even if it is only a medium-sized city, is
too large to survey individually. Statistical and econometric methodology examines and analyzes
a sample of data from the population. After analyzing the data, we make statistical inferences.
These are conclusions or judgments about a population based on the data analysis. Great care
must be taken when drawing inferences. The inferences are conclusions about the particular
population from which the data were collected. Data on households from southern Australia
may, or may not, be useful for making inferences, drawing conclusions, about households from
the southern United States. Do Melbourne, Australia, households have the same food spending
patterns as households in New Orleans, Louisiana? That might be an interesting research topic.
If not, then we may not be able to draw valid conclusions about New Orleans household behavior
from the sample of Australian data.
k
2.2.1
Data Generating Process
k
The sample of data, and how the data are actually obtained, is crucially important for subsequent
inferences. The exact mechanisms for collecting a sample of data are very discipline specific (e.g.,
agronomy is different from economics) and beyond the scope of this book.1 For the household
food expenditure example, let us assume that we can obtain a sample at a point in time [these are
cross-sectional
data] consisting of N data pairs that are randomly selected from the population.
)
(
Let yi , xi denote the ith data pair, i = 1, … , N. The variables yi and xi are random variables,
because their values are not
) until they are observed. Randomly selecting households makes
( known
the first observation
pair
y
,
x
1 1 statistically independent of all other data
(
)
( pairs,
) and each observation pair yi , xi is statistically independent of every other data pair, yj , xj , where i ≠ j. We
)
(
further assume that the random variables yi and xi have a joint pdf f yi , xi that describes their
distribution of values. We often do not know the exact nature of the joint distribution (such as
bivariate normal; see Probability Primer, Section P.7.1), but all pairs drawn from the same population are assumed to follow the same joint pdf , and, thus, the data pairs are not only statistically
independent but are also identically distributed (abbreviated i.i.d. or iid). Data pairs that are iid
are said to be a random sample.
If our first assumption is true, that the behavioral
( rule)y = β1 + β2 x + e holds for all households in the population, then restating (2.1) for each yi , xi data pair
yi = β1 + β2 xi + ei ,
i = 1, … , N
(2.1)
This is sometimes called the data generating process (DGP) because we assume that the observable data follow this relationship.
............................................................................................................................................
1 See,
for example, Paul S. Levy and Stanley Lemeshow (2008) Sampling of Populations: Methods and Applications,
4th Edition, Hoboken, NJ: John Wiley and Sons, Inc.
k
k
52
CHAPTER 2
The Simple Linear Regression Model
2.2.2
k
The Random Error and Strict Exogeneity
The second assumption
) simple regression model (2.1) concerns the “everything else”
( of the
term e. The variables yi , xi are random variables because we do not know what values they
take until a particular household is chosen and they are observed. The error term ei is also a
random variable. All the other factors affecting food expenditure except income will be different
for each population household if for no other reason that everyone’s tastes and preferences are
different. Unlike food expenditure and income, the random error term ei is not observable; it
is unobservable. We cannot measure tastes and preferences in any direct way, just as we cannot
directly measure the economic “utility” derived from eating a slice of cake. The second regression assumption is that the x-variable, income, cannot be used to predict the value of ei , the effect
of the collection of all other factors affecting the food expenditure by the ith household. Given
2
an income value xi for the ith household, the best (optimal)
) predictor of the random error ei is
(
the conditional expectation, or conditional
E ei |xi . The assumption that xi cannot be used
)
( mean,
to predict ei is equivalent to saying E ei |xi = 0. That is, given a household’s income we cannot do any better than predicting that the random error is zero; the effects of all other factors on
food expenditure average out, in a very specific way, to zero. We will discuss other situations in
which this might or might
)not be true in Section 2.10. For now, recall(from)the Probability
( )Primer,
(
Section P.6.5, that E ei |xi = 0 has two implications. The first is E ei |xi = 0 =⇒ E ei = 0; if
the conditional expected value of the random error is zero, then the unconditional expectation
of the random error is also zero. In the population, the average effect of all the omitted factors
summarized by the random error( term) is zero.
(
)
The second implication is E ei |xi = 0 =⇒ cov ei , xi = 0. If the conditional expected value
of the random error is zero, then ei , the random error for the ith observation, has covariance zero
and correlation zero, with the corresponding observation xi . In our example, the random component ei , representing all factors affecting food expenditure except income for the ith household,
is uncorrelated with income for that household. You might wonder how that could possibly be
shown to be true. After all, ei is unobservable. The answer is that it is very hard work. You
must convince yourself and your audience that anything that might have been omitted from the
model is not correlated with xi . The primary tool is economic reasoning: your own intellectual
experiments (i.e., thinking), reading literature( on the
) topic and discussions with colleagues or
classmates. And we really can’t prove that E ei |xi = 0 is true with absolute certainty in most
economic models. (
)
( We) noted that E ei |xi = 0 has two implications. If either of the implications is not true, then
E ei |xi = 0 is not true, that is,
(
)
( )
(
)
E ei |xi ≠ 0 if (i) E ei ≠ 0 or if (ii) cov ei , xi ≠ 0
)
(
In the first case, if the population average of the random errors ei is not
zero,
then
E
e
|x
i
i
( )
( ) ≠ 0.
In a certain sense, we will be able to work around the
) when E ei ≠(0, say) if E ei = 3,
( case
as you will see below. The second implication of E ei |xi = 0 is that cov ei , xi = 0; the random error for the ith observation has
) covariance and correlation with the ith observation
( zero
variable x is said to be exogeon the explanatory variable. If cov ei , xi = 0, the (explanatory
)
nous, providing our first assumption that the pairs yi , xi are iid holds. When x is exogenous,
regression analysis
)can be used successfully to estimate β1 and β2 . To differentiate
(
) the weaker
(
from
the
stronger
condition
E
e
= 0, we say
|x
condition cov ei , xi = 0, simple( exogeneity,
i
i
)
(
)
that x is strictly exogenous if E ei |xi = 0. If cov ei , xi ≠ 0, then x is said to be endogenous.
When x is endogenous, it is more difficult, sometimes much more difficult, to carry out statistical
inference. A great deal will be said about exogeneity and strict exogeneity in the remainder of
this book.
............................................................................................................................................
2 You
will learn about optimal prediction in Appendix 4C.
k
k
k
2.2 An Econometric Model
E X A M P L E 2.1
A Failure of the Exogeneity Assumption
Consider a regression model exploring the relationship
between a working person’s wage and their years of
education, using a random sample of data. The simple
regression model is WAGEi = β1 + β2 EDUCi + ei , where
WAGEi is the hourly wage rate of the ith randomly selected
person
and EDUC
) i is their years of education. The pairs
(
WAGEi , EDUCi from the random sample are assumed to be
iid. In this model, the random error ei accounts for all those
factors other than EDUCi that affect a person’s wage rate.
What might some of those factors be? Ability, intelligence,
2.2.3
perseverance, and industriousness are all important characteristics of an employee and likely to influence their wage
rate. Are any of these factors which are bundled into ei likely
to be correlated with EDUCi ? A few moments reflection
will lead you to say “yes.” It is very plausible that those
with higher education have higher ability, intelligence,
perseverance, and industriousness. Thus, there is a strong
argument that EDUCi is an endogenous regressor in this
regression and that the strict exogeneity assumption fails.
The Regression Function
The importance
( of) the strict exogeneity assumption is the following. If the strict exogeneity
assumption E ei |xi = 0 is true, then the conditional expectation of yi given xi is
)
)
(
(
E yi |xi = β1 + β2 xi + E ei |xi = β1 + β2 xi , i = 1, … , N
(2.2)
)
(
The conditional expectation E yi |xi = β1 + β2 xi in (2.2) is called the regression function, or
population regression function. It says that in the population the average value of the dependent
variable for the ith observation, conditional
( on )xi , is given by β1 + β2 xi . It also says that given a
change in x, Δx, the resulting change in E yi |xi is β2 Δx holding all else constant, in the sense
that given xi the average of the random errors is zero, and any change in x is not correlated with
any corresponding change in the random error e. In this case, we can say that a change
in
) x leads
(
to, or causes, a change in the expected (population average) value of y given xi , E yi |x(i .
)
The regression function in (2.2) is shown in Figure 2.2, with y-intercept β1 = E yi |xi = 0
and slope
)
)
(
(
ΔE yi |xi
dE yi |xi
β2 =
=
(2.3)
Δxi
dxi
where Δ denotes “change in” and dE(y|x)∕dx denotes the “derivative” of E(y|x) with respect to x.
We will not use derivatives to any great extent in this book, and if you are not too familiar with
the concept you can think of “d” as a stylized version of Δ and go on. See Appendix A.3 for a
discussion of derivatives.
E( y| x)
Average expenditure
k
53
E(y| x) = β1 + β2 x
ΔE( y| x)
β2 =
Δx
ΔE(y| x)
dE(y | x)
=
Δx
dx
β1
x
Income
FIGURE 2.2 The economic model: a linear relationship between average
per person food expenditure and income.
k
k
k
54
CHAPTER 2
The Simple Linear Regression Model
E X A M P L E 2.2
Strict Exogeneity in the Household Food Expenditure Model
The strict exogeneity assumption is that the average of
everything else affecting the food expenditure of the ith
household, given the income of the ith household, is zero.
Could this be true? One test of this possibility is the question
“Using the income of the ith household, can we predict the
value of ei , the combined influence of all factors affecting
food expenditure other than income?” If( the )answer is yes,
then strict exogeneity fails. If not, then E ei |xi = 0 may be a
plausible assumption. And if it is, then equation (2.1) can be
interpreted as a causal model, and β2 can be thought of as the
marginal effect of income on expected (average) household
food expenditure, (holding
) all else constant, as shown in
equation (2.3). If E ei |xi ≠ 0 then xi can be used to predict a
nonzero value for ei , which in turn will affect the value of yi .
In this case, β2 will not capture all the effects of an income
change, and the model cannot be interpreted as causal.
Another important consequence of the assumption of strict exogeneity is that it allows us to think
of the econometric model as decomposing the dependent variable into two components: one that
varies systematically as the values of the independent variable change and another that is random
That is, the econometric model yi = β1 + β2 xi + ei can be broken into two parts:
)
( “noise.”
E yi |xi = β1 + β2 xi and the random error, ei . Thus
)
(
yi = β1 + β2 xi + ei = E yi |xi + ei
k
The values
( of) the dependent variable yi vary systematically due to variation in the conditional
mean E yi |xi = β1 + β2 xi , as the value of the explanatory variable changes, and the values of the
dependent variable yi vary randomly due to ei . The conditional pdf s of e and y are identical except
for their location, as shown in Figure 2.3. Two values of food expenditure y1 and y2 for households
with x = $1000 of weekly income are shown in Figure 2.4 relative to their conditional mean. There
will be variation in household expenditures on food from one household to another because of
variations in tastes and preferences, and everything else. Some will spend more than the average
value for households with the same income, and some will
( spend less.)If we knew β1 and β2 , then
we could compute the conditional mean expenditure E yi |x = 1000 = β1 + β2 (1000) and also
the value of the random errors e1 and e2 . We never know β1 and β2 so we can never compute e1
and e2 . What we are assuming, however, is that at each level of income x the average value of all
that is represented by the random error is zero.
2.2.4
Random Error Variation
We have
) the assumption that the conditional expectation of the random error term is
( made
zero, E ei |xi = 0. For the random error term we are interested in both its conditional mean,
f(∙∣x = 1000)
f (e∣x = 1000)
E(e∣x = 1000) = 0
f (y∣x = 1000)
E(y∣x = 1000) =
β1 + β2(1000)
FIGURE 2.3 Conditional probability densities for e
and y.
k
k
k
2.2 An Econometric Model
55
f(y∣x = 1000)
f(y∣x = 1000)
e1
e2
y1
y2
E(y∣x = 1000) = β1 + β2(1000)
y
FIGURE 2.4 The random error.
or expected value, and its variance. Ideally, the conditional variance of the random error is
constant.
)
(
(2.4)
var ei |xi = σ2
This is the homoskedasticity (also spelled homoscedasticity) assumption. At each xi the
variation of the random error component is the same. Assuming the population relationship
yi = β1 + β2 xi + ei the conditional variance of the dependent variable is
)
(
)
(
)
(
var yi |xi = var β1 + β2 xi + ei |xi = var ei |xi = σ2
k
The simplification works because by conditioning on xi we are treating it as if it is known and
therefore not random. Given xi the component β1 + β2 xi is not random, so the variance rule (P.14)
applies.
This was an explicit assumption in Figure 2.1(b) where the pdf s f (y|x = 1000) and
f (y|x( = 2000)
have the same variance, σ2 . If strict exogeneity holds, then the regression function
)
is E yi |xi = β1 + β2 xi , as shown in Figure 2.2. The conditional distributions f (y|x = 1000) and
f (y|x = 2000) are placed along the conditional mean function in Figure 2.5. In the household
expenditure example, the idea is that for a particular level of household income x, the values
of household food expenditure will vary randomly about the conditional mean due to the
assumption that at each x the average value of the random error e is zero. Consequently, at each
level of income, household food expenditures are centered on the regression function. The conditional homoskedasticity assumption implies that at each level of income the variation in food
f(y∣x)
y
re
ditu
d
Foo
en
exp
0)
0)
x=
f(y∣
100
x=
f(y∣
200
β1 + β2x = E(y∣x)
μy∣1000
x=
100
0
x=
200
0
Hou
μy∣2000
seho
ld in
com
e
x
FIGURE 2.5 The conditional probability density functions for y, food
expenditure, at two levels of income.
k
k
k
56
CHAPTER 2
The Simple Linear Regression Model
expenditure about its mean is the same. That means that at each and every level of income
( we)are
equally uncertain about how far food expenditures might fall from their mean value, E yi |xi =
β1 + β2 xi . Furthermore, this uncertainty
does not depend on income or anything else. If this
)
(
assumption is violated, and var ei |xi ≠ σ2 , then the random errors are said to be heteroskedastic.
2.2.5
Variation in x
2.2.6
Error Normality
)
(
In a regression analysis, one of the objectives is to estimate β2 = ΔE yi |xi ∕Δxi . If we are to hope
that a sample of data can be used to estimate the effects of changes in x, then we must observe
some different values of the explanatory variable x in the sample. Intuitively, if we collect data
only on households with income $1000, we will not be able to measure the effect of changing
income on the average value of food expenditure. Recall from elementary geometry that “it takes
two points to determine a line.” The minimum number of x-values in a sample of data that will
allow us to proceed is two. You will find out in Section 2.4.4 that in fact the more different values
of x, and the more variation they exhibit, the better our regression analysis will be.
k
In the discussion surrounding Figure 2.1, we explicitly made the assumption that food expenditures, given income, were normally distributed. In Figures 2.3–2.5, we implicitly made the
assumption of conditionally normally distributed errors and dependent variable by drawing classically bell-shaped curves. It is not at all necessary for the random errors to be conditionally
normal in order for regression analysis to “work.” However, as you will discover in Chapter 3,
when samples are small, it is advantageous for statistical inferences that the random errors, and
dependent variable y, given each x-value, are normally distributed. The normal distribution has a
long and interesting history,3 as a little Internet searching will reveal. One argument for assuming regression errors are normally distributed is that they represent a collection of many different
factors. The Central Limit Theorem, see Appendix C.3.4, says roughly that collections of many
random factors tend toward having a normal distribution. In the context of the food expenditure
model, if we consider that the random errors reflect tastes and preferences, it is entirely plausible
that the random errors at each income level are normally
When the assumption
of con)
)
( distributed.
(
ditionally normal errors is made, we write ei |xi ∼ N 0, σ2 and also then yi |xi ∼ N β1 + β2 xi , σ2 .
It is a very strong assumption when it is made, and as mentioned it is not strictly speaking necessary, so we call it an optional assumption.
2.2.7
Generalizing the Exogeneity Assumption
)
(
So far we have assumed that the data pairs yi , xi have been drawn from a random sample and
are iid. What happens if the sample values of the explanatory variable are correlated? And how
might that happen?
A lack of independence occurs naturally when using financial or macroeconomic time-series
data. Suppose we observe the monthly report on new housing starts, yt , and the current
)30-year
(
fixed mortgage rate, xt , and we postulate the model yt = β1 + β2 xt + et . The data yt , xt can be
described as macroeconomic time-series data. In contrast to cross-section data where we have
observations on a number of units (say households or firms or persons or countries) at a given
point in time, with time-series data we have observations over time on a number of variables. It is
customary to use(a “t”)subscript to denote time-series data and to use T to denote the sample size.
In the data pairs yt , xt , t = 1, … , T, both yt and xt are random because we do not know the values
............................................................................................................................................
3 For
example, Stephen M. Stigler (1990) The History of Statistics: The Measurement of Uncertainty, Reprint Edition,
Belknap Press, 73–76.
k
k
k
2.2 An Econometric Model
57
until they are observed. Furthermore, each of the data series is likely to be correlated across time.
For example, the monthly fixed mortgage rate is likely to change slowly over time
making
the
(
)
rate at time t correlated with the rate at time t − 1. The assumption that the pairs yt , xt represent
random iid draws from a probability distribution is not realistic. When considering the exogeneity
assumption for this case, we need to be concerned not just with possible correlation between xt
and et , but also with possible correlation between et and every other value of the explanatory
variable, namely, xs , s = 1, 2, … , T. If xs is correlated with xt , then it is possible that xs (say, the
mortgage rate in one month) may have an impact on yt (say, housing starts in the next month).
Since it is xt , not x(s that)appears in the equation yt = β1 + β2 xt + et , the effect of xs will be included
in et , implying E et |xs (≠ 0. We
) could use xs to help predict the value of et . This possibility is
That is, independence of the pairs
ruled
out
when
the
pairs
y
,
x
t t( are )assumed to be(independent.
(
)
)
yt , xt and the assumption E et |xt = 0 imply E et |xs = 0 for all s = 1, 2, … , T.
To extend the strict
assumption to models where the values of x are correlated,
( exogeneity
)
we need to assume E et |xs = 0 for all (t, s) = 1, 2, … , T. This means that we cannot predict the
random error at time t,( et , using
any of the values of the explanatory variable. Or, in terms of
)
our earlier notation, E ei |xj = 0 for all (i, j) = 1, 2, … , N. To write this assumption in a more
)
(
convenient form, we introduce the notation 𝐱 = x1 , x2 , … , xN . That is, we are using x to denote
all sample observations on the explanatory
variable. Then, a more general way of writing the
(
)
strict exogeneity
assumption
is
E
e
|𝐱
=
0,
i = 1, 2, … , N. From this assumption, we can also
i
(
)
write E yi |𝐱 = β1 + β2 xi for i = 1, 2, … , N. This assumption is discussed further in
( the) context of alternative types of data in Section 2.10 and in Chapter
9.
The
assumption
E
)
(ei |𝐱 )= 0,
(
i = 1, 2, … , N, is a weaker assumption than assuming E ei |xi = 0 and that the pairs yi , xi are
independent, and it enables us to derive a number of results for cases where different observations
on x may be correlated as well as for the case where they are independent.
k
k
2.2.8
Error Correlation
( )
In addition
( )
( ) to possible correlations between a random error for one household ei or one time
period et being correlated with the value of an explanatory variable for another household xj
( )
or time period xs , it is possible that there are correlations between the random error terms.
With cross-sectional data, data on households, individuals, or firms collected at one point in
time, there may be a lack of statistical independence between random errors for individuals who
are spatially connected. That is, suppose that we collect observations on two (or more) individuals
who live in the same neighborhood. It is very plausible that there are similarities among people
who live in a particular neighborhood. Neighbors can be expected to have similar incomes if the
homes in a neighborhood are homogenous. Some suburban neighborhoods are popular because
of green space and schools for young children, meaning households may have members similar in
ages and interests. We might add a spatial component s to the error and say that the random errors
ei (s) and ej (s) for the ith and jth households are possibly correlated because of their common
location. Within a larger sample of data, there may be clusters of observations with correlated
errors because of the spatial component.
In a time-series context, your author is writing these pages on the tenth anniversary of Hurricane Katrina, which devastated the U.S. Gulf Coast and the city of New Orleans, Louisiana, in
particular. The impact of that shock did not just happen and then go away. The effect of that huge
random event had an effect on housing and financial markets during August 2005, and also in
September, October, and so on, to this day. Consequently, the random
errors
) in the population
(
) re(
lationship yt = β1 + β2 xt + et are correlated over time, so that cov et , et+1 ≠ 0, cov et , et+2 ≠ 0,
and so on. This is called serial correlation, or autocorrelation, in econometrics.
The starting point in regression analysis(is to assume
that there is no error correlation. In
)
time-series models, we start
by
assuming
cov
e
,
e
|𝐱
=
0
for
t ≠ s, and for cross-sectional data
t s
)
(
we start by assuming cov ei , ej |𝐱 = 0 for i ≠ j. We will cope with failure of these assumptions
in Chapter 9.
k
k
58
CHAPTER 2
The Simple Linear Regression Model
2.2.9
Summarizing the Assumptions
We summarize the starting assumptions of the simple regression model in a very general way.
In our summary we use subscripts i and j but the assumptions are general, and apply equally to
time-series data. If these assumptions hold, then regression analysis can successfully
)estimate
(
the(unknown
population
parameters
β
and
β
and
we
can
claim
that
β
=
ΔE
y
|x
1
2
2
i i ∕Δxi =
)
dE yi |xi ∕dxi measures a causal effect. We begin our study of regression analysis and econometrics making these strong assumptions about the DGP. For future reference, the assumptions
are named SR1–SR6, “SR” denoting “simple regression.”
Econometrics is in large part devoted to handling data and models for which these assumptions may not hold, leading to modifications of usual methods for estimating β1 and β2 , testing
hypotheses, and predicting outcomes. In Chapters 2 and 3, we study the simple regression model
under these, or similar, strong assumptions. In Chapter 4, we introduce modeling issues and diagnostic testing. In Chapter 5, we extend our model to multiple regression analysis with more than
one explanatory variable. In Chapter 6, we treat modeling issues concerning the multiple regression model, and starting in Chapter 8 we address situations in which SR1–SR6 are violated in
one way or another.
Assumptions of the Simple Linear Regression Model
)
(
SR1: Econometric Model All data pairs yi , xi collected from a population satisfy the
relationship
yi = β1 + β2 xi + ei , i = 1, … , N
k
SR2:( Strict Exogeneity
The conditional expected value of the random error ei is zero. If
)
𝐱 = x1 , x2 , … , xN , then
(
)
E ei |x = 0
If strict exogeneity holds, then the population regression function is
(
)
E yi |x = β1 + β2 xi , i = 1, … , N
and
(
)
yi = E yi |x + ei , i = 1, … , N
SR3: Conditional Homoskedasticity The conditional variance of the random error is
constant.
)
(
var ei |x = σ2
SR4: Conditionally Uncorrelated Errors The conditional covariance of random errors ei
and ej is zero.
(
)
cov ei , ej |x = 0 for i ≠ j
SR5: Explanatory Variable Must Vary In a sample of data, xi must take at least two different values.
SR6: Error Normality (optional) The conditional distribution of the random errors is
normal.
)
(
ei |x ∼ N 0, σ2
The random error e and the dependent variable y are both random variables, and as we have
shown, the properties of one variable can be determined from the properties of the other. There
is, however, one interesting difference between them: y is “observable” and e is “unobservable.”
k
k
k
2.3 Estimating the Regression Parameters
59
If the regression parameters
) β1 and β2 were known, then for any values yi and xi we could
(
=
y
−
β
+
β
x
calculate
e
i
1
2 i . This is illustrated in Figure 2.4. Knowing the regression function
(
) i
E yi |𝐱 = β1 + β2 xi we could separate yi into its systematic and random parts. However, β1 and
β2 are never known, and it is impossible to calculate ei .
What comprises the error term e? The random error e represents all factors affecting y other
than x, or what we have called everything (else. )These factors cause individual observations yi
to differ from the conditional mean value E yi |𝐱 = β1 + β2 xi . In the food expenditure example,
what factors can(result
) in a difference between household expenditure per person yi and its conditional mean E yi |𝐱 = β1 + β2 xi ?
1. We have included income as the only explanatory variable in this model. Any other economic
factors that affect expenditures on food are “collected” in the error term. Naturally, in any
economic model, we want to include all the important and relevant explanatory variables in
the model, so the error term e is a “storage bin” for unobservable and/or unimportant factors
affecting household expenditures on food. As such, it adds noise that masks the relationship
between x and y.
2. The error term e captures any approximation error that arises because the linear functional
form we have assumed may be only an approximation to reality.
3. The error term captures any elements of random behavior that may be present in each individual. Knowing all the variables that influence a household’s food expenditure might not
be enough to perfectly predict expenditure. Unpredictable human behavior is also contained
in e.
k
If we have omitted( some
) important factor, or made any other serious specification error, then
assumption SR2 E ei |𝐱 = 0 will be violated, which will have serious consequences.
2.3
k
Estimating the Regression Parameters
E X A M P L E 2.3
Food Expenditure Model Data
The economic and econometric models we developed in the
previous section are the basis for using a sample of data
to estimate the intercept and slope parameters, β1 and β2 .
For illustration we examine typical data on household food
expenditure and weekly income from a random sample of
40 households. Representative observations and summary
statistics are given in Table 2.1. We control for household
size by considering only three-person households. The
values of y are weekly food expenditures for a three-person
household, in dollars. Instead of measuring income in dollars, we measure it in units of $100, because a $1 increase in
income has a numerically small effect on food expenditure.
Consequently, for the first household, the reported income
is $369 per week with weekly food expenditure of $115.22.
For the 40th household, weekly income is $3340 and weekly
food expenditure is $375.73. The complete data set of
observations is in the data file food.
k
T A B L E 2.1
Food Expenditure and Income Data
Observation
(household)
Food
Expenditure ($)
i
1
2
39
40
Sample mean
Median
Maximum
Minimum
Std. dev.
yi
115.22
135.98
⋮
257.95
375.73
Summary Statistics
283.5735
264.4800
587.6600
109.7100
112.6752
Weekly
Income ($100)
xi
3.69
4.39
29.40
33.40
19.6048
20.0300
33.4000
3.6900
6.8478
k
60
CHAPTER 2
The Simple Linear Regression Model
Remark
In this book, data files are referenced with a descriptive name in italics such as food.
The actual files which are located at the book websites www.wiley.com/college/hill and
www.principlesofeconometrics.com come in various formats and have an extension that
denotes the format, for example, food.dat, food.wf1, food.dta, and so on. The corresponding
data definition file is food.def .
We assume that the expenditure data in Table 2.1 satisfy the assumptions SR1–SR5. That is, we
assume that the regression model yi = β1 + β2 xi + ei describes a population relationship and that
the random error has conditional expected value zero. This implies that the conditional expected
value of household food expenditure is a linear function of income. The conditional variance of y,
which is the same as that of the random error e, is assumed constant, implying that we are equally
uncertain about the relationship between y and x for all observations. Given x the values of y for
different households are assumed uncorrelated with each other.
Given this theoretical model for explaining the sample observations on household food
expenditure, the problem now is how to use the sample information in Table 2.1, specific values
of yi and xi , to estimate the unknown regression parameters β1 and β2 . These parameters represent
the unknown intercept and slope(coefficients
for the food expenditure–income relationship. If we
)
represent the 40 data points as yi , xi , i = 1, … , N = 40, and plot them, we obtain the scatter
diagram in Figure 2.6.
Remark
0
100
200
300
400
500
600
700
It will be our notational convention to use i subscripts for cross-sectional data observations,
with the number of sample observations being N. For time-series data observations, we use
the subscript t and label the total number of observations T. In purely algebraic or generic
situations, we may use one or the other.
y = weekly food expenditure in $
k
0
10
20
x = weekly income in $100
FIGURE 2.6 Data for the food expenditure example.
k
30
40
k
k
2.3 Estimating the Regression Parameters
61
(
)
Our problem is to estimate the location of the mean expenditure line E yi |𝐱 = β1 + β2 xi . We
would expect this line to be somewhere in the middle of all the data points since it represents
population mean, or average, behavior. To estimate β1 and β2 , we could simply draw a freehand
line through the middle of the data and then measure the slope and intercept with a ruler. The
problem with this method is that different people would draw different lines, and the lack of a
formal criterion makes it difficult to assess the accuracy of the method. Another method is to draw
a line from the expenditure at the smallest income level, observation i = 1, to the expenditure
at the largest income level, i = 40. This approach does provide a formal rule. However, it may
not be a very good rule because it ignores information on the exact position of the remaining
38 observations. It would be better if we could devise a rule that uses all the information from all
the data points.
2.3.1
The Least Squares Principle
To estimate β1 and β2 we want a rule, or formula, that tells us how to make use of the sample
observations. Many rules are possible, but the one that we will use is based on the least squares
principle. This principle asserts that to fit a line to the data values we should make the sum of the
squares of the vertical distances from each point to the line as small as possible. The distances are
squared to prevent large positive distances from being canceled by large negative distances. This
rule is arbitrary, but very effective, and is simply one way to describe a line that runs through the
middle of the data. The intercept and slope of this line, the line that best fits the data using the
least squares principle, are b1 and b2 , the least squares estimates of β1 and β2 . The fitted line
itself is then
(2.5)
ŷ i = b1 + b2 xi
k
The vertical distances from each point to the fitted line are the least squares residuals. They are
given by
(2.6)
ê i = yi − ŷ i = yi − b1 − b2 xi
These residuals are depicted in Figure 2.7a.
Now suppose we fit another line, any other line, to the data. Denote the new line as
ŷ ∗i = b∗1 + b∗2 xi
where b∗1 and b∗2 are any other intercept and slope values. The residuals for this line, ê ∗i = yi − ŷ ∗i ,
are shown in Figure 2.7b. The least squares estimates b1 and b2 have the property that the sum of
their squared residuals is less than the sum of squared residuals for any other line. That is, if
SSE =
N
∑
i=1
ê 2i
is the sum of squared least squares residuals from (2.6) and
SSE∗ =
N
∑
i=1
2
ê ∗i =
N (
∑
i=1
yi − ŷ ∗i
)2
is the sum of squared residuals based on any other estimates, then
SSE < SSE∗
no matter how the other line might be drawn through the data. The least squares principle says
that the estimates b1 and b2 of β1 and β2 are the ones to use, since the line using them as intercept
and slope fits the data best.
The problem is to find b1 and b2 in a convenient way. Given the sample observations on y
and x, we want to find values for the unknown parameters β1 and β2 that minimize the “sum of
squares” function
N (
) ∑
)2
(
yi − β1 − β2 xi
S β1 , β2 =
i=1
k
k
k
62
CHAPTER 2
The Simple Linear Regression Model
y
y = b1 + b2 x
y4
e4
y3
y4
e3
y3
y2
e2
y1
e1
y2
y1
x1
x2
x3
x4
x
(a)
y
e3*
y*2
y*1
y* = b*1 + b2* x
e4*
y3
y4*
y*
3
e2*
e*1
k
y = b1 + b2 x
y4
y2
k
y1
x1
x2
x3
x4
x
(b)
FIGURE 2.7 (a) The relationship among y, ê , and the fitted
regression line. (b) The residuals from another
fitted line.
This is a straightforward calculus problem, the details of which are given in Appendix 2A. The
formulas for the least squares estimates of β1 and β2 that give the minimum of the sum of squared
residuals are
The Ordinary Least Squares (OLS) Estimators
∑(
b2 =
where y =
∑
yi ∕N and x =
∑
)(
)
xi − x yi − y
)2
∑(
xi − x
b1 = y − b2 x
(2.7)
(2.8)
xi ∕N are the sample means of the observations on y and x.
We will call the estimators b1 and b2 , given in equations (2.7) and (2.8), the ordinary least
squares estimators. “Ordinary least squares” is abbreviated as OLS. These least squares estimators are called “ordinary,” despite the fact that they are extraordinary, because these estimators
are used day in and day out in many fields of research in a routine way, and to distinguish them
k
k
2.3 Estimating the Regression Parameters
63
from other methods called generalized least squares, and weighted least squares, and two-stage
least squares, all of which are introduced later in this book.
The formula for b2 reveals why we had to assume [SR5] that in the sample xi must take at least
two different values. If xi = 5, for example, for all observations, then b2 in (2.7) is mathematically
undefined and does not exist since its numerator and denominator are zero!
If we plug the sample values yi and xi into (2.7) and (2.8), then we obtain the least squares
estimates of the intercept and slope parameters β1 and β2 . It is interesting, however, and very
important, that the formulas for b1 and b2 are perfectly general and can be used no matter what
the sample values turn out to be. This should ring a bell. When the formulas for b1 and b2 are
taken to be rules that are used whatever the sample data turn out to be, then b1 and b2 are random
variables. When actual sample values are substituted into the formulas, we obtain numbers that
are the observed values of random variables. To distinguish these two cases, we call the rules or
general formulas for b1 and b2 the least squares estimators. We call the numbers obtained when
the formulas are used with a particular sample least squares estimates.
• Least squares estimators are general formulas and are random variables.
• Least squares estimates are numbers that we obtain by applying the general formulas to the
observed data.
The distinction between estimators and estimates is a fundamental concept that is essential to
understand everything in the rest of this book.
E X A M P L E 2.4a
k
Estimates for the Food Expenditure Function
Using the least squares estimators (2.7) and (2.8), we can
obtain the least squares estimates for the intercept and slope
parameters β1 and β2 in the food expenditure example using
the data in Table 2.1. From (2.7), we have
)(
)
∑(
xi − x yi − y
18671.2684
= 10.2096
=
b2 =
)2
∑(
1828.7876
x −x
i
and from (2.8)
b1 = y − b2 x = 283.5735 − (10.2096)(19.6048) = 83.4160
A convenient way to report the values for b1 and b2 is to write
out the estimated or fitted regression line, with the estimates
rounded appropriately:
ŷ i = 83.42 + 10.21xi
This line is graphed in Figure 2.8. The line’s slope is 10.21,
and its intercept, where it crosses the vertical axis, is 83.42.
The least squares fitted line passes through the middle of the
data in a very precise way, since one of the characteristics of
the fitted line based on the least squares parameter estimates
is that it( passes
through the point defined by the sample
)
means, x, y = (19.6048, 283.5735). This follows directly
from rewriting (2.8) as y = b1 + b2 x. Thus, the “point of the
means” is a useful reference value in regression analysis.
Interpreting the Estimates
Once obtained, the least squares estimates are interpreted
in the context of the economic model under consideration.
k
The value b2 = 10.21 is an estimate of β2 . Recall that x,
weekly household income, is measured in $100 units. The
regression slope β2 is the amount by which expected weekly
expenditure on food per household increases when household weekly income increases by $100. Thus, we estimate
that if weekly household income goes up by $100, expected
weekly expenditure on food will increase by approximately
$10.21, holding all else constant. A supermarket executive
with information on likely changes in the income and the
number of households in an area could estimate that it will
sell $10.21 more per typical household per week for every
$100 increase in income. This is a very valuable piece of
information for long-run planning.
Strictly speaking, the intercept estimate b1 = 83.42 is
an estimate of the expected weekly food expenditure for a
household with zero income. In most economic models we
must be very careful when interpreting the estimated intercept. The problem is that we often do not have any data points
near x = 0, something that is true for the food expenditure
data shown in Figure 2.8. If we have no observations in the
region where income is zero, then our estimated relationship
may not be a good approximation to reality in that region.
So, although our estimated model suggests that a household
with zero income is expected to spend $83.42 per week on
food, it might be risky to take this estimate literally. This is
an issue that you should consider in each economic model
that you estimate.
k
k
CHAPTER 2
The Simple Linear Regression Model
500
y
300
400
y = 83.42 + 10.21 x
200
–
y = 283.57
100
y = weekly food expenditure in $
600
700
64
0
–
x = 19.60
0
10
20
x = weekly income in $100
30
40
FIGURE 2.8 The fitted regression.
k
k
Elasticities Income elasticity is a useful way to characterize the responsiveness of consumer
expenditure to changes in income. See Appendix A.2.2 for a discussion of elasticity calculations
in a linear relationship. The elasticity of a variable y with respect to another variable x is
ε=
percentage change in y 100(Δy∕y) Δy x
•
=
=
percentage change in x 100(Δx∕x) Δx y
In the linear economic model given by (2.1), we have shown that
β2 =
ΔE(y|x)
Δx
so the elasticity of mean expenditure with respect to income is
ε=
E X A M P L E 2.4b
ΔE(y|x)
x
x
•
= β2 •
Δx
E(y|x)
E(y|x)
(2.9)
Using the Estimates
To estimate this elasticity we replace β2 by b2 = 10.21. We
must also replace “x” and “E(y|x)” by something, since in
a linear model the elasticity is different on each point on the
regression line. Most commonly,
( ) the elasticity is calculated at
the “point of the means” x, y = (19.60, 283.57) because it is
a representative point on the regression line. If we calculate
the income elasticity at the point of the means, we obtain
19.60
x
= 0.71
ε̂ = b2 = 10.21 ×
283.57
y
k
k
2.3 Estimating the Regression Parameters
This estimated income elasticity takes its usual interpretation.
We estimate that a 1% increase in weekly household income
will lead to a 0.71% increase in expected weekly household
expenditure
( )on food, when x and y take their sample mean
values, x, y = (19.60, 283.57). Since the estimated income
elasticity is less than one, we would classify food as a “necessity” rather than a “luxury,” which is consistent with what
we would expect for an average household.
Prediction
The estimated equation can also be used for prediction or
forecasting purposes. Suppose that we wanted to predict average weekly food expenditure for a household with a weekly
income of $2000. This prediction is carried out by substituting x = 20 into our estimated equation to obtain
ŷ i = 83.42 + 10.21xi = 83.42 + 10.21(20) = 287.61
We predict that a household with a weekly income of $2000
will on average spend $287.61 per week on food.
Computer Output
65
different and uses different terminology to describe the output. Despite these differences, the various outputs provide the
same basic information, which you should be able to locate
and interpret. The matter is complicated somewhat by the fact
that the packages also report various numbers whose meaning
you may not know. For example, using the food expenditure
data, the output from the software package EViews is shown
in Figure 2.9.
In the EViews output, the parameter estimates are in
the “Coefficient” column, with names “C,” for constant term
(the estimate b1 ) and INCOME (the estimate b2 ). Software
programs typically name the estimates with the name of the
variable as assigned in the computer program (we named
our variable INCOME) and an abbreviation for “constant.”
The estimates that we report in the text are rounded to
two significant digits. The other numbers that you can
∑ 2
recognize at this time are SSE = ê i = 304505.2, which
is called “Sum squared resid,” and the sample mean of y,
∑
y = yi ∕N = 283.5735, which is called “Mean dependent var.”
We leave discussion of the rest of the output until later.
Many different software packages can compute least squares
estimates. Every software package’s regression output looks
k
k
Dependent Variable: FOOD_EXP
Method: Least Squares
Sample: 1 40
Included observations: 40
C
INCOME
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
F-statistic
Prob(F-statistic)
Coefficient
Std. Error
t-Statistic
Prob.
83.41600
10.20964
43.41016
2.093264
1.921578
4.877381
0.0622
0.0000
0.385002
0.368818
89.51700
304505.2
−235.5088
23.78884
0.000019
Mean dependent var
S.D. dependent var
Akaike info criterion
Schwarz criterion
Hannan-Quinn criter
Durbin-Watson stat
283.5735
112.6752
11.87544
11.95988
11.90597
1.893880
FIGURE 2.9 EViews regression output.
2.3.2
Other Economic Models
We have used the household expenditure on food versus income relationship as an example to
introduce the ideas of simple regression. The simple regression model can be applied to estimate
k
k
66
CHAPTER 2
The Simple Linear Regression Model
the parameters of many relationships in economics, business, and the social sciences. The applications of regression analysis are fascinating and useful. For example,
• If the hourly wage rate of electricians rises by 5%, how much will new house prices increase?
• If the cigarette tax increases by $1, how much additional revenue will be generated in the
state of Louisiana?
• If the central banking authority raises interest rates by one-half a percentage point, how much
will consumer borrowing fall within six months? How much will it fall within one year? What
will happen to the unemployment rate in the months following the increase?
• If we increase funding on preschool education programs in 2018, what will be the effect on
high school graduation rates in 2033? What will be the effect on the crime rate by juveniles
in 2028 and subsequent years?
The range of applications spans economics and finance, as well as most disciplines in the social
and physical sciences. Any time you ask how much a change in one variable will affect another
variable, regression analysis is a potential tool.
Similarly, any time you wish to predict the value of one variable given the value of another
then least squares regression is a tool to consider.
2.4
k
Assessing the Least Squares Estimators
Using the food expenditure data, we have estimated the parameters of the regression model
yi = β1 + β2 xi + ei using the least squares formulas in (2.7) and (2.8). We obtained the least
squares estimates b1 = 83.42 and b2 = 10.21. It is natural, but, as we shall argue, misguided,
to ask the question “How good are these estimates?” This question is not answerable. We will
never know the true values of the population parameters β1 or β2 , so we cannot say how close
b1 = 83.42 and b2 = 10.21 are to the true values. The least squares estimates are numbers that
may or may not be close to the true parameter values, and we will never know.
Rather than asking about the quality of the estimates we will take a step back and examine
the quality of the least squares estimation procedure. The motivation for this approach is this: if
we were to collect another sample of data, by choosing another set of 40 households to survey, we
would have obtained different estimates b1 and b2 , even if we had carefully selected households
with the same incomes as in the initial sample. This sampling variation is unavoidable. Different
samples will yield different estimates because household food expenditures, yi , i = 1, … , 40, are
random variables. Their values are not known until the sample is collected. Consequently, when
viewed as an estimation procedure, b1 and b2 are also random variables, because their values
depend on the random variable y. In this context, we call b1 and b2 the least squares estimators.
We can investigate the properties of the estimators b1 and b2 , which are called their sampling
properties, and deal with the following important questions:
1. If the least squares estimators b1 and b2 are random variables, then what are their expected
values, variances, covariances, and probability distributions?
2. The least squares principle is only one way of using the data to obtain estimates of β1 and β2 .
How do the least squares estimators compare with other procedures that might be used, and
how can we compare alternative estimators? For example, is there another estimator that has
a higher probability of producing an estimate that is close to β2 ?
We examine these questions in two steps to make things easier. In the first step, we investigate the
properties of the least squares estimators conditional on the values of the explanatory variable in
the sample. That is, conditional on x. Making the analysis conditional on x is equivalent to saying
that, when we consider all possible samples, the household income values in the sample stay the
k
k
k
2.4 Assessing the Least Squares Estimators
67
same from one sample to the next; only the random errors and food expenditure values change.
This assumption is clearly not realistic but it simplifies the analysis. By conditioning on x, we are
holding it constant, or fixed, meaning that we can treat the x-values as “not random.”
In the second step,
in Section 2.10, we return to the random sampling assumption
)
( considered
and recognize that yi , xi data pairs are random, and randomly selecting households from a population leads to food expenditures and incomes that are random. However, even in this case and
treating x as random, we will discover that most of our conclusions that treated x as nonrandom
remain the same.
In either case, whether we make the analysis conditional on x or make the analysis general by treating x as random, the answers to the questions above depend critically on whether
the assumptions SR1–SR5 are satisfied. In later chapters, we will discuss how to check whether the
assumptions we make hold in a specific application, and what we might do if one or more assumptions are shown not to hold.
Remark
We will summarize the properties of the least squares estimators in the next several sections.
“Proofs” of important results appear in the appendices to this chapter. In many ways, it is
good to see these concepts in the context of a simpler problem before tackling them in the
regression model. Appendix C covers the topics in this chapter, and the next, in the familiar
and algebraically easier problem of estimating the mean of a population.
k
The Estimator b2
Formulas (2.7) and (2.8) are used to compute the least squares estimates b1 and b2 . However,
they are not well suited for examining theoretical properties of the estimators. In this section, we
rewrite the formula for b2 to facilitate its analysis. In (2.7), b2 is given by
)(
)
∑(
xi − x yi − y
b2 =
)2
∑(
xi − x
2.4.1
This is called the deviation from the mean form of the estimator because the data have their
sample means subtracted. Using assumption SR1 and a bit of algebra (Appendix 2C), we can
write b2 as a linear estimator,
N
∑
(2.10)
b2 = wi yi
i=1
where
x −x
wi = ∑( i
)2
xi − x
(2.11)
The term wi depends only on x. Because we are conditioning our analysis on x, the term wi is
treated as if it is a constant. We remind you that conditioning on x is equivalent to treating x as
given, as in a controlled, repeatable experiment.
Any estimator that is a weighted average of yi ’s, as in (2.10), is called a linear estimator.
This is an important classification that we will speak more of later. Then, with yet more algebra
(Appendix 2D), we can express b2 in a theoretically convenient way,
∑
b2 = β2 + wi ei
(2.12)
where ei is the random error in the linear regression model yi = β1 + β2 xi + ei . This formula is
not useful for computations, because it depends on β2 , which we do not know, and on the ei ’s,
k
k
k
68
CHAPTER 2
The Simple Linear Regression Model
which are unobservable. However, for understanding the sampling properties of the least squares
estimator, (2.12) is very useful.
2.4.2
The Expected Values of b1 and b2
The OLS estimator b2 is a random variable since its value is unknown
until
) a sample is collected.
(
What we will show is that if our model assumptions hold, then E b2 |𝐱 = β2 ; that is, given x the
of
expected value of b2 is equal to the true parameter β2 . When the expected value of any(estimator
)
a parameter equals the true parameter value, then that estimator is unbiased. Since E b2 |𝐱 = β2 ,
the least squares estimator b2 given x is an unbiased estimator of β2 . In Section
( ) 2.10, we will show
that the least squares estimator b2 is unconditionally unbiased also, E b2 = β2 . The intuitive
meaning of unbiasedness comes from the sampling interpretation of mathematical expectation.
Recognize that one sample of size N is just one of many samples that we could have been selected.
If the formula for b2 is used to estimate β2 in each of those possible samples, then, if our assumptions are valid, the average value of the estimates b2 obtained from all possible samples will
be β2 .
We will show that this result is true so that we can illustrate the part played by the assumptions of the linear regression model. In (2.12), what parts are random? The parameter β2 is not
random. It is a population parameter we are trying to estimate. Conditional on x we can treat xi
as if it is not random. Then, conditional on x, wi is not random either, as it depends only on the
values of xi . The only random factors in (2.12) are the random error terms ei . We can find the conditional expected value of b2 using the fact that the expected value of a sum is the sum of the
expected values:
k
)
(
)
(
)
(
∑
E b2 |x = E β2 + wi ei |x = E β2 + w1 e1 + w2 e2 + · · · + wN eN |x
(
)
(
)
(
)
( )
= E β2 + E w1 e1 |x + E w2 e2 |x + · · · + E wN eN |x
)
∑ (
= β2 + E wi ei |x
(
)
∑
= β2 + wi E ei |x = β2
k
(2.13)
The rules of expected values are fully discussed in the Probability Primer,
P.5,
)
( and)
( Section
Appendix B.1.1. In the last line of (2.13), we use two assumptions. First, E wi ei |𝐱 = wi E ei |𝐱
because conditional on x the terms wi are not random, and constants can
( be)factored out of expected
values.
Second,
we
have
relied
on
the
assumption
that
E
)
(
) ei |𝐱 = 0. Actually, if
(
E ei |𝐱 = c, where c is any constant value, such as 3, then E b2 |𝐱 = β2 . Given x, the OLS
estimator
) b2 is an unbiased estimator of the regression parameter β2 . On the other hand, if
(
E ei |𝐱 ≠ 0 and it depends on x in
) way, then b2 is a biased estimator of β2 . One leading
( some
case in which the assumption E ei |𝐱 = 0 fails is due to omitted variables. Recall that ei
contains everything else affecting yi other than xi . If we have omitted
) anything that
( is )important
(
and that is correlated with x then we would expect that E ei |𝐱 ≠ 0 and E b2 |𝐱 ≠ β2 . In
Chapter 6 we discuss this omitted variables bias. Here we have shown that conditional on x,
and under
( )SR1–SR5, the least squares estimator is linear and unbiased. In Section 2.10, we show
that E b2 = β2 without conditioning on x.
The unbiasedness of the estimator b2 is an important sampling property. On average, over
all possible samples from the population, the least squares estimator is “correct,” on average, and
this is one desirable property of an estimator. This statistical property by itself does not mean that
b2 is a good estimator of β2 , but it is part of the story. The unbiasedness property is related to what
happens in all possible samples of data from the same population. The fact that b2 is unbiased
does not imply anything about what might happen in just one sample. An individual estimate
(a number) b2 may be near to, or far from, β2 . Since β2 is never known we will never know, given
k
k
2.4 Assessing the Least Squares Estimators
69
one sample, whether our estimate is “close” to β2 or not. Thus, the estimate b2 = 10.21 may be
close to β2 or not.
(
)
The least squares estimator b1 of β1 is also an unbiased estimator, and E b1 |𝐱 = β1 if the
model assumptions hold.
2.4.3
Sampling Variation
To illustrate how the concept of unbiased estimation relates to sampling variation, we present
in Table 2.2 least squares estimates of the food expenditure model from 10 hypothetical random
samples (data file table2_2) of size N = 40 from the same population with the same incomes as
the households given in Table 2.1. Note the variability of the least squares parameter estimates
from sample to sample. This sampling variation is due to the fact that we obtain 40 different
households in each sample, and their weekly food expenditure varies randomly.
The property of unbiasedness is about the average values of b1 and b2 if used in all possible samples of the same size drawn from the same population. The average value of b1 in these
10 samples is b1 = 96.11. The average value of b2 is b2 = 8.70. If we took the averages of estimates from more samples, these averages would approach the true parameter values β1 and β2 .
Unbiasedness does not say that an estimate from any one sample is close to the true parameter
value, and thus we cannot say that an estimate is unbiased. We can say that the least squares
estimation procedure (or the least squares estimator) is unbiased.
2.4.4
k
The Variances and Covariance of b1 and b2
Table 2.2 shows that the least squares estimates of β1 and β2 vary from sample to sample. Understanding this variability is a key to assessing the reliability and sampling precision of an estimator.
We now obtain the variances and covariance of the estimators b1 and b2 . Before presenting the
expressions for the variances and covariance, let us consider why they are important to know. The
variance of the random variable b2 is the average of the squared distances
) between the possible
(
values of the random variable and its mean, which we now know is E b2 |𝐱 = β2 . The conditional
variance of b2 is defined as
{[
(
)
(
)]2 | }
var b2 |x = E b2 − E b2 |x |x
|
T A B L E 2.2
Estimates from 10 Hypothetical Samples
Sample
b1
b2
1
2
3
4
5
6
7
8
9
10
93.64
91.62
126.76
55.98
87.26
122.55
91.95
72.48
90.34
128.55
8.24
8.90
6.59
11.23
9.14
6.80
9.84
10.50
8.75
6.99
k
k
k
70
CHAPTER 2
The Simple Linear Regression Model
f2(b2∣x)
f1(b2∣x)
β2
FIGURE 2.10 Two possible probability density
functions for b2 .
k
It measures the spread of the probability( distribution
( of b)2 . In Figure 2.10 are graphs of two
)
possible probability distributions of b2 , f1 b2 |𝐱 and f2 b2 |𝐱 , that have the same mean value but
different variances.
(
(
)
)
The pdf f2 b2 |𝐱 has a smaller variance than f1 b2 |𝐱 . (Given) a choice, we (are interested
in
)
b
b
have
the
pdf
f
|𝐱
,
rather
than
f
|𝐱
.
With
the
estimator precision
and
would
prefer
that
b
2
2 2
1 2
(
)
probability
is
more
concentrated
around
the
true
parameter
value β2 givdistribution f2 b2(|𝐱 , the
)
ing, relative to f1 b2 |𝐱 , a higher probability of getting an estimate that is close to β2 . Remember,
getting an estimate close to β2 is a primary objective of regression analysis.
The variance of an estimator measures the precision of the estimator in the sense that it tells
us how much the estimates can vary from sample to sample. Consequently, we often refer to
the sampling variance or sampling precision of an estimator. The smaller the variance of an
estimator is, the greater the sampling precision of that estimator. One estimator is more precise
than another estimator if its sampling variance is less than that of the other estimator.
We will now present and discuss the conditional variances and covariance of b1 and b2 .
Appendix 2E contains the derivation of the variance of the least squares estimator b2 . If the
regression model assumptions SR1–SR5 are correct (assumption SR6 is not required), then the
variances and covariance of b1 and b2 are
[
]
∑ 2
xi
(
)
2
var b1 |x = σ
(2.14)
)2
∑(
N xi − x
(
)
σ2
var b2 |x = ∑(
)2
xi − x
[
]
(
)
−x
2
cov b1 , b2 |x = σ ∑(
)2
xi − x
(2.15)
(2.16)
At the beginning of this section we said that for unbiased estimators, smaller variances are better than larger variances. Let us consider the factors that affect the variances and covariance
in (2.14)–(2.16).
1. The variance of the random error term, σ2 , appears in each of the expressions. It reflects the
dispersion of the values y about their expected value E(y|x). The greater the variance σ2 , the
greater is that dispersion, and the greater is the uncertainty about where the values of y fall
relative to their conditional mean E(y|x). When σ2 is larger, the information we have about β1
and β2 is less precise. In Figure 2.5, the variance is reflected in the spread of the probability
distributions f (y|x). The larger the variance term σ2 , the greater is the uncertainty in the
statistical model, and the larger the variances and covariance of the least squares estimators.
k
k
k
2.4 Assessing the Least Squares Estimators
yi
yi
y
y
x
(a)
xi
71
yi = b1 + b2 xi
x
xi
(b)
FIGURE 2.11 The influence of variation in the explanatory variable x on precision of
estimation: (a) low x variation, low precision: (b) high x variation, high
precision.
k
)2
∑(
2. The sum of squares of the values of x about their sample mean,
xi − x , appears in each
of the variances and in the covariance. This expression measures how spread out about their
mean are the sample values of the independent or explanatory variable x. The more they are
spread out, the larger the sum of squares. The less they are spread out, the smaller the sum of
squares. You may recognize this sum of squares as the numerator of the sample variance of
)2
∑(
the x-values. See Appendix C.4. The larger the sum of squares,
xi − x , the smaller the
conditional variances of the least squares estimators and the more precisely we can estimate
the unknown parameters. The intuition behind this is demonstrated in Figure 2.11. Panel (b)
is a data scatter in which the values of x are widely spread out along the x-axis. In panel (a),
the data are “bunched.” Which data scatter would you prefer given the task of fitting a line
by hand? Pretty clearly, the data in panel (b) do a better job of determining where the least
squares line must fall, because they are more spread out along the x-axis.
3. The larger the sample size N, the smaller the variances and covariance of the least squares
estimators; it is better to have more sample data than less. The sample size N appears in
each of the variances and covariance
because each of the sums consists
(
)
)2of N terms. Also,
∑(
N appears explicitly in var b1 |𝐱 . The sum of squares term
xi − x gets larger as N
increases because each of the terms in the sum is positive or zero (being zero if x happens
to equal
for an observation). Consequently, as N gets larger, both
(
)its sample
( mean value
)
var b2 |𝐱 and cov b1 , b2 |𝐱 get smaller, since the sum (of squares
appears in their denomina)
tor. The sums in the numerator and denominator of var b1 |𝐱 both get larger as N gets larger
and(offset) one another, leaving the N in the denominator as the dominant term, ensuring that
var b1 |𝐱 also gets smaller as N gets larger.
(
)
∑
4. The term xi2 appears in var b1 |𝐱 . The larger this term is, the larger the variance of the least
squares estimator b1 . Why is this so? Recall that the intercept parameter β1 is the expected
value of y given that x = 0. The farther our data are from x = 0, the more difficult it is to
interpret β1 , as in the food expenditure example, and the more difficult it is to accurately
∑
estimate β1 . The term xi2 measures the squared distance of the data from the origin,
x) = 0.
(
∑
If the values of x are near zero, then xi2 will be small, and this will reduce var b1 |𝐱 . But
∑
if the values (of x) are large in magnitude, either positive or negative, the term xi2 will be
large and var b1 will be larger, other things being equal.
(
)
5. The sample mean of the x-values appears in cov b1 , b2 |𝐱 . The absolute magnitude of the
covariance increases with an increase in magnitude of the sample mean x, and the covariance
has a sign opposite to that of x. The reasoning here can be seen from Figure 2.11. In panel (b)
k
k
k
72
CHAPTER 2
The Simple Linear Regression Model
the least squares fitted line must pass through the point of the means. Given a fitted line
through the data, imagine the effect of increasing the estimated slope b2 . Since the line must
pass through the point of the means, the effect must be to lower the point where the line hits
the vertical axis, implying a reduced intercept estimate b1 . Thus, when the sample mean is
positive, as shown in Figure 2.11, there is a negative covariance between the least squares
estimators of the slope and intercept.
2.5
The Gauss–Markov Theorem
What can we say about the least squares estimators b1 and b2 so far?
• The estimators are perfectly general. Formulas (2.7) and (2.8) can be used to estimate the
unknown parameters β1 and β2 in the simple linear regression model, no matter what the
data turn out to be. Consequently, viewed in this way, the least squares estimators b1 and b2
are random variables.
• The least squares estimators are linear estimators, as defined in (2.10). Both b1 and b2 can
be written as weighted averages of the yi values.
• If assumptions SR1–SR5
(
) hold, then the
( least
) squares estimators are conditionally unbiased.
This means that E b1 |𝐱 = β1 and E b2 |𝐱 = β2 .
• Given x we have expressions for the variances of b1 and b2 and their covariance. Furthermore,
we have argued that for any unbiased estimator, having a smaller variance is better, as this
implies we have a higher chance of obtaining an estimate close to the true parameter value.
k
Now we will state and discuss the famous Gauss–Markov theorem, which is proven in
Appendix 2F.
Gauss–Markov Theorem:
Given x and under the assumptions SR1–SR5 of the linear regression model, the estimators
b1 and b2 have the smallest variance of all linear and unbiased estimators of β1 and β2 . They
are the best linear unbiased estimators (BLUE) of β1 and β2 .
Let us clarify what the Gauss–Markov theorem does, and does not, say.
1. The estimators b1 and b2 are “best” when compared to similar estimators, those that are
linear and unbiased. The theorem does not say that b1 and b2 are the best of all possible
estimators.
2. The estimators b1 and b2 are best within their class because they have the minimum variance.
When comparing two linear and unbiased estimators, we always want to use the one with
the smaller variance, since that estimation rule gives us the higher probability of obtaining
an estimate that is close to the true parameter value.
3. In order for the Gauss–Markov theorem to hold, assumptions SR1–SR5 must be true. If any
of these assumptions are not true, then b1 and b2 are not the best linear unbiased estimators
of β1 and β2 .
4. The Gauss–Markov theorem does not depend on the assumption of normality (assumption
SR6).
5. In the simple linear regression model, if we want to use a linear and unbiased estimator, then
we have to do no more searching. The estimators b1 and b2 are the ones to use. This explains
k
k