Gra6036- Multivartate Statistics with
Econometrics (Psychometrics)
Distributions
Estimators
Ulf H. Olsson
Professor of Statistics
Two Courses in Multivariate Statistics
• Gra 6020 Multivariate Statistics
• Applied with focus on data analysis
• Non-technical
• Gra 6036 Multivariate Statistics with Econometrics
• Technical – focus on both application and understanding “basics”
• Mathematical notation and Matrix Algebra
Ulf H. Olsson
Course outline Gra 6036
• Basic Theoretical (Multivariate) Statistics mixed with econometric
(psychometric) theory
• Matrix Algebra
• Distribution theory (Asymptotical)
• Application with focus on regression type models
•
•
•
•
Logit Regression
Analyzing panel data
Factor Models
Simultaneous Equation Systems and SEM
• Using statistics as a good researcher should
• Research oriented
Ulf H. Olsson
Evaluation
• Term paper (up to three students) 75%
• 1 – 2 weeks
• Multipple choice exam (individual) 25%
• 2 – 3 hours
Ulf H. Olsson
Teaching and communication
• Lecturer 2 – 3 weeks: 3 hours per week (UHO)
• Theory and demonstrations
• Exercises 1 week: 2 hours (DK)
• Assignments and Software applications (SPSS/EVIEWS/LISREL)
• Blackboard and Homepage
• Assistance: David Kreiberg (Dep.of economics)
Ulf H. Olsson
Week
hours
Read
2
Basic Multivariate Statistical
Analysis. Asymptotic Theory
3
Lecture notes
3
Logit and Probit Regression
3
Compendium: Logistic
Regression
4
Logit and Probit Regression
3
Compendium: Logistic
Regression
5
Exercises
2
6
Panel Models
3
Book chapter (14): Analyzing
Panel Data: Fixed – and
Random-Effects Models
7
Panel Models
3
Book chapter (14): Analyzing
Panel Data: Fixed – and
Random-Effects Models
8
Exercises
2
Ulf H. Olsson
9
Factor Analysis/ Exploratory
Factor Analysis
3
Structural Equation Modeling.
David Kaplan, 2000
10
Confirmatory Factor Analysis
3
Structural Equation Modeling.
David Kaplan, 2000
11
Confirmatory Factor Analysis
3
Structural Equation Modeling.
David Kaplan, 2000
12
Exercises
2
13
Simultaneous Equations
3
Structural Equation Modeling.
David Kaplan, 2000
15
Structural Equations Models
3
Structural Equation Modeling.
David Kaplan, 2000
16
Structural Equations Models
3
Structural Equation Modeling.
David Kaplan, 2000
17
Exercises
2
Ulf H. Olsson
Any Questions ?
Ulf H. Olsson
Univariate Normal Distribution
•
•
Ulf H. Olsson
Cumulative Normal Distribution
Ulf H. Olsson
Normal density functions
1/ 2 ( x   )2 /  2 
1
( x |  ,  ) 
e
 2
2
1
(u ) 
e
2
u N (0,1)
u2
2
,u 
x

Ulf H. Olsson
The Chi-squared distributions
N (0,1) then z  u
If u
2
 (1)
2
If z1 , z2 ,....zn are n independent  2 (1)  var iables
n
then  zi
 2 ( n)
i 1
E (  (n))  n
2
Var (  2 (n))  2n
Ulf H. Olsson
The Chi-squared distributions
If u1 , u2 ,....un are n independent N (0,  2 )  var iables
n
then  (u 2i /  i )
 2 ( n)
i 1
If u1 , u2 ,....un are n independent N (  ,  2 )  var iables
n
then  (u 2i /  i )
 2 (n,  )
i 1
E (  (n,  ))  n  
2
Var (  2 (n,  ))  2n  4
Ulf H. Olsson
Bivariate normal distribution
Ulf H. Olsson
Standard Normal density functions
1
 (u ) 
e
2
(u, v) 
u 2
2

1
2 (1   )
 (u1.u2 ,..., un ) 
2
e
1
( u 2  2 uv v 2 )
2
2 (1  )
1
(2 )
n/2

1/ 2
e
1
(  u ' 1u )
2
;
u'  (u1 , u2 ,....un )
Ulf H. Olsson
Estimator
• An estimator is a rule or strategy for using the data to estimate the
parameter. It is defined before the data are drawn.
• The search for good estimators constitutes much of econometrics
(psychometrics)
• Finite/Small sample properties
• Large sample or asymptotic properties
• An estimator is a function of the observations, an estimator is thus
a sample statistic- since the x’s are random so is the estimator
Ulf H. Olsson
Small sample properties

Unbiased E ( )  

Biased

E ( )  


1 is more efficient : Var (1 )  Var ( 2 )
Ulf H. Olsson
Large-sample properties

Consistency : lim n P(     n     )  1
for all  .

Asymptotic unbiased : lim n E ( n )  




Var ( 1 ) 

 1 is Asymptotic Efficent : lim n
1

Var ( ) 
2 

for all 
Ulf H. Olsson
Introduction to the ML-estimator
Let  be the data matrix
  ( x1 , x2 ,......, xk ); where xi are vectors
The Likelihood function is as a function of the unknown
parameter vector  :
k
f ( x1 , x2 ,......, xk , )   f ( xi , )  L( | X )
i 1
Ulf H. Olsson
Introduction to the ML-estimator
• The value of the parameters that maximizes this function are the
maximum likelihood estimates
• Since the logarithm is a monotonic function, the values that
maximizes L are the same as those that minimizes ln L
The necessary conditions for max imiz in g L ( ) is
 ln L( )
0


We denote the ML  estimator  ML



L( )  L is the Likelihood function evaluated at 
Ulf H. Olsson
Introduction to the ML-estimator
• In sampling from a normal (univariate) distribution with
mean  and variance 2 it is easy to verify that:


 ML   ML
2
 ML
1 n
  xi and
n i

1 n
  ( xi  x) 2
n i
•MLs are consistent but not necessarily unbiased
Ulf H. Olsson
Two asymptotically Equivalent Tests
Likelihood ration test
Wald test
The Likelihood Ratio Test
Let  be a vector of parameters to be estimated .


Two ML  estimates  U and  R

The likelihood ratio is  
LR

LU
The l arg e sample distribution of  2 ln 
is  2 (d )
Ulf H. Olsson
The Wald Test
If x
N   ,  , then ( x   )'  ( x   ) is  (d )
1
2
H 0 : c( )  q,
then under H 0


W  (c( )  q) 'U (c( )  q)
1
is  (d )
2
Ulf H. Olsson
Example of the Wald test
• Consider a simpel regression model
y   x 
H0 :    0 ,

we know

|  0 |

s ( )
 z or (t ) ;


W  (   0 ) 'Var (   0 ) (   0 )  z
1
2
is  2 (1)
Ulf H. Olsson
Likelihood- and Wald. Example from
Simultaneous Equations Systems
•
•
•
•
N=218; # Vars.=9; # free parameters = 21;
Df = 24;
Likelihood based chi-square = 164.48
Wald Based chi-square = 157.96
Ulf H. Olsson
Assessing Normality and Multivariate
Normality (Continuous variables)
Skewness
Kurtosis
Mardias test
Bivariate normal distribution
Ulf H. Olsson
Positive vs. Negative Skewness
Exhibit 1
These graphs illustrate the notion of skewness. Both
PDFs have the same expectation and variance. The one
on the left is positively skewed. The one on the right is
negatively skewed.
Ulf H. Olsson
Low vs. High Kurtosis
Exhibit 1
These graphs illustrate the notion of kurtosis. The PDF on
the right has higher kurtosis than the PDF on the left. It is
more peaked at the center, and it has fatter tails.
Ulf H. Olsson
J-te order Moments
• Skewness
• Kurtosis
Population central moments
 j  E ( X   ) j , j  1;   E ( X );
X is continuous and random
3
Skewness :  1 
(  2 )3/ 2
4
Kurtosis :  2  2  3
2
Ulf H. Olsson
Skewness and Kurtosis
 1 and  2 can be estimated from a sample.
We can test H 0 : Skewnes  0 and H 0 : Kurtosis  0
by z and  2  tests
We can even estimate and test for multi var iate kurtosis.
Multi var iate kurtosis :  2, p  E (( X   ) '  1 ( X   )) 2
Ulf H. Olsson
To Next week
• Down load LISREL 8.8. Adr.: http://www.ssicentral.com/
• Read: David Kaplan: Ch.3 (Factor Analysis)
• Read: Lecture Notes
Ulf H. Olsson