Multivariate Regression
British Butter Price and Quantities from
Denmark and New Zealand 1930-1936
I. Hilfer (1938). “Differential Effect in the Butter Market,” Econometrica, Vol. 6,
#3, pp.270-284
Data
• Time Horizon: Monthly 3/1930-10/1936
• Response Variables:
– Y1 ≡ Price of Danish Butter (Inflation Adjusted)
– Y2 ≡ Price of New Zealand Butter (Inflation Adjusted)
• Predictor Variables:
– X1 ≡ Danish Imports
– X2 ≡ New Zealand/Australia Imports
– X3 ≡ All Other Imports
Danish and NZ Butter Prices in Britain (3/30-10/36)
750.0
700.0
650.0
Adjusted Price
600.0
550.0
Denmark
500.0
NZ
450.0
400.0
350.0
300.0
250.0
1
5
9
13
17
21
25
29
33
37
41
month
45
49
53
57
61
65
69
73
77
81
Danish Prices vs NZ Imports
Danish Prices vs Danish Imports
750
750
700
700
650
650
600
Danish Prices
Danish Price
600
550
500
450
550
500
450
400
400
350
350
300
300
250
250
125
150
175
200
225
250
275
0
300
100
200
300
500
600
700
NZ Prices vs NZ Imports
NZ Prices vs Danish Imports
750
750
700
700
650
650
600
600
550
550
NZ Prices
NZ Prices
400
NZ Imports
Danish Imports
500
500
450
450
400
400
350
350
300
300
250
250
125
150
175
200
225
Danish Imports
250
275
300
0
100
200
300
400
NZ Imports
500
600
700
Multivariate Regression Model
p Responses
k Predictors
n observations
Y  Xβ  E
where :
Y11  Y1 p   Y1' 

  
Y      
Yn1  Ynp  Yn' 

  
1 X 11  X 1k 
X      
1 X n1  X nk 
 e11  e1 p   e1' 

  
E     
en1  enp  e'n 

  
  01

11
β
 

  k1
 0 p 
 1 p  β'0 

   β1 

  kp 
  ei1     12   1 p 
  

V (e i )  V             Σ
    
2 
e



ip
1
p
p

  
Least Squares Estimates
^
B  ( X' X) 1 X' Y
 S12  S1 p 
^
^ '
^

1

 


 Y  X B   Y  X B  
Σ        
n  (k  1) 


 S1 p  S p2  






1
1
1
 Y' (I  X( X' X) X)Y  
Y' (I  PX )Y 
 
 n  (k  1) 
 n  (k  1) 




y

y
 ij

ij 

where : S 2j  i 1 
n  (k  1)
n
^
2
^
^



y

y
y

y
 ij

ij  ik
ik 


S jk  i 1 
n  (k  1)
n
Note: This assumes independence across months
Butter Price Example
•
•
•
•
p=2 Response Variables (Danish, NZ Prices)
k=3 Predictors (Danish, NZ, Other Imports)
n=80 Months of Data
First and last 4 months (x0 is used for intercept term):
Month(t)
1
2
3
4

77
78
79
80
Y1(t)
678.5
593.2
570.9
599.3

487.1
496.9
483.6
474
Y2(t)
590
546.6
558.2
577.9

448.2
465.8
417.8
385.4
x0(t)
1
1
1
1

1
1
1
1
x1(t)
181
175
187
260

206
188
190
196
x2(t)
197
194
129
61

170
245
299
324
x3(t)
125
125
189
279

445
352
380
312
Butter Price Example
X'X
80
16066
26690
15647
INV(X'X)
1.27431
-0.004153
-0.001081
-0.000344
Y'Y
20967695
17594889
16066
3319752
5139280
3293189
26690
5139280
10274494
4551068
15647
3293189
4551068
3909851
-0.004153183 -0.00108058 -0.00034378
1.84557E-05 2.2308E-06 -1.5207E-06
2.2308E-06
1.456E-06 7.5069E-07
-1.5207E-06 7.5069E-07 2.0386E-06
Y'PY
17594889.22 20674850.4 17342086.8
14935051.93 17342086.8 14658806.1
X'Y
40444.9
8059701
13362790
7697519
B-Hat
980.1346
-1.12371
-0.48986
-0.43702
Sigma-Hat
3853.213
3326.348
33857.5
6812621
10794292
6635274
905.7373
-0.89519
-0.69075
-0.36961
3326.348
3634.813
^
P D (t )  980.13  1.12 I D (t )  0.49 I NZ (t )  0.44 I O (t )
^
P NZ (t )  904.74  0.90 I D (t )  0.69 I NZ (t )  0.37 I O (t )
Testing Hypotheses Regarding 
• Many times we have theories to be tested regarding
regression coefficients
• The most basic is that none of the predictors are
related to any of the responses
• Others may be that the regression coefficients for
one or more predictor(s) is same for two or more
responses
• Others may be that the effects of two or more
predictors are the same for one or more response(s)
• Tests can be written in the form of H0: LM = d for
specific matrices L,M,d
Matrix Set up for General Linear Tests
H 0 : Lβ M  d  (Lβ  cj)M  0 (c  0 if d  0)
where L is a matrix for predictors and M is for responses,
c is column vec tor of constants, and j is a row vector of 1s.
Step 1 : Set up the hypothesis (H ) matrix :
'


^
1 



1
H  M '  L β  cj  L( X' X) L'  L β  cj M




Step 2 : Set up the error SSCP (E) matrix :
^
^
^

'
E  M '  Y' Y  β X' X β M


Step 3 : Set up (at least one of) 4 test statistics
Step 4 : Convert te st statistic( s) to approximat e F - statistics
Step 5 : Compare statistics with appropriat e F critical values
Three Statistics Based on H and E
Common Elements among Statistics :
q1  rank (H  E) q2  rank L( X' X) 1 L'
  n  (k  1) s  min( q1 , q2 )
m1  0.5 q1  q2  1 m2  0.5(  q1  1)
Wilks' Lambda :  
H
HE
where : r    (q1  q2  1) 2
 1  1 t
 FW   1 t
 
 rt  2u  
 ~ Fq1q2 ,rt  2u

 q1q2 
 q 2q 2  4
 21 2 2
u  (q1q2  2) 4 t   q  q  5
1
2
1

Pillai' s Trace : V  trace H(H  E)
1

 V
 FP  
 s V
Hotelling - Lawley' s Trace : U  trace E 1 H  
if q12  q22  5  0
otherwise
 2m2  s  1  
 ~ Fs ( 2 m1  s 1), s ( 2 m2  s 1)

 2m1  s  1 
 U  2(m2 s  1)  
 ~ Fs ( 2 m1  s 1), 2 ( m2 s 1)
Case 1 : m2  0 : FH   


s
s
2
m

s

1
 
1

   q1q2  2   
 4  



 U 
 b 1   
Case 2 : m2  0 : FH    
 ~ Fq1q2 , 4q1q2  2  b 1
q1q2
 c 




  q1q2  2  
2   b  1  

q1  2m2 q2  2m2 


where : b 
c 
22m2  1m2  1
2m2
Testing Relation Between Price and Quality (I)
  01  02 

  '

β0
11
12


Y  Xβ  Ε
β

  21  22  β1 




32 
 31
 11 12  0 0
0 1 0 0 
0 0 
1
0


0 0 
H 0 : β1  0  Lβ M    21  22   0 0 L  0 0 1 0
M
d



0 1

  31  32  0 0
0 0 0 1
0 0
 .004153
 .001081  .000344 
980.13 905.74
 1.27431
  1.12  0.90 
  .004153 1.85  10 5 2.23 10 6  1.52 10 6 
^

1


β
( X' X)  
6
  0.49  0.69 
  .001081 2.23 10
1.46E - 06 7.51E - 07 




6

0
.
44

0
.
37

.
000344

1
.
52

10
7.51E
07
2.04E
06




'
^
1 
 ^
 227476.2 225046.8
1
H   L β  L( X' X) L'  L β   




  225046.8 329677.3
292844.2 252802.4
E  Y' (I  X( X' X) 1 X' )Y  

252802.4
276245.8




Testing Relation Between Price and Quality (II)
'
^
1 
 ^
 227476.2 225046.8
H   L β  L( X' X) 1 L'  L β   




  225046.8 329677.3
292844.2 252802.4
E  Y' (I  X( X' X) 1 X' )Y  

252802.4 276245.8




q1  rank( H  E)  2 q2  rank L( X' X) 1 L'  3   80  (3  1)  76 s  min( 2,3)  2
2(3)  2
2 2 32  4
 2  3  1
m1  0.5 3  2  1  0 m2  0.5(76  2  1)  36.5 r  76  
1 t 
2
  76 u 
4
2 2  32  5
 2 
Wilks' Lambda :  
E
HE

16987896750
 0.195411
86934276335
 1  0.1954111/ 2  76(2)  2(1) 

  31.55 df1  6, df 2  150 F.05, 6,150  2.160
 FW  
1 / 2 
0
.
195411
2
(
3
)



1
Pillai' s Trace : V  trace H (H  E)  0.348478  0.736181  1.084659


 1.084659  2(36.5)  2  1 
  30.02 df1  6, df 2  152 F.05, 6,152  2.159
 FP  

 2  1.084659  2(0)  2  1 
Hotelling - Lawley' s Trace : U  trace (E 1 H )  0.35007  2.33411  2.68419
 FHL
b  1.084888 c  1.318386

2(3)  2  
 4 

 2.68419   1.084888  1 

 33.34 df1  6, df 2  98.24 F.05, 6,100  2.191



1
.
318386
2
(
3
)






Testing for a Differential Effect
• Hypothesis: Price of Danish and NZ Butter is equally
“effected” by quantities of each type
–
–
–
–
H1: Common Effects for each quantity / price, common intercepts
H2: Common Effects for each quantity / price, different intercepts
H3: Common Effects for quantities, different effects across prices
H4: Differential Effects for quantities, common effects across prices
H : 11   21   31  12   22   32 ,  01   02
1
0
H 02 : 11   21   31  12   22   32
H : 11   21   31, 12   22   32
3
0
H :  01   02 , 11  12 ,  21   22 ,  31   32
4
0
Matrix Forms for H1:H4
 01   02

 0 

 0 



11
12

  

 0 
 21   22
H1 : 
 
 31   32

 0 
11  12    21   22  0

  
11  12    31   32  0
11  12

 0 

 0 



21
22

  
  0 
H2:
 31   32

  











11
12
21
22

 0 
11  12    31   32  0
 11  12 12   22  0 0
H3: 
  0 0 






12
23 


 11 13
  01   02  0
     0 
H 4 :  11 12    
  21   22  0

  



31
32

 0 
1
0

0
 L1  
0
0

0
0
0

 L 2  0

0
0
0 0 0
1 0 0
0
1 0
 1
M


1
 1
0 0
1
 
1  1 0

1 0  1
1 0 0
0
1 0
 1

0 0
1 M2   

 1
1  1 0
1 0  1
0
 L3  
0
1
0
 L4  
0

0
1  1 0
1 0
M3  

1 0  1
0 1 
0 0 0
1 0 0
 1
M4   
0 1 0
 1

0 0 1
H and E Matrices
'


^

1




'
1 '
H k  M k  L k β  L k ( X ' X ) L k  L k β M k




Ek  M 'k Y' (I  X( X' X) 1 X' )YM k
^
Hypothesis
1
2
3
4
H
662762
115751
29741.2
-6159.4
649483
Hypothesis
1
2
3
4
-6159.4
59334.3
E
63485.1
63485.1
292844
252802
63485.1

0.08742
0.3542
0.35824
0.08904
252802
276246
FW
132.236
27.7139
25.1533
194.379
q1
1
1
2
q2
6
5
2
s
1
1
2
m1
2
1.5
-0.5
m2
38
38
38
r
78
77.5
75.5
u
1
0.75
0.5
t
1
1
2
1
4
1
1
38
77
0.5
1
df1 (all)
6
5
4
4
df2W
76
76
150
76
V
0.91258
0.6458
0.67817
0.91096
Note: F.05,6,75=2.22, F.05,5,75=2.34, F.05,4,150=2.44, F.05,4,75=2.49
All 4 hypotheses are rejected
FP
135.716
28.4432
20.2658
199.494
df2P
78
78
158
78