Special Topic:
Matrix Algebra and the ANOVA
•
Matrix properties
•
Types of matrices
•
Matrix operations
•
Matrix algebra in Excel
•
Regression using matrices
•
ANOVA in matrix notation
Definition of a matrix
 a matrix is a rectangular array of elements
 a11 a12
A
a 21 a 22
a13 
a 23 
 Matrix order (dimensions or size)
– m=#rows, n=#columns  m x n
 2 5 6
A

 1 2 3
matrix element
a13 = 6
order is 2 x 3
Matrix Algebra and the ANOVA
 a single value is called a ‘scalar’
B=6
 a single row is called a ‘row vector’
B   25 31 18 7
 a single column is called a ‘column vector’
25 
 31
B   25, 31, 18, 7    
18 
 
7
Types of Matrices
 A square matrix has equal numbers of rows and columns
 In a symmetric matrix,
aij = aji
a31 = a13 = 4
2
0
A
4
11

0
6
0
5
4
0
3
8
11
5

8
1 
 In a diagonal matrix, all off-diagonal elements = 0
 An identity matrix is a diagonal matrix with diagonals = 1
1
0
I =
0

0
0
1
0
0
0
0
1
0
0
0

0

1
Common Variance, Independence
 eij are independent, with common variance
1
0
2
I = 
0
0

0
1
0
0
0
0
1
0
0
 2


0 2
0
  
0
0


1
0
0
2
0
0
0
0
2
0
0

0
0
2
 
 Off-diagonal elements are zero, showing that
there is no covariance (there is independence)
Trace
 The trace of a matrix is the sum of the elements
on the main diagonal (aii)
2
0

A4
0

11
0
6
0
0
0
4
0
3
8
9
0 11
0 0

8 9
1 8

0 8 
tr(A) = 2 + 6 + 3 + 1 + 8 = 20
Matrix Addition and Subtraction
 Add or subtract corresponding elements of each
matrix
 The order (dimensions) of the matrices must be
the same
 4 6 2 9 2 5  13 8
8 3 0    4 7 1  12 10

 
 
7
1 
 4 6 2 9 2 5   5 4 3
8 3 0    4 7 1   4 4 1

 
 

Matrix Multiplication
 Take the sum of crossproducts of rows from the first
matrix with columns from the second matrix
 The number of columns in the first matrix must be
the same as the number of rows in the second matrix
A
rxn
B
nxc
4
2 5 1 8  
3 6 9 4  x  1

 9
7 3 3 5  
5
M
rxc
1
 62

6
  119
2 
  83
0
34 

57

31 
m11 = 2*4 + 5*1 + 1*9 + 8*5 = 62
Transpose of a Matrix
 To transpose a matrix, exchange rows and
columns
 2 1
A  5 2 


6 3 
a21  a12 = 5
2 5 6
A  

1 2 3
 A prime () or a (T) is used to denote a transpose
 Note that AA gives the uncorrected sum of
squares and crossproducts for the columns of A
 65 30 
AA  

30 14 
Sum of crossproducts
Sum of squares on the diagonal
Inverse of a Matrix
 Taking the inverse of a matrix is analagous to
division in math
 It’s easy for diagonal matrices
1
6
0
0
 6 0 0


1




1
A 0 3 0 A  0 3 0




1
0
0
9
0
0



9

 Use (-1) as an exponent to denote an inverse
Inverting a 2x2 Matrix
 Inverting a 2 x 2 matrix is not too hard
 Find the determinant (D), often written as |M|
a b
M

 c d
D = ad - bc

M   c
D
1
d
D

a 
D 
b
D
2 5
M

3 9 
D = 2*9 – 5*3 = 3

M   3
3
1
9
3
 For larger matrices, use a computer!

2 
3 
5
3
Linear Dependence
a b
M

 c d
D = ad - bc
 2 6
M

3 9 
D = 2*9 – 6*3 = 0
 The matrix is singular because one column can be
obtained by multiplying another by a constant (3 in this
case)
|M| = 0
 The rank of a matrix = the number of linearly
independent columns (1 in this case)
 A nonsingular matrix is full rank – the rank equals the
total number of columns
Properties of Full Rank Matrices
 A square, nonsingular matrix has a unique inverse
2 1 3
 0.25926 0.407407 0.33333
A   7 8 6  A 1   0.2037 0.03704 0.166667




 4 9 5 
 0.574074 0.25926 0.166667
Using Excel: MDETERM(G3:I5) = 54
The determinant ≠ 0, so there is a unique inverse
 For a full rank matrix
A-1A = AA-1 = I
 If A-1 exists, then (A-1)-1 = A
1 0 0
A 1A  I  0 1 0 


0 0 1
 If A-1 exists, and B-1 exists then (AB)-1 = B-1A-1
Idempotent Matrices
 A matrix is idempotent if it can be multiplied by
itself and the result is the original matrix
AA = A
 2 2 4 
A   1 3
4


 1 2 3 
 2 2 4 
AA   1 3
4


 1 2 3 
 Idempotent matrices must be square
 The trace of an idempotent matrix is equal to its
rank
Generalized Inverse
 A generalized inverse (M–) can be obtained for
any matrix, but the solution is not unique
MM–M = M
Matrix Algebra in Excel
 A little cumbersome, but may be handy for a
limited number of calculations
 Addition and subtraction are the same as always
– Use the usual shortcuts: fill down, fill right, copy, paste
 To transpose a matrix, there
are two options:
1. Copy the original matrix, select
a single destination cell, use
“paste special” and select the
option “Transpose”
2. Use the matrix function
TRANSPOSE
Matrix Functions in Excel
 Examples: MMULT, TRANSPOSE, MINVERSE
 Steps for Matrix operations (on a PC)
– Select destination cells (must be the right dimensions)
– Enter the matrix formula
– Press F2
– Press Ctrl-Shift-Enter
Regression in Matrix Notation
Y = X + ε
Linear model
Parameter estimates
b = (X
X)-1XY
Correction for mean
Y


CF 
i
n
Source
df
SS
MS
Regression
(uncorrected)
p
bXY
MSR
Residual
N-p
YY – bXY
MSE
Total (uncorrected)
N
YY
p = number of parameters estimated in the model
N = total number of observations
2
 nY 2
Regression example
 Fit a quadratic curve Yi = b0 + b1Xi + b2Xi2
Linear model in matrix notation
6 
13 
 
17 
16 
  
1
1
1
1
2
4
6
8
Solution: b = (XX)-1XY
Using Excel, SAS or R
X
2
4
6
8
Y
6
13
17
16
Y = X + ε
4
 e1 
b0   

16   e 2
 b1   
36     e3 
b2   

64
e 4 
Yi = -5.5 + 6.7Xi -0.5Xi2
ANOVA example
 CRD with 3 treatment levels, 2 reps
Linear model
 1 2 3
1
1

1
X
1

1
1

1
1
0
0
0
0
0
0
1
1
0
0
Y = X + ε
0* 1* *2
0  1
0  1
 
0  1


0
1
 
1  1


1  1
1
1
0
0
0
0
0
0

1
1

0

0
TRT
1
1
2
2
3
3
Y
3
4
1
4
8
6
Reparameterize to make a
nonsingular matrix (rank = 3)
Let 3 = 0
    3
*
0
1*  1  3
   2  3
*
2
ANOVA example using PROC IML
proc iml;
X={1 1 0,
1 1 0,
1 0 1,
1 0 1,
1 0 0,
1 0 0}
;
Y={3, 4, 1, 4, 8, 6};
CF=sum(Y)*sum(Y)/countn(Y);
XPX=X`*X; XPXinv=inv(XPX); XPY=X`*Y;
B=XPXinv*XPY;
SSTotal=(Y`*Y)-CF;
SSTrt=B`*XPY-CF;
SSE=SSTotal-SSTrt;
YHAT=X*B;
Resid=Y-YHAT;
print SSTotal SSTrt SSE B YHAT Resid;
QUIT;
Recalculating the parameters of interest
proc glm;
class trt;
model Y=trt/solution;
SAS uses a generalized inverse.
The result is the same as setting 3=0
 0*
1*
 2*
Y  (30*  1*  *2 ) / 3  (3  33  1  3  2  3 ) / 3
 (3  1  2  3 ) / 3      (3 * 7  3.5  4.5) / 3  4.3333
Y1  0*  1*    3  1  3    1
Y2  0*  *2    3  2  3    2
Y3  0*    3