Special Topic:
Matrix Algebra and the ANOVA
•
Matrix properties
•
Types of matrices
•
Matrix operations
•
Matrix algebra in Excel
•
Regression using matrices
•
ANOVA in matrix notation
Definition of a Matrix
 a matrix is a rectangular array of elements
 a11 a12
A
a 21 a 22
a13 
a 23 
 Matrix order (dimensions or size)
– m=#rows, n=#columns  m x n
 2 5 6
A

 1 2 3
matrix element
a13 = 6
order is 2 x 3
Types of Matrices
 a single value is called a ‘scalar’
B=6
 a single row is called a ‘row vector’
B   25 31 18 7
 a single column is called a ‘column vector’
25 
 31
B   25, 31, 18, 7    
18 
 
7
Types of Matrices
 A square matrix has equal numbers of rows and columns
 In a symmetric matrix,
aij = aji
a31 = a13 = 4
2
0
A
4
11

0
6
0
5
4
0
3
8
11
5

8
1 
 In a diagonal matrix, all off-diagonal elements = 0
 An identity matrix is a diagonal matrix with diagonals = 1
1
0
I =
0

0
0 0 0
1 0 0

0 1 0

0 0 1
Common Variance, Independence
 eij are independent, with common variance
1
0
2
I = 
0
0

0 0 0
 2


1 0 0 2
0
  
0 1 0
0


0 0 1
0
0
0
2
0
0
0
2
0
0

0
0
2
 
 Off-diagonal elements are zero, showing that
there is no covariance (there is independence)
Trace
 The trace of a matrix is the sum of the elements
on the main diagonal (aii)
2
0

A4
0

11
0
4
6
0
0
0
0
3
8
9
11
0 0

8 9
1 8

0 8 
0
tr(A) = 2 + 6 + 3 + 1 + 8 = 20
Matrix Addition and Subtraction
 Add or subtract corresponding elements of each
matrix
 The order (dimensions) of the matrices must be
the same
 4 6 2 9 2 5  13 8
8 3 0    4 7 1  12 10

 
 
7
1 
 4 6 2 9 2 5   5 4 3
8 3 0    4 7 1   4 4 1

 
 

Matrix Multiplication
 Take the sum of crossproducts of rows from the first
matrix with columns from the second matrix
 The number of columns in the first matrix must be
the same as the number of rows in the second matrix
A
rxn
B
nxc
4
2 5 1 8  
3 6 9 4  x  1

 9
7 3 3 5  
5
M
rxc
1
 62

6
  119
2 
  83
0
34 

57

31 
m11 = 2*4 + 5*1 + 1*9 + 8*5 = 62
Transpose of a Matrix
 To transpose a matrix, exchange rows and
columns
 2 1
A  5 2 


6 3 
a21  a12 = 5
2 5 6
A  

1 2 3
 A prime () or a (T) is used to denote a transpose
 Note that AA gives the uncorrected sum of
squares and crossproducts for the columns of A
 65 30 
AA  

30 14 
Sum of crossproducts
Sum of squares on the diagonal
Inverse of a Matrix
 Taking the inverse of a matrix is analagous to
division in math
 It’s easy for diagonal matrices
1
6
0
0
 6 0 0


1




1
A 0 3 0 A  0 3 0




1
0
0
9
0
0



9

 Use (-1) as an exponent to denote an inverse
Inverting a 2x2 Matrix
 Inverting a 2 x 2 matrix is not too hard
 Find the determinant (D), often written as |M|
a b
M

 c d
D = ad - bc

M   c
D
1
d
D

a 
D 
b
D
2 5
M

3 9 
D = 2*9 – 5*3 = 3

M   3
3
1
9
3
 For larger matrices, use a computer!

2 
3 
5
3
Linear Dependence
a b
M

 c d
D = ad - bc
 2 6
M

3 9 
D = 2*9 – 6*3 = 0
 The matrix is singular because one column can be
obtained by multiplying another by a constant (3 in this
case)
|M| = 0
 The rank of a matrix = the number of linearly
independent columns (1 in this case)
 A nonsingular matrix is full rank – the rank equals the
total number of columns
Properties of Full Rank Matrices
 A square, nonsingular matrix has a unique inverse
2 1 3
 0.25926 0.407407 0.33333 
A   7 8 6  A 1   0.2037 0.03704 0.166667 




 4 9 5 
 0.574074 0.25926 0.166667 
Using Excel: MDETERM(G3:I5) = 54
The determinant ≠ 0, so there is a unique inverse
 For a full rank matrix
A-1A = AA-1 = I
 If A-1 exists, then (A-1)-1 = A
1 0 0
A 1A  I  0 1 0 


0 0 1
 If A-1 exists, and B-1 exists then (AB)-1 = B-1A-1
Idempotent Matrices
 A matrix is idempotent if it can be multiplied by
itself and the result is the original matrix
AA = A
 2 2 4 
A   1 3
4


 1 2 3 
 2 2 4 
AA   1 3
4


 1 2 3 
 Idempotent matrices must be square
 The trace of an idempotent matrix is equal to its
rank
Generalized Inverse
 A generalized inverse (M–) can be obtained for
any matrix, but the solution is not unique
MM–M = M
Matrix Algebra in Excel
 A little cumbersome, but may be handy for a
limited number of calculations
 Addition and subtraction are the same as always
– Use the usual shortcuts: fill down, fill right, copy, paste
 To transpose a matrix, there
are two options:
1. Copy the original matrix, select
a single destination cell, use
“paste special” and select the
option “Transpose”
2. Use the matrix function
TRANSPOSE
Matrix Functions in Excel
 Examples: MMULT, TRANSPOSE, MINVERSE
 Steps for Matrix operations (on a PC)
– Select destination cells (must be the right dimensions)
– Enter the matrix formula
– Press F2
– Press Ctrl-Shift-Enter
Regression in Matrix Notation
Y = X + ε
Linear model
Parameter estimates
b = (X
X)-1XY
Correction for mean
Y


CF 
i
n
Source
df
SS
MS
Regression
(uncorrected)
p
bXY
MSR
Residual
N-p
YY – bXY
MSE
Total (uncorrected)
N
YY
p = number of parameters estimated in the model
N = total number of observations
2
 nY 2
Regression example
 Fit a quadratic curve Yi = b0 + b1Xi + b2Xi2
Linear model in matrix notation
6 
13 
 
17 
16 
  
1
2
1
4
1
6
1
8
Solution: b = (XX)-1XY
Using Excel, SAS or R
X
2
4
6
8
Y
6
13
17
16
Y = X + ε
4
 e1 
b0   

16   e 2
 b1   
36     e3 
b2   

64 
e 4 
Yi = -5.5 + 6.7Xi -0.5Xi2
Working with Matrices in R
 Create a matrix
 Add matrices
C <- A + B
 Multiply matrices
D <- A %*% B
 Transpose a matrix
Xp <- t(X)
 Invert a matrix
XpXinv <- solve(XpX)
ANOVA example
 CRD with 3 treatment levels, 2 reps
Linear model
 1 2 3
1
1

1
X
1
1

1
Y = X + ε
0* 1* *2
1 0 0
1
1
1 0 0


0 1 0
1

0 1 0
1
1
0 0 1


0 0 1
1
0 0
0 0

1 0

1 0
0 1

0 1
TRT
1
1
2
2
3
3
Y
3
4
1
4
8
6
Reparameterize to make a
nonsingular matrix (rank = 3)
Let 1 = 0
    1
*
0
  2  1
*
1
  3  1
*
2
Recalculating the parameters of interest
R Code for a one-way ANOVA
R uses the first treatment as the standard
ONE.crd<-lm(Y ~ TRT, data=ONE) The result is the same as setting 1=0
*
summary(ONE.crd)
0    1



*
0
*
1
*
2
1*  2  1
*2  3  1
Y1  0*    1
Y2  0*  1*    1  2  1    2
Y3  0*  2*    1  3  1    3
Y  (30*  1*  2* ) / 3  (3  31  2  1  3  1 ) / 3
 (3  1  2  3 ) / 3      (3 * 3.5  1.0  3.5) / 3  4.3333