COVARIANCE NOTATION
An easier method for performing calculations with covariances (as compared to summation notation) is
as follows. Let X, Y, W, and Z be random variables. Let α and β be parameters.
var    0
cov  , X   0
cov  X , X   var  X 
cov  X ,  Y )   cov  X , Y  
var  X   cov  X ,  X    2 var  X 
var  X  Y   var  X   var Y   2 cov  X , Y 
cov  X  Y ,W  Z   cov  X ,W   cov  X , Z   cov Y , W   cov Y , Z 
ERROR COVARIANCE MATRICES
An error covariance matrix shows the structure of covariances among different error terms. By
definition, error covariance matrices are symmetric (the number in the ith row and jth column is the same
as the number in the jth row and ith column). The number, or “element”, in the ith row and jth column is
the covariance between the ith and jth errors.
Let a simple time series data set be comprised of T time periods. Let ut be the error term at time t. The
error covariance matrix, Ω, is a T x T matrix that takes the form:
 cov(u1 , u1 ) cov(u1 , u2 )
 cov(u , u ) cov(u , u )
1
2
2
2



cov(u1 , uT )
cov(u1 , uT ) 




cov(uT , uT ) 
OLS Assumptions
Under the standard OLS assumptions, the covariance of error terms across time periods is zero and the
variance of the error term at a point in time is constant across time. This means that:
 2 if t1  t2
cov(ut1 , ut2 )  
0 if t1  t2
This implies that the error covariance matrix takes the form:
 2 0

0 2




0
0



2
 
Heteroskedasticity
In the presence of heteroskedasticity, the variance of the error term is no longer constant across time.
This means that:
 t2 if t1  t2  t
cov(ut1 , ut2 )  
0 if t1  t2
This implies that the error covariance matrix takes the form:
 12 0

0  22



 0
0




 T2 
Serial Correlation
In the presence of serial correlation, the covariance of the error term across different time periods is no
longer zero. The form (AR or MA) and order (1, 2, etc.) of the serial correlation describes the pattern of
the covariances.
Serial Correlation: AR(1) error
AR(1) errors take the form ut   ut 1   t where  t ~ IN  0,  2  . This implies that:
 2 if t1  t2

cov(ut1 , ut2 )    2 if t1  t2  1

0 if t1  t2  1
and that the error covariance matrix takes the form:
 2
 2
 
 0




 0
 2
2
0
 2
0
 2
0
2
 2
0 

0 


0 
 2 

 2 
Serial Correlation: AR(2) error
AR(2) errors take the form ut   ut 2   t where  t ~ IN  0,  2  . This implies that:
 2 if t1  t2
 2
  if t1  t2  1
cov(ut1 , ut2 )   2 2
   if t1  t2  2
0 if t  t  2
1
2

and that the error covariance matrix takes the form:
 2

2
 
  2 2

0






 0
 2
2
 2
 2 2
 2
2
0

 2
2
2
0
 2
0
 2
 2 2
2
 2
 2 2
2
 2 2
0
 2
2
 2
0 

0 



0 

 2 2 
 2 

 2 
Serial Correlation: MA(1) error
MA(1) errors take the form ut   t 1   t where  t ~ IN  0,  2  . This implies that:
cov(ut1 , ut2 )   n 2 , n  t1  t2
(note that an MA(1) process is the same as an AR(∞) process). The error covariance matrix takes the
form:
 2

2
 
  2 2

    3 2
  4 2


  T 2

 2
2
 2
 2 2
 3 2
 2 2
 2
2
 2
 2 2
 3 2
 2 2
 2
2
 2
 4 2
 3 2
 2 2
 2
2
 T 1 2
 T  2 2
 T 3 2
 T 4 2
 T 2 

 T 1 2 
 T 2 2 

 T 3 2 
 T 4 2 
2




Combined Heteroskedasticity and AR(1) Serial Correlation
In the presence of heteroskedasticity and an order 1 autoregressive error, we have: ut   ut 1   t
where  t ~ IN  0,  t2  .
 t2 if t1  t2  t
 2
cov(ut1 , ut2 )    min
t1 ,t2  if t1  t2  1

0 if t1  t2  1
The error covariance matrix takes the form:
  12
 2
  1
 0




 0
 12
 22
0

2
2
0
 T2 2
0
 T21
 T21
0 

0 


0 
 T21 

 T2 
Panel Data
Let a panel data set be comprised of S cross-sections and T time periods where the data is arranged first
by cross-section then by time period. The error covariance matrix, Ω, is an ST x ST matrix that takes the
form:
 1,1

1,2



1, S
1,2
 2,2
1, S 




S ,S 
where each submatrix  s1 , s2 is a T x T matrix of error covariances for cross-sections s1 and s2 across all
combinations of time periods. For example, let us,t be the error term associated with cross-section s at
time t. The submatrix  s1 , s2 is constructed as:
 s1 , s2
 cov(us1 ,1 , us2 ,1 ) cov(us1 ,1 , us2 ,2 )

 cov(us1 ,1 , us2 ,2 ) cov(us1 ,2 , us2 ,2 )


cov(us1 ,1 , us2 ,T )
cov(us1 ,1 , us2 ,T ) 




cov(us1 ,T , us2 ,T ) 
Expanding Ω, we have:
  cov  u1,1 , u1,1 

  cov  u1,1 , u1,2 


 cov  u , u 
1,1 1,T

  cov  u , u 
1,1
2,1

  cov  u1,1 , u2,2 



 cov u , u
 1,1 2,T 



  cov  u1,1 , uS ,1 

  cov  u1,1 , uS ,2 


 cov  u , u 
1,1
S ,T

cov  u1,1 , u1,2 
cov  u1,2 , u1,2 
cov  u1,1 , u2,2 
cov  u1,2 , u2,2 
cov  u1,1 , uS ,2 
cov  u1,2 , uS ,2 
cov  u1,1 , u1,T  




cov  u1,T , u1,T  
cov  u1,1 , u2,T  




cov  u1,T , u2,T  
 cov  u1,1 , u2,1  cov  u1,1 , u2,2 

 cov  u1,1 , u2,2  cov  u1,2 , u2,2 


cov  u1,1 , u2,T 

 cov  u2,1 , u2,1  cov  u2,1 , u2,2 

 cov  u2,1 , u2,2  cov  u2,2 , u2,2 


cov  u2,1 , u2,T 

cov  u1,1 , u2,T  




cov  u1,T , u2,T  
cov  u2,1 , u2,T  




cov  u2,T , u2,T  
cov  u1,1 , uS ,T  




cov  u1,T , uS ,T  
 cov  u1,1 , uS ,1  cov  u1,1 , uS ,2 

 cov  u1,1 , uS ,2  cov  u1,2 , uS ,2 


cov  u1,1 , uS ,T 

 cov  uS ,1 , uS ,1  cov  uS ,1 , uS ,2 

 cov  uS ,1 , uS ,2  cov  uS ,2 , uS ,2 


cov  uS ,1 , uS ,T 

cov  u1,1 , uS ,T  




cov  u1,T , uS ,T  
















cov  uS ,1 , uS ,T   




cov  uS ,T , uS ,T   

Note that there are two possible types of heteroskedasticity: within individuals and between individuals.
Within individuals heteroskedasticity manifests as non-constant diagonal elements of Ω. Between
individuals heteroskedasticity manifests as non-constant diagonal elements of the Σ. For each type of
heteroskedasticity, the heteroskedasticity can be common or individual specific. Common
heteroskedasticity is heteroskedasticity that is the same for all individuals. For example:
Common within-individuals heteroskedasticity
2

 if t1  t2  t and s1  s2
cov(us1 ,t1 , us2 ,t2 )   t

0 if t1  t2  1 or s1  s2
Common across-individuals heteroskedasticity
2

 if t1  t2  t and s1  s2
cov(us1 ,t1 , us2 ,t2 )   t

0 if t1  t2  1 or s1  s2
Individual specific within-individuals heteroskedasticity
2

 if t1  t2  t and s1  s2
cov(us1 ,t1 , us2 ,t2 )   s ,t

0 if t1  t2  1 or s1  s2
Individual specific across-individuals heteroskedasticity
2

 s ,t if t1  t2  t and s1  s2
cov(us1 ,t1 , us2 ,t2 )  

0 if t1  t2  1 or s1  s2
Notice that common within-individuals heteroskedasticity is identical to heteroskedasticity as seen in a
time series data set with the exception that the same heteroskedasticity structure is repeated for each
individual.
Similarly, serial correlation can occur within-individuals or between-individuals and can be common or
individual specific.