Independent Observations
(I) Covariance functions
Let
Y1 
 1 
Y 
 
2
Y   , E Y      2 , V Y    2V  


,
 
 
Yn 
 n 
where
2
functions.
may be unknown and
Suppose
V   is a matrix of known
Y1 , Y2 ,, Yn
are
independent
VarYi  depends only on  i . Then,
0
V1 1 
 0
V2  2 
V    
 


0
 0




 .

 Vn  n 


0
0
Note:
V1 , V2 ,, Vn  may be taken to be identical.
(II) Quasi-likelihood functions
Motivating example:
Let Y ~ N
 ,  . Then, the log-likelihood function is
2
1
and
2

y  
l    
2
2
.
Thus, the score function for  is
s   
l  
y


2
.
Note that


y
y
 st dt  
y t
2
1 
t2   
 |y 
dt  2 
yt 
 
2
 


1 
 2  y2 
 2  y   y  

 
2

1  2 y 2  2 y   2  y 2
 2 

2


1

 y   

2
2
2
 l  
◆
Denote
U 
Y 
 2V   .
U has the following properties in common with the score function:
 
E U   0, Var U   E U 2 
 U
1



E
2
 V  
 

 .

Then, since the score function is the derivative of the log-likelihood
function, the integral
2
Q  


y
y t
dt
 2V t  ,
if exists, should behave like a log-likelihood function for  under
the very mild assumptions.
Q  is referred to as the
quasi-likelihood, or as the log quasi-likelihood for  based on
the data y.
Example:
Y 
2
U



Var
Y



Let V     . Then,
and
 2 .
y t
y  2t dt

1   y


dt

1
dt

y

2 
 y t

 Q   


1

2

y log t  |

y
  y
 y log      
2

1
1

2
 y  y log  y 
 y log    
 the log - likelihood of Poisson random vaiable
Example:
Y 
2
2 2
U





V



Var
Y



Let
. Then,
and
 2 2 .
3
y t
y  2t 2 dt
1
1   y

 2   2 dt   dt 
y t
 y t

 Q   


1  y 

 | y  log t  |y 
2 
  t



1 
y




1


log


log
y

 2  

 1
1  y
  log    2 1  log  y 
2 
  
 
y
   log  


 the log - likelihood of Gamma random variable
E Y    , then
Note that for a Gamma random variable Y with
VarY    2
◆
The quasi-likelihood function for the complete data is the sum of the
individual contributions
Q  
n
 Qi i  
i 1
n

i 1
i
yi
yi  t
dt
 2Vi t  .
The quasi-deviance function corresponding to a single observation is
D , y   2 Q   2 
2
y

which does not depend on
y t
dt ,
V t 
 2.
(III) Parameter estimation
Let the parameters of interest,
 p1 , related to the dependence of
 on the covariate x. Therefore, we can write  i   i   .
4
Thus,
Qi  i 
Qi  i   i
yi   i


Dir ,
2
 r
 i
 r
 Vi  i 
where Dir 
 i
. Therefore,
 r
Q  n Qi  i  n yi   i

 2
Dir
 r
 r
i 1
i 1  Vi  i 
V11 1 
0

0
V21  2 

 Dnr 
 


0
 0

1
D
 2 1r

1 t 1
DrV  y   
2

D2 r
where Dr  D1r

D2 r
  y1  1 


y


2
2



   


 Vn1  n   yn   n 


0
0
Dnr  . Further,
t
Q 
1
t
1
y   ,

D
V

2
U   
where

Dn p  D1

D2
Dp

 D11
D
21

 

 Dn1
D12
D22




Dn 2

D1 p 
D2 p 
 .

Dnp 
The covariance matrix of U   is
  E U  U t  
 U   
 E
i  VarU 



1

2



D tV
1
D
5
Note:
Under the usual limiting conditions on the eigenvalues of i  , the
asymptotic variance-covariance matrix of
 
̂ is

Var ˆ  i1   2 DtV 1 D

1
.
That is, i  plays the same role as the Fisher’s information for
◆
ordinary likelihood functions.
̂ , the Fisher’s scoring method
To obtain the parameter estimate
is
ˆ
n 1

ˆ tVˆ 1D
ˆ
 ˆn  D
n n
n

1
ˆ tVˆ 1  y  ˆ , n  0,1, ,
D
n n
n
where
ˆ  D ˆ , Vˆ  V  ˆ , ˆ    ˆ
D
 n
 n
n
n
n
 n
To estimate
 2 , we can use the following statistic
~ 2 
where X
2
.
n
Yi  ˆ i   X 2
1

n  p i 1 Vi ˆ i 
n p
2
,
is the generalized Pearson statistic.
Note:
Vi Yi    2Vi  i 
 2 
 Y  i 
Var Yi  Var Yi   i 

 Var  i

Vi  i 
Vi  i 


V


i
i 

 Var Z i , Z i 
Yi   i
Vi  i 
6
Thus, the statistic similar to the sample variance is
2
 Yi  ˆ i 
n
2



Zi
2

n
ˆ
Vi  i  
i 1 

Yi  ˆ i 
1

i 1



n p
n p
n  p i 1 Vi ˆ i 
n
7
.