On Bartlett Correction of Empirical Likelihood in the Presence of Nuisance Parameters BY SONG XI CHEN Department of Statistics, Iowa State University, Ames, Iowa 50011-1210, USA songchen@iastate.edu AND HENGJIAN CUI Department of Statistics and Financial Mathematics Beijing Normal University, 100875, China hjcui@bnu.eud.cn SUMMARY Lazar & Mykland (1999) showed that an empirical likelihood defined by two estimating equations with a nuisance parameter need not be Bartlett correctable. This paper shows that Bartlett correction of empirical likelihood in the presence of a nuisance parameter depends critically on the way the nuisance parameter is removed when formulating the likelihood for the parameter of interest. We establish in the broad framework of estimating functions that the empirical likelihood is still Bartlett-correctable if the nuisance parameter is profiled out given the value of the parameter of interest. Some key words: Bartlett correction; Empirical likelihood; Estimation equation; Nuisance parameter. 1. INTRODUCTION Since its introduction by Owen (1988, 1990), empirical likelihood has become an useful tool for conducting nonparametric or semiparametric inference. Empirical likelihood has been shown in a wide range of situations as outlined in Owen (2001) to admit limiting chisquared distributions, which is a nonparametric version of the Wilks theorem in the context of parametric likelihood. Another key property of empirical likelihood which also resembles 1 that of a parametric likelihood is Bartlett correction. Bartlett-correctability is a secondorder property which implies that a simple mean adjustment to the likelihood ratio leads to its distributional approximation to the limiting chi-squared distribution being improved by one order of magnitude. That the empirical likelihood is Bartlett-correctable has been established for a range of situations; see for example Hall and La Scala (1990) for the case of the mean parameter, DiCiccio et al. (1991) for smooth functions of means, Chen & Hall (1993) for quantiles, Chen (1993, 1994) for linear regression and Cui & Yuan (2001) for quantiles in the presence of auxiliary information. Jing & Wood (1996) showed that the exponentially tilted empirical likelihood for the mean is not Bartlett-correctable. Indeed, Corcoran (1998) showed that Kullback-Leibler divergence is the unique member of a large class of divergence measures that produces Bartlett-correctable empirical likelihood statistics. However, Lazar & Mykland (1999) showed that in some circumstances, where the empirical likelihood is defined by two estimating equations and when a nuisance parameter is present, even the use of KullbackLeibler divergence can fail to guarantee Bartlett correctability. In contrast to the result of Lazar & Mykland (1999), Chen (1994) had earlier proved that empirical likelihood is Bartlett-correctable in the context of simple linear regression when one coefficient is treated as a nuisance parameter. It appears that the result obtained by Lazar & Mykland (1999) is due to absence of a regular Edgeworth expansion for the signed square root of the empirical likelihood ratio. In the present paper, we confirm that the result of Chen (1994) holds in general. We consider the Bartlett property in a broader situation where there are r estimating equations and the dimension of the nuisance parameter is p, with p < r, which is within the framework of the empirical likelihood for generalised estimating equations introduced in Qin & Lawless (1994). It is found that, if the nuisance parameter is profiled out given the parameter of interest, the empirical likelihood is still Bartlett-correctable. This indicates that the Bartlett correctability of the empirical likelihood is dependent on the method of nuisance parameter 2 removal when one formulating the likelihood for the parameter of interest, rather than on any fundamental differences between estimating equations and the smooth function of means. It is expected that a corresponding result holds for parametric likelihood as well, namely that the Bartlett correction property only holds in general when the nuisance parameter is ‘profiled out’. The paper is organized as follows. In Section 2 we establish the empirical likelihood defined on a set of generalized estimating equations in the presence of nuisance parameters. The Bartlett correction is established in Section 3. All the algebraic manipulations required to establish the Bartlett correction is given in the Appendix. 2. EMPIRICAL LIKELIHOOD WITH NUISANCE PARAMETERS Consider a random vector X with unknown distribution function F which depends on a r-dimensional parameter (θ, ψ) ∈ Rr−p × Rp . Here the interest is on the parameter θ while treating ψ as a p-dimensional nuisance parameter. We assume that the parameter (θ, ψ) is defined by r (r > p) functionally unbiased estimating equations g j (x, θ, ψ), j = 1, 2, · · · , r such that E{g j (X1 , θ0, ψ0 )} = 0 where (θ0, ψ0 ) is the true parameter value. In particularly, we define g(X, θ, ψ) = g 1 (X, θ, ψ), g 2 (X, θ, ψ), · · · , g r (X, θ, ψ) T . Assume that {X1 , X2 , · · · , Xn } is an independent and identically distributed sample drawn from F . Let V = Cov{g(Xi , θ0, ψ0)} and we assume the following regularity conditions: (i) V is a r × r positive definite matrix and the rank of E{∂g(X, θ0, ψ0)/∂ψ} is p; (1) (ii) For any j, 1 ≤ j ≤ p, all the fourth order partial derivatives of g j (x, θ0, ψ) with respect to ψ are continuous in a neighborhood of θ0 and are bounded by some integrable function G(x) in the neighborhood; (iii) Eg(X, θ0 , ψ0)15 < ∞ and the characteristic function of g(X, θ0 , ψ0) satisfies the Cramér condition: lim sup|t|→∞ |E[exp{itT g(X, θ0 , ψ0)}]| < 1. 3 To simplify derivations, let us first rotate the original estimating functions by defining wi (θ, ψ) =: T V −1/2g(Xi , θ, ψ), where T is a r × r orthogonal matrix such that T V −1/2E ∂g(X, θ 0 , ψ0 ) T U= Λ 0 ∂ψ r×p U = (ukl )p×p is an orthogonal matrix and Λ = diag(λ1 , · · · , λp ) is a non-singular p × p diagonal matrix. Furthermore, let us define Ω = ω kl p×p = UΛ−1 . Let p1 , · · · , pn be non-negative weights allocated to the observations. The empirical likelihood for the parameter (θ, ψ) is L(θ, ψ) = subject to pi = 1 and the constraints n pi i=1 pi wi (θ, ψ) = 0. Let (θ, ψ) = −2 log{nn L(θ, ψ)} be the log empirical likelihood ratio. Standard derivations in the empirical likelihood show that (θ, ψ) = 2 n log{1 + λT (θ, ψ)wi(θ, ψ)}, i=1 where λ(θ, ψ) satisfies: n−1 n i=1 1+ wi (θ, ψ) T λ (θ, ψ)wi (θ, ψ) = 0. (2) To obtain the empirical likelihood ratio at θ0 , we need to profile out the nuisance parameter ψ. To simplify notation, let us write wi (ψ) = wi (θ0 , ψ) and let ψ̃ =: ψ̃(θ0 ) be the minima of (θ0 , ψ) given θ = θ0 and λ̃ = λ(θ0 , ψ̃) be the solution of (2) at (θ0, ψ̃). Let (θ̂, ψ̂) be the maximum empirical likelihood estimate of parameter (θ, ψ). Since the number of estimating functions equal to the dimension of parameter (θ, ψ), then (θ̂, ψ̂) = 0. This means that the log empirical likelihood ratio for θ0 is just r(θ0 ) =: (θ0 , ψ̃(θ)) = 2 n i=1 4 log{1 + λ̃T wi (ψ̃)}. In order to develop an expansion for r(θ0 ), we need to derive expansions for λ̃ and ψ̃ first. We notice from Qin and Lawless (1994) that (λ̃, ψ̃) are the solutions of Q1n (λ, ψ) = n−1 Q2n (λ, ψ) = n−1 n i=1 n i=1 wi (ψ) =0 1 + λT wi (ψ) (3) (∂wi(ψ)/∂ψ)T λ = 0. 1 + λT wi (ψ) (4) Let η = (λT , ψ T )T , η0 = (0, ψ0 ), ⎛ ⎞ ⎜ Q1n (η) ⎟ Q(η) = ⎝ Q2n (η) ⎠ ⎛ and S = E ⎞ ∂Q(0, ψ0) ⎜ −I S12 ⎟ =⎝ ⎠, ∂η S21 0 T . To facilitate easy expressions, we standardize Q to where S21 = U(Λ, 0) and S12 = S21 Γ(η) = S −1 Q(η). Let wij (ψ) and Γj (η) denote respectively the j-th component of wi(ψ) and Γ(η). The following α − A system of notations was first used by DiCiccio, Hall and Romano (1988): αj1 ...jk = E{wj1 (ψ0 )...wjk (ψ0)} Aj1 ...jk = n−1 n wj1 (ψ0 )...wjk (ψ0) − αj1 ...jk . i=1 We also need to define β j,j1 ...jk = E ∂ k Γj (0, ψ 0) ∂ηj1 ...∂ηjk , B j,j1 ...jk = ∂ k Γj (0, ψ0) − β j,j1 ...jk ∂ηj1 ...∂ηjk and ∂ n wip (ψ0) ∂ lwij (ψ0) ∂ m wik (ψ0) ... ∂ψ j1 ...∂ψ jl ∂ψ k1 ...∂ψ km ∂ψ p1...∂ψ pn n 1 ∂ n wip (ψ0) ∂ lwij (ψ0 ) ∂ m wik (ψ0 ) = ... n i=1 ∂ψ j1 ...∂ψ jl ∂ψ k1 ...∂ψ km ∂ψ p1...∂ψ pn γ j,j1 ...jl ;k,k1 ...km ;...;p,p1...pn = E C j,j1 ...jl ;k,k1 ...km ;...;p,p1...pn − γ j,j1 ...jl ;k,k1 ...km ;...;p,p1...pn . 3. EXPANSIONS TO THE LIKELIHOOD RATIO 5 Since 0 = Γj (ψ̃(θ0 ), λ̃) = B j + β j,k (η̃ k − η0k ) + B j,k (η̃ k − η0k ) + 1 j,kl k β (η̃ 2 − η0k )(η̃ l − η0l ) + 12 B j,kl (η̃ k − η0k )(η̃ l − η0l ) + 1 j,klm k β (η̃ 6 + 1 j,klmn k β (η̃ 24 − η0k )(η̃ l − η0l )(η̃ m − η0m )(η̃ n − η0n ) + 1 B j,klmn (η̃ k 24 − η0k )(η̃ l − η0l )(η̃ m − η0m )(η̃ n − η0n ) + Op (n−5/2 ). − η0k )(η̃ l − η0l )(η̃ m − η0m ) + 16 B j,klm (η̃ k − η0k )(η̃ l − η0l )(η̃ m − η0m ) Here and throughout the paper, we use the tensor notation where if a superscript is repeated a summation over that superscript is understood. After inverting the above expansion we have for j ∈ {1, · · · , r + p}, η̃ j − η0j = −B j + B j,k B k − 12 β j,kl B k B l − B j,k B k,lB l + 12 β k,lmB j,k B l B m + β j,kl B k,m B mB l − 12 β j,kl β k,mn B m B n B l − 12 B j,kl B k B l + 16 β j,klm B k B lB m + Op (n−2 ) where j, k, l, m, ∈ {1, 2, · · · , r + p}. This implies that for j ∈ {1, · · · , r} λ̃j = −B j + B j,q B q − 12 β j,uq B u B q − B j,u B u,q B q + 12 β u,qsB j,u B q B s + β j,uq B u,s B s B q − 1 j,uq u,st s t q β β B BB 2 − 12 B j,uq B u B q + 16 β j,uqs B u B q B s + Op (n−2 ) (5) where q, s, t, u ∈ {1, · · · , r + p}, and for k ∈ {1, · · · , p} ψ̃ k = −B r+k + B r+k,q B q − 12 β r+k,uq B u B q − B r+k,u B u,q B q + 12 β u,qs B r+k,u B q B s + β r+k,uq B u,s B s B q − 12 β r+k,uq β u,st B s B tB q − 12 B r+k,uq B u B q + 16 β r+k,uqs B u B q B s + Op (n−2 ). (6) Derivations given in the appendix show that for a ∈ {1, · · · , r − p} n−1 (θ0 ) = Ap+a Ap+a − Ap+a p+b Ap+aAp+b − 2ω kl C p+a,k Ap+a Al + 2γ p+a;p+b,k ω kl Ap+a Ap+b Al + 23 αp+a p+b p+c Ap+aAp+b Ap+c + γ p+a,kl ω km ω ln Ap+aAm An + Aji B i,q B q B j [2, i, j] − B j,u B j,q B u B q 6 − 2C j,k B j,q B r+k B q + γ j,kl B r+k B r+l B j,q B q + 2γ j,kl B j B r+l B r+k,q B q − 2γ j;i,l (B j B i B r+l,q B q + B r+l B i B j,q B q [2, j, i]) + 2αjih B j B i B h,q B q + ( 12 β j,uq β r+k,st γ j,k − 14 β j,uq β j,st )B u B q B s B t − 12 γ j,kl β j,uq B u B q B r+l B r+k + (γ i;j,k β i,uq + γ j;i,k β i,uq − γ j,kl β r+l,pq )B u B q B j B r+k + 2γ j;i;h,k B j B i B h B r+k − (γ j;i,lk + γ j,l;i,k )B j B i B r+l B r+k + 13 γ j,klm B j B r+k B r+l B r+m − 1 jihg j i h g α B BB B 2 + (γ j;i,l β r+l,uq − αjih β h,uq )B j B i B u B q − C j,kl B j B r+k B r+l + 2C j;i,l B j B iB r+l − 23 Ajih B j B iB h + Op (n−5/2). (7) Let R = R1 + R2 + R3 be a signed root decomposition of n−1 (θ0 ) such that n−1 (θ0 ) = Rq Rq + O(n−5/2 ) where Rj = Op (n−j/2 ) for j = 1, 2 and 3. Clearly, R1 and R2 can be determined from the terms of Op (n−1 ) and Op (n−3/2) respectively in (7). Specifically, for a, b, c, d, e ∈ {1, · · · , r−p} and l, k, m, n, o, v, m, n ∈ {1, · · · , p} Rq1 = Rq2 = 0 for q ≤ p, Rp+a = − 12 Ap+a 2 + p+b Rp+a = Ap+a 1 and Ap+b − ω kl C p+a,k Al + γ p+a;p+b,k ω kl Ap+b Al 1 p+a p+b p+c p+b p+c α A A 3 + 12 γ p+a,kl ω km ω ln Am An . p+a from (7) and expressing all the remaining After removing terms induced by Rp+a 2 R2 p+a p+a terms in terms of As and Cs, we have Rq3 = 0 for q ≤ p and Rp+a = Rp+a 3 31 + R32 + R33 where Rp+a = 31 − 3 p+a p+c p+c p+b p+b A A A 8 + ω ml C p+b,m Al 1 ml nl p+a,m p+b,n p+b ω ω C C A 2 − αl p+a p+b p+a Ap+b + 12 ω lm C p+b,l Ap+a p+b Am + ω ml ω kn C l,k C p+a,m An + ω lm C p+a;p+b,l Ap+b Am ω nl C p+c,n Ap+b Ap+c − αp+a − 5 p+a p+b p+c p+c p+d p+b p+d α A A A 6 + 1 p+a p+b p+c p+b p+c A A A 3 + αl p+b p+c ω mn C p+c,m Ap+b An + 12 ω km ω ln C p+a,kl Am An p+a p+b 7 ω kl γ p+c;p+d,k + 49 αp+a p+b p+e p+e p+c p+d α − 1 kl ml p+a;p+b,k p+c;p+d,m ω ω γ γ 2 − 14 αp+a p+b p+c p+d Ap+b Ap+c Ap+d , Rp+a = − 12 γ m,kl ω kn ω lo ω vm C p+a,v An Ao − 14 γ p+b,kl ω kn ω lm Ap+a 32 + γ p+b,kl ω lo ω kn ω vn C p+a,v Ap+b Ao − γ p+b,kl ω lo ω kn An p+a p+b Am An Ap+b Ao − γ p+a,kl ω ln ω mo ω kv C v,m Ap+a An Ao + γ p+a;p+b,lω ln ω on C p+c,o Ap+b Ap+c − γ p+a;p+b,l ω ln An p+c Ap+bAp+c − γ p+a;p+b,r+m ω mn ω lo C o,m Ap+b An − (γ m;p+a,l + γ p+a;m,l )ω ln ω om C p+b,o Ap+b An − (γ p+a;p+b,l + γ p+a;p+c,l )ω ln ω ko C p+b,k An Ao − ( 12 γ p+c;p+a,l + γ p+a;p+c,l )ω ln Ap+b Rp+a = 33 1 kl p+a p+b p+c p+c,k p+b l ω α C A A 3 p+c Ap+b An and + 12 ω m m ω n n ω ov ω lk γ k,m n γ p+a,ol + γ p+b,m n γ p+a;p+b,o − 13 γ p+a,m n o AmAn Av + ωm mωn n 1 p+a p+b p+c p+c,m n α γ 3 + 1 p+a,m ;p+c p+b;p+c,n γ γ 2 − 1 ol kl p+a,m o p+b,n k ω ω γ γ 2 + ω n n ω ol αl + ω lo γ p+b,n l (γ o,m ;p+a + γ o;p+a,m ) + 12 ω lo γ o,m n γ p+a;p+b,l − 12 γ p+a;p+b,m n − 12 γ p+a,m ;p+b,n Ap+b Am An p+a p+b p+c,on γ + 12 γ p+c,m ;p+a (γ p+c;p+b,n + γ p+b;p+c,n ) + αp+a p+b p+d 2 p+d;p+c,n (3γ + γ p+c;p+d,n ) − γ p+a;p+b;p+c,n − ω ol γ p+a;p+b,l γ p+c,on Ap+b Ap+c An . 4. BARTLETT CORRECTABILITY The key in checking if the empirical likelihood is Bartlett correctable or not is to examine if the third and the fourth order joint cumulants of R are at the orders of n−3 and n−4 respectively. This is the path taken by DiCiccio, Hall and Romano (1991), Jing and Wood (1996) and Lazar and Mykland (1999). A formal establishment of the Bartlett correction can be made by developing Edgeworth expansions for the empirical likelihood ratio under condition (1). The joint third-order cumulants of R is cum(Rp+a , Rp+b , Rp+c ) = E(Rp+a Rp+b Rp+c ) − E(Rp+a )E(Rp+b Rp+c )[3] + 2E(Rp+a )E(Rp+b )E(Rp+c )[3] 8 p+b p+c p+a p+b p+c p+a p+b p+c −3 = E(Rp+a 1 R1 R1 ) + E(R2 R1 R1 )[3] − E(R2 )E(R1 R1 )[3] + O(n ). Note that p+e −1 de E(Rp+d 1 R1 ) = n δ , p+d p+e −2 p+a E(Rp+a 1 R1 R1 ) = n α −1 E(Rp+a − 16 αp+a 2 ) = n p+b p+b p+d p+e − ω kl γ p+a,k;l + 12 ω km ω lm γ p+a,kl + O(n−2 ). p+d p+e By working out E(Rp+a 2 R1 R1 ) it may be shown that p+d p+e p+a p+d p+e p+a p+d p+e 1 −3 E(Rp+a 2 R1 R1 ) = E(R2 )E(R1 R1 ) − 3 E(R1 R1 R1 ) + O(n ) (8) which readily implies that cum(Rp+a , Rp+b , Rp+c ) = O(n−3 ). (9) The joint fourth-order cumulants of R is cum(Rp+a , Rp+b , Rp+c , Rp+d ) = E(Rp+a Rp+b Rp+c Rp+d ) − E(Rp+a Rp+b )E(Rp+c Rp+d )[3] − E(Rp+a )E(Rp+b Rp+c Rp+d )[4] + 2E(Rp+a )E(Rp+b )E(Rp+c Rp+d )[6] − 6E(Rp+a )E(Rp+b )E(Rp+c )E(Rp+d ) p+b p+c p+d p+a p+b p+c p+d p+a p+b p+c p+d = E(Rp+a 1 R1 R1 R1 ) + E(R2 R1 R1 R1 )[4] + E(R3 R1 R1 R1 )[4] (10) p+b p+c p+d p+a p+b p+c p+d + E(Rp+a 2 R2 R1 R1 )[6] − E(R1 R1 )E(R1 R1 )[3] p+b p+c p+d p+a p+b p+c p+d − E(Rp+a 2 R1 )E(R1 R1 )[12] − E(R3 R1 )E(R1 R1 )[12] p+b p+c p+d p+a p+b p+c p+c − E(Rp+a 2 R2 )E(R1 R1 )[6] − E(R2 )E(R1 R1 R1 )[4] p+b p+c p+d p+a p+b p+c p+d −4 − E(Rp+a 2 )E(R2 R1 R1 )[12] + 2E(R2 )E(R2 )E(R1 R1 )[6] + O(n ). From (8), we have p+b p+c p+d p+b p+c p+d p+b p+c p+d −4 E(Rp+a 2 ){E(R1 R1 R1 )[4]+E(R2 R1 R1 )[12]−2E(R2 )E(R1 R1 )[6]} = O(n ) which means the sum of the last three terms in (10) is O(n−4 ). We examine in the following the other terms in (10). 9 Let t1 = αp+a p+b p+c p+d t3 = αp+a p+b p+c + αp+b t4 = αp+a , t2 = δ ab δ cd + δ acδ bd + δ ad δ bc , αp+d p+e p+e p+c p+d p+a p+e p+e α p+b p+e αp+c p+d p+e + αp+a p+b p+d αp+c p+e p+e + αp+a p+c p+d p+b p+e p+e p+c p+e p+b p+d p+e + αp+a p+d p+e p+b p+c p+e α and + αp+a α α . It is easy to check that p+b p+c p+d p+a p+b p+c p+d −3 −4 E(Rp+a 1 R1 R1 R1 ) − E(R1 R1 )E(R1 R1 )[3] = n (t1 − t2 ) + O(n ). (11) Derivations given in the appendix show that p+b p+c p+d p+a p+b p+c p+d E(Rp+a 2 R1 R1 R1 )[4] − E(R2 R1 )E(R1 R1 )[12] = n−3 [−6t1 + 2t2 − 16 t3 + 23 t4 − {ω kl γ p+a,k;l αp+b + 12 {γ p+a,kl ω km ω lm αp+b p+c p+d p+c p+d }[4] }[4]] + O(n−4 ), (12) p+b p+c p+d p+a p+b p+c p+d E(Rp+a 2 R2 R1 R1 )[6] − E(R2 R2 )E(R1 R1 )[6] = n−3 {3t1 − t2 + 16 t3 − 59 t4 + 13 ω kl (γ p+a,k;l αp+b − 16 ω km ω lm γ p+a,kl αp+b p+c p+d p+c p+d [2, a, b][6]} + O(n−4 ) )[2, a, b][6] (13) and p+b p+c p+d p+a p+b p+c p+d −3 1 −4 E(Rp+a 3 R1 R1 R1 )[4] − E(R3 R1 )E(R1 R1 )[12] = n (2t1 − 9 t4 ) + O(n ). (14) Combining (11), (12), (13) and (14), we see all the terms of order n−3 cancel each other and hence cum(Rp+a , Rp+b , Rp+c , Rp+d ) = O(n−4 ) (15) This and (9) mean that the empirical likelihood ratio for θ0 admits Bartlett correction despite the presence of the nuisance parameters. 10 5. AN EXAMPLE To connect our finding with that of Lazar and Mykland (1999), we present here an example which would mean the empirical likelihood was not Bartlett correctable by applying the results of Lazar and Mykland (1999), but it is Bartlett correctable based on some known result. Assume a bivariate random vector X = (Z, Y )T has a distribution F , and {Xi = (Zi , Yi )}ni=1 be an independent and identically distributed sample drawn from F . Let θ0 = E(X) = 0 and ψ = E(Y ). The interest here is to infer θ while treating ψ as a nuisance parameter. There are two estimating equations: g 1 (X, θ, ψ) = Z −θ and g 2 (X, θ, ψ) = Y −ψ. We further assume that Cov(X) = I2, E(Z 3 ) = 0 and X admits other conditions assumed in (1). To be consistent with the notations used in Lazar and Mykland (1999), we let ⎛ U. = xi , T (yi − ψ), 0 , U.. = ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ − x2 − i xi (yi − ψ) − − ⎞ xi (yi − ψ) 0 (yi − ψ)2 −n 0 ⎟ ⎟ ⎟ , −n ⎟ ⎟ 0 ⎠ κr = n−1 E(Ur ), κrs = n−1 E(Urs ), κr,s = n−1 Cov(Ur , Us ), κr,ts = n−1 Cov(Ur , Uts ) and κrs,tu = n−1 Cov(Urs , Utu ). It is easy to see that κr =: EUr = 0, ⎛ (κrs ) = ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ −1 0 ⎞ ⎛ ⎞ ⎛ 0 ⎟ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ 1 0 0 ⎟ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ 0 −1 −1 0 −1 0 ⎟ ⎟ ⎟, ⎟ ⎠ and (κr,s ) = (κr,s )+ , where (κr,s ) = + 0 1 0 0 0 0 ⎟ ⎟ ⎟, ⎟ ⎠ (κrs ) = (κrs )−1 = ⎞ −1 0 0 0 −1 0 −1 1 stands for the Moore-Penrose inverse. i i = κi,ακα,rs = κi,rs , βrst = κi,α κα,rst = κi,rst for i = 1, 2, 3, and Define βrs i Vi Ur = Vr , Urs = Vrs + βrs i i and Urst = Vrst + βrs Vit [3] + βrst Vi . 11 0 ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ Moreover let v denote the cumulants of V . These notations are fully consistent with Lazar and Mykland (1999). It is easy to verify that (v rs) = (κrs ), v 11 = v 23 = −v 33 = −1, v 13 = 0 and v111 = −E(x3), and v113 = v133 = v13,13 = v1133 = 0. Hence b0 = b1 = c = 0 in equation (5) of Lazar and Mykland (1999). Moreover, ω̃4 = 37 −1 n v111v111v 11v 11v 11 + O(n−2 ) 18 = − 37 n−1 {E(x3)}2 + O(n−2 ), ρ4 = − 37 {E(x3 )}2 = 0. 18 18 This means that the fourth order cumulants of R were not at the order of n−4 , and thus the empirical likelihood would not be Bartlett correctable. However, we note that the empirical likelihood ratio statistic (θ0 ) = 2 n log{1 + λ̂1 g 1 (Xi , 0, ψ̂) + λ̂2 g 2 (Xi , 0, ψ̂)} i=1 where λ̂1 , λ̂2 , ψ̂ are the solutions of ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ n xi = 0, 2 i + λ (yi − ψ) i=1 1 + n yi − ψ = 0 and 1 2 i=1 1 + λ xi + λ (yi − ψ) n −λ2 = 0. 1 2 i=1 1 + λ xi + λ (yi − ψ) λ1 x From the third equation, λ̂2 = 0, and λ̂1 should satisfy Therefore, (θ0 ) = 2 n xi i=1 1 + λ̂1 xi n i=1 n = 0 and ψ̂ = i=1 n i=1 yi /(1 + λ̂1 xi ) 1/(1 + λ̂1 xi ) . log(1 + λ̂1 xi ). This is essentially the empirical likelihood for the mean, and is known to be Bartlett correctable. 6. DISCUSSION The Bartlett factor for the empirical likelihood ratio (θ0 ) can be derived by deriving the first two cumulants of R. p+a −1 p+a Since E[Rp+a + O(n−2 ) where μp+a = − 16 αp+a 1 ] = 0 and E[R2 ] = n μ ω kl γ p+a,k;l + 12 γ p+a,kl ω km ω lm , we have cum(Rp+a ) = n−1 μp+a + O(n−2 ). 12 p+b p+b − Derivations given in the appendix show that cum(Rp+a , Rp+e ) = n−1 δ ae + n−2 Δae + O(n−3 ) (16) where Δae = 1 p+a p+e p+b p+b α 2 − 13 αp+a p+b p+c αp+e p+b p+c − ω kl γ p+a,k;l;p+e [2, a, e] − 12 ω km ω ln γ p+a,kl αmn + ω ml [γ p+b;p+b,m αl p+a p+e − p+e 1 p+a p+e p+c p+b p+b p+c α α 36 [2, a, e] + 12 (γ p+b;p+e,m − γ p+e;p+b,m )αl p+a p+b ][2, a, e] − ω ml ω nl γ p+a;p+b,n γ p+e;p+b,m + ω kl ω mn γ p+a,k;l γ p+e,m;n + 2ω ml γ p+e,m;l;p+a [2, a, e] − ω ml ω nl γ p+a,m;p+e,n − 13 ω mn γ p+b,m;n αp+a + ω kl [ 23 γ p+e,k;l αp+a p+b p+b − γ p+a;p+e,k αl p+b p+b p+e p+b ][2, a, e] + 1 p+b,kl km lm p+a p+e p+b γ ω ω α 6 − 1 p+a,kl lm ov km p+e,o;v γ ω ω ω γ ][2, a, e] − 12 ω km ω ln ω k m ω l n γ p+a,kl γ p+e,k l . 2 + ω kv ω ln ω mn γ p+a,m;v γ p+e,kl [2, a, e] Let cα be the upper α quantile of the χ2r−p distribution with density function gr−p . Then, √ by developing an Edgeworth expansion for nR under conditions (1), it may be shown that P {(θ0 ) < cα} = α − n−1 Bc cα gr−p (cα ) + O(n−2 ) and P {l(θ0 ) < cα (1 + n−1 Bc )} = α + O(n−2 ), where Bc = (r − p)−1 {μT μ + d a=1 Δaa} is the Bartlett factor. ACKNOWLEDGEMENT The authors would like to thank Professor Nicole Lazar for informative discussions. The project is supported by an Academic Research Grant of the National University of Singapore (R-155-000-018-112) and the NSFC (10071009) of China. 13 REFERENCES CHEN S. X. (1993). On the coverage accuracy of empirical likelihood regions for linear regression model. Ann. Inst. Statist. Math. 45, 621-637. CHEN, S. X. (1994). Empirical likelihood confidence intervals for linear regression coefficients. J. Multivariate Anal. 49, 24-40. CHEN, S. X. & HALL, P. (1993). Smoothed empirical likelihood confidence intervals for quantiles. Ann. Statist. 21, 1166-1181. CUI H. J. & YUAN X. J. (2001). Smoothed empirical likelihood confidence interval for quantile in the partial symmetric auxiliary information. J. Sys. Sci. and Math. Scis. 21(2), 172-181. DICICCIO, T. J., HALL, P. & ROMANO, J. P. (1991). Empirical likelihood is Bartlett correctable. Ann. Statist. 19, 1053-1061. DICICCIO, T. J. & ROMANO, J. P. (1989). On adjustments based on the signed root of the empirical likelihood ratio statistic. Biometrika 76, 447-56. JING, B. Y. and WOOD, A. T. A. (1996). Exponential empirical likelihood is not Bartlett correctable. Ann. Statist. 24 365-369. LAZAR, N. A. & MYKLAND, P. A. (1999). Empirical likelihood in the presence of nuisance parameters. Biometrika 86, 203-211. OWEN, A. B. (1990). Empirical likelihood ratio confidence regions. Ann. Statist. 18, 90-120. OWEN, A. B. (1991). Empirical likelihood for linear models. Ann. Statist. 19, 1725-1747. QIN J. & LAWLESS, J. (1994). Empirical likelihood and general estimation equations. Ann. Statist. 22, 300-325. APPENDIX Basic formulae We start with providing some basic formulae which will be used throughout the appendix. 14 From the definition of the S matrix in Section 2, it can be easily checked that ⎛ S −1 T −1 T T −1 ⎜ −I + S12 (S12 S12 ) S12 S12 (S12 S12 ) ⎟ =⎝ ⎛ ⎞ T T (S12 S12)−1 S12 T (S12 S12)−1 ⎠ = ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎞ 0 0 0 −Ir−p Ω 0 T Ω 0 ΩΩT ⎟ ⎟ ⎟ ⎟. ⎟ ⎠ From the definitions of B and A, ⎛ B= ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ B .. . 1 Br ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ = ⎛ ⎞ ⎜ S −1 ⎝ A ⎟ 0 ⎠ ⎛ = ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎞ 0 ⎟ ⎟ ⎟ −A2 ⎟ ⎟ ⎠ ΩA1 where AT = (A1, · · · Ar )T =: (AT1 , AT2 )T . Here A1 = (A1, · · · , Ap)T and A2 = (Ap+1 , · · · , Ar )T constitute a partition of vector A. The special form of S −1 given early means that for positive integers k and a B k = 0 for k ≤ p; B p+a = −Ap+a for a ≤ r − p, and B r+k = ω kl Al for k ≤ p. (A.1) Let B1 = (B 1, · · · , B r )τ and B2 = (B r+1 , · · · , B r+p )τ . Since SB = (Aτ , 0τp×1 )τ which means that −B1 + S12 B2 = A. As S12 = (γ j,k )r×p and from (A.1) we have γ j,k B r+k = Aj I(j ≤ p) (A.2) where I(·) is the indicator function. Since ⎛ ⎜ (B u,q )r+p×r+p = S −1 ⎝ S21(B j,k )r×p = (C k,m )τp×r ⎞ −(A ) (C ) ⎟ ij (C i,l)T i,l 0 ⎠, (A.3) and S21(B j,r+a )r×p = 0. As S21 = (γ j,k )τ , these mean γ j,k B j,l = C l,k for l ≤ r and k ≤ p and γ j,k B j,r+a = 0. 15 (A.4) Furthermore, (A.3) also implies the following which bridges B s,t with Ajm and C j,m : ⎛ ⎞ k,l ⎜ (B ) ⎜ ⎜ ⎜ ⎜ ⎝ (B k,r+l ) ⎟ ⎟ ⎟ (B p+a,l) (B p+a,p+b) (B p+a,r+l) ⎟ ⎟ ⎠ (B r+k,l ) (B r+k,p+b ) (B r+k,r+l ) ⎛ = (B k,p+b ) (ω ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ mk C l,m ) ⎞ (ω (Ap+a,l) mk C p+b,m ) 0 −(C p+a,l) (Ap+a,p+b ) (ω km (ω nm C l,n − Aml )) (ω km (ω nm C p+b,n − Am,p+b )) (ω km C m,l) ⎟ ⎟ ⎟ ⎟. ⎟ ⎠ It can be also shown that for a fixed h ∈ {1, · · · , r} ⎛ ⎞ ⎜ 0 ⎜ ⎜ ⎛ = ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ Ω 0 0 0 ∂ 2 Q1 ∂λh∂ψ ∂ 2 Q2 ∂λh∂ψ ⎞ ⎟ ⎠ ⎞ T 0 Ω 0 −Ir−p Ω ∂ 2 Q1 ∂λh∂λ ⎟ ⎜ ⎟×E⎝ 2 ⎟ ∂ Q2 ⎠ ∂λh∂λ T ΩΩ (β s,th ) = ⎜ 0 −Ir−p ⎜ ⎝ ⎛ ΩT ⎟ ⎟ 0 0 ΩΩT 0 klh (2α ⎟ ⎟ ⎟ ⎟× ⎟ ⎠ ⎞ kp+bh ) (2α (2αp+alh ) −(γ ) (2αp+ap+bh ) −(γ h;k,m + γ k;h,m )T −(γ h;p+a,m + γ p+a;h,m )T (β s,tr+k ) = ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛ = ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎞ 0 0 0 −Ir−p Ω 0 0 0 0 −Ir−p Ω 0 T Ω 0 ΩΩT T Ω 0 ΩΩT ⎛ ⎟ ⎟ ⎜ ⎟ ⎟×E⎝ ⎟ ⎠ ⎞ ⎟ ⎟ ⎟ ⎟× ⎟ ⎠ 16 ∂ 2 Q1 ∂ψk ∂λ ∂ 2 Q1 ∂ψk ∂ψ ∂ 2 Q2 ∂ψk ∂λ ∂ 2 Q2 ∂ψk ∂ψ +γ k;h,m ) ⎟ ⎟ h;p+a,m p+a;h,m ⎟ ; −(γ +γ ) ⎟ ⎟ ⎠ (γ h,kl ) Similarly for a fixed k ∈ {1, · · · , p} ⎛ h;k,m ⎞ ⎟ ⎠ ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎞ −(γ l,k;m + γ l;m,k ) −(γ l,k;p+a + γ l;p+a,k ) (γ l,mk ) ⎟ ⎟ ⎟ . −(γ p+a,k;m + γ p+a;m,k ) −(γ p+a,k;p+b + γ p+a;p+b,k ) (γ p+a,mk ) ⎟ ⎟ (γ l,mk )T (γ p+a,mk )T 0 ⎠ It may be shown from the above equation that β l,p+a p+c = −ω ol (γ p+c;p+a,o + γ p+a;p+c,o ), β l,r+m β p+a,p+b p+c = −2αp+a = γ p+c;p+a,m + γ p+a;p+c,m , β l,p+a r+n = ω ol γ p+a;on , β p+a,p+b r+n = γ p+a,n;p+b + γ p+a;p+b,n , β r+k,p+a p+c = 2ω ko αo β r+k,p+a r+n = ω ko ω mo γ p+a,mn − ω ko (γ o,n;p+a + γ o;p+a,n ) and β r+k,r+m p+c = ω ko ω no γ p+c,nm − ω ko (γ p+c;o,m + γ o,p+c;m ). p+b p+c , β p+a,r+m p+c β l,r+m r+n = 0, p+a p+c p+c β r+k,r+m β p+a,r+m r+n = ω ol γ p+c;om r+n = ω ko γ o,mn , = −γ p+a,mn − ω ko ω no (γ p+c;p+a,n + γ p+a;p+c,n ), Derivation of (7) Now we are ready to expand n−1 l(θ0) = n−1 n i=1 [λ̃T wi (ψ̃) − 12 {λ̃T wi (ψ̃)}2 + 13 {λ̃T wi (ψ̃)}3 − 14 {λ̃T wi (ψ̃)}4] + Op (n−5/2 ) By substituting expansions for λ̃ and ψ̃ given in (5) and (6) into the above equation, we have for a, b, c, d, e ∈ {1, 2, · · · , r − p}, f, g, h, i, j ∈ {1, 2, · · · , r}, k, l, m, n, o ∈ {1, 2, · · · , p} and q, s, t, u ∈ {1, 2, · · · , r + p}: n−1 l(θ0) = −2B j Aj − B j B j + 2B j,q B q (Aj + B j ) − β j,uq B u B q (Aj + B j ) − 2B j,u B u,q B q (Aj + B j ) + β u,qs B j,u B q B s (Aj + B j ) − β j,uq β u,st B q B s B t(Aj + B j ) − B j,uq B u B q (Aj + B j ) + 13 β j,uqs B u B q B s (Aj + B j ) + 2β j,uq B u,s B s B q (Aj + B j ) + 2γ j,k − B j,q B q B r+k + − 1 u,qs j,u q s r+k β B B B B 2 + 1 j,uq u q r+k B B B B 2 1 j,uq u q r+k β B B B 2 + B j,u B u,q B q B r+k − β j,uq B u,s B q B s B r+k + 12 β j,uq β u,st B q B s B tB r+k − 16 β j,uqs B u B q B s B r+k − 12 β j,uq B u B q B r+k,s B s [2, j, r + k] 17 + B j,u B u B r+k,q B q + 14 β j,uq β r+k,st B u B q B s B t + 2C j,k B j B r+k − B j,q B q B r+k [2, j, r + k] + 12 β j,uq B u B q B r+k [2, j, r + k] + γ j,kl {−B j B r+k B r+l + B j,q B r+k B r+l B q [3, j, r + k, r + l] − 1 j r+k r+l,uq u q B B β B B [3, j, r 2 + 1 j,klm j r+k r+l r+m γ B B B B 3 + k, r + l]} − C j,kl B j B r+k B r+l − B j,u B u B j,q B q − 14 β j,uq β j,stB u B q B s B t + β j,uq B u B q B j,s B s − B j B i Aji + B j B i,pB pAji [2, j, i] − 12 β j,uq B u B q B i Aji[2, j, i] + 2γ j;i,l B j B i B r+l − B j B i B r+l,q B q + 12 β r+l,uq B j B i B u B q − B r+lB i B j,q B q [2, j, i] + 1 j,uq u q i r+l β B B B B [2, j, i] 2 − 2 jih j i h α B BB 3 + 2B j B iB r+l C j;i,l − (γ j;i,lk + γ j,l;i,k )B j B i B r+l B r+k + 2αjih B j B i B h,q B q − αjih β j,uq B u B q B iB h − 23 Ajih B j B iB h + 2γ j;i;h,k B j B iB h B r+k − 12 αjihg B j B iB h B g + Op (n−5/2 ) By using those formulae given at the beginning of the appendix, it is shwon in Chen and Cui (2002) that n−1 (θ0 ) = −2Aj B j − B j B j − Aji B j B i + 2C j,k B j B r+k + 2γ j;i,l B j B iB r+l − 23 αjih B j B i B h − γ j,kl B j B r+k B r+l + Aji B i,q B q B j [2, i, j] − B j,u B j,q B u B q − 2C j,k B j,q B r+k B q + γ j,kl B r+k B r+l B j,q B q + 2γ j,kl B j B r+l B r+k,q B q − 2γ j;i,l (B j B iB r+l,q B q + B r+l B iB j,q B q [2, j, i]) + 2αjih B j B iB h,q B q + ( 12 β j,uq β r+k,st γ j,k − 14 β j,uq β j,st )B u B q B sB t + (γ i;j,k β i,uq + γ j;i,k β i,uq − γ j,kl β r+l,pq )B u B q B j B r+k − 12 γ j,kl β j,uq B u B q B r+l B r+k − (γ j;i,lk + γ j,l;i,k )B j B i B r+l B r+k + 13 γ j,klm B j B r+k B r+lB r+m + 2γ j;i;h,k B j B i B h B r+k − 12 αjihg B j B i B h B g + (γ j;i,lβ r+l,uq − αjih β h,uq )B j B iB u B q − C j,klB j B r+k B r+l + 2C j;i,l B j B i B r+l − 23 Ajih B j B i B h + Op (n−5/2). (A.5) It can be shown from (A.1), (A.2) and (A.4) that −2Aj B j − B j B j = Ap+aAp+a and −Aji B j B i + 2C j,k B j B r+k + 2γ j;i,l B j B iB r+l − 23 αjih B j B iB h − γ j,kl B j B r+k B r+l = −Ap+a p+b Ap+a Ap+b − 2ω kl C p+a,k Ap+a Al + 2γ p+a;p+b,k ω kl Ap+a Ap+b Al 18 + 2 p+a p+b p+c p+a p+b p+c α A A A 3 + γ p+a,kl ω km ω ln Ap+aAm An . These and (A.5) readily imply (7). Derivations of (12). p+b p+c p+d p+a p+b p+c p+d The fives terms involved in E(Rp+a 2 R1 R1 R1 )[4] − E(R2 R1 )E(R1 R1 )[12] are respectively p+b − 12 {E(Ap+a p+b Ap+b Ap+b Ap+c Ap+d )[4] − E(Ap+a Ap+b Ap+b )E(Ap+c Ap+d )[12]} = n−3 (−6t1 + 2t2 − 12 t3 − 2t4 ) + O(n−4 ); − ω kl {E(C p+a,k AlAp+b Ap+c Ap+d )[4] − E(C p+a,k Al Ap+b )E(Ap+c Ap+d )[12]} = −n−3 {ω kl (γ p+a,k;l αp+b + γ p+a,k;p+d αl p+b p+c p+c p+d + γ p+a,k;p+b αl p+c p+d + γ p+a,k;p+c αl p+b p+d )}[4] + O(n−4 ); γ p+a;p+b ,k ω kl {E(Ap+b AlAp+b Ap+c Ap+d )[4] − E(Ap+b AlAp+b )E(Ap+c Ap+d )[12]} = n−3 {ω kl (γ p+a;p+b,k αl p+c p+d + γ p+a;p+c,k αl p+b p+d 1 p+a p+b p+c α {E(Ap+b Ap+c Ap+b Ap+c Ap+d )[4] 3 + γ p+a;p+d,k αl p+b p+c )}[4] + O(n−4 ); − E(Ap+b Ap+c Ap+b )E(Ap+c Ap+d )[12]} = n−3 ( 13 t3 + 83 t4) + O(n−4 ) and 1 p+a,kl km ln γ ω ω {E(Am An Ap+b Ap+c Ap+d )[4] 2 = n−3 ( 12 γ p+a,kl ω km ω lm αp+b p+c p+d − E(Am An Ap+b )E(Ap+c Ap+d )[12]} [4]) + O(n−4 ). Combining these five terms give (12). Derivations of (13). p+b Since Rp+a = Ap+a , the terms involved is closely related to 15 terms in Rp+a 1 2 R2 . The terms involved are E(Ap+a p+b Ap+b p+c Ap+b Ap+c Ap+c Ap+d )[6] − E(Ap+ap+b Ap+bp+c Ap+b Ap+c ) ×E(Ap+c Ap+d )[6] = n−3 (12t1 − 3t2 + 6t3 + 13t4 ) + O(n−4 ); 19 ω mn {E(C p+a,k C p+b,m Al An Ap+c Ap+d )[6] − E(C p+a,k C p+b,m AlAn )E(Ap+c Ap+d )[6]} = n−3 {ω ml (γ p+b,m;p+d γ p+a,k;p+c + γ p+a,k;p+d γ p+b,m;p+c )[6]} + O(n−4 ); ω mn γ p+a;p+b ,k γ p+b;p+c ,m {E(Ap+b Ap+c Al An Ap+c Ap+d )[6] − E(Ap+b Ap+c AlAn ) ×E(Ap+c Ap+d )[6] = n−3 {ω ml (γ p+a;p+c,k γ p+b;p+d,m + γ p+a;p+d,k γ p+b;p+c,m )[6]} + O(n−4 ); 1 p+a p+b p+c p+bp+d p+e α α {E(Ap+b Ap+c Ap+d Ap+e Ap+c Ap+d )[6] 9 ×E(Ap+c Ap+d )[6]} = n−3 ( 23 t3 + 16 t ) 9 4 − E(Ap+b Ap+c Ap+d Ap+e ) + O(n−4 ); E(Am An Am An Ap+c Ap+d )[6] − E(Am An Am An )E(Ap+c Ap+d )[6]} = O(n−4 ); ω kl {E(C p+a,k Ap+b p+b Ap+b AlAp+c Ap+d )[6] − E(C p+a,k Ap+b p+b = n−3 {ω kl (γ p+a,k;p+c αlp+bp+d + γ p+a,k;p+d αlp+bp+c + 2γ p+a,k;l αp+b ω kl γ p+a;p+c ,k {E(Ap+b p+b Ap+b Ap+c Al Ap+c Ap+d )[6] − E(Ap+b ×E(Ap+c Ap+d )[6]} = n−3 {ω kl (γ p+a;p+d,k αl − 16 αp+a p+b p+c p+b p+c Ap+b Al )E(Ap+c Ap+d )[6]} p+c p+d p+b + γ p+a;p+c,k αl )[6]} + O(n−4 ); Ap+b Ap+c Al) p+b p+d )[6]} + O(n−4 ); {E(Ap+bp+d Ap+b Ap+c Ap+d Ap+c Ap+d )[2, a, b][6] −E(Ap+bp+d Ap+b Ap+c Ap+d )E(Ap+c Ap+d )[2, a, b][6]} = n−3 (−2t3 − ω km ω ln γ p+a,kl {E(Ap+b p+b Ap+b Am An Ap+c Ap+d )[6] − E(Ap+b p+b 16 t) 3 4 + O(n−4 ); Ap+b AmAn ) ×E(Ap+c Ap+d )[6]} = n−3 (2ω km ω lm γ p+a,kl αp+b p+c p+d [6]) + O(n−4 ); ω kl ω mn γ p+a;p+b ,m {E(C p+b,k Ap+b AlAn Ap+c Ap+d )[6] − E(C p+b,k Ap+b AlAn )E(Ap+c Ap+d )[6]} = n−3 {ω kl ω ml (γ p+a;p+d,m γ p+b,k;p+c + γ p+a;p+c,m γ p+b,k;p+d )[6]} + O(n−4 ); 20 ω kl αp+a p+b p+c = n−3 (2ω kl αp+a {E(C p+b,k Ap+b Ap+c AlAp+c Ap+d )[6] − E(C p+b,k Ap+b Ap+c Al )E(Ap+c Ap+d )[6] p+c p+d p+b,k;l γ [6]) + O(n−4 ); E(C p+b,o AmAn Av Ap+c Ap+d )[6] − E(C p+b,o AmAn Av )E(Ap+c Ap+d )[6] = O(n−4 ); E(Ap+b Ap+c Ap+d AlAp+c Ap+d )[6] − E(Ap+b Ap+c Ap+d Al )E(Ap+c Ap+d )[6] = O(n−4 ); E(Ap+b AmAn Av Ap+c Ap+d )[6] − E(Ap+b Am An Av )E(Ap+c Ap+d )[6] = O(n−4 ); ω km ω ln αp+a p+b p+c p+b,kl γ {E(Ap+b Ap+c Am An Ap+c Ap+d ) − E(Ap+b Ap+c Am An ) ×E(Ap+c Ap+d )} = n−3 (2ω km ω lm αp+a p+c p+d p+b,kl γ ) + O(n−4 ). Derivation of (14). Due to the fact that E(Al Ap+a) = 0 for l ∈ {1, · · · , p}, there is no need to compute terms in Rp+a involving Al . It may be shown that 3 3 {E(Ap+a p+c Ap+c p+b Ap+b Ap+bAp+c Ap+d )[4] 8 − E(Ap+a p+c Ap+c p+b Ap+b Ap+b ) ×E(Ap+cAp+d )[12]} = n−3 (3t4 ) + O(n−4 ); ω ml {E(C p+b ,m Alp+a Ap+b Ap+b Ap+c Ap+d )[4] − E(C p+b ,m Alp+a Ap+b Ap+b ) ×E(Ap+c Ap+d )[12]} = n−3 [ω ml {(γ p+d,m;p+c + γ p+c,m;p+d )αl + (γ p+b,m;p+c + γ p+c,m;p+b )αl p+a p+d p+a p+b + (γ p+b,m;p+d + γ p+d,m;p+b )αl p+a p+c }[4]] + O(n−4 ); ω ml ω nl {E(C p+a,m C p+b ,n Ap+b Ap+b Ap+c Ap+d )[4] − E(C p+a,m C p+b ,n Ap+b Ap+b ) ×E(Ap+c Ap+d )[12]} = n−3 [ω ml ω nl {γ p+a,m;p+c (γ p+b,n;p+d + γ p+d,n;p+b ) + γ p+a,m;p+b (γ p+c,n;p+d + γ p+d,n;p+c ) +γ p+a,m;p+d (γ p+b,n;p+c + γ p+c,n;p+b )}[4]] + O(n−4 ); 21 −αp+a p+b l ω nl {E(C p+c ,n Ap+b Ap+c Ap+b Ap+c Ap+d )[4] − E(C p+c ,n Ap+b Ap+c Ap+b ) ×E(Ap+c Ap+d )[12]} = n−3 [−ω nl {(γ p+d,n;p+c + γ p+c,n;p+d )αl + (γ p+b,n;p+c + γ p+c,n;p+b )αl − 56 αp+a p+b p+c p+a p+d p+a p+b + (γ p+b,n;p+d + γ p+d,n;p+b )αl p+a p+c }[4]] + O(n−4 ); {E(Ap+c p+d Ap+b Ap+d Ap+aAp+c Ap+d )[4] − E(Ap+c p+d Ap+b Ap+d Ap+a ) t ) + O(n−4 ); ×E(Ap+c Ap+d )[12]} = n−3 (− 20 3 4 1 {E(Ap+a p+b p+c Ap+b Ap+c Ap+bAp+c Ap+d )[4] 3 − E(Ap+a p+b p+c Ap+b Ap+c Ap+b ) ×E(Ap+c Ap+d )[12]} = n−3 (8t1 ) + O(n−4 ); (αl p+a p+b − 14 αp+a p+b p+e p+e p+c p+d ω kl γ p+c ;p+d ,k + 49 αp+a p+b p+c p+d α − 12 ω kl ω ml γ p+a;p+b ,k γ p+c ;p+d ,m ) × {E(Ap+b Ap+c Ap+d Ap+b Ap+c Ap+d )[4] − E(Ap+b Ap+c Ap+d Ap+b ) ×E(Ap+c Ap+d )[12]} = n−3 [−6t1 + 32 t 9 4 + ω kl {(γ p+d,k;p+c + γ p+c,k;p+d )αl + (γ p+b,k;p+d + γ p+d,k;p+b )αl − p+a p+c 1 kl ml ω ω {γ p+a;p+b,k (γ p+c;p+d,m 2 p+a p+b + (γ p+b,k;p+c + γ p+c,k;p+b )αl p+a p+d }[4] + γ p+d;p+c,m ) + γ p+a;p+c,k (γ p+b;p+d,m + γ p+d;p+b,m ) + γ p+a;p+d,k (γ p+b;p+c,m + γ p+c;p+b,m )}[4]] + O(n−4 ); γ p+a;p+b ,l ω ln ω on {E(C p+c ,o Ap+b Ap+c Ap+b Ap+c Ap+d )[4] − E(C p+c ,o Ap+b Ap+c Ap+b ) ×E(Ap+cAp+d )[12]} = n−3 [ω ln ω on {γ p+a;p+b,l(γ p+c,o;p+d + γ p+d,o;p+c ) + γ p+a;p+c,l (γ p+b,o;p+d + γ p+d,o;p+b ) + γ p+a;p+d,l (γ p+b,o;p+c + γ p+c,o;p+b )}[4]] + O(n−4 ); −γ p+a;p+b ,l ω ln {E(An,p+c Ap+b Ap+c Ap+b Ap+c Ap+d )[4] − E(An,p+c Ap+b Ap+c Ap+b ) ×E(Ap+c Ap+d )[12]} = n−3 {−2ω ln (γ p+a;p+b,l αn p+c p+d + γ p+a;p+c,l αn + O(n−4 ). 22 p+b p+d + γ p+a;p+d,l αn p+b p+c )[4]} Derivations of (16). p+e p+a p+e The main task in deriving (16) is to work out E(Rp+a 2 R2 ) and E(R1 R3 ) for a, e ∈ p+a {1, · · · , r − p}. There are 15 terms in Rp+a 2 R2 . Those 15 terms and their corresponding expectations, denoted in Jiae for i = 1, · · · , 15 are reported below: (1). 14 Ap+a p+b J1ae = 14 [(αp+a Ap+e p+c Ap+b Ap+c , p+e p+b p+b − δ ae) + αp+a p+b p+b p+e p+c p+c α + αp+a p+b p+c p+e p+b p+c α (2). ω kl ω mn C p+a,k C p+e,m AlAn , J2ae = ω kl ω ml γ p+a,k;p+e,m + ω kl ω mn (γ p+a,k;l γ p+e,m;n + γ p+a,k;n γ p+e,m;l ); (3). ω kl ω mn γ p+a;p+b,k γ p+e;p+c,m Ap+b Ap+c Al An , J3ae = ω kl ω ml γ p+a;p+b,k γ p+e;p+b,m ; (4). 19 αp+a p+b p+c J4ae = 19 [αp+a αp+e p+d p+e Ap+b Ap+c Ap+d Ap+e , p+b p+b p+e p+d p+d α + 2αp+a p+b p+c αp+e p+b p+c ]; (5). 14 ω km ω ln ω k m ω l n γ p+a,kl γ p+e,k l AmAn Am An , J5ae = 14 γ p+a,kl γ p+e,k l [ω km ω lm ω k m ω l m + ω km ω ln (ω k m ω l n + ω k n ω l m )] (6). 12 ω kl C p+a,k Ap+e p+b J6ae = 12 ω kl (γ p+a,k;p+b αl Ap+b Al [2, a, e], p+e p+b (7). − 12 ω kl γ p+a;p+c,k Ap+e J7ae = − 12 ω kl γ p+a;p+b,k αl (8). − 16 αp+a p+b p+c J8ae = − 16 (2αp+a p+d αp+e (9). − 14 ω km ω ln γ p+a,kl Ap+e [2, a, e]; p+b p+c + αp+a p+b p+b αp+e Ap+b Am An [2, a, e], p+b p+b [2, a, e]; (10). −ω kl ω mn γ p+a;p+b,m C p+e,k Ap+b Al An [2, a, e],) ae = −ω kl ω ml γ p+a;p+b,m γ p+b;p+e,k [2, a, e]; J10 (11). − 13 ω kl αp+a p+b p+c )[2, a, e]; Ap+bAp+c Ap+d [2, a, e], p+b J9ae = − 14 ω km ω lm γ p+a,kl αp+e p+b p+b Ap+b Ap+c Al[2, a, e], p+e p+b Ap+e p+b p+c p+b + γ p+a,k;lαp+e C p+e,k Ap+b Ap+c Al[2, a, e], ) 23 p+d p+d )[2, a, e]; ]; ae J11 = − 13 ω kl αp+a p+b p+b p+e,k;l γ [2, a, e]; (12). − 12 ω km ω ln ω ov γ p+a,kl C p+e,o Am An Av [2, a, e], ae = − 12 γ p+a,kl [ω km ω ln ω on γ p+e,o;m + ω km ω ln ω om γ p+e,o;n + ω lm ω ov ω km γ p+e,o;v ] [2, a, e]; J12 (13). 1 kl p+a;p+b,k p+e p+c p+d p+b p+c p+d l ω γ α A A A A [2, a, e], 3 ae = 0; J13 (14). 1 km ln ov p+a,kl p+e;p+b,o p+b m n v ω ω ω γ γ A A A A [2, a, e], 2 ae = 0; J14 (15). 1 km ln p+a p+b p+c p+e,kl p+b p+c m n ω ω α γ A A A A [2, a, e], 6 ae = 16 ω km ω lm αp+a J15 p+b p+b p+e,kl γ [2, a, e]; Note that J1ae =: J1ae + J4ae + J8ae = 14 (αp+a 7 p+a − 36 α p+b p+c p+e p+b p+c α p+e p+b p+b − δ ae ) + 1 p+a p+b p+b p+e p+c p+c α α 36 ]. J2ae = ω kl ω ml γ p+a,k;p+e,m + ω kl ω mn (γ p+a,k;l γ p+e,m;n + γ p+a,k;n γ p+e,m;l ). J3ae = ω kl ω ml γ p+a;p+b,k γ p+e;p+b,m . J5ae = 14 γ p+a,kl γ p+e,k l [ω km ω lm ω k m ω l m + ω km ω ln (ω k m ω l n + ω k n ω l m )] J6ae = ω kl (γ p+a,k;p+b αlp+ep+b + γ p+a,k;l αp+ep+bp+b )[2, a, e]. J7ae = − 12 ω kl γ p+a;p+b,k αlp+ep+b [2, a, e]. 1 km lm p+a,kl p+ep+bp+b ae = − 12 ω ω γ α [2, a, e] J9ae + J15 ae ae = −ω kl ω ml γ p+e;p+b,m γ p+b;p+a,k [2, a, e]. J11 = − 13 ω kl αp+a J10 p+b p+b p+e,k;l γ [2, a, e]. ae = − 12 γ p+a,kl [ω km ω ln ω on γ p+e,o;m + ω km ω ln ω om γ p+e,o;n J12 +ω lm ω ov ω km γ p+e,o;v ][2, a, e]. Combine the above terms, we arrive at p+e −2 ae −3 E(Rp+a 2 R1 )[2, a, e] = n J16 + O(n ) where ae = −(αp+a J16 p+e p+b p+b − δ ae ) − ω kl γ p+a,k;l;p+e [2, a, e] + ω kl γ p+a;p+b,k αl 24 p+b p+e [2, a, e] + 2 p+a α 3 p+b p+c p+e p+b p+c α 1 + ω km ω ln γ p+a,kl αmn 2 p+e [2, a, e]. p+e ae There are 25 terms in Rp+a 3 R1 , whose expectations are denoted by J16+i for i = 1, · · · , 25. ae = 34 [(αp+a J17 p+e p+c p+c − δ ae ) + αp+a ae = ω ml [γ p+e,m;l;p+a + γ p+b,m;p+b αl J18 ae = 12 ω lm [γ p+b,l;mαp+a J19 p+e p+b p+b p+c p+a p+e p+b p+c + αp+a p+e p+c + γ p+b,m;p+e αl p+a p+b ][2, a, e], + γ p+b,l;p+e αm αp+e p+a p+b αp+b p+b p+c ][2, a, e], ae J20 = − 12 ω ml ω nl [γ p+a,m;p+e,n + γ p+a,m;p+b γ p+b,n;p+e + γ p+a,m;p+e γ p+b,n;p+b ][2, a, e], ae = ω ml ω kn [γ l,k;n γ p+a,m;p+e + γ l,k;p+e γ p+a,m;n ][2, a, e], J21 ae = −ω nl [(γ p+e,n;p+b + γ p+b,n;p+e )αl J22 ae = −2ω mn γ p+c,m;n αp+a J23 ae J24 = − 53 [2αp+a p+e p+c ae = 12 ω km ω lm γ p+a,kl;p+e [2, a, e], J25 ae = 2αp+a J27 p+e p+b p+b ae J28 = − 32 αp+a αp+a + 16 9 +2αl p+a p+e p+a p+e ][2, a, e], + αp+a p+e p+c αp+c p+d p+d ], ae J26 = ω lm γ p+a;p+e,l;m [2, a, e], , p+e p+b p+b p+b p+e + γ p+c,n;p+c αl , p+b p+c p+e p+b p+c α p+a p+b αp+e + ω kl (γ p+b;p+e,k + γ p+e;p+b,k )αl p+b p+e p+a p+b [2, a, e] − 12 ω kl ω ml γ p+a;p+b,k (γ p+b;p+e,m + γ p+e;p+b,m )[2, a, e] ω kl γ p+c;p+c,k + 89 αp+a p+e p+e αp+e p+c p+c − 12 ω kl ω ml γ p+a;p+e,k γ p+c;p+c,m [2, a, e], ae J29 = − 12 ω kn ω ln ω vm γ m,kl γ p+a,v;p+e [2, a, e], ae = ω lo ω kn ω vn γ p+e,kl γ p+a,v;o [2, a, e], J31 ae J30 = − 12 γ p+b,kl ω km ω lm αp+a ae J32 = −ω lo ω kn γ p+e,kl αon p+a p+e p+b [2, a, e], ae = −ω ln ω mn ω kv γ p+a,kl γ v,m;p+e [2, a, e], J33 ae = ω ln ω on [γ p+a;p+b,l(γ p+e,o;p+b + γ p+b,o;p+e ) + γ p+a;p+e,l γ p+c,o;p+c ][2, a, e], J34 ae J35 = −ω ln [2γ p+a;p+b,l αnp+bp+e + γ p+a;p+e,l αnp+cp+c ][2, a, e], ae = −ω mn ω lo γ p+a;p+e,r+m γ o,m;n [2, a, e], J36 ae = −ω ln ω om (γ m;p+a,l + γ p+a;m,l )γ p+e,o;n [2, a, e], J37 ae = − 12 ω ln (γ p+c;p+a,l + γ p+a;p+c,l )αnp+ep+c [2, a, e], J38 25 , ], ae J39 = −ω ln ω kn (γ p+a;p+b,l + γ p+a;p+c,l )γ p+b,k;p+e [2, a, e], ae J40 = 13 ω kl γ p+c,k;l αp+a ae = ω m m ω n m [ 13 αp+a J41 p+e p+c [2, a, e] and p+e p+c p+c,m n + 12 γ p+c,m ;p+a (γ p+c;p+e,n + γ p+e;p+c,n ) γ + 12 γ p+a,m ;p+c γ p+e;p+c,n + ω lo γ p+e,n l (γ o,m ;p+a + γ o;p+a,m ) + 12 ω lo γ o,m n γ p+a;p+e,l − 12 ω ol ω kl γ p+a,m o γ p+e,n k − 12 γ p+a;p+e,m n − 12 γ p+a,m ;p+e,n ][2, a, e]. In summary, we have: 1 ae ae ae ae ae ae J42 =: J1ae + J16 + J17 + J24 + J27 + J28 = αp+a p+e p+b p+b − 13 αp+a 2 1 p+a p+b p+b p+e p+c p+c 1 p+a p+e p+c p+b p+b p+c + 36 α α − 36 α α − ω kl γ p+a,k;l;p+e [2, a, e] + 12 ω km ω ln γ p+a,kl αmn + ω ml [γ p+b;p+b,m αl − p+a p+e p+e αp+e [2, a, e] + (γ p+b;p+e,m + 2γ p+e;p+b,m )αl 1 ml nl p+a;p+e,n p+b;p+b,m ω ω [γ γ 2 p+b p+c p+a p+b ][2, a, e] + γ p+a;p+b,n (γ p+b;p+e,m + γ p+e;p+b,m )][2, a, e], ae ae ae ae ae ae ae J43 =: J18 + J19 + J20 + J22 + J23 + J26 = ω ml [2γ p+e,m;l;p+a + ( 12 γ p+b,m;p+e − γ p+e,m;p+b )αl − 1 ml nl p+a,m;p+e,n ω ω [γ 2 − ω mn γ p+b,m;n αp+a p+a p+b ][2, a, e] + γ p+a,m;p+b γ p+b,n;p+e + γ p+a,m;p+e γ p+b,n;p+b ][2, a, e]. p+e p+b , ae ae ae ae ae ae J44 =: J3ae + J6ae + J7ae + J9ae + J15 + J10 + J11 + J35 + J38 = ω kl ω ml γ p+a;p+b,k γ p+e;p+b,m + ω kl [( 12 γ p+e,k;p+b − 3γ p+e;p+b,k )αl + 2 p+e,k;l p+a p+b p+b γ α 3 − γ p+a;p+e,k αl 1 km lm p+e,kl p+a ω ω γ α − 12 p+b p+b p+b p+b p+a p+b ][2, a, e] [2, a, e] − ω kl ω ml γ p+e;p+b,m γ p+b;p+a,k [2, a, e], ae ae ae ae ae ae ae ae ae ae ae J45 =: J29 + J30 + J31 + J32 + J33 + J34 + J36 + J37 + J39 + J40 = − 12 ω kn ω ln ω vm γ m,kl γ p+a,v;p+e [2, a, e] − 12 γ p+b,kl ω km ω lm αp+a p+e p+b . + ω kv ω ln ω mn (γ p+a,m;v − γ v,m;p+a )γ p+e,kl [2, a, e] − ω lm ω kn γ p+e,kl αmn p+a [2, a, e] − ω mn ω lo γ p+a;p+e,r+m γ o,m;n [2, a, e] + ω kn ω ln [γ p+a;p+b,k (γ p+e,l;p+b − γ p+e;p+c,l ) + γ p+a;p+e,k γ p+b,l;p+b ][2, a, e] 26 p+b p+c − ω ln ω om (γ m;p+a,l + γ p+a;m,l )γ p+e,o;n [2, a, e] + 13 ω kl γ p+b,k;l αp+a p+e p+b [2, a, e], ae ae ae ae ae J46 =: J2ae + J5ae + J12 + J21 + J25 + J41 = ω kl ω mn (γ p+a,k;l γ p+e,m;n + γ p+a,k;n γ p+e,m;l ) + [ 14 ω km ω lm ω k n ω l n − 12 ω km ω ln ω k m ω l n ]γ p+a,klγ p+e,k l − 1 p+a,kl km ln on p+e,o;m γ [ω ω ω γ 2 + ω km ω ln ω om γ p+e,o;n + ω lm ω ov ω km γ p+e,o;v ][2, a, e] + 12 ω lo γ o,m n γ p+a;p+e,l ][2, a, e] + ω ml ω kn [γ l,k;n γ p+a,m;p+e + γ l,k;p+e γ p+a,m;n ][2, a, e] + ω m m ω n m [ 13 αp+a + p+e p+b p+b,m n γ 1 p+a,m ;p+b p+e;p+b,n γ γ 2 1 p+a p+b p+b p+e p+c p+c α α 36 − p+e p+b p+b p+a p+e − 13 αp+a p+b p+c αp+e p+b p+c 1 p+a p+e p+c p+b p+b p+c α α 36 − ω kl γ p+a,k;l;p+e [2, a, e] − 12 ω km ω ln γ p+a,kl αmn + ω ml [γ p+b;p+b,m αl + ω lo γ p+e,n l (γ o,m ;p+a + γ o;p+a,m ) and ae ae ae ae ae ae J47 =: J42 + J43 + J44 + J45 + J46 = 12 αp+a + + 12 γ p+b,m ;p+a (γ p+b;p+e,n + γ p+e;p+b,n ) p+e [2, a, e] + 12 (γ p+b;p+e,m − γ p+e;p+b,m )αl p+a p+b ][2, a, e] − ω ml ω nl γ p+a;p+b,n γ p+e;p+b,m + ω kl ω mn γ p+a,k;lγ p+e,m;n + 2ω ml γ p+e,m;l;p+a [2, a, e] − ω ml ω nl γ p+a,m;p+e,n − 13 ω mn γ p+b,m;n αp+a + ω kl [ 23 γ p+e,k;l αp+a − p+b p+b − γ p+a;p+e,k αl p+b p+b p+e p+b ][2, a, e] 1 km lm p+e,kl p+a p+b p+b ω ω γ α [2, a, e] + 16 γ p+b,kl ω km ω lm αp+a p+e p+b . 12 + ω kv ω ln ω mn γ p+a,m;v γ p+e,kl [2, a, e] − 12 γ p+a,kl ω lm ω ov ω km γ p+e,o;v ][2, a, e] + [ 14 ω km ω lm ω k n ω l n − 12 ω km ω ln ω k m ω l n ]γ p+a,klγ p+e,k l . Then, we get ae cum(Rp+a , Rp+e ) = n−1 δ ae + n−2 (J47 − μp+a μp+e ) + O(n−3 ) =: n−1 δ ae + n−2 Δae + O(n−3 ) where Δae = 1 p+a p+e p+b p+b α 2 − 13 αp+a p+b p+c αp+e p+b p+c − ω kl γ p+a,k;l;p+e [2, a, e] − 12 ω km ω ln γ p+a,kl αmn 27 − p+e 1 p+a p+e p+c p+b p+b p+c α α 36 [2, a, e] + ω ml [γ p+b;p+b,m αl p+a p+e + 12 (γ p+b;p+e,m − γ p+e;p+b,m )αl p+a p+b ][2, a, e] − ω ml ω nl γ p+a;p+b,n γ p+e;p+b,m + ω kl ω mn γ p+a,k;l γ p+e,m;n + 2ω ml γ p+e,m;l;p+a [2, a, e] − ω ml ω nl γ p+a,m;p+e,n − 13 ω mn γ p+b,m;n αp+a + ω kl [ 23 γ p+e,k;l αp+a p+b p+b − γ p+a;p+e,k αl p+b p+b p+e p+b ][2, a, e] + 1 p+b,kl km lm p+a p+e p+b γ ω ω α 6 − 1 p+a,kl lm ov km p+e,o;v γ ω ω ω γ ][2, a, e] − 12 ω km ω ln ω k m ω l n γ p+a,kl γ p+e,k l . 2 + ω kv ω ln ω mn γ p+a,m;v γ p+e,kl [2, a, e] 28