1 The Probabilities of Large Deviations for the Chi-square and Log-likelihood Ratio Statistics Sherzod Mirakhmedov Institute of Mathematics. National University of Uzbekistan 100125, Tashkent, Durmon yuli st., 29 e-mail: shmirakhmedov@yahoo.com Abstract. A new large deviation results for the Pearson’s chi-square and Log-likelihood ratio statistics are obtained. Here attention is focused on the case when the number of groups increases to infinity and the probabilities of groups’ decreases to zero, as the sample size tends to infinity. 1. Introduction. Let (1 ,...,N ) be a random vector having a multinomial distribution M n, p1 ,..., pN , where p1 ... pN 1 , pm 0 , m 1,..., N and 1 ... N n . Many tests of goodness-of-fit can be reduced to testing a hypothesis about the parameters p1 ,..., pN . Most notably test statistics for this kind of testing problem are Pearson’s chi-square ( 2 ) statistic (m npm )2 , npm m 1 N N2 and the log-likelihood ratio (LR) statistic: N m m 1 npm N 2m ln . In the present paper we study P TN xN , where TN is standardized version of the N2 or N statistics and xN , in the situation when max pm 0 , N as n . Similar problems have been considered in the literature. We refer to Kallenberg (1985), in particularly his results for the Cramer’s zone, viz. xN o( N ) , proposes that N min pm c 0 and additionally N o( n ) , nN 3/2 xN for the N2 case. We refer also to Sirajdinov et al (1989) and Ivchenko and Mirakhmedov (1995). In these papers the large deviation problem in Cramér’s zone has been studied for the class of statistics h1 (1 ) ... hN (N ) , where hm (u ) is a Borel function defined on the non-negative axis and satisfying the Cramer condition: max E exp H hk ( ) , 1 k N H 0 , here Poi( ) , the Poisson distribution with expectation lim(n / N ) (0, ) . The n Cramér’s condition is fulfilled for the N , but for the N2 does not -------------MSC (2000): 60F10, 62G10, 62G20 Key words and phrases. Chi-square statistic, Log-likelihood ration statistic, multinomial distribution, large deviations, Poisson distribution. 2 The assertions of the present paper corresponds to the case xN o( N 1/6 ) , nevertheless they covers the situations which does not covered by aforementioned papers. Our results, together with corresponding results of Kallenberg (1985), allowed to explore the intermediate asymptotic relative efficiency due to Inglot (1999) of the 2 and LR goodness of fit tests; the results are object of separate paper Mirakhmedov (2016). Also our proofs are based on the technique different from those aforementioned authors. Core part of our technique is that we reduce the cumulants of statistics of type RN to the cumulants of a sum of independent random variables (r.v.), next we use a modified variant of Lemma 2.3 of Saulis and Statulevicius (1991). The rest of the paper is organized as follows. The main results are presented in Section 2; in Section 3 the proofs are given; for the reader’s convenience, the auxiliary Assertions are collected in Appendix. In what follows Ck is an universal positive constant; all asymptotic statements are considered as n ; F stands for “ r.v has the distribution F ”. 2. Results Let pmax max pk , pmin min pk , n n / N and (u ) be the standard normal distribution 1 k N 1 k N function. 2.1. Chi-square statistic. Set n max 1, (npmin )1 , N 1 2N , m 1 npm N2 N2 1 n N 1 Np m 1 m 1 2 N N2 N n1 . Theorem 2.1. Let pmax o(1) . Then for all xN such that 0 xN o min N3 / N2 n (2.1) 1/3 1/4 , n1/6 , pmax it holds P N2 xN N N 1 ( xN ) 1 o(1) (2.2) and P N2 xN N N ( xN ) 1 o(1) . (2.3) Corollary 2.1. Let n C0 1/ 2 and C1 Npmin Npmax C2 N 1/3 , then for all xN such that 0 xN o( N 1/6 ) the relations (2.2) and (2.3) are valid. Corollary 2.2. Let pm N 1 1 (n)dm,n , m 1,..., N , where (n) 0 and N d m 1 m,n 0, 1 N 2 d m,n d 2 . N m1 (2.4) 3 Then for arbitrary n and xN such that 0 xN o N 1/6 min(1, n2/3 ) the relations (2.2) and (2.3) are valid. Remarks 2.1. For xN the relation (2.2) can be written in the form log P N2 xN N N xN2 / 2(1 o(1)) . From Kallenberg (1985, Eq. (2.17)) such kind relation follows if log N o( xN2 ) , xN o( N 1/6 ) and N o(n3/8 ) . Last condition excludes the case recommended by Mann and Wald (1942) who obtained the relation N cn2/ 5 concerning the optimal choice of the number of groups in chi-square goodness of fit test. 2.2. Likelihood ratio statistic N . Here we restrict our attention to the case n ; the corresponding results for the case n (0, ) follows from Sirajdinov et al (1987). Theorem 2.2. Let n and C3 Npmin Npmax C4n3 , (2.5) 1/4 Then for all xN such that 0 xN o min( N 1/6 , pmax ) it holds P N xN 2 N N 1 ( xN ) 1 o(1) , (2.6) and P N xN 2 N N ( xN ) 1 o(1) . (2.7) Corollary 2.3. Let n and pm N 1 1 o(1) , m 1,..., N , then for all xN such that 0 xN o( N 1/6 ) the relations (2.6) and (2.7) are valid. Remarks 2.2. Let n , Npmax O(min( N 1/3 , n3 )) , xN and xN o( N 1/6 ) . Then from (2.6) it follows that log P N xN 2 N N xN2 / 2 1 o(1) . From Kallenberg (1985, Eq. (2.13)) such relation follows under additionally conditions N o n3/7 and log N o( xn2 ) . 3. Proofs In what follows 1 ,..., N are independent r.v.s with m Poi(npm ) , Poi(n ) and k ( ) is a cumulant of the k th order of the r.v. , N RN hm m and m 1 N TN hm m , (3.1) m 1 where (1 ,..., N ) is a multinomial r.vec. as in introduction, hm ( x) , m 1,..., N are Borel functions defined on the non-negative axis. Main Lemma. Let E hm (m ) O Ehms (m ) , s (3.2) 4 E hm (m ) (m npm )2 O s a (npm )b Ehms (m ) , s (3.3) for all m 1, 2,..., N , some non-negative real a and b , and each integer s : 3 s k , where an integer k such that k p i 1 ji o(1) , (3.4) for any k-subset ( j1 ,..., jk ) of the set (1, 2,..., N ) . Then for all k satisfying (3.4) and 3 k o Kn (a, b) , b where Kn (a, b) n1b pmax 1/( a 1) (3.5) , a max(1, a) , it holds ( RN ) k k (TN ) 1 o 1 . (3.6) Proof. We start from the following Lemma 3.1. For any non-negative k it holds n ER n k N S ETNk exp i N d , n n where def n 2 nP{S N 0} 1 n !e n 1 2 nn n 2 ((1,...,N )) Proof. It is well known that 1 1 o . n ((1,..., N ) / SN 0) , where (3.7) ( X ) stands for the distribution of a random vector X , S N m npm . Hence ERNk E TNk S N 0 . On the other N m 1 hand E TNk ei SN E ei SN E TNk S N . Now Lemma 3.1 follows by Fourier inversion. Notice that Eq. (3.7) follows due to fact that S N Poi(n) and Stirling’s formula. n Let k be as in (3.4) and (3.5). Use Lemma 3.1 to write k ERNk n l ,k l l 1 n E h j1 ( j1 ) n k1 ... h jl ( jl ) kl S exp i N d , n (3.8) where l ,k is the summation over all l-tuples (k1 ,..., kl ) with positive integer components such that k1 ... kl k ; l is the summation over all l-tuples ( j1 ,..., jl ) such that ji jr for i r and jm 1,2,..., N ; m 1,2,..., l . l l i 1 i 1 Set Sl , N ( ji np ji ) , dl , N p ji , l n E h j1 ( j1 ) n k1 ... h jl ( jl ) kl 1,..., k and write S exp i N d n 5 n E h j1 ( j1 ) n n E h j1 ( j1 ) n k1 k1 ... h jl ( jl ) ... h jl ( jl ) kl S S Sl , N exp i l , N E exp i N d n n 2 Sl , N S N Sl , N exp i E exp i exp 1 dl , N d n n 2 2 exp 1 d E h j1 ( j1 ) l,N 2 n n kl E h j1 ( j1 ) k1 ... h jl ( jl ) kl k1 ... h jl ( jl ) kl Sl , N 1 d exp i n n 2 def exp 1 d d J1 J 2 J 3 . l , N 2 n We have i ( m npm ) it / E exp exp npm e n n 1 (3.9) 2 it 3 exp p pm , m n 6 n 2 (3.10) and i (m npm ) 2 pm 2 2 E exp exp 2npm sin exp 2 , n 2 n 2 2 2 because sin u / 2 u / , u . (3.11) In order to get an upper bound for the J1 we first write the integral J1 as the sum of two n / 2 and integrals, J1 and J1 say, over intervals n/2 n respectively. Next, noting that S N Sl , N is the sum of independent r.v.s m npm over the set of indexes (1,..., N ) except ( j1 ,..., jl ) we use to E exp i (S N Sl , N ) / n in the J1 and J1 the equation (3.10) and inequality (3.11) respectively. Then quite clear algebra gives n /2 3 kl k1 2 J1 E h j1 ( j1 ) ... E h jl ( jl ) exp 1 dl , N exp (1 dl , N ) 1 d 2 6 n n /2 2 S N Sl , N E exp i n exp 2 1 dl , N d n n 2 3 n /2 6 2 E h j1 ( j1 ) ... E h jl ( jl ) exp 1 dl , N 6 n (1 dl ,N )d 12 n /2 kl k1 n C5 exp 1 dl , N . 4 Hence 1 J1 O E h j1 ( j1 ) n k1 ... h jl ( jl ) kl . (3.12) since (3.2) and (3.4). Next apply inequality eit 1 it t 2 / 2 to get: for all k satisfying (3.5) 6 2 kl l k1 C6 J2 E h j ( j ) ... h jl ( jl ) ji np ji n 1 1 i 1 ki C7 l l E h ji ( ji ) ji np ji n i 1 C8 E h j1 ( j1 ) C9 E h j1 ( j1 ) h jm ( jm ) k1 ... h jl ( jl ) k1 l m i , m 1 ... h jl ( jl ) 2 k1 o 1 E h j1 ( j1 ) km kl k l a ki (np ji )b n i 1 kl (npmax )b 1 max(1,a ) k n ... h jl ( jl ) kl , (3.13) because (3.3) and the fact that k1a ... kla k max(1,a ) . By a simple algebra we obtain J 3 2 E h j1 ( j1 ) k1 ... h jl ( jl ) 1 o 1 . kl (3.14) since the condition (3.4). Now apply (3.12), (3.13) and (3.14) in the (3.9) to get E h j1 ( j1 ) k1 ... h jl ( jl ) kl S exp i N d n 2 1 o 1 E h j1 ( j1 ) k1 ... h jl ( jl ) kl . (3.15) Remark that in (3.13) and (3.14), and hence in (3.15), the o(1) is uniform in all l-tuples (k j1 ,..., k jl ) and k satisfying the condition (3.5). The relations (3.7), (3.8) and (3.15) imply ERNk ETNk 1 o 1 , (3.16) for every integer k satisfying the condition (3.5). Main Lemma follows from (3.16) and Assertion 3. Remark 3.1. The condition (3.3) is used to estimate of the integral J 2 only. Let the condition (3.3) is fulfilled with a1 , b1 for all m (1, 2,..., N ) and with a2 , b2 for all m (1, 2,..., N ) \ , then Eq. (3.6) is valid for every integer k such that 3 k o min( Kn (a1 , b1 ), Kn (a2 , b2 ) instead of (3.5). Let x stand for the integer part of x , Poi( ) and v ( ) E . v Lemma 3.2. For any integer v 2 one has v /2 v ( ) v ! cl ,v l , (3.17) l 1 where for all of l 1, 2,..., v / 2 , 0 cl ,v 1/ l ! , also (3.18) 7 (v 1)cl ,v 1 lcl ,v cl 1,v1 , (3.19) where l 1, 2,..., (v 1) / 2 if v is an even, and l 1, 2,..., (v 1) / 2 1 , (v 1)c( v1)/2,v1 c( v1)/2,v1 if v is an odd; here we put c0,v 1 0 . Proof. Apply the Bruno’s formula to Eei ( ) exp ei 1 i to get 1 v ! cl ,v l , km l 1 m 2 km !( m !) v2 v v ( ) v ! k ... k 2 where is summation over all non-negative k2 ,..., kv such that 2k2 ... vkv v and v 1 , and hence 0 cl ,v 1/ l ! . On the other hand Kenney and km m 2 km !( m !) l k2 ... kv , cl ,v Keeping (1953) give v 1 ( ) vv 1 ( ) Just to keep notation simple we put Let is even, then (v 1) / 2 d v ( ) . d v/2 (3.20) and cˆi ,v v !cl ,v . Note that cˆ1,v and (v 1) / 2 (v 1. 2) / 2 1 l 1 l 1 l 1 l 2 1 for any integer 0. Therefore (3.17) and (3.20) gives cˆl ,v1 l vcˆl ,v1 l 1 lcˆl ,v l vcˆl 1,v1 lcˆl ,v l . For the even v the property (3.19) follows. Let now v is odd, then (v 1) / 2 1. (v 1) / 2 and From (3.17) and (3.20) obtain 1 1 l 1 l 1 l 1 l 2 l 1 cˆl ,v1 l vcˆl ,v1 l 1 lcˆl ,v l vcˆl 1,v1 l lcˆl ,v l vcˆl 1,v 1 lcˆl ,v l vcˆ ,v 1 1 . l 2 This proves (3.19) for the odd v . Lemma 3.2 is proved completely. Using (3.15) we obtain v ( ) d d v ( ) v / 2 v ( ) . This together with (3.20) imply v ( ) v1 ( ) v / 2 v ( ) . (3.21) Use formula (3.20) for v 2 ( ) , next again apply (3.20) to the derivative of v 1 ( ) , after this use the inequalities 2 d2 d 2 v ( ) v/2 d d v 1 2 v ( ) ( ). v ( ) (because lcˆl ,v 1 cˆl ,v ( see (3.19)), d d v ( ) v / 2 v ( ) and Then one can observe that for any integer 0 ( 1)v ( ) v2 ( ) 2v( v)v ( ) . (3.22) In the sequel some notations regarding to the general statistic RN ( see (3.1)) are useful : 8 N N m 1 m 1 AN Ef m ( m ) , n n1 cov f m ( m ), m , N N2 Varf m ( m ) , N2 N2 n n2 . (3.23) m 1 Note that under the conditions (3.2) and (3.3) ERN AN o( N ) and VarRN N2 (1 o(1)) . 2 N We notice that formula for the and 2 N (3.24) given in (2.1) are the special case of (3.23) corresponding to the chi-square statistic; that is why the same symbols are used. Proof of Theorem 2.1. The chi-square statistic is (3.1) kind with f m ( x) x npm / npm , hence 2 AN N , N2 and N2 as in (2.1). Set m m npm / npm . We have Em2 s (m npm )2 4s(npm 2s) Em2 s , since (3.22), hence the conditions (3.2) and (3.3) are fulfilled with b 1 and a 1 if npm 2s , and with b 0 and a 2 if npm 2s . It is evident that the condition (3.4) is 1 fulfilled for the k o( pmax ) . Therefore, from Main Lemma and Remark 3.1 it follows that N 2 1 o 1 , k m m1 ( N2 ) k (3.25) 1/2 for every integer k such that 3 k o min pmax , n1/3 . We would remind the notation see that (2k )! 22k e / 2k (k !)2 . n 1 max 1,(npmin ) Due to this, putting m . Since 2 m mme ( 2 m E 2 m k 1 l 0 k 1 (np m 2 m ) / Var obtain E mk 2k (2k )!(Varm2 ) k /2 (npm ) l (k !) 2 27 n / Varm2 m k 2 m! e m mm e m one can and using Lemma 3.2 we for k 3 , since 2 1 ) l k nk 2 and Varm (npm ) 2 . Hence, by Assertion1 l 0 k m k ! 2 28 n / Varm2 k 2 . That is k k ! 2 2 2 m n N 2 2 28 n k m k ! m1 8 k 2 k 2 Varm2 , for k 3 . Therefore N2 , (3.26) N because m , m=1,2,… are independent r.v.s, and N2 Var m2 , see notation (2.1). Now (3.25), m 1 (3.26) and the fact that N2 Var N2 1 o(1) , see (3.23), imply k 2 N / Var 2 N N k m2 / Var N2 m1 13 2 2 2 n N k ! 3 N k 2 , 9 1/2 for all k satisfying 3 k o min pmax , n1/3 . Theorem 2.1 follows from this and Assertion 2 with 1/2 213 n N2 / N3 and k min pmax , n1/3 . Proof of Corollary 2.1. Under conditions of Corollary 2.1 we have N2 1 (2c0 )1 N2 N2 1/ 4 and n max(1, 2 / c1 ) , also pmax c21/ 4 N 1/ 6 . Corollary 2.1 follows. Proof of Corollary 2.2 is straightforward since in this case n max(1, n1 ) , N2 N 2 d 2 2 (n)n1 (1 o 1 and N2 2 N 1 (2n )1 (1 o(1)) 2 N n . Proof of Theorem 2.2. Observe that the N is RN kind statistic with f m ( x) x ln x / npm (we assume that 0 ln 0 0 ). Write f m (m ) npm 1 ˆm ln 1 ˆm , (3.27) N where ˆm m npm / npm 1 . Put gm ( x) f m ( x) n ( x npm ) . It is evident that RN g m (m ) , m 1 hence it is sufficient to check the conditions of Main Lemma for gm (m ) . We shall use the inequality 1 1 x 2l 1 x 2l 2 0 (1 x) ln(1 x) x x 2 x3 ... , 2 6 2l (2l 1) (2l 1) (3.28) x 1 and integer l 0 , to find, see denotes (3.23), AN 1 N 1 N 1 N 1 1 O 1 O , 2 12 m1 npm npm 2 n n 1 (3.29) 1 1 N 1 2 1 O 1 O n , 12n m1 npm npm N N2 Var g m (m ) m 1 (3.30) 1 N 1 N 1 N 1 1 O 1 O . 2 6 m1 npm npm 2 n (3.31) From (3.28) we have x (1 x) ln(1 x) x x 2 / 2 , hence c n npm (1 n )ˆm gm (m ) npm (1 n )ˆm ˆm2 , (3.32) since ˆm 1 , npm c1n and 1 n (c0n2 )1 by (3.30). From Lemma 3.2 we obtain Eˆm O !(npm ) /2 for any integer 3.2, whereas for an odd 1 (to see this for an even 1 we can directly use Lemma 1 we first use the inequality E ˆm Eˆm1 /( 1) and then apply Lemma 10 3.2). Therefore, it is not hard to understand that the conditions (3.2) and (3.3) with a 1 and b 1 1/ 2 are satisfied. Thus, Main Lemma gives: for all integer k such that 3 k o pmax N g (m ) 1 o 1 . k m m1 ( N ) k On the other hand using (3.32), inequality (2k )! 22k e / (3.33) 2k (k !)2 and the condition (2.4) we obtain k k !(npm )k / 2 k k 2 11 k 2 E gm (m ) 2k npm (1 n )k E ˆm Eˆm2 k 2k 1 (2k )! (k !) 2 . 2k c0n Therefore, Assertion 1 imply k gm (m ) (k !)2 212 k 2 , because Var gm (m ) 21 (1 o(1)) . Hence 1 N g (m ) (k !) 2 211 N 1/ 2 k N m m 1 k 2 , since (3.29) and (3.31). Thus from (3.33) taking into account that k ( N1 N ) (k !)2 212 N 1/ 2 k 2 N 2 N Vargm ( m ) we obtain m 1 , 1/2 for all integer k such that 3 k o pmax . Noting that Var N N2 1 o(1) , see (3.24), Theorem 1/2 2.2 follows from Assertion 2 with k o pmax and 212 N 1/2 . Proof of Corollary 2.3 is straightforward. Appendix. Below is a r.v. with E 0 , Var 1 , k ( ) and k ( ) cumulant and moment of k th order respectively of the r.v. . The following conditions and assertions are taken from the book by Saulis and Statulevicius (1991). Bernstein condition ( B ) : there exists the constants 0 and B 0 such that k ( ) (k !) 1 Bk 2 , for all k =3,4,…. Statulevicius condition ( S ) : there exists the constants 0 and 0 such that k ( ) (k !) 1 ( k 2) , for all k =3,4,…. Assertion 1. If satisfy condition ( B ) then it also satisfy condition ( S ) with 2B . 1 Assertion 2. Let a r.v. depending on a parameter satisfy condition ( S1 ) (i.e. v 1 ) for all k such that 3 k k . Then for all x such that 0 x 121 min k , 1/ 3 there exist a constant c such that P x 1 ( x) 1 C ( x 1) 1/3 and P x ( x) 1 C ( x 1) 1/3 . In the book of Saulis and Statulevicius (1991) no assertion in the form of Assertion 2 is given. However, Assertion 2 can be proved by reasoning alike to those presented by Saulis and 11 Statulevicius (1991) at page 35, in course of proof of Lemma 2.3, see relations (2.62), (2.65) and (2.66) and Lemma 2.2 of the book. Assertion 3. One has ( ) k ! 1 k here m1 m2 ... mk 1 k m1 m2 ... mk 1! l 1 1 l ( ) ml ! l ! ml is summation over all non-negative integer m1 , m2 ,..., mk such that m1 2m2 ... kmk k . References. 1. Kallenberg W.C.M.,(1985). On moderate and large deviations in multinomial distributions. Ann. Statist. 13, 1554-1580. 2. Inglot T., (1999). Generalized intermediate efficiency of goodness-of-fit tests. Math. Methods in Statist; 8, p.487-509. 3. Ivchenko G.I. and Mirakhmedov S.,(1995). Large deviations and intermediate efficiency of the decomposable statistics in multinomial scheme. Math. Methods in Statist. 4, p.294-311. 4. Mann H.B. and Wald A. (1942). On the choice of the number of intervals in the application of the chi-square test. Ann. Math. Statist. 13, 306-317. 5. Saulis L, Statulevicius V., (1991). Limit theorems for large deviations. Dordrecht, Boston, London: Kluwer Academic Publishers, 232 p. 6. Sirajdinov S.Kh, Mirakhmedov S.A. and Ismatullaev S.A., (1989). Large deviation probabilities for decomposable statistics in a polynomial scheme. Theory Probabl.Appl. 34, p.706-719. 7. Kenney J.F. and Keeping E.S. (1953). Mathematics of Statistics (Van Nostrand, London) part 2. 8. Mirakhmedov S.(2016). Asymptotic Intermediate Efficiency of the Chi-square and Likelihood Ratio Goodness of Fit Tests. Submitted.