Efficiency and Relative Efficiency of Tests. Chi-Square Tests Scientific Seminar “Asymptotic Statistics” Olena Korzhevska Table of Contents • Relative Efficiency of Tests – Asymptotic Power Functions. Consistency. Asymptotic Relative Efficiency • Efficiency of Tests – Asymptotic Representation Theorem. Testing Normal Means. Local Asymptotic Normality. One-Sample Location. Two-Sample Problems • Chi-Square Tests – Quadratic Forms in Normal Vectors. Pearson Statistic. Testing Independence. Goodness-of-Fit Tests. Asymptotic Efficiency 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 2/42 1. Relative Efficiency of Tests Asymptotic Power Functions • The relative efficiency of two sequences of tests is the quotient of the numbers of observations needed with the two tests to obtain the same level and power. • Testing problem: ๐ป0 : ๐ ∈ Θ0 ๐ฃ๐ . ๐ป1 : ๐ ∈ Θ1 • The power function of a test that rejects ๐ป0 if a test statistics ๐๐ falls into critical region ๐พ๐ : ๐ โผ ๐๐ ๐ = Ρ๐ (๐๐ ∈ ๐พ๐ ) • The test is of the level ๐ถ if its size sup ๐๐ ๐ : ๐ ∈ Θ0 does not exceed ๐ผ. • The sequence of tests is asymptotically of level ๐ถ if limsup sup ๐๐ ๐ ≤ ๐ผ. ๐→∞ 14/02/2015 ๐∈Θ0 Olena Korzhevska. Asymptotic Statistics Seminar 4/42 Asymptotic Power Functions • The test with power function ๐๐ is better than the test with power function ๐๐′ if both ๐๐ (๐) ≤ ๐′๐ (๐), ๐ ∈ Θ0 , ๐๐ (๐) ≥ ๐′๐ (๐), ๐ ∈ Θ1 . • Aim: to compare tests asymptotically. • Consider 2 sequences of tests, with power functions ๐๐ and ๐′๐ (Tests of each sequences are of the same type). 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 5/42 Asymptotic Power Functions • First idea – compare limiting power functions of the form : ๐ ๐ = lim ๐๐ ๐ . ๐→∞ • Example(Sign test). ๐1 , ๐2 , … , ๐๐ r.v. form the distribution with unique median ๐. - Test: ๐ป0 : ๐ = 0 ๐ฃ๐ . ๐ป1 : ๐ > 0, - Test statistics: ๐๐ = ๐−1 ๐๐=1 1{๐๐ >0} , - Distribution function of the observations ๐น ๐ฅ − θ , ๐ ๐ = 1 − ๐น −θ , 1 − ๐น −θ ๐น −๐ ๐2 ๐ = , ๐ ๐ 1 ๐ 0 = , 2 1 ๐ 0 = . 4 - ๐ ๐๐ − ๐ ๐ โ ๐(0, ๐ 2 (๐)) asymptotically, - under ๐ป0 : ๐ ๐๐ − 1/2 โ ๐(0,1/4) 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 6/42 Asymptotic Power Functions • Example(Sign test). - Test that rejects ๐ป0 if ๐ ๐๐ − 1/2 > ๐ง๐ผ /2 has power function: ๐ง๐ผ ๐๐ ๐ = ๐๐ ( ๐ ๐๐ − ๐ ๐ > − ๐ ๐ ๐ −๐ 0 2 ๐ง๐ผ − ๐ ๐น 0 − ๐น −๐ 2 =1− Φ + ๐(1) ๐ ๐ - as ๐น 0 − ๐น −๐ > 0 for every ๐ > 0, it follows that for ๐ผ = ๐ผ๐ → 0 sufficiently slowly 0 ๐๐ ๐ = 0, ๐๐ ๐ = 1 ๐๐ ๐ > 0. - In this case the limit power function corresponds to the perfect test with all error probabilities equal to zero. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 7/42 Asymptotic Power Functions How do we compare tests? We need to make the problem of discriminating between the null and the alternative hypotheses more difficult as n increases. It is natural to consider a shrinking alternative, that converges to the null. To test: ๐ป0 : ๐ = 0 ๐ฃ๐ . ๐ป1 : ๐๐ > 0, with ๐๐ → 0 Example(Sign test, continued). (on a board) In this situation a reasonable method for asymptotic comparison of 2 sequences of tests is to consider local limiting power functions: ๐ โ = lim ๐๐ ๐→∞ 14/02/2015 โ ๐ , โ ≥ 0. Olena Korzhevska. Asymptotic Statistics Seminar 8/42 Asymptotic Power Functions Theorem: Suppose that ๐๐ , ๐, and ๐ are such that, for all โ and ๐๐ = โ/ ๐, ๐(๐๐ −๐(๐๐ )) โ ๐(๐๐ ) ๐๐ ๐(0,1) ๐ is differentiable in 0, ๐ is continuous in 0. Then the tests that reject ๐ป0 : ๐ = 0 for large values of ๐๐ and are asymptotically of level ๐ผ satisfy, for all โ, โ ๐′ 0 ๐๐ → 1 − Φ ๐ง๐ผ − โ . ๐ 0 ๐ 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 9/42 Asymptotic Power Functions Proof: Substituting โ = 0 shows that the asymptotic level of the test is ๐ผ iff ๐ป0 : ๐ = 0 is rejected for ๐(๐๐ −๐(0)) ๐(0) Thus, ๐๐ ๐๐ = ๐๐๐ = ๐๐๐ ๐ ๐๐ − ๐ 0 ๐ ๐๐ −๐ ๐๐ ๐(๐๐ ) → 1 − Φ ๐ง๐ผ − 14/02/2015 > > ๐ง๐ผ . > ๐ 0 ๐ง๐ผ ๐ 0 ๐ง๐ผ − ๐ ๐ ๐๐ −๐ 0 ๐(๐๐ ) ๐′ 0 โ ๐ 0 Olena Korzhevska. Asymptotic Statistics Seminar 10/42 Asymptotic Power Functions • ๐′ 0 ๐ 0 - slope of the sequence of tests. • Example (Sign test): The sign test has slope • Example (t-test):๐๐ = ๐ ๐๐ , ๐ ๐−๐ ๐๐ ๐′ 0 ๐ 0 = 2f 0 . โ ๐ 0,1 . ๐ Reject H0 if ๐๐๐ > ๐ง๐ผ . ๐ ๐ ๐ − ๐ ๐ = 14/02/2015 โ/ ๐ ๐ ๐ ,๐ ๐ = ๐(๐−โ / ๐) ๐ ๐ = 1. ๐′ 0 ๐ 0 +โ 1 ๐ 1 − ๐ โ ๐(0,1) โ/ ๐ = 1/๐. Olena Korzhevska. Asymptotic Statistics Seminar 11/42 Asymptotic Power Functions Example (Sign test vs. t-test): • ๐1 , ๐2 , … , ๐๐ random sample from a ๐(๐ฅ − ๐)-density, ๐symmetric about 0, has unique median & finite 2๐๐ moment. • Test: ๐ป0 : ๐ = 0 that the observations are symmetrically distributes around 0. Compare the performance of sign and ttest. • Suffices to compare the slopes of 2 tests: 1 ๐ 2f 0 and , respectively. • For ๐(0,1) the slopes are 2/๐ and 1. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 12/42 Asymptotic Power Functions Relative efficiency of the sign test versus the ttest for some distributions. DISTRIBUTION EFFICIENCY(SIGN/T-TEST) Logistic ๐ 2 /12 Normal 2/๐ Laplace 2 Uniform 1/3 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 13/42 Consistency • Definition: A sequence of tests with power functions ๐ โผ ๐๐ ๐ is asymptotically consistent at level ๐ผ against alternative ๐ if it is asymptotically of the level ๐ผ and ๐๐ ๐ → 1. • If a family of sequences of tests contains for every level ๐ผ a sequence that is consistent against every alternative, then the corresponding tests are simply called consistent. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 14/42 Consistency ๐๐ Lemma 1: ๐๐ a sequence of statistics: ๐๐ ๐(๐) for every ๐. Then the family of tests that reject the null hypothesis ๐ป0 : ๐ = 0 for large values of ๐๐ is consistent against every ๐ such that ๐ ๐ > ๐(0). Lemma 2: Suppose that ๐๐ , ๐, and ๐ are such that, for all โ and ๐๐ = โ/ ๐, ๐(๐๐ −๐(๐๐ )) โ ๐(๐๐ ) ๐๐ ๐(0,1), ๐′ (0) > 0, ๐ – continuous at 0 and σ 0 > 0. Suppose that the tests that reject ๐ป0 for the large values of ๐๐ have nondecreasing power functions ๐ โผ ๐๐ ๐ . Then this family of tests is consistent against every alternative ๐ > 0. Moreover, if ๐๐ 0 → ๐ผ, then ๐๐ ๐๐ → ๐ผ when ๐๐๐ → 0, or ๐๐ ๐๐ → 1 when ๐๐๐ → ∞. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 15/42 Consistency • Example(t-test): The two-sample t-statistics (๐๐ − ๐๐ )/๐ converges in probability to E(๐ − ๐)/๐, where ๐ 2 = lim ๐ฃ๐๐(๐๐ − ๐๐ ). n→∞ If the null hypothesis postulates that E๐ = ๐ธ๐, then the test that rejects the null hypothesis for the large values of the t-statistics is consistent against every alternative for which E๐ > ๐ธ๐. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 16/42 Asymptotic relative efficiency • Sequence of tests can be ranked in quality by comparing their asymptotic power functions. • For the test statistics we have seen so far this comparison involves “slopes” of the tests. • The concept of relative efficiency yields a method to quantify the interpretation of the slopes. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 17/42 Asymptotic relative efficiency • Sequence of testing problems to test: ๐ป0 : ๐ = 0 vs. ๐ป1 : ๐ = ๐๐ . • Requirement: tests need to attain asymptotic level ๐ผ and power ๐พ ∈ (๐ผ, 1). • ๐๐ is a power function of a test if n observations are available, ๐๐ is minimal number of observations such that both ๐๐๐ (0) ≤ ๐ผ and ๐๐๐ (๐๐ ) ≥ ๐พ. • The limit (if exists) lim ๐๐,2 ๐→∞ ๐๐,1 is called (asymptotic) relative efficiency or Pitman efficiency of the first sequence of tests with respect to second one. • A relative efficiency larger than 1 indicates that fewer observations are needed with the first sequence of tests, which may then be considered the better one. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 18/42 Asymptotic relative efficiency Theorem: Consider stat. models (๐๐,๐ : ๐ ≥ 0) : ๐๐,๐ − ๐๐,0 ๐ ๐๐,๐ −๐๐ ๐๐ Let ๐๐,1 , ๐๐,2 – sequences of statistics: ๐๐ ๐๐ ๐→0 0, ∀๐. โ ๐ 0,1 , ∀๐๐ → 0, ๐๐ functions: ๐๐ − differentiable at 0, ๐′ ๐ 0 > 0, and ๐๐ −continuous at 0, ๐๐ 0 > 0, i ∈ 1,2 . Then the relative efficiency of the tests that reject ๐ป0 : ๐ = 0 for large values of ๐๐,๐ is equal to ๐1′ (0)/๐1 (0) ๐2′ (0)/๐2 (0) 2 , ∀ ๐๐ ↓ 0, ∀๐๐ → 0 independently of ๐ผ > 0 and ๐พ ∈ ๐ผ, 1 . If the power function of the test based on ๐๐,๐ are nondecreasing for every n, then the assumption of asymptotic normality of ๐๐,๐ can be relaxed to asymptotic normality under every sequence ๐๐ = ๐(1/ ๐) only. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 19/42 2. Efficiency of Tests Asymptotic Representation Theorem • Randomized test (test function) ๐ in an experiment (๐, ๐ด, ๐โ : โ ∈ ๐ป) is a measurable map ๐: ๐ โผ [0,1] on the sample space. • The power function of a test ๐ is the function โ โผ ๐ โ = ๐ธโ ๐ ๐ . Theorem: Let the sequence of experiments โฐ๐ = (๐๐,โ : โ ∈ ๐ป) converge to a dominated experiment โฐ= (๐โ : โ ∈ ๐ป). Suppose that a sequence of power functions ๐๐ of tests in โฐ๐ converges poinwise, i.e., ๐๐ โ → ๐(โ), for every h and some arbitrary function ๐. Then ๐ is a power function in the limit experiment, i.e., there exists a test ๐ in โฐ with ๐ โ = ๐ธโ ๐ ๐ for every h. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 21/42 Testing Normal Means • Suppose X is ๐๐ (โ, Σ)-distributed, Σ – known, h – unknown. • Test: ๐ป0 : ๐ ๐ โ = 0 vs. ๐ป1 : ๐ ๐ โ > 0, for known vector c, ๐ ๐ Σ๐ > 0 Proposition: The test that rejects ๐ป0 if ๐ ๐ ๐ > ๐ง๐ผ ๐ ๐ Σ๐ is uniformly most powerful at level ๐ผ for testing the ๐ป0 : ๐ ๐ โ = 0 vs. ๐ป1 : ๐ ๐ โ > 0, based on X. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 22/42 Local Asymptotic Normality • If the model (๐๐ : ๐ ∈ Θ) is differentiable in quadratic mean, then the local experiment converges to the Gaussian experiment (recall yesterday last talk!) ๐๐๐0 +โ/ −1 ๐ ๐ : โ ∈ ๐ → ๐ โ, ๐ผ : โ ∈ ๐ ๐ ๐ 0 • The sequence of power functions ๐ โฆ ๐๐ (๐) in original experiments induces the sequence of power functions h โฆ ๐๐ (๐0 + โ/ ๐) in the local experiments. Suppose ๐๐ ๐0 + โ ๐ → ๐ โ ∀โ, some ๐. Then by the asymptotic representation theorem, this limit ๐ is the power function in the Gaussian limit experiment. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 23/42 Local Asymptotic Normality • Suppose ๐-real, ๐๐ is of asymptotic level ๐ผ to test: ๐ป0 : ๐ ≤ ๐0 vs. ๐ป1 : ๐ > ๐0 Then, ๐ 0 = lim ๐๐ ๐0 ≤ ๐ผ, and hence ๐ corresponds to a level ๐ผ ๐→∞ test for: ๐ป0 : โ = 0 vs. ๐ป1 : โ > 0 in the limit experiment. • By Proposition for testing normal means, ๐ must be bounded by the power function of the uniformly most powerful level ๐ผ test in the limit experiment. Thus ∀h,(c=1, Σ = ๐ผ๐−1 ๐๐ ๐๐๐๐๐๐ ๐๐ก๐๐๐) 0 โ lim ๐๐ ๐0 + ≤ 1 − Φ ๐ง๐ผ − โ ๐ผ๐0 ๐→∞ ๐ 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 24/42 Local Asymptotic Normality • As stated earlier, sequence of power function ๐๐ ๐0 + โ/ ๐ → 1 − Φ(๐ง๐ผ − โ๐ ) for every h, has slope s. From the upper bound, ๐ผ๐0 is the largest possible slope. • The relative efficiency of the best test and the test with a slope s is: ๐ผ๐0 /๐ 2 which can be interpreted as the number of observations needed with the given sequence of tests with the slope s divided by the number of observations needed with the best test to obtain the same power. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 25/42 Local Asymptotic Normality Theorem 15.4: Let Θ ⊂ ๐ ๐ -open, ๐: Θ โผ ๐ -differentiable in ๐0 , with ๐ ≠ 0: ๐ ๐0 = 0. Let (๐๐ ,๐ : ๐ ∈ Θ) be locally asymptotically normal at ๐0 with nonsingular I, ๐๐ → ∞ -const. Then, ๐ โฆ ๐๐ (๐) of any sequence of level ๐ผ tests for testing: ๐ป0 : ๐(๐) ≤ 0 vs. ๐ป1 : ๐(๐) > 0 satisfy for every h: ๐๐0 โ > 0: ๐๐0 โ โ limsup๐๐ ๐0 + ≤ 1 − Φ ๐ง๐ผ − . ๐๐ ๐→∞ ๐ ๐๐0 ๐ผ๐−1 ๐ ๐0 0 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 26/42 Local Asymptotic Normality Addendum: Let ๐๐ be statistics such that ๐๐0 ๐ผ๐−1 Δ 0 ๐,๐0 ๐๐ = + ๐๐๐,๐ 1 . 0 −1 ๐ ๐๐0 ๐ผ๐0 ๐๐0 Then the sequence of tests that reject ๐ป0 for the values of ๐๐ > z๐ผ is asymptotically optimal in the sense that the sequence for every h ๐๐0 +๐๐−1 โ ๐๐ ≥ ๐ง๐ผ → 1 − Φ ๐ง๐ผ − ๐๐0 โ ๐ ๐๐0 ๐ผ๐−1 ๐๐ 0 0 *(Δ๐,๐0 - sequence of statistics that converges in distribution under ๐0 to a normal ๐๐ (0, ๐ผ๐0 )-distribution). 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 27/42 Local Asymptotic Normality • The point ๐0 in the theorem is on the boundary of ๐ป0 and ๐ป1 . • If the dimension k>1, then this boundary is (k-1)-dimentional, and there are many possible values for ๐0 . • If dimension k=1, the boundary point ๐0 is typically unique −1/2 and hence known, and we could use Tn = I๐0 Δ๐,๐0 to construct an optimal sequence of tests for the problem ๐ป0 : ๐ = ๐0 .There are known as score tests. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 28/42 One-Sample Location • ๐1 , ๐2 , … , ๐๐ sample from a ๐(๐ฅ − ๐)-density, ๐-symmetric about 0, has finite ๐ผ๐ , may be known or (partially) unknown. • To test: ๐ป0 : ๐ = 0 vs. ๐ป1 : ๐ > 0. • For fixed ๐, ( ๐๐=1 ๐ ๐ฅ๐ − ๐ : ๐ ∈ R ) is locally asymptotically normal at ๐ = 0 with Δ๐,0 = −๐−1/2 ๐๐=1 ๐/๐′ (๐๐ ), norming rate ๐, Fisher information ๐ผ๐ . • From the preceding sections, the best asymptotic level ๐ผ power function for known ๐ is 1 − Φ ๐ง๐ผ − โ ๐ผ๐ . • ๐๐ = − 1 1 ๐ ๐ผ๐ ′ ๐ ๐ ๐=1 ๐ ๐๐ + ๐๐0 (1) • Than according to the Theorem 15.4, the sequence of tests that reject ๐ป0 if ๐๐ > ๐ง๐ผ attains bound and hence is asymptotically optimal. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 29/42 One-Sample Location Example(t-test): The standard normal density ๐0 possesses score function ๐0′ /๐0 ๐ฅ = −๐ฅ and I๐0 = 1. Consequently, if the underlying distribution is normal, then the optimal test statistics should satisfy: Tn = ๐๐๐ /๐ + ๐๐0 (๐−1/2 ). The t-statistics ๐๐ /๐๐∗ fulfill the requirements. That is the case because for normally distributed observations the ttest is uniformly most powerful for every finite n and hence is certainly asymptotically optimal. *t-statistics simply replaces unknown standard deviation ๐ by an estimate ๐๐ 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 30/42 One-Sample Location In this example, t-statistics simply replaces the unknown standard deviation ๐ by an estimate. This approach can be followed for the most scale families. Under some regularity conditions, the statistics ๐ 1 1 ๐0′ ๐๐ ๐๐ = − ๐0 ๐๐ ๐ ๐ผ๐0 ๐=1 Should yield asymptotically optimal tests, given a consistent sequence of scale estimators ๐๐ . 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 31/42 3. Chi-Square Tests Quadratic Forms in Normal Vectors • ๐๐2 โ • ๐ 2 ๐ ๐=1 ๐ ๐ 2 ๐ ๐=1 ๐ for i.i.d. ๐ 0,1 -distributed ๐1 , ๐2 , … , ๐๐ โ ๐ 2 of standard normal vector ๐ = (๐1 , … , ๐๐ ) Lemma: If vector ๐ is ๐๐ (0, Σ)-distributed, then ๐ 2 is distributed as ๐๐=1 ๐2๐ ๐๐2 for i.i.d. ๐ 0,1 -distributed ๐1 , … , ๐๐ and ๐1 , … , ๐๐ the eigenvalues of Σ. Proof: There exists an orthogonal matrix ๐ : ๐Σ๐๐ = ๐๐๐๐(๐๐ ). Then the vector ๐๐~๐๐ (0, ๐๐๐๐(๐๐ )), which is the same as the distribution of the vector ( ๐1 ๐1 , … , ๐๐ ๐๐ ). Now ๐ 2 = ๐๐ 2 14/02/2015 has the same distribution as ๐ ๐=1 Olena Korzhevska. Asymptotic Statistics Seminar 2 ๐๐ ๐๐ . 33/42 Pearson Statistics • Suppose we observe ๐๐ = (๐๐,1 , … , ๐๐,๐ ) with multinomial distribution corresponding to ๐ trials and ๐ classes having probabilities ๐ = (๐1 , … , ๐๐ ). • The Pearson statistics for the testing ๐ป0 : ๐ = ๐ is given by ๐ ๐ถ๐ a = ๐=1 ๐๐,๐ − ๐๐๐ ๐๐๐ 2 Theorem: If the vector ๐๐ is multinomially distributed with the parameters ๐ and ๐ = ๐1 , … , ๐๐ > 0, then the sequence ๐ 2 ๐ถ๐ a → ๐๐−1 under ๐. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 34/42 Pearson Statistics • The Pearson statistic is oddly asymetric in the observed and true frequencies(which is motivated be the form of the asymptotic covariance matrix). • The method to symmetrize the statistic leads to the Hellinger statistic ๐ ๐ป๐2 a =4 ๐=1 ๐๐,๐ − ๐๐๐ ๐ 2 ๐๐,๐ + ๐๐๐ 2 =4 ๐๐,๐ − ๐๐๐ 2 ๐=1 • Up to a multiplicative constant it’s a Hellinger distance between the discrete probability distribution on {1, … , ๐} with probability vectors ๐ and ๐๐ /๐, respectively. ๐ • As (๐๐ /๐ − ๐) → 0, ๐ป๐2 is asymptotically equivalent to ๐ถ๐ . 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 35/42 Testing Independence • Suppose that each element of a population can be classified by two characteristics, having ๐ and ๐ levels, respectively : ๐11 โฏ ๐1๐ ๐1 . โฎ โฑ โฎ โฎ ๐๐1 โฏ ๐๐๐ ๐๐ . ………………………… ๐.1 … ๐.๐ ๐ • Classification for a random sample of size ๐ from the population – matrix ๐๐,๐๐ : multinomially distributed with parameters ๐ and probabilities ๐๐๐ = ๐๐๐ /๐. • ๐ป0 : ๐๐๐ = ๐๐ ๐๐ − ๐๐๐ก๐๐๐๐๐๐๐ ๐๐๐ ๐๐๐๐๐๐๐๐๐๐๐ก for unknown probability vectors ๐๐ and ๐๐ . 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 36/42 Testing Independence • The ML-estimators of ๐ and ๐ under ๐ป0 : ๐๐ = ๐๐,๐. /๐ and ๐๐ = ๐๐,.๐ /๐ • Modified Pearson statistic with these estimators: ๐ ๐ ๐ถ๐ ๐๐ โจ๐๐ = ๐=1 ๐=1 ๐๐,๐๐ − ๐๐๐ ๐๐ 2 ๐๐๐ ๐๐ Corollary: If the (๐ × ๐) matrices ๐๐ are multinomially distributed with parameters ๐ and ๐๐๐ = ๐๐ ๐๐ > 0, then the sequence ๐ถ๐ ๐๐ โจ๐๐ converges in distribution to the 2 ๐(๐−1)(๐−1) -distribution. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 37/42 Testing Independence Example: Google wants to test the performance of new search algorithms. Google might test three algorithms using a sample of 10,000 google.com search queries. Search algorithm No new search New search Total current test 1 test 2 3511 1749 1818 1489 751 682 5000 2500 2500 Total 7078 2922 10000 • To test: ๐ป0 : The algorithms each perform equally well. ๐ป1 : The algorithms do not perform equally well. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 38/42 Testing Independence Example: ML estimators for ๐ and ๐: ๐๐ = ๐๐,๐. /๐, ๐๐ = ๐๐,.๐ /๐, • ๐๐๐ ๐๐ = ๐๐,๐. โ ๐๐,.๐ /๐ – expected count of each cell (ij). Search algorithm current test 1 test 2 No new search 3511 (3539) 1749 (1769.5) 1818 (1769.5) New search 1489 (1461) 751 (730.5) 682 (730.5) Total 5000 2500 2500 • ๐ถ๐ ๐๐ โจ๐๐ = ๐๐๐ ๐๐๐ฃ๐๐ ๐๐๐ข๐๐ก−๐๐ฅ๐๐๐๐ก๐๐ ๐๐๐ข๐๐ก 2 ๐๐ฅ๐๐๐๐ก๐๐ ๐๐๐ข๐๐ก Total 7078 2922 10000 = 6.120 • ๐๐ = ๐ − 1 ๐ − 1 = 2 − 1 3 − 1 = 2 • p−value = 0.047, thus we reject ๐ป0 at significance level ๐ผ = 0.05. That is, the data provide convincing evidence that there is some difference in performance among the algorithms. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 39/42 Goodness-of-Fit Tests • Given a random sample ๐1 , ๐2 , … , ๐๐ from a distribution ๐, we want to test H0 : ๐ ∈ ๐ซ0 • Testing goodness-of-fit typically focuses on no particular alternative, that is why ๐ 2 statistics are reasonable. • Partition ๐ =∪๐ ๐๐ of the sample space into finitely many sets • โ๐ ๐ด = ๐−1 (1 ≤ ๐ ≤ ๐: ๐๐ ∈ ๐ด) fraction of observations in ๐ด • Vector ๐(โ๐ ๐1 , … , โ๐ ๐๐ ) is multinominal distributes, modified chi-squared statistics is given: ๐ ๐ โ๐ ๐๐ − ๐ ๐๐ ๐=1 14/02/2015 2 ๐ ๐๐ Olena Korzhevska. Asymptotic Statistics Seminar 40/42 Asymptotic Efficiency • The asymptotic null distributions of various versions of the Pearson statistic enable us to set critical values but by themselves do not give information on the asymptotic power of the tests. • The asymptotic power can be measured in various ways: – the most important method – to consider local limiting power functions (discussed earlier) – A second method to evaluate the asymptotic power is by Bahadur efficiencies 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar 41/42 Thank you for attention. 14/02/2015 Olena Korzhevska. Asymptotic Statistics Seminar