advertisement

Confidence Intervals Lecture XXI Interval Estimation • As we discussed when we talked about continuous distribution functions, the probability of a specific number under a continuous distribution is zero. • Thus, if we conceptualize any estimator, either a nonparametric estimate of the mean or a parametric estimate of a function, the probability of the true value equal to the estimated value is obviously zero. • Thus, usually talk about estimated values in terms of confidence intervals. Specifically, as in the case when we discussed the probability of a continuous variable, we define some range of outcomes. Confidence Intervals • Amemiya notes a difference between confidence and probability. Most troubling is our classic definition of probability as “a probabilistic statement involving parameters.” This is troublesome due to our inability without some additional Bayesian structure to state anything concrete about probabilities. • Example 8.2.1. Let Xi be distributed as a Bernoulli distribution, i=1,2,…,n. Then, T X ~ N p, p(1 p) / n A Therefore, we have Z Tp A ~ N 0,1 p1 p n • Why? By the Central Limit Theory • Given this distribution, we can ask questions about the probability. Specifically, we know that if Z is distributed N(0,1), then we can define k P Z k Table of Normal Probabilities k 1.0000 1.5000 1.6449 1.7500 1.9600 2.0000 2.3263 0.1587 0.0668 0.0500 0.0401 0.0250 0.0228 0.0100 0.3173 0.1336 0.1000 0.0801 0.0500 0.0455 0.0200 • The values of k can be derived from the standard normal table as Tp P k k p1 p n • Assuming that the sample value of T is t, the confidence is defined by t p C k k p1 p n • Building on the first term Tp p 1 p P k P T p k n p 1 p n p1 p 2 P T p k 2 n 2 k2 k2 2 2 P T 2Tp p p p 0 n n 2 k 2 P p 1 n k2 2 p 2T T 0 k n • Using this probability, it is possible to define two numbers h1(T) and h2(T) for which this inequality holds. Ph1 T p h2 T k where 2 2 k k k 2 2T 2T 41 T n n n h1 T , h2 T k2 21 n 2 2 Table 1. Sample Statistics for T for 3 Draws Draw 1 Draw 2 Statistic p=.4 p=.6 p=.4 p=.6 T 0.300 0.500 0.480 0.600 S 0.463 0.505 0.505 0.495 T(1-T)/50 0.030 0.035 0.035 0.034 T 0.320 0.500 0.430 0.590 S 0.469 0.503 0.498 0.494 T(1-T)/100 0.022 0.025 0.025 0.024 T 0.347 0.547 0.393 0.587 S 0.478 0.499 0.490 0.494 T(1-T)/150 0.018 0.020 0.019 0.020 T 0.360 0.570 0.400 0.605 S 0.481 0.496 0.491 0.490 T(1-T)/200 0.016 0.017 0.017 0.017 Draw 3 p=.4 p=.6 0.480 0.660 0.505 0.479 0.035 0.032 0.380 0.590 0.488 0.494 0.024 0.024 0.380 0.593 0.487 0.493 0.019 0.020 0.380 0.575 0.487 0.496 0.017 0.017 Table 2. Empirical Confidence Intervals for Samples Draw 1 Draw 2 Draws p=.4 p=.6 p=.4 p=.6 50 Lower Bound 0.191 0.366 0.348 0.462 Upper Bound 0.438 0.634 0.615 0.724 100 Lower Bound 0.237 0.404 0.337 0.492 Upper Bound 0.417 0.596 0.528 0.681 150 Lower Bound 0.275 0.467 0.319 0.507 Upper Bound 0.426 0.624 0.473 0.662 200 Lower Bound 0.297 0.501 0.335 0.536 Upper Bound 0.429 0.637 0.469 0.670 Draw 3 p=.4 p=.6 0.348 0.522 0.615 0.776 0.291 0.492 0.478 0.681 0.306 0.513 0.460 0.669 0.316 0.506 0.449 0.642 • Example 8.2.2. Let Xi ~ N(m,s2), i=1,2,…n where m is unknown and s2 is known. We have T X ~ N m ,s 2 n T m P k k s2 n • Example 8.2.3. Suppose that Xi~N(m,s2), i=1,2,3,…n with both m and s2 unknown. Let TX be an estimator of m and 1 n 2 S i 1 X i X n 2 be the estimator of s2. • Then the probability distribution tn1 S 1 T 1 n 1 or the Student’s t distribution with n-1 degrees of freedom is the probability distribution. • Theorem 5.4.1: Let X1, X2,…Xn be a random sample from a N(m,s2) distribution, and let 1 n X i 1 X i n 1 n 2 X i X S i 1 n 1 2 • X and S2 are independent random variables • X ~ N m ,s 2 n • (N-1)S2/s2 has a chi-squared distribution with n-1 degrees of freedom. • The proof of independence is based on the fact that S2 is a function of the deviations from the mean which, by definition, must be independent of the mean. • The chi-square is defined as: f x 1 p 2 2 x p p 1 x 2 2 e 2 • In general, the gamma distribution function is defined through the gamma function: t 0 1 t e dt 1 1 0 1 t t e 1 t t e dt f t • Substituting X=bt gives the traditional two parameter form of the distribution function f x , b 1 b x x 1 b e • The expected value of the gamma distribution is b and the variance is b2. • Lemma 5.4.1. (Facts about chi-squared random variables) We use the notation cp2 to denote a chi-squared random variable with p degrees of freedom. ▫ If Z is a N(0,1) random variable, the Z2~c12, that is, the square of a standard normal random variable is a chi-squared random variable. ▫ If X1, X2,…Xn are independent, and Xi~cpi2, then X1+X2+…Xn~c2p1+p2+…pn, that is, independent chisquared variables add to a chi-square variable, and the degrees of freedom also add. • The first part of the Lemma follows from the transformation of random variables for Y=X2 which yields: fY y 1 2 f y f y y X X • Returning to the proof at hand, we want to show that (N-1)S2/s2 has a chi-squared distribution with n-1 degrees of freedom. To demonstrate this, note that n 1 S 2 n n 2 S 2 n 1 2 n 1 X n X n 1 n 1 n 1 n 2 Sn X X i n j 1 j n 1 i 1 2 2 1 n 1 1 n 1 1 n 1 1 2 n 1 Sn X n j 1 X j X n i 1 X i j 1 X j X n n n n n 2 n 1 n 1 n 1 1 n 1 1 2 n 1 S X X X X j Xn n n n 1 i i 1 j 1 n n n n 2 2 • If n=2, we get 1 2 S X 2 X1 2 2 2 • Given that (X2-X1)/√2 is distributed N(0,1), S22~c12 and by extension for n=k, (k-1)Sk2~ck-12. • Given these results for the chi-square, the distribution of the Student’s t then follows. X m S n X m S s n 2 s 2 • Note that this creates a standard normal random variable in the numerator and c 2 n 1 in the denominator n 1 • The complete distribution found by multiplying the standard normal time the chi-squared distribution times the Jacobian of the transformation yields: p2 1 2 1 fT t 1 p 1 p p 2 2 2 t 1 p 2