Lecture 5: Confidence Intervals Taeyong Park Carnegie Mellon University in Qatar October 1, 2019 Park (CMUQ) 70-207 October 1, 2019 1 / 31 Roadmap: Where are we now? Probability theory; sampling distributions ⇒ Confidence intervals and hypothesis testing ⇒ Applying to mean difference tests (t tests and Anova) and proportion difference tests (z tests and χ2 tests). Park (CMUQ) 70-207 October 1, 2019 2 / 31 Sampling distribution of the sample mean Sampling distribution of the sample mean For random sampling with a large sample size n , the sampling distribution of the sample mean y is approximately normal: σ2 y ∼ N µ, n µ ¶ The mean of the distribution is equal to population mean µ. The standard deviation of the distribution is equal to σ is a population standard deviation. pσ , n where σ p is also called a standard error. n Park (CMUQ) 70-207 October 1, 2019 3 / 31 What is next? We are hunting a population mean µ. E.g, what is the average salary of EAI 2,500 employees? We sample from the population and calculate a sample statistic y. We use a sample statistic like y to estimate a population parameter like µ. Park (CMUQ) 70-207 October 1, 2019 4 / 31 Point estimation Point estimation for µ. Sample mean: y = Ï n 1X yi n i =1 Terminology: a sample mean is a point estimator; a realized sample mean value y is a point estimate. Example: You are the project manager for a company who is deciding whether or not to place a new product on the market. You commission a poll of consumers in the region to find out whether they would need the new product. Your pollster gives you a point estimate: 54.5 percent of consumers need the product. Park (CMUQ) 70-207 October 1, 2019 5 / 31 Point estimation Point estimation for µ. Sample mean: y = Ï n 1X yi n i =1 Terminology: a sample mean is a point estimator; a realized sample mean value y is a point estimate. Example: You are the project manager for a company who is deciding whether or not to place a new product on the market. You commission a poll of consumers in the region to find out whether they would need the new product. Your pollster gives you a point estimate: 54.5 percent of consumers need the product. How much can we believe it to be close to µ? Park (CMUQ) 70-207 October 1, 2019 5 / 31 Interval estimation A point estimate is OK, but there is always uncertainty around point estimates: A point estimate (sample statistic) is hardly the same as the population parameter. Alternatively, interval estimation. Park (CMUQ) 70-207 October 1, 2019 6 / 31 Interval estimation In the example above, your pollster could give you interval estimates: We can be 25% confident that the true percent point of consumers who need the product is between 52 and 57. We can be 95% confident that the true percent point of consumers who need the product is between 45.75 and and 62.25. We can be 100% confident that the true percent point of consumers who need the product is between 0 and 100. Trade-off between the width of your interval estimates and the confidence you can have: The more confident you are, the wider your interval is, and hence the less precise your estimate becomes. Park (CMUQ) 70-207 October 1, 2019 7 / 31 An interval estimator for µ Confidence interval A confidence interval for a population parameter is a range of numbers within which a parameter is believed to fall given repeated sampling. Confidence interval for µ: [ y ± Margin of error]. How to calculate the margin of error in [ y ± Margin of error]? Ï Ï Use the sampling distribution of y . Information about the probability that y will be within a given distance of µ. Park (CMUQ) 70-207 October 1, 2019 8 / 31 Sampling distribution and confidence interval Recall the following question from the last lecture: Ï Ï Suppose µ = 51800 and σ = 4000 for the EAI salary data. What is the probability that y will be within of $500 of µ when the sample size is 30? µ ¶ The sampling distribution of the sample mean: y ∼ N µ, 500 500 = p . σy 4000/ 30 µ F z= F Then, we calculate P Park (CMUQ) −500 500 p ≤z≤ p 4000/ 30 4000/ 30 70-207 σ2 . n ¶ October 1, 2019 9 / 31 Sampling distribution and confidence interval Recall the following question from the last lecture: Ï Ï Suppose µ = 51800 and σ = 4000 for the EAI salary data. What is the probability that y will be within of $500 of µ when the sample size is 30? µ ¶ The sampling distribution of the sample mean: y ∼ N µ, 500 500 = p . σy 4000/ 30 µ F z= F Then, we calculate P −500 500 p ≤z≤ p 4000/ 30 4000/ 30 σ2 . n ¶ Generally, we can ask the question: P (µ − k ≤ y ≤ µ + k)? A confidence interval is [ y − k ≤ µ ≤ y + k ], where k is Margin of error. ⇒ Confidence interval for µ is [ y ± Margin of error]. Margin of error is a value like 500 in the above example: z × σ y . Ï σ y is from the sampling distribution, and z is from the confidence level you determine, mostly 95% or 99%. Park (CMUQ) 70-207 October 1, 2019 9 / 31 Confidence interval for µ with σ known µ is unknown: Our target. What about σ? Generally, σ is unknown. But in some applications, when large amounts of relevant historical data are available, σ is estimated prior to sampling. We begin with σ known, and will consider σ unknown later. Park (CMUQ) 70-207 October 1, 2019 10 / 31 Confidence interval for µ with σ known µ is unknown: Our target. What about σ? Generally, σ is unknown. But in some applications, when large amounts of relevant historical data are available, σ is estimated prior to sampling. We begin with σ known, and will consider σ unknown later. Example: Ï Ï Ï Lloyd’s Department Store wants to learn about the amount spent per shopping trip. Lloyd’s has been using the weekly survey for several years ⇒ Distribution and σ = $20. It uses a new random sample of 100 customers to compute a 95% confidence interval for the amount spent per shopping trip. Park (CMUQ) 70-207 October 1, 2019 10 / 31 Confidence interval for µ with σ known 95% of a normal distribution falls within 1.96 standard deviations from the mean. This holds true for every normal distribution. µ ¶ σ2 Given x ∼ N µ, , x falls within 1.96σx with probability 0.95. n Ï Whether x or y is a just matter of terminology. Once a sample selected, if x falls between µ − 1.96σx and µ + 1.96σx , then the interval x ± 1.96σx will contain µ. Here, NOTE that x falls between µ − 1.96σx and µ + 1.96σx with probability 0.95. That is, with 95 times out of 100 repeated samples, x occurs such that the interval x ± 1.96σx contains the population mean µ. Park (CMUQ) 70-207 October 1, 2019 11 / 31 Confidence interval for µ with σ known ³ 2 ´ 20 Getting back to the Lloyd’s example, x ∼ N µ, 100 and σx = 2. 95% of all x values from repeated samples will be within ±3.92 of µ. Park (CMUQ) 70-207 October 1, 2019 12 / 31 Interpretation of confidence interval 95% C.I.: Among 100 confidence intervals, 95 intervals contain the population mean. We are 95% confident that the population mean lies in the 95% C.I. Park (CMUQ) 70-207 October 1, 2019 13 / 31 A common mistake when Interpreting confidence intervals The probability that a one single confidence interval contains µ? Park (CMUQ) 70-207 October 1, 2019 14 / 31 A common mistake when Interpreting confidence intervals The probability that a one single confidence interval contains µ? INCORRECT interpretation: The probability that a confidence interval contains the population mean is 95%. Park (CMUQ) 70-207 October 1, 2019 14 / 31 Confidence interval for µ with σ known: A how-to guide The margin of error in [ y ± Margin of error] is given by z α/2 × σ ȳ , where z α/2 is a z-value, and σ ȳ is the standard deviation of the sampling distribution of sample mean (a.k.a. standard error). σ Therefore, [ y ± Margin of error] = y± z α/2 × p . n z α/2 is determined by the confidence level (mostly 95% or 99%). α = 1-the confidence level (e.g., 95% confidence level ⇔ α = 0.05, α/2 = 0.025.) α/2: how much area we need under the curve to the right. z α/2 for the 95% confidence level: Find a z-value corresponding to the probability (1 − 0.95)/2 = 0.025 on the right tail. Park (CMUQ) 70-207 October 1, 2019 15 / 31 Exercise z α/2 for the 99% confidence level? 0.005 = 2.58 Given a sample of n = 100 with y = 9.6, find a 95% confidence interval for the population mean. Suppose σ = 4. 9.6+-0.005 *4/10 Find a 99% confidence interval for the population mean. Lower bound and upper bound Park (CMUQ) 70-207 October 1, 2019 16 / 31 A z-table Standard Normal Distribution Table (Right-Tail Probabilities) z 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 Park (CMUQ) .00 .5000 .4602 .4207 .3821 .3446 .3085 .2743 .2420 .2119 .1841 .1587 .1357 .1151 .0968 .0808 .0668 .0548 .0446 .0359 .0287 .0228 .0179 .0139 .0107 .0082 .0062 .0047 .0035 .0026 .0019 .0013 .0010 .0007 .0005 .0003 .01 .4960 .4562 .4168 .3783 .3409 .3050 .2709 .2389 .2090 .1814 .1562 .1335 .1131 .0951 .0793 .0655 .0537 .0436 .0351 .0281 .0222 .0174 .0136 .0104 .0080 .0060 .0045 .0034 .0025 .0018 .0013 .0009 .0007 .0005 .0003 .02 .4920 .4522 .4129 .3745 .3372 .3015 .2676 .2358 .2061 .1788 .1539 .1314 .1112 .0934 .0778 .0643 .0526 .0427 .0344 .0274 .0217 .0170 .0132 .0102 .0078 .0059 .0044 .0033 .0024 .0018 .0013 .0009 .0006 .0005 .0003 .03 .4880 .4483 .4090 .3707 .3336 .2981 .2643 .2327 .2033 .1762 .1515 .1292 .1093 .0918 .0764 .0630 .0516 .0418 .0336 .0268 .0212 .0166 .0129 .0099 .0075 .0057 .0043 .0032 .0023 .0017 .0012 .0009 .0006 .0004 .0003 .04 .4840 .4443 .4052 .3669 .3300 .2946 .2611 .2296 .2005 .1736 .1492 .1271 .1075 .0901 .0749 .0618 .0505 .0409 .0329 .0262 .0207 .0162 .0125 .0096 .0073 .0055 .0041 .0031 .0023 .0016 .0012 .0008 .0006 .0004 .0003 .05 .4801 .4404 .4013 .3632 .3264 .2912 .2578 .2266 .1977 .1711 .1469 .1251 .1056 .0885 .0735 .0606 .0495 .0401 .0322 .0256 .0202 .0158 .0122 .0094 .0071 .0054 .0040 .0030 .0022 .0016 .0011 .0008 .0006 .0004 .0003 70-207 .06 .4761 .4364 .3974 .3594 .3228 .2877 .2546 .2236 .1949 .1685 .1446 .1230 .1038 .0869 .0721 .0594 .0485 .0392 .0314 .0250 .0197 .0154 .0119 .0091 .0069 .0052 .0039 .0029 .0021 .0015 .0011 .0008 .0006 .0004 .0003 .07 .4721 .4325 .3936 .3557 .3192 .2843 .2514 .2206 .1922 .1660 .1423 .1210 .1020 .0853 .0708 .0582 .0475 .0384 .0307 .0244 .0192 .0150 .0116 .0089 .0068 .0051 .0038 .0028 .0021 .0015 .0011 .0008 .0005 .0004 .0003 .08 .4681 .4286 .3897 .3520 .3156 .2810 .2483 .2177 .1894 .1635 .1401 .1190 .1003 .0838 .0694 .0571 .0465 .0375 .0301 .0239 .0188 .0146 .0113 .0087 .0066 .0049 .0037 .0027 .0020 .0014 .0010 .0007 .0005 .0004 .0003 .09 .4641 .4247 .3859 .3483 .3121 .2776 .2451 .2148 .1867 .1611 .1379 .1170 .0985 .0823 .0681 .0559 .0455 .0367 .0294 .0233 .0183 .0143 .0110 .0084 .0064 .0048 .0036 .0026 .0019 .0014 .0010 .0007 .0005 .0003 .0002 October 1, 2019 17 / 31 Determining the sample size Now suppose we want the population mean to be estimated within a certain margin of error. That is, we want to choose a sample size large enough to provide a desired margin of error. Park (CMUQ) 70-207 October 1, 2019 18 / 31 Determining the sample size Now suppose we want the population mean to be estimated within a certain margin of error. That is, we want to choose a sample size large enough to provide a desired margin of error. Using the example above, y = 9.6, σ = 4, 95% confidence level. Margin of error = z 0.025 × pσn = 1.96 × p4n . We want to choose n such that we estimate µ within a margin of error of 0.5. Ï Ï 4 When n = 100 above, the margin of error = 1.96 × p100 = 0.784. What is your guess? n greater than 100 or smaller than 100? Park (CMUQ) 70-207 October 1, 2019 18 / 31 Determining the sample size Now suppose we want the population mean to be estimated within a certain margin of error. That is, we want to choose a sample size large enough to provide a desired margin of error. Using the example above, y = 9.6, σ = 4, 95% confidence level. Margin of error = z 0.025 × pσn = 1.96 × p4n . We want to choose n such that we estimate µ within a margin of error of 0.5. Ï Ï 4 When n = 100 above, the margin of error = 1.96 × p100 = 0.784. What is your guess? n greater than 100 or smaller than 100? Solve 1.96 × p4n ≤ 0.5. Park (CMUQ) 70-207 October 1, 2019 18 / 31 Confidence interval for µ with σ unknown σ is generally unknown in reality. With σ unknown, we use a sample standard deviation s to estimate σ, and use a t distribution instead of relying on a normal sampling distribution. CI for µ with σ known Using a normal distribution directly from the sampling distribution of the sample mean, y ± z α/2 pσn . CI for µ with σ unknown Using s = qP (y i −y)2 n−1 Park (CMUQ) and a t distribution, y ± t α/2 psn . 70-207 October 1, 2019 19 / 31 Confidence interval for µ with σ unknown We use a t distribution (t score) instead of the standard normal distribution (z score) to account for the error produced by estimating pσn using psn . Especially, when the sample size is small, the error is large. Then, how does a t distribution account for such an increased error? Ï It makes you less confident about your inference by increasing the margin of error. Park (CMUQ) 70-207 October 1, 2019 20 / 31 The t distribution and the standard normal Park (CMUQ) 70-207 October 1, 2019 21 / 31 Notes on the t distribution Symmetric, bell-shaped, zero mean. It has “thicker” tails than the standard normal distribution ⇒ A larger margin of error and a wider CI. Unlike the standard normal distribution (σ = 1), the t distribution’s dispersion depends on degrees of freedom (n − 1), sometimes listed as df. Text As df increases, the t distribution becomes closer and closer to the standard normal distribution. ⇒ When n is large, no big difference between z α/2 and t α/2 . Park (CMUQ) 70-207 October 1, 2019 22 / 31 Exercise Given a sample y = 9.6, S = 4, n = 25, find a 95% confidence interval for the population mean µ. Use the t distribution and the t table in the next page. Given a sample y = 9.6, S = 4, n = 31, find a 99% confidence interval for the population mean µ. Use the t distribution and the t table in the next page. 0.005, n = 30 Park (CMUQ) 70-207 October 1, 2019 23 / 31 Exercise n - 1 = 25 -1 0.025 Given a sample y = 9.6, S = 4, n = 25, find a 95% confidence interval for the population mean µ. Use the t distribution and the t table in the next page. Ï 4 = [7.9488, 11.2512] 9.6 ± 2.064 × p 25 Given a sample y = 9.6, S = 4, n = 31, find a 99% confidence interval for the population mean µ. Use the t distribution and the t table in the next page. Ï 4 = [7.624342, 11.57566] 9.6 ± 2.75 × p 31 Park (CMUQ) 70-207 October 1, 2019 24 / 31 t Table t Table cum. prob t .50 t .75 t .80 t .85 t .90 t .95 t .975 t .99 t .995 t .999 t .9995 one-tail 0.50 1.00 0.25 0.50 0.20 0.40 0.15 0.30 0.10 0.20 0.05 0.10 0.025 0.05 0.01 0.02 0.005 0.01 0.001 0.002 0.0005 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.816 0.765 0.741 0.727 0.718 0.711 0.706 0.703 0.700 0.697 0.695 0.694 0.692 0.691 0.690 0.689 0.688 0.688 0.687 0.686 0.686 0.685 0.685 0.684 0.684 0.684 0.683 0.683 0.683 0.681 0.679 0.678 0.677 0.675 1.376 1.061 0.978 0.941 0.920 0.906 0.896 0.889 0.883 0.879 0.876 0.873 0.870 0.868 0.866 0.865 0.863 0.862 0.861 0.860 0.859 0.858 0.858 0.857 0.856 0.856 0.855 0.855 0.854 0.854 0.851 0.848 0.846 0.845 0.842 1.963 1.386 1.250 1.190 1.156 1.134 1.119 1.108 1.100 1.093 1.088 1.083 1.079 1.076 1.074 1.071 1.069 1.067 1.066 1.064 1.063 1.061 1.060 1.059 1.058 1.058 1.057 1.056 1.055 1.055 1.050 1.045 1.043 1.042 1.037 3.078 1.886 1.638 1.533 1.476 1.440 1.415 1.397 1.383 1.372 1.363 1.356 1.350 1.345 1.341 1.337 1.333 1.330 1.328 1.325 1.323 1.321 1.319 1.318 1.316 1.315 1.314 1.313 1.311 1.310 1.303 1.296 1.292 1.290 1.282 6.314 2.920 2.353 2.132 2.015 1.943 1.895 1.860 1.833 1.812 1.796 1.782 1.771 1.761 1.753 1.746 1.740 1.734 1.729 1.725 1.721 1.717 1.714 1.711 1.708 1.706 1.703 1.701 1.699 1.697 1.684 1.671 1.664 1.660 1.646 12.71 4.303 3.182 2.776 2.571 2.447 2.365 2.306 2.262 2.228 2.201 2.179 2.160 2.145 2.131 2.120 2.110 2.101 2.093 2.086 2.080 2.074 2.069 2.064 2.060 2.056 2.052 2.048 2.045 2.042 2.021 2.000 1.990 1.984 1.962 31.82 6.965 4.541 3.747 3.365 3.143 2.998 2.896 2.821 2.764 2.718 2.681 2.650 2.624 2.602 2.583 2.567 2.552 2.539 2.528 2.518 2.508 2.500 2.492 2.485 2.479 2.473 2.467 2.462 2.457 2.423 2.390 2.374 2.364 2.330 63.66 9.925 5.841 4.604 4.032 3.707 3.499 3.355 3.250 3.169 3.106 3.055 3.012 2.977 2.947 2.921 2.898 2.878 2.861 2.845 2.831 2.819 2.807 2.797 2.787 2.779 2.771 2.763 2.756 2.750 2.704 2.660 2.639 2.626 2.581 318.31 22.327 10.215 7.173 5.893 5.208 4.785 4.501 4.297 4.144 4.025 3.930 3.852 3.787 3.733 3.686 3.646 3.610 3.579 3.552 3.527 3.505 3.485 3.467 3.450 3.435 3.421 3.408 3.396 3.385 3.307 3.232 3.195 3.174 3.098 636.62 31.599 12.924 8.610 6.869 5.959 5.408 5.041 4.781 4.587 4.437 4.318 4.221 4.140 4.073 4.015 3.965 3.922 3.883 3.850 3.819 3.792 3.768 3.745 3.725 3.707 3.690 3.674 3.659 3.646 3.551 3.460 3.416 3.390 3.300 0.000 0.674 0.842 1.036 1.282 1.645 1.960 2.326 2.576 3.090 3.291 0% 50% 60% 70% 80% 90% 95% Confidence Level 98% 99% 99.8% 99.9% two-tails df 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 40 60 80 100 1000 z Park (CMUQ) 70-207 0.001 October 1, 2019 25 / 31 Confidence interval for a population mean µ CI for µ [ y ± Margin of Error], y is the sample mean. CI for µ with σ known Using a normal distribution directly from the sampling distribution of the sample mean, y ± z α/2 pσn . CI for µ with σ unknown Using s = qP (y i −y)2 n−1 and a t distribution, y ± t α/2 psn . When n is large, z α/2 and t α/2 are similar. Park (CMUQ) 70-207 October 1, 2019 26 / 31 What about for a population proportion p ? For categorical data, we record the proportions of observations in the categories. Ï The proportions of employees who participated in a management training / who didn’t. Ï The proportions of nations with low tariffs / moderate tariffs / high tariffs. Ï The proportions of students majored in Business Administration / Information Systems / Computer Science. Park (CMUQ) 70-207 October 1, 2019 27 / 31 Confidence interval for a population proportion p CI for µ [ y ± Margin of Error], y is the sample mean. Recall ³that we ´ computed the margin of error based on σ2 y ∼ N µ, n , σ y = pσn ⇒ margin of error = z α/2 × pσn . Park (CMUQ) 70-207 October 1, 2019 28 / 31 Confidence interval for a population proportion p CI for µ [ y ± Margin of Error], y is the sample mean. Recall ³that we ´ computed the margin of error based on σ2 y ∼ N µ, n , σ y = pσn ⇒ margin of error = z α/2 × pσn . CI for p [p ± Margin of Error], p is the sample proportion. r ¶ p(1 − p) p(1 − p) Compute the margin of error: p ∼ N p, , σp = n n r p(1 − p) ⇒ margin of error = z α/2 × . n µ Park (CMUQ) 70-207 October 1, 2019 28 / 31 Confidence interval for a rpopulation proportion p In the margin of error z α/2 × p(1 − p) , p is not known. n We substitute p for p . CI for p r [p ± Margin of Error] = p ± z α/2 × Park (CMUQ) 70-207 p(1 − p) n October 1, 2019 29 / 31 Confidence interval for a rpopulation proportion p In the margin of error z α/2 × p(1 − p) , p is not known. n We substitute p for p . CI for p r [p ± Margin of Error] = p ± z α/2 × p(1 − p) n Increased statistical error due to substituting p for p ? Should use t α/2 instead of z α/2 ? q p(1−p) s Compare y ± t α/2 pn and p ± z α/2 . n Ï Two estimates vs. one estimate. Park (CMUQ) 70-207 October 1, 2019 29 / 31 Confidence interval for a rpopulation proportion p In the margin of error z α/2 × p(1 − p) , p is not known. n We substitute p for p . CI for p r [p ± Margin of Error] = p ± z α/2 × p(1 − p) n Increased statistical error due to substituting p for p ? Should use t α/2 instead of z α/2 ? q p(1−p) s Compare y ± t α/2 pn and p ± z α/2 . n Two estimates vs. one estimate. Substituting p for p for the margin of error doesn’t produce an additional error. ⇒ Always use the standard normal and z α/2 . Ï Park (CMUQ) 70-207 October 1, 2019 29 / 31 Exercise Source: Rasmunssen Reports, 2012 Question: “Will today’s children be better off than their parents?” Population: U.S. Adults Sample size: N = 1000 Sample Data: 240 said yes. Find a 95% confidence interval for the proportion of adults who think that today’s children will be better off than their parents. Interpret your result substantively. Park (CMUQ) 70-207 October 1, 2019 30 / 31 Exercise Point estimate: p = 0.24. r r p(1 − p) .24(1 − .24) Margin of error : z 0.025 × = 1.96 = 0.026. n 1000 Confidence interval : 0.24 ± 0.026 = [0.214, 0.266] Interpretation Ï I am 95% confident that the proportion of the U.S. adults who think that today’s children will be better off than their parents is between 0.214 and 0.266. Park (CMUQ) 70-207 October 1, 2019 31 / 31