Chapter 7 – Sampling Distribution Parameters are numerical descriptive measures for populations. For normal distribution, the location and shape are described by µ and σ. Often the values of parameters are unknown. Hence, you must rely on the sample to learn about these parameters. If you want the sample to provide reliable information about the population, you must select your sample randomly. The numerical descriptive measures you calculate from the sample are called statistics. These statistics vary or change for each different random sample you select; that is, they are random variables. Definition: The Sampling distribution of a statistic is the probability distribution for the possible values of the statistic that result when random sample of size 𝑛 are repeatedly drawn from the population. i.e The probability distributions for the statistic are called Sampling Distributions. They tell us what values of the statistics can occur and how often each value occurs. Example: A population consists of 𝑁 = 5 numbers: 3, 6, 9, 12, 15. If a random sample of size 𝑛 = 3 is selected without replacement. 1. Find the sampling distributions for the sample mean 𝑥̅ . Sample values 𝑥̅ 3, 6, 9 6 3, 6, 12 7 3, 6, 15 8 3, 9, 12 8 3, 9, 15 9 3, 12, 15 10 6, 9, 12 9 6, 9, 15 10 6, 12, 15 11 9, 12, 15 12 The sampling distribution for 𝑥̅ 𝑥̅ 𝑃(𝑥̅ ) 6 0.1 7 0.1 8 0.2 9 0.2 10 0.2 11 0.1 12 0.1 2. Find 𝐸(𝑥̅ ) 𝐸(𝑥̅ ) = ∑ 𝑥̅ ∗ 𝑃(𝑥̅ ) = 0.6 + 0.7 + 1.6 + 1.8 + 2 + 1.1 + 1.2 = 9 3. Find the population mean µ= 3+6+9+12+15 5 =9 Note 𝐸(𝑥̅ ) = µ If the population mean µ is unknown, you might choose several statistics as an estimator; the sample mean 𝑥̅ is more widely used. The Central Limit Theorem (CLT): If random samples of 𝑛 observations are drawn from a nonnormal population with mean µ and standard deviation σ, then, when 𝑛 is large (𝑛 ≥ 30), the sampling distribution of the sample mean 𝑥̅ is 𝜎 approximately normally distributed with mean µ and standard deviation 𝑛 . √ Note: The approximation becomes more accurate as 𝑛 becomes large. Theorem: If random samples of 𝑛 observations are drawn from a normal population with mean µ and standard deviation σ, the sampling distribution of the 𝜎 sample mean 𝑥̅ is normally distributed with mean µ and standard deviation 𝑛 . √ ̅ i.e The sampling distribution of the sample mean 𝒙 if the sample of 𝑛 measurements is selected from: A population with mean µ and standard deviation σ 𝑥̅ will have mean µ and standard deviation population has a normal distribution 𝑥̅ exactly 𝑁(µ, 𝑛 ) regardless of the sample size 𝑛 population has a nonnormal distribution 𝑥̅ approximately 𝑁(µ, 𝑛 ) for large samples ( 𝑛 ≥ 30) 𝜎 √𝑛 𝜎2 𝜎2 Definition: The Standard deviation of a statistic used as an estimator of a population parameter is also called the Standard Error of the estimator ( SE ). Note: 𝑥: 𝑁(µ, 𝜎 2 ) → 𝑍 = 𝑥̅ : 𝑁(µ, 𝜎2 𝑛 )→𝑍= 𝑥− µ 𝜎 𝑥− µ 𝜎 ⁄ √𝑛 ∶ 𝑁 (0, 1) ∶ 𝑁 (0, 1) Example: if we take a sample of size 36 from a population with mean 2 and variance 25. Find: 1. Sampling distribution of 𝑥̅ 2. 𝐸(𝑥̅ ) 3. 𝑣𝑎𝑟(𝑥̅ ) 4. 𝑆𝐸(𝑥̅ ) 5. The distribution of 6(𝑥̅ −2) 5 6. 𝑃(𝑥̅ ≤ 3) 7. 𝑃(0 ≤ 𝑥̅ ≤ 2) Solutions: 1. 𝑛 = 36 ≥ 30. Hence, by CLT 𝑥̅ 𝑎𝑝𝑝𝑟𝑜𝑥𝑖𝑚𝑎𝑡𝑒𝑙𝑙𝑦 𝑁(µ, 𝜎2 𝑛 ). 25 For this example 𝑥̅ : 𝑁(2, 36 ) 2. 𝐸(𝑥̅ ) = µ = 2 3. 𝑣𝑎𝑟(𝑥̅ ) = 4. 𝑆𝐸(𝑥̅ ) = 25 36 5 6 (𝑥̅ −2) 6(𝑥̅ −2) = 5 is approximately 5⁄6 𝑥̅ − µ 3−µ ≤ 3) = 𝑃(𝜎⁄ 𝑛 < 𝜎⁄ 𝑛) √ √ 3−2 = 𝑃( 𝑍 < 5⁄6 ) = 𝑃(𝑍 < 1.2) = 0.8849 0−2 2−2 ≤ 𝑥̅ ≤ 2) = 𝑃 ( 5⁄6 < 𝑍 < 5⁄6 ) 5. Sampling distribution of 6. 𝑃(𝑥̅ 7. 𝑃(0 = 𝑃(−2.4 < 𝑍 < 0 ) = 𝑃 (𝑍 < 0) − 𝑃(𝑍 < −2.4) = 0.5 − 0.0082 = 0.4918 𝑁(0,1 ) Example: suppose that 𝑥: 𝑁(20, 36). Take a random sample of size 25. Find: 1. The distribution of 𝑥̅ 2. 𝑃(𝑥 ≤ 25) 3. 𝑃(𝑥̅ > 1) Solutions: 36 1. 𝑥̅ : 𝑁(20, 25 ) 2. 𝑃(𝑥 ≤ 25) = 𝑃( 𝑥−µ 𝜎 ≤ 25−20 ) 6 = 𝑃(𝑍 ≤ 0.83) = 0.7967 𝑥̅ −𝜇 3. 𝑃(𝑥̅ > 24) = 𝑃 (𝜎⁄ √ > 𝑛 24−20 ) 6⁄5 = 𝑃(𝑍 > 3.33) = 𝑃(𝑍 < −3.33) = 0.0004 Example: the distribution of the grades of Stat 101 is normal with mean 70 and variance 16. If we take a random sample of 36 students. Find: 1. The probability that the mean of grades of 36 students will be greater than 72. 2. The probability that the total of grades of 36 students will be less than 2556. Solutions: 1. 𝑃(𝑥̅ > 72) = 𝑃 (𝑍 > 72−70 ) 4⁄6 = 𝑃(𝑍 > 3) = 𝑃(𝑍 < −3) = 0.0013 ∑𝑥 2. 𝑃(∑ 𝑥 < 2556) = 𝑃 ( 𝑛 < 2556 𝑛 ∑𝑥 ) = 𝑃 ( 36 < 2556 36 ) = 𝑃(𝑥̅ < 71) = 𝑃(𝑍 < 1.5) = 0.9332 H.W: suppose a random sample of size 25 observations is selected from a population that is normally distributed with mean 106 and standard deviation 12 find: 1. The mean and the standard deviation of the sampling distribution of the sample mean 𝑥̅ . 2. The probability that 𝑥̅ exceed 110. 3. The probability that the sample mean deviates from the population mean by no more than 4. Chapter 8 – Large Sample Estimation Inference Statistics: making inferences about a population using information contained in a sample. Population → Parameter → 𝜇, 𝜎 2 Sample → Statistic → 𝑥̅ , 𝑆 2 We estimate the parameter by statistic Example: The population mean 𝜇 estimate by 𝑥̅ 𝜇̂ = 𝑥̅ Methods for making inferences about population parameters fall into one of two categories: 1. Estimation: estimating or predicting the value of the parameter. 2. Hypothesis testing: making a decision about the value of a parameter based on some preconceived idea. Definition: An estimator is a rule, usually expressed as a formula, that tells us how to calculate an estimate based on information in the sample. Definition: An estimate: the value that calculated by the estimator. Type of estimators: 1. Point estimation: A single number calculated to estimate the parameter. We estimate 𝜇̂ 𝑏𝑦 𝑥̅ , 𝜎̂ 2 𝑏𝑦 𝑆 2 , and 𝜎̂ 𝑏𝑦 𝑆 Example: suppose we have the following random sample: 12, 18, 10, 6, 14 from a distribution with mean µ and variance 𝜎 2 1. Find the point estimate of 𝜇 (i.e. calculate 𝑥̅ ) 𝜇̂ = 𝑥̅ = ∑𝑥 𝑛 = 12+18+10+6+14 5 = 12 2. Find the point estimate of 𝜎 2 ∑(𝑥𝑖 − 𝑥̅ )2 ∑ 𝑥𝑖2 − 𝑛𝑥̅ 2 2 2 𝜎̂ = 𝑆 = = = 20 𝑛−1 𝑛−1 3. Find the point estimate of 𝜎 𝜎̂ = 𝑆 = √20 = 4.47 An estimator is unbiased if the mean of its sampling distribution equal the parameter of interest. For example, 𝐸(𝜇̂ ) = 𝐸(𝑥̅ ) = 𝜇. Then 𝑥̅ is unbiased. Of all the unbiased estimators, we prefer the estimator whose sampling distribution has the smallest spread or variability( i.e. minimum variance) Example: let 𝑥1 , 𝑥2 , ⋯ , 𝑥16 be a random sample from 𝑁(𝜇, 25). Which is the best estimator for 𝜇, 𝑥1 or 𝑥̅ Solution: 𝑥1 : 𝑁(𝜇, 25) 25 𝑥̅ : 𝑁(𝜇, 16) 𝐸(𝑥̅ ) = 𝜇 𝑎𝑛𝑑 𝐸(𝑥1 ) = 𝜇 hence, they are unbiased 25 But 𝑣𝑎𝑟(𝑥1 ) = 25 𝑎𝑛𝑑 𝑣𝑎𝑟(𝑥̅ ) = 16 Note that the variance of 𝑥̅ is smaller than the variance of 𝑥1 , then 𝑥̅ is better than 𝑥1 as estimator. Definition: Error of estimation is the distance between an estimate and the true value of the parameter. Definition: The Margin of Error: the maximum distance between the estimator and the true value of the parameter. It denoted by 𝑀𝐸 or (1 − 𝛼)𝑀𝐸 and calculated as 𝑀𝐸 = ± 𝑍𝛼 ∗ 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑒𝑟𝑟𝑜𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑜𝑟 2 Where α = level of significance (0% − 100%) 𝛼 𝑍𝛼 = 𝑃1− 𝛼 it is the (1 − 2 ) % percentile. 2 2 Example: for given 𝛼 = 0.1 find 𝑍𝛼 and 𝑍𝛼 2 Solution: 𝑍𝛼 = 𝑍0.1 = 𝑃0.9 i.e. if 𝑍: 𝑁(𝑜, 1) find 𝑐 such that 𝑃(𝑍 < 𝑐) = 0.9 From the table 𝑐 = 𝑍0.1 = 1.28 𝑍𝛼 = 𝑍0.05 = 𝑃0.95 = 1.645 2 H.W: for given 𝛼 = 0.05 find 𝑍𝛼 and 𝑍𝛼 2 Example: given a random sample of size 𝑛 = 36 we found that the average of this sample was 65. If we know that 𝜎 = 20. Find the margin error for µ when 𝛼 = 0.05 Solution: the 95% 𝑀𝐸 of µ = ± 𝑍𝛼 ∗ 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑒𝑟𝑟𝑜𝑟 𝑜𝑓 𝑥̅ 2 = ± 𝑍𝛼 ∗ 2 𝜎 √𝑛 𝛼 = 0.05 → 𝑍𝛼 = 𝑍0.025 = 𝑃0.975 = 1.96 2 20 = 3.33 6 √𝑛 the 95% 𝑀𝐸 of µ = ±1.96 ∗ 3.33 = ±6.6 𝑆𝐸(𝑥̅ ) = 𝜎 = If σ is unknown, we replace it by 𝑆. Where 𝑆 is the sample standard deviation such that 𝑛 ≥ 30. The 95% margin error = ±1.96 𝑆 √𝑛 Example: A sample of 36 observations has mean 18 and standard deviation 2.5 1. Find the point estimate of 𝜇 𝜇̂ = 𝑥̅ = 18 2. Find a 95% margin of error Margin of error = ±𝑍0.025 = ±1.96 2.6 = ±0.8493 6 𝑆 √𝑛 = ±1.96 𝑆 √𝑛 2.Intervat Estimation: Is a rule for calculating two numbers ( say a and b) to create an interval that you are fairly certain contains the parameter of interest. This interval called Confidence Interval and denoted by 𝐶. 𝐼 Definition: the probability that a confidence interval will contain the estimated parameter is called the confidence coefficient, and denoted by 1 − 𝛼. Example: a 95% confidence interval means that the confidence coefficient or the probability that the interval will contain the estimated parameter is 0.95 i.e. 𝑃(𝑎 < 𝜇 < 𝑏) = 0.95 Definition: A 100(1 − 𝛼)% 𝐶. 𝐼 for the population mean µ is constructed as [ 𝑥̅ ± 𝑍𝛼 ∗ 2 𝜎 √𝑛 ] = [ 𝑥̅ − 𝑍𝛼 ∗ 2 𝜎 √𝑛 , 𝑥̅ + 𝑍𝛼 ∗ 2 𝜎 √𝑛 ] If σ is unknown we replace it by S (if 𝑛 ≥ 30) such that [ 𝑥̅ ± 𝑍𝛼 ∗ 2 𝑆 √𝑛 ] Example: suppose that we have a random sample of size 25 from a normal distribution with mean µ and variance 16. We found that the sample mean is 12 and the sample variance is 15. 1. Find a 90% 𝐶. 𝐼 for µ Solution: 𝑛 = 25, 𝜎 2 = 16, 𝜇 =? , 𝑥̅ = 12 𝑎𝑛𝑑 𝑆 2 = 15 Confidence level : 100(1 − 𝛼)% = 90% 𝛼 = 0.1 𝑍𝛼 = 𝑍0.05 = 𝑃0.95 = 1.645 2 Hence, the 90% 𝐶. 𝐼 for µ is [ 𝑥̅ ± 𝑍𝛼 ∗ 2 𝜎 √𝑛 4 ] = [ 12 ± 1.645 ∗ 5 ] = [ 12 ± 1.31 ] = [ 10.7, 13.31 ] This means that: If we choose 100 intervals of the same length, we expect to find that 90 of them contains the true mean µ OR We are 90% confidence that µ lies within the interval [ 10.7, 13.31 ] 2. Find a 95% 𝐶. 𝐼 for µ Solution: Confidence level : 100(1 − 𝛼)% = 95% 𝛼 = 0.05 𝑍𝛼 = 𝑍0.025 = 𝑃0.975 = 1.96 2 Hence, the 95% 𝐶. 𝐼 for µ is [ 𝑥̅ ± 𝑍𝛼 ∗ 2 𝜎 √𝑛 4 ] = [ 12 ± 1.96 ∗ 5 ] = [ 12 ± 1.568 ] = [ 10.43, 13.57 ] The lower confidence limit = 10.7 The upper confidence limit = 13.31 Note that in point estimation we have high accuracy, but in interval estimation we have a high confidence. The increased width is necessary to increase the confidence. The only way to increase the confidence without increasing the width of the interval is to increase the size, n. Best interval: 1. Has larger confidence coefficient. 2. Has the shortest interval width (length). H.W: Find a 95% 𝐶. 𝐼 for µ when 𝑛 = 36, 𝑥̅ = 13.1 𝑎𝑛𝑑 𝑆 2 = 3.42 Chapter 9 – Large Sample test of hypotheses In practical situation, statistical inference can involve either estimating a population parameter or making decisions about the value of the parameter. For example, suppose you need to test that the average income of some workers is greater than 250J D. Definition: A hypothesis is a statement that may be a correct or not. Large – sample test of hypothesis for µ Steps to perform the test: Step1: write the Null and Alternative hypothesis. The Alternative hypothesis, 𝐻𝑎 𝑜𝑟 𝐻1, is generally the hypothesis that the researcher wishes to support. It may be one of the following three choices: µ is greater than 𝜇0 → 𝐻𝑎 : 𝜇 > 𝜇0 µ is less than 𝜇0 → 𝐻𝑎 : 𝜇 < 𝜇0 µ is different from 𝜇0 → 𝐻𝑎 : 𝜇 ≠ 𝜇0 The Null hypothesis, 𝐻0 , is a contradiction of the alternative hypothesis. It has the equality all the time; 𝐻0 : 𝜇 = 𝜇0 . Step2: Find the test statistic which is a single number calculated from the sample data. The point estimator for µ is 𝑥̅ . If 𝑥̅ : 𝑁(𝜇, 𝜎2 𝑥̅ −𝜇 ), then the test statistic is 𝑍 = 𝜎⁄ 𝑛 √𝑛 . Since we want to make a decision about specific number 𝜇0 , we will use the test statistic 𝑍𝑐 = 𝑥̅ − 𝜇0 𝜎 ⁄ √𝑛 If 𝜎 is unknown and 𝑛 ≥ 30, we estimate the test statistic by 𝑍𝑐 = Step3: Find the critical value In case of Upper side or right tailed test 𝐻𝑎 : 𝜇 > 𝜇0 Lower side or left tailed test 𝐻𝑎 : 𝜇 < 𝜇0 Two sides or two tailed test 𝐻𝑎 : 𝜇 ≠ 𝜇0 The critical value C.V 𝑍𝛼 −𝑍𝛼 ±𝑍𝛼 2 𝑥̅ −𝜇0 𝑆⁄√𝑛 . Step4: State your conclusion Reject 𝐻0 or don't reject 𝐻0 (accept 𝐻0 ) In case of: 1. 𝐻𝑎 : 𝜇 > 𝜇0 we reject 𝐻0 if 𝑍𝑐 > 𝑍𝛼 The rejection region is 2.𝐻𝑎 : 𝜇 < 𝜇0 we reject 𝐻0 if 𝑍𝑐 < −𝑍𝛼 The rejection region is 3. 𝐻𝑎 : 𝜇 ≠ 𝜇0 we reject 𝐻0 if |𝑍𝑐 | > 𝑍𝛼 2 i.e. 𝑍 > 𝑍𝛼 𝑜𝑟 𝑍 < −𝑍𝛼 2 The rejection region is 2 Example: a random sample of size 36 given the following results, 𝑥̅ = 72, 𝑆 2 = 100. For 𝛼 = 0.05 test wither the population mean, µ, is greater than 70 or not. Step1: 𝐻0 : 𝜇 = 70 vs 𝐻𝑎 : 𝜇 > 70 𝑥̅ −𝜇0 𝑆⁄√𝑛 Step2: 𝑍𝑐 = 72−70 10⁄6 = = 1.2 Step3: C.V is 𝑍𝛼 = 𝑍0.05 = 1.645 Step4: since 𝐻𝑎 > 70 we reject 𝐻0 if 𝑍𝑐 > 𝑍𝛼 but 1.2 < 1.645 We don't reject 𝐻0 (or we accept 𝐻0 ). Example: given that 𝑛 = 100, 𝑥̅ = 22, 𝑆 2 = 49. For 𝛼 = 0.01 test the hypothesis that the population mean is not equal 21 (or is different than 21). 1. 𝐻0 : 𝜇 = 21 vs 𝐻𝑎 : 𝜇 ≠ 21 2. 𝑍𝑐 = 𝑥̅ −𝜇0 𝑆⁄√𝑛 = 22−21 7⁄10 = 1.43 3. C.V is ± 𝑍𝛼 = ± 𝑍0.005 = ± 2.575 2 4. Since 𝐻𝑎 ≠ 21 we reject 𝐻0 if |𝑍𝑐 | > 𝑍𝛼 2 but 1.43 < 2.575 we don't reject 𝐻0 Example: let 𝑥1 , 𝑥2 , ⋯ 𝑥36 be a random sample such that 𝑥̅ = 7, 𝑆 2 = 9. Conduct the test. Do the data provide sufficient evidence to indicate that 𝜇 < 10. 𝛼 = 0.05 1. 𝐻0 : 𝜇 = 10 vs 𝐻𝑎 : 𝜇 < 10 2. 𝑍𝑐 = 𝑥̅ − 𝜇0 𝑆⁄√𝑛 = 7 − 10 = −6 3⁄6 3. C.V is − 𝑍𝛼 = −𝑍0.05 = −1.645 4. Since 𝐻𝑎 ≠ 21 we reject 𝐻0 if 𝑍𝑐 < −𝑍𝛼 −6 < −1.645 we reject 𝐻0 which means that there is sufficient evidence to indicate that µ is not equal to 10. Testing hypothesis by using the P-value Definition: The P-value is the smallest value of α for which 𝐻0 can be rejected. It is a probability calculated using the test statistics. Steps to perform the test: Step1: write the Null and Alternative hypothesis. Step2: Find the test statistic 𝑍𝑐 = 𝑥̅ −𝜇0 𝑆⁄√𝑛 Step3: Find the P-value If 𝐻𝑎 : 𝜇 > 𝜇0 , the 𝑃 − 𝑣𝑎𝑙𝑢𝑒 = 𝑃(𝑍 > 𝑍𝑐 ). If 𝐻𝑎 : 𝜇 < 𝜇0 , the 𝑃 − 𝑣𝑎𝑙𝑢𝑒 = 𝑃(𝑍 < 𝑍𝑐 ). If 𝐻𝑎 : 𝜇 ≠ 𝜇0 , the 𝑃 − 𝑣𝑎𝑙𝑢𝑒 = 𝑃(|𝑍| > 𝑍𝑐 ) = 2 𝑃(𝑍 > 𝑍𝑐 ). Step4: State your conclusion Reject 𝐻0 if 𝑃 − 𝑣𝑎𝑙𝑢𝑒 ≤ 𝛼 Don't reject 𝐻0 if 𝑃 − 𝑣𝑎𝑙𝑢𝑒 > 𝛼 Example: In a random sample of size 100. We found that the average weight of students are normally distributed with the following results: 𝑥̅ = 58.4, 𝑆 2 = 25. Test the hypothesis that the population mean is different than 60 using 𝛼 = 0.05 Solution: 1. 𝐻0 = 60 vs 𝐻𝑎 ≠ 60 2. 𝑍𝑐 = 𝑥̅ − 𝜇0 𝑆 ⁄ √𝑛 = 58.4 − 60 = −1.2 5⁄10 5. Since 𝐻𝑎 ≠ 60, the 𝑃 − 𝑣𝑎𝑙𝑢𝑒 = 𝑃(|𝑍| > 1.2) = 2 𝑃(𝑍 < −1.2) = 2 ∗ 0.1151 = 0.2302 6. Since 𝐻𝑎 ≠ 60 we reject 𝐻0 if 𝑃 − 𝑣𝑎𝑙𝑢𝑒 ≤ 𝛼 but 0.2302 > 0.05 we don't reject 𝐻0 Example: Find the P-value for the following large-sample Z test: a. A right tailed test with observed 𝑧 = 1.15 𝑃 − 𝑣𝑎𝑙𝑢𝑒 = 𝑃(𝑍 > 𝑍𝑐 ) = 𝑃(𝑍 > 1.15) = 𝑃(𝑍 < −1.15) = 0.1251 b. A two tailed test with observed 𝑧 = −2.78 𝑃 − 𝑣𝑎𝑙𝑢𝑒 = 𝑃(|𝑍| > 𝑍𝑐 ) = 2𝑃(𝑍 > 2.78) = 2𝑃(𝑍 < −2.78) = 0.0054 c. A left tailed test with observed 𝑧 = −1.81 𝑃 − 𝑣𝑎𝑙𝑢𝑒 = 𝑃(𝑍 < 𝑍𝑐 ) = 𝑃(𝑍 < −1.81) = 0.0351 H.W: A random sample of n=35 observations produced a mean 𝑥̅ = 2.4 and 𝑆 = 0.29. suppose your research objective is to show that the population mean µ exceeds 2.3 a. give the null and alternative hypotheses for the test. b. locate the rejection region for the test using a 5% significance level. c. Find the standard error of the mean. d. conduct the test. Do the data provide a sufficient evidence to indicate that 𝜇 > 2.3 e. calculate the P-value for the test statistic in part d. f. use the P-value to draw a conclusion at the 5% significance level.