advertisement

Power: Example Questions and Outline Model Answers Question 1 Two groups of 20 schoolchildren (20 subjects per group) were tested in a study of anagram solving. The mean time to solve the anagrams for group 1 (who had not received special literacy training) was 10.5 seconds (standard deviation= 6.7 seconds). The corresponding figures for group 2 (who had received the literacy training) were 13.9 and 6.9 seconds. (a) Explain the elements which appear in the approximate formula for calculating the statistical power of a two-tailed test, given below. (20% of question marks) (b) Use the formula to calculate the observed power of this study, for tests at the 5% significance level. Remember to adjust the formula because 1-tailed tests would be carried out in this study. Based on the information given below you will be able to express your answer as a pair of values within which the observed power lies. Show your working. (35% of question marks) (c) If there were a real group difference as large as the one observed in the data, then what is your estimate of how often the null hypothesis would fail to be rejected? What is the name for this quantity? (10% of question marks) (d) By using the approximate formula below, estimate the sample size that would be required in each group in order that the above experiment would have 80% power to detect a group difference equivalent to that measured in the experiment. Base your calculations on one-tailed tests at the 5% significance level. The formula is for a two-tailed test and needs adjusting appropriately. Show your working. (35% of question marks) Approximate Power Formula power = z-1{d*sqrt(n/2) – z(1-/2)} where sqrt(x) is the square-root of x Approximate Sample Size Formula required sample size per group = 2*{z(power) +z(1-/2)}2/d2 Data That May Be Needed in The Calculations .050 .100 .200 .300 .400 .500 .600 .700 .800 .900 z(1-/2) 1.960 1.645 1.282 1.036 .842 .674 .524 .385 .253 .126 x -1.95 -1.65 -1.30 -1.00 -0.85 -0.65 -0.50 -0.25 -0.05 0.00 z-1{x} 0.026 0.049 0.097 0.159 0.198 0.258 0.309 0.401 0.480 0.500 Note: the z-1{x} function is symmetrical about x=0; so z-1{1.95}=1-z-1{-1.95} etc. Question 1: MODEL ANSWER Note that the perfect answer can be pretty short:(a) Elements to explain: z and z-1; d; n; α. z( ) is the inverse cumulative distribution function of the normal distribution -- the function takes a probability and converts it into a z-score. Thus, a probability of 0.975 is converted to a z-score of +1.96 because the z-score below which 97.5% of the normal distribution lies is +1.96. The function z-1( ) is the inverse of z and is the cumulative distribution function of the normal distribution -- the function takes a z-score and converts it to a probability. Thus, a z-score of +1.96 is converted to a probability of 0.975, i.e. 97.5% of the normal distribution lies below this z-score. (An explanation of z and z-1 without the exact jargon, but which is accurate, will also get full marks here.) d is a measure of the between-groups effect size: it is the mean difference between the groups divided by their pooled standard deviation. n is the number of subjects in each of the 2 groups (and the group size must be equal for this formula). α is the Type I error rate, conventionally 5%. (b) In the example n=20 and α=0.05. To calculate observed power we need the observed effect size. To calculate the observed value of d: the mean difference between the groups is 3.4 secs and the groups have s.d. of 6.7 and 6.9. When we pool the s.d. of equal-sized groups we can take the average s.d. i.e. 6.8. Thus d=3.4/6.8=0.5. The adjustment to the formula that is required for a one-tailed test is to replace z(1-α/2) with z(1-α). To see this, consider doing a z-test on a mean which is supposed to be drawn from a standard normal distribution (mean=0; s.d.=1). To test (two-tailed, with α=0.05) if the observed value of the mean is not significantly different from zero then the critical z-values are z(0.975)=+1.96 and z(0.025)=-1.96. In other words, the value must lie between z(α/2) and z(1-α/2). To test (one-tailed, with α=0.05) if the observed value is not significantly above zero the critical z-value is z(0.95), i.e. the value must be below z(1-α). the term in the {} brackets must be worked out first:{0.5*sqrt(10) - z(0.95)} from the table given if α=0.1 then z(1- α/2)=z(0.95)=1.645 so the {} term works out to be -0.06 thus, from the table, z-1{-0.06} lies between 0.480 and 0.401, but much closer to 0.480. Thus, the observed power for this experiment is between 40 and 48%. (c) Between 52% and 60% of the time (but closer to 52%). This is known as the Type II error rate. (d) To calculate the required sample size for 80% power by a 1-tailed t-test we must use an adjusted formula once again. Again, replace z(1-α/2) with z(1-α). [Check: a 1-tailed test should increase power and so reduced required number of subjects. z(1-α/2) would be replaced with a smaller value z(1-α) in the formula for required sample size and so this is correct]. the {} term is [z(0.8) + z(0.95)]2 to look up z(0.8) we need to look up z(1-0.4/2) thus we need to use the table for α=0.4. Thus, the {} term =[0.842 + 1.645]2 = 6.185 and so the required sample size in each group = 2*6.185/0.52= 49.5 = 50.