advertisement

Stat 921 Notes 4 Reading: Observational Studies, Chapters 2.5-2.7 I. Testing General Hypotheses (Section 2.6.1) So far, we have considered testing the null hypothesis of no treatment effect. There is an extension to test any hypothesized treatment effect. Let rT rC be the effect of treatment. Suppose we want to test H 0 : 0 (for no treatment effect 0 0 ). Under H 0 , rCsi rTsi si Z si . Thus, we can compute rC under H 0 based on the observed responses R and the observed treatment assignment Z ; call this value of rC under H 0 , rC ( 0 ) . Our test statistic is t ( Z , rC (0 )) and we reject for large values of the test statistic. The null hypothesis of the test statistic can be computed because H 0 , rC rC (0 ) . Example: Suppose we want to test the additive treatment effect H 0 : rT rC 1 Unit Group Observed Adjusted Ranks of Response Response Adjusted Responses I Zi Ri Zi Ri 1 qi 1 1 9 8 7 2 0 1 1 1 3 0 3 3 2 4 0 4 4 3 5 1 7 6 5 6 1 11 10 8 7 1 8 7 6 8 0 5 5 4 Let t ( Z , rC (0 )) be the Wilcoxon rank sum statistic, i.e., the sum of the ranks of the adjusted responses in the treated group. Then, the observed test statistic is T 7 5 8 6 26 . This is the largest possible rank sum for N 8 and the p-value is 1 8 1/ 70 0.014 . 4 For the creativity study, suppose we want to test the hypothesis that the intrinsic motivation increases scores by 2. intrinsic=c(12,12,12.9,13.6,16.6,17.2,17.5,18.2,19.1,19.3,19.8,20.3,20.5,20.6,21.3, 21.6,22.1,22.2,22.6,23.1,24,24.3,26.7,29.7); extrinsic=c(5,5.4,6.1,10.9,11.8,12,12.3,14.8,15,16.8,17.2,17.2,17.4,17.5,18.5,18.7, 18.7,19.2,19.5,20.7,21.2,22.1,24); wilcox.test(intrinsic-2,extrinsic,exact=TRUE,alternative="greater"); Wilcoxon rank sum test with continuity correction 2 data: intrinsic - 2 and extrinsic W = 330, p-value = 0.1274 alternative hypothesis: true location shift is greater than 0 Warning message: In wilcox.test.default(intrinsic - 2, extrinsic, exact = TRUE, alternative = "greater") : cannot compute exact p-value with ties There is no strong evidence against the hypothesis that the intrinsic treatment has an additive effect of 2. For a two-sided test, wilcox.test(intrinsic-2,extrinsic,exact=TRUE); Wilcoxon rank sum test with continuity correction data: intrinsic - 2 and extrinsic W = 330, p-value = 0.2548 alternative hypothesis: true location shift is not equal to 0 Note, when there are ties, wilcox.test does not compute exact pvalues and instead uses the normal approximation. To obtain exact p-values, we can install the exactRankTests package. library(exactRankTests); wilcox.exact(intrinsic-2,extrinsic,alternative="greater"); Exact Wilcoxon rank sum test data: intrinsic - 2 and extrinsic W = 330, p-value = 0.1276 alternative hypothesis: true mu is greater than 0 3 II. Confidence Intervals by Inverting a Test Under the model of an additive treatment effect, rTsi rCsi , a 1 confidence set for is obtained by testing each value of and collecting all values not rejected into a set A . For an effect increasing statistic, which includes all the tests from Chapter 2.4.3 of the book (e.g., Wilcoxon rank sum, Wilcoxon signed rank, difference in means, etc.), the test statistic is a decreasing function of . The argument is the following: * * Let . Let rC ( ), rC ( ) denote the potential responses * under control under , respectively based on the observed * * responses R , i.e., rC ( R Z ), rC ( R Z ) . Then, for any * z, (rsi rsi )(2 zsi 1) 0 for all s, i . Then, for an effect * increasing statistic, t ( Z , rC ( )) t ( Z , rC ( )) Since the test statistic is a decreasing function of , we can find the confidence interval by the bisection method (see Chapter 9 of Numerical Recipes in C, http://www.nrbook.com/a/bookcpdf.php ) The function uniroot in R finds the zero of a one-dimensional monotonic function using a bisection method. # Find one-sided 95% lower confidence interval for tau pval.minus.alpha.func=function(tau0,ytreated,ycontrol,alpha=.05){ pval=wilcox.exact(ytreated-tau0,ycontrol,alternative="greater")$p.value; pval-alpha; 4 } lower.ci.limit=uniroot(pval.minus.alpha.func,c(10,10),ytreated=intrinsic,ycontrol=extrinsic)$root; > lower.ci.limit [1] 1.400071 A one-sided 95% confidence interval for is approximately (1.40, ) . Two Sided Confidence Interval A two sided 95% confidence interval can be found by taking the intersection of two 97.5% one-sided confidence intervals (this is the shortest interval containing all that are rejected by neither of two one-sided, 0.025 level tests). # Find two-sided confidence inteval lower.twosided.ci.limit=uniroot(pval.minus.alpha.func,c(10,10),ytreated=intrinsic,ycontrol=extrinsic,alpha=.025)$root; upper.twosided.ci.limit=uniroot(pval.minus.alpha.func,c(10,10),ytreated=intrinsic,ycontrol=extrinsic,alpha=.025,side="less")$root; > lower.twosided.ci.limit [1] 1.000062 > upper.twosided.ci.limit [1] 6.599908 III. Point Estimates: Unbiased Estimates of the Average Effect (Section 2.7.1) Point Estimates: Unbiased Estimates of the Average Effect 5 Randomized experiments enable us to obtain unbiased estimates of the average treatment effect. Suppose there are N subjects and m of the subjects are randomly assigned to treatment, the rest to take the control Consider estimating the average causal effect of the treatment among the population of these N subjects, 1 N ACE rTsi rCsi , by the differences between the sample N i 1 mean of the outcomes in the treated group and the sample mean of the outcomes in the control group, N N 1 1 ˆ ACE (1 Z i ) Ri . Zi Ri N m m i 1 i 1 ˆ is unbiased estimator of ACE . Proposition: ACE Proof: Taking the expectation over the distribution of N equally likely random assignments, we have m 6 N 1 1 N ˆ E ( ACE ) E Z i Ri (1 Z i ) Ri N m i 1 m i 1 N 1 1 N E Z i rTsi (1 Z ) r i Csi N m i 1 m i 1 N 1 N 1 ( m / N ) rTsi ( N m) / N rCsi m i 1 N m i 1 1 N rTsi rCsi N i 1 ■ Comment: The proposition says that a randomized experiment provides an unbiased estimate of the mean treatment effect 1 N among the subjects in the study, N rTsi rCsi . It follows that i 1 if the units are randomly sampled from an infinite population, a randomized experiment provides an unbiased estimate of the mean treatment effect in the population over repeated experiments (where each experiment consists of randomly sampling the units and then randomly assigning the sampled units to the treatments). An unbiased estimate of the median treatment effect in the population cannot be obtained. To see this, consider two populations of units, one in which P(r 6, r 4) 1/ 3, P(r 8, r 6) 1/ 3, P( r 10, r 8) 1/ 3 , and another in which Ti Ci Ti Ci Ti 7 Ci P(rTi 10, rCi 4) 1/ 3, P(rTi 8, rCi 8) 1/ 3, P( rTi 6, rCi 6) 1/ 3 In the first population, the median treatment effect is 2 while in the second population, the median treatment effect is 0. But the marginal distributions of rCi and rTi are the same for the two populations, so the distribution of the treated and control subject outcomes will be the same in repeated experiments in which the units are randomly drawn from an infinite population. 8