Multiple comparisons http://newyorkette.com/wp-content/070430_cartoon_8_a12131_p465.gif How we decide which means are truly different from one another? • The ANOVA only test the null hypothesis that the treatment means were all sampled from the same distribution Two general approaches • A posteriori comparisons (unplanned; after the fact) • A priori comparisons (planned; before the fact) Effects of early snowmelt on alpine plant growth Three treatment groups and 4 replicates per treatment: 1. Unmanipulated 2. Heated with permanent solar-powered heating coils that melt spring snow pack earlier in the year than normal 3. Controls, fitted with heating coils that are never activated Data Unmanipulated Control Heated 10 12 12 13 9 11 11 12 Y2 10.75 12 13 15 16 Y3 14.00 Y1 11 .75 Y 12.17 ANOVA table for one-way layout Source df Sum of squares Mean square Among groups 2 22.16 11.08 Within groups 9 19.50 2.17 11 41.67 Total P=tail of F-distribution with (a-1) and a(n-1) degrees of freedom F-ratio 5.11 P-value 0.033 A posteriori comparisons • We will use Tukey’s “honestly significant differences (HSD) • It controls for the fact that we are carrying out many simultaneous comparisons. • the P-value is adjusted downward for each individual test to achieve an experimentwise error rate α=0.05 The HSD 1 1 MS residual n n j i HSD q 2 Where: q= is the value from a statistical table of the studentized range distribution n = sample size The HSD 1 1 2.17 4 4 HSD 3.199 2.3562 2 Y3 14.00 Y1 11 .75 3.25, P<0.05 1, NS Y2 10.75 2.25, NS Dunnett’s test SEt MS residual 1 1 n n j i For each treatment vs. control pair, CV= d(m,df) SEt; Where m (number of treatments) includes the control and d is found in tables for Dunnett’s test. Use Dunnett’s test by comparing treatment furthest from control first, then next furthest from control, etc. Dunnett’s test 2 SEt 2.17 1.0416 4 d control_ vs _ heated dtest X control X treatment SEt 10.75 14 3.1201 1.0416 d control_ vs _ unmanipulated 10.75 11.72 0.96003 1.0416 d-critical(m=3,df=9)=2.61 But… • Occasionally posterior tests may indicate that none of the pairs of means are significantly different from one another, even if the overall F-ratio led to reject the null-hypothesis! • This inconsistence results because the pairwise test are not as powerful as the overall F-ratio itself. A priori (planned) comparisons • They are more specific • Usually they are more powerful • It forces you to think clearly about which particular treatment differences are of interest A priori (planned) comparisons • The idea is to establish contrasts, or specified comparisons between particular sets of means that test specific hypothesis. • These test must be orthogonal or independent of one another • They should represent a mathematical partitioning of the among group sum of squares To create a contrast • Assign an number (positive, negative or 0) to each treatment group • The sum of the coefficients for a particular contrast must equal 0 (zero) • Groups of means that are to be averaged together are assigned the same coefficient • Means that are not included in the comparison of a particular contrast are assigned a coefficient of 0 Contrast I (Heated vs. Non-Heated) Unmanipulated (1) Y1 11 .75 Control (1) Heated (-2) Y2 10.75 Y3 14.00 a MS contrast MS heated _ vs _ unheated n( ciYi ) 2 i 1 a 2 c i 1 i 4 ((1)(11.75) (1)(10.75) (2)(14.00)) 2 20.16 2 2 2 1 1 2 F-ratio= 20.16/2.17=9.2934 With 1 df F-critical1,9 =5.12 Contrast I (Heated vs. Non-Heated) using formula for non equal samples Unmanipulated (1) Y1 11 .75 k Control (1) Heated (-2) Y2 10.75 Y3 14.00 SS contrast ni Yi Y , k _ groups 2 i MS heated _ vs _ unheated 8(11.25 12.167) 2 (4)(14.00 12.167) 2 20.16 F-ratio= 20.16/2.17=9.2934 With 1 df F-critical1,9 =5.12 More information on Sokal and Rohlf (2000) Biometry Contrast II (Control vs. Unmanipulated) Unmanipulated (1) Y1 11 .75 Control (-1) Y2 10.75 Heated (0) Y3 14.00 a MS contrast n( ciYi ) 2 i 1 a 0 (1)(1) (1)( 1) (2)(0) 2 c i i 1 4 ((1)(11.75) (1)(10.75) (0)(14.00)) 2 MS unmanipulated _ vs _ control 2 2 2 1 1 F-ratio= 2/2.17=0.9217 With 1 df F-critical1,9 =5.12 In order to create additional Orthogonal contrasts • If there are a treatment groups, at most there can be (a-1) orthogonal contrasts created (although there are many possible sets of such orthogonal contrasts). • All of the pair-wise cross products must sum to zero. In other words, a pair of contrast Q and R is independent if the sum of the products of their coefficients CQi and CRi equals zero. a n c c 0 i Qi Ri i 1 A priori contrasts Source df SS MS F-ratio P 2 22.16 11.08 2.11 0.033 Heated vs. Non-Heated 1 20.16 20.16 9.29 <0.025 Control vs. Unmanipulated 1 2.00 2 0.92 NS 9 19.50 2.17 11 41.66 Treatments Residual