Data Analysis Tutorial (See teleassessment paper and DataAnalysis.doc for examples) References [1] Anderson, M, Whitcomb, P, DOE Simplified: Practical Tools for Effective Experimentation, Productivity Inc., 2000. [2] Montgomery, D, Runger, G, Applied Statistics and Probability for Engineers, Wiley, 1994. [3] Montgomery, D, Design and Analysis of Experiments, 5th ed, Wiley, 2001. [4] Portney, LG, Watkins, MP, Foundations of Clinical Research: Applications to Practice, Appleton & Lange, 1993 [5] Glantz, SA, Primer of Biostatistics, 4th ed, McGraw Hill, 1997. Web resources Excel for one-way ANOVA http://www.mnstate.edu/wasson/ed602excelss13.htm http://www.sytsma.com/phad530/aovexel.html http://www.isixsigma.com/library/content/c021111a.asp Excel for independent t-test http://www.mnstate.edu/wasson/ed602excelss11.htm Excel for dependent t-test http://www.mnstate.edu/wasson/ed602excelss12.htm Power analysis http://www.biostat.ucsf.edu/sampsize.html Calculating effect size indices http://web.uccs.edu/lbecker/Psy590/es.htm G-Power freeware for power analysis http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/ Engineering statistics handbook http://www.itl.nist.gov/div898/handbook/index.htm UMN Stats Consulting Service (exc!!) http://www.stat.umn.edu/consulting/ 533580180 Page 1 of 10 3/8/2016 533580180 Page 2 of 10 3/8/2016 Tutorials Tutorial on Repeated Measures (Blocked) Design and Analysis Refs: [4] 374-377, 387-390 [3] 126-133 [2] 660-667 [1] 34-35 For repeated measures design [4] (or blocked analysis [1]). Uses calculation method of [4] which is the same as [2] and [3]. See Table 20.4 in [4] for an example. (For alternate method, see tutorial in Appendix and [1] Tables 2-3 and 2-4) All calculations and tables created in Excel. See end of tutorial for letting built-in Excel tools do the hard work. For example, see file repeatedmeasuresexample.xls 1. Determine what is a block. Blocks can be subjects or anything else where variation is expected. Every treatment must be conducted once and only once in every block. 2. Create data table with one row for each block and one column for each treatment. For example, see [4] Table 20.4. [2] and [3] have tables in transpose order. 3. Create column for sums of rows, Sum(Si) and create row for sums of columns, Sum(Xi) 4. Compute from the table. See [4] Table 20.4 for math formulas. See Excel example file for computation formulas n = number of blocks k = number of treatments Sum(X) = sum of all data points Sum(X^2) = sum of all squared data points 2 Sum(X)*2/nk = X ij n * k 2 1 Sum(colsum^2) = X i n 2 1 Sum(rowsum^2) = X j k SST = Total sum of squares 533580180 Page 3 of 10 3/8/2016 SSTr = Treatment sum of squares SSBl = Block sum of squares SSE = Error sum of squares DFTr = Treatment degrees of freedom = (k-1) DFBl = Block degrees of freedom = (n-1) DFE = Error degrees of freedom = (k-1)*(n-1) MSTr = Treatment mean square = SSTr/DFTr MSE = Error mean square = SSE/DFE F = F stat = MSTr/MSE Fcrit is from alpha=.05 F stat table with DFnum=DFTr, DFden=DFE Or, rather than doing all this by hand, in Excel-speak, a repeated measures study can be analyzed with the “ANOVA: Two-factor without replication” tool. This is much simpler than the above and also gives exact values for F crit for any DF and gives prob values. To use Excel: Create data table with one row for each block and one column for each treatment. Label rows and columns. See bottom of file repeatedmeasuresexample.xls Tools > Data Analysis… > ANOVA: Two-factor without replication (If Data Analysis is not available, Tools > Add Ins… > Analysis ToolPak) Click in Input Range, then highlight data, including the header rows. Check Labels Click in Output Range, then in an empty cell to mark the top left of the results box. Click OK Because the treatments are the columns, the F value of interest is in the row marked “Columns” The P value is the probability that the treatments are equal. The Fcrit value is the 0.05 confidence level Above the ANOVA, the Average column gives the mean for each treatment. Interpret results: If F > Fcrit, treatment means are significantly different and can reject null hypothesis with 95% confidence. 533580180 Page 4 of 10 3/8/2016 If F > Crit F, treatment means are significant for P < 0.05 If treatment means are different, perform a multiple comparison test to determine which of the treatments are different. Compute Scheffe comparison ([4] p 405) Scheffe min sig difference = (k 1) Fcrit 2 * MS E n k = number of treatments Fcrit is from the Excel result or from a table MSE is the MS value for the Error row n = number of blocks = number of samples in each group. For two treatments to be different, their means must differ by more than Scheffe. Scheffe will avoid Type I errors, but has little power; it is a conservative measure. ([4] p. 417). For good balance between power and Type I error, try Tukey’s HSD ([4] p. 401) T HSD min sig diff = q MS E n MSE = mean square error n = number of subjects in each group = number of blocks q is from Table A.6 in [4] (p. 622) In the table r = number of treatments, DFe is error dof To present comparisons, arrange means in ascending order and underline those that are not different ([4] p. 408). Paired T-Test Tutorial The paired t-test is run with the Excel “t-test: Paired Two Sample for Means” Data Analysis tool. Line up data in two side-by-side columns. The null hypothesis is that the two treatments are equal. The results will appear in the designated output range. There is a significant difference between treatments if the absolute value of “t Stat” is greater than “t Critical two-tail”, or if “P(T<=t) two-tail” is less than 0.05. The paired two tailed t-test is the same as a repeated measures ANOVA, so instead of a paired t-test, can run the “ANOVA: Two-factor without replication” Data Analysis tool. 533580180 Page 5 of 10 3/8/2016 The two treatments are different if for the row labeled Columns, F > F crit. The P-value is the same as the “P(T<=t) two-tail” in the paired t-test. Tutorial on Power analysis for 2-tailed T-test Following Appendix C in [1]. Also see [5] pp. 158-180, particularly Fig. 6-6 on p. 164. You want to analyze the power of the study to detect a difference. What you would like to draw is a "power function" which is a plot of the power (00 to 100%) of a t test to detect a difference on the vertical axis against the difference on the horizontal axis (in the units of interest, e.g. Newton-meters for muscle torque). Assume that comparisons are via 2-tailed T-test and that acceptance level is alpha=0.5. Go to table C.2 Pick off the line for N, the number of subjects in each group for the study (22/2 = 11?). If different number of subjects in each group, compute N via Equation C.7. For the sake of discussion, assume N=11. Create an Excel table for this row with one column of D values and one column of Pwr values. Plot (see enclosed) For example, this power function plot says that for N=11, if the actual effect size index of the test was D=0.5, the power to detect the difference was 31%. Or, to detect a difference with power of 80%, the effect size index would have to be somewhere between 1.0 and 1.2. "Effect size index" D is not a particularly useful number, so translate the D column into a column of effect sizes in the units you care about. Equation C.1 shows how to do this. For this you need to know s, the common std deviation. Calculate s using Eq. C.2 where s1 and s2 are the std deviations for each population (Eq 17.5 and 17.6). The new column of numbers is d*s and is the "effect size" which is the difference in means. Create a new plot which is exactly what you want, a plot of the power of the test to pick out differences in the means. For unpaired t-test, d = X1 X 2 s d = effect size index Xbar1- Xbar2 = diff in group means, or the difference you want to detect 533580180 Page 6 of 10 3/8/2016 s = pooled std deviation = sqrt(variance) if variances not equal, s = s12 s 22 2 For paired t-test, must adjust the effect size index d. See [4] page 653. gPower cannot be used to automatically calculate effect size for paired t-tests d d 2 sd dbar = mean of difference scores, or the desired difference you want to detect sd = std dev of difference scores This formula is easy because you just create a column of data differences and compute the SD. Another formula (Eq. C.4) is d d 1 r where r = correlation coefficient for paired data and d X1 X 2 s Power calculations for ANOVA See [4] p. 655 Effect size index = f Eq. C.9 shows how to calculate f Use pooled standard deviation (=SD=sigma) for s K = number of treatments (groups) Hopefully there are = sample sizes For post hoc, Sm=diff to detect =difference from grand mean that would be useful to detect. So if you don’t care about any particular treatment, this is really the same as eq. C.1. Calculations 533580180 Page 7 of 10 3/8/2016 Use gPower, a freeware program for power analysis instead of looking up numbers in the table. See next section. Tutorial on gPower gPower is a freeware program for power analysis. See References for where to download. Read the on-line tutorial, after you read Appendix C in [1] and the tutorial above. For post-hoc analysis, use to determine the power of the test to detect a particular difference given the known deviations and known subject size. Choose Tests > Means for t-test power analysis. Post hoc, Two-tailed. Calculate Effect size d from above tutorial Alpha=.05 Sample size: n1, n2 are number in each group. Push Calculate. Looking for 80% or better power, the higher the power the better. Choose Tests > F-test (ANOVA) for anova power analysis Calculate effect size f from above tutorial. If you care about specific means to detect, use the Calculate Effect size tool in gPower or crunch eq. C.10 Alpha=.05 Groups=number of treatments Total samples=treatments*number of samples per group=total number of data points. For a more powerful Power Analysis program, try PASS from NCSS (http://www.ncss.com/pass.html) Expensive, but covers all designs. Prof. Richard DiFabio in PT has a copy. 533580180 Page 8 of 10 3/8/2016 Appendix Blocked Design Tutorial using methods in [1] Refs: [4] 374-377, 387-390 [3] 126-133 [2] 660-667 [1] 34-35 For repeated measures design [4] (or blocked analysis [1]). Uses calculation method of [1] which is simple to understand, with language of [4] which is more standard. See Table 20.4 in [4] and Tables 2-3 and 2-4 in [1] for example. Note: do not using this method, but using method described in tutorial in the main body above 5. Create data table with one row for each treatment and one column for each block where blocks can be subjects or anything else where variation is expected. 6. For mean use AVERAGE(), for variance use VAR(). 7. Create row of Means to compute within block means. 8. Create column of Means to compute within treatment means. 9. Create column of Variance to compute within treatment variances 10. Compute grand mean (= mean of column of means) 11. Create row of Diff = within block mean – grand mean 12. Create new table where each data entry is data – Diff 13. Compute within treatment means and within treatment variances F statistic is F MS t / MS e where MSt is mean square of treatment and MSe is mean square of error. MS = SS/df where SS = sum of squares and df = degrees of freedom. Formulas for SS are in Table 20.4, but [1] provides a simpler computation. MSe is also the variance of blocks pooled. Sqrt(MSe) is the pooled standard deviation ([1] p. 30). 14. Compute from the new table: 533580180 Page 9 of 10 3/8/2016 k number of treatments n number of blocks DFnum k 1 dof num DFden (k 1) * (n 1) dof denom MS t n * var( within treatment means) MS e ( within treatment means) /( k 1) F MS t / MS e Crit F from 5% F table ( Appendix 1 3 in [1], Table A.3( I ) in [4] If F > Crit F, treatment means are significant for P < 0.05 11. Compute Fisher Least Significance Difference See example at bottom of blocktestdata.xls. Note: made file blocktestdata.xls that follows Table 2-3 example in [1] to verify methods. For Table 2-3 without blocking, DF num = number of treatments minus 1 = 6-1=5. DF den is total number of observations (6*4=24) minus DF used to compute means = 6, DF=24-6=18. See [1] p30 For Table 2-4 in [1], DFnum = 6-1 = 5 (number of treatments minus 1). DFden=(n-1)*(k1) (number of treatments minus 1 times number of blocks (subjects) minus 1) = 5*3=15. See [4] p. 390. For Table 2-4 in [1], the mean of the within dot variance is sum(var)/(k-1) = sum(var)/5. In Table 2-3, it is calculated as sum(var)/k. This distinction is not clear from the text. For blocktestdata.xls, used equation on p. 34 in [1] to compute F, and Tables in back of [1] to find F crit.. At bottom of blocktestdata.xls, used data from [4] Table 20.4, p. 390 to test if their method is same as equations for [1]. Note row/column reversal from book table. Get same answer for F. 533580180 Page 10 of 10 3/8/2016

Download
# StatsTutorial