Lab Exercise 3 (Permutation tests)
The goal is to develop a macro to carry out a permutation test based on the difference
of two means for comparing two samples.
1. From the web page get the pigeons data and paste it into columns 10 and 11. Then
using the unstack columns command in the Manipulate menu, unstack the data into cols
1 and 2.
2. State the research hypothesis, statistical model, and statistical hypotheses.
3. The test statistic will be the difference in means. (This is essentially the two-sample t
statistic.) Compute and save in k1 the difference in means.
PERMUTATION PRICIPLE: If the null hypothesis is true then we can consider the two
samples as coming from the same population (or identical populations).
a. We combine the data (stack it) and randomly permute it.
b. Then separate it into two samples (unstack it).
c. Finally compute the difference in means.
If we repeat this 1000 times and compute 1000 differences in means, then the 1000
differences in means are all equally likely possible values of the difference in means.
Hence, the permutation p-value is simply the proportion of differences out of 1000
that are greater than or equal to the observed difference in means. (This for an
alternative hypothesis that specifies pop mean 1 greater than pop mean 2.) For more
explanation of permutation tests see Chapt 2 of the text by Higgins.
Relevant Minitab commands: stack and unstack (Manipulate menu), sample from
column (Calculate>Random Data menu.
4. Write a gmacro to compute the permutation p-value for the given problem in 1-3
above. You should work it through by randomly generating a difference in means and
store it in row 1 of some column. Then use History to get the command language to
paste into your macro and then make the necessary modifications to generate 1000
differences.
5. Write a report including
i. A copy of your macro
ii. Answers to 2 and 3 above
iii. The permutation p-value and your conclusions.
iv. Now use Minitab to carry out a two-sample t-test and compare the results
with your answer in iii.