What is the difference between an alpha level and a p-value? Science sets a conservative standard to meet for a researcher to claim that s/he has made a discovery of a real phenomenon. The standard is the alpha level, usually set of .05. Assuming that the null hypothesis is true, this means we may reject the null only if the observed data are so unusual that they would have occurred by chance at most 5 % of the time. The smaller the alpha, the more stringent the test (the more unlikely it is to find a statistically significant result). Once the alpha level has been set, a statistic (like r) is computed. Each statistic has an associated probability value called a p-value, or the likelihood of an observed statistic occurring due to chance, given the sampling distribution. Alpha sets the standard for how extreme the data must be before we can reject the null hypothesis. The p-value indicates how extreme the data are. We compare the p-value with the alpha to determine whether the observed data are statistically significantly different from the null hypothesis: If the p-value is less than or equal to the alpha (p< .05), then we reject the null hypothesis, and we say the result is statistically significant. If the p-value is greater than alpha (p > .05), then we fail to reject the null hypothesis, and we say that the result is statistically nonsignificant (n.s.).