Setting the scene: what is ANOVA? ANOVA is an acronym for Analysis of Variance. It covers a series of tests that explores differences between groups or across conditions. The type of ANOVA employed depends on a number of factors: how many independent variables there are; whether those independent variables are explored between- or within-groups; and the number of dependent variables being examined ANOVA explores the amount of variance that can be ‘explained’. . For example, we could give a questionnaire to a group of people. Responses to the questions can be allocated scores according to how they have been answered. Because people are different, it is quite likely that the response scores will differ between them. To examine that, we find the overall (grand) mean score and see how much the scores vary either side of that. The amount that the scores vary is called the variance. When using ANOVA tests, we can partition that variance into separate pots. We call those pots the ‘sum of squares’ The overall variance is found in the ‘total sum of squares’: this illustrates how much the scores have varied overall to that grand mean. Sometimes the scores will vary a lot, other times they will vary a little, and occasionally the scores will not vary at all. When the scores vary, we need to investigate the cause of the variation. The scores may have varied because of some fundamental differences between the people answering the questions. For example, income may vary across a sample of people according to the level of education. If those differences account for most of the variation in the scores, we could say that we have ‘explained’ the variance. At the other extreme, the scores may have varied simply due to random or chance factors: this is the variance that we cannot explain. ANOVA tests seek to explore how much variance can be explained – this is found in the ‘model sum of squares’. Any variance that cannot be explained is found in the ‘residual sum of squares’ (or error). We express the sum of squares in relation to the relevant degrees of freedom (we will see how later). This produces the model mean square and residual mean square. If we divide the model mean square by the residual mean square, we are left with something called an ‘F ratio’– this illustrates the proportion of the overall variance that has been explained (in relation to the unexplained variance). The higher the F ratio, the more likely it will be that there is a statistically significant difference in mean scores between groups (or conditions for within-group ANOVAs). What is independent one-way ANOVA? An independent one-way ANOVA explores differences in mean scores from a single (para- metric) dependent variable (usually) across three or more distinct groups of a categorical independent variable (the test can be used with two groups, but such analyses are usually undertaken with an independent t-test). We explored the requirements for determining whether data are parametric in Chapter 5, but we will revisit this a little later in this chapter. As we saw just now, the variance in dependent variable scores is examined according to how much of that can be explained (by the groups) in relation to how much cannot be explained (the error variance). The main outcome is illustrated by the ‘omnibus’ (overall) ANOVA test, but this will indicate only whether there is a difference in mean scores across the groups. \ Research question for independent one-way ANOVA We can illustrate independent one-way ANOVA by way of a research question. A group of higher education researchers, FUSS (Fellowship of University Student Surveys), would like to know whether contact time varies between university courses. To explore this they collect data from several universities and investigate how many hours are spent in lectures, according to three courses (law, psychology and media). FUSS expect that there is a difference but they have not predicted which group will spend more time attending lectures. In this example, the explained variance will be illustrated by how mean lecture hours vary across the student groups, in relation to the grand mean. For the record, all outcomes in this chapter are based on entirely fictitious data! Theory and rationale Identifying differences Essentially, all of the ANOVA methods have the same method in common: they assess explained (systematic) variance in relation to unexplained (unsystematic) variance (or ‘error’). With an independent one-way ANOVA, the explained variance is calculated from the group means in comparison to the grand mean (the overall average dependent variable score from the entire sample, regardless of group). The unexplained (error) variance is similar to the standard error that we encountered in Chapter 4. In our example, if there is a significant difference in the number of hours spent in lectures between the university groups, the explained variance must be sufficiently larger than the unexplained variance. We examine this by partitioning the variance into the model sum of squares and residual sum of squares. We can see how this is calculated in Box 9.3. It is strongly recommended that you try to work through this manual example as it will help you understand how variance is partitioned and how this relates that to the statistical outcome in SPSS. Post hoc tests If no specific prediction has been made about differences between the groups, post hoc tests must be used to determine the source of difference. We can also choose to use post hoc tests in preference to non-orthogonal planned contrasts. However, there must be a significant ANOVA outcome in order for post hoc tests to be employed. If we try to run these tests on a nonsignificant ANOVA outcome it might be regarded as ‘fishing’. Also, we run post hoc tests only if there are three or more groups. If there are two groups we can use the mean scores to indicate the source of difference. Post hoc tests explore each pair of groups to assess whether there is a significant difference between them (such as Group 1 vs. 2, Group 2 vs. 3 and Group 1 vs. 3). Most post hoc tests account for multiple comparisons automatically (so long as the appropriate type of test has been selected – see later). How SPSS performs independent one-way ANOVA We can perform an independent one-way ANOVA in SPSS. To illustrate, we will use the same data that we examined manually earlier. We are investigating whether there is a significant difference in the number of lecture hours attended according to student groups (law, psychology and media). There are ten students in each group. This means that we will not need to consult Brown–Forsythe F or Welch's F outcomes (see assumptions and restrictions). However, because we do not know whether we have homogeneity of variance between the groups, we still need to potentially account for that in choosing the correct post hoc test (so we should request the outcome for Games–Howell in addition to Tukey – see Box 9.8). We can be confident that a count of lecture hours attendance represents interval data, so that the parametric criterion is OK. However, we do not know whether the data are normally distributed; we will look at that shortly. Testing for normal distribution To examine normal distribution, we start by running the Kolmogorov–Smirnov/Shapiro– Wilk tests (we saw how to do this for between-group studies in Chapter 7). If the outcome indicates that the data may not be normally distributed, we could additionally run z-score analyses of skew and kurtosis, or look to ‘transform’ the scores (see Chapter 3). We will not repeat the instruction for performing Kolmogorov–Smirnov/Shapiro–Wilk tests in SPSS (but refer to Chapter 3 for guidance). However, we should look at the outcome from those analyses (see Figure 9.3). Because we have a sample size of ten (for each group), we should refer to the Shapiro– Wilk outcome. Figure 9.3 indicates that lecture hours appear to be normally distributed for all courses: Law, W (10) = .976, p = .943); Psychology, W (10) = .893, p = .184); and Media, W (10) = .976, p = .937). You will recall that KS and SW tests investigate whether the data are significantly different to a normal distribution, so we need the significance (‘Sig.’) to be greater than .05. Running independent one-way ANOVA in SPSS Now we can perform the main analysis: We explored the various post hoc options earlier. We know that we have equal group sizes (ten in each group), so we are pretty safe in selecting Tukey. However, we do not yet know whether we have equality of variances, so we should also select Games–Howell just in case. Interpretation of output Figure 9.8 shows the lecture hours appear to be higher for psychology than for the other two courses, and appear to be higher for law than media. However, we cannot make any inferences about that until we have assessed whether these differences are significant. We need to check homogeneity of (between-group) variances. Figure 9.9 indicates that significance is greater than .05. Since the Levene statistic assesses whether variances are significantly different between the groups we can be confident that the variances are equal. Figure 9.10 indicates that there is a significant difference in mean lecture hours across the courses. We report this as follows: F (2, 27) = 12.687, p 6 .001. The between–groups line in SPSS is equivalent to the model sum of squares we calculated in Box 9.3. The within–groups line equates to the residual sum of squares (or error). There are conventions for reporting statistical information in published reports, such as those suggested by the American Psychological Association (APA). The British Psychological Society also dictates that we should adhere to those conventions. When reporting ANOVA outcomes, we state the degrees of freedom (df), which we present (in brackets) immediately after ‘F’. The first figure is the between groups (numerator) df; the second figure is the within groups (denominator) df. The ‘Sig’ column represents the significance (p). It is generally accepted that we should report the actual p value (e.g. p = .002, rather than p 6 .05). When a number cannot be greater than 1 (as is the case with probability values) we drop the ‘leading’ 0 (so we write ‘p = 002’, rather than ‘p = 0.002’). The only exception is when the significance is so small that SPSS reports it as .000. In this case we report the outcome as p 6 .001. We cannot say that p = 0, because it almost certainly is not (when we explored the ANOVA outcome manually earlier, we saw that p = .0001; this is less than .001, but it is not 0). Welch/Brown–Forsythe adjustments to ANOVA outcome If we find that homogeneity of variance has been violated, we should refer to the Welch/Brown– Forsythe statistics to make sure that the outcome has not been compromised. The tests make adjustment to the degrees of freedom (df) and/or F ratio outcomes (shown under ‘Statistic’– Figure 9.11). We do not need to consult this outcome, as we did satisfy the assumption of homogeneity of variance. However, it is useful that you can see what the output would look like if you did need it. Post hoc outcome As we stated earlier, if we have three or more groups (as we do), the ANOVA outcome tells us only that we have a difference; it does not indicate where the differences are. We need to refer to post hoc tests to help us with that. Figure 9.12 shows two post hoc tests: one for Tukey and one for Games–Howell. We selected both because, at the time, we were not certain whether we had homogeneity of variances. As we do, we can refer to the Tukey outcome. The first column confirms the post hoc test name. The second column is split into three blocks: Law, Psychology and Media (our groups). The third column shows how each group compares to the two remaining groups (in respect of lecture hours). For example, Law vs. Psychology (row 1) and Law vs. Media (row 2). The fourth column shows the difference in the mean lecture hours between that pair of course groups. To assess the source of difference we need to find between-pair differences that are significant (as indicated by cases where ‘Sig.’ is less than .05). Presenting data graphically We can also draw a graph for this result. However, never just repeat data in a graph that has already been shown in tables. The illustration is shown in Figure 9.16, in case you ever need to run the graphs. We can use SPSS to ‘drag and drop’ variables into a graphic display with between- group studies (it is not so straightforward for within-group studies).
