EDF 802
Dr. Jeffrey Oescher
Formative Exercise for Topic 4 – Factorial ANOVA
I.
Purpose: The purpose of this formative exercise is to assess your statistical analysis skills relative to a
research question for which the data has already been collected. Generally speaking you will be
evaluated on conducting the appropriate inferential analysis, drawing conclusions related to the
research questions, and describing the inferential logic used in your analyses.
As always, I would like to encourage your discussing this assignment with other class members.
Often these discussions are very helpful in understanding all of the issues in quantitative research
design and analysis. I expect, however, that you will actually write the paper independently of others.
II. Research Questions
A. A researcher was interested in investigating the effect of the time-of-day a test was taken on
students’ test scores. A review of the literature on this topic consistently indicated performance
differences between morning, midday, and late afternoon testing times that were attributed to
physical and psychological factors. It also indicated the effects of testing time were likely to be
different for males and females.
To investigate this problem, 24 students were randomly selected from a population of 9 th grade
students. Each student was randomly assigned to one of three testing-time groups. The first group
was tested in the morning (M), the second around noon (N), and the third in the late afternoon (A).
The subjects were tested with a standardized reading examination. Scores on this exam could
range from 0 to 50.
Your task is to statistically analyze this data for the researcher. Her obvious concern is the
comparison of the scores fro each of the three time periods, but she is also concerned about the
interaction of gender as well as any differences related to it. She is not concerned about either
Type I or Type II errors.
Group 1 (M)
Sex Time Score
F
M
43
F
M
40
F
M
49
M
M
50
M
M
39
M
M
44
M
M
43
M
M
48
Sex
F
F
F
F
F
M
M
M
Group 2 (N)
Time Score
N
36
N
35
N
31
N
23
N
37
N
36
N
41
N
35
Sex
F
F
F
M
M
M
M
M
Group 3 (A)
Time Score
A
40
A
42
A
34
A
36
A
31
A
42
A
36
A
33
III. Performance Criteria: The assessment examines four specific areas: 1) requisite information to an
inferential analysis (e.g., statistical hypothesis, alpha level, etc.), 2) the inferential test results, 3)
conclusions relating the statistical results to the research question, and 4) the inferential logic used in
this process. Each area is assessed according to a set of criteria unique to that area. These criteria are
discussed below.
Each specific criterion will be assessed on a three (3) point scale. The end points of this scale (i.e., a
‘3’ and a ‘1’) are described in narrative in terms of the characteristics underlying responses at these
levels. You have received previously a set of practice exercises in which these criteria are explicated.
-1-
A. Requisite Information: There is information upon which all inferential testing is based. This
section assesses four (4) dimensions of such information. They are 1) the statistical hypothesis, 2)
the alpha level, 3) power and sample size, and 4) the test statistic and sampling distribution.
1.
Statistical Hypotheses
Level 3 Response: The statistical hypotheses are stated explicitly. They accurately reflect the
research question. They express the appropriate relationship between/among the population
parameters and are algebraically correct.
Level 1 Response: The statistical hypotheses are not stated or do not accurately reflect the
research question. If present, they express inappropriate relationships between/among the
population parameters or are algebraically incorrect.
2.
Alpha Level
Level 3 Response: The alpha level is explicitly stated. It reflects a reasonable balance
between the consequences of Type I and Type II errors. A discussion of these concerns is
present.
Level 1 Response: The alpha level is not explicitly stated or does not reflect a reasonable
balance between the consequences of Type I and Type II errors. No discussion of Type I or
Type II errors is present.
3.
Test Statistic and Sampling Distribution
Level 3 Response: The correct statistical test(s) is stated explicitly. The appropriate sampling
distribution (i.e., the distribution and degrees of freedom) is identified.
Level 1 Response: The statistical tests and sampling distribution are not stated or are
incorrect.
B. Results: The results of a inferential test are clearly the most important objective of an analysis.
This section assesses three issues that are related to these results. They are 1) the statistical
information describing the sample and the performance on relevant variables, 2) the inferential
statistical result, and 3) the statistical conclusion.
1.
Statistical Information Describing the Sample and Relevant Variables
Level 3 Response: The sample is accurately described both statistically and narratively. Data
describing the demographic characteristics of the sample and variables relevant to the
research questions is included. Categorical data is clearly and accurately summarized in terms
of frequencies and/or proportions and is presented clearly in narrative format. Interval/ratio
data is summarized in terms of means, standard deviations, and sample size. This data is
presented clearly and accurately in a format that follows appropriate guidelines (see Huck
(2008)) or the APA Manual – Fifth Edition (2002)). An accurate summary of all descriptive
data is presented in textual form.
Level 1 Response: The sample is not described or is described inaccurately. Demographic
data is ignored or inaccurately summarized. Statistical summaries are incorrectly computed or
are presented in inappropriate formats. The narrative summary of all data repeats, rather than
summarizes, the data contained in the discussion and/or tables.
2.
Inferential Test Results
Level 3 Response: The inferential test results are stated explicitly in terms of the test statistic
(e.g., F2,45=13.50, p=.318) and statistical conclusion (i.e., acceptance or rejection of H0). The
-2-
tests of the assumptions underlying the procedure are reported accurately as are the level of
power for the analysis and the effect size. The test statistic is computed correctly and is
reported according to appropriate guidelines (see Huck (2008)) or the APA Manual – Fifth
Edition (2002)).
Level 1 Response: The inferential test results are not stated or are incorrectly stated.
Assumptions underlying the procedure are not addressed or are incorrectly tested. Power is
not reported or is incorrectly calculated. If the results are presented, the presentation does not
follow appropriate guidelines.
3.
Statistical Conclusions
Level 3 Response: The statistical conclusion is stated and follows logically from the test
results.
Level 1 Response: The statistical conclusion is not stated or is not a logical extension of the
test results.
C. Conclusions Relating the Statistical Results to the Research Question: The results of any
inferential analysis should be interpreted relative to the research question. This section assesses the
extent to which this occurs on a single dimension.
1.
Conclusions Relating the Statistical Results to the Research Question
Level 3 Response: The statistical results are discussed in terms of the research question, not
the statistical conclusions. The discussion focuses on the relationship between/among the
independent and dependent variables. The discussion appropriately reflects the statistical
results in this context.
Level 1 Response: Conclusions are not discussed or reflect only a statistical orientation. They
do not discuss the results in the context of the research question driving the analysis.
D. Inferential Logic: Knowledge of the logic of inferential testing can simplify the understanding and
appropriate use of a host of statistical procedures. The section assesses the extent to which you
have conceptualized the procedure analyzed above in terms of this logic. A single dimension is
used to assess your knowledge.
1.
Discussion of Inferential Logic
Level 3 Response: The logic of inferential testing is stated accurately in general form. The
application of this logic to the specific problem being analyzed in this assignment is
appropriately discussed (e.g., statistical hypothesis, specific sampling distribution, observed
value of the test statistic, etc.).
Level 1 Response: The logic of inferential testing is not stated or stated inaccurately. The
application of this logic to the specific problem being analyzed in this assignment is not
discussed or is discussed inappropriately.
-3-