Turner, J. Using statistics in small-scale language education research: Focus on non-parametric data
Chapter Three: Practice Problem Key
Practicing the Concepts—Variable Roles
For each of the brief problems that follow, identify the variables and indicate which have been
assigned the role of independent variable and which have been assigned the role of dependent
variable.
1. A researcher plans to investigate whether a teacher’s amount of experience has an impact on
the students’ success in learning a foreign language.
Independent variable(s): teacher’s amount of experience
Dependent variables(s): students’ success in learning
2. Jean (that’s me) plans to do a research problem to investigate whether students who are
enrolled in morning classes have the same degree of success in learning as students who are
enrolled in late afternoon classes.
Independent variable(s): time of class meeting
Dependent variables(s): students’ success in learning
3. I wonder whether learners whose first language is Spanish or French find learning Japanese
more challenging than learners whose first language is Thai or Vietnamese.
Independent variable(s): learner first language
(Though really, the variable is a characteristic of the first language—whether the learner’s
first language is tonal or not (+/- tonality). Spanish and French are not tonal languages; Thai
and Vietnamese are tonal languages. Japanese, by the way, is not a tonal language.)
Dependent variables(s): degree of challenge in learning Japanese
4. Do you think that extroverts are more likely to develop a high degree of oral fluency than
introverts?
Independent variable(s): degree of extroversion or if the variable is nominal,
introversion/extroversion
Dependent variables(s): oral fluency
5. Can you think of any moderator variables that might play a role in Studies 1 – 4?
In the first study, I might include teacher’s type of experience as a moderator variable. In
Study 2, I think I’d include as a moderator variable how learners characterize themselves—so
the variable could be labeled as morning people/night owl. In Study 3, I might include
number of languages spoken as a moderator variable. The sex of the learners might be an
important consideration in Study 4 so I believe I’d include it as a moderator variable.
1
Turner, J. Using statistics in small-scale language education research: Focus on non-parametric data
6. Can you think of any control variables that might play a role in Studies 1 – 4?
For Study 1, I think it would be a good idea to focus on students in just one type of school—I
think I’d first study learners in high school, so the control variable could be labeled type of
school, or student’s level of education. In Study 2, I think I’d introduce student’s program of
study as a control variable, and include as participants only those who are in the applied
linguistics program. For Study 3, I’d only include learners who are between the ages of 20 –
25, so age would be a control variable. In Study 4 I’d have two control variables. This first is
language of study—I’d only include people who are learning Spanish. The second control
variable that I’d introduce is age. As in Study 3, I’d only include participants who are
between the ages of 20 and 25.
7. Write two brief research problems of your own.
No answer key for this one!
8a. What is the independent variable and how many levels does it have?
type of curriculum; it has two levels (new and existing)
8b. What is the dependent variable?
reading ability
8c. What type of scale do you think “measures” the dependent variable?
I think the researcher defined reading ability as an interval scale variable, because it appears
that the variable has mathematical properties. (The researcher calculated the “average” as
part of the analysis of the data).
8d. Is there an explicit control variable?
The researcher doesn’t say that there is a control variable, so there isn’t an explicit one.
However, I believe the researcher had access to students in grades K – 6 and chose to include
only first-graders as participants in this study, so perhaps learner’s grade level is a control
variable.
8e. Is there an explicit moderator variable?
The researcher doesn’t say that she compared the outcomes for the native speakers and nonnative speakers of English assigned to the two different types of curriculum, but I think she
should. If she does, learner’s first language, would be a moderator variable.
8f. Identify two major threats to the internal validity of the study and what might be done to
attempt to reduce each.
Here are a few that might threaten the soundness of the study.
1) teachers’ familiarity with the curriculum
Teachers who participate in the study should be trained to use the curriculum that they
teach—whether it is the new curriculum of the existing one.
2) teachers’ attitude toward the curriculum
What teachers think of the curriculum they teach could have an impact on their teaching—I
think I’d exclude teachers from participating in the study if they reported having a negative
attitude toward the curriculum they use.
2
Turner, J. Using statistics in small-scale language education research: Focus on non-parametric data
8g. Identify two major threats to the external validity of the study and what might be done to
attempt to reduce each.
1) The fact that only first-graders participated in the study threatens the external validity of
the study. The only way this threat could be reduced is by including students from other
grade levels as participants; however, the researcher may not have sufficient resources to do
this.
2) If the researcher didn’t make learner first language a moderator variable, I think one
would have to be very careful about generalizing the findings of the study to both native
speakers of English and non-native speakers of English. To reduce this threat, I recommend
that this factor, learner first language (with two levels, native speaker of English and nonnative speaker of English), be studied as a moderator variable.
8h. Into which of the four categories of research design does this study best fit?
I used the three ordered questions on page 80 in the textbook as a guide.
1) Is there an experimental treatment?
Yes, there is an innovation; the students who use the new curriculum represent an
experimental condition. The students who use the existing curriculum are in the nonexperimental condition.
2) Are there legitimate comparison groups?
If the number of first-graders in the district is sufficiently large to be considered a population,
it appears that the random selection of the 60 participants from the population would result in
their being comparable to the population of first-graders, thus there are legitimate
comparison groups.
3) Are the legitimate comparison groups formed through random selection and random
assignment?
If one believes that the district is sufficiently large to be considered a population, random
selection of the participants indicates that the design of the study is true experimental.
Note: If the school district is so small that the first-graders cannot be considered a population,
the design must be considered pre-experimental.
9a. What is the independent variable and how many levels does it have?
travel to Spanish-speaking country (immersion in Spanish-speaking country); it has two
levels, yes and no.
9b. What is the dependent variable?
Spanish ability (or Spanish proficiency)
9c. What type of scale do you think “measures” the dependent variable?
I think it’s an interval scale variable, though it could be an ordinal scale that has
mathematical properties (see pp. 18-19 in the textbook).
3
Turner, J. Using statistics in small-scale language education research: Focus on non-parametric data
9d. Is there an explicit control variable?
There doesn’t seem to be any explicit control variables, but there are several implicit ones,
including teacher (she only included her students!)
9e. Is there an explicit moderator variable?
I think she treated sex as a moderator variable.
9f. Identify two major threats to the internal validity of the study and what might be done to
attempt to reduce each.
There are a lot of serious threats to the soundness of this study! Here are a few:
1) duration of time the learner spent in a Spanish-speaking country
I think the nominal independent variable, travel to Spanish-speaking country, which has only
two levels, is inadequate. It should have been defined to include several more categories
reflecting length of time spent in a Spanish-speaking country.
2) purpose of travel to a Spanish-speaking country
The researcher doesn’t take into account why the learners travelled to a Spanish-speaking
country or what they did while they were there. The researcher should create a moderator
variable to categorize the participants according to their purpose for the travel.
9g. Identify two major threats to the external validity of the study and what might be done to
attempt to reduce each.
1) a very limited sampling of learners of Spanish
All of the participants in this study were the former students of one teacher. I assume there
were other teachers of Spanish at the school—their students should have been included too!
2) There are so many threats to the soundness of this study that it simply doesn’t have any
external validity! I suggest the teacher create a new research design and start over!
9h. Into which of the four categories of research design does this study best fit?
To answer this question, I used the ordered three questions on page 80 as a guide, but needed
to answer only the first one to identify the research design.
1) Is there an experimental treatment?
No, there is no innovation or change. The teacher simply reviewed the data that she had;
there is no experimental treatment so the design is ex post facto.
4