11 Research
Methods
Prof. TIAN Bing
Shaanxi Normal
University
I. overvie w & history

An Introduction
IV. Methods and Testing

Traditional Thoughts of Education

Research M ethods

Foreign Language Education

Language Testing.

(Pedagogical) Lexicography
V. Learning
II. Lg Description

Language Descriptions

Language Corpora.

Stylistics.

Discourse Analysis. vs CA

Second Language Learning.

Individual Differences in Second
Language Learning.

Social Influences on Language
Learning.
VI. Teaching
III. Cognitive & Social

Fashions in Language Teaching

Language Acquisition: L1 vs L2

Language, Thought, and Culture.

Language and Gender.

Language and Politics.

Language Teacher Education.

World Englishes.

The Practice of LSP

Bilingual Education.
M ethodology.

Computer Assisted Language
Learning
Fig. 0 A Bird’s-Eye-Vie w of Applied Linguistic Studies
1. Research Methods for Applied Linguistics:
Scope, Characteristics, and Standards




1 The Scope of Applied Linguistics Research
2 Characteristics of Applied Linguistics Research
3 Standards for Sound Applied Linguistics
Research
4 Ethical considerations
1.1 Defining applied linguistics
research

1 Definitions that listed the types of research (e.g., “An investigation of

a particular topic, or problem, through a document search and/or
empirical study (the conducting of experiments) and analysis” and
“Investigation through the reading of literature, experimentation
and/or any other type of data gathering . . .”)
2 Definitions that listed the topics of research (e.g., “In its widest
sense, to seek new ways to improve language education and
intercultural communication training” and “Searching for information
on how students process information, internalize data and retain it
for communicative purposes.”)


3 Definitions that covered the purpose of research (e.g., “The search
for information that will help practitioners (in this case, teachers) better
carry out their jobs . . .” and “Systematic study of language issues
and use in order to improve delivery of services to our students.”)
4 Definitions that enumerated the steps in the process of research
(e.g., “Working toward truth, proving theories, trying out new
approaches – and then compiling results, analyzing results and
sharing with colleagues” and “Stating a hypothesis; gathering data;
testing the hypothesis; relating the conclusions to issues at hand.”)

any systematic and principled inquiry in
applied linguistics.
2 Characteristics of Applied
Linguistics Research






Applied linguistics research can be described from many different
perspectives including at least
(1) the contextual factors involved in applied linguistics research,
(2) van Lier’s parameters of educational research design,
(3) Grotjahn’s data collection methods, data types, and data
analysis procedures,
(4) other sets of research characteristics, and
(5) the qual–quant continuum.
2.1 Contextual factors in applied
linguistics research


1 International and national contexts. International and national
organizations and governmental bodies support a fair amount of
applied linguistics research. Since they control the money, their
political priorities tend to influence who will do such research and
how.
2 Professional contexts. Within applied linguistics, the types of
research that are popular at any given time vary; for a few years,
interest in one type of research may increase at the expense of
another, then interest may be rekindled for that latter type. In other
words, even research can have its trends and fads.


3 Institutional contexts. Institutional contexts in applied linguistics
research can refer to everything from entire school districts, to
individual language programs, or even to very specific individual
tutoring situations. Within these institutional contexts a number of
factors can influence the type and quality of research: the size of
the institution, availability of resources to support research,
institutional policies and priorities, the institution’s past experiences
with researchers, and even the personalities of the various
administrators and teachers involved.
4 Local contexts. Local contexts refer to the specific
circumstances in which the research will take place. The context
may be a classroom, laboratory, private home, Internet bulletin
board, or even a coffee shop. A number of factors in the local context
may turn out to be important to the success or failure of a research
study:





(a) physical context (e.g., class size, layout of the school, etc.),
(b) time context (e.g., minutes per class, classes per day, etc.),
(c) social context (e.g., language backgrounds, ethnic mix of the
students, etc.),
(d) pedagogical context (e.g., teaching methods used, preferred
learning styles of the students and teachers, etc.),
(e) psychological context (e.g., comfort level of participants with
regard to research studies, etc.).

5 Personal contexts. Individual researchers have certain
preconceptions about the aspects of applied linguistics that ought to be
researched, the form that research should take, and their role in the
research process. Such preconceptions and preferences arise from
individual differences in abilities, personalities, motivations, priorities,
training, etc. and may influence the types of research a particular
individual or group will be interested in doing.
2.2 Van Lier’s parameters of educational
research design


As shown in Figure 19.4 (from van Lier, 1988) research can also be
described in terms of an intervention axis (i.e., degree of
intervention, from intervention to non-intervention) and a selectivity
axis (i.e., selectivity of focus, from highly selective to non-selective).
For example, on the intervention axis, research can take the form
of a formal experimental design with a randomly assigned treatment
and control groups, which would be an intervention study, or it can take
the form of a series of informal classroom observations, which
would be more in the direction of a non-intervention study. On the
selectivity axis, research can be highly selective in focus (e.g.,
obligatory use of the definite article by immigrant Chinese
adolescent men from Guangdong province), or non-selective (e.g., all
language related behaviors observed of all participants in a population
of students).





The selectivity and intervention axes also create what van Lier
calls four territories:
1 controlling, in which the researcher conducts a carefully planned
experiment restricted both in participants and content focus;
2 measuring, in which intervention is minimal but the data focus
is highly restricted;
3 asking/doing, in which the researcher might intervene to ask
participants to talk about what they are thinking;
4 watching, in which the researcher just observes with both
intervention and selectivity kept to a minimum.
2.3 Grotjahn’s data collection methods,
data types, and data analysis procedures

According to Grotjahn (1987), research can be classified in terms
of data collection methods (i.e., experimental vs. nonexperimental), data types (i.e., qualitative vs. quantitative), and
data analysis procedures (statistical vs. interpretive).
2.4 Other sets of research
characteristics

2.4.1 Time orientation

Another way to classify research is according to time orientation, or the
amount of time invested in gathering data. Along those lines,
studies are sometimes classified as either cross-sectional or
longitudinal (e.g., see Larsen-Freeman & Long, 1991).
Cross-sectional studies are those conducted over a short period of
time, often with a relatively large number of participants. For instance,
a study might gather language proficiency data (using a test),
motivation data (using a questionnaire), and personal information
data (on the same questionnaire) from 300 students in a one-shot
cross-sectional study.


In contrast, longitudinal studies are typically carried out over a
relatively long period of time, often with a small number of participants.
For example, a study might dedicate five years to following five
students of varying backgrounds, making careful observations of
their language proficiency growth, their motivation to learn
languages, their personal characteristics, etc.

Other ways of describing the time orientation have recently
surfaced like prolonged engagement (observations and
involvement with a group of people over a long period of time),
persistent observations (frequent observations over that long period
of time), and the cyclical nature (data collection, analysis,
interpretation, followed by further data collection, analysis,
interpretation, etc.) of longitudinal studies in the exploratoryinterpretative tradition (see especially, Davis, 1995, pp. 444–5).
2.4.2 Theory generation



Studies can also differ in terms of theory generation, which can
take two forms: hypothesis forming and hypothesis testing.
Hypothesis forming research may begin with some very general
framing questions, but will typically have no hypotheses to start
with. In such a study, the researcher will make every effort to
keep an open mind and form hypotheses about what is going on
only after a great many observations of various sorts.
The resulting hypotheses are typically considered part of the
interpretation, and the researcher often goes back to the
participants to ask them if the hypotheses are reasonable (in a
process called member checking). One of the great strengths often
cited for qualitative research is its potential for forming new
hypotheses.


In contrast, hypothesis testing research begins with a set of
research questions and hypotheses. Sometimes the hypotheses
are stated; more often they are implicit in the research questions.
The statistical analyses in such studies are designed to formally
test the probability that the hypotheses are true and typically
include some form of probability statement, like p < 0.01, which
indicates that there is less than a one percent probability that the
observed difference (or relationship) is a chance fluctuation.
One of the strengths often cited for quantitative research is its
potential for hypothesis testing.
2.4.3 Variable description



Similarly, variable description can take two forms in applied linguistics
research: variable definition and variable operationalization.
In variable definition research, the researcher attempts to begin with
no preconceived notions of what the important variables in the
study will be, or how they will be defined.
As the study progresses, the process of discovering and describing
variables serves to gradually define them.


In variable operationalization research, the researcher clearly
outlines the variables of interest from the outset (particularly the
dependent, independent, and moderator variables) and explains how
each one was operationalized, that is, how each one was observed or
measured and quantified.
For instance the variable Japanese language proficiency might be
operationalized a scores on a particular Japanese proficiency test,
or the variable nationality might be operationalized as 1 for
Chinese, 2 for Japanese, and 3 for Korean based on asking each
of the participants what passport they hold, etc.
2.4.4 Researcher perspective


The researcher’s perspective is sometimes described as either
emic or etic. Researchers adopting the emic perspective make every
effort to understand the point of view of the participants and to examine
how the interpretations drawn from the research relate to those
views through practices such as member checking (getting the
participants’ reactions to the interpretations drawn from the research).
Researchers assuming the etic perspective take an outsider’s
view during the data gathering process, often attempting to be as
objective as possible (for more on the emic/etic distinction, see Davis,
1995, p. 433).
2.5 The qual–quant continuum



2.5.1 The qualitative versus quantitative dichotomy
Reichardt and Cook (1979, p. 10) summarize the differences between
qualitative and quantitative research as shown in Table 19.2. Notice
that the column on the left in Table 19.2 is labeled “Qualitative
paradigm” and that the column to the on the right is labeled
“Quantitative paradigm,” with each column containing adjectives
and adjectival phrases describing each “paradigm.”
A number of these distinctions are uncontroversial and make eminent
sense; for example, the fact that the qualitative paradigm advocates
qualitative methods and the quantitative paradigm advocates
quantitative methods and the fact that the qualitative paradigm is
typically naturalistic where the quantitative paradigm would more
accurately be characterized as controlled.



However, I disagree with other distinctions, especially those that begin
with un-. “Uncontrolled observation” and “ungeneralizable” both
seem to me to be unfair characterizations of qualitative research.
The observations in qualitative research are often well planned and
structured in their own ways, as in a well-designed interview
schedule, a classroom observation checklist, or a carefully planned
discourse coding scheme.
“Ungrounded” appears to me to be an equally unfair characterization of
quantitative research because such research is sometimes quite
exploratory. I would also argue that the use of the terms subjective
and objective has become outdated partly because the two terms have
become highly loaded over the years (Porter, 1998) and partly
because those loaded meanings do not accurately characterize the
two types of research.
2.5.2 General problems with the qualitative
versus quantitative dichotomy




Such a qualitative versus quantitative approach also has a number of
general problems:
1 Dichotomizing qualitative versus quantitative research leaves out
altogether secondary research types like literature reviews.
2 It treats as monolithic at least seven very distinct qualitative
research techniques (case study research; introspection research;
discourse analysis research; interactional analysis research;
classroom observation research; interviews; and questionnaires).
3 It represents as monolithic at least ten qualitative research
traditions that come from a variety of other fields like
anthropology and theology (see Table 19.3, adapted slightly from
Lazaraton, 1995, p. 460).




4 It presents as monolithic at least six very different quantitative
research techniques (interviews; questionnaires; descriptive;
exploratory; quasiexperimental; and experimental).
5 It ignores the way survey research, including interviews and
questionnaires, is both qualitative and quantitative.
6 It ignores the ways researchers often combine qualitative and
quantitative research techniques (as shown above in the
Grotjahn’s analysis in Table 19.1).
7 It confuses research methods (interpretive, survey, and
statistical) and research techniques (like those listed in the second
and fourth points above).
2.5.3 The qual–quant interactive continuum

Perhaps a more constructive and accurate approach would be to view
qualitative and quantitative research as a matter of degrees, a
continuum, rather than a clear-cut dichotomy like the one shown in
Table 19.2. As Newman and Benz (1998) put it, “All behavioral
research is made up of a combination of qualitative and quantitative
constructs” (p. 9). They advocate the notion of a qual–quant research
continuum, as opposed to a dichotomy, and insist that the continuum
be considered “interactive.”

Figure 19.5 shows how 12 research characteristics can be
combined to create an almost infinite number of possible
interrelationships and thus to describe a wide variety of different
research types.