Research Methods of Applied
Linguistics and Statistics
(3)
Types of research
Constructing Research Designs
Research Types
◼
◼
◼
◼
Data sources:
primary, secondary
Objectives:
descriptive, exploratory, correlational,
explanatory
Time for data collection:
cross-sectional, longitudinal (panel, trend)
Data types:
qualitative, quantitative (survey,
experimental)
◼
◼
Primary: studies based on primary, or original,
data sources, such as classroom observations or
real students, or their test scores, or their
responses to a questionnaire.
Secondary: studies based on secondary sources
such as other researhers’ books and articles,
including library research and literature reviews.
◼
◼
Descriptive: studies involving the collection of
data in order to test hypotheses or to answer
questions concerning the current status of the
subjects of the study.
Exploratory: studies conducted into an issue or
problem where there are few or no earlier
studies to refer to. The focus is on gaining
insights and familiarity for later investigation.
◼
◼
Correlational: The basic question for descriptive
research is - "What are the values of a number of
variables for a given sample of subjects. The basic
research question for correlation research is - What is
the relationship between two or more variables for a
given set of subjects. Notice that we said relationship
between variables and not the effect of one variable on
another variable.
In descriptive research we are just describing our
subjects in terms of one or more variables, while in
correlational research we are looking at the relationship
between the variables.
◼
◼
Explanatory: studies conducted in order to
explain any behaviour in L2 learning and
teaching. It could be done by using
questionnaires, group discussions, interviews, etc.
Cross-sectional: studies in which groups of
participants of different ages are observed and
compared at a given time
◼
Longitudinal: It is research that studies a person
or group over a set period of time, normally to
track the effect of some variable and as such
permits causal pathways to be determined . For
example, when trying to discover the effect of
language aptitude on L2 proficiency, a
researcher may track 50 children over a 30 year
period in order to find a common factor
between those children who are quicker and
effective L2 learners.
Constructing Research Designs
◼
Determining the design
◼
Classification of designs
Research Design 1: Quantitative /
Qualitative
◼
Quantitative research is empirical research involving
analysis of numerical data Its aim is to determine the
relationship between one thing (an independent
variable) and another (a dependent or outcome variable)
in a population. Quantitative research designs are either
descriptive (subjects usually measured once) or
experimental (subjects measured before and after a
treatment). A descriptive study establishes only
associations between variables. An experiment
establishes causality.
◼
Qualitative research is empirical research
involving analysis of data such as words (e.g.,
from interviews), pictures (e.g., video), or
objects (e.g., an artifact). Its aim is building
theory. It is generally inductive in approach, is
based originally on the naturalistic assumption
that reality is mind dependent (i.e., can only be
known as it is interpreted and has "meaning" for
the observer), is usually of single-subject design,
and generally deals with nominal data
Philosophical base
Phenomenology (quali.) Logical positivism (quan.)
Data from personal
involvement
Subjective
Data from experiment
Empathy
Distance
Data in the form of
string
Data in form of number
Objective
Methodology
Synthetic
Analytic
From part to whole
From whole to part
Internal structure
External structure
Process-oriented
Product-oriented
Dynamic reality
hypothesis
Static reality hypothesis
Purpose
Induction
Deduction
Hypothesis generating Hypothesis confirming
Exploratory, descriptive Inferential, confirmatory,
explanatory
Result of pattern or
Formation of theory
hypothesis
Features for both designs
High validity, low
reliability
Preference of
content to form
Low generalizing
capacity
High explanatory
power
Low validity, high
reliability
Preference of form to
content
High generalizing
capacity
Low explanatory
power
Mixed design: Balanced
Mixed design: Unbalanced
Literature review of quantitative and
qualitative studies by Lazaraton (2000)
◼
Current Trends in Research Methodology
and Statistics in Applied Linguistics
Research Design: experimental
research
An experimental study is a study in which the
researcher manipulates one or more
independent variables and measures their
effect(s) on one or more dependent variables
while controlling the effect of extraneous
variables.
Major components of experimental
research
The population
A comparison which involves at least 2 groups
or 2 conditions.
◼ The treatment
A treatment in which one or more independent
variables are manipulated.
◼ The measurement of the treatment
The measurement of dependent variable(s) as
the result of the treatment.
◼
Letters standing for components
◼
◼
◼
◼
X: an experimental treatment (teaching method,
exposure to a material…)
－: no treatment
O: measurement of the effects of the treatment
or a test
R: randomization, or the random assignment of
subjects to groups in order to control for
extraneous variables.
Types of experimental studies
Pre-experimental
One-group posttest
only study:
X O1
One-group pre/post
test study:
O1 X O 2
No control group; no
randomization
Quasi-experimental
True-experimental
Posttest only
nonequivalent groups:
EG: X O1
CG: O2
Pre/post test
nonequivalent groups:
EG: O1 X O2
CG: O3
O4
Posttest only
equivalent groups
EG: R X O1
CG: R O2
Pre/post test
equivalent groups:
EG: R O1 X O2
CG: R O3
O4
a control group; no
randomization
a control group +
randomization
Pre-experimental: single group
design
One-shot design
◼ One group pretest-posttest design
◼ Time-sampling design
◼
Example for one-shot design
An English teacher used the immersion
teaching method for teaching English
communicative use. After 3 weeks, a test is
administered to the class. The class
performs well on the test.
◼XO
◼ It is also called a pilot study design
◼
One group pretest-posttest
design
It is also called repeated measures. It uses
◼
◼
the subjects as their own controls and eliminates
the need for a control group design.
Strength: It is efficient because some extraneous
variables, attrition and loss of subjects controlled.
Weakness: validity might be affected by history,
attrition and maturation.
Time-sampling design
◼
◼
◼
It is also referred to as time-series designs.
O 1 O 2 O n X O4 O 5 O n
O1 X O2 → O3 -- O4 →O5 X O6 →O7 -- O8
Quasi-experimental: posttest
only nonequivalent groups
• EG: X O1
CG:
O2
• O1-O2
• The treatment effect could also be
attributed to a series of extraneous
variables.
Posttest/posttest nonequivalent
groups
EG: O1 X O2
CG: O3
O4
◼ (O2-O1) – (O4-O3)
◼ Without randomization, the confounding effects
cannot be fully controlled.
True-experimental: posttest only
equivalent groups
EG: R X O1
CG: R O2
O1-O2
The design assumes that the 2 groups are
identical on all relevant features, but it is still
sensitive to selection bias, mortality,
Hawthorn effects.
Pre/post test equivalent groups
EG: R O1 X O2
CG: R O3
O4
Factorial Designs
◼
Factorial designs are similar to the true
experimental designs and include all of the
elements found in those designs, such as
randomization, pre- and post-testing, and
treatments. The difference is that the effects of
independent variables and a moderating variable
may be tested at the same time.
Example for a factorial design
◼
◼
In a study to test for the effect of language laboratory
training on pronunciation, it is also decided to
measure for the effect of such training on learners of
different proficiency levels, as well as the effect of
different kinds of language laboratory practice.
Types of practice (contextualized / decontextualized
drills) , proficiency levels of the learner (beginners /
advanced learners )
Exemplary Figure for Factorial
Design
Experimental study by Robb, et al.
(1986)
◼
Salience of feedback on error and its effect on EFL
writing quality
◼
Types of feedback: complete correction, coded
feedback; yellow highlighting of errors, listing the
number of errors
The compositions were graded on 19 features which
were collapsed into 7 categories.
◼