Mixed-methods data analysis
Graduate Seminar in English Language Studies
Suranaree, March 2011
Richard Watson Todd
KMUTT
http://arts.kmutt.ac.th/crs/research/mmda.ppt
Overview



Pure quantitative research
Pure qualitative research
Mixed-methods research


Collecting both QUANT and QUAL data using
different instruments
Mixed-methods data analysis


Usually only QUAL data collected
Data is treated both quantitatively and qualitatively
Quantitative or qualitative?


QUANT – QUAL distinction in applied
linguistics research
QUANT: data is numbers; uses statistics


Experimental research; surveys
QUAL: data is words; uses thematic or
narrative interpretation

Conversation analysis; ethnography
Mixed-methods research


“A mixed methods study involves the
collection or analysis of both quantitative and
qualitative data in a single study with some
attempts to integrate the two approaches at
one or more stages of the research process”
(Dörnyei, 2007)
Purposes:


Achieve a fuller understanding
Triangulate findings
Examples of mixed-methods research

Poor example



Research into attitudes: survey a large number
and interview a predetermined small number of
subjects
Purpose: unclear
Similar, slightly better example


Research into attitudes: survey a large number of
subjects, then, selecting based on questionnaire
responses, interview a small number
Purpose: follow-up on interesting results
Examples of mixed-methods research

An example of the opposite



Interview a small number to gain insights to
design a questionnaire, then survey a large
number
Purpose: informing instrument design
Another similar example


Research into beliefs: interview 4 teachers but
survey 80 students
Purpose: accounting for practicality in using
instruments
Examples of mixed-methods research

An example focusing on triangulation





Research into strategies: comparing results from
different instruments
Much strategy research involves the use of SILL
SILL asks respondents to identify how often they
use a particular strategy
Strategy use is context-dependent
Research question: Will recent context of learning
change responses to SILL?
Examples of mixed-methods research

Method





Single subject
Time 1: read academic articles
Time 2: read short stories for pleasure
Responded to SILL twice
Interviewed 4 times (background interview, after
SILL responses, summary interview)
Examples of mixed-methods research

SILL responses


Interview responses


Showed major differences between 2 times e.g. “If I guess
the meaning of a word, later I will check whether my guess
is correct by using a dictionary.” rated Always at Time 1;
Never at Time 2
Showed that recent learning contexts influenced different
ratings
Triangulation to confirm results or triangulation to
provide different perspectives
Mixed-methods data analysis

“The most common perception of mixed methods
research is that it is a modular process in which
qualitative and quantitative components are carried
out either concurrently or sequentially. Although this
perception is by and large true, it also suggests that
the analysis of the data should proceed
independently for the QUANT and QUAL phases
and mixing should occur only at the final
interpretation stage. This conclusion is only partially
true … we can also start integrating the data at the
analysis stage, resulting in what can be called mixed
methods data analysis”

Dörnyei (2007)
Mixed-methods data analysis (MMDA)

From Dörnyei, MMDA means
 Quantitising qualitative data
 Qualitising quantitative data
Quantitising qualitative data

Quantitising is often done unconsciously





Conducting a keyword analysis
Use of IELTS scores in research
Quantitising helps a qualitative analysis by
allowing a reliability check
Quantitising can be used to count and
compare frequency of themes
Quantitising allows further statistical analysis
of data, but information is always lost when
converting QUAL to QUANT
Qualitising quantitative data


Not common
Narrative profile formation

Using quantitatively obtained questionnaire data
in a qualitative description of a subject
More complex MMDA

Nature of QUANT data




Concise
Allows further analysis (inferential statistics)
Provides summary information
Nature of QUAL data



Detailed and informative
Allows insight into cases
Provides in-depth information
More complex MMDA

What purposes can mixing QUANT and
QUAL data analysis serve?




Illustration for insight
Concise summary to give overview
Preliminary overview to inform analysis
Providing a more well-rounded and more
persuasive analysis
MMDA: Illustration for insight



In many QUANT studies, it is easy to get lost
in the numbers and forget what they mean
If the numbers are derived from QUAL data, it
is useful to give a QUAL example to
concretise the QUANT findings
In Case 1, the original data is QUAL; this is
quantitised for analysis; a QUAL example is
given to concretise the data and to show how
the quantitative analyses was applied
MMDA: Summarising for an overview



In some QUAL research (primarily involving
categorisation or thematisation), the lengthy,
detailed data make it difficult to see the
overall pattern
It can be useful to provide a QUANT
summary as an overview
In Case 2, the data is QUAL and analysed in
a QUAL way, but the overall pattern of results
is presented as QUANT
MMDA: Preliminary overview to inform
analysis




In QUAL studies with large amounts of data, it is
difficult for the researcher to ensure that all relevant
issues have been identified
It is also difficult to see underlying patterns that can
be drowned in the sheer quantity of data
It is useful to conduct a preliminary QUANT analysis
to ensure all issues and underlying patterns are
identified
In Case 3, QUAL data is treated qualitatively to find
keywords which then inform a QUAL thematic
analysis
MMDA: Providing a more well-rounded
and more persuasive analysis




In QUAL studies with large amounts of data,
restricting analysis to either QUANT or QUAL
cannot provide a full picture of the data
QUAL provides detailed description of the
data
QUANT provides generalisations of patterns
to the whole data set
In Case 4, QUAL and QUANT analyses are
used together to produce a fuller description
of the data
Uses of MMDA

Use

Pattern

Illustration for insight
Summarise for
overview
Inform analysis
Provide full picture

QUANT → QUAL
QUAL → QUANT






QUANT → QUAL
Mix of QUANT and
QUAL