Mapping the Landscape of Mixed
Methods Research
by John W. Creswell, Ph.D.,
Department of Educational Psychology,
University of Nebraska-Lincoln
Founding Co-editor, Journal of Mixed
Methods Research
University of Manitoba, Canada
March 26, 2010
© Please do not duplicate or use these slides without the express permission of the author.
1
Positioning Myself
View research as set of interactive
components; not always linear
 Focus on rigorous data collection
and analysis
 Applied research methodologist
 Often work with individuals in a
step-by-step process

First generation mixed methods
researcher

I am still learning how to
discuss/think about this topic

How Do We Understand the Social World
and Social Life?

Quantitative
methods
 Trends, graphs,
statistics
 Cause-and-effect
(factors that
influence)
 Comparison of
groups
 Frames study
using theory

Qualitative
methods
 Individual stories
 Different
perspectives
 Complexity of
situation
 Builds up to a theory
4
The Al Gore Documentary on Global Warming
5
Stories and Trends
Whitechuck
Glacier 1973
Whitechuck
Glacier 2006
6
The Story of Mixed Methods Research

How mixed methods
 began and initially developed
 has become defined
 grew in philosophical ways
 developed technically
 became a “movement”
 has come under criticism
 continues to evolve as a methodology
HOW MIXED METHODS RESEACH
DEVELOPED AND IS DEFINED
How, when, and why did mixed methods begin?


How?
 Evolved over last twenty years
 Quantitative research to qualitative to mixed methods
When?


Beginnings in 1988-1989
 Jennifer Greene – USA- University of Illinois - evaluator
 John Hunter and Allen Brewer – USA -Northwestern and Boston College sociologists
 Alan Bryman - England – management
 John Creswell – USA – education – my chapter on combining, my teaching of
proposal development class – lst generation
 Fielding – sociologist in UK
Why?



Legitimacy of qualitative research
Increased interest in having two strands – quan and qual – in studies
Individuals looking for new methodologies
What emerged?
Quantitative Data
Qualitative Data
Mixed Methods
Method
Paradigm
Perspective
Methodology
Using mixed
methods in other designs
10
Six core characteristics of mixed methods
research






The collection of both qualitative and quantitative data
(open- and closed-ended) in response to research
questions
The analysis of both qualitative and quantitative data
Persuasive and rigorous procedures for the qualitative
and quantitative methods
The integration of these two data sources (merging,
connecting, embedding)
The use of a specific mixed methods design that
involves a concurrent or sequential integration (and
equal or unequal emphases)
An approach to research that has a philosophical
foundation
11
“Mixing” the two types of data… timing,
implementation, priority
Converge data:
Qual
Results
Quan
Connect data:
Qual
Quan
Results
Embed the data:
Quan data
Qual data
12
Typical Scenarios:





Surveys and focus groups data are merged and
compared.
Survey is first completed and then focus groups used as
a follow-up to explain the quantitative results.
Focus groups are first conducted. Information learned is
then used to construct an instrument for a follow-up with
a random sample.
An experiment is conducted and qualitative data are
collected before the experiment begins and after the
experiment has concluded.
A longitudinal study is underway with multiple studies
organized to address a single overarching research
objective. The studies are both quantitative and
qualitative .
A primer on quantitative research






Researcher makes decisions before study (variables,
hypotheses, instruments)
Researcher reduced the inquiry to a small number of
variables to study and a large number of people
Researcher asks specific closed-ended questions
Researcher analyzes numbers
Researcher places importance on reliability, validity,
generalizability, replicability, control, and lack of bias
Researcher writes the report using a standard structure
14
A primer on qualitative research
 Researcher makes decisions based on views of
participants
 Researcher opens the inquiry up to understand the
complexity of the situation
 Researcher asks open-ended questions – single
phenomenon
 Researcher analyzes words and images
 Researcher places emphasis on individual meaning,
context, and self-reflexivity
 Researcher writes the report using a flexible, literary
structure
15
Reasons for “mixing”






The insufficient argument – either quantitative or
qualitative may be insufficient by itself
Multiple angles argument – quantitative and qualitative
approaches provide different “pictures”
The more-evidence-the-better argument – combined
quantitative and qualitative provides more evidence
Community of practice argument – mixed methods may
be the preferred approach within a scholarly community
Eager-to-learn argument – it is the latest methodology
“Its intuitive” argument – it mirrors “real life”
PHILOSOPHICAL DEVELOPMENTS
Four paradigm perspectives
Postpositivism
Determination
Reductionism
Empirical observation
and measurement
Theory verification
Constructivism
Understanding
Multiple participant
meanings
Social and historical
construction
Theory generation
Advocacy/Participatory
Political
Empowerment issueoriented
Collaborative
Change-oriented
Pragmatism
Consequences of actions
Problem-centered
Pluralistic
Real-world practice
oriented
18
What are the paradigm stances in mixed
methods?





Paradigms cannot be mixed – hence mixed methods is
untenable (“incompatibility” thesis)
Multiple paradigms can be used – but they need to be
explicit and honored – and they lead to new insights
(“dialectic” – Greene, 2007)
Multiple paradigms can be used and they relate to
design and stages in the research (Creswell and Plano
Clark, 2007)
There is one paradigm for mixed methods (pragmatism
or transformative – Tashakkori and Teddlie, Mertens)
The paradigm to use is based on the researcher’s
community of scholars (Morgan)
19
The fit between paradigm and designs
stance (Creswell & Plano Clark, 2007)
Postpositivist Paradigm
Survey
Constructivist Paradigm
Focus
Group
Interviews
THE DEVELOPMENT OF TECHNIQUES
Technique Developments





Types of designs; diagrams, detailed
procedures, notation
Scripts for purpose statements
Mixed methods research questions
Analysis strategies for merging data
Point of interface strategies for sequential
data
22
Types of Mixed Methods Designs
Concurrent Mixed Methods Designs
Convergent Parallel Design
QUAN
Data &
Results
Interpretation
QUAL
Data &
Results
Embedded Design
QUAN
Pre-test
Data &
Results
Intervention
qual
Process
QUAN
Post-test
Data &
Results
Interpretation
23
Sequential Designs Mixed Methods Designs
Explanatory Design
QUAN
Data &
Results
Following up
qual
Data &
Results
Interpretation
Exploratory Design
QUAL
Data &
Results
Building to
quan
Data &
Results
Interpretation
Sequential Embedded Design
Beforeintervention
qual
QUAN
Intervention
Trial
Afterintervention
qual
Interpretation
24
Emancipatory Theory Design
A Framework
(Participatory action research, community health research, critical
theory, case study, narrative study, experimental study, etc. )
QUAL
Data &
Results
Building to
quan
Data &
Results
Interpretation
OR
QUAN
Data &
Results
Interpretation
QUAL
Data &
Results
25
Multiphase (or Multi-project) Design
QUAN
Data &
Results
QUAL
Data &
Results
Interpretation
quan
Data &
Results
QUAL
Data &
Results
Interpretation
Building to
26
Detailed diagram
27
28
Write a Purpose Statement for a
Convergent Design
Intent
Design
Data
Purpose
Rationale
This mixed methods study will address _________________
(overall content-aim of the study). A convergent mixed
methods design will be used, and it is a type of design in
which different but complementary data will be collected on
the same topic. In this study, _________________
(quantitative instruments) will be used to test the theory of
_____________ (the theory) that predicts that
__________________ (independent variables) will influence
________________ (positively, negatively) the __________
(dependent variables or outcomes) for ____________
(participants) at __________ (the research site). Concurrent
with this data collection, qualitative _______________ (type
of qualitative data, such as interviews) will explore
____________ (the central phenomenon) for
_________________ (participants) at _____________(site).
The reason for collecting both quantitative and qualitative
data are to bring together the strengths of both forms of
29
research to ___________________ (e.g., compare results,
validate results, corroborate results).
A new type of research question:
A mixed methods question
Three ways to write this question:
Methodologically-focused:

To what extent do the qualitative results confirm the quantitative results?
Content-focused:

How do the interviews with adolescent boys support the quantitative
results that their self-esteem changes during the middle school years?
Hybrid of quantitative and qualitative elements:

What results emerge from comparing the exploratory qualitative data
about boy’s self-esteem with outcome quantitative instrument data
measured on a self-esteem instrument?
Example of Joint Display (Li, Marquart, & Zercher, 2000)
31
Example of Joint Display (Lee & Greene, 2007)
32
Point of interface strategies in
sequential designs
Exploratory Sequential Design = developing and testing an
instrument
Phase I
QualitativeExploring a topic
Phase II
Developing an
instrument
Quotes=items
Codes =variables
Themes=scales
Phase III
Administering
the instrument
Testing
psychometric
properties
33
THE “MOVEMENT” OF MIXED
METHODS RESEARCH
34
Describing mixed
methods as a
“movement.”
“The emergence of mixed
methods as a third
methodological movement
in the social and behavioral
sciences began during the
1980’s.” (p. 697)
-Tashakkori &
Teddlie (2003)
35
Tracking empirical studies…
Selected Mixed Methods Studies in Trauma Research (Creswell & Zhang, in
progress)
Authors
(Year)
Journal
Topic
Quan Data
Qual Data
Integration
Impact
Factor
Jones,
Kafetsios
(2005)
Clinical Child
Psychology and
Psychiatry
Political Violence and
Psychological Well-being
HTQ
and
HSCL-25
Open-ended
questions and
observations
Quan results are explored in a
qual followup
4.434
Monneuse et
al.
(2008)
Journal of the Surgeons
Attitudes about Injury among High Questionnaire
School Students
Open-ended
questions
Quan results are explored in a
qual followup
3.101
Evans
et al.
(2007)
Journal of Traumatic
Stress
Intrusive Memories Related to
Violent Crime
Semi-structured
interview
Semi-structured
interview
Quan results are explored in a
qual follow-up and vice versa
1.884
Yassi
et al.
(2004)
Nursing Research
Factors Associated with Staff
Injuries
Telephone survey
Interview and focus
group
Qual findings help to develop
or select quan instruments
1.748
Feeney,
Ylvisaker
(2003)
Journal of Head Trauma
Rehabilitation
Context-Sensitive Behavioral
Supports for Young Children with
TBI
MAS
and
ABCI
Interview of openended questions
Quan results are explored in a
qual followup
1.643
Banyard,
Williams
(2007)
Child Abuse & Neglect
Women’s Voice on Recovery
from Child Sexual Abuse
Structured Interview
In-depth, openended interview
Quan results are explored in a
qual followup
1.506
Igreja
et al.
(2006)
The Journal of Nervous
and Mental Disease
Women’s Posttraumatic Suffering
After the War
HTQ
and
SRQ
In-depth interview
Comparison of the data
sources
1.496
Arditti
(2003)
Journal of Loss and
Trauma
Family Visiting at a Local Jail
Semi-structured
interview
Semi-structured
interview
Quan results are explored in a
qual followup and vice versa
0.528
Gaskell
(2007)
Developmental
Neurorehabilitation
Evaluating Rehabilitative Activity
Holidays for Burn-injured Children
SDQ and Selfperception Profile
for Children
Qual questionnaire
Comparing the data sources
NA
Noting
books
available…
37
MIXED METHODS
CONFERENCE
Programme
2008
www.mixedmethods.org
Cambridge University, UK
Seeing the development of
mixed methods conferences…
Noting journals
devoted to
mixed methods
research…
Founding Editors: John W. Creswell
and Abbas Tashakkori
Current Editors: Max Bergman
(Switzerland), Donna Mertens (USA)
39
Additional mixed methods journals…



Field Methods
Quality and Quantity
Multiple Approaches to Research (on-line)
Other journals that publish mixed methods studies
(e.g., International Journal of Social Research
Methodology)
 Special issue journals

CONTROVERSIES
41
Eleven key controversies (Creswell, in
press)











How should mixed methods be defined?
Does the use of “quantitative” and “qualitative” in mixed methods create an artificial
and unnecessary binary distinction?
What is driving the interest in mixed methods?
Is mixed methods a “new” approach?
Can we “mix” paradigms in mixed methods?
Does mixed methods privilege post-positivism?
Is there a dominant, meta-narrative discourse emerging in mixed methods?
Should there be a bilingual nomenclature for mixed methods?
Have we arrived at an understanding of the types of research designs in mixed
methods?
Is mixed methods claiming other designs as “their own” ?
What is the value added by mixed methods beyond conducting a quantitative or a
qualitative study?
Source: Creswell, J. W. (in press). Controversies in mixed methods. In N. K. Denzin
and Y. S. Lincoln (Eds.) SAGE handbook of qualitative research (4th ed.). Thousand
Oaks, CA: Sage.
Some Writing on the Controversies
•Is there a post-positivist leaning to mixed methods?
Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10, 42-61.
Giddings, L. S. (2006). Mixed-methods research: Positivism dressed in drag?
Journal of Research in Nursing, 11(3), 195-2003.
Holmes, C. A. (2007). Mixed(up) methods, methodology and interpretive
frameworks. Contributed paper for the Mixed Methods Conference, Cambridge,
University, July 10-12, 2006.
•Is there a dominant discourse in mixed methods research?
Freshwater, D. (2007). Reading mixed methods research: Contexts for criticism.
Journal of Mixed Methods Research, 1(2), 134-146.
43
Some questions I have heard
at University of Manitoba
• I am collecting both quantitative and qualitative data
my study, but am not calling it “mixed methods.” Should I?
• What skills are essential for conducting a mixed methods
study?
• I have the qualitative skills, but really I am not competent
in quantitative research, can I conduct a mixed methods
study?
• You have not highlighted the importance of research
questions that drive good research. Shouldn’t a person
know their question before they choose their method?
• Most journals in my field have a 3000 word limit. How can
I publish my mixed methods study with such limitations?44
What will hold mixed methods back?




Time/resources to conduct studies
Lack of understanding the methodology
Difficult to change research behaviors
Disagreements within the mixed methods community
What will push mixed methods forward?






Discipline expansion - unique discipline
adoption
International expansion and cooperation - little
“Atlantic gap,” little North American gap
Increased graduate student interests
Refinement and expansion of techniques
More advocates (and more critics)
The need for an empirical study of the value
added by mixed methods to qualitative or
quantitative
Summary







Mixed methods is a recent methodological
development
There are different philosophical stances
We know the basic feature of this approach
Many techniques for conducting mixed methods
research have developed
Mixed methods has grown into an interdisciplinary,
world-wide movement
Yet, critics have emerged, and challenges exist
The future is positive, but there is much work left to
be done
Additional readings
Books:







Creswell, J. W., & Plano Clark, V. L. (2007). Designing and
conducting mixed methods research. Thousand Oaks, CA:
Sage.
Creswell, J. W. (2008). Research design: Qualitative,
quantitative, and mixed methods approaches (3rd ed.).
Thousand Oaks, CA: Sage Publications.
Greene, J. C. (2007). Mixed methods in social inquiry. San
Francisco: Jossey-Bass.
Mertens, D. M. (2005). Research methods in education and
psychology: Integrating diversity
with quantitative and
qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage
Publications.
Plano Clark, V. L., & Creswell, J. W. (2008). The mixed
methods reader. Thousand Oaks, CA: Sage.
Tashakkori, A. & Teddlie, C. (1998). Mixed methodology:
Combining qualitative and quantitative approaches.
Thousand Oaks, CA: Sage Publications.
Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of
mixed methods in social and behavioral research. Thousand
Oaks, CA: Sage Publications.
48
Additional readings
Articles and Chapters:







Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixedmethod evaluation designs. Educational Evaluation and Policy Analysis, 15
(2), 195-207.
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003).
Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in social and behavioral research (pp.
209-240). Thousand Oaks, CA: Sage.
Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological
issues in conducting mixed methods research. In M.M. Bergman (Ed.),
Advances in mixed methods research. London: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual
framework for mixed-method evaluation designs. Educational Evaluation and
Policy Analysis, 11 (3), 255-274.
Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods
sequential explanatory design: From theory to practice. Field Methods, 18(1),
3-20.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of
Mixed Methods Research, 1, 48-76.
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological
triangulation. Nursing Research, 40, 120-123.
49
Mapping the Landscape of Mixed
Methods Research
by John W. Creswell, Ph.D.,
Department of Educational Psychology,
University of Nebraska-Lincoln
Founding Co-editor, Journal of Mixed
Methods Research
University of Manitoba, Canada
March 26, 2010
© Please do not duplicate or use these slides without the express permission of the author.
50