Two Cultures:
Contrasting Qualitative
and Quantitative Research
Gary Goertz
James Mahoney
Contents
1 Introduction
2 Mathematical prelude: a short introduction to logic and set theory for social
scientists
I Causal models and inference
3 Causes-of-effects versus effects-of-causes
4 Causal models
5 Asymmetry
6 Hume’s two definitions of cause
II Within-case analysis
7 Within-case versus cross-case causal analysis
8 Causal mechanisms and process tracing
9 Counterfactuals
III Concepts and measurement
10 Concepts: ontology and epistemology
11 Meaning and measurement
12 Semantics, statistics, and data transformations
13 Conceptual opposites and typologies
IV Research design and generalization
14 Case selection and hypothesis testing
15 Generalizations
16 Scope
Qualitative vs. Quantitative Research
-- There are many differences across
nearly all aspects of methodology.
-- These differences are not well
captured by the idea of words vs.
numbers.
-- There is no single difference that
drives or explains all other differences.
Culture: A shared set of values,
beliefs, norms, and practices.
Alternative cultures are often
associated with different toolkits or
resources for solving problems.
We seek to promote cross-cultural
understanding and communication.
We believe this understanding must
be founded upon a recognition and
appreciation of differences.
Our goal is not to criticize either
qualitative or quantitative
research.
We maintain a kind of
anthopological neutrality about
both cultures.
There is a place for qualitative,
quantitative, and multi-method
research in the social sciences.
Multi-method research is essential for
projects that require the analyst to
pursue both qualitative and
quantitative goals.
The nature of the research
question and the goal of research
shapes whether qualitative,
quantitative, or multi-method
research is most appropriate.
One Culture, Many Cultures, or Two
Cultures?
KKV: One culture founded on
mainstream quantitative techniques.
Quantitative tradition: Many subcultures
– e.g., frequentist vs. Bayesian
approaches.
Qualitative tradition: Many
subcultures too.
Split between behavioral and “causal
inference” approaches (e.g., QCA,
process tracing) vs. “post-positivist”
approaches (e.g., interpretive
analysis, critical theory, postmodern
approaches).
Our two cultures approach:
(1) Both of our cultures share with
KKV a concern with scientific
inference, including especially
causal inference.
This means that interpretive, critical
theory, and postmodern
approaches tend to drop out of our
discussion.
One could write another book
focused on the differences between
our “causal inference cultures” and
“post-positivist cultures.”
That is not the book we wrote.
(2) We insist that there are two main
cultures oriented to causal
inference: qualitative and
quantitative cultures.
Some Evidence for existence of
two cultures: formal
organizations, graduate training,
informal networks.
Other types of data: methods
books, exemplary studies, and
research articles.
We are focusing on actual practices,
not necessarily best practices.
In this book, we are neither judging
nor coaching researchers.
We are looking at what they are
actually doing.
We do hope that our descriptions are
informative and useful.
Also, we are not focusing on
“possible practices.”
For instance, one might reconfigure
fuzzy-set methods to do most of
what regression analysis does. It
is a possible or hypothetical
practice.
But no one does it in actual
practice.
The qualitative research culture is the
less well known of the two cultures.
A couple of its big defining features
are:
(1) A focus on individual cases and
the use of within-case analysis;
(2) The implicit or explicit use of
mathematical logic and set theory.
Process tracing tests provide a good
example of both within-case
analysis and set theory.
For example, consider a “hoop test.”
Hoop test:
Passing a hoop test is necessary
but not sufficient for the validity of
a given hypothesis.
This kind of test can eliminate a
given hypothesis but it cannot
always provide strong support
that the hypothesis is valid.
Example
Hypothesis: O.J. Simpson
intentionally caused the death of
Ron Goldman.
Hoop test:
Was O.J. in the general area at the
time that Goldman was killed?
Some hoop tests are harder to
pass than others:
(1) Was O.J. on the planet Earth at
the time that Goldman was
killed?
(2) Was O.J. at the Nicole Brown
Simpson home at the time that
Goldman was killed?
Failing a hoop test always
eliminates a hypothesis.
Passing a hoop test lends support
in favor of a hypothesis in
proportion to the degree that it is
a hard test.
What makes a hoop test easy or
hard?
The difficulty of a hoop test is
related to the frequency at which
the necessary condition is
typically or normally present.
Hoop tests that make reference to
rarely present necessary
conditions constitute difficult hoop
tests.
Set of
people
on Earth
Goldman’s
murderer
Goldman’s
murderer
Set of people
near Nicole
Brown Simpson’s
home at time of
murder
Other hoop tests:
(1) Is O.J. right handed?
(2) Did O.J. have motive to carry
out a violent murder?
(3) Does O.J.’s hand fit the glove?
Logic and set theory are basic to
most qualitative methods, including
within-case methods such as
process tracing.
Logic and set theory are not the
same mathematics as statistics and
probability theory.
Within-case analysis is not the same
approach as cross-case analysis.
Qualitative research is different from
quantitative research.
Causes-of-effects versus effectsof-causes
Causes-of-effects approach: Start
with an outcome to be explained
and work backward to its causes.
Effects-of-causes approach: Start
with a potential cause and ask
about its effect (if any) on an
outcome.
Contemporary quantitative
research:
Favors effects-of-causes questions.
In particular, quantitative research
seeks to estimate the average
causal effect of a treatment or
independent variable.
KKV: Define causal effect in terms of
average causal effect.
Morton and Williams (2010: 35): “A lot
of political science quantitative research
– we would say the modal approach –
focuses on investigating the effects of
particular cases. Sometimes this
activity is advocated as a part of an
effort to build toward a general model of
the causes of effects, but usually if such
a goal is in a researcher’s mind, it is
implicit.”
The Neyman-Rubin-Holland model of
causality “is purely a model of the
effects of causes. It does not have
anything to say about how we move
from a set of effects to a model of
the causes of effects” (Morton and
Williams 2010: 99).
What about regression models that try
to maximize variation explained?
“If your goal is to get a big R2, then your
goal is not the same as that for which
regression analysis was designed. The
best regression model usually has an
R2 that is lower than could otherwise be
obtained. The goal of getting a big R2 .
. . is unlikely to be relevant to any
political science question” (King 1986:
677).
We have found that few statistic
articles now use R2 as a basis for
explanation or evaluation of a
causal model.
Often the R2 in published work is
quite low. (This is not intended as a
criticism).
Qualitative researchers: They often
try to comprehensively explain
outcomes.
What caused World War I?
What caused sustained high growth
in Korea and Taiwan?
What caused social revolutions in
France, Russia, and China?
What caused the end of the Cold
War?
To answer causes-of-effects
questions, qualitative researchers
must also employ an effects-ofcauses approach. That is, they
must establish that causes had
certain effects.
But qualitative researchers do not
usually estimate average causal
effects.
Instead, they tend to understand
causal effects in terms necessary
conditions and INUS conditions
(i.e., conditions that are jointly
sufficient for the outcome).
So qualitative researchers have their
own way of thinking about the
effects of causes. It is rooted in
logic and set theory.
Qualitative researchers often seek
general explanations that apply to
more than one case.
However, for them, to provide a
convincing general explanation
entails providing a convincing
explanation of individual cases.
Basic principle of qualitative
research:
A good general explanation of Y is
also a good explanation of all
individual cases of Y.
Qualitative researchers need to be
sure that their causal model works
in their individual cases.
They do not view the estimation of a
significant average effect as the end
point or the main goal of research.
Quantitative and experimental
research:
Less oriented toward the individual
case.
Generating a good explanation for
each individual case is not the main
goal.
In fact, the Neyman-Rubin-Holland
model of causality seems to
assume it is impossible to estimate
a causal effect for the individual i,
which is precisely why one
estimates an average effect for a
population of causes.
Within-case analysis goes hand in
hand with the effort to say
something about the Xs that caused
a particular Y.
There is an affinity between withincase analysis and answering
causes-of-effects questions in
qualitative research.
Conclusion:
(1)To understand and evaluate
research, one must take into
consideration research goals;
(2) Qualitative and quantitative
researchers often have different
research goals;
(3) Failure to recognize this fact
generates miscommunication and
misunderstanding.
END