Political Science 30:
Political Inquiry
Qualitative Research Design II

Selecting on the Dependent Variable
Mills’ Method of Agreement
 Mills’ Method of Difference


Examples
Clem Miller
 Dreze and Sen

Selecting on the Dependent
Variable

Selecting cases according to the value of
the dependent variable that they take on
is more controversial than selecting on
the independent variable.
It allows you to look at extreme values or
divergent cases.
 “However, if this design is to lead to
meaningful … causal inferences, it is crucial
to select observations without regard to
values of the explanatory variables. K.K.V.”

Selecting on the Dependent
Variable: Method of Agreement

When you use Mills’ Method of Agreement,
you select cases that take on the same
values of the dependent variable.

This helps you to rule out possible causes,
because independent variables that vary over
these cases can’t cause the dependent var.

This method can only disprove a hypothesis,
because it can’t find a correlation.
Selecting on the Dependent
Variable: Method of Agreement

This design could help us rule out “early
industrialization” as a cause of whether
a country has a viable socialist party.
Case
France
Early
Industrialization?
No
Viable Socialist
Party?
Yes
Britain
Yes
Yes
Selecting on the Dependent
Variable: Method of Difference

When you use Mills’ Method of
Difference, you select cases that take on
different values of the dependent
variable.
After you have selected your cases, you
determine what values they take on for
some independent variables.
 Perhaps one independent variable will vary
across your cases, and explain the D.V.

Selecting on the Dependent
Variable: Method of Difference

Adding a country that has no viable
socialist party can add causal leverage
to our early investigation.
Case
Early Indust.? Feudalism? Viable Social
Party?
France
No
Yes
Yes
Britain
Yes
Yes
Yes
USA
Yes
No
No
Example: Clem Miller

This Congressman writing home to his
supporters tells the story of two different
lobbying efforts on behalf of farmers.

Let’s assume that his process of case
selection came in the same order that his
narrative is written.

If so, he selected on independent variables
and used the “Most Similar Systems” design.
Example: Clem Miller
Case
Walnut
Growers
Poultry
Men
Economic Sympathy of
Well
Trouble?
Officials? Organized?
Govt.
Help?
Example: Dreze and Sen

Both countries began a new political
regime at mid-century with large
populations and little wealth.

They have diverged since then: “There is
little doubt that as far a morbidity, mortality,
and longevity are concerned, China has a
large and decisive lead over India. (p. 205)”

“What has brought about that lead is a matter
of very considerable interest. (p. 206)”
Comparing Many Research
Methods: Measurement Validity

Measurement validity judges the gap
between your conceptual definition and
your operational definition.
Qualitative – Highest measurement validity
 The measurement validity of other research
methods (Lab experiments, quasi/natural
experiments, and quantitative research)
really depends on what you are trying to
measure.

Comparing Many Research
Methods: Internal Validity

Internal validity judges how well a
research design has tested a causal
relationship, in the cases examined.
Random assignment is the key.
Lab Experiments – highest
 Quasi/Natural experiments – medium
 Qualitative research – medium
 Quantitative research – lowest

Comparing Many Research
Methods: External Validity

External validity is how confident we can
be that a causal relationship identified in
our cases can be generalized to the
outside world. Random sampling is key
Quantitative research – highest
 Quasi/Natural experiments – medium
 Lab experiments – low
 Qualitative research – low

Comparing Many Research
Methods: Overall Lessons
 The
best research design depends
on your research question and the
particular problems that it poses.
 The
best research uses a mixture of
methods to test a hypothesis.