Assessing the Trivialness, Relevance,
and Relative Importance of
Necessary or Sufficient
Conditions in Social Science*
Gary Goertz
Political scientists of all stripes have proposed numerous necessary or sufficient
condition hypotheses. For methodologists a question is how can we assess the importance of these necessary conditions. This article addresses three central questions about the importance of necessary or sufficient conditions. The first concerns
the "trivialness" of necessary or sufficient conditions. The second is how much a
necessary or sufficient condition is "relevant?" The third important question deals
with the relative importance of necessary or sufficient conditions: for example, ifX~
and X2 are necessary or sufficient conditions, is one more important than the other?
The article develops measures to assess the importance of necessary or sufficient
conditions in three related contexts: (1) Venn diagrams, (2) 2 x 2 tables, and (3)
fuzzy sets. Two empirical examples are discussed at length: ( 1) Skocpol's States and
Social Revolutions: A Comparative A nalysis ~?1eFrance, Russia, and China and (2)
Ragin's (2000) analysis of the causes of IMF riots.
Introduction
Political scientists o f all stripes---e.g., game theorists, quantitative scholars, social
constructivists--have proposed numerous necessary condition hypotheses (see the
chapters in Goertz and Starr, 2003 for examples). For example, the social and economic requisites o f democracy have been debated for more than 40 years (Lipset,
1959). The agreement o f all veto players is a necessary condition for policy change
(Tsebelis, 1999). Hypotheses about necessary or sufficient conditions play a cenGary Goertz is professor in the Department of Political Science at the University of Arizona. He is
the author or co-author of four books and more than 25 amcles on issues of international politics,
methodology, and conflict studies. He is the author of Social Science Concepts (Princeton University Press, 206), Contexts ofhlternational Polittcs (Cambridge, 1994), co-author with Paul Diehl of
Territorial Changes and International Col~ict (Routledge, 1992), and War and Peace in lntetwational Rivah'l' (University of Michigan Press, 2000).
Studies in Conzparative International Development, Summer 2006, Vol. 41, No. 2, pp. 88-109.
Goertz
89
tral role in qualitative macrocomparative studies. A question for methodologists is
how can we assess the importance of necessary or sufficient conditions?
The concept of a trivial necessary condition forms part of the background knowledge of most social scientists. Since the concept is not taught in methods courses, it
is not clear where these intuitions come from or whether they are the same for all. A
major goal of this article is to provide an in-depth analysis of the concept of a trivial
necessary condition. As Downs illustrates (see below), common examples of trivial
necessary conditions invoke factors which are constant for all values of the dependent variable, as illustrated by the hypothesis that air is a necessary condition for
armies to operate. We shall see that this common notion oftrivialness is valid. Yet
the analysis oftrivialness leads to a second alternative conceptualization that is not
part of our collective intuition. This second approach starts from the opposite side
by saying that the most nontrivial necessary condition is one that is also sufficient.
I call this dimension the relevance of necessary conditions. These two approaches
lead to different measures of how important a necessary condition is.
Finally, I address the issue of the relative importance of necessary or sufficient
conditions. If a given theory has multiple necessary or sufficient conditions, how
can we determine if one is more important than another? It is common in statistical
analyses to assess the relative importance of causal factors; qualitative methods
scholars have this same need.
If the concept of a trivial necessary condition forms part of our background
knowledge, such is not the case for sufficient conditions. However, we can fruitfully ask the same question about sufficient conditions: How trivial are they? While
this may seem like a nonsensical question, we shall see that it has a perfectly reasonable interpretation that mirrors nicely that of a trivial necessary condition.
With the publication of Fuzzy-Set Social S(ience, Charles Ragin provided social
scientists with an impressive set of new tools for the analysis of social and political
behavior. One thing that Ragin does not really discuss is how we might evaluate the
importance of a necessary or sufficient condition. Fuzzy-set social science tells
you how to find necessary or sufficient conditions, but only provides hints about
how to analyze the importance of those conditions. For example, Goldthorpe criticizes qualitative comparative analysis ([QCA ], Ragin, 1987): "In an application of
QCA, we should note that the independent variables are simply shown to be causally relevant or not; no assessment of the relative strengths of different effects or
combinations of effects is, or can be, made" ( 1997: 7).
Questions of importance arise almost without fail in the discussion of causal
explanations. For example, is resource mobilization more important than political
opportunity structure in explaining social movements? In the specific case analyzed by Ragin and which I take up here, is urbanization more important than IMF
pressure as a cause of IMF riots?
This article has two overall goals. The first is to provide conceptual tools for the
nonmethodologist so that she can understand, make, and address important critiques of her own and others' work. All the key conceptual points are made with
Venn diagrams and simple 2 x 2 tables, assuming no mathematical background. As
an ongoing example, I use Skocpol's well-known States and Social Revolutions: A
Comparative Analysis of France, Russia, and China (1979). At the core of this work
lie two necessary and one sufficient condition hypotheses.
90
Studies in Comparative International Development / Summer 2006
The second goal of the article is to provide a series of quantitative measures of
the key concepts. While requiring no extensive mathematical knowledge per se, the
discussion is inherently technical. In addition, these measures deal with issues that
are a continuation of Ragin's (2000) work on fuzzy sets, and as such assume knowledge of that book. Since my measures extend those discussed in his book, I use as
ongoing examples his two empirical analyses of IMF riots and welfare states to
show how assessing importance is a natural next step in fuzzy-set methodology.
In short, this article can be read in two ways. The first is to read the nonstarred
sections. This is a complete article, assuming no methodological background and
focusing on the key conceptual issues. Second, one could read the whole article,
including the starred sections, to understand the technical details. The nonstarred
sections use Skocpol's States and Social Revolutions as an ongoing example to give
substance and intuition to the methodological discussion.
The Trivialness and Relevance of Necessary Conditions
In some respects, it is not natural to ask about the importance of a necessary or
sufficient condition. I propose that one way to attack the importance question is
through its opposite trivialness. It seems clear that i f X is a trivial necessary condition, then it has little or no importance. Since we have some intuitions about
trivialness (at least for necessary conditions) we can perhaps leverage that into an
analysis o f the importance o f necessary conditions.
Braumoeller and Goertz (2000) provide the only explicit analysis of trivialness
that I am aware of(also see Dion, 1998). They start with a typical consideration of
trivial necessary conditions:
The search for necessary conditions is problematic because the utility of a necessary
condition is contingent and poorly understood. There are an infinite number of necessary
conditions for any phenomenon. For example, it is true that all armies require water and
gravity to operate, but the contribution of such universals is modest (Downs, 1989: 234).
Gravity is a trivial necessary condition because it is constant across all cases of
armies. Braumoeller and Goertz extend this basic idea to define a trivial necessary
condition as being one that is present in all cases in the universe of analysis, both
when the dependent variable is present and absent.
However, the analysis of a trivial necessary condition assumes that it is a necessary condition. An essential first step in the analysis of the importance of necessary
conditions is to define a necessary condition itself. In terms of set theory:
X is a necessary condition for Y if Y is a subset o f X.
Skocpol's States and Social Revolutions has two such hypotheses: state crisis
and peasant revolt are necessary for the occurrence of social revolution. For example, the set of social revolutions is a subset of the cases of state crisis. In other
words, whenever we see social revolution (Y) we find state crisis present (Xt). There
already exists significant literature on assessing whether a given factor is a necessary condition (e.g., Dion, 1998; Braumoeller and Goertz, 2000; Ragin, 2000). For
Goertz
91
Figure 1
Trivial Necessary Conditions: A Set Theoretic Perspective
Xs: Universe
/
J
x4
/
the purposes of this article, this evaluation has already been conducted, but we need
further analysis of the importance o f these necessary conditions.
In Figure 1, Y is contained in X z, hence Y never occurs unless X 2 does. The rectangle (variable Xs) is the universe of cases. The variables X I, X 2, X 3, and X 4 are all
necessary conditions for the set Y, since Y is a subset of them all. X 3 is more trivial
than X2, which is more trivial than XI because the set Y as a proportion of the set X 2
decreases from X 1to X 4. The variable X 5 is a completely trivial necessary condition
because no matter what Y is, it must always be a subset o f X 5, the universe of cases.
I believe that this constitutes the commonly held intuition about what a trivial necessary condition is (illustrated by Downs's remarks above). X 5 is a trivial necessary
condition because it always occurs. X s is a necessary condition since Yis a subset of
X 5, but it is a trivial one.
In the context of States and Social Revolutions, state crisis would be a trivial
necessary condition if it occurred virtually all the time. If a state were chronically
unstable, then it is not surprising that when social revolution does occur it is during
a time of state crisis. This is exactly what Downs was referring to when he used
examples such as gravity is necessary for armies, a condition that is necessary but
trivially so.
Downs illustrates that most people have an intuitive notion o f a trivial necessary
condition. But there is no literature on the converse way of assessing necessary
conditions through what I call its "relevance" (to give it a name). Instead of defining the importance through trivialness, we can define it in terms of a condition that
is maximally relevant. Here we have an intuition which we can build on:
A maximally important, relevant, necessary condition is also a sufficient condition.
In general a necessary condition is more important the more sufficient it is.
Studies in Comparative International Development / Summer 2006
92
Table 1
Necessary Conditions for Social Revolutions (S. R.)
Table lb
Peasant Revolt (ER.)
Table la
State Breakdown (S.B.)
S.R.
~S.R.
~S.B.
S.B.
0
279
3
13
~ER.
S.R.
~S.R.
0
286
P.R.
3
6
We can see that necessary conditions become more important as the difference
between X, and Y decreases. In other words, as the set X, shrinks in its coverage o f Y
its importance increases. Since YCXbecause Xis a necessary condition, the smallest possible set X, is when X=Y. Hence, X, is most relevant when X and Y are equal.
Tables la and lb give the empirical data for Skocpol's two necessary conditions
for social revolution. These data come from an extensive theoretical and empirical
analysis of her book; for details, I refer the reader to Goertz (2006: Chapters 7 and
9). A word should be said about the zero values, notably those with zero on all three
variables (two independent and one dependent). As part of our analysis, Jim Mahoney
and I developed a list of countries that fall within the scope of Skocpol's theory. I
use 10-year blocks to count the zero values. Since social revolutions typically take
more than one year to occur, it seemed reasonable to use a longer period to count
when they do not occur. While 10 years is arbitrary, so would be using a year.
Nothing of importance in the analyses to follow hinges on this. We can see from
Tables l a and lb that state crisis and peasant revolt are necessary conditions since
when state breakdown (S.B.) or peasant revolt (ER.) are absent, social revolution
(S.R.) does not occur. By definition i f X is a necessary condition for Y it must be
present when Y occurs. A key principle is:
Zero in the (~X, Y) cell means X is a necessary condition.
The hypothetical data in Tables 2a and 2b illustrate perfectly trivial and perfectly
relevant necessary condition hypotheses respectively. What makes X trivial is that
~Xnever occurs, which permits a new insight into the logic of necessary conditions
in 2 x 2 tables:
The (~X, ~Y) cell is the "trivialness cell" o f a 2 x 2 table.
To assess trivialness, we look at the content of the (~X, --Y) cell. If it is zero, then
you have a trivial necessary condition; if there are a reasonable number of observations, then the necessary condition is not trivial.
In practice, this is often an easy test to pass. For example, in States and Social
Revolutions, there are many periods where there is neither state crisis nor social
revolution. This cell is by far the one with the most observations in Tables la and
Goertz
93
Table 2
Trivial and Relevant Necessary Conditions in 2 x 2 Conditions
Table 2a
Trivial Necessary Condition
Y
~Y
~X
X
0
0
lO0
500
Table 2b
Relevant Necessary Condition
Y
~)"
~X
X
0
500
lO0
0
lb. We often can conclude without any formal measures (described below) that the
necessary condition is not trivial. Yet, there are cases such as the democratic peace
where it may not be so obvious. It may be very difficult in some time periods to find
cases of no war and democracy because democracies were so rare. It is possible that
only a small percent of all dyads are joint democracies.
Table 2b illustrates a completely relevant necessary condition. Recall relevance
is defined as being sufficient. We can think of the cases in the (X, ~Y) cell as
counterexamples to the proposition that X is sufficient for Y: as the number of these
counterexamples decreases, the support for the sufficiency hypothesis increases.
As we move more and more cases from the -'Y to the Y cell, the importance of the
necessary condition goes up. Eventually, maximal relevance is achieved when the
(iV, ~Y) cell has zero cases and we have the classic diagonal table of a necessary and
sufficient condition. This is what Goertz (2003) called the sufficiency effect of a
necessary condition: how much the presence of a necessary condition X helps produce Y. We have a principle for the relevance of necessary conditions in 2 x 2 tables:
The (X, ~Y) cell is the relevance cell tbr necessary conditions.
Looking at the Skocpol data in Tables la and lb, we can see that the factors of
state breakdown and peasant revolt are not maximally relevant. If they were, the
cells with 13 and 6 would be zero, and we would see more social revolutions as a
result. While both variables have a sufficiency effect that is important, they are not
sufficient by themselves to bring about social revolution.
In sum, we can see that the necessary condition hypotheses in Skocpol's States
and Social Revolutions are clearly nontrivial and relevant. There are many observations in the trivialness cell. At the same time, there are occasions when state breakdown or peasant revolts occur without producing social revolution, so they are not
maximally relevant. The starred sections below provide a quantitative measure of
these two concepts. Yet, as the Skocpol example illustrates, for many qualitative
analyses, an examination of the data is often adequate to demolish convincingly
claims of trivialness.
It is clear that the concepts of trivialness and relevance are not the same. This is
most easy to see because they refer to different cells of a 2 x 2 table. While most
people have an intuitive and correct feel for the concept of a trivial necessary con-
94
Studies in Comparative International Development / Summer 2006
dition, it is important to realize that it has a partner that has not been recognized. We
can also assess necessary conditions by how much they are also sufficient conditions.
When Trivial Necessary Conditions Are Important: Scope
While it is not always obvious, necessary conditions are often used in scope conditions. Theory Z applies only if certain scope conditions are met. These scope conditions themselves are necessary conditions.
For necessary conditions used in scope statements, we want ones that are in
some sense trivial. Recall that the basic notion oftrivialness is a variable that occurs
often. For scope conditions this is a good thing.
Skocpol limited her scope to politically ambitious agrarian states that have not
experienced colonial domination (1979: 33-42, 287-290). She explicitly excludes
cases in which the possibilities for revolution have been shaped by the legacies of
colonialism, dependence in the international economy, and the rise of modern militaries differentiated from dominant classes. These are quite a limitation: nearly all
modern Third World countries are excluded by Skocpol's scope statement.
This twist on the trivialness of necessary condition in scope statements is a constant concern. Most obvious, it arises when we begin to add more necessary conditions onto scope, which then decreases the number of cases where the scope applies
and which eventually might be quite small. For example, Gruber gives four important necessary conditions for the application of his theory of international institution formation. He argues that "while the argument I have been making is a
conditional one, however, it should be noted that none of the necessary conditions
is especially restrictive or demanding" (2000: 46). What he does not mention is that
all four necessary conditions together might reduce significantly the scope of his
theory.
In short, trivialness concerns push in different directions, depending on whether
the necessary condition is used in scope statements or the theory itself.
*The Mathematics of Trivialness and Relevance
Fuzzy-set analysis extends the tradition of Boolean or Aristotelean logic by allowing continuous values between zero and one instead of dichotomous (or "crisp" in
fuzzy-set terminology) values. The fuzzy-set "membership score" can be seen as a
continuous value of the X or Y variable. Space considerations prevent a complete
exposition of why Xis necessary for Ymeans that the fuzzy-set value of Xis greater
than or equal to the fuzzy-set value of Y. Basically the set theoretic notion of containment (i.e., C) becomes the relationship of less than (i.e., <): in an important
sense a number which is less than another is "contained in" the larger number. This
fuzzy-set sense of containment for necessary conditions produces triangular data
like those in Figure 2.
We can apply the basic insight of Venn diagrams and 2 x 2 tables to the fuzzy-set
definition of a necessary condition. I f X = l for all Y, then it is a necessary condition
for Y, since Y must lie in the [0,1] interval. Figure 2 illustrates this for the lower
triangular data typical of a necessary condition. The dots represent a standard non-
95
Goertz
Figure 2
Trivial Necessary Conditions: A Fuzzy-Set Perspective
1.0
0.9
0.8
0.7
Y
0.6
v
0.5
m
0.4
m
0.3
0.2
0.1
0.0
I
I
0.0 0.1
I
I
~
I
I
I
I
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
I
1.0
X
Nontrivial necessary condition: . . . . . .
Trivial necessary condition: 9 9 9 9 **
trivial necessary condition 9 The asterisks are a trivial necessary condition 9 Notice
that they all hover very near the X =1 vertical line9 Perfect trivialness is achieved in
the case where X = I for all cases in the population 9
I propose then that the fuzzy-set trivialness of variable X be based on the distance between x, and 1. If x, is always 1, then lhe trivialness score will be zero since
the distance between x, and 1 is always zero. As x, moves away from 1, its importance increases and the trivialness score moves away from zero.
We now know the minimum trivialness score for each x, which is zero, but we do
not know what its maximum "importance" score can be. In Figure 1, a necessary
condition becomes important as it approaches Y. In fuzzy sets, a necessary condition must be greater than or equal to the membership score o f the dependent variable. In Figure 2, this means that x, > y,. Hence we can "standardize" how trivial x,
is by how far from 1.00y, is: ( l - x ) / (l-y,).
I suggest that the trivialness of X b e defined as the average distance between x,
and 1, standardized by the m a x i m u m importance that this value can attain based on
y,. The measure of trivialness, T,,~.~, o f X is the average distance from x, to 1.00
standardized by how fary, is from 1.00:
N
"
Studies in Comparative International Development / S u m m e r 2006
96
A completely trivial necessary condition has x, = 1 for all i, so a completely
trivial necessary condition has a Tn~.~value o f zero. This means that the further Tn~,is
away from zero the more nontrivial, i.e., important, the necessary condition is.
Going at things from the other direction, by definition of necessity, i.e., x, ->y,,
maximal importance (i.e., least nontrivial) occurs when x, =%, which gives a maximum T,,~ score of 1.00. This means that i f X i s a necessary and sufficient condition,
it has the maximum importance score of 1.0, which confirms our intuitive understanding o f the ultimate importance o f a necessary condition as being also sufficient (see below for more on this).
We can ask what relevance looks like in a fuzzy-set context. A necessary and
sufficient condition in fuzzy set is one that lies on the X = Y diagonal line. As
illustrated in Figure 3, a very relevant necessary condition is one where all the observations lie on or just below the diagonal line. This contrasts with trivialness,
which puts all the cases near the X = 1 line.
Just as I defined a trivialness score, ir , as the relative distance to the X = 1
vertical line, we can create a measure of the relevance of a necessary condition,
R,,,.,, which is how close, relatively, X i s to sufficiency in the fuzzy-set sense. Recall
that by definition a necessary condition is one where x, > y,. If trivialness is 1 - x,,
then relevance is closeness to y,. We can define the relevance measure of a necessary condition as ~ (y,/x,). As above, maximal relevance is achieved when x, = y,,
which means that the maximum value o f the ~ 0 ' , / x , ) is N, and which we then use
to standardize the measures so that it ranges from zero to one:
1
R ..... = - - ~ , y , / x ,
(2)
N
In short, when R,~, c is near zero, it indicates a nonrelevant necessary condition,
while relevance increases as R,,~,. does.
We now have two measures o f the importance o f necessary conditions. One, T,,ec,
evaluates necessary conditions based on the criterion oftrivialness, while the other,
R.ec, uses the standard of sufficiency to determine how strong a necessary condition
is. How do they compare?
First, we need to notice that they are not identical. One way to look at this is to
ask when the two measures are equal.
1 - x,
y,
1 - y,
x,
x, (1 - x , ) = y , (1 - y , ) which is true when
x, = y, or when
y, = 1 - x,
When we are on the X = Y diagonal, the two measures are identical. Recall that
both measures achieve their m a x i m u m (1.00) when x, = y , which is what we want
since it is a sufficient condition and therefore the least trivial necessary condition)
It turns out that the Y = 1 - X diagonal line divides the two measures, while the
two measures are identical for all points on the line. As illustrated in Figure 4, for
points to the southwest, Tn~c gives higher importance scores, while for observations
97
Goertz
Figure 3
Relevance of Necessary Conditions: A Fuzzy-set Perspective
1.0
0.9--
0.8
,.-%
,%
0.7
* ' , .~"~ *
0.6 "
y
0.5
0.4
:g.
0.3
0.2
0.1
0.0
,
,.~.
*I
I
I
I
I
I
I
I
I
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
I
1.0
X
Average necessary condition: . . . . . .
Very relevant necessary condition: 9 9 9 9 **
to the northeast, it is the R~,,cmeasure that is the larger o f the two. This makes sense
when we think about what the Y = 1 - X line represents. Recall that the key relevance cell was ()I, ~X). In terms o f Figure 4, this is now the Y = 1 X line. As we
move toward northeast, the relevance measure becomes more important than the
trivialness one. We found that (~X, ~Y) cell is key for the trivialness measure, which,
in terms o f Figure 4, is the origin. It is not surprising that as we move southwest
toward the origin (i.e., the (~X, ~Y) cell), the trivialness measure trumps the relevance one.
Why this is true can be seen with some simple examples. We must recall t h a t X i s
necessary for I1, therefore x, >_3;- Take y , and x, to be the point (.7,.6) that is in the
R,,,., region o f Figure 4. Now R,,,,, - 9
= .86 > .75 = ( 1 - .7) / ( 1 - .6) = 7,,,,. If we
take the point (.3,.4), which is in the T,,, part o f the graph, we find that T,,,,,= ( 1 .4)
/ ( 1 - .3) - .86 > .75 = .3 / .4 = Roec.
Notice that I have chosen two examples that are symmetric across the Y = 1 - X
diagonal, as can be seen in Figure 4. Using this example, we can see that T,,,.,for the
point (.3,4) equals R,,, for the point (.7,.6) as well as the converse. This is a product
o f the symmetry o f the two around the Y - 1 Xline. For the point (.7,.6), Xis .7, so
98
Studies in Comparative International Development / Summer 2006
Figure 4
Comparing Trivialness and Relevance Perspectives
on Necessary of Sufficient Condition Importance
1.0-- \
0.9-
,, ,,
R~/>
T~f
/
0.80.7" " " -, -,
0.6Y
0.5-
Tsuf > Rsuf
0.4-
(.3,.4)* /
,.6)
/ , ,
Rnec > Tnec
" ,, ".
*(.4,.3)
0.30.2-
T~
" ,. ".
",,,,,,
> Rnec
" , ,,
0.10.0
I
I
I
I
I
I
I
[
I
I
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
X
1 - X yields .3, which is the X-value of the point (. 3,.4). A similar calculation gives
the two corresponding Y-values.
Notice that the R,~, and T,ec measures divide the lower triangular necessary condition region of the graph in two equal parts. As a result, if the data happen to be
uniformly and symmetrically distributed across both regions, we will obtain the
same overall values for how important X is from each measure.
Is T,er better than Rnecor vice versa? Both are based on valid notions of relevance
and trivialness. Given the symmetry o f their behavior, it seems that a good overall
measure of the importance of a necessary condition would be the mean of the two.
I propose that the measure of importance of a necessary condition, i.e., TR..... be the
mean of the two.
Ragin (2006) has proposed alternative measures that deal with determining the
trivialness of necessary or sufficient conditions. He uses the concept of"coverage"
to assess the importance of necessary of sufficient conditions. Coverage is appropriate since as Figure 1 illustrates, the trivialness of a necessary condition depends
on how X covers Y. The exact measure he uses differs from those defended here.
Instead of the basic ~(x,/y,) formula used here, he suggests ~ x , / ~ y,. My measure
keeps paired values of X and Y together while his splits them before summing: in
short, I compare-then-sum and he sums-then-compares.
If the data are well-behaved (Why should data be well-behaved if children are
not?) then the two produce similar results. The main differences between the two
Goertz
99
Table 3
Relevance and Trivialness for Necessary Conditions for IMF Riots
Importance
Urbanization
IMF Pressure
T,,~,
R,,.,
TR.....
0.60
0.41
0.50
0.46
0.39
0.43
occur when there is a contrast between small numbers that produce a high ratio,
e.g., .2/.3 and small proportions with large denominators,. 1/.9. For example, the
dataset {(.3,2) (.3,.2) (.9,. 1) (.9,. 1)} yields ~ v , / ~ x, = .25, while ~ (y,/x,) = .39.
*The Trivialness and Relevance of Necessary Conditions in IMF Riots
We have seen above that both the trivialness and relevance perspectives on the importance of necessary conditions have validity. It is useful to see how they perform
with real-life data. To this end, I use the data on IMF riots that Ragin analyzed in
Fuzzy-Set Social Science. In Chapter 8, he found there were two necessary conditions for IMF riots: urbanization and IMF pressure.
Many Third World countries needing debt (re)financing were forced to go to the
IMF for funding. Typically, the IMF puts policy conditions on such loans, which
often include reduced funding for public services or reduced price supports for
necessities such as food and fuel. The resulting large increases in prices often led to
what were called "IMF riots." The dependent variable in Ragin's analysis is the
severity (measured of course via fuzzy set) of the riots.
There is a debate about the causes of these riots (for a survey, see the original
analysis by Walton and Ragin, 1990). The independent variables that they included
in their analysis range from more proximate causes such as "IMF pressure" to more
structural variables like the degree of urbanization, which reflects in part the size of
the groups most affected by price hikes in necessities. Additional independent variables included: (1) degree of economic hardship, (2) degree of dependence on foreign investment, (3) the degree to which a government is "activist," and (4) the
degree of political liberalization in the 1980s.
In analyzing the trivialness and relevance of necessary conditions, we must only
operate on necessary conditions. As stated above, Ragin cites two necessary conditions for IMF riots: urbanization and IMF pressure, but he does not ask about how
trivial these necessary conditions might be. My analysis oftrivialness starts where
his analysis of necessary conditions ended. Table 3 explains how these two necessary conditions perform on the T,.... R ..... and TR,~.~.measures.
We can see from the results in Table 3 that T,,~cand R,,,~ can produce different
evaluations of the importance of necessary conditions. This is particularly striking
in the case of urbanization, which has a value of.60 from the trivialness perspective
on importance, while the relevance view gives it a significantly lower value of.41.
As we saw above in Figure 4, the distribution of cases can play a major role in
explaining these kinds of differences. If most of the observations lie in above the Y
= l - X diagonal, then we would expect R,~c to be larger than T,,~r and, vice versa, if
a significant majority of cases lie below that diagonal.
100
Studies in Comparative International Development / Summer 2006
Overall, these results suggest that neither of these necessary conditions is trivial
or nonrelevant. Since neither measure is near zero, the evidence suggests that both
urbanization and IMF pressure are significant necessary conditions for IMF riots.
The Relative Importance of Necessary Conditions
While intuitions often serve well in thinking about trivialness, they often lead astray
when considering the question of the relative importance of necessary conditions.
At first blush it might make little sense to ask ifXl is more necessary than X 2. It is
better to rephrase the question: Is XI a more important necessary condition than
in causing Y?.
It is useful to think about this in the context of what Jack Levy and I have called
"powder keg" models of explanation (Goertz and Levy, 2007). The powder keg
image must be one of the most popular in historians' metaphor box. In terms of case
studies and qualitative methods, the powder keg model is often an expression of a
necessary condition theory. The powder keg and the match are individually necessary and jointly sufficient for the explosion. For example, Skocpol (1979) notes
that the breakdown of the government during state crisis opens a window of opportunity, which when combined with the spark of peasant revolt results in social revolution. Since the powder keg model is conjunctural, i.e., involves interaction terms,
we can ask the same question of"2 x 3 = 6." Which is more important in producing
6, the 2 or the 3?
The answer to these two questions is the relative frequency with which X 1and X 2
occur. If2s are rarer than 3s, then 2 is a more important cause of 6. We take the least
common factor to be the more important cause. To see this intuition, take some
powder keg scenarios that vary the relative frequency of the spark and the keg. A
smoker lights up and there is a gas leak in his house, resulting in the explosion of
his house. Lighting a match and the presence of gas due to the leak (like 2 and 3 in
the production of 6) are both causes of the explosion. Yet when asked for the cause
of the explosion, people will say it was the gas leak. Gas leaks are relatively rare,
while the smoker has lit thousands of matches. On an oil rig, where gas is often
present, the cause of an explosion will be the careless worker who lights a cigarette.
Honor6 and Hart (1985) show how this principle is embodied in most Western
legal systems. Courts have to decide in many individual cases that are causally
complex. While it is not the only causal principle used, the relative frequency rule
plays a key role. Normal events, situations, and occurrences are not seen as important causes, rare and unusual actions are more often seized upon as the main cause)
The key principle is the relative frequency with which the necessary conditions
occur: The less often the necessary condition occurs the more important it is vis-hvis the other necessary conditions.
Tables la and lb show how frequently state crisis and peasant revolt occur within
the scope of Skocpol's theory. We are looking at the X column of the 2 x 2 tables.
Since we are comparing factors that are necessary conditions they must have the
same values in the (X, Y) cell. Relative importance really comes down to comparing
the values of the (X, -~Y) cell. In other words: how often, comparatively, does state
crisis (X) not lead to social revolution (~Y)? As I noted above, the (X, ~Y) cell is the
relevance cell. In short, the relative importance of necessary conditions is a ques-
Goertz
I01
tion about their relative relevance) From the data in Table la, we know that it is 13
times for state crisis, while for peasant revolt it is only 6 times, so peasant revolts
are 13/6 = 2.2 times more important than state breakdowns in causing social revolutions.
Of Skocpol's two main causal factors, it is fair to say that most readers have
focused their attention on the state breakdown variable. For example, students of
Skocpol who have developed their own major theories of social revolution have
zeroed in on this variable (Goldstone, 1990; Goodwin, 2001 ). Yet my results clearly
suggest that, empirically speaking, the peasanl revolt variable is the causally more
important one.
Precisely because it is more difficult for peasants to stage large scale revolts than
for states to experience major crises, it is appropriate to view peasant revolts as the
more important cause, Alternatively, we can make the same point by thinking in
terms of correlations. The correlation between state crisis and social revolution is
much lower than that for peasant revolt and social revolution. In sum, we can assess
the relative importance of necessary conditions. In the case of States and Social
Revolutions, this leads to a significant reappraIsal of the implications of that work. 4
*The Mathematics of Relative Importance
It has become standard practice in event history methods to evaluate relative importance in the percentage change in the dependent variable for a standard deviation
change in the independent variable. If a standard deviation change in X t yields a 50
percent increase in the probability of Ywhile lbrX~ the change is 100 percent, then
X2 is more important than X~. In an analogous way, we naturally want to know if
necessary condition X t is more or less important than necessary condition ~ .
Often the variables in the model reflect major theoretical differences. So debates
about the relative importance ofX~ vis-g~-visX~ impinge on the evaluation of theory
1 versus theory 2. While both ,V~ and X2 might be statistically significant, the researcher often wants to argue that X t '+explains more variance" than X 2.
We might suspect that IMF pressure is the key variable in explaining IMF riots
since, after all, the dependent variable is IMF riots. Yet structuralist approaches
such as world system and dependency perspectives (Walton and Ragin, 1990:879
880) stress the importance of the structural +'preconditions" for such riots. In short,
we have many reasons to want to assess the relative importance of necessary conditions.
The measure of ++relative importance," which 1 propose, takes advantage of an
idea that Ragin proposes in his analysis of conjunctural sufficient conditions for
welfare provision (297-299). The most important of the conjunctural sufficient
conditions is the one that has the maximum score the most often. Ragin finds that
"strong left parties" is the most important sufficient condition because it provides
the maximum more often than the other sufficient conditions.
We can generalize this logic and apply it to necessary conditions. Necessary
condition 1 is more important than necessary condition 2 if the value of necessary
condition 1 is smaller than that of necessary condition 2. In terms of trivialness, a
necessary condition is more important the further away it is from 1. For each case,
we can see which necessary condition is the smallest (i.e., the furthest from 1.00).
102
Studies in Comparative International Development / Summer
2006
Therefore, xl is the most important necessary condition if it is the minimum over all
the K necessary conditions in that particular case, i.e., Xl, = min(xl~, x~2,...,x~x). In
general, X~ is more important if it is the minimum in a higher percentage of cases
than X 2. One measure of relative importance, Inec, is the ratio of the percentage of
times X 1, X 2, etc., provide the minimum value.
Table 4 gives the Inec values for the urbanization and IMF pressure variables. It
turns out that urbanization has the minimum score 60 percent of the time while IMF
pressure has the lowest value for only 40 percent of the cases. This means that Inec
rates urbanization much more important than IMF pressure, 50 percent more important. This might lead us to conclude that the domestic context is more important
in explaining IMF riots than the actions of the IMF itself.
The Inecmeasure implicitly employs a dichotomous weighting scheme: a variable
gets a weight of 1.00 if it is the minimum and zero otherwise. This does not take
into account that there may be little difference between the minimum and the next
lowest value. For example, the I, ecmeasure treats the two cases OfXy,x 2 with values
(.2,.8) and (.2,.25) in exactly the same way: in both casesX t gets a value of 1 while
X 2 gets a value of 0.
We might then want to develop a more refined and sensitive weighting scheme
that takes into account how close the other necessary conditions are to the smallest
one. One can do this by considering "how close" the other necessary condition(s)
are to the minimum: the closer they are the larger their weight should be. When they
are equal then they should get the same weight as the minimum (which is the procedure I applied above when there were ties). Formally, this can be expressed in terms
of the relative trivialness of the two necessary conditions. Looking at the individual
values of variable X~, i.e., Xl,, i = 1 ..... N over K necessary conditions:
1-Xlz
where Xl.m,. =min(xll.Xi2.Xl3,...Xlk)
(3)
n
Notice that trivial necessary conditions, i.e., Xl, = 1 receive zero weight because
1 -x~, = 1 - 1 = 0. The relative importance score for X1 is the average of the relative
importance scores for each case, i.e., x~,.
It is clear from the preceding sections that in addition to a trivialness measure of
relative importance we can define a relevance measure as:
IRnec(Xl,) = Xl'm'n wherexl,mm = min(xll,Xlz,xl3,.
Xl t
,)Xlk
(4)
These two relative importance measures have strong links to the measures of the
importance of necessary conditions developed above. The measures of the importance of a necessary condition tout court examine its "necessariness" and are relative to the dependent variable Y. The relative importance measures developed in this
section look at the relationship between necessary conditions. The necessariness
measures answer the question "how trivially necessary is X for Y?.," while the relative importance measures respond to the query "is XI more necessary than ~ ? " The
latter question is about the relations among the Xs.
Table 3 gives the relative importance scores for the urbanization and IMF pressure variables for the measures discussed above. Urbanization is the more impor-
103
Goertz
Table 4
The Relevance Importance of Necessary Conditions for IMF Riots
Relative Importance
Urbanization
IMF Pressure
percent
1.50
0.60
1.00
.40
ITs,
1.26
IR .....
ITR.e ~
1.10
1.18
I,e `
1.00
1. O0
tant necessary condition for all indicators, but how much more important than IMF
pressure varies significantly. Not surprising, The most extreme value 1.50 occurs
with the Inecmeasure, which gives zero weight to the necessary condition that is not
the minimum. All other measures give some weight to the nonminimum; thus we
expect the continuous weighting scheme to reduce the contrast between the two
variables. Instead of receiving zero weight, the larger of the two necessary conditions receives some nonzero positive weight (unless it has value 1.00). Usually, this
will tend to make the measurement of relative importance of necessary conditions
closer to each other.
As we saw above, trivialness and relevance measures can produce different results. Here, the trivialness version of relative importance, i.e., I T , ec, makes urbanization much more important than IMF pressure with a value of 1.26; in contrast
with the relevance measure, I R ..... the two come out about equal at 1.10. A good
overall measure of relative importance would be the mean of 1Tn~ c and IRne ~. In the
case of IMF riots, this mean, i.e., I T R n e ~, produces the value of I. 18. In short, it
seems that overall the urbanization necessary condition is roughly 20 percent more
important than the IMF pressure necessary condition.
This section has presented a basic measure, with variations, that describes the
relative importance of necessary conditions. We need to remember that they all rest
on the basic notions oftrivialness or relevance developed in the preceding section.
I propose that the I T R , c L.measure gives a good overall view of the relative importance of necessary conditions in fuzzy-set methods.
The Importance of Sufficient Conditions
All the principles and techniques discussed above for necessary conditions can be
applied directly to the analysis of the trivialness, relevance, and relative importance
of sufficient conditions. My treatment will therefore be much briefer. A key issue is
to develop our intuitions about what "trivial sufficient conditions" are. It is common to speak of trivial necessary conditions, but as far as I know, no one has analyzed trivial sufficient conditions. Similarly, it seems natural to talk of sufficiency
as the ultimate level of relevance of a necessary condition; in contrast, it is less
obvious to say that necessity is the ultimate relevance value for a sufficient condition.
In Figure 1, X becomes increasingly trivial as the set X~ becomes larger. By the
inverse principle for sufficient conditions, trivialness means that X b e c o m e s more
trivial as X s h r i n k s in size. Figure 5 shows how this works. The sufficient condi-
Studies in Comparative International Development / Summer 2 0 0 6
104
Figure 5
Trivial Sufficient Conditions: A Set Theoretic Perspective
Universe
/fj.t
i
"~'~.
Y=X1/
'"
\
/i/
iX
\
i X2
I
\
5~
I
/
/
\
\
/
\
tions X 1- X s become progressively more trivial because they form a smaller and
smaller subset of Y. Ultimately, the most trivial sufficient condition is one that never
occurs, i.e., the null set. Here we have the first example of the inverse of the necessary condition principle for sufficient conditions: X i s a trivial necessary condition
if it always occurs; X i s a trivial sufficient condition if it never occurs.
While such a sufficient condition may seem to hold no interest, this is not always
the case. For example, if the standard view of the democratic peace is correct, then
we have the sufficient condition hypothesis: "If all nations of the world are democracies then world peace will occur." The democratic peace example suggests that
the empirically empty set of sufficient conditions can be a theoretically important
one, but one that in practice is virtually impossible to attain. Similarly, a common
folk theorem in the game theoretic literature on war proposes that certainty is sufficient for peace (Bueno de Mesquita and Lalman, 1992; Niou, Ordeshook, and Rose,
1989; Wagner, 1993). While governments may never achieve certainty, the theorem
that certainty is sufficient for peace plays a fundamental role in bargaining theories
of war. In short, trivial sufficient conditions are ones that are hard to achieve, and
are the inverse of trivial necessary conditions that are trivial because they are easy
to achieve.
Recall that a maximally relevant necessary condition is one that is sufficient. We
need to invert this notion to arrive at what a relevant sufficient condition is: a maximally relevant sufficient condition is also a necessary condition.
The approach to the relative importance of necessary conditions I have adopted
stems from Ragin's (2000) use of it for sufficient conditions. Situations ofequifinality
by definition mean that there are multiple paths to the same outcome. It is natural to
ask if one path is more common or "important" than another path. Ragin's techniques provide the researcher with a way to determine the set of paths, but do not
provide much in the way of methods to evaluate the relative importance of paths.
105
Goertz
The obvious measure of relative importance is to count how often each path is
taken. For necessary conditions, it was one of the two necessary conditions that
occurred least frequently. Since we are now in the realm of sufficient conditions, it
is the path that is taken most often in practice. If one path is taken five times while
the other 10, then the second path is twice as important as the first.
In short, the logic oftrivialness, relevance, and relative importance for sufficient
conditions mirrors that of necessary conditions. The general principle is that we
must do the converse for sufficient conditions of what we do for necessary ones.
*The Mathematics of the Importance of Sufficient Conditions
The key to transferring the measures of trivialness and relevance from necessary
conditions to sufficient ones lies in inverting lhe set or fuzzy-set relationship between Xand Y. Xis a necessary condition for Y if Y C X o r Y<_Xin terms of fuzzyset membership scores. For sufficient conditions, this becomes Y_DX or Ye X. All the
measures oftrivialness, relevance, and relative nmportance for sufficient conditions
follow from their necessary condition homologues once this substitution is made.
The fuzzy-set measure oftrivialness of a sufficient condition looks analogous to
that for necessary conditions. Trivialness for necessary conditions was when X = 1
Figure 6
Trivialness of Sufficient Conditions: A Fuzzy-Set Perspective
1.o
:r
0.9
~:**
0.8
~'"
0.7
**
0.6
I
0.5
"~:*
0.4
:,
0.3
,
0.2
0.1
0.0
j
I
I
I
I
~
I
0.0 0. l 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
X
Nontrivial sufficient condition: . . . . . .
Trivial sufficient condition: * * * * **
I
1.0
106
Studies in Comparative International Development / Summer 2006
for the whole population. Not surprising, trivial sufficient conditions are those where
X = 0 for all cases. Figure 6 illustrates trivialness for sufficient conditions in a
fuzzy-set framework. I f X i s zero (or close to zero) for all X, then it is easy to fulfill
the fuzzy-set requirement for sufficient conditions that Y2 X.
Figure 6 clearly shows that my measures of trivialness and relevance will play
out in the same manner as for necessary conditions. The equation for trivialness is:
rsu: -
1
N
(5)
Recall that by the definition of sufficiency y, ~ x,. The ratio will always lie between
0 and 1. When X is a trivial sufficient condition, both it and the ratio will be zero.
Relevance is how much the points are near the X = Y diagonal, just as it is for
necessary conditions (see Figure 3). The only difference is that for sufficient conditions the points must be on or above the diagonal, while for necessary conditions
they must be on or below that line.
The fuzzy sets of relevance works in the same manner as relevance for necessary
conditions. In Figure 3, relevant sufficient conditions will be those that hug the X =
Ydiagonal from above, instead of below, as is the case for necessary conditions.
The equation for relevance is:
--
y,)/
x,
In equation (6), when Y= 1, the ratio (1 - y , ) / (1 - x ) is zero. As more cases have
values o f one (or near one), the less relevant the sufficient condition is. As we move
toward the Y= 1 line from the X = Y diagonal the relevance of the sufficient condition decreases.
Figure 7 summarizes the relationships between trivialness and relevance for necessary or sufficient conditions. Trivialness involves changes on the X dimension,
while relevance means changes along Y. For example, trivialness of a necessary
condition means moving toward the X = 1 line, while trivialness for a sufficient
condition implies movement toward the X = 0 one. Analogously, a sufficient condition is increasing nonrelevant as Y increases toward 1.00; a necessary condition is
increasingly nonrelevant as Y moves toward zero.
Conclusions
The issues concerning the importance of necessary or sufficient conditions must be
addressed by those who use fuzzy-set methods as well as by those who more generally make necessary or sufficient condition claims. It is always possible for critics
similar to Downs to say that a given necessary condition is trivial. The concepts and
quantitative measures developed here give a concrete means for responding to such
criticism.
These empirical measures can also inform theoretical debates about the relative
importance of different variables in multivariate explanations. For example, Skocpol
argues that state crisis and peasant insurrection are two necessary conditions for
social revolution. In her analysis, she stresses the state crisis variable, which is
Goertz
i 07
Figure 7
Summarizing Trivialness and Relevance for Necessary or Sufficient Conditions
1.0
l
0.8
SC Irrelevance /
0.6
/
,<
Y
SC Trivialness /
0.4
/
/
./
./
/
/
/
/
/
/
/
/
/
/
/
/
NC Trivialness
>
/
/ /NC Irrele~ ance
/111//
0.2
/
0.0
0.0
I
I
I
I
I
0.2
0.4
0.6
0.8
1.0
A
SC - Sufficient condition
NC - Necessawy condition
]Vote: Arrows indicate increasing tnvlalness or irrelevance
certainly the factor that has drawn the most attention by commentators on her work.
However, it is the case that empirically the peasant insurrection variable is more
important, The measures of relative importance proposed here provide one means
for evaluating these kinds of claims.
More than just giving measures of trivialness and relevance, 1 have presented a
coherent and complete analysis of the concept of a trivial necessary or sufficient
condition. While the concept of a trivial necessary condition belongs to the discourse of social scientists, it has not been subjected to rigorous analysis. As this
article shows, there are really two forms oftrivialness: one that I call trivialness and
the other that I have termed relevance. Both approaches give us useful and crucial
information about the importance of necessary or sufficient conditions. I have shown
in this article how this logic holds uniformly in the three principal contexts where
these issues are likely to arise: ( 1 ) Venn diagrams, (2) 2 x 2 tables, and (3) fuzzy-set
analyses. At the same time, I have developed measures for evaluating quantitatively
the trivialness and relevance of necessary or sufficient conditions for fuzzy-set
methods.
Given the large number of necessary condition hypotheses in the social science
108
Studies in Comparative International Development / S u m m e r 2006
literature (see Goertz, 2003, for 150 examples), the need for the conceptual and
quantitative tools for evaluating the trivialness, relevance, and relative importance
of necessary or sufficient conditions is clear. This article provides one set of answers to these important questions.
Notes
* I would hke to thank Bear Braumoeller, Jim Mahoney, and the editors and reviewers of SCID for
comments on earlier drafts. Special thanks go to Charles Ragm for his many suggestions and
comments, in particular the one that led to the basic measure of trlvialness proposed below.
1. In the equations for trivialness and relevance, two special cases that merit a brief discussion are
when the denominators are zero. For the trivialness measure, the denominator 1 - y is zero when
v - 1. Since we are dealing with necessary conditions, this can only occur when x = 1 (i.e., the
northeast corner point [1,1]). Similarly, for the relevance measure, it is zero when x = 0. Again,
because we are dealing with necessary conditions, this can only occur when y = 0, i.e., at the
origin (0,0). What should be the values of trwialness and relevance in these two cases? The
answer to this question involves recalling that for all cases on the X = Ydiagonal the trwlalness
and the relevance measures are equal to each other. Since the points (0,0) and ( 1,1 ) also he on the
diagonal, we can use the rule that iftrivialness or relevance is undefined due to division by zero,
it is equal to the other measure9 Essentially, th~s means defining these two special cases to have
value 1.
2. This result is supported by two related propositions about the avallabihty ofcounterfactual alternatives: (1) exceptions tend to evoke contrasting normal alternatives, but not vice versa, and (2)
an event is more likely to be undone by altering exceptional rather than routine aspects of the
causal chain that led to it (Kahneman and Miller, 1986: 143).
3. One might suggest using the trivialness cell to assess relative importance instead of relevance
one. In general, this would not be a good idea since the (~X, ~Y) cell is often problematic, see
Mahoney and Goertz (2004) for an extensive discussion.
4. The significance of the concept of relatwe importance extends beyond just necessary conditions
to general interaction terms9 I have often heard that it is impossible to assess the individual
contributions of an interaction term. The concepts and measures of relative importance discussed
here also work for interaction terms.
References
Braumoeller, B. and G. Goertz. 2000. "The Methodology of Necessary Conditions." Amerwan Journal o f Political Science 44: 844-858.
Bueno de Mesquita, B., and D. Lalman. 19929 War and Reason: Domestic and International bnperatives. New Haven, CT: Yale University Press.
Dion, D. 1998. "Evidence and Inference in the Comparative Case Study." Comparative Politics 30:
127-145.
Downs, G. 19899 "The Rational Deterrence Debate." l~brld Politics 41: 225-237.
Goertz, G. 2003a. "The Substantive Importance of Necessary Condition Hypotheses." In Necessa O,
Conditions" Theopy, Methodology. and Applications, eds., Goertz, G. and H. Starr. New York:
Rowman and Littlefield.
9 2003b. International Notwzs and Dectsionmaking: A Punctuated Equihbrium Model. New
York: Rowman and Littlefield.
- - .
2006. Soctal Science Concepts: A User,s" Guide. Princeton: Princeton University Press.
Goertz, G., and J. Levy. 2007. "Causal Explanations, Necessary Conditions, and Case Studies." In
Explaining War and Peace Case Studies and Necessary Condition Colmterfactua&, eds. G.
Goertz and J. Levy. London: Routledge.
Goertz, G. and H. Starr, eds., 2003. Necessary Conditions." Theory, Methodology and Applications.
New York: Rowman and Llttlefield.
Goertz
109
Goldstone, J. 1990. Revolumm and Rebellion m the Early Modern World. Berkeley, CA: University of
California Press.
Goldthorpe, J. 1997. '~Current Issues in Comparative Macrosoclology: A Debate on Methodological
Issues." Comparattve Social Research 16: 1-26.
Goodwin, J. 2001. No Other Way Out: States and Revolutionary Movements, 1945-1991. Cambridge:
Cambridge University Press.
Gruber, L. 2000. Ruhng the W~rld" Pm*'erPolitws and tile Rtse ~)/Suptvnatl~mal hzstitutions. Princeton:
Princeton University Press.
Honor6, T., and H.L.A. Hart. 1985. Causation in the Law 2nd ed. Oxtbrd Oxford Umversity Press.
Kahneman, D., and D. Miller. 1986. "'Norm Theory: Comparing Reahty lo Its Alternatives." Psvehological Review 93:418 439.
Lipset, S. 1959. "Some Social Requisites of Democracy: [ conomlc Development and Pohtlcal Legitimacy." Amerwan Pohtwal Science Review 53: 69-105.
Mahoney, J., and G. Goertz. 2004. The Possibility Principle: "Choosing Negative Cases in Comparative Research." Amertcan Polittcal Science Review ')8: 653-660.
Niou, E., R Ordeshook, and G. Rose. 1989. The Balance ~,! Power. Stability m International Systems.
Cambridge Cambridge University Press.
Ragin, C. 1987. The Comparative Method. Movmg bevomt Quahtative and Quantitative Strategies.
Berkeley, CA: University of Cahfornia Press.
- - .
2000. Fuzzy-Set Social Science. Chicago: Umverslty of Chicago Press.
-. 2006. "Set Relations in Social Research: Evalualmg Their Consistency and Coverage. PohticalAnal)wis" 14:291 310.
Skocpol, T. 1979. States and Social Revolutions: ,4 Comparative Anal3wis o/France, Russta, and
China. Cambridge: Cambridge Umversity Press.
Tsebehs, G. 1990. "Veto Players and Law Production in Parhamentary Democracies: An Empirical
Analysis." American Political Setence Review 93:591-608.
Wagner, R. 1993. "The Causes of Peace.'" In Stopping the l~tllin~: How Civil 14"~lrsEnd, ed., R. Llckhder.
New York: New York Umvcrslty Press.
Walton, J. and Ragm, C. 1~,~r "'Global and National Sources of Pohtlcal Protest: Third World Responses to the Debt ('rJsis.'" Amerwan Sociological Review 55 876-890.