S-2
11/24/09
4:16 PM
Page 43
2
Evaluation Issues in
Peace Implementation
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
GEORGE DOWNS AND STEPHEN JOHN STEDMAN
Compared to the formidable scholarly literature dealing with the problem
of negotiating peace agreements in civil wars, the literature devoted to the
problem of implementing peace agreements is modest. This is somewhat
surprising, as it is painfully clear that the distance between a peace agreement and anything that warrants the term real peace is as large as the distance between the ideals expressed in the most ambitious domestic program
and their concrete realization. Many more peace agreements are signed
than are actually put into practice, and once signed, the stakes are often far
higher.
A large part of the explanation for why the literature on the implementation of peace agreements in civil wars is not more well developed is that
it is difficult to think of an environment that is less conducive to the conduct of evaluation research. The number of cases is small and the measurements of potentially important variables, such as the military capability and
political goals of different parties, are often unreliable. Implementation
strategies tend to be poorly specified and no two are ever identical. Worse,
there are literally dozens of contextual variables—ranging from the amount
of international resources available to the United Nations, to the salience of
a given conflict for international public opinion—that vary widely across
cases and inevitably influence both the choice and the success of an implementation strategy. All of this leaves both the policymaker and the analyst
continually uncertain about what a given strategy actually comprises, what
would have occurred had a strategy not been employed (the counterfactual), and whether the superiority of one strategy over another reflects the
greater merits of the first or the fact that the second strategy had to cope
with a more demanding context.
In the absence of systematic evaluation, however, our understanding of
43
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
44
4:16 PM
Page 44
EVALUATING IMPLEMENTATION STRATEGIES
what does and does not work in peace implementation is prey to any number of fallacies, many of which are brought about by a tendency to separate
strategy and context. What works in Guatemala is expected to work in
Bosnia, even though the cases differ dramatically in most key respects.
Practitioners frequently attribute successful outcomes to their actions and
strategies, and attribute failure to the implementation context. Critics, on
the other hand, tend to focus on incorrect strategies as causes of failure and
attribute success to context or outside factors.
This chapter examines three issues associated with the implementation
of peace agreements in civil wars. First, we describe some of the difficulties that plague estimating the success of individual operations and how the
international system as a whole has performed to date. Second, we conduct
a very modest determinants study in order to see which contextual variables
are associated with implementation success as well as to describe the difficulties associated with such studies. Finally, we discuss the problems associated with determining and improving the relative effectiveness of different implementation strategies. As we shall see, this raises a host of sensitive
issues, many of which are associated with the uncomfortable fact that the
strategy and resources needed to succeed in some cases are unavailable
because no major or regional power sees it in their interest to provide them.
We argue that cases of peace implementation differ dramatically in
terms of the difficulty of the implementation environment and in the willingness of international actors to provide necessary resources and troops.
We find that three of the most important environmental variables in determining failure are (1) the presence of spoilers (factions or leaders who
oppose the peace agreement and use violence to undermine it); (2) neighboring states that oppose the peace agreement and funnel resources to
potential spoilers; and (3) the domestic availability of valuable, easily marketable commodities such as gems or timber. With regard to the importance
of such commodities, no civil war peace agreement has been fully, successfully implemented where these are present. On the willingness side, we find
that the most important variable in accounting for successful implementation is the extent to which a major or regional power defines a particular
case as in its national security interest. Only when such interest is present
has peace implementation succeeded against the opposing efforts of spoilers and hostile neighbors. Evidence suggests that in the absence of such
major or regional power interest, the United Nations can only succeed in
the least difficult environments.
Estimating Effectiveness: Measuring Success
The first step in evaluating institutional effectiveness is to develop some
reliable way to measure implementation success. Traditionally, this prob-
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 45
EVALUATION ISSUES IN PEACE IMPLEMENTATION
45
lem has been approached by developing one or more outcome indicators,
each of which measures the extent to which a goal that is considered central
to the operation’s success has been achieved. In this section we discuss various measures that have been used to evaluate successful peace implementation and describe their potential pitfalls. We begin by focusing on mandate achievement as a measure of success, and then turn to various
objective indicators, such as resolution of the underlying conflict. The evaluation of subgoals such as human rights, elections, disarmament, and
refugee repatriation as measures of implementation success is also discussed.
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
Mandate Achievement
Until relatively recently, the most common indicator of operation success
was the degree to which an operation achieved the goals that were laid out
in the UN’s mandate authorizing the mission. The fact that mandate
accomplishment is defined by the United Nations itself makes it “endogenous.” This endogeneity may seem attractive, as it protects the measure
from criticism that it represents a goal that the agent being evaluated never
intended to accomplish in the first place, but this is actually less of a virtue
than it might first seem because it makes evaluation the prisoner of ambition.
This creates two quite different problems. One is that it allows an
institution to be successful by doing nothing more than lowering its goals
or by setting very modest goals in the first place rather than by accomplishing something substantive. Agencies and institutions that have considerable autonomy with regard to goal setting but are nonetheless operating
in a politically charged environment often do this, particularly when they
believe that failure may impact their funding level or future mission.1 A
common criticism of the Implementation Force (IFOR) in effecting the
Dayton Peace Agreements stems from how the U.S. military limited its
mission to separation of the warring armies in Bosnia, and refused to take
on the more ambitious tasks necessary for creating peace: the capture of
war criminals, the protection of refugees who sought to return home to disenfranchised areas, and the disarmament of the contending armies.2 It
makes little sense to declare IFOR a success when its aspirations are so
low.
While it is sometimes possible for organizations to inflate their success
rate by purposely minimizing performance goals, agencies with little autonomy with respect to goal setting suffer from precisely the opposite problem. The political masters of these agencies use them as scapegoats by
offloading their responsibilities onto the institution in the form of ambitious
goals that the political masters want to get credit for having, but avoid the
blame for not achieving. This has been a recurrent problem for the United
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
46
4:16 PM
Page 46
EVALUATING IMPLEMENTATION STRATEGIES
Nations in the 1990s as the Security Council and member states have
repeatedly demanded that the institution pursue ambitious policies, such as
safe havens in Bosnia, or state- and nation-building in Somalia, without
providing the requisite resources to succeed. The member states and the
Security Council garner symbolic value for proclaiming morally worthy
goals, and are able to blame the organization when it fails to achieve them.
In either of these instances, the evaluator reaches conclusions about the
agency’s success that are uninformative. They provide next to no information about how well the institution performs the tasks that compose its mission relative to other institutions or its own past performance. Perhaps just
as bad, using mandate performance as a measure of success keeps evaluators from asking questions about the appropriateness of the mandate, such
as the extent to which the mandate is sensible given the institution’s
resources—questions that are regularly asked in connection with the performance of economic and military strategy.
Mandate accomplishment as a measure of success also creates comparability problems. We cannot overlook the possibility that a mission that is
only partly successful in fulfilling an ambitious mandate nonetheless may
generate benefits far higher than those brought about by a mission that is
nominally more successful by fulfilling a weak mandate. For example, the
UN Transitional Authority in Cambodia (UNTAC) has come under extensive criticism for failing to achieve all of the political, military, and economic goals of its ambitious mandate.3 But there can be little doubt that
UNTAC accomplished substantially more than did the U.S. mission
Operation Support Hope in Rwanda, which, while described by a former
high-ranking general as a success based on its limited mandate,4 did nothing to prevent or reduce armed hostilities in Eastern Zaire—hostilities that
eventually led to hundreds of thousands of deaths, refugees, and displaced
persons, and an interstate war involving the armies of eight different
African states.
This problem of noncomparability in mandate accomplishment is only
exacerbated by the fact that today’s peace operations vary substantially.
Peace implementation can include a wide array of tasks. UN peacekeepers
have been asked to disarm and demobilize warring troops and repatriate
refugees in Cambodia, Mozambique, Guatemala, and Angola; reform legal
institutions in El Salvador and Bosnia; investigate human rights abuses in
Cambodia, Angola, and El Salvador; hold elections in El Salvador,
Mozambique, Namibia, Cambodia, and elsewhere; and oversee truth commissions in El Salvador and Guatemala. In ongoing civil wars where the
consent of the warring parties for intervention has been absent—Somalia,
Liberia, and Sierra Leone—soldiers have been asked to enforce peace and
create the conditions under which peace implementation can go forward.
Among the other important limitations of mandate fulfillment as a
measure of success is the fact that mandates are often strategically ambigu-
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 47
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
47
ous and that this calculated ambiguity increases with the complexity of the
operation. Michael Doyle, Ian Johnstone, and Robert Orr argue that experience has taught the military planners connected with large, multidimensional peacekeeping operations to anticipate that the mission will unfold in
ways that were not anticipated when the decision to undertake the mission
was made: parties will renege on their agreements, the tasks of relocation
or disarmament will prove more difficult to achieve, and so forth.5 In order
to reserve the capacity to flexibly adapt to these changing conditions, planners want mandates to be more ambiguous than they might be in a simpler,
more traditional mission. This is entirely sensible from the planner’s standpoint, but it makes it much more difficult for the researcher to confidently
gauge the degree to which a given mission achieves its mandate.
Finally, mandates for peace implementation missions in the 1990s have
suffered from inflation of goals—goals that have been included in the mandate less for realistic calculation of possible accomplishment, than as a way
of appeasing nongovernmental organizations and domestic public opinion.
As the 1990s progressed, the lobbying of NGOs, especially those involved
in human rights, pressured member states to include more and more subgoals in UN peace operation mandates. The inclusion of such goals as
accountability for war crimes, de-mining, stopping the use of child soldiers,
improving the plight of women in society, and protecting civilians has
become a kind of humanitarian “pork,” with each NGO striving to ensure
that its single issue is included in operation mandates. But given the large
number of such goals in most peace implementation mandates, and given
the limited resources to achieve such goals, implementers inevitably must
make decisions about which subgoals are most important to their overriding task of implementing a peace agreement. Nor are all good things
achievable in the short term of peace implementation, a problem we will
return to shortly in discussing objective indicators of success.
Other Measures of Success
The limitations of mandate accomplishment as the sole measure of peace
implementation success have led researchers to supplement it with other
indicators. Marjorie Brown suggests two others: whether or not the operation led to a resolution of the underlying disputes of the conflict, and
whether or not the operation reduced conflict in the area.6 Paul Diehl looks
at whether the operation limited armed conflict and the extent to which the
operation facilitated conflict resolution.7 Duane Bratt uses four indicators
of success: mandate performance; the degree of conflict resolution; whether
or not the operation kept conflict from spreading in the region; and the
extent to which the operation reduced both military and civilian casualties
below the rates that existed before the operation began.8 All of these measures of success possess the virtue of being less susceptible than mission
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
48
4:16 PM
Page 48
EVALUATING IMPLEMENTATION STRATEGIES
mandate to the problem of endogeneity. They prevent a mission from being
declared a success simply because its mandate required very little, or from
being declared a failure because it failed to achieve overly ambitious goals,
even if it produced significant benefits. The new measures rely less on
ambition and more on the kind of tangible accomplishments that dominate
estimates of success in most other fields. If a conflict widens and increasing
numbers of lives are lost or, conversely, if three democratic elections take
place in succession over five years, it will be reflected in the level of success that these measures assign to an operation.
Of course, no outcome measure is without its limitations, and these
newer and more objective indicators of success have their own problems.
One of these has to do with the decision about how the indicators are combined into a scale. This can be done by assuming that the indicators are
equally important or by weighing them in importance. The decision is up to
the analyst, and the fact that different analysts will decide the issue differently means that estimates about the overall success or failure of a given
program are likely to vary more than they would in the case of an individual measure of success such as mandate accomplishment. In addition, a
given multiple indicator has its own internal comparability problems in that
although two separate missions are assigned the same score, it is based on
the achievement of different things. Thus, Bratt, for example, classifies an
operation as being partially successful if it achieves any one of three different objectives.9 This raises the potential problem that two partial successes
may be as unalike as they are alike, especially with respect to the resources
and strategies that were necessary to achieve partial success in the two
instances.
Another difficulty involves the number and breadth of indicators that
are included in the multiple measure. Currently, there is an understandable
tendency to include indicators that the analyst believes best characterize the
successful elimination of factors that resulted in conflict in the first place.
For example, a study carried out by DFI for the Pentagon requires that a
successful peace operation lead to economic stabilization as well as to the
installation of a second democratically elected government.10
There is no doubt that a peace operation that produced economic stabilization and two democratically elected governments would be more successful than one that simply produced a ceasefire. It is also true that one
that produced economic stabilization, a growth rate of 5 percent or better,
and three actively contested democratic elections would be better still. By
this logic, it follows that the best measure of success and the one that will
lead to the most nuanced understanding of what different peacekeeping
missions have accomplished is the one that contains the most complete list
of conditions believed to prevent the outbreak of conflict and improve the
quality of life in a state torn apart by civil war.
Yet this is almost certainly not the case. One reason lies in the fact that
the passage of time is the enemy of inference. The passage of time intro-
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 49
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
49
duces the equivalent of noise in the form of a growing number of influencing factors that are not directly associated with a peace mission. The more
time that passes from the conclusion of a peace mission, the more likely it
is that any number of other extraneous factors (e.g., business cycles,
famines, unusually good or bad weather, the policies of a neighboring state,
the behavior of the first elected leaders) are actually responsible for what
has taken place rather than the technology of the peace mission itself. As
the potential impact of such exogenous factors increases, the quality of our
inferences about the contribution of the peace operation itself tends to fall
until the inferences reach the point where they break down completely. This
is one of the reasons why elementary school educators tend to measure
teacher and school performance by indicators such as student reading and
math scores rather than by student success in getting into the college of
their choice, getting a good job, or having a happy marriage. It is not that
the educators value a fourth-grade reading score more than they do the
impact of a high-quality university education or professional and personal
success; they just believe that any correlation between the quality of fourthgrade instruction and these outcomes is likely to be more spurious than
real.
Some performance indicators, especially those suggested by scholars
of peacebuilding, run the further danger of setting such high standards of
success that we are left with a world of undifferentiated failure. Exemplary
of this pitfall is the work of Roland Paris, who argues that one should judge
the success of UN peace operations in the 1990s by the standard of “creating conditions that will allow peace to endure long after the peacekeepers
have left” and asserts that by this standard only one operation—Namibia—
can be judged a success. 11 Correspondingly, the list of failures then
includes not only unmitigated disasters such as Rwanda and Angola, but
also operations where war has ended, such as Mozambique, El Salvador,
Bosnia, and Nicaragua. By failing to discriminate between catastrophic
failures such as Rwanda and Angola, where hundreds of thousands die, and
flawed successes such as El Salvador, Cambodia, and Mozambique, where
war has ended but citizens face the mundane reality of life in povertyridden insecurity, Paris holds evaluation hostage to an unreasonable standard of success and ensures that it will yield very little information that can
be used to improve future missions. It is not that we should avoid striving
to attain good things like economic growth, equitable development, and
good governance; it is that these goals form a useless standard for evaluating implementation actions that take place in a short period of time.
Subgoal Achievement
Given the problematic character of summary evaluations, analysts engaged
in the task of assessing peace implementation often change their focus from
overall mission objectives to mission subgoals such as demobilization,
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
50
4:16 PM
Page 50
EVALUATING IMPLEMENTATION STRATEGIES
human rights, and refugee repatriation because they are more manageable.
They require less information because they represent only one small subroutine in a much larger program. They also promise more robustness
because there are fewer dimensions that have to be matched to another
case. That is, it is frequently more plausible to argue that the task of repatriation in two given states was more similar than was the overall task of
peace implementation.
There is a limit to the extent precision is possible, of course. The
processes connected with subgoals like human rights, demobilization, or
repatriation are never exactly the same in any two cases, and an insistence
that every detail of two such processes be the same in every case would
doom any attempt to formulate generalizations before it got off the ground.
Perhaps a bigger danger lies in the fact that different subgoals interact and
success in one area can affect success in another area. The importance of
this interaction will be missed when analyzing subgoals in isolation. This
can lead to any number of mistaken inferences. For example, Human
Rights Watch not too surprisingly asserts that the failure to protect and further human rights in Angola best explains the UN’s failure to implement
peace there.12 But as Tonya Putnam demonstrates in Chapter 9, the protection and furtherance of human rights during peace implementation rest
completely on the ability of the implementers to persuade the warring parties to demobilize their soldiers and turn their armies into political parties.
In the absence of progress in those subgoals, implementers have little
chance to achieve higher standards of human rights.
Moreover, much of what we have said above about mandate achievement applies to evaluation of subgoals. Again, to use the human rights
example, the ability of peacekeepers to successfully pursue human rights
during implementation differs greatly according to the conflict environment. Pursuing human rights in Somalia, for example, where there was no
existing rule of law, no civil society, and no functioning state, and where
spoilers sought to destroy the peace agreement, was much more difficult
than pursuing human rights in El Salvador, where the two warring parties
agreed to make reform of judiciary and police institutions a key aspect of
the peace agreement.
Operationalizing Success and Failure
In this project we chose to operationalize mission success by scoring two
variables: (1) whether large-scale violence is brought to an end while the
implementers are present; and (2) whether the war is terminated on a selfenforcing basis so that the implementers can go home without fear of the
war rekindling. We assessed each case of peace implementation by these
measures and then further subjected the cases to an informal, counterfactual
analysis about the specific contribution of the implementers to the result. If
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 51
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
51
the peacekeepers could not bring large-scale violence to a close, as in
Rwanda, Angola, Somalia, or Sri Lanka, the mission was coded as a failure.
If the peacekeepers brought the war to a close, but could not leave for fear
of the war restarting, as in Bosnia, we coded the case as a partial success. If
the peacekeepers brought the war to a close and departed without the war
restarting in a two-year period, as in El Salvador, Mozambique, Guatemala,
and Nicaragua, we coded the case as a success.
We then went further to ask what would have occurred in the program’s absence. It is the difference between this score and what occurred in
the presence of the program that represents a mission’s impact. It makes no
sense to credit outside implementers with achieving something that would
have happened anyway or to blame them for being ineffective when without their efforts the situation would have become far worse. In the absence
of an estimate of the counterfactual, the researcher is implicitly assuming
that it was the status quo at the time the mission was initiated: everything
that happened after that is assumed to have been caused by the implementers.
Because the task of estimating the counterfactual in an area where both
theory and data are scarce is so difficult, analysts in new fields such as
peace implementation tend to omit any reference to it, partly on the implicit
grounds that any such estimate would be arbitrary and thus have little credibility anyway. This makes sense given the circumstances, but it risks introducing a significant upward bias in our estimate of how successful a given
institution has been in addressing the problem assigned to it. To understand
why, one must keep in mind that most large-scale institutional interventions
take place in contexts where there is some sort of crisis, such as a recession
or epidemic. In the absence of intervention some of these crises will grow
increasingly worse (for example, the recession will turn into a depression),
but more often than not the situation will eventually get better by itself or
“regress to the mean” and the atypical state of affairs is replaced by the typical.
Does the lack of counterfactual analysis lead us to credit peace operations too much or to denigrate them unnecessarily? An examination of the
sixteen peace operations in this study led us to alter our initial evaluations
of two cases. In the case of Liberia, which would have been coded as a success based on the criterion of ending the war on a self-sustaining basis, we
downgraded it to a partial success on the basis of evidence that the actions
of the implementers actually delayed resolution of the war. In the case of
Cambodia, which would have been coded as a failure based on the criterion
of having ended the violence, we upgraded it to a partial success because
the country was much more secure when the peace operation was terminated, even though violence continued at low levels. Finally, a case could also
be made that in three of the cases, Nicaragua, El Salvador, and Guatemala,
the wars would have ended at some point in the 1990s without UN involve-
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
52
4:16 PM
Page 52
EVALUATING IMPLEMENTATION STRATEGIES
ment in the implementation of the peace agreements; at best, the role of
outside implementers was to hasten the end of wars whose termination was
ordained by a changed international and regional context. In the end, we
rejected reclassifying the cases as less than successes because we felt that
by quickening what may have come about in any case, outsiders contributed to saving thousands of lives and helped stabilize the agreements.
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
Evaluating Overall Institutional Effectiveness
Evaluation of how the United Nations and/or the international system as a
whole have performed to date in peace implementation also suffers from
vexing problems. For people who are not very interested in program evaluation, this may not seem too troubling. Can’t one simply pick the set of outcome indicators that one believes is the most informative, measure the
extent to which success has been achieved in each implementation mission
that has been undertaken, add them up and calculate the average performance measure on each indicator, and then either take the average score over
all the indicators or weigh them as one thinks best? The answer is that this
is a good place to start, but a potentially dangerous and poor place to stop
because it tells us less about the overall success that the UN and the international system have had than one might first think and may be of little predictive value.
Evaluation of institutional effectiveness must consider that some missions are more difficult and costly to conduct than others, and the UN is
likely to have a lower success rate in these. For example, implementing
peace accords in Cambodia, where the fighters grudgingly signed a peace
agreement and had over 120,000 men-at-arms, was much more difficult and
expensive than implementing peace accords in Guatemala, where the guerrillas were small in number (estimated between 1,500 and 3,000) and were
substantially defeated. Ending the civil war in Somalia, where the state had
collapsed, militias had proliferated, and none had consented to intervention,
was a far more difficult mission than bringing peace to El Salvador, where
the warring parties supported UN participation in implementing a settlement that emerged after several years of negotiations and no factions used
violence to undermine the peace.
The fact that the success rate will vary with the mix of variously difficult missions that are undertaken and that this mix has been changing over
time means that any simple estimate of trends in the institution’s average
effectiveness will be fundamentally ambiguous. If that estimate shows that
the UN has become less effective over time, it may mean that the organization is not functioning as well as it once was, or it could just as easily mean
that the organization is functioning better but the level of mission difficulty
has increased at an even faster rate. Variance in the difficulty of missions
causes the same problems when it comes to forecasting the likely success
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 53
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
53
of a mission that the UN is undertaking. Unless the future mission is neither more nor less difficult than the average mission, the forecast will be
too high or too low.
The connection between mission type and mission difficulty also
means that the UN’s overall success rate may depend more on which operations it chooses to undertake than on the strategic decisions it makes about
how to conduct its operations. In itself this fact is unremarkable; the history
of warfare makes it clear that whom one chooses to fight is at least as
important as how one conducts the subsequent war. One should be cautious
about interpreting data about trends in UN peacekeeping effectiveness as
measured by mission accomplishment unless the nature of the mission is
controlled for—something that is difficult to do and rarely attempted.
The impact of variation in the difficulty of missions that the UN or the
international system chooses to undertake on its overall success rate raises
the important related question of how we should deal with peace implementation opportunities that have been selected out by the UN; that is,
cases that it has chosen to pass up for whatever reason (e.g., Sudan or
Chechnya). This is an important issue that can make a dramatic difference
in our estimate of overall success, but the boundaries of what is often
referred to as the potential treatment population (i.e., the number used in
the denominator of the fraction that represents the program’s success rate)
are not always obvious and depend on the use to which the statistic will be
put. For example, if we are trying to estimate the effectiveness or success
of an employment program, does it make more sense to measure it by the
percentage of people graduating from the program who have found jobs,
the percentage of those entering the program who have found jobs, or the
percentage of those eligible for the program who have participated, graduated, and found jobs? The people who operate the program would always
prefer that the first measure be used because the smaller base rate will yield
a higher percentage or rate of success. If the program is a pilot project
being used to determine how well a certain type of program is likely to
work if it is widely employed, the sponsors of those funding the project
would find the second statistic the most appropriate because program
dropouts represent program failures. If the program is a national one
charged with the task of providing the unemployed with job training, voters
would probably feel that the third statistic is the one that is important
because the program is supposed to serve all of the unemployed, not just
some of them.
In the case of peace implementation, the standard approach is to look
at the success rate of the operations that are undertaken, but the justification for adopting this approach seems largely one of convenience. It implicitly assumes that it makes sense to equate the mission of the United Nations
with whatever it chooses to do, and that intervention opportunities that are
declined have been declined for good reason. But it is likely that there are
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
54
4:16 PM
Page 54
EVALUATING IMPLEMENTATION STRATEGIES
systematic selection effects at work. Research by Michael Gilligan and
Stephen Stedman indicates that the United Nations intervenes much less in
civil wars in Asia than it does in other regions, and that it avoids civil wars
where the government possesses a large army.13 The UN Security Council
will never authorize an operation in one of the permanent five member
states of the Security Council; civil wars such as Chechnya will never be
addressed. The Security Council is loath to authorize peace missions when
there is an absence of consent by the warring parties. It is unlikely that the
Security Council would intervene in a civil war in a regional power such as
India without the consent of that power.
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
In Search of the Determinants of Success
Having devised outcome measures that they understand even if they do not
trust them completely, evaluators usually turn to the task of trying to identify contextual factors that explain the variation in institutional performance
that they reveal. This is considerably more difficult than devising outcome
measures, for any number of reasons: the number of cases is small and difficult to handle statistically; the measurement of potentially important variables, such as extent of state authority and the extent to which parties are
genuinely committed to finding a peaceful solution to their conflict, is often
difficult and unreliable; and interaction and selection effects create any
number of interpretation problems.
With respect to interaction, the impact of a given factor is likely to vary
across types of missions. To use an obvious example, the commitment of
resources that are likely to be present when one or more major powers are
interested in having a conflict resolved is likely to be much more important
in cases where the implementers attempt to enforce rather than monitor
compliance. If we decide not to worry about this and plunge ahead and
obtain a regression estimate of the impact of major power interest, that estimate will be determined by the mix of consensual and enforcement missions that has accumulated to that point. Any estimate of the average effect
of major power interest will be systematically too high or too low depending on the nature of the mission. To come close to an accurate prediction we
would need to either estimate the impact of major power interest separately
on consensual and enforcement missions or have a term in our predictive
equation that represented the interaction between major power interest and
mission complexity. The second technique has the advantage of conserving
more degrees of freedom, but in a field where the number of cases is so
small, any attempt to statistically estimate interaction is problematic.
The related but more elusive problem of selection arises from the fact
that peacekeeping choices are the residue of strategies. For example, suppose the United States and its major European allies have a policy of play-
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 55
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
55
ing a major military role in the implementation of peacekeeping only in
cases where they believe their participation is absolutely essential in order
to achieve success. If one is unaware of this fact or ignores it and looks at
cases of peace implementation in the 1990s, one would see that those missions where the United States has played a preponderant military role—
Bosnia and Somalia—have registered a failure and a partial success.
Conversely, one would also see that U.S. military participation has been
absent in the clear successes—Mozambique, El Salvador, Nicaragua, and
Guatemala. The temptation is to conclude that the participation of the
United States in peace operations is either irrelevant or detrimental to success when in fact its participation may have been essential to whatever was
achieved in the most difficult cases and might have dramatically improved
the results in the less difficult cases that were only marginal successes.
In a study such as this one, where the sample is small and composed
only of peace implementation missions that were undertaken, theory is too
undeveloped for us to analyze the data against the backdrop of a reliable
selection equation. The only way to deal with interaction and selection
effects is to be extremely cautious in interpreting the findings and try to
think about the effects that we might be missing. With that cautionary note
in mind, we conducted a rough analysis of the determinants of the successful implementation of peace agreements in the sixteen cases included in the
CISAC-IPA study.
The general literature on policy implementation provides crucial
insight into the conditions under which a peace agreement might succeed or
fail. A policy implementation perspective requires that we pay attention to
the environment surrounding implementation and recognize that some
environments are more conducive to implementation than others. Such a
perspective also looks to the coalitions that support implementation and
their willingness to invest resources. As a first cut, therefore, our project
ascertained what makes some peace implementation environments more
difficult and challenging than others. Our second cut was to determine
which cases get the most resources.
Based on findings from the scholarly literature on peacemaking in civil
war, we created a difficulty score for peace implementation based on the
following eight variables: the number of warring parties; the lack of either
a peace agreement before intervention or a coerced peace agreement; the
likelihood of spoilers; a collapsed state; the number of soldiers; the presence of disposable natural resources; the presence of hostile neighboring
states or networks; and demands for secession.
• The number of warring parties. The difficulty of implementation
increases when there are more than two warring parties. 14 Strategies
become less predictable, balances of power become more tenuous, and
alliances become more fluid. In Cambodia, for example, any action that the
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
56
4:16 PM
Page 56
EVALUATING IMPLEMENTATION STRATEGIES
United Nations might have taken against the Khmer Rouge had to be
weighed against the effects such action would have had on the National
United Front for a Cooperative, Independent, Neutral, and Peaceful
Cambodia (FUNCINPEC), which relied on the Khmer Rouge to balance
against the State of Cambodia.15 In cases where a proliferation of warring
parties occurred, as in Somalia and Liberia, implementers constantly found
it difficult to craft solutions that would address the concerns of all the warring factions. Where any factions found themselves excluded, the peace
agreement faced their violent opposition.
• The absence of a peace agreement signed by all major warring parties before intervention and with a minimum of coercion. The United
Nations has usually required a detailed peace agreement among the warring
parties—as a sign of their consent to a peace mission—as a precondition
for UN involvement. In the 1990s, however, the United Nations intervened
in many ongoing wars and, in several instances, either it or a regional
organization or a state intervened in the hope of using force to compel a
peace agreement: the UN in Somalia, the Economic Community of West
African States Cease-Fire Monitoring Group (ECOMOG) in Liberia and
Sierra Leone, India in Sri Lanka, the North Atlantic Treaty Organization
(NATO) in Bosnia, and Syria in Lebanon. Intervention in the absence of a
peace agreement likely will trigger violent opposition by parties who value
the preintervention status quo. The absence of a peace agreement implies a
lack of problem solving and trust- and confidence-building among the warring factions, thus producing a more difficult implementation environment.
• The likelihood of spoilers. The presence of spoilers in peace agreements poses daunting challenges to implementation.16 One critique of the
spoiler concept, however, is that spoilers are only recognized after the fact.
This criticism can be addressed by attempting to gauge whether prospective
implementers judged that they were likely to face violent challenges during
implementation. A more sophisticated criticism of the spoiler concept is
that potential spoilers are always present and whether an actor actually
engages in spoiling behavior depends on the existence of a special opportunity structure.17 There is, as we shall see, some evidence that this is at least
partially the case.
• Collapsed state. The lack of state institutions and governing capacity
places great demands on peace implementers. In addition to bringing fighting to a close, the implementers must create and build up a modicum of
state capacity in order for the peace to have a chance to sustain itself.
• Number of soldiers. At some level, numbers matter. High numbers of
soldiers pose greater demands for verification and monitoring and, hence, a
greater potential for successful cheating. Moreover, greater numbers of soldiers require more personnel for monitoring and more resources for demobilization. We scored cases where there were more than 50,000 soldiers as
being difficult to implement.
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 57
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
57
• Disposable natural resources. If warring parties have access to disposable resources such as gems, minerals, or timber, implementation
becomes more difficult. Such resources not only provide armies with a
means for continued fighting, they also become the reward against which
they weigh the benefits of peace.18 A key difference between Mozambique
and Angola is that, in the latter country, the Union for the Total
Independence of Angola (UNITA) had access to diamonds, which emboldened its spoiler behavior, whereas the Mozambique National Resistance
(Renamo) lacked access to such resources, which effectively limited the
benefits of returning to war.19
• Hostile neighboring states or networks. Civil wars rarely take place
in otherwise stable regions. As Peter Wallensteen and Margareta Sollenberg
observe, many civil wars today intersect with regional conflicts and interstate competition.20 From this it would follow that the attitude of the surrounding states toward a peace agreement in a neighbor’s civil war plays a
key role in supporting or undermining the prospects of peace. Spoilers to a
peace agreement, for example, are likely to be much stronger and more
vocal if they are confident that they can count on neighboring states for
sanctuary, guns, fuel, and capital.21 Likewise, in regions where weak states
have little control over borders, well-organized private or semiofficial networks can allow neighboring states to take advantage of such state decrepitude to support spoilers in the war-torn country.
• Wars of secession. There is a plausible argument that negotiated settlements are more difficult to attain and implement where civil wars are
fought over national sovereignty.22 Such conflicts often revert to all-ornothing struggles that make the job of would-be implementers more difficult than in cases where warring parties share a common identity and at
least agree on a unitary future for their country.
The more these indicators are present, the greater the difficulty of
bringing the conflict to an end. The conflict environment, however, is only
one aspect of implementation success or failure. International willingness is
also crucial; low degrees of interest and commitment lead either to no intervention or, alternatively, to an intervention with an extremely limited strategy set in the sense that implementers will be constrained by the resources
they can deploy and the range of subgoals they can pursue. Constraints on
the strategy set need not be a problem when the implementation environment is easy. But difficult environments and constrained strategy sets can
be a recipe for disaster, for example, when the Security Council authorizes
a mission to implement a peace agreement in Somalia or Angola, but does
not provide adequate resources.
We scored our cases on three indicators of interest and commitment,
what we refer to as a “willingness score”: major or regional power interest,
commitment of resources, and acceptance of risk of casualties to soldiers.
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
58
4:16 PM
Page 58
EVALUATING IMPLEMENTATION STRATEGIES
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
• Major or regional power interest. A key sign of a high commitment
to a mission is whether large, powerful states support intervention and publicly define the conflict as important to their own vital security interests.
The more remote a mission is from a powerful state’s vital security interests, the more likely it is being undertaken for symbolic reasons that are
unlikely to inspire the outlay of more than a very modest amount of
resources. We created a simple estimate of security interest based on
whether or not a civil war was geographically proximate to a major or
regional power and whether there were statements by major or regional
powers at the time of intervention that enunciated security, as opposed to
humanitarian, motivations for intervention.
• Resource commitment. Crucial for successful implementation is the
willingness of states to provide adequate financial resources for a mission.
But such resources vary by case, and this is often known before a mission
even begins. In Cambodia, for example, the Security Council provided
extreme leeway to planners to judge what resources were needed for a successful mission. In Rwanda, on the other hand, the Security Council rejected the figures provided by its own field mission, and instead imposed
extreme limits on resources available.
• Acceptance of risk to soldiers. We scored this separately from financial resource commitment because policymakers evaluate the risk to their
own troops differently than they evaluate financial resources.
Table 2.1 lists the cases in our study, the principal implementer, the
conflict difficulty score, the international willingness score, and the outcome. The higher the difficulty score, the more difficult the implementation
environment; the higher the willingness score, the greater the willingness
of international actors to commit to the effort.
If we just look at the bivariate regressions of implementation outcome
on each of these variables, we find that only four variables are significant at
the 0.01 level. These are the existence of a spoiler, the presence of disposable resources, the presence of a neighboring state that is hostile to the
peace agreement, and the presence of major power interest. Consistent with
expectations, the presence of the first three factors reduces the chances of
successful implementation, and the last improves it. The summary variables, conflict and willingness, together explain about 65 percent of the
variance in mission outcome. All of the other variables have an impact that
is in the predicted direction but not significant with the small number of
cases.
Since we were curious about the extent to which the role of spoiler was
itself an endogenous strategy, the selection of which partly depends on the
availability of financial resources, we also looked at the extent to which the
presence of spoilers can be predicted by the presence of disposable
resources and/or the presence of a hostile neighboring state. The prevalence
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 59
59
EVALUATION ISSUES IN PEACE IMPLEMENTATION
Table 2.1
Peace Implementation Cases
Zimbabwe
Sri Lanka
Namibia
Nicaragua
Lebanon
Liberia
Angola I
Cambodia
Mozambique
El Salvador
Somalia
Rwanda
Angola II
Bosnia
Guatemala
Sierra Leone
Time
Period
Principal
Implementer
Difficulty
Score
Willingness
Score
Outcome
1980
1987–1989
1989–1990
1989–1990
1990–
1990–1998
1991–1993
1991–1994
1992–1995
1992–1994
1993
1993–1994
1994–1999
1995–
1996–1997
1996–1998
Great Britain
India
UN
UN
Syria
ECOWAS
UN
UN
UN
UN
UN
UN
UN
NATO
UN
ECOWAS
4
6
0
1
5
6
4
5
3
1
5
3
4
6
0
6
1.0
1.7
1.7
1.5
2.7
2.1
0.4
2.2
1.2
1.5
1.4
0.4
0.9
2.2
1.5
0.7
Success
Failure
Success
Success
Partial success
Partial success
Failure
Partial success
Success
Success
Failure
Failure
Failure
Partial success
Success
Failure
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
Notes: Difficulty score: 0 = lowest, 8 = highest. Willingness score: 0 = lowest, 3 = highest.
of spoilers makes this a little like reading tea leaves, but the presence of
disposable resources appears slightly related, and the presence of a hostile
neighbor more strongly so. In every case where there is a hostile neighbor,
there is a spoiler, and in the six cases where there is no hostile neighbor,
spoilers appear only twice. Of course, one cannot overlook the possibility
that instead of a hostile neighboring state providing the resources that allow
a spoiler to survive, the existence of a spoiler gives a potentially hostile
neighboring state reason to believe that it can effectively subvert the peace
agreement.
Figure 2.1 illustrates the intersection of conflict difficulty, the degree
of major or regional power interest, and implementation outcome. The
cases of Guatemala, El Salvador, Namibia, and Nicaragua all scored as low
on difficulty of implementation contexts and high on great or regional
power interest. Not surprisingly, all were successful. Mozambique and
Zimbabwe fell in the low-to-medium range of implementation difficulty
and, although their interest scores were not as high as the cases mentioned
above, their peace agreements were successfully implemented.
To the right of the graph are the more difficult cases. Where major or
regional power interest was low, as in Angola, Somalia, and Rwanda, or
high, as in Sri Lanka, implementation failed. In the most difficult cases,
substantial major or regional power interest was enough to compensate for
the difficulty of environment to produce partial success in Cambodia,
Lebanon, Bosnia, and Liberia, but was not enough to avoid failure in Sierra
Leone.
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 60
60
EVALUATING IMPLEMENTATION STRATEGIES
Figure 2.1
Interest and Difficulty: Case Outcomes
Nicaragua
El Salvador
1 ●
●
Guatemala
Cambodia
Lebanon Bosnia
■
■
0.8
Namibia
Interest score
●
Mozambique
●
●
Zimbabwe
0.6
Liberia
■
▲
Sierra
Leone
▲
Sri Lanka
0.4
Rwanda
▲
0.2
▲
Angola I
Angola II
Somalia
▲
0
0
2
4
6
8
Difficulty score
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
● success
■ partial success
▲ failure
In terms of interaction, it is interesting to note that both cases where
the level of conflict was high (i.e., where the aggregate conflict scores were
5 or 6), and where little major power interest was manifest, were total failures, while only one of the five high-conflict cases where major power
interest was present was a total failure. Major or regional power interest
may not be able to guarantee complete success in difficult environments,
but it is usually able to guarantee at least partial success. More unsettling
from the standpoint of the UN as an international body and the evolution
away from realpolitik is the fact that in the absence of such interest, there is
no evidence of even partial success.
One might expect that the role of major powers on the Security
Council acts to regularly select out difficult missions in preference to easier
and less costly ones. Not much can be said about this in the absence of
knowing what proportion of the total number of potential peacekeeping
opportunities fall into the challenging category (and knowing whether there
are other attributes that seem likely to contribute to the difficulty of a case
but are entirely “selected out”), but it is nonetheless the case that almost
half of the cases in the sample are classified as difficult.
If major powers don’t seem to go out of their way to avoid highconflict cases, they do appear to be slightly more discriminating with
respect to which cases are undertaken. The average major power interest in
high-conflict cases is 0.72 as opposed to 0.58 for lower-conflict cases, and
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 61
EVALUATION ISSUES IN PEACE IMPLEMENTATION
61
these figures underestimate the extent of discrimination since four of the
low-conflict cases are in Central America, where the United States could be
expected to be interested in any case. If these are eliminated, the major
power interest score of the low-conflict cases drops to 0.38.
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
Strategies and Implementation Success
The majority of peacekeeping researchers and practitioners are less interested in the broad issue of the determinants of successful implementation
than they are in the comparative effectiveness of different peacekeeping
strategies. Unfortunately, the comparative evaluation of strategies is even
more problematic than are determinants studies. One reason for this is
that the term strategy can refer to virtually anything, from the UN’s decisions about which missions to undertake, to its decisions about what
goals to pursue, to its decisions about how to pursue them. Moreover,
while strategies often share much in common, they invariably differ in
any number of details. This leaves the evaluator uncertain about what it is
that should be evaluated as well as what has been evaluated, and creates a
lot of confusion about what the conclusions of a given study actually
mean.
Another reason why strategic evaluation tends to be more problematic
is that selection and interaction effects are often ignored or dealt with poorly. In many ways this makes little sense. The operation of selection and
interaction is usually far more obvious in the case of strategy choice than it
is in the operation of a given determinant. At an abstract level researchers
and policymakers know that strategies are usually tailored to fit a specific
context rather than the result of historical serendipity, and that when comparing strategies this makes it difficult to sort out the impact of context and
strategy. They are well aware of the fact that education researchers worry
that the success rates of top-rated schools or special experimental programs
may have more to do with the background of the children who gravitate to
good school districts or enroll in voluntary experimental programs than
with the quality of the schools or programs themselves. Peacekeeping
researchers know they must also worry about whether the greater success
rate of one particular strategy stems from its greater superiority or the fact
that the missions in which it was employed were easier than those where
other strategies were used. Too often, however, individual psychology,
organizational interest, and strategic misrepresentation lead policymakers
and researchers to forget this.
In the case of peace implementation the tendency is to look at the correlation between a grand strategy of confidence-building (reminiscent of
traditional peacekeeping) and success in places like El Salvador,
Guatemala, Mozambique, and Namibia, and the correlation between a
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
62
4:16 PM
Page 62
EVALUATING IMPLEMENTATION STRATEGIES
grand strategy of coercion (compellence or deterrence) and failure in places
like Sri Lanka, Somalia, and Sierra Leone and only partial successes in
Lebanon and Bosnia, and therefore to assume that confidence-building is a
superior strategy to coercion. This, of course, ignores that the cases represent radically different implementation contexts, ignores the deadly failures
of confidence-building in Rwanda and Angola, and fails to ask the counterfactual—whether confidence-building would have led to stable peace in
Somalia or Sri Lanka. To put it differently, evaluations of peacekeeping in
the 1990s often suffered from a belief that the UN should always employ
small numbers of lightly armed troops on the basis that the rate of mission
failure is correlated with troop strength and the heaviness of armor.
There are exceptions, of course, but most of them are relatively recent.
For example, both Doyle and Stedman make the normative case that choice
of strategy should be determined by the difficulty of the case and the threat
posed by spoilers.23 In Doyle’s work, as the environment of peace implementation becomes more difficult, the amount of transitional authority
exerted by implementers and their willingness to act coercively must
increase. In Stedman’s work, when implementers are faced with violent
challenges by powerful leaders or factions who are unalterably opposed to
a specific peace agreement or whose demands on the peace process are too
high, implementers will have to choose coercive strategies to prevail.
Confidence-building is an inappropriate strategy against ethnic extremists
who are willing to commit genocide, as in Rwanda, in order to undermine a
peace agreement.
Nonetheless, a skeleton account of international effectiveness in peace
implementation in the 1990s demonstrates a reluctance to connect strategy
and context when thinking about the effectiveness of strategies. What
jumps out of the data is the fact that the UN’s earliest successes in peace
implementation occurred in the least difficult cases: Namibia (1989),
Nicaragua (1989), and El Salvador (1990–1994). Each of these cases
scored low in conflict difficulty and moderate to high in international willingness. Possibly because there was so little variance in peacekeeping context, lessons that were learned from these cases tended to emphasize the
effectiveness of one universally applicable strategy. As a consequence,
peacekeepers emerged with the belief that traditional peacekeeping—
impartiality, neutrality, and consent—and its theoretical underpinning of
confidence-building could be successfully applied to any case or mandate.
This overconfidence in the ability of the United Nations to implement
peace agreements led to overcommitment. Between 1991 and 1994, the
United Nations authorized implementation missions to Angola, Cambodia,
Guatemala, Mozambique, Rwanda, and Somalia, devoting little apparent
thought to the extent to which these cases resembled those where the earlier
successes took place. In short order, and with the exceptions of Guatemala
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 63
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
63
and Mozambique, the United Nations found itself in much more dangerous
situations, using inappropriate strategies without adequate resources.
In Somalia, the one case where the United Nations followed a coercive
strategy, the institution interpreted its experience as a failure of strategy—
that coercion is incompatible with peace implementation. Just as it ignored
context in the case of success, it ignored context in the case of failure. By
the end of 1993 the UN updated its selection of strategies based on their
relative success—that is, it employed more frequently those strategies that
experience had shown to be the most successful. Because the strategy associated with the most difficult case—coercion—had the lowest success rate,
the UN found itself applying strategies suited to relatively easy missions in
every case, with the effect of reducing the organization’s overall success
rate. It was as if a hospital suddenly decided to suspend bone marrow treatments on the grounds that the patients who received them had a higher
mortality rate than those who did not without bothering to figure out that
part of the justification for giving a patient a very expensive and debilitating cure might be the fact that the patient is much sicker than the average
person with the disease.
While it is possible to attribute part of the UN’s overconfidence in the
traditional peacekeeping model to its string of early successes, the decision
to ignore the great contextual differences between Somalia and the other
peacekeeping failures on the one hand and the early successes on the other
likely stemmed from causes other than a lack of experience or analytic
bewilderment. Two other closely related factors are likely to have played a
role: the norms of UN decisionmakers and the nature of the UN organizational routines that they create. With respect to their personal value system,
individuals are often attracted to a career at the UN in the first place
because they esteem neutrality, nonpartisanship, and roles of mediator and
peacemaker. They do not want to be viewed as, or view themselves as, part
of a partisan militia allied against one side. As a consequence, they have a
motivated bias in favor of traditional peacekeeping methods that makes
them reluctant to acknowledge the limitations of their preferred intervention technology or the virtues of methods that they abhor and that invariably limit their operational autonomy.
Strategic misrepresentation also seems to be operating in some of these
cases. As much as UN decisionmakers are drawn to the traditional peacekeeping process, they are even more drawn to ending conflict and playing a
central role in conflict intervention wherever it takes place. Both of these
goals were placed in jeopardy by the failures of the 1990s, and the UN
responded to this crisis in ways that were not always productive. As a consequence of the mounting number of peacekeeping failures and the UN’s
failure to protect safe havens in Eastern Bosnia in 1995, leading to the
slaughter of 7,000 Muslim men and boys, the United States made it a point
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
64
4:16 PM
Page 64
EVALUATING IMPLEMENTATION STRATEGIES
to exclude the UN from any substantive role in implementing the Dayton
Accords in Bosnia. Regional intervention—along the lines of Nigeria and
ECOMOG in Liberia and Sierra Leone—was advocated as a replacement
for UN missions. As the 1990s were coming to a close, a new SecretaryGeneral, Kofi Annan, fought against what he perceived to be the marginalization of the United Nations in peace operations, and the Secretariat
fought for inclusion in some of the most controversial operations of the
decade: Kosovo, East Timor, Sierra Leone, and the Democratic Republic of
Congo (DRC)—all extremely difficult operations that posed major challenges.
One can see this as an attempt to ensure that the United Nations would
regain a central role in peace implementation, but the fact that at least three
of these operations are unlikely to have ever received the attention of any
major or regional power without the efforts of the Secretary-General suggests that another motivation is at work. In cases such as East Timor, Sierra
Leone, and the DRC, many strategic options that are theoretically available
to the UN are not “incentive compatible,” in the sense that they are policies
that are not in the self-interest of critical actors to implement. In more conventional language, they are not politically feasible. The fact that great
and/or regional powers must play a key role in implementation creates
potential incompatibilities between the strategies that the UN might like to
see employed and the incentives of the major powers to support those
strategies.
As the difficulty of the implementation environment increases, there is
a need for greater scope and assertiveness of the transitional authority that
is supplied by international actors. Similarly, the greater the difficulty of
the implementation environment, the greater the need for coercive strategies of implementation. This means that the strategies that are available to
international implementers are a function of great and regional power interest: selection acts to truncate the strategy set. When great or regional powers do not see a particular case as affecting their security interests, they do
not provide the resources and commitment necessary for coercive strategies.
Incentive compatibility not only affects choice of grand strategy, but
also the availability of mechanisms that are needed to provide strategic
coherence and coordination. As Bruce Jones argues in Chapter 4, the more
difficult the implementation environment, the greater the need for strategic
coordination. When international actors suffer from a lack of unity or do
not fully support an operation, would-be spoilers can take advantage of
international splits to attack the peace process and threaten would-be
peacemakers. Similarly, the more coercive the strategy, the greater the
civil-military tensions in implementation.24 When international troops are
at risk, troop-donating states tend to insist on greater day-to-day interfer-
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 65
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
EVALUATION ISSUES IN PEACE IMPLEMENTATION
65
ence in the political conduct of the mission. Such interference can defeat
the ability of an implementation force to diagnose and respond effectively
to spoilers.
As Jones points out, strategies such as “Friends of the SecretaryGeneral” are fundamentally a function of great and regional power interest.
The willingness of states to join a “Friends” groups is indicative of a prior
judgment that the specific case is in the state’s interest. The result is that
such mechanisms for coordination will be unavailable in some cases: there
were no Friends of Somalia, as there were no great or regional powers who
had security interests in a peaceful Somalia.
Incentive compatibility and the gap between what is needed in some
missions and what the great or regional powers are willing to provide has
led to organizational pathologies within the United Nations that, unless
addressed, may lead to the end of UN involvement in making peace. The
Secretariat seems to have learned that there are some cases where the
Security Council will only authorize a mission if it perceives that the case
will be safe and easy. Where threats or dangers appear, the reaction of the
Security Council is to avoid deployment altogether or to cut and run. Faced
with such knowledge as early as Rwanda, UN bureaucrats became reluctant
to share worrisome conflict assessments and tended only to “ask for what
the traffic will bear.”25 This led to UN intervention in cases where, if anything went wrong, the UN would fail. The need to present optimistic scenarios to the Security Council precluded contingency planning, since the
basic premise of such planning is that less-than-optimistic scenarios are
possible. In some cases the need for the UN to present a successful face to
its political masters led missions such as Angola in 1997 and Sierra Leone
in 1999 to ignore blatant noncompliance by the warring parties. At its
worst, as in East Timor in 1999, the UN Secretariat engaged in reckless
gambling by selling the Security Council on a mission by downplaying its
difficulty, in the hopes that if it went bad, the Security Council and the
member states would be embarrassed into response.
Conclusion
On the basis of the findings of this chapter, we recommend the following:
• When selecting what peace agreements the UN should implement,
objective assessments must be made of the degree of difficulty of the case,
the appropriate strategy that is necessary for success, and the likelihood of
major or regional powers supplying the appropriate resources.
• When selecting what peace agreements the UN should implement,
major power or regional power interest should be treated as a hard con-
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
66
4:16 PM
Page 66
EVALUATING IMPLEMENTATION STRATEGIES
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
straint. This means that without major or regional power interest, the
United Nations should be extremely cautious in trying to implement cases
where spoilers are present, neighboring states are hostile, and disposable
resources are available. Experience suggests that UN intervention can make
a bad situation on the ground even worse, and failure may both jeopardize
the UN’s role in peacekeeping and increase the severity of incentive compatibility problems.
• Given the importance of judgment about difficulty of implementation environment, there is a need at the United Nations for better strategic
assessment concerning possible peace implementation missions. The
Secretariat must act more like Consumer Reports and less like used-car
salesmen, attempting to sell the Security Council every mission that
arises.
• Given the importance of the role that spoilers play in implementation
failure, there is a need for intelligence gathering and assessment concerning
the motives, intentions, and capabilities of parties who sign peace agreements and parties who are omitted from peace agreements.
• Given that such intelligence is fallible and that there will be missions
where unanticipated violent challenges may erupt during implementation,
there is a need for the United Nations to improve its contingency planning
for peace operations.
• Before attempting to implement a peace agreement in a country
where there are easily marketed valuable commodities (spoils) or that is
adjacent to a state hostile to the peace process, the implementer should
have the strategy, resources, and commitment to regulate these commodities and the inflow of assistance to spoilers.
There is no easy answer to any of the connections between the UN’s
success rate and the difficulty of the potential missions that arise.
Regardless of how we categorize missions, some are more difficult and
costly to conduct than others and the UN is likely to have a lower success
rate in connection with these even with selection operating. It follows that
if the UN wants to increase its success rate, the easiest way to do so will be
to focus on cases where there are only two parties, a functioning state
exists, the numbers of combatants are low, and there are no expected spoilers.
Is this a sensible strategy? On the one hand, the answer seems obviously no. It seems like the equivalent of a hospital choosing to turn away cancer patients and people in need of transplants in favor of orthopedic procedures and gall bladder operations. On the other hand, if the UN is under
constant financial pressure and there are situations where the probability of
success is virtually nonexistent, there is no more to be gained by ignoring
these facts in the context of peacekeeping than there is in the context of
healthcare.
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
Appendix 2.1
Coding of Cases by Difficulty and Willingness Variables
Difficulty
Bosnia
Liberia
Sierra Leone
Sri Lanka
Lebanon
Cambodia
Somalia
Angola I
Angola II
Zimbabwe
Rwanda
Mozambique
El Salvador
Nicaragua
Guatemala
Namibia
50,000
soldiers
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
Likely
Hostile
spoilers neighbors
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
More
than
two Disposable Weak Collapsed
Difficulty Power Resource
Risk
Interest commitment lives
parties resources agreement state
Secession scoreb
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
6
6
6
6
5
5
5
4
4
4
3
3
1
1
0
0
Notes: a. Outcomes: 0 = failure, 1 = partial success, 2 = success.
b. The higher the conflict score, the higher the level of difficulty.
c. The higher the willingness score, the higher the willingness of international actors to end the conflict.
1.0
0.7
0.7
0.5
1.0
1.0
0.2
0.2
0.2
0.5
0.2
0.5
1.0
1.0
1.0
0.7
1.0
0.7
0.7
0.5
0.7
1.0
1.0
0.2
0.7
0.5
0.2
0.7
0.5
0.5
0.5
1.0
0.2
0.7
0.7
0.7
1.0
0.2
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Page 67
Civil War
Outcomea
Willingness
Willingness
scorec
2.2
2.1
2.1
1.7
2.7
2.2
1.4
0.4
0.9
1.0
0.4
1.2
1.5
1.5
1.5
1.7
4:16 PM
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
1
1
0
0
1
1
0
0
0
2
0
2
2
2
2
2
S-2
11/24/09
4:16 PM
68
Page 68
EVALUATING IMPLEMENTATION STRATEGIES
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
Notes
1. George Downs and Patrick Larkey, The Search for Government Efficiency:
From Hubris to Helplessness (Philadelphia: Temple University Press, 1986).
2. See Chapter 18 on Bosnia by Elizabeth Cousens in this book.
3. See Chapter 17 on Cambodia by Sorpong Peou in this book.
4. George A. Joulwan and Christopher C. Shoemaker, Civilian-Military
Cooperation in the Prevention of Deadly Conflict: Implementing Agreements in
Bosnia and Beyond (New York: Carnegie Corporation of New York, 1998), pp. 19,
23–24.
5. Michael W. Doyle, Ian Johnstone, and Robert C. Orr, “Strategies for
Peace: Conclusions and Lessons,” in Doyle, Johnstone, and Orr, eds., Keeping the
Peace: Multidimensional UN Operations in Cambodia and El Salvador
(Cambridge: Cambridge University Press, 1997), pp. 388–389.
6. Marjorie Ann Brown, “United Nations Peacekeeping: Historical Overview
and Current Issues,” report for Congress (Washington, D.C.: Congressional
Research Service, 1993).
7. Paul Diehl, International Peacekeeping (Baltimore: Johns Hopkins
University Press, 1993).
8. Duane Bratt, “Assessing Peacekeeping Success,” in Michael Pugh, ed.,
The UN, Peace, and Force (New York: Frank Cass, 1997), pp. 64–81.
9. Ibid.
10. Barry M. Blechman, William J. Durch, Wendy Eaton, and Julie Werbel,
Effective Transitions from Peace Operations to Sustainable Peace: Final Report
(Washington, D.C.: DFI International, September 1997).
11. Roland Paris, “The Perils of Liberal International Peacebuilding,”
International Security 22, no. 2 (fall 1997), pp. 54–89.
12. Human Rights Watch, Angola Unravels: The Rise and Fall of the Lusaka
Peace Process (New York: Human Rights Watch, 1999).
13. Michael Gilligan and Stephen John Stedman, “Where Do the Peacekeepers
Go?” paper prepared for the World Bank, October 29, 2001.
14. Gerardo L. Munck and Chetan Kumar, “Civil Conflicts and the Conditions
for Successful International Intervention: A Comparative Study of Cambodia and El
Salvador,” Review of International Studies 21, no. 2 (1995): 159–181; and Michael
W. Doyle and Nicholas Sambanis, “International Peacebuilding: A Theoretical and
Quantitative Analysis,” American Political Science Review 94, no. 4 (December
2000): 779–801.
15. See Chapter 17 on Cambodia by Sorpong Peou in this book; and Stephen
John Stedman, “Spoiler Problems in Peace Processes,” International Security 22,
no. 2 (fall 1997): 32–36.
16. Stedman, “Spoiler Problems.”
17. Paul Collier and Anke Hoeffler have made a similar argument in connection with the factors that inspire rebellion in the first place. See Greed and
Grievance in Civil War, Policy Research Working Paper no. 2355 (Washington,
D.C.: World Bank, Development Research Group, May 2000).
18. There is a burgeoning literature on the political economy of civil wars and
its effect on prospects for war termination. See, for example, Mats Berdal and
David Malone, eds., Greed and Grievance: Economic Agendas in Civil Wars
(Boulder, Colo.: Lynne Rienner, 2000).
19. Stedman, “Spoiler Problems,” pp. 47–48.
20. Peter Wallensteen and Margareta Sollenberg, “Armed Conflicts, Conflict
Termination, and Peace Agreements, 1989–96,” Journal of Peace Research 34, no.
3 (fall 1997): 339–358.
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
S-2
11/24/09
4:16 PM
Page 69
EVALUATION ISSUES IN PEACE IMPLEMENTATION
69
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
21. Stedman, “Spoiler Problems,” p. 51.
22. Chaim Kaufmann, “Possible and Impossible Solutions to Ethnic Civil
Wars,” International Security 20, no. 4 (spring 1996): 136–175.
23. See Chapter 3 by Michael Doyle in this book; and Stedman, “Spoiler
Problems.”
24. Karen Guttieri, “Civil-Military Relations in Peace Implementation,” draft
prepared for this project.
25. See Chapter 16 by Gilbert Khadiagala in this book.
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.
11/24/09
4:16 PM
Page 70
Copyright © 2002. Lynne Rienner Publishers. All rights reserved.
S-2
Ending Civil Wars : The Implementation of Peace Agreements, edited by Stephen John Stedman, et al., Lynne Rienner
Publishers, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=6191482.
Created from columbia on 2023-02-13 00:49:27.