Public Sector Reforms and Evaluation: Trajectories and Trends.
An International Overview
Hellmut Wollmann
in: Hellmut Wollmann (ed.),
Evaluating Public Sector Reforms: An International and Comparative Perspective,
Special Issue of Revista Internacional de Estudios Politicos,
September 2001 (ISSN 1516- 5973), pp. 11-40
1.
The ambitious agenda of this special issue of IJPS
This special issue of the International Journal of Political Studies (IJPS) has two ambitious
(interrelated) aims.
First, in addressing the current state of public sector reforms in the international context the
publication aims at covering a wider geographical range than most of the available literature
which has largely concentrated on New Public Management (insert empty space) reforms in
the Anglo-Saxon and also Scandinavian countries (with the important exception of
Pollitt/Bouckaert 2000 who have included France and Germany in their comparative analysis
of as many as ten countries). In the country selection represented in this volume the reader
will, not surprisingly,find the “usual suspects” of the international debate (USA, Australia,
New Zealand, U.K., Sweden, Norway, Denmark, Netherlands), but also countries and regions
hitherto less in the lime light or even ignored, such as Germany, Switzerland and Spain (as
Continental European cases - with France being unfortunately left out), Japan and Latin
America (with a focus on Brazil and Chile). Adding up to some 15 countries under inspection
in this volume hopefully some some new comparative insights will be gained into the trajectories and patterns of public sector reform on a broader regional basis of countries and of
“families” of countries.
Second, and more importantly, the articles of this volume are mandated to dwell on the question as to whether, when and how evaluation as an analytical procedure to monitor and assess
the process and results of public sector reforms and measures has been institutionalized in the
countries concerned. With some few exceptions (such as the seminal piece by Pollitt 1995),
neither the specific conceptual and methodological problems which are posed by evaluating
public sector reforms nor the state of the art (in terms of stock-taking of the available pertinent evaluation research) has so far been taken up explicitly and in some depth.1 As these issues have thus so far largely remained conceptually, methodologically and empirically unmapped territory our special issue may truly claim to bear some novelty and innovation, hopefully breaking some ground for further discussions in the relevant arenas.
The purpose of this introductory piece is to identify some argumentative threads running
through the articles of the volume and to make an attempt to systematize some of the observa-
tions and findings spelt out in the contributions. Following the two-staged topic of the special
issue our introduction will come in two sections, first on public sector reform policy and then
on the evaluation thereof .
2.
Trajectories of public sector reforms – an international overview
2.1. New Public Management (NPM) reforms and “traditional” reforms
In the following we assume that, in empirically studying a country’s course (trajectory) of
public sector reform, a distinction should be made between the reform concepts and components which directly derive from NPM (in the narrow sense), on the one hand, and between
other reform concepts and elements which are rooted in other (we shall call them: “traditional”) reform strands, on the other hand. If such a distinction is not made and “traditional” reform measures are indiscriminately claimed for, and included in NPM (which is the case in
much of NPM-devoted literature), the analytical picture becomes unduly blurred and also misleading.

NPM is generally recognized as being a bundle of concepts (Pollitt speaks somewhat ironically of a “shopping basket”, Pollitt 1995) which is not rooted in one consistent theory,
but exhibits a (tension-fraught) “hybrid” character in that it combines economic organization theory and management theory, the former setting on contractualism (with inherent
centralizing tendencies) and latter on managerialism (“let managers manage” with decentralizing and devolutionary tendencies) (see e.g. Christensen/Lagreid 2001: 19 with further references). Besides catchwords, such “lean state”, privatizing, downsizing, if not
minimizing, deregulation, that hail from neo-liberal policies in which NPM has been politically embedded, the important keywords of “genuine” NPM are: competition, marketization, outsourcing, contracts, private sector management techniques, agencification and the
like.

By contrast, traditional public sector reform concepts can be identified as essentially rooted in the idea of changing the political and administrative structures on their political and
democratic premises. Such reform concepts were originally contrived and pursued in earlier reform periods. They encompass government/cabinet reform (e.g. strengthening the
executive leadership of the prime minister), introduction of planning and evaluation pro-
1
Reference may be made, at this point, of a recent conceptual etc. discussion on public sector reform evaluation in Switzerland, see Bussmann et al. 1997, Ritz 1999, and in Germany see Wollmann 2000a.
cedures, political decentralization and administrative deconcentration, democratization,
citizen rights, citizen participation.
2.2. The country profiles of public sector reform in a comparative ranking
In drawing on the empirical evidence provided by the country reports in this volume, in using
the categorization submitted and applied by Pollitt/ Bouckaert (in this volume and in
Pollitt/Bouckaert 2000: 93 ff., with the distinction made between maintaining, modernizing,
marketizing and minimizing patterns of public sector modernization) and in employing the
differentiation just suggested between (“genuine”) NPM concepts and “traditional” ones, the
attempt shall be made to highlight some of the relevant country profiles as follows:

“In general terms the ‘Anglo-Saxon’ countries (Australia, New Zealand, UK, USA) have
gone further and faster down the roads of marketization and minimizing than have most
continental European countries” (Pollitt/Bouckaert in this volume), whereby New Zealand
has generated the theoretically “purest” formulation of NPM doctrine (see Halligan in this
volume). The UK and New Zealand, being unitary and centralist countries, have made
strong use of “agencification” (unlike Australia and the US which, being federal, have
hardly employed it). The four Anglo-Saxon countries are rated by Pollitt/Bouckaert as, by
and large, “marketizing and minimizing”.

Among the Scandinavian countries Sweden presents herself also as a reform-active country, whereby a strong strand of traditional decentralization policy (strengthening the
“classical” multi-functional model of local self-government) (see Premfors 1998) goes
hand in hand with a significant dose of NPM dose (particularly in the field of performance
management and output measurement) (see Christensen et al. in this volume). Pollitt/Bouckaert (insert empty space) rate Sweden as a “modernizing” case (see this volume). By
contrast, Norway has manifested comparatively limited public sector reform activities
holding largely on to her traditional administrative structures and mores (see Christensen
et al. in this volume). Pollitt/Bouckaert who did not include Norway in their sample
would probably rank the country as falling between the maintaining and modernizing category.

Germany is an ambivalent case of public sector reforms (Schröter/Wollmann 1997). On
the one hand, the country has a strong tradition of traditional reforms and institutional
adaptation in its multi-level and multi-actor system resulting in a reform pattern of
“fragmented incrementalism” (see Derlien 1996, Wollmann 1996a, 1997, 2001a,
Benz/Goetz 1996). On the other hand, it has, since the early 1990s, become a late-comer
to NPM reforms with the local government level taking a lead (see Wollmann and Jann/
Reichard in this volume). While Pollitt/Bouckaert rate the German trajectory as still
largely exhibiting the maintaining pattern, it might be seen, since the recent inroad of
NPM, as moving towards becoming a (late) modernizer.

Since the late 1970s – after the elapse of the Franco regime and the country’s return to
democracy – Spain’s institutional trajectory has, at first, been essentially shaped by its political decentralization (in creating autonomous regions and introducing local selfgovernment) as the major institutional lever for democratization. It is only quite recently
(since 1997) that, apparently not least in response to the reform pressures coming from
European integration, comprehensive NPM-guided reform measures have been tackled
(see Osuna/Velez in this volume). Hence, Spain may be ranked as a (very late) modernizer.

Japan is another late-comer to NPM-inspired reforms. Only since 1995 comprehensive
reforms which include traditional reform elements, such as cabinet reform, decentralization and strengthening of local self-government, as well as NPM-specific components,
particularly extended agencification, have been inaugurated (see Muramatsu/Matsunami
in this volume)2.

In most Latin America countries, since their transition to democracy in the early 1980s,
their institutional development has been primarily marked by political decentralization (be
it by “federalizing” the regional level, be it by strengthening local self-government) as an
instrument of democratization (see Bresser-Pareira in this volume). In Brazil, as the most
advanced case of public sector reforms in the region, at the same time civil service and
administrative reforms were undertaken which were largely oriented to the, as it were,
“classical” civil service and (“Weberian”) bureaucracy patterns meant to overcome the
patrimonialism, clientelism and corruption of traditional administrative structures and mores. Until recently, NPM-guided modernization measures were explicitly rejected (or
postponed) on the assumption and understanding that they should be taken up only after
the “classical” reform measures had been put in place. At the same time, such NPM“attentism” was stipulated and justified also by international experts, including the World
Bank, under the concept of “sequencing”. It is only recently (since 1995) that Brazil has
pronouncedly turned to NPM-inspired public sector reforms, just as the international consultants community, including the World Bank, has conceptually changed gear in abandoning their earlier “sequencing” belief and now setting on the NPM doctrine.
2.3. Explaining the cross-country variance of public sector reform trajectories
In the recent study of public sector reforms, as it is impressively evidenced also in the contributions to this volume, a great deal of conceptual and empirical efforts has been devoted not
only to identify the very profiles of the countries’ trajectories of public sector modernization,
but also to recognize the factors (methodologically speaking: the “independent variables”)
which account for and “explain” the varied profiles (as “dependent variable”). Because of
their theoretical implications and their richness of empirically founded insights these „explanatory“ efforts shall also be taken up in the context of this introductory piece, although necessarily again only in a sketchy manner.
From the recent comparative work on the development of public sector reforms mention
should be made, because of their conceptual and empirical merits, first of all, of the analytical
frames and explanatory schemes that have been submitted and employed by Pollitt and
Bouckaert (see Pollitt/Bouckaert 2000 and in this volume) and by Christensen, Laegreid and
Wise (see Christensen/Laegreid ed. 1991, 1991 with an explication of their “transformative”
approach, Christensen et al. in this volume). Notwithstanding some conceptual and terminological differences, both teams have focused on the socio-economic context, on the institutional and cultural tradition and on the institutional setting (polity) as the three sets of variables that presumably influence the country’s trajectory most strongly. Yet, reference should
also be made of the copious body of conceptual and empirical knowledge which has been
accumulated, since the early 1990s, in the social science research conducted, under the rubric
“transformation research”, on the huge institutional change in the post-socialist countries in
Central Eastern Europe, including East Germany (see Wollmann 1996c), which after all was a
process of unprecedented public sector restructuring and modernization. In the analytical
frame and explanatory schemes guiding that research on institutional transformation the institutional and cultural “legacies” of the past loomed large as presumed determinants of the cur-
2
Nota bene: In their country report the authors lump the bundle of intended reform measures together under
the NPM label.
rent and future institutional development, with the concept of “path-dependency” being given
prominent attention3.
In largely conforming with the above mentioned analytical schemes, while partly deviating
from them in the selection or grouping of relevant factors, subsequently we submit an explanatory scheme in which the following factors are taken into consideration:

The country’s socio-economic context, particularly the budgetary situation.

The country’s institutional and cultural tradition and “path-dependencies” (such as
Rechtsstaat versus “public interest”, Roman Law versus Common Law, statist versus
“state-less”, managerialist traditions etc.).

The country’s institutional setting (polity) (such as federal versus unitary, multi-level versus one-level, centralist versus decentralized, but also majoritarian versus consociational
etc.).

The constellation of relevant political and socio-economic actors (political beliefs, strategic interests, but also “will and skill” of key actors).

The discourse coalition, reform coalition, “advocacy coalition” (possibly made up of
academics, think tanks, consultants, practitioners etc.) involved in the preparation and implementation of reforms.

The starting conditions, that is, the very institutional etc. state of the public sector from
which the reform is going to depart and to “take off” (such as its expenditure size4, its personnel size5, its institutional centrality6, the degree of public sector involvement in the delivery of social and health services, the state of the public service etc.).
In the following the attempt shall be made to “test” the explanatory power of these factors in
drawing on the empirical evidence picked from the articles of this volume as well as from
related work. In conceptual terms, certain crucial dimensions of the process and state of public sector reforms (as “dependent variable”) will sequentially be singled out and it will be
asked, in turn, which factors (as “independent variable”) can plausibly explain it. As a caveat
3
The concept of path dependency was first applied to the (institutional) transition and transformation in Latin
American and former socialist Central East European countries, particularly by Karl and Schmitter 1991. For
an application to the comparative study of local government development in post-socialist countries see
Wollmann 1997b, 2000d.
4
As indicated, for instance, by the volume of governmental expenditures as percentage of GDP.
5
As indicated by the number of government employees as percentage of the total employment.
6
As indicated, for instance, by the number of central government personnel as percentage of the total public
sector personnel.
it should to warned, however, that these remarks can hardly be anything but exploratory at
this point.
2.4. The “take off” of public sector reforms
The “take-off” decision taken in the countries to embark upon (large-scale) public sector reforms has in most cases been triggered, no doubt, by an economic and budgetary crisis which
instigated the political decision-makers, regardless of political party complexion, to seek remedy in (far-ranging) public sector reforms, largely of the NPM persuasion.

Probably the most striking case in point can be found in New Zealand where in 1984 it
was the ruling Labour government that, in view of a dismal economic and budgetary situation, decided to abandon its previous pro-active welfare state policy and to turn to a downright neo-liberal policy, including comprehensive NPM-inspired public sector reforms.

In Australia, too, it was in view of a deep economic crisis that the governing Labour Party
that in 1983 inaugurated a far-reaching public sector reforms (meant at first, however, to
strengthen the control of political government over public administration).

Also in Sweden dramatic economic and budgetary developments in 1982 and in the early
1990s triggered comprehensive public sector reforms mainly engineered by the ruling Social Democrats (except for the short-lived conservative-liberal (“ bourgeois”) government
intermezzo during 1991-1994).

Germany, being a conspicuous late-comer to NPM, has finally turned to NPM-inspired
public sector reforms since the early 1990s, following German Unification, when the public debts, due to the continuous enormous financial burdens from Unification, kept skyrocketing. The Social-democrat-led federal government that took office in fall 1998 has
stepped up the public sector reform drive.

Japan, too, being another (very) late-comer to NPM, has, at last, embarked upon farreaching NPM-inspired public sector reforms after 1996 when the budgetary plight following the burst of the country’s “bubble” economy ever worsened.

The USA makes a deviant case in that the country has engaged in public sector policies,
although it found itself in a relatively comfortable economic and budgetary situation. In
this case, a powerful “discourse and reform coalition” and the reform-prone managerialist
tradition of the country were probably instrumental (see below).

By contrast, Norway corroborates the “crisis thesis”, as the country is economically and
budgetarily well-off due to its off-shore oil riches and hence has not seen much cause to
embark upon major public sector reforms.
2.5. Scope and intensity of public sector reforms
The scope and intensity of the public sector reform package decided upon and put in place in
a country has been significantly determined by the very starting conditions of that country.

The fact that New Zealand (along with the U.K.) has gone furthest in pursuing minimizing
and marketizing strategies (see Pollitt /Bouckaert in this volume), including a heavy dose
of privatization, outsourcing and agencification , can be plausibly explained, first of all, by
the institutional deficits, if not modernization “lags” which marked the starting conditions
of New Zealand (as well as of Great Britain) in terms of an oversized state-run economic
sector (as a result of previous nationalization), over-unitary and over-centralist government structures and a quasi-monopoly of the public sector in the delivery of social and
health services. To a considerable extent, the wide range of the NPM modernization bundle and its neo-liberal zeal can be seen as a political and conceptual reaction to these, as it
were, outdated starting conditions.

Sweden’s starting conditions appear somewhat contradictory. On the one hand, although
being a unitary state, the country has been traditionally characterized by a (by international standards unusually, if not uniquely) high degree of decentralization (evidenced by the
all but minimal personnel size of the central government level and the “multi-purpose”
local self-government model), and, on the other hand, by a quasi-monopoly of the public
sector in the delivery of social and health services. Since the early 1980s Sweden’s public
sector reforms have moved essentially in two directions. For one, the country has been
even further decentralized by functionally and politically strengthening its traditional multi-function local self-government. Second, in obvious response to the hitherto prevalent
quasi-monopoly of public sector “production” of social and health service the country has
embarked upon NPM-guided reforms (such as performance management and marketization).

Germany’s starting conditions were marked by its federal decentralized structures, a
strong (multi-)functional and political role of local self-government and the predominance
of ngo’s, that is non-public non-profit welfare organizations, in the delivery of the social
services (with primary health care being provided entirely by “private” medical doctors).
This should explain why, well into the 1980s, little need was seen to take to NPM reforms
at least with regard to agencification, outsourcing and the like. After Germany finally
joined the international NPM bandwagon in the early 1990s, the reform debate has been
mainly directed at the traditional (Weberian) legal rule-bound hierarchical bureaucracy
model as well as the traditional civil service system.

Japan’s very recent move to tackle public sector reforms can, to a significant extent, be
interpreted as catching up with delayed traditional reforms, such as reorganizing central
government and decentralization. At the same time, the creation of NPM-type “agencies”
can be seen as a response to an over-centralization of the Japanese central government
ministries.
2.6. Institutional (level) comprehensiveness and decision-style of public sector reforms
Both the institutional comprehensiveness and the decision and implementation speed of public
sector reforms significantly depends on the institutional setting (polity) of the country, whereby the unitary/federal and multi-level/one level divide seems crucial.

In unitary countries the national parliament and government, that is, the ruling political
majority, have the power to decide public sector reform policies and measures, in principle, for all political and administrative levels of the country. Once the decision is made by
the central level actors, implementation can, to a large extent, be centrally enforced.
Hence, the public sector reform activities are bound to take a “top down” direction.

In federal countries the picture is quite different. In federal countries the political and legislative jurisdiction is largely restricted to federal governmental institutions and personnel
(which in the case of Germany is less than 10 percent and in the case of the USA some 15
percent of the total number of public employees, as compared to 87 percent in New Zealand). By contrast, the federal government level has no legislative jurisdiction over the
States/Länder and local government levels in most administrative reform matters. Under
these institutional conditions, public sector reform may evolve “bottom up” rather than
“top down” . The ensuing multi-level and multi-actor system has, in the German case, resulted in the local government often taking the reform lead and in an overall and longterm reform pattern of “fragmented incrementalism”.
2.7. Conceptual and theoretical profile of public sector reforms
The conceptual focus and theoretical underpinning of public sector reform may be significantly influenced by the “discourse coalition”, “reform coalition” (Wagner/Wittrock/Wollmann
1991) or “advocacy coalition” (Sabatier 1988) involved in the formulation and implementation of such policies and often establishing an important link with the international discourse
community.

New Zealand has been identified as the country with the most pronounced “theoretical”
underpinning of its NPM policy, such as public choice and principal-agent theory. This
“theoretical” NPM profile can be seen as emanating from the discourse and reform coalition which was made up of a small group of Labour Party leaders, business leaders, neoliberal university economists and like-minded officials of the Treasury who formed an almost “secretive elite” (Halligan 2001: 85) in masterminding the reform strategy and its
theoretical underpinning.

In the USA the discourse on public sector modernization has evolved and has been conducted with a more checkered profile which was shaped by economic organization theory
(public choice and the like) as well as by the managerialist school of thought, while it was
overshadowed and dramatized by guru-type bestsellers, such as the “Reinventing Government” book by Osborne/Gaebler 1992. The discourse and reform coalition can be seen
made up mainly of conservative think tanks, consultants and business leaders.

Reflecting what has been called the country’s “consensualist” policy-style Sweden has a
long tradition of establishing “commissions” consisting of academics, business leaders,
trade unionists and government officials for the preparation of legislative drafts and policy
decisions (see Vedung 1992: 72 ff.). Thus, the public sector reform projects, too, have
been accompanied by a broad public discourse in which three separate discourse communities (“traditionalists”, “decentralists” and “economists”) have been distinguished, each
affiliated with particular ministries and socio-economic groups (see Premfors 1991, 1998:
150). The discourse pluralism has probably been conducive to the compromise-oriented
and pragmatic stance of the Swedish reform debate.

In Germany the discourse on public sector reform has experienced a conspicuous shift and
rift since the 1990s. Whereas well into the 1980s the discourse and reform coalition was
largely carried by law trained academics, like-minded practitioners and, to some extent,
political scientists, constituting what can be called “traditional modernizers”, since the
early 1990s a discourse and reform coalition of “NPM-modernizers” has become dominant consisting of economists, NPM-sympathetic academics and consultants (see Wollmann 1996b: 22 ff., Schröter/Wollmann 1997, Derlien 2000: 153).
3.
Receptiveness to, or reluctance towards, NPM-reforms
A country‘s receptiveness to, or reluctance towards, adopting the NPM reform message and
postulates is probably determined largely by its institutional and cultural traditions.

The concept of applying private sector-derived managerialism to the public sector has
found easy acceptance in the USA for reasons which are deeply ingrained in the country’s
history and culture. First, the idea of the State as self-standing entity and of a cognitive,
legal and institutional divide between the State and Society has been rejected by American
political thinking since the very foundation of the country7. Second (and following from
this), private sector managerialism has been seen as a guideline for public sector organization and reform since the 19th century progressivist and good government movements and
has found its scientific elevation in Fredrick W. Taylor’s „scientific management“ (see
Hood 1998: 16, Pollitt 1990: 15).

The cases of the U.K. and New Zealand are somewhat more complicated. On the one
hand, the build-up of the advanced welfare state which under Labour governments resulted in a “strong” welfare state with pronounced public sector structures. On the other hand,
the British (and analogously New Zealand’s) political cultures and thinking are still rooted
in the Common Law tradition in which, unlike the Roman Law tradition of the Continental European countries, a divide between the State and Society, between public law and
private is cognitively and legally not recognized and in which public administration is
conceived as being guided by “public interest” regards rather than by strict legal regulation. Hence, the concepts of contractualism, marketization, managerialism and the like
have had an easy entry and acceptance cognitively as well as normatively to the AngloSaxon world.

By contrast, Germany is still firmly rooted, first, in the idea of the Constitutional State as a
self-standing democratically and politically controlled public sphere with a basic political
and social mandate. Second, it is still premised in a Rechtsstaat tradition, that is, in the
7
See Dwight Waldo 1980: „We did not want a European-style state, we did not need a European style state,
and we did not develop a European-style state“ (quoted from Stillman 1998: 172).
principle that public administration needs to be (politically) directed and controlled by
legislative regulation and, concomitantly, by comprehensive judicial review. By the same
token, the traditional (Weberian) model of legal rule-bound hierarchical public administration is intrinsically geared to the Rechtsstaat principle, including judicial review. In the
German case the politico-democratic profile of the public sector and the legality and judiciability of public administration has become institutionally and normatively even more
firmly entrenched because of the country’s traumatic historic experience with dictatorial
and lawless regimes.
3.1. (Progressing) convergence or (continuing) divergence of the countries’ trajectories?
To summarize, on the one hand, under the onrush of global and international trends, particularly under the world-wide pressure on the state budgets, the governments have turned to neoliberal and NPM-inspired public sector reform policies and will probably continue to do so.
Hence, it has been concluded that the countries are going to move along on basically convergent trajectories of public sector modernization: “At least in OECD countries, the current
disparity between a few pioneering countries and other more cautious counties can be expected to narrow... Reform is becoming a functional imperative it and of itself” (Thoenig in
this volume).
However, on the basis of the empirical findings, particularly evidenced by the persistence of
country-specific or country family-specific institutional and cultural traditions and by the institutional and normative profile of the political systems, it can plausibly be assumed and predicted that the countries (and country families) will continue, at least for the foreseeable, to
exhibit particularities conducive to maintain, if not enhance divergence (for a similar argument see Pollitt/Bouckaert 2000: 96, Christensen/Laegreid 2001: 20 ff.).
Referring to the ideal-type exaggeration in which J. Olsen (Olsen 1988, see also Christensen/
Laegreid 2001: 14 f.) has juxtaposed the “Sovereign State” and the “Supermarket State” as
two models of governance, two scenarios can be recognized.
First, one might discern a type of public sector trajectory in which the main function of the
(proprammatically “lean”) public sector is seen in just enabling public services, while the
delivery of the services should essentially rely on marketization, outsourcing and on “one
function” agencies – with the traditional democratically elected and politically accountable
local self-government being marginalized. Some of these (ideal-type) features can be identified with the real-type development in Anglo-Saxon countries.
By contrast, another pattern may be seen (again with some ideal-type exaggeration) in a public sector trajectory in which, notwithstanding the absorption of significant neo-liberal and
NPM impulses to increase the cost-efficiency,, a main function of the state continues to pertain to legal regulation and law-application and in which the bulk of public tasks continues to
be carried out by (to a significant extent still “Weberian”) public administration, primarily on
the decentralized level of multi-function local self-government. This type of development can
be recognized in Germany and, perhaps with an even stronger emphasis on decentralized
multi-functional local government, in Sweden (probably standing for most other Scandinavian
countries).
While recently in Latin American countries, in abandoning the previous “sequencing” formula, comprehensive NPM reforms have been tackled before the democratization-induced
decentralization has been institutionally and politically consolidated and before a (patrimonialism and corruption-resistent) “classical” civil service has been put in place, it remains to be
seen whether this is a viable and “delivering” modernization strategy.
With regard to the Central Eastern European countries (unfortunately not represented in
country sample of this volume) it would appear analytically likely and normatively advisable
that they, for the time being, heed the rationale of the “sequencing” formula and set their
course on institutional trajectories in which the development of a Rechtsstaat-guided public
administration and of politically and functionally strong local self-government (on the basis of
a multi-function model) should be given top priority. (The sobering experience which, for
instance, Latvia has made with the spree of agencification might be seen as a writing on the
wall, for this example see Pollitt/Bouckaert in this volume ).
4.
Evaluating public sector reforms
Turning now to the evaluation of public sector reforms, at the outset some definitional and
conceptual remarks should be in order, since due to the novelty of this territory the definitional and conceptual mapping tools are still quite unsettled.
4.1. Object of evaluation: Public sector reform policies as distinct from “substantive” policies.

Substantive policies have been called the, as it were, “normal” policies (such as social
policy, employment policy, youth policy housing policy etc.) which essentially aim at
having an impact on the socio-economic etc. situation and development in the “policy environment”. In the history of policy and program evaluation the classical example, of
course, are the Social Action Programs which were started by the US federal government
in 1964 in order to combat the misery of ethnic minorities in US big cities. The evaluation
which the US Congress required to be conducted on those “War on Poverty” Programs
ushered in the rapid expansion of policy and program evaluation in a growing range of
“substantive” policies, including an upsurge of “social experiments”, with the US taking
the lead and European countries following suit8.

By contrast, public sector reform policies are, by definition, directed, in their immediate
intention, at remolding the political and administrative structures. Thus, one may speak, in
an institutionalist parlance, of institution policy9 or, in a policy science diction, of polity
policy10 or also of meta-policy making11.
4.2. The „sequence of goals“ problematique of public sector reform policies
Whereas „substantive policies“ can, in a certainly oversimplified manner, be seen as aiming at
„one“ policy goal (say, the reduction of unemployment, the improvement of the environment)
which is meant to be attained by appropriate measures, the public sector reform policies have
a more complicated „architecture“ which can be described as a “sequence of interrelated
goals”. Typically, public sector reform policy is directed (goal 1) at effecting certain changes
in the political and administrative structures (organization, personnel, instruments, “culture”
etc.) in order to (goal dimension 2) to improve the performance (efficiency, effectiveness etc.)
of public sector activities.
Hence, in the evaluation of public sector policies essentially two distinct analytical schemes
should be distinguished:
8
For an early overview see Levine et al. ed.1981, Levine 1981, for later international overviews see Wagner/Wollmann 1986, Ray ed. 1990, Derlien 1990, Mayne ed. 1992.
9
For a Swiss discussion on the distinction between „substantive policy“ (substanzielle Politik) and „institution
policy“ (Institutionenpolitik) see Knoepfel/Bussmann 1997:59, Ritz 1999: 28.
10
Referring to the distinction made in policy science and policy studies between policy (as the contents of policy-making), politics (as the process of policymaking) and polity (as the institutional setting thereof).
11
quoting a term coined by Yezekel Dror.

Evaluation of goal 1: Has the intended institutional change (of the public sector) been
attained? Evaluation of institutional change

Evaluation of goal(s) 2: Have the intended „results“ of the institutional reform been
achieved? Evaluation of results
4.3.. Evaluation of institutional change
The evaluative effort to assess the institutional change effected by public sector policies needs
to be put in a wider conceptual frame in which the institutional change is conceived as a "process over time”. In this process several phases can be analytically discerned, starting with the
formulation and formation of the concepts (discourse, decision-making), proceeding to the
implementation and resulting in the realization of reforms. (It goes without saying that in the
real world such neat analytical distinctions of phases do not exist, instead they tend to be fluid
and interconnected). Conceptually speaking, the observable public sector reforms put in place
can be conceived as the “dependent variable” while the factors which have shaped them are
the “independent variables”. For an elaboration of the conceptual problems involved and on
some conceptual frameworks which have been submitted for analyzing institutional change in
public sector reform policies reference shall be made to the concepts discussed earlier in this
paper..
4.4.. Evaluation of the „results“ of public sector reforms: Conceptual and methodological
problems
While in the evaluation of institutional change the latter is conceptually treated as a dependent variable that needs to be explained, in the evaluation of results of public sector reforms the
reformed institutional structure are, conceptually speaking, looked upon as independent variables, whereas the intended changes/goals of the reform policy (such as, improvement of performance, efficiency, outputs etc.) are seen as dependent variables – the crucial evaluation
question being whether the observable changes of the latter (outputs etc.) can be causally
related the former (institutional changes).
In the methodological section of their excellent introductory contribution to this volume
Pollitt/Bouckaert give a penetrating “five point” overview of the many intricate conceptual
and methodological problems which “result” evaluations has to cope with (see also Pollitt
1995)12 and among which the following loom large (see also Christensen/Laegreid 2001: 30):

the goals and objectives that serve as a measuring rod are hard to identify, not the least
because the modernization measures mostly come in bundles,

goals are hard to translate into operationalizable and measurable indicators,

good data to empirically “fill in” the indicators are hard to get and the more meaningful an
indicator is, the more difficult it is to obtain viable data,

the more “remote” (which often means: the more relevant) the goal dimension is (such as,
outcomes, “systemic” effects, see Pollitt/Bouckaert 2000: 120 ff., or effects on the
“broader political-democratic context”, see Christensen/Laegreid 2001: 32), the harder it
becomes to operationalize them,

side-effects/unintended consequences13 are hard to trace,

methodologically robust research designs (quasi-experimental, “controlled” time-series
etc.) are rarely applicable (ceteris paribus conditions mostly hard, if not impossible to establish, N too small, “before”-data not available etc.).
Hence, the conceptual and methodological problems of conducting evaluation on public sector reform policies and measures are formidable, particularly when it is undertaken in the
classical vein of ex-post evaluation guided by rigorous conceptual and methodological standards.
Vis-a-vis these manifold conceptual and methodological hurdles “full-fledged” evaluation of
public sector reforms is bound to cope with (and also in view of the reluctance which policymakers and top administrators often exhibit towards getting researchers from outside intimately involved and having them carry out “in-depth” evaluations) Thoenig proposes (in this
volume) a type of “quasi-evaluation” which would be less fraught with conceptual and methodological predicaments than a “full-fledged” evaluation and more disposed to focus on,
and restrict itself to an information- and data-gathering and descriptive function of evaluation
rather than on an explanatory one. Thoenig perceives more than one advantage in such “quasievaluation” approach. First, such conceptually and methodologically “lean” evaluation de-
12
13
See also Knoepfel/Bussmann 1997: 58 ff. Ritz 1999, Wollmann 2000a for a discussion of the conceptual and
methodological problems of evaluation of public sector reform policies.
See Jann/Reichard in this volume: “No organizational development of only modest complexity will haven
without the most common on all social phenomena – unintended and even contra-intuitive processes and results”.
signs may find easier access and wider application in the otherwise hurdle-rich evaluation
territory (whereby he causticly remarks that “there is no surer way of stifling evaluation at
birth than to confine it in the ghetto of methodology”, in this volume). Second, such conceptually and methodologically slimmed down variant of „quasi-evaluation“ may be conducive to
having a more “trustful” communication between the policy-maker and the evaluator and to
promote a „gradual learning process in successive stages that fosters an information culture”
(Thoenig in this volume).
4.5.. The institutions and actors involved in evaluation
At this point it may be useful also to call to mind the array of institutions and actors which are
typically involved in evaluation, be it that they initiate and commission or that they carry out
evaluation. Usually the following distinction and typology is made:

Internal (“in house” or agency-based) evaluation which is carried by the operating unit
itself or by its “home” agency (ministry etc.) in a “self-evaluation” function. In fact, this
internal self-evaluation operation is a key procedure and component in the entire monitoring and feedback system which is pivotal to NMP’s (internal) management and accounting system.

External evaluation within which two variants can be distinguished: initiated/carried out
inside the executive (for instance, by the Finance Minister, Prime Minister’s Office etc.)
and initiated/carried out within the government at large (especially by the parliament, the
court of audit, an adhoc government/parliamentary/”royal-type” commission and the like);

With regard to external evaluation an additional important distinction can be seen in evaluation as „contractual research“(Wollmann 2001b) that is, evaluation research carried out
by outside (ideally independent) research units and teams (commercial, semi-public nonprofit or university-based) that are commissioned (and financed) by governmental institutions (ministries, parliament, court of audit, including the agency itself that operates the
project to be evaluated) to conduct the evaluation study.

Finally, mention should also be made of academic research which, following the “intrascientific” selection of topics, concepts and methods and funded by (independent) foundations or from university resources, turns to the study of public sector reform policies under
an implementation and/or evaluation perspective in what may be called “applied basic research”.
4.6.. Taking stock of evaluation on public sector reform policies: Just a Glimpse
Thanks to wide geographical coverage which has been attained in this special issue of IJPS it
provides a mosaic of the “state of the art” of the evaluation on public sector reform policies
probably more comprehensive than in any other one publication presently on the market.
In view of that scope and richness of information only a few sketchy and selective glimpses
must again suffice to highlight some of the salient profiles and trends.
4.6.1.. Internal (in-house) evaluation
The introduction of internal (agency-based) monitoring, information, and feedback loop
mechanisms which is a conceptual and instrumental key component of the entire NPM management reform concept and machinery has made remarkable advances. In most countries
such procedures of internal monitoring and self-evaluation have been employed as a pivotal
procedure and instrument in the performance management systems hinging on indicatorbased performance measuring, controlling loops and cost-achievement accounting.. The lead
in performance accounting has probably taken by New Zealand (see Halligan in this volume),
while most other countries, such as Sweden, have followed suit (see Jann/Reichard in this
volume). Even in Germany which has been a late-comer to NPM on most other NPM scores,
internal monitoring and “controlling” mechanisms have been put in place as an element of the
NPM-derived “New Steering Model” (see Wollmann in this volume).
4.6.2.. External evaluation
From the rich mosaic of cross-country information enclosed in the articles of this volume the
following trends in external evaluation should, quite selectively, be highlighted in a tentative
„ranking“.

Australia which has been an active public sector modernizer since the early 1980s has
been in the fore also in conducting comprehensive (“ex post”) evaluation on these reform
policies and measures. In 1991, after a decade of intensive reforms, the first (and internationally probably the most extensive yet) evaluation was undertaken by the Task Force on
Management Improvement, a quasi-independent group of public servants (see Halligan in
this volume). In addition, for instance, a prominent business leader was commissioned to
do further (more sectoral) reviews. Vis-a-vis these and a spree of other evaluation activities the Australian State Services Commission went as far as speaking of a “picture of
evaluation overkill” (quoted by Halligan in this volume).

In the USAan evaluation tradition has emerged since the 1960s which has resulted, inter
alia, in the political disposition and practice of the US Congress to regularly write evaluation requirements into relevant legislation (see e.g. Freeman/Solomon 1981). Following
this line, the Civil Service Reform Act of 1978, for instance, stipulated evaluation to be
carried out thereon. Similarly, extensive evaluation was conducted under the 1992 Government Performance and Results (see Christensen et al. in this volume) At the same time,
the General Accounting Office and the Office of Management and Budget as well as other
traditional institutional advocates of policy evaluation have pushed evaluation also on
public sector reform policies(see Pollitt/Bouckaert in this volume).

New Zealand which has been the most active, if not aggressive promoter of NPM reforms
has a strong showing also in initiating major evaluative reviews, by first, in 1991, mandating the Steering Group for Review of State Sector Reforms and then, in 1994, commissioning Allan Schick, an internationally renowned expert, to conduct a comprehensive review14 (see Halligan in this volume).

Rooted in a policy evaluation tradition which evolved in the 1960s and even earlier15,
hinting at a “scientific revolution” in policy-making and centering around its traditional
“committee” system, and additionally ingrained in a “freedom of information” culture,
Sweden has manifested a broad interest and readiness also to evaluate the public sector reform policies and measures which have got under way since the early 1980s. For instance,
the Expert Group on the Public Sector has provided regular reports assessing the productivity in government activities (Christensen et al. in this volume).

Germany presents a somewhat deviant and puzzling case. On the one hand, the country
does have a noticeable tradition of (“substantive”) policy evaluation (in having been,
along with Sweden, among the evaluation frontrunners among European countries in the
1960s16, bordering on a “scientific revolution” in policy-making17, and continuing to employ the evaluation of “substantive” policies in a significant volume). On the other hand,
conspicuously little external evaluation of public sector modernization measures has so far
14
15
16
17
Allan Schick concluded that although the reforms were „more comprehensive and rigorous than those introduced in other countries, they have been neither complete nor perfect“ (quoted from Halligan in this volume).
During the 1960s Sweden was one of the frontrunners among European countries in the introduction and
application of evaluation, for an international. overview see Levine 1981: 50 2: “Evaluation is ... endemic
throughout the Swedish system”.
See Levine 1981, Derlien 1990: 148.
During the late 1960s and early 1970s when the US set the pace in conducting large-scale „social experimentation“, Germany was probably the one country (besides the US) that (temporarily) went furthest in venturing
into „social experimentation“ („experimentelle Politik“) premised on the conduct of evaluation (see Hellstern/Wollmann 1983).
been undertaken, probably due, first of all, to the fact that Germany has been a latecomer
to the recent upsurge of public sector reforms (see Jann/Reichard in this volume). It
should be added, however, that there has been a swell of university- and public research
institution-based research in which, with evaluative implications, the recent wave of public sector reforms has been studied, with a focus on the local government level where most
reform activities have taken place so far (see Wollmann in this volume). Furthermore, the
research copiously conducted on the institutional transformation in East Germany should
be mentioned which, after all, was an event and project of public sector modernization on
an unprecedented scale (see Wollmann 1996b).
The above touched upon country cases (as well as the other cases in this volume’s mosaic)
suggest that the evolution and application of external evaluation has been strongly influenced
by a constellation of factors in which the country’s evaluation tradition, the emergence and
existence of an institutional advocate of external evaluation and the intensity and duration of
the recent public sector reform wave loom large. The USA is a case of an evaluation tradition
well entrenched in legislative practices and institutional watchdogs (such as GAO) as well as
in its information-eager scientific culture, while Sweden is similarly rooted in an evaluation
tradition marked by a culturally and scientifically grounded interest in the transparency of
political and administrative matters. In Australia and New Zealand it seems to be the transparency postulate itself on which the NPM doctrine essentially hinges that has propelled the
relevant actors to undertake external evaluations. Moreover, these four evaluation-active
countries indicate that intensity and duration of the public sector reforms has been another
crucial factor to elicit external evaluation.
As to the broader cross-country picture and pattern of the conduct of external evaluation on
public sector reforms mention should finally be made of the observation that the stock of
hitherto available evaluation seems to be marked by a “north/ south divide” (see Jann/ Reichard in this volume) or a Common Law /Roman Law tradition divide (see Thoenig in this volume). At this point, these clues shall only be taken notice of as hopefully thought- and research-stimulating ideas.
4.7. Current state and perspective of evaluation on public sector reforms
Summarizing and winding up this introductory overview, the mapping and stock-taking which
is undertaken in this special issue on evaluation of public sector reforms provides an ambivalent, kind of „glass half full and half empty“ picture.
On the one hand, significant advances can be observed, particularly in the countries which
have taken the lead in NPM reforms. This holds true particularly for the introduction and
employment of internal evaluation and self-evaluation in the context of performance management, monitoring, cost-achievement-accounting, controlling and the like, but also with
regard to external evaluation, including some remarkable attempts at comprehensive ex-post
evaluations and reviews.
On the other hand, even in the countries which are most advanced in the development of internal and external evaluation it needs to be emphasized that these moves and achievements
fall conspicuously short of the promises and (high) standards which would logically follow
from the key function and pivotal role which in the NPM doctrine is ascribed and prescribed
to the creation of transparency through evaluation and to the utilization of thus generated
knowledge. As measured against these programmatically high standards and postulates of
NPM the current modernization practice still shows a glaring discrepancy and reveals a paradox which has repeatedly (and rightly) been addressed in the literature.
No doubt, the introduction, institutionalization and utilization of evaluation in public sector
reform policies and measures faces an uphill fight on more than one front. On the conceptual
and methodological line it is confronted by a mine-field of conceptual and methodological
problems (of which Pollitt/ Bouckaert give an account in this volume). On the front of the
political and administrative practice the employment of evaluation, particularly in the classical vein (such as,ex-post evaluation committed to rigorous conceptual and methodological
standards and carried out by independent researchers) encounters a host of obstacles. First it
has to reckon with the “natural” reluctance of political and administrative leaders to let outside researchers undertake independent evaluative investigations that may have, in their view,
unexpected, embarrassing and unwelcome results and may provide ammunition to the political opposition, to the public at large and the media (see Thoenig in this volume). Furthermore,
it has to count with recalcitrance, if not boycotting from the administrative rank and file who
eye administrative reforms with skepticism and suspicion, particularly in times of budgetary
squeeze in which such reforms easily smack of job reduction and work-load increase.
While, as it has been evidenced also by the contributions to this volume, further advances of
evaluation on public sector modernization hinge essentially on institutional as well as cultural
changes, this summary will conclude with an institutionalist focus and perspective in assuming that the further development of evaluation will largely depend on the existence and density of institutional advocates in the country’s politico-administrative system.

Executive government/administration. Whereas the operating administrative units (sectoral ministries, agencies) themselves can be assumed of having little interest in initiating
evaluation, leave alone external evaluation, executive government actors with crosscutting (budgetary, planning etc.) interests may be much more readily disposed and eager
to impose, and conduct evaluation on the modernization activities of such sectoral ministries and agencies. In fact, in the cases of Australia and New Zealand the central government finance ministries have been instrumental in promoting NPW as well as evaluation
thereon (see Halligan in this volume). In Japan the Management and Coordination Agency which is subordinated to the Government cabinet has recently been charged with taking
a lead function in evaluating the reform process (see Muramatsu/Matsunami in this volume). The Swedish Agency for Public Management deals with regular reporting (see
Christensen et al. in this volume). There are moves in Germany to strengthen the evaluation function of the Chancellor’s Office, along with its intended lead in public sector modernization (see Jann/Reichard in this volume).

Parliament should be assumed to act as a key player in the promotion of evaluation on
public sector reforms. Yet, parliaments have so far largely refrained from becoming seriously involved. The US Congress is an exception in following its traditional practice of
legislatively requiring evaluation. Even in other active NPM-reform countries the parliaments were quite slow in resuming an active reform role (which, incidentally, would also
imply redefining their own role). This “parliamentary restraint” seems particularly pronounced in countries with an accepted state and Rechtsstaat tradition (with regard to
which Jann/Reichard’s north/south divide and Thoenig’s Common Law/Roman Law divide could make some sense).
The Court of Audit should also be seen as a crucial institutional candidate to advocate evaluation on public sector reforms. Again, General Accounting Office in the USA (with its traditional evaluation profile) and also Sweden’s Court of Audit are important cases in point (see
Christensen et al. in this volume). Most recently, Japan’s Court of Audit has been explicitly
assigned such an evaluation function on public sector reforms (see Muramatsu/Matsunami in
this volume). In Continental Europe, the courts of audit have so far refrained from such an
involvement..
Lastly, the scientific community should become the core of an „advocacy coalition“ in the
demand and persuasion that the evaluation of public policies, including public sector reform
policies, by independent research should be regular practice of policy-making in modern
government. The growing number of evaluation societies which have come into existence in
the international and national contexts point at an exciting and encouraging development. For
one, they indicate the will and readiness of both researchers and practitioners involved in
evaluation activities to improve the conceptual and methodological quality of evaluation and,
at the same, to set up and to subscribe to ethical standards to ensure the intellectual honesty
and moral integrity of their work. Secondly, the evaluation societies (in some of which working groups on public sector modernization have been established) can play a critical role in
forming “advocacy coalitions” in calling for (and possibly even “lobbying for”) independent
high-quality evaluation to be conducted in and on policy-making, not least in and on public
sector reform policies.
Final note: While the point of cautioning against an easy trans-national transfer of “good practice” is well taken by W. Jann and Ch. Reichard (in this volume), there can be no doubt that
the accumulation and the diffusion of the internationally available stock of knowledge and
experience in carrying out public sector reforms and in institutionalizing and conducting evaluation thereon make for an indispensable and critical potential in order to make progress in
either domain. May this volume, through its wide coverage of countries and regions and its
assembly of leading experts in the field, make a noticeable contribution to this mandatory,
albeit arduous inter- and trans-national learning process.
References
Benz, Arthur/Goetz, Klaus H. 1996, The German Public Sector: National Priorities and the
International Reform Agenda, in: Benz, Arthur/ Goetz, Klaus H. (eds.) 1996, A New
German Public Sector?, Aldershot etc.: Dartmouth, pp. 1-26.
Bussmann, Werner/Klöti, Ulrich/Knöpfel, Peter (eds.) 1997, Einführung in die Politikevaluation, Basel: Helbing & Lichterhan.
Christensen, Tom/Laegreid, Per (eds.) 2001, New Public Management, Aldershot etc.: Ashgate.
Christensen,Tom/Laegreid 2001, A Transformative Perspective on Administrative Reforms,
in: Christensen, Tom/Laegreid, Per (eds.) 2001, New Public Management, Aldershot
etc.: Ashgate, pp. 13-39.
Derlien, Hans-Ulrich 1990, Genisis and Structure of Evaluation Efforts in Comparative Perspective, in: Rist, Ray C. (ed.) 1990, Program Evaluation and the Management of Government, New Brunswick/London: Transaction, pp. 147-177.
Derlien, Hans-Ulrich 1996, Patterns of Postwar Administrative Development in Germany, in:
Benz, Arthur/Goetz, Klaus H. (eds.), A New German Public Sector?, Aldershot: Dartmouth, pp. 27 ff.
Derlien, Hans-Ulrich 2000, Actor Constellation, Opportunity Structure and Concept Feasibility in German and British Public Sector Reforms, in: Wollmann, Hellmut/Schröter,
Eckhard (eds.) 2000, Comparing Public Sector Reform in Britain and Germany, Aldershot: Ashgate, p. 150-170.
Derlien, Hans-Ulrich, Patterns of Postwar Administrative Development in Germany, in: Benz,
Arthur/Goetz, Klaus H. (eds.) 1996, A New German Public Sector?, Aldershot etc.:
Dartmouth, pp. 27-44.
Hellstern, Michael/Wollmann, Hellmut (Hrsg.) 1983, Experimentelle Politik, Opladen:
Westdeutscher Verlag.
Freeman, Howard/Solomon, Marian A. 1981, The Next Decade of Evaluation Research, in:
Halligan, John 2001, The Process of Reform in the Era of Public Sector Transformation, in;
Christensen, Tom/Laegreid, Peer (eds.) 2001, New Public Management, Aldershot etc.:
Ashgate, pp. 73-89.
Hood, Christopher 1998, The art of the state: culture, rhetoric and public management, Oxford, Oxford University Press.
Karl, T. L. and Schmitter, P. C. 1991, Modes of transition in Latin America, Southern and
Eastern Europe, International Social Science Journal, no. 43, pp. 269-284.
Knoepfel, Peter/Bussmann, Werner 1997, Die öffentliche Politik als Evaluationsobjekt, in:
Bussmann, Werner/Klöti, Ulrich/ Knöpfel, Peter (eds.) 1997, Einführung in die Politikevaluation, Basel: Helbing & Lichterhan, pp. 58-77.
Levine, Robert A. 1981, Program Evaluation and Policy Analysis in Western Nations: An
Overview, in: Levine, Robert A./Solomon, Marian A./Hellstern, Gerd-Michael/ Wollmann, Hellmut (eds.) 1981, Evaluation Research and Practice. Comparative and International Perspectives, Beverly Hills/London: Sage, pp. 12-27.
Levine, Robert A./Solomon, Marian A./Hellstern, Gerd-Michael/Wollmann, Hellmut (eds.)
1981, Evaluation Research and Practice. Comparative and International Perspectives,
Beverly Hills/London: Sage, pp. 12-27.
Mayne, J./Bemelsmans-Videc, M.L./Hudson, J./Conner, R. (eds.) 1992, Advancing Public
Policy Evaluation, Amsterdam: North-Holland publ.
Olsen, J.P. 1988, Administrative Reform and Theories of Organization, in: Campbell, C./Peters, Guy (eds.) 1988, Organizing Governance: Governing Organizations, Pittsburgh:
University of Pittsburg Press, pp. 233-254.
Osborne, D./Gaebler, T. 1992, Reinventing government: how the entrepreneurial spirit is
transforming the public sector, Reading, Mass., Adison Wesley.
Pollitt, Christopher 1990, Managerialism and the Public Services. The Anglo-American Experience, Oxford.
Pollitt, Christopher 1995, Justification by works or by faith? Evaluating the New Public Management, in: Evaluation, vol. 1, no. 2, October, pp. 133-154.
Premfors, R. 1991, The „Swedish Model“ and public sector reform, in: Western European
Politics, vol. 14, no. 3.
Premfors, R. 1998, Reshaping the democratic state: Swedish experiences in a comparative
perspective , in: Public Administration, 76:1, pp. 141-159.
Rist, Ray C. (ed.) 1990, Program Evaluation and the Management of Government, New
Brunswick/London: Transaction.
Ritz, Adrian 1999, Die Evaluation von New Public Management, Bern.
Sabatier, Paul 1988, An Advocacy Coalition Framework of Policy Change and the Role of
Policy-Oriented Learning Therein, in: Policy Sciences, vol. 21, pp. 129-168.
Schröter, Eckhard/Wollmann, Hellmut 1997, Public sector reforms in Germany: Whence and
where? A case of ambivalence?, in: Administrative Studies/Hallinnon Tutkimus, vol. 3,
pp. 184-200.
Stillman, Richard J. 1998, Creating the American State, Tuscaloosa/London: University of
Alabama Press.
Vedung, Evert 1992, Five Observations on Evaluation in Sweden, in: Mayne, J./BemelsmansVidec, M.L./Hudson, J./Conner, R. (eds.) 1992, Advancing Public Policy Evaluation,
Amsterdam: North-Holland publ., pp. 71-87.
Wagner, Peter/Wollmann, Hellmut 1986, Fluctuations in the development of evaluation research: Do regime shifts matter?, in: International Social Science Journal, pp. 205-218.
Wittrock, Björn/Wagner, Peter/Wollmann, Hellmut 1991, Social science and the modern
state: policy knowledge and the political institutions in Western Europe and the United
States, in: Wagner, Peter/Weiss, Carol/Wittrock, Björn/Wollmann, Hellmut (eds.) Social Science and Modern States, Cambridge; Cambridge University Press, pp. 28 ff.
Wollmann, Hellmut 1996a, Modernization of the Public Sector and Public Administration in
the Federal Republic of Germany – (Mostly) A Story of Fragmented Incrementalism, in:
Muramatsu, Michio/Naschold, Frieder (ed.), State and Administration in Japan and
Germany, Berlin/New York: deGruyter, pp. 79-101.
Wollmann, Hellmut 1996b, Verwaltungsmodernisierung. Ausgangsbedingungen, Reformanläufe und aktuelle Modernisierungsdiskurse, in: Reichard, Christoph/Wollmann, Hellmut (Hrsg.), Kommunalverwaltung im Modernisierungsschub?, Basel usw.: Birkhäuser,
S. 1-46.
Wollmann, Hellmut 1996c, The Transformation of Local Government in East Germany: Between Imposed and Innovative Institutionalization, in: Benz, Arthur/ Goetz, Klaus H.
(eds.) 1996, A New German Public Sector?, Aldershot etc.: Dartmouth, pp. 137-164.
Wollmann, Hellmut 1997a, Modernization of the Public Sector and Public Administration in
the Federal Republic of Germany – (Mostly) a Story of Fragmented Incrementalism, in:
Muramatsu, Michio/Naschold, Frieder (eds.), State and Administration in Japan and
Germany, Berlin: deGruyter, pp. 79 ff.
Wollmann, Hellmut 1997b, Institution building and decentralization in formerly socialist
countries: the cases of Poland, Hungary, and East Germany, in: Government & Policy,
vol. 15, pp. 463-480.
Wollmann, Hellmut 2000a, Evaluierung und Evaluierungsforschung von Verwaltungspolitik
und -evaluierung – zwischen Analysepotential und -defizit, in: Stockmann, Reinhard
(ed.), Evaluationsforschung, Opladen: Leske + Budrich, pp. 196 –229.
Wollmann, Hellmut 2000b , Comparing Institutional Development in Britain and Germany:
(Persistent) Divergence or (Progressing) Convergence?, in: Wollmann, Hellmut/Schröter, Eckhard (eds.) 2000, Comparing Public Sector Reform in Britain and Germany, Aldershot: Ashgate, pp. 1-26.
Wollmann, Hellmut 2001a, Germany’s trajectory of public sector modernisation: continuities
and discontinuities, in: Policy & Politics, vol. 29, No. pp. 151-169.
Wollmann, Hellmut 2001b, Contractual Research and Policy Knowledge, in: International
Encyclopedia of the Social and Behavioral Sciences, vol. 5 (forthcoming