Masteruppsats i offentlig förvaltning [VT/2012]
Förvaltningshögskolan, Göteborgs universitet
Seth Sander 870714-4930
Handledare: Lena Lindgren
Examinator: Patrik Zapata
Efficient Privatization of SOEs –
The Importance of Operationalization in
Policy Evaluation.
Abstract
In the beginning of this thesis a problem regarding efficiency operationalization in previous
studies of SOE privatization was introduced. The purpose of the thesis was then established as
discerning in what ways various operationalizations of efficiency influenced the obtained
results and their validity.
After a brief introduction to the history of privatizations and the progression of ideas and
feelings regarding privatization previous empirical studies of privatization were detailed. It
was established that these previously conducted empirical studies of privatization were quite
similar in terms of their operationalizations of efficiency as well as their end results. From
these previous studies an operationalization was derived based on their similarities, which was
labeled operational efficiency. I then argued that these previous empirical studies were in fact
evaluations based on the definition of evaluation supplied by Vedung. From the basis of
evaluation theory I proceeded to question the operationalizations in the detailed evaluations of
privatization and, by extension, the validity of their conclusions. The basis of the criticism of
the operationalizations in the detailed evaluations of privatization was based primarily on the
lack of adherence to goals. In order to identify what the potential consequences of the
operationalizations in the detailed privatization evaluations could be two analyses of a recent
case, Apoteket AB, were carried out. One of the analyses was carried through the
identification and operationalization of the goals of the privatization; this process was labeled
goal fulfillment. The other was carried out using the previously defined operational efficiency.
The results of these two analyses were then compared. Despite the purposes of the analyses
being the same the results provided information regarding completely different aspects of the
privatization and the connection between operational efficiency and goal fulfillment was
nonexistent. I then argued that the construct validity of the measurement operational
efficiency was sorely lacking and that by extension the conclusions regarding the efficiency of
privatization as a policy tool are questionable.
2
Abstract ............................................................................................................................ 2
Introduction ..................................................................................................................... 4
What is the purpose of this paper? .............................................................................................................................4
Important Definitions and Distinctions ....................................................................................................................6
What is an SOE? ...................................................................................................................................................................... 6
Can Apoteket AB be considered an SOE? .................................................................................................................... 7
What is a privatization? ..................................................................................................................................................... 7
Can the reregulation of the Swedish pharmaceuticals monopoly be considered a privatization? . 8
Outline of the Thesis ......................................................................................................... 9
A Short Historical Background of Privatization................................................................. 11
Privatization and Efficiency, Defining Operational Efficiency and Goal Fulfillment............ 14
Empirical Studies of Privatization............................................................................................................................ 14
An Alternative View of Efficiency .................................................................................................................................18
What, besides goal fulfillment, is needed in order to assess efficiency? .................................................. 27
Methodology – Measuring the Effects of Privatization, Dual Approaches ......................... 28
Sources and Collection of Data .................................................................................................................................. 31
Data used to measure changes in operational efficiency ..................................................................................32
Empirical Results, Goal Fulfillment – Pre- and Post-Privatization Data ............................. 34
Empirical Results – Pre- and post-privatization operational efficiency.............................. 47
Conclusions .................................................................................................................... 52
Reflections...................................................................................................................... 56
Thoughts regarding my choice of case ................................................................................................................... 56
The need for further research.................................................................................................................................... 56
Summary ........................................................................................................................ 58
References ...................................................................................................................... 59
Printed Sources ................................................................................................................................................................ 59
Public Sources, Annual Reports and other economic data. ........................................................................... 60
Web Based Sources ........................................................................................................................................................ 62
3
Introduction
What is the purpose of this paper?
Privatization is a relatively new phenomenon often said to have started with the Thatcher
government in Britain.1 Since then privatization has become widespread, primarily but not
exclusively, in the industrialized world.2 Sweden has not been an exception to this, somewhat
surprisingly considering a history of nearly unbroken rule by a social democratic political
majority.
Over the years a multitude of empirical studies regarding privatization of state owned
enterprises (SOEs) have been conducted. In much of the recent empirical analysis of
privatization efficiency has been assessed through various comparisons of SOE performance
before and after a privatization. 3 As I will show later in this thesis the pre- and postprivatization performance of former SOEs is generally measured through a number of key
indicators of SOE efficiency. The more comprehensive studies repeat this process for a large
number of former SOEs in different countries. An overview of these empirical studies of
newly privatized SOEs shows very convincing evidence for the benefits of privatization in
terms of increased efficiency and output of SOEs.4
While these results are quite convincing and clear-cut I find that there is an aspect of this
empirical research that warrants further investigation and it has to do with the way the term
efficiency is operationalized in these previous studies. Closer examination of the term
efficiency reveals that the factors taken in to account in the previous studies may be
insufficient.5 Vedung for example describes efficiency in terms of “The programs’ capacity to
produce predicted outcomes that fulfill the goals of the program (…)”.6 I do not believe that
the operationalizations present in the previous studies account for goal fulfillment as an
efficiency criterion and it is this problem that I have studied.
1
Megginson et.al (1998) p. 56
Dyck (2001)
3
For a more detailed description see section 4 starting at page 13 of this thesis. A definition of what does and
does not constitute an SOE is provided in the following section.
4
Jordahl (2008)
5
For a comprehensive discusison of this see pages 14 – 19 of this thesis.
6
Vedung (2009) p. 31
2
4
By analyzing a recent privatization through the use of the conventional operationalization and
methodology found in previous studies as well as an alternative, theoretically derived,
operationalization and methodology I will attempt to discern in what ways these
operationalizations of efficiency can influence the obtained results and their validity. The
central question guiding this thesis will therefore be: How does the operationalization of
efficiency in the detailed previous empirical studies of privatization affect results?
In order to answer this question I will:
A: Analyze previous empirical studies of privatization in order to identify an established
operationalization and methodology.
B: Theoretically derive an alternative operationalization and methodology.
C: Analyze a recent case of privatization using both operationalizations and methodologies
separately in order to be able to compare the results to one another and analyze how results
are affected by the disparate operationalizations.
Any research that can increase the general understanding of how the effects of privatizations
of SOEs are affected by evaluation-methodology would be worthwhile and could potentially
improve the quality of political decision-making on a national and supranational level. But
perhaps more importantly this thesis has the potential to create awareness of the need for more
interdisciplinary scientific analysis of privatizations as opposed to analyzing the economic
and sociopolitical effects independently. In fact, large-scale privatizations are still being
planned, for example in Sweden7 and in the interest of improving these future decisions more
research on the subject of privatization is crucial.
To this end I am going to examine the most recent privatization in Sweden, namely the state
owned pharmaceutical retail monopoly Apoteket AB. Apoteket AB was, until July 2009, the
sole provider of retail pharmaceuticals in Sweden. In 2009 the market was reregulated and a
large section of Apoteket’s assets were sold. The privatization process will be described in
more detail in a later section. The primary reasons for choosing this particular case were its
size and the fact that it was recent. Completion of the thesis requires effects of the
privatization to be measured and at the time I reasoned that these effects may be more evident
7
Odell (2009)
5
for a large privatization as opposed to a smaller one. The fact that the privatization was quite
recent meant that it had yet to be evaluated by any national government agency and I also
assumed that data regarding the privatization would be abundant.
Before the examination of this case can begin there are some important definitions and
distinctions that need to be made.
Important Definitions and Distinctions
A definition of the term efficiency is obviously very important for this thesis and such a
definition will be forthcoming. Unfortunately placing a definition of efficiency in this section
interfered with flow of the thesis as a whole and it is therefore part of a later section.
Another important questions is likely whether my chosen case, Apoteket AB, is even an
example of an SOE privatization. This question has at least two parts:
1. Can Apoteket AB prior to the reregulation of the retail pharmaceuticals market be
considered an SOE?
2. Was the reregulation actually a case of privatization?
Each of these questions in turn need to be answered in at least two parts:
1a. What is an SOE?
1b. Can Apoteket AB be considered an SOE?
2a. What is a privatization?
2b. Does the reregulation of the Swedish pharmaceuticals monopoly fall under this
definition of privatization?
What is an SOE?
1a: There are two defining features of an SOE. The first is that the nation state is the
controlling owner of the enterprise and the second is that goods or services are sold.8 The
important distinction to make here, especially in the Swedish context, is between an SOE and
a state owned organization that provides a service but is primarily financed indirectly through
taxes. In the Swedish case a school or hospital that provides the overwhelming majority of
their services essentially free of direct charge are not to be considered SOEs as opposed to for
8
The World Bank (1995) p. 26
6
example formerly state owned Svensk Stål AB (SSAB) a steel/mining company which sold its
goods and services both prior to and after privatization. This is an important distinction to
make, as nearly every part of the organizational structure, as well as the goals and general
culture of these types of organizations is likely to be entirely different. Thus it stands to
reason that the theories explaining the privatization of one do not necessarily apply to the
other. This may well be an interesting subject for future research to examine but privatizations
of non-SOEs are not covered within the scope of this thesis.
Can Apoteket AB be considered an SOE?
1b: The primary purpose of the company was to fulfill the role as sole vendor of
pharmaceuticals in Sweden. While Apoteket AB did in fact provide certain free public service
functions such as the poison information hotline, (giftinformationscentralen) these services
were not part of the privatization and are still provided in a public service capacity by
Apoteket AB after the reregulation.9 The privatization encompassed the business side of
Apoteket AB which sold pharmaceuticals to consumers in order to finance its own existence.
Therefore this criterion is fulfilled.
What is a privatization?
2a: The defining feature of a privatization is the relinquishment of state control of an
organization in favor of private actors, however this definition is oversimplified. In fact, in
order for an SOE to be considered privatized there is no need for complete liquidation of the
state’s stake; this is generally referred to as a partial privatization where part of the ownership
is sold but the state retains voting control. A full privatization is often handled through a sale
of stock in the SOE to the point where the state is no longer the controlling owner.10 A sale of
an SOE to another state or other company controlled by the state is generally not considered a
privatization. 11 In the scientific literature the term privatization is occasionally supplanted by
other terms such as divestiture. In this thesis I have chosen to use the term privatization as I
believe it to be the most widely used.
9
Prop. 2008/09:145 (2009)
For a discussion of what constitutes controlling ownership see La Porta et.al (1999)
11
Megginson et.al (1998) p. 62
Jordahl (2008) p. 9.
10
7
Can the reregulation of the Swedish pharmaceuticals monopoly be considered a
privatization?
2b, What differentiates the case of Apoteket AB from the most common types of privatization
is that the enterprise Apoteket AB was broken up in to two main categories; one that is still
under state ownership and one that was broken up into clusters which in turn were sold to
private business. Therefore while approximately 60% of Apoteket AB’s assets were sold, the
voting rights and the ownership of the brand name Apoteket AB, along with the remainder of
the assets are still in the hands of the state.12 The 60% of Apoteket AB’s assets that were sold
to private investors now directly compete with the remainder of the SOE in a mostly open
market. Considering that there was significant transfer of ownership from public to private the
reregulation can be considered a privatization even though what is left of the SOE is still
under state ownership. A privatization of 60% of an SOEs assets is a clear example of a
privatization and therefore this criterion is fulfilled.
12
Prop 2008/09:145 (2009) p. 84f
8
Outline of the Thesis
The historical background that follows this section serves to put the current theoretical and
empirical studies of privatization into a historical context and describe the process which has
led to where the field is today.
The following theoretical section will begin with a brief overview of the privatization
literature. This overview is not to be considered all-encompassing but serves the purpose of
giving the reader an introduction to the central ideas of the empirical work that has been
conducted on privatizations of SOEs. I will then move on to discuss what I consider
methodological shortfalls in the empirical study of privatization and the contribution I wish to
make to the general understanding of the privatization phenomenon and its evaluation. The
methodological shortfalls in the previous literature pertain to the definition of efficiency and I
will argue that the narrow definition of efficiency used in many empirical studies does not
effectively assess the effects of an SOE privatization on a society. I will then present an
alternative way of assessing the efficiency of a privatization.
In the theoretical section I give an account of a number of studies of privatizations and
identify the parameters used to measure the efficiency of a privatization effort. I also argue
that this way of assessing the efficiency of privatizations may not be optimal and propose an
alternative way of measuring the efficiency of privatizations mostly by supplanting
operational efficiency with goal fulfillment. I also identify the general goals of my chosen
case as increased consumer access and lower prices of pharmaceuticals. In order to assess
these goals more easily I have formulated a number of hypotheses pertaining to each goal.
These hypotheses will later be tested and the results of the hypotheses tests will be used as a
basis for an assessment of goal fulfillment. The methodological section presents these
hypotheses and defines how they will be tested and what data will be used. It also more
clearly defines how the assessment using the established efficiency definition will be carried
out.
The empirical section summarizes the data gathered and tests the hypotheses based on the
data from before and after the privatization of Apoteket AB. The results of these tests, as well
as their implications for fulfillment of the privatization’s goals are then analyzed in a short
analysis section. The pre- and post-privatization data points for the theoretically derived
performance indicators are also presented and synthesized.
9
In the conclusions section the results of the goal fulfillment assessment will then be compared
to the results of the analysis of operational efficiency and analyzed using the theoretical tools
developed earlier in the thesis.
10
A Short Historical Background of Privatization
Privatization is often identified in political discourse as pertaining to a certain ideology or
political standpoint such as, but not limited to, neo-liberalism13 but I believe this view can be
misleading. In the past few decades, governments from all parts of the political spectrum have
been carrying out privatizations of various SOEs.14 Instead I believe a great deal more can be
understood if privatization is viewed as a phenomenon not exclusive to governments with a
certain political tradition, i.e. conservative, socialist or liberal, but rather as a paradigm shift
from the use of one form of policy too to another. What is considered favorable ownership
structure changes over time and what is popular one day may not be so the next; and as a
matter of definition, widespread privatization must have been preceded by widespread state
ownership or there would be nothing to privatize.
Shirley & Walsh describe how state ownership experienced surging popularity during the
1930s, 1940s and 1950s among many developed nations and somewhat later among
developing nations. At the time this was seen as the cure for market failures, which were
considered the cause of all manner of problems and economic injustices. In parts of the
developing world the rise of SOEs was also brandished as a means of state economic
independence and a move away from foreign meddling in the economy. Although there was
some theoretical criticism of state ownership at the time it did not have any significant impact
until the 1960s and 70s.15
During the 50s and 60s the attitudes toward SOEs in the developed world started to change.
While the idea of state ownership was still generally looked upon favorably there was
increasing concern over poor performance and inefficiency of SOEs. Nellis describes how the
general mindset regarding SOEs changed and how the World Bank, among other
organizations started to credited governments for the purpose of reforming SOEs:
“The Bank had assisted member governments in this field in hope that, in addition to
providing basic and needed services in an efficient and cost-effective manner (…) public
enterprises would assist the development of strategic sectors, gain access to commercial
13
Alliansfritt Sverige (2012)
Corpwatch (2012)
Socialistalternative (2012)
The World Bank (1995) p. 4
14
Megginson et.al (1998) p. 44
15
Shirley & Walsh, (2001) p. 3
11
credit that would be denied to small private businesses, fill entrepreneurial gaps,
empower numerically large but economically weak segments of the population, maintain
employment levels, and raise the level of savings and investment.”16
Nellis goes on to explain that by the end of the 70s there were considerable doubts as to
whether these goals were achievable through reform. Despite repeated reforms at great
expense the reformed SOEs failed to meet the reform goals. Most SOEs continued to perform
as inefficiently as before and the performance of some actually worsened. In some developing
countries the burden of lossmaking SOEs overwhelmed budgets and banking systems. The
economic turmoil even created problems for profitable firms, which could not raise sufficient
investment capital. 17 These realizations initiated a paradigm shift at the World Bank and
among researchers: “By the mid-1980s, the dominant view among Bank staff working on
SOE reform was that the available set of improvement mechanisms, short of ownership
change, was hard to apply and even more difficult to sustain, and that more hope was offered
by privatization”.18
The conservative British government led by Margaret Thatcher in the late 1970s coined the
term privatization to describe the transfer of ownership from public to private and this is often
thought of as the start of a privatization paradigm. 19 The privatizations of the Thatcher
government were not the first privatizations of that era (both West Germany and Chile had
sold SOEs prior to that). However the impact of the wave of privatizations in Britain in the
early 1980s was much greater.20
The privatizations carried out by the Thatcher government were quite controversial at the
time. There was substantial political resistance from opposition politicians to privatizations
and even threats of re-nationalization by the Labour party.21 Despite political opposition the
initial British privatizations were well-received and Margaret Thatcher secured a second term
in office in 1983. The apparent success of the British privatization effort sparked interest
16
Nellis (2002) p. 2f
Nellis (2002) p. 3
18
Nellis (2002) p. 3f
19
Shirley & Walsh (2001) p. 3
Megginson et.al (1998) p. 44
20
Jordahl (2008) p. 11
21
Megginson et.al (1998) p. 47
17
12
among other western European countries and similar programs of privatization were
undertaken. In France alone 22 large SOEs were sold during 1986-87.22
The same period also saw major privatizations in countries such as Japan and the United
States. The privatization of the Japanese SOE Nippon Telephone and Telegraph (NTT) was
by far the largest ever undertaken at the time, worth approximately USD 77 billion. In the
United States the most notable privatization was that of Conrail and it, while much smaller
than the privatization of NTT, received significant publicity. Both of these major
privatizations were well-received and generally deemed quite successful.23
These early privatizations did not only draw significant political interest but scientific as well,
but most of the empirical findings generally lacked statistical significance and the results of
these studies were mostly inconclusive.24
More recent empirical work (which will be discussed at greater length in a later section) has
provided less ambiguous results. Megginson et.al stipulate that:
“More recent theoretical and empirical studies have offered stronger support for the dual
propositions that private firms outperform SOEs and that privatization itself increases the
operating efficiency of divested firms.”25
As privatizations have continued to change ownership structures in countries around the
world the privatization paradigm has become quite influential, in no small part due to the
abundance of empirical research endorsing it.26
22
Jordahl (2008) p. 11
Megginson et.al (1998) p. 58f
24
Megginson et.al (1998) p. 45
25
Megginson et.al (1998) p. 46
26
Dyck (2001) p. 59
23
13
Privatization and Efficiency, Defining Operational Efficiency and
Goal Fulfillment.
Empirical Studies of Privatization
Privatizations, public/private partnerships and other collaborations have become very popular
during the last three decades. This process has been described as a paradigm shift from a
public/private relationship that was seen in terms of clearly defined lines and clear
hierarchical exertions of power from top to bottom to a “blurring of the lines” between public
and private.27 This paradigm shift has spawned a vast amount of scientific literature in areas
such as governance, privatization and new public management. The apparent success of many
of these privatizations and partnerships in increasing operational efficiency and solving
certain problems has helped spread these ideas around the world through organizations such
as the World Bank.
The empirical literature concerning the matter of public versus private ownership can be
divided in to two categories by way of methodology.28 The first category, generally made up
of earlier studies, compared the performance of SOEs to that of private enterprises in
comparable fields and over time. This approach generally suffers from the methodological
problem of differentiating the ownership effects from other differences that invariably exist
between different enterprises.29 The World Bank explains the second category thus:
“More recent work compares the performance of SOEs before and after divestiture,
divested with undivested SOEs, or divested firms with a hypothetical counterfactual in
which the same firm is assumed to continue under state ownership.”30
The rest of this section will be devoted to the methodology and conclusions of this second
category of empirical privatization research. In the following summary of empirical work
from this category of privatization analysis I have attempted to account for the studies and
authors that I found to be widely referenced in empirical studies of privatization.
27
Peters & Pierre (2010)
The World Bank (1995) p. 37
29
The World Bank (1995) p. 37
30
The World Bank (1995) p. 37
28
14
One of the most comprehensive studies of post-privatization performance of former SOEs
across several fields and nations is the one presented by Megginson et.al in The Financial and
Operating Performance of Newly Privatized Firms. Megginson et.al were able to compare the
pre- and post privatization performance of 61 former SOEs in 18 countries privatized between
1961 and 1990. The results of this study are relatively uniform and paint a clear picture of the
effects on the privatized firms.31 Megginson et.al measured several aspects of the 61 former
SOEs such as profitability, efficiency, capital investment spending, output and employment.
Statistical analysis of these categories showed significant increases in nearly all of the above
categories.32 The change in the individual categories is listed below.

Profitability: 69.1 percent of the firms showed expanding profit margins and the
median increase of the benchmark measure of profitability was 2.49 percent.

Efficiency: The median increase of sales per employee was 12 percent and the median
increase of net income per employee was 25.1 percent.

Capital Investment Spending: The median increase of the benchmark measure, capital
expenditures divided by sales, was 5.21 percent.

Output: 75.4 percent of the firms experienced increases in sales and the median
increase was 24.14 percent.

Employment: 64 percent of firms showed increased number of employees and the
median increase was 276 employees. This result was not statistically significant.
In a survey of the theoretical and empirical literature on the differences between public and
private ownership, Shirley & Walsh set out to answer a number of questions, one of which
regards how SOEs and private firms are affected by market structure and how this may affect
privatization outcomes. Performance of firms was measured in terms of operational efficiency
defined as “the present value of outputs from a given set of inputs.” 33 They find that while
both market structure and ownership affect performance, private firms outperform public ones
even in competitive markets. In fact, the efficiency increase as a result of privatization is more
pronounced in a competitive environment.34 Shirley & Walsh explain that:
31
Megginson et.al (1998) p. 47f
Megginson et.al (1998) p. 77ff
33
Shirley & Walsh (2001) p. 5
34
Shirley & Walsh (2001) p.10
32
15
“The empirical literature suggests that while market structure has a positive impact
performance, this impact fails to dominate the ownership effect. The argument that
market-structure dominance rests on cases in which public and private firms in
competitive environments perform equally well, and these cases are rare.”35
In an overview of influential literature on privatizations of SOEs Jordahl presents relatively
uniform conclusions regarding the effects of privatization on factors such as efficiency,
productivity, profitability, output and debt. The only ambiguity regarding the studies that
Jordahl examines stems from the effects of privatization on employment levels in SOEs. The
results are especially conclusive when data is available from both before and after a
privatization:
“Nearly all studies that assess the same company before and after a privatization find that
profitability, efficiency, output, and investments increase while accumulation of debt
decreases. How employment at the privatized firms is affected is unclear.”36 (My
translation)
A more in depth empirical study of privatizations is Galal et.al’s Welfare Consequences of
Selling Public Enterprises. In the study, Galal et.al study privatizations in the United
Kingdom, Chile, Malaysia and Mexico in order to answer the question as to whether a country
is better or worse off following a privatization of an SOE. In order to discern whether the
privatization was beneficial they compare a number of factors relating to the SOEs
performance five years’ prior and five years post privatization. They then compare the factual
privatization scenario with a counterfactual non-privatization scenario in order to identify any
differences that can be traced to the privatization. The factors used to measure the pre- and
post-privatization performance of SOEs are: productivity, capital, consumer surplus, two
measures of profitability and other returns.37
The empirical research paints quite a uniform picture of privatization resulting in an increase
of an SOE’s operational efficiency. Indeed, the World Bank concludes that: “We have
35
Shirley & Walsh (2001) p. 11
Jordahl, (2008) p. 23
37
Galal et.al (1994) p. 4ff
36
16
presented evidence that SOEs are less efficient than the private sector in competitive markets
and, with effective regulation of private firms, in monopoly markets as well.”38
Besides the similarities of the conclusions drawn in the above presented empirical studies
there is another glaring similarity, namely that of methodology. The above-described studies
all measure the efficiency of the privatization in terms of some sort of comparison between
pre- and post-privatization SOE performance. In fact even the measurements used to assess
this performance are relatively consistent. For example all of the studies presented above use
some form of measurement of productivity, nearly all measure output, profitability and
investment, while some also measure employment. The authors whose work is detailed above
refer to this work as empirical study of privatization but I believe that a different, more
detailed label can shed more light on what has actually been done. It is my view that what is
described above, as empirical study of privatization is in fact evaluation; consider the
following definition supplied by Vedung. According to Vedung, evaluation is to
systematically, post factually, assess the value of an intervention by measuring its effects with
the purpose of providing practical guidance going forward.39 By this definition follows that
post factually assessing the effects on performance of a privatization in order to assess the
viability of privatizations in general is a perfect example of evaluation. Labeling the above
presented empirical studies as evaluation raises the specter of evaluation theory and the
analytical insight it can provide in the analysis of the above detailed empirical studies.
The realization that the above-presented empirical studies are in fact evaluations gives rise to
questions of evaluation theory. Shadish et.al specify five principles and practices that a good
theory of evaluation should give comprehensive account of. I would like to focus on the first
one which is “What methods to use to produce credible knowledge;”.40 In terms of the abovepresented studies; does the measurement of pre- and post-privatization SOE performance
produce credible knowledge about the efficiency of privatizations? Rossi et.al refer to this
aspect of evaluation theory as construct validity. The concept of construct validity is defined
by Rossi et.al as “the extent to which the variables used to measure program constructs
convincingly represent the constructs in the program logic model”.41 A program logic model,
as mentioned in the citation by Rossi et.al, is a causal model that is designed to plausibly
38
The World Bank. (1995) p. 50
Vedung (2009) p. 19-33
40
Donaldson & Lipsey (2006) p. 59
41
Rossi et.al (2004)
39
17
explain how a program is intended to achieve desired outcomes.42 In other words construct
validity in this case pertains to the capacity to generalize measurements such as productivity
and output to the desired outcomes of the privatization. The concept of construct validity
thereby not only stresses the importance of assessing an evaluations operationalization, it also
defines the yardstick against which to assess these operationalizations, namely the program
logic model, or in other words the goals or purpose of the program itself. 43
It is with basis in this principle of evaluation theory that empirical studies of privatization
need to be examined. For while the previously detailed parameters do provide significant
information about the performance of the SOE I will argue that they are an insufficient base
on which to evaluate a privatization and its efficiency. As I will argue in the following
section, there is much more to efficiency than simply measurements of performance. The
above-mentioned parameters will form the basis for a subsequent evaluation of the efficiency
of the privatization of Apoteket AB. This evaluation will provide a baseline that will serve as
a comparison to a second evaluation using a different definition of efficiency. The efficiency
definition based on the parameters of productivity, output, profitability, investment and
employment will henceforth be referred to as operational efficiency. The prefix operational is
not used in this context in order to convey any additional meaning but simply to distinguish
this definition of efficiency from the one developed in the following section.
An Alternative View of Efficiency
As I have just shown significant weight is given to the measurement of pre- vs. postprivatization efficiency in the above-detailed privatization literature. In this capacity the term
efficiency, occasionally associated with the prefix operational, is often used as a synonym for
productivity. In this section I will argue that this measure of efficiency provides inadequate
information on which to base an evaluation of SOE privatization.
Efficiency is a complicated term with several distinct, as well as changing, meanings and there
is even relatively abundant literature devoted entirely to research on the subject. Despite the
inherent complexity I will attempt to keep the terminological discussion simple.
42
43
McLaughlin & Jordan (1999) p. 2
Rossi et.al (2004)
18
Despite the confusion surrounding efficiency, there is some degree of fundamental agreement;
efficiency is the relationship between a cost and achieved change.44 A cost (often in terms of
resources and time) is made in order to achieve something (such as manufacturing a car) and
the efficiency is the relationship between the cost and the car. Since resources aren’t endless,
there needs to be some allocation and some form of measurement as to the relative attainment
of the invested resources. Unfortunately efficiency, even in a relatively straightforward
process such as manufacturing, is rarely as simple as productivity. In other words the number
of cars that can be made from a given set of resources is not the only relevant measurement;
productivity only becomes relevant when it is compared to the quality of the output. 45 E.g. it
doesn’t matter how many cars are made if they aren’t roadworthy. Therefore any serious
attempt at measuring efficiency also needs to take into consideration to what extent set goals
are met.
The connection between efficiency and goal fulfillment is echoed by Vedung who describes
the definition of efficiency as: “The programs capacity to produce predicted outcomes that
fulfill the goals of the program (…)”46 From this definition follows that the outcomes that are
of interest when measuring efficiency must be produced and predicted by the intervention
program and also serve to fulfill the goals of the intervention; or in other words. When
analyzing a private sector business the distinction between the use of goals and operational
efficiency may not yield any significant differences as one could argue that the goals of such
businesses can be more easily defined in terms of economics measurements. Not so for the
public sector; the state as owner of an SOE has many concerns and obligations that may not
necessarily, for better or worse, coincide with productivity or profit maximization. 47
Essentially an evaluation based on operational efficiency runs the risk of becoming an
evaluation of the former SOE and not an evaluation of the privatization.
Unfortunately the realization that goals need to be accounted for creates more questions than
it answers:
A. Whose goals are important when measuring the success, or lack thereof, of the
privatization of an SOE?
44
Johansson, (2011) p. 9
45
Oskar Johansson (2011) p. 8-25
Evert Vedung (2009) p. 31
47
Lindgren (2009) p. 48
46
19
B. Are there any formalized goals, and if so what are they?
C. If there are formalized goals, how do we measure them?
These questions will be addressed in the following section using my chosen case of Apoteket
AB as an example.
Whose goals are important when measuring the success, or lack thereof, of the
privatization of an SOE?
A: Whose goals to measure performance against, is not necessarily obvious. In fact, when a
large SOE such as Apoteket AB is to be privatized a large number of actors are going to have
differing goals and visions for the privatization, official or otherwise. In the case of the
privatization of Apoteket AB, dozens of organizations, both public and private, were involved
directly with the process of shaping the particulars of the privatization. 48 Add to that the
significant influence of evaluators and investigators and the picture of who in fact is in charge
of the privatization process becomes muddled indeed. Ultimately though, the government as
the owner formally has the final authority to privatize or not to privatize an SOE. One could
argue that government ownership is simply a proxy for ownership by the citizenry but since
the citizens are not in the business of formalizing specific goals it would be extremely
difficult to achieve any measure of validity when operationalizing these hypothetical goals.
On the one hand one could argue that government does not necessarily have the interests of
the citizenry in mind. On the other hand, whether the government actually has its citizens’
interests in mind is not necessarily relevant. What is important is whether the official goals
reflect the interests of the citizens and this seems quite likely, as any potential ulterior motives
are not likely to be displayed to the general public.
Ultimately it is the government that will take the decision as to whether or not to privatize an
SOE and therefore the measuring criterion of privatization success should reasonably have the
government’s goals in mind. Simply measuring operational efficiency does not do that
accurately. In this case an increase in value of output for a given set of inputs could for
example be the result of a price increase, effectively shifting costs from business owners to
consumers.
48
Prop 2008/09:145 (2009)
SOU 2008:4 (2008)
20
Are there formalized goals and if so what are they?
B: The existence of formalized goals and their explicitness is likely to vary from case to case
but it seems unlikely that something as major as the privatization of an SOE would be
conducted entirely without goals or expectations. Indeed the privatization literature gives
plenty of examples of varying goals for privatizations around the world.49
In the case of Apoteket AB there were two clearly formalized goals for the privatization:
1. Increased consumer access
2. Lower prices of pharmaceuticals
Increased consumer access
B1: What is implied by increased consumer access is not self-evident but public documents
relating to the privatization effort give a more in-depth description. The factors mentioned
that explicitly relate to consumer access to pharmaceuticals are: service hours, number &
proximity of pharmacies and level of service.
The goal to increase service hours is relatively simple and straightforward. It was assumed
that a privatization of the pharmaceuticals market and the diversity and competition that
would follow would cause pharmacies to have longer service hours than was experienced
during the state monopoly as each owner attempts to improve its market share.50
In regards to the number of pharmacies it was assumed that the increased competition and
productivity that the privatization created would enable the establishment of additional
pharmacies. On the other hand, there was some concern that the sparsely populated nature of
certain parts of Sweden would make those areas undesirable to retailers. Prior to the
privatization the state monopoly was obliged to provide access to retail pharmaceuticals in all
parts of the country and there were concerns that a removal of this obligation would
jeopardize access to retail pharmaceuticals in sparsely populated areas. 51 Therefore it was
decided that one of the conditions for private investors to be allowed to acquire a cluster of
49
Shirley & Walsh (2001)
Jordahl (2008)
Megginson et.al (1998)
50
Prop. 2008/09:145 (2008) p. 80f
51
Dir. 2006:§136 (2006) p. 5
21
pharmacies was that all pharmacies in the cluster were to remain operational for at least three
years when terms would be renegotiated.52
The wording used in the formulation of this goal is interesting in the sense that the rural and
urban areas are held to different standards. On the one hand the goal is to increase the number
of pharmaceuticals retailers but on the other hand the goal is also to “safeguard” the access to
pharmaceuticals in sparsely populated areas.53 The nature of the word safeguard (Swedish:
trygga) implies protecting or preserving. This implies that the deciding factor for rural areas
may not be the number of pharmacies as much as guaranteeing that there is at least one
pharmacy that can be considered reasonably accessible.
As opposed to the other two factors of consumer access, level of service is quite ambiguous.
Upon closer examination of public documents it becomes clear that level of service pertains
primarily to the ability of the pharmaceuticals retailers to live up to the obligation of supply
that applied to the state monopoly. Apoteket AB was required to supply all nationally and
European Economic Area (EEA) approved medication to consumers within 24 hours.54 There
was special concern that this capacity may be threatened primarily in rural populated areas.55
Lower prices of pharmaceuticals
B2: There are two distinct parts of the goal of lower prices on pharmaceuticals: the cost to
consumers and the cost to government.
Part of the basis for the goal of lowered prices of pharmaceuticals is the fact that prior to the
privatization the prices paid to manufacturers by Apoteket AB for pharmaceuticals
(Manufacturer to Pharmacy Price (MPP)) with active patents was the second highest in the
OECD.56 This was seen as a significant problem as active patent pharmaceuticals are by far
the largest cost driver, accounting for approximately 70 percent of the value of all sales.57 It
was assumed that private businesses would be more effective at negotiating the price of these
pharmaceuticals and that this would create the possibility for reduced pharmacy-to-consumer
52
53
54
55
56
57
WP/PM 2011:49 (2011) p. 18
Dir. 2006:§136 (2006) Page 1
Moïse & Docteur (2007) p. 43
SOU 2008:4 (2008) p. 477f
SOU 2008:4 (2008) p. 33
SOU 2008:4 (2008) p. 33f
22
prices (PCP) through lowered MPP.58 While the goal to reduce the prices of pharmaceuticals
does not specify at what stage of the process from manufacturing to retail it aims to reduce
prices, the fact that the supposed beneficiaries are both consumers and government leads me
to believe that PCP is the price that is to be lowered. But while reducing the price is all well
and good, a reduction of price is not the actual purpose but the means to an end. In clearer
terms this means that the goal is not to in fact necessarily to reduce the prices paid but to
reduce the aggregate cost. Any reduction in price that does not reduce the actual cost is
entirely meaningless and could for example be the result of deflation or other circumstances.
But that is not to say that price information is entirely useless as it may prove useful as part of
an analysis of cost fluctuation.
The costs for consumers and government for retail pharmaceuticals are connected through a
system of cost sharing. Consumers pay for all costs of prescription medication up to a cap
after which government pays for all subsequent costs. Non-prescription medication on the
other hand is generally not subsidized and the consumer pays in full. 59 This system has
historically resulted in the consumer paying for approximately 30 percent of the total costs of
retail pharmaceuticals with the state and local governments paying the remainder of the
costs.60 As the cost-sharing scheme does not apply equally to all types of medication this
creates a situation where primarily consumers enjoy any cost reduction in non-prescription
medication. Any cost reductions in prescription medication will primarily benefit government
as consumers already above the cap make the majority of the purchases of prescription
medication from retailers.61
How do we measure the fulfillment of the formalized goals?
C: How to measure the goals of the privatization is important to determine in order to reliably
assess the efficiency of the privatization. In order to create simplicity and structure my chosen
assessment parameters will culminate in a number of hypotheses that will aid the process of
analyzing the empirical results.
Increased consumer access
58
SOU 2008:4 (2008) p. 37
Moïse & Docteur (2007) p. 15
60
SOU 2008:4 (2008) p. 32f
61
SOU 2008:4 (2008) p. 111
59
23
C1: The goal of increased consumer access has three parts: service hours, number and
proximity of pharmacies, and level of service. The varying nature of these factors does not
lend itself well to assessment as a whole, rather I believe each factor needs to be assessed
individually and the results compiled in to a general evaluation of goal fulfillment.
Measuring potential changes in service hours of pharmacies across the country pre- as well as
post-privatization should satisfactorily assess the goal fulfillment of increased service hours of
pharmacies. Detailing any potential change in service hours for all pharmacies individually is
obviously quite unwieldy. With this in mind an average change value, if any, seems adequate
to assess goal fulfillment. Data detailing the average service hours of retail pharmacies preand post-privatization was obtained from the Swedish Pharmacy Association. For the goal of
increased service hours to be fulfilled it is my assessment that the average service hours
should have increased as a result of the privatization. This leads us to the first hypothesis:
#1 H0: The average service hours of pharmacies have not increased post privatization.
Number and proximity of pharmacies has two parts. The general number of pharmacies can
be assessed with relative ease; it is simply a matter of collecting numerical data from before
and after the privatization and comparing the two. But in order to attribute an increase in the
number of pharmacies to the privatization we also need to put any change in the number of
pharmacies in relation to any trend in the number of pharmacies that might have existed prior
to the privatization. For example, if the number of pharmacies in Sweden was growing by an
average of 50 pharmacies per year prior to the privatization, a continued increase of 50
pharmacies per year post privatization would not be attributable to the privatization. For the
goal of an increased number of pharmacies to be fulfilled it is my assessment that the
aggregate number of pharmacies must have increased post privatization when compared to the
projected change of the years leading up to the privatization. Data detailing the number of
retail pharmacies for the various years was obtained from two separate sources. Apoteket AB
supplied the pre-privatization data while the post-privatization data was obtained from The
Medical Products Agency. This leads us to our second hypothesis:
2# H0: The aggregate number of pharmacies has not increased post-privatization compared to
the average change in the number of pharmacies of five years prior to privatization.
24
The question of proximity on the other hand is a different matter. In order to measure the goal
fulfillment of safeguarding the access to pharmaceuticals in rural areas the data needs to
reveal the number of pharmacies located in these areas before and after privatization.
Unfortunately the number of pharmacies in rural areas alone does not necessarily accurately
portray the average citizens access to pharmacies, as they may not be spread evenly across
these areas. To account for this problem I will be using a measure the number of people that
have to travel more than a certain amount of time by car in order to access a pharmacy. This
measurement is based on an area type model created by the Swedish Rural Agency (Swedish:
Glesbyggdsverket). This model has been used in numerous studies in order to describe
changes in access to various services and markets. 62 The data detailing travel time to the
nearest pharmacy based on the area type indexation model was obtained from The Swedish
Agency for Growth Policy Analysis. In addition to this there is also the matter of measuring
the relative vulnerability of this access. One way of assessing the vulnerability of consumer
access to pharmaceuticals would be to measure what effect the closing of the nearest
pharmacy would have on the required travel time. In other words, a small difference between
the distance to the nearest pharmacy and the second nearest pharmacy would indicate low
vulnerability and vice versa. Again, data for all these factors needs to be acquired from before
and after privatization and trended where necessary. The data detailing the change in travel
time that would result from a hypothetical closing of the nearest pharmacy was also supplied
by The Swedish Agency for Growth Policy Analysis. For the goal of increased pharmacy
access in rural areas to be fulfilled it is my assessment that the number of pharmacies in rural
areas needs to have increased post privatization when compared to the projected change of the
years leading up to the privatization. It is also my assessment that the average travel time to
the nearest pharmacy should have decreased as a result of the privatization and that the travel
time to the second nearest pharmacy should have decreased as well. This leads us to our third,
fourth and fifth hypotheses:
3# H0: The number of pharmacies located in rural areas has not increased post privatization
compared to the average change in the number of pharmacies of five years prior to
privatization.
62
WP/PM 2011:49 p. 12f
25
4# H0: The average travel time required to reach the nearest pharmacy has not decreased post
privatization compared to the average change in the number of pharmacies of five years prior
to privatization.
5# H0: The average travel time required to reach the second nearest pharmacy has not
decreased post privatization compared to the average change in the number of pharmacies for
five years prior to privatization.
Level of service is a quite intangible concept that needs to be focused into a defined
measurement that lends itself to assessment more easily. The goal of increased level of service
is not explained extensively in public documents but seems to pertain primarily to the
capacity of retailers to deliver all nationally and EEA approved medication to consumers
within 24 hours. For obvious reasons detailing the success rate of medication availability
within 24 hours of every pharmacy individually is impractical. Therefore it is my assessment
that an average success rate of availability of sought medication within 24 hours post
privatization compared to the pre privatization equivalent would be the best way to measure
the fulfillment of this goal. The data detailing the success rate of delivery of sought
prescription pharmaceuticals at the first visit pre-privatization was supplied by the Dental and
Pharmaceuticals Benefits Agency and post-privatization data by the Swedish Pharmacy
Association. This leads us to the sixth hypothesis:
#6 H0: The average success rate of delivery of sought prescription medication within 24 hours
has not improved post privatization.
Lower prices of pharmaceuticals
C2: There are two distinct parts of the goal for lower cost of pharmaceuticals: the cost to
consumers and the cost to government. In order to gather as much information as possible
from which to assess the fulfillment of the goal of lower cost of pharmaceuticals it is prudent
to examine both prescription and non-prescription medication. In order to assess the change in
the cost of pharmaceuticals to consumers and government the pre-privatization cost trend
must also be taken in to consideration.
Unfortunately there are some problems trending the costs of prescription medication. Because
of the fact that a significant amount of the cost of pharmaceuticals can be derived from
26
pharmaceuticals with active patents, the expiration and introduction of patents on
pharmaceuticals with widespread use cause large fluctuations in the aggregate cost of
pharmaceuticals. The cost-sharing scheme causes this effect to have major impact on the costs
for government but a much lesser impact on the cost for consumers as consumers of
expensive patented pharmaceuticals will quickly reach the consumer payment cap. The nature
of this effect means it does not lend itself well to trending as it occasionally causes large
fluctuations from one year to the next. Fortunately, while this does lend a certain degree of
uncertainty to the cost development results, patent introduction or expiration dates are not
complete unknowns and should be able to be accounted for to some degree when the
empirical data is analyzed.
In order to fulfill the goal of reduced pharmaceuticals costs it is my assessment that the costs
of non-prescription and/or prescription medication should show reduction compared to the
trend of the past few years. The data pertaining to cost development of pharmaceuticals preprivatization was obtained from Official Reports of the Swedish Government while the postprivatization data for aggregate pharmaceuticals cost was obtained from Pharmaceutical
Service AB while non-prescription cost data was obtained from The National Board of Health
and Welfare. Pharmaceuticals consumption data was supplied by Pharmaceutical Service AB.
This leads us to the seventh and eighth hypotheses:
7# H0: The costs of prescription pharmaceuticals have not decreased post privatization
compared to the average change of the years previous to the privatization.
8# H0: The costs of non-prescription pharmaceuticals have not decreased post privatization
compared to the average change of the years previous to the privatization.
What, besides goal fulfillment, is needed in order to assess efficiency?
An SOE, or any other business for that mater, does not operate in a vacuum, there are also
societal costs associated with privatization of SOEs. While the degree of interference in
private business may vary, nearly every political system is likely to have some form of
government supervision of private business. Regulation and supervision are likely to increase
with a decrease in direct government involvement in favor of private alternatives. 63 This
63
Jessop (2010) p. 64ff
27
supervision, regardless of its scope, is not free and if it is likely to increase. This cost increase
should be accounted for when measuring the general success of the privatization.
Methodology – Measuring the Effects of Privatization, Dual
Approaches
The hypotheses pertaining to increased consumer access are, as demonstrated in the section
above, the following:
#1 H0: The average service hours of pharmacies have not increased post privatization.
#2 H0: The aggregate number of pharmacies has not increased post privatization compared to
the average change in the number of pharmacies during three years prior to privatization.
#3 H0: The number of pharmacies located in rural areas has not increased post privatization
compared to the average change in the number of pharmacies during three years prior to
privatization.
#4 H0: The average travel time required to reach the nearest pharmacy has not decreased post
privatization compared to the average change in time required during three years prior to
privatization.
#5 H0: The average travel time required to reach the second nearest pharmacy has not
decreased post privatization compared to the average change in travel time during the three
years prior to privatization.
#6 H0: The average success rate of delivery of sought medication within 24 hours has not
improved post privatization compared to pre-privatization.
And the hypotheses pertaining to the goal of lowered prices of pharmaceuticals are:
#7 H0: The costs of prescription pharmaceuticals have not decreased post privatization
compared to the average change in costs during years previous to the privatization.
28
#8 H0: The costs of non-prescription pharmaceuticals have not decreased post privatization
compared to the average change in costs during years previous to the privatization.
Input
While the above hypotheses are designed to measure goal fulfillment there is another
parameter that needs to be taken into consideration in order to measure the efficiency of the
privatization and that parameter is input. As per the efficiency definition in the previous
section, goal fulfillment needs to be contrasted to the size of the cost that was paid in order to
achieve the stated goals. The primary sacrifice that was made to achieve the privatization
goals was in terms of increased workload for state agencies and a resulting increase in costs.
There are three state agencies which were deemed to be directly affected by the privatization
of the Swedish pharmaceuticals monopoly (Swedish translations in parentheses): The Medical
Products Agency (Läkemedelsverket), The Dental and Pharmaceutical Benefits Agency
(Tandvårds- och läkemedelsförmånsverket) and The National Board of Health and Welfare
(Socialstyrelsen).64
The Medical Products Agency was deemed to have to bear the brunt of the workload increase
resulting from the privatization. Among other new tasks, the new process of issuing licenses
to prospective retailers was deemed a significant burden:
”With the suggested system for licensing application, notification of pharmacy
establishment and notification of changed circumstances, The Medical Products Agency
will need to administer and assess these applications and notifications. This will be a
completely new task for The Medical Products Agency which, depending on when and to
what extent new retailers seek to establish themselves on the market, will require
considerable resources both in terms of manpower and restructuring.”65 (My translation)
In addition to the above-mentioned costs the privatization was deemed to require significant
increases in supervision of pharmacies as well as of pharmaceuticals imports. Many of the
tasks that the agency performed prior to the privatization and continues to perform post
privatization also needed to be restructured. All of The Medical Products Agency’s cost
64
65
SOU 2008:4 p. 581ff
SOU 2008:4 p. 582
29
increases will however be financed through fees imposed upon the private retailers and will
therefore not burden government spending.66
The Dental and Pharmaceutical Benefits Agency on the other hand has its day-to-day
operation financed by the state government and it was also projected to receive an increased
workload.67 The agency was primarily tasked with monitoring and to some extent evaluating
certain aspects of the systems for exchanges of pharmaceuticals with generic competition and
imports of specialty pharmaceuticals. The primary task of the agency, which was to organize
the prescription pharmaceuticals price cap scheme, will remain unchanged. This will keep the
bulk of the costs for the agency unchanged. Apart from certain one-off costs, which will be
discussed below, the agency was to receive an increase to its annual budget of SEK 5-10
million (USD Approx. 1.1 million).68
The National Board of Health and Welfare has the supervision of pharmacies as its primary
responsibility in relation to the pharmaceuticals market. The expected increase in the
workload was explained as follows:
“The pharmaceuticals market is not seen by the agency as a large patient safety risk but
the large quantities of pharmaceuticals that are handled still require certain resources to
be allocated in order to ensure safety. The supervision is more dependent on the
organizational form of, rather than on the sheer number of, pharmacies. Apoteket AB had
a well functioning safety system functioning identically across the country which allowed
supervision to be systematized.”69
In order to cover the additional costs associated with the complications to the supervision that
the privatization entailed, an annual budget increase of SEK 4,5 million (Approx. USD 0,65
million) was deemed sufficient.70
As mentioned above there were certain one-off costs associated with the privatization. These
primarily pertained to such things as evaluations, investigations and the reorganization of
66
SOU 2008:4 p. 582
SOU 2008:4 p. 582f
68
SOU 2008:4 p. 582f
69
SOU 2008:4 p. 584
70
SOU 2008:4 p. 584
67
30
certain public service functions performed by Apoteket AB. According to performed analyses
these costs were in the area of SEK 100-150 million (USD Approx. 17,85 million).71
Aggregately these costs amount to an annual cost increase of approximately SEK 12 million
(USD 1,7 million) and a one-off cost of approximately SEK 125 million (USD Approx. 17,85
million). But importantly the privatization of the pharmaceuticals market did not only
generate costs, the privatization included the sale of a significant number of pharmacies and
relating infrastructure divided into eight clusters. The aggregate sales revenue of these eight
clusters was approximately SEK 6 billion (USD Approx. 860 million).72 Subtracting the oneoff costs from the aggregate sales revenue leave SEK 5 billion 875 million to cover the annual
costs of the privatization. The remaining sales revenue is adequate to cover the annual cost
increase of approximately 12 million for approximately 490 years. Alternatively an inflation
adjusted interest rate of only 0.2 percent on the remaining sales revenue would suffice in
order to cover the increase in annual costs.
Sources and Collection of Data
Testing the previously formulated hypotheses will require a significant amount of data from
several points in time. The magnitude and spread of the data required to test these hypotheses
would require an inordinate amount of time and effort to collect personally and would likely
prove incompatible with the restrictions placed on this thesis in terms of available time. For
example the data pertaining to the travel time to the nearest pharmacy for well over nine
million individuals at several different points in time would be virtually impossible to
measure personally within the scope of this thesis. Fortunately the nature of this data is such
that much of it can be mined from official statistics available from public sources. Performing
secondary analysis of official statistics as opposed to personally collecting the required data
provides the opportunity to test the hypotheses within the scope of this Masters thesis.
Besides saving time there are other benefits to secondary analysis of official statistics, not the
least of which is the quality of the data. The data available in official statistics is generally of
such quality as invariably requires resources to collect that are far beyond what can be
expected from personal data-collection. 73 For example some of the data used in this thesis
71
SOU 2008:4 p. 586
Riksrevisionen (2011)
73
Bryman (2008) p. 297
72
31
was obtained by agencies dealing exclusively with data collection with the authenticity of the
collected data guaranteed through the threat of criminal persecution. It would for example not
be feasible to collect data regarding the aggregate sales value of non-prescription
pharmaceuticals individually from every pharmacy in the nation and compile it personally.
The primary problem with using secondary data is that I can not personally guarantee the
authenticity of the data in the same way as if I had collected it personally. And while the
quality or authenticity of secondary data should never be taken absolutely for granted I
believe that the quality of the data acquired to test the hypotheses of this thesis can sensibly be
quite assured.
Many of the hypotheses also require longitudinal analysis. In other words the testing of many
of the hypotheses requires comparable data from several points in time to be compared. When
comparing data from two or more points in time it is important that the two data points are of
comparable quality and I have in these cases, where possible, compared annual data from the
same source. Where pre- and post-privatization data was not available from the same source I
have done my best to ensure that the data was collected using comparable methods and
definitions.
Data used to measure changes in operational efficiency
The data gathered in order to assess the efficiency of the pharmaceuticals market post
privatization has been obtained from the available annual reports of the largest actors on the
reregulated pharmaceuticals market. Considering the number of minor actors with small
market shares the increased workload and cost of examining annual reports from all market
actors would far outweigh the benefit of the added data. The five largest actors in the
reregulated retail pharmaceuticals market account for 90.02 percent of the market measured as
percentage of the number of total pharmacies, which should be sufficient in order to assess
any significant changes in efficiency. This is especially true considering the privatization
method of sales of clusters, which should significantly decrease the possibility of
geographical or other biases as the clusters were designed to spread pharmacies both in terms
of output and geography.
The parameters that were previously determined as pertaining to the operational efficiency of
the privatization are productivity, output, profitability, investment and employment. In order
to measure productivity I have chosen to use two different measures, output sales value
32
divided by the number of employees and operating profit divided by the number of
employees. In order to measure output I have used output sales value before tax. In order to
measure profitability I have used operating profit (EBIT), defined as sales revenue minus cost
of goods sold and all expenses except for interest and taxes. The measurement of investment I
have chosen only takes into consideration investment spending. This was done as SOE sales
of pharmacies to private actors generated excessive revenue that would obscure any
difference in investment spending. Employment was measured as the average number of
employees for the given year.
As the retail pharmaceuticals market was reregulated in July 2009 the data for that year is not
usable as the data gathered in order to assess the operational efficiency is based on full years.
33
Empirical Results, Goal Fulfillment – Pre- and Post-Privatization
Data
#1
The service hours of retail pharmacies under the monopoly were relatively uniform with most
pharmacies being open between 10:00 and 18:00 weekdays. A smaller number of pharmacies
were open on Saturdays but then only until 14:00 with no retail pharmacies open for business
on Sundays. Approximately 10 percent of all pharmacies were closed during the summer
months. 74 According to the Swedish Pharmacy Association the service hours of retail
pharmacies prior to the privatization correspond to an average of 43hrs per week.75
According to the Swedish Pharmacy Association the average service hours for retail
pharmacies in 2011 was 53hrs/week.76
The first hypothesis regarding the goal fulfillment of increased consumer access to
pharmaceuticals was average service hours of pharmacies. The empirical results show that
pharmacies were open an average of 43hrs per week prior to privatization. After privatization
this increased to as much as 53hrs per week corresponding to an increase in service hours of
approximately 23 percent. H0 for #1; the average service hours of pharmacies have not
increased post privatization, can therefor be rejected in favor of H1.
74
Bengtsson (2006) p. 15
Sveriges Apoteksförening (2011) p. 22
76
Sveriges Apoteksförening (2011) p. 22
75
34
#2
Figure 1. Number of retail pharmacies. Y Axis = Number of retail pharmacies, X Axis =
Year and growth rate
Number of retail pharmacies
1500
1000
500
0
2006
2007,
2.6%
2008,
0.8%
2009, 2010, 2011,
2.0% 22,2% 8,7%
Figure 1 shows the number of retail pharmacies prior to the privatization of the state
monopoly as well as the annual growth rate. The growth rates of 2.64 percent, 0.78 percent
and 2.00 percent for the years 07, 08 and 09 respectively yield a average growth rate of 1.81
percent. There were 918 retail pharmacies in Sweden when the state monopoly was privatized
in 2009.77
According to the Swedish Pharmacy Association there were 1122 retail pharmacies in
Sweden in 2010 and according to the Medical Products Agency there were 1220 retail
pharmacies in March 2011.78
The second hypothesis regards the number of retail pharmacies in Sweden. The empirical
results show that there were 918 pharmacies at the time of privatization and that the average
growth rate of the number of retail pharmacies during the period leading up to the
privatization was 1.81 percent per year. For H0, the aggregate number of pharmacies has not
increased post privatization compared to the average change in the number of pharmacies of
the three years prior to privatization, to be justifiable the number of retail pharmacies for 2010
and 2011 should be in the vicinity of 935 for 2010 and 952 for 2011. The empirical results
77
Apoteket AB (2008) (1) p. 1
SOU 2008:4 (2008) p. 115
Läkemedelsverket (2011)
78
Läkemedelsverket (2011)
35
show that the actual numbers were 1122 for 2010 and 1220 for 2011. The actual numbers
correspond to an average growth rate of approximately 15.48 percent or approximately 8.56
times faster than prior to privatization. H0 for #2; the aggregate number of pharmacies has not
increased post privatization compared to the average change in the amount of pharmacies of
the three years prior to privatization, can therefor be rejected in favor of H1.
#3
When describing the number of pharmacies located in rural areas it is of obvious importance
to clearly define a rural area. For this purpose I will be using an indexation model developed
by the Swedish agency for growth policy analysis. The indexation model indexes areas by the
travel time required to reach a community of 200, 1000, 3000, 30000 and 60000 inhabitants
and assigns each area a number corresponding to its relative proximity between 0 and 500
where 500 represents the lowest proximity. 79 For the purpose of this study I will be
considering areas with a score of 200 or less as rural. While the indexation model is complex
and an index of 200 can correspond to a variety of different situations an index of 200 is
designed to be equivalent to a travel time of 45 minutes to a area of at least 30 000
inhabitants.
In 2009 the number of pharmacies located in rural areas as defined above was 176, which
represents approximately 19 percent of the total number of pharmacies for the same year.80
In 2011 the number of pharmacies located in rural areas was 193, 81 which represents
approximately 15.82 percent of the total number of pharmacies for the same year.
The third hypothesis pertaining to the goal of increased access was the number of pharmacies
located in rural areas. The number of retail pharmacies located in rural areas increased by 17,
or approximately 9.5 percent. H0 for #3; The aggregate number of pharmacies has not
increased post privatization compared to the average change in the number of pharmacies of
the three years prior to privatization, can therefor be rejected in favor of H1. It is worth noting
that while the number of pharmacies located in rural areas did increase the growth was slower
than in urban areas.
79
WP/PM 2011:49 p. 15
WP/PM 2011:49 p. 17
81
WP/PM 2011:49 p. 17
80
36
#4
When calculating fluctuations in the travel time to the nearest pharmacy one needs to keep in
mind that the full extent of any potential change in travel times may not be a result of retailers
establishing closer to the customers but of the customers relocating from areas with few
pharmacies to areas with many. For example urbanization or discrepancies in population
growth between rural and urban areas could create an inherent bias showing increased access
that is not a result of increased pharmacy proximity in rural areas. During the years leading up
to, and following, the privatization there is evidence to suggest that population growth has
been greater in urban areas.82
Table 1. Pre-privatization travel times to nearest pharmacy.
2009
Population
9 247 154
40 Min +
30 - <40
Min
20 -< 30
Min
10 - <20
5 - <10 Min
< 5 Min
8 006
21 683
107 421
897 275
1 485 288
6 727 481
Table 1 displays the aggregate population of Sweden for 2009 of approximately 9.2 million
individuals and divides them according to travel time to the nearest pharmacy. The table
shows that in 2009 the vast majority (≈89 percent) live within 10 minutes of a pharmacy.83
The average travel times to the nearest pharmacy were calculated by the Swedish Agency for
Growth Analysis using population data and the National Road Database (NVDB). The
database includes all public roads as well as speed limit information for these roads enabling
the calculation of travel time to nearest pharmacy.84
82
Statistiska Centralbyrån (2012)
WP/PM 2011:49 p. 23
84
WP/PM 2011:49 p. 14
83
37
Table 2. Post-privatization travel times to nearest pharmacy.
2011
Population
9 405 640
40 Min +
30 - <40 Min
20 -< 30 Min
10 - <20
5 - <10 Min
< 5 Min
7 398
20 334
99 275
872 135
1 425 701
6 980 797
Table 2 displays the population of Sweden for 2011 of approximately 9.4 million individuals
and divides them according to travel time to the nearest pharmacy. The table shows that in
2011 the vast majority (≈89.3 percent) live within 10 minutes of a pharmacy.85
Table 3. Travel Time Comparison
Population and Travel Time Fluctuations
2009
2011 % Change
Population
9 247 154 9 405 640
1.71%
40 Min +
30 - <40 Min
20 -< 30 Min
10 - <20
5 - <10 Min
< 5 Min
8 006
7 398
21 683
20 334
107 421
99 275
897 275
872 135
1 485 288 1 425 701
6 727 481 6 980 797
-7.59%
-6.22%
-7.58%
-2.80%
-4.01%
3.77%
The fourth hypothesis pertains to the average travel time to the nearest retail pharmacy. The
empirical results show the number of people in all travel time categories have decreased post
privatization with the exception of the less than five minutes category despite an overall
population increase of 1.71 percent.
During the period the percentage of individuals whose travel time exceeded 20 minutes
decreased from 1.48 percent to 1.35 percent, a decrease of approximately 8.9 percent. On the
other hand these results need to be interpreted with some caution as outside factors such as
depopulation of rural areas in favor of urban areas could contribute to these fluctuations.
85
WP/PM 2011:49 p. 23
38
While there is evidence suggesting that there is some discrepancy in population growth
between rural and urban areas during the period in question it is my opinion that H0 can be
rejected in favor of H1. The size of the decreases coupled with the evidence of increased
number of pharmacies in rural areas leads me to believe that privatization has indeed had an
effect on the average travel times to the nearest retail pharmacy. How large this effect may be
is on the other hand very difficult to tell.
#5
Unfortunately I was not able to obtain data regarding the travel time to the second nearest
pharmacy pre-privatization and I was therefore unable to test this hypothesis. Even though
there is nothing to compare the post-privatization data to I have detailed it below as some may
find it of interest. The government agency that collected and presented the post-privatization
data informed me via email on March 22 2012 that the pre-privatization data did not currently
exist and that there were no plans at the time to gather this data.
Table 4. Travel time difference between nearest and second nearest pharmacy.
2011
40+ Min
34 469
30 <40Min
38 783
20 <30Min
132 273
10 <20Min
853 860
5 - <10Min
936 084
<5 Min
7 410 171
Table 4 displays the resulting travel time increase if the closest pharmacy was unavailable or
closed.86
#6
According to the Dental and Pharmaceuticals Benefits Agency the degree to which sought
prescription medication was available at the customer’s initial visit to a retail pharmacy prior
to privatization was 96 percent.87
86
87
WP/PM 2011:49 p. 30
The Dental and Pharmaceuticals Benefits Agency (2011) p. 15
39
According to the Dental and Pharmaceuticals Benefits Agency the average degree to which
pharmacies were able to supply sought medication was 96 percent for original
pharmaceuticals and 94 percent for generic alternatives for 2011.88
The sixth and final hypothesis regarding consumer access was the average success rate for
delivery of sought medication. The empirical results show that prior to privatization the
average success rate was approximately 96 percent. By 2011 the success rate was 94 percent
for generic pharmaceuticals and 96 percent for originals. As generic pharmaceuticals account
for approximately 85 percent of aggregate pharmaceuticals sales in Sweden the rate of
successful delivery of sought prescription pharmaceuticals seems to have dropped, although
not dramatically. H0 for #6, the average success rate of delivery of sought medication within
24 hours has not improved post privatization compared to pre privatization it can therefore be
accepted with a small caveat. The caveat is that the statistics supplied by the Dental and
Pharmaceuticals Benefits Agency pertain to the success rate of delivery at the initial visit and
not, as per the definition of the law, successful delivery within 24 hours.
#7
Testing whether the cost of pharmaceuticals has changed as a result of the privatization is not
without certain inherent difficulties. In order to test for a potential change, an estimated cost
trajectory for a non-privatization scenario needs to be calculated. Unfortunately trending the
cost of pharmaceuticals is not as simple as it may seem. The system of compulsory
substitution of prescription medication with generic competition that was introduced in 2002
altered the annual growth of pharmaceuticals to such an extent that any years prior to the
change simply aren’t comparable. There is also the matter of accounting for changes in the
volume of pharmaceuticals consumed when analyzing aggregate cost data. Lastly there is the
issue of patent expiration and the introduction of so called “blockbuster” pharmaceuticals.
These “blockbuster” pharmaceuticals account for a large enough part of the value of
aggregate sales that their introduction or patent expiration can have dramatic effects on the
aggregate data. Ergo a specific year with many expirations and few introductions of
“blockbuster” pharmaceuticals may skew the results for the specific year.
88
The Dental and Pharmaceuticals Benefits Agency (2011) p. 14
40
Figure 2. Pharmaceutical retail sales value. Y Axis = Retail sales value billions of SEK.
X Axis = Year and growth rate.
Pharmaceutical Retail Sales in Sweden
(Billions of SEK)
40
30
20
10
0
Figure 2 shows the retail sales value of pharmaceuticals sold for human use in Sweden during
the period 2001 – 2011 and the percentage growth compared to the previous year for each
year individually. The sharp decline in sales value growth from 2002 and onward is thought to
have been greatly influenced by the introduction of mandatory substitution of pharmaceuticals
with generic competition.89 For the purpose of trending the potential sales value growth had
the pharmaceuticals market not been privatized the figures prior to 2002 are incomparable.
Therefor trending of pharmaceutical retail sales value must be based on the years 2003 –
2008. The growth percentages for those years were 2.2 percent, 2.5 percent, 2.8 percent and
5.1 percent 6.5 percent and 5.2 percent respectively90 which corresponds to an average yearly
growth rate of approximately 4.05 percent.
Meanwhile the consumption rate of pharmaceuticals also increased during the period leading
up to the privatization by approximately 2.5 percent.91 Unfortunately I have not been able to
find any data detailing whether there may be some inherent bias as to which types of
pharmaceuticals this growth may encompass. For example if growth was overrepresented in
pharmaceuticals with active patents as opposed to pharmaceuticals with generic competition a
89
Moïse & Docteur (2007) p. 32
SOU 2008:4 p. 104
91
Moïse & Docteur, (2007) p. 33
90
41
2.5 percent increase in consumption may lead to a larger than 2.5 percent increase in sales
value or vice versa. I have not been able to find any evidence to suggest that any inherent bias
of the above described nature may exist. While this lack of information is regrettable
consideration to consumption rates is necessary in order to proceed with any evaluation of to
what extent increased consumption is a cost driver.
While it is an inherently uncertain business to assess the impact of patent introductions and
expirations on the aggregate cost of pharmaceuticals for an entire year there is some evidence
to suggest that there were more major patent expirations than introductions during the period
2002 – 2005. This may have influenced the aggregate sales value of pharmaceuticals
negatively for this period resulting in lower sales value growth than would otherwise have
been the case.92 While introduction of new and expensive pharmaceuticals on the market is a
very real cost driver the factors of interest for the purpose of analyzing the privatization are
changes that can be directly attributed to the privatization. The aggregate sales value of
pharmaceuticals in Sweden in 2008 was 34.586 billion.93
The total retail sales value of retail pharmaceuticals in 2011 was SEK 36.7 billion.94 The
growth factors of 2.64 percent, 1.41 percent and 1.94 percent for the years 2009 - 2011 yield
an average growth rate of approximately 1.997 percent.
Table 5. Post Privatization Consumption growth (DDD/1000)
Year
Table 5 displays the consumption growth (DDD/1000
Change
2009
1.59% inhabitants) for each full year post privatization.95 The growth
2010
0.35% factors of 1.59 percent, 0.35 percent and 0.23 percent yield an
2011
0.23%
average growth rate of approximately 0.723 percent.
For the year 2010 there is some evidence that patent expirations had a dampening effect on
price increases with a relatively low corresponding introduction of pharmaceuticals. 96
92
SOU 2008:4 p. 106ff
SOU 2008:4 p. 104.
94
Apotekens Service AB (2011) p. 3
95
SOU 2008:4 p. 4
96
Lindhé et.al (2011)
93
42
The first hypothesis regarding the cost of retail pharmaceuticals was: the costs of prescription
pharmaceuticals have not decreased post privatization compared to the average change for the
years previous to the privatization. The empirical data shows that in 2008, the last full year
prior to privatization, the retail sales value of pharmaceuticals sold for human use in Sweden
was SEK 34.586 billion. The average growth of expenditures for the control period was 4.05
percent. Ceteris paribus the retail sales value of pharmaceuticals sold for human use in
Sweden in 2010 and 2011 should be in the vicinity of SEK 36.799 billion and SEK 37.985
billion respectively. The empirical data shows that the actual numbers for 2010 and 2011 were
approximately SEK 36.0 billion and SEK 36.7 billion corresponding to an average growth
rate of 1.997 percent. In other words the average growth was slowed by approximately 63
percent.
Regarding the consumption of pharmaceuticals the empirical results show that the period
prior to privatization saw an annual average increase in consumption of approximately 2.5
percent. For 2010 and 2011 the consumption growth rate was 0.35 percent and 0.23 percent,
which yields an average growth rate of 0.29 percent. Unfortunately I have not been able to
identify exactly to what extent the consumption increases are spread across different
categories or price ranges of pharmaceuticals. Assuming that the spread of consumption of
retail pharmaceuticals was evenly spread across different price ranges the average sales value
growth minus the consumption rate growth for the two periods is 4.05 - 2.5 = 1.55 pre
privatization and 1.997 – 0.29 = 1.707 post privatization.
As mentioned earlier, there is some indication that there were more major patent expirations
than introductions during the period 2002 – 2005, which may have influenced the preprivatization sales-value growth results negatively. The same is likely to be true for 2010.
It is my opinion that, taken together, the above results are insufficient in order to either accept
or reject H0: The costs of non-prescription pharmaceuticals have not decreased post
privatization compared to the average change of the years previous to the privatization.
#8
Trending of the cost for non-prescription medication is simpler than for prescription
medication. As non-prescription pharmaceuticals are not part of the mandatory generic
substitution system data from years prior to 2002 can be used for trending purposes. The
43
prevalence of patents on “blockbuster” medication is also lower for non-prescription
pharmaceuticals.
Figure 3. Sales value of non-prescription pharmaceuticals. Y Axis = Retail sales value
millions of SEK. X Axis = Year and growth rate
Pharmaceutical Retail Sales Value of nonprescription pharmaceuticals (SEK millions)
4000
3000
2000
1000
0
Figure 3 is based on data from the National Board of Health and Welfare and shows the cost
of non-prescription pharmaceuticals increasing steadily between 2002 and 2011.97 The preprivatization growth factors translate into an average growth rate of approximately 7.06
percent per year. For 2008 the aggregate cost of non-prescription pharmaceuticals in Sweden
was approximately SEK 3.103 billion.98
The consumption of non-prescription pharmaceuticals measured as DDD/1000 inhabitants
during the five years prior to privatization increased at an average rate of 1.53 percent per
year.99
Table 6. Consumption of non-prescription pharmaceuticals 2009 – 2011 (DDD/1000)
Year
Consumption Change
2009
-0.73%
2010
3.16%
2011
-1.20%
97
SOU 2008:4 Page 104.
Socialstyrelsen (2003-2012)
98
Socialstyrelsen (2003-2012)
99
Socialstyrelsen (2003-2012)
44
Table 6 displays the average growth in consumption measured as DDD/1000 inhabitants for
non-prescription pharmaceuticals between 2009 and 2011. The average growth rate was
approximately 0.41 percent100
The second hypothesis regarding the cost of retail pharmaceuticals was: the costs of nonprescription pharmaceuticals have not decreased post privatization compared to the average
change of the years previous to the privatization. The empirical data shows that the average
growth rate of retail sales value of non-prescription pharmaceuticals during the period prior to
the privatization was 7.06 percent per year and that the average increase in consumption of
pharmaceuticals was 1.53 percent per year. The aggregate cost for 2008 was approximately
SEK 3.103 billion. Ceteris paribus the retail sales value for the years 2010 and 2011 should
be in the vicinity of SEK 3.593 billion and 3.866 billion respectively in order for H0 to be
acceptable.
The empirical data shows that post privatization the growth rates for 2010 and 2011 were
approximately 8.75 percent and 2.18 percent respectively yielding an average growth rate of
approximately 5.47 percent. The aggregate sales values were approximately SEK 3.62 and
3.70 billion respectively corresponding well to the predicted 3.593 and 3.866. The growth rate
of consumption measured as DDD/1000 inhabitants was 0.41 percent post privatization. The
sales value growth rate minus consumption growth rate for the two periods equals 5.53
percent prior to privatization and 5.06 percent post-privatization.
While the post-privatization change in sales value growth minus consumption growth of
approximately 0.47 percent is a decrease, the small sample size available of only two years
post privatization is cause for concern as to the conclusions that can be drawn from this. For
example the standard deviation for the years 2002 – 2008 is 1.5 percentage points,
approximately three times the difference between the two averages.
It is my opinion that, taken together, the above results are insufficient in order to either accept
or reject H0: The costs of non-prescription pharmaceuticals have not decreased post
privatization as a result of privatization compared to the average change during the years
previous to the privatization.
100
Socialstyrelsen (2012) p. 46ff
Socialstyrelsen (2011) p. 44ff
Socialstyrelsen (2012) p. 47ff
45
Goal - Increased Consumer Acess
#1: The privatization goal of increased consumer access to pharmaceuticals was divided into
six hypotheses. The testing of these hypotheses revealed that the privatization of the Swedish
pharmaceuticals monopoly caused: 1) the average service hours of pharmacies to increase, 2)
the aggregate number of retail pharmacies to increase, 3) the aggregate number of retail
pharmacies in rural areas to increase, 4) the average travel time to the nearest retail pharmacy
to decrease and 6) caused a minor drop in the rate of successful delivery of pharmaceuticals
within 24 hours. Out of the six factors originally identified as pertaining to the goal
fulfillment of increased consumer access, four parameters show significant improvement
while one parameter shows decline and one parameter, 5) remains inconclusive due to lack of
available data.
Goal - Lower prices of pharmaceuticals
#2: The goal of lowered prices of pharmaceuticals had two distinct parts: lowered cost to
consumers and lowered cost to government. The structure of the pharmaceuticals cost-sharing
scheme means that reductions in the cost of non-prescription pharmaceuticals will primarily
benefit consumers while reductions in the cost of prescription pharmaceuticals will primarily
benefit government. Testing the hypotheses regarding both costs of prescription and nonprescription pharmaceuticals yielded inconclusive results. In other words there is insufficient
evidence to safely ascertain whether prices have in fact changed as a result of the
privatization.
46
Empirical Results – Pre- and post-privatization operational
efficiency
The following tables will display the empirical data gathered pertaining to the measurement of
operational efficiency of the privatization. The parameters output, operating profit,
employment, productivity (1) and (2) as well as capital investment. Productivity (1) pertains
to output divided by employees while productivity (2) pertains to operating profit divided by
employees. Unfortunately due to the fact that this thesis is due before the due date of 2011
annual reports a fair number of 2011 annual reports were unavailable at the time of writing.
The unavailable annual reports show up as empty columns in the following tables.
Table 7 – Apoteket AB pre-privatization efficiency parameters.
Output Sales Value
Apoteket AB (SEK Millions)
2007
2008
30418,85
31699,6
Operating Profit
Employees
Productivity (1)
Productivity (2)
Capital Investment
Number of Pharmacies
82
146
8016,75
7999,5
3,794
0,010
3,963
0,018
429
893
253
900
Table 7 displays the pre-privatization for the state owned monopoly Apoteket AB. This data
will be used as the pre-privatization baseline to which the post privatization data will be
compared. The data contained in the table was collected from the annual reports of Apoteket
AB for the years 2007 and 2008.101
101
Apoteket AB (2008b)
Apoteket AB (2007)
47
Table 8. Post-privatization efficiency parameters Kronans Droghandel AB & Apoteksgruppen.
Kronans Droghandel (SEK Millions) Apoteksgruppen (SEK Millions)
2010
2011
2010
2011
Output Sales Value
4443
3700
3500
Operating Profit
41
-232,6
-72,6
1196
950
950
3,714882943
0,034280936
3,894736842
-0,244842105
3,684210526
-0,076421053
Capital Investment
111
37
37
Number of Pharmacies
209
150
155
Employees
Productivity (1)
Productivity (2)
Table 8 displays the post privatization 2010 data for the newly established actor on the
reregulated market Kronans Droghandel Apotek AB and the 2010 and 2011 data for another
new actor, Apoteksgruppen i Sverige AB. Table 9 displays 2010 data for two other newly
established actors, ApoPharm Holding AB and Doc Morris AB. ApoPharm Holding AB is the
owner of the largest private chain of retail pharmacies, Apoteket Hjärtat, as well as a handful
of small actors.102
Table 9 – Post-privatization efficiency parameters ApoPharm & Doc Morris.
Output Sales Value
Apopharm Holding (SEK Millions) Doc Morris (SEK Millions)
2010
2011
2010
2011
6772
205
Operating Profit
-210
-158
Employees
2057
134
3,29217307
-0,1020904
1,52985075
-1,1791045
Capital Investment
119
154
Number of Pharmacies
260
50
Productivity (1)
Productivity (2)
102
ApoPharm Holding AB (2011)
Apoteksgruppen i Sverige AB (2010)
Apoteksgruppen i Sverige AB (2011)
Admenta Sweden AB (2011)
Kronans Droghandel Apotek AB (2011)
48
Table 10 – Post-privatization efficiency parameters Apoteket AB
Output Sales Value
Apoteket AB (SEK Millions)
2010
16274
2011
15243
Operating Profit
-710
109
Employees
3946
3691
4,124
-0,180
4,130
0,030
261
341
322
349
Productivity (1)
Productivity (2)
Capital Investment
Number of Pharmacies
Table 10 displays the post-privatization data for the still state owned but former monopoly
Apoteket AB. All of the post privatization data presented in the above tables was gathered
from annual reports acquired directly from the businesses where possible and from the
Swedish Companies Registration Office where not.103
Table 11 – Aggregate data comparison and 100% of market estimation and change factors.
Year
Output Sales Value
Monopoly Market 90% of Market 100% Estimation Change 08-10 Percentage
2008
2010
2010
31699,6
31394
34878,734
3179,134
Operating Profit
10,03%
82
-1269,6
-1410,5256
-1492,5256
-1820,15%
8016,75
8283
9202,413
1185,663
14,79%
3,954
0,010
3,790172643
-0,153277798
3,790172643
-0,153277798
-0,164
-0,164
-4,15%
-1598,52%
Capital Investment
253
682
757,702
504,702
199,49%
Number of Pharmacies
900
1010
1122
222
24,67%
Employees
Productivity (1)
Productivity (2)
Table 11 displays the aggregate data for the previously presented tables. All numbers are
millions of SEK. The 90% of market collumn is the aggregate data from the five largest actors
on the reregulated market accounting for 90.02 percent of the total number of pharmacies in
Sweden. The 100% estimation category is simply the 90% of market column data multiplied
by a factor of 1,111, thereby assuming that the remaining 10 percent of pharmacies not
included in the gathered data are comparable in terms of output, operating profit, employees,
productivity and investment per pharmacy to the 90 percent. The Change 08-10 column
103
Apoteket AB (2010)
Apoteket AB (2011)
49
presents the difference between the monopoly market data and the 100% estimation for 2010
column data and the percentage category simply shows the change as a percentage.
Table 11 shows that output had increased by approximately 10 percent after the first full year
since the reregulation of the market. But while output increased operating profits were down
from a profit of SEK 82 million to a deficit of approximately SEK 1,411 billion. Employment
saw a sizeable increase of approximately 1186 persons, or 14,79 percent of the preprivatization workforce. Average productivity was also down, average output per employee
was down 4,15 percent while the average operating profit per employee went negative. The
far and away largest increase was in the capital investment category, which increased by
nearly 200 percent. Worth noting regarding the capital investment category is that the
standard deviation of pre privatization capital investment was rather large.
Operational Efficiency
It was previously determined that the measurement of operational efficiency would be based
upon the parameters of output, profitability, employment, productivity and investment. As can
be seen at a glance the data does not paint a uniform picture of the privatization results, while
some parameters show improvement others show decline. Therefore weighting of the various
parameters is required in order to provide an assessment of operational efficiency.
One way of bringing more order to the parameters could be to divide them into two groups.
The first group of parameters includes the parameters output, employment and capital
investment. These parameters can be grouped together on the basis that they primarily provide
information regarding the scope of the market, i.e. how much goods are sold, how many
people are employed in order to sell them and how much is invested in the future sales of
these goods. The other group of parameters includes the parameters productivity and
profitability. These parameters can be grouped together on the basis that they primarily
provide information regarding the yield generated by the parameters included in the first
group. When the data is assessed it is clear that the parameters which show improvement are
the ones included in the first group, i.e. the parameters pertaining to the scope of the market
while the parameters pertaining to the yield of said parameters shows decline.
50
Productivity and operational performance are undoubtedly important factors of operational
efficiency. On the other hand the fact that the post-privatization data is only made up of one
year and in such close proximity to the privatization date poses some analytical problems. In a
monopoly market, which existed pre-privatization, output tends to be lower than in a
competitive market. 104 One would expect that the expansion evident in the measured
parameters is carried out in the hope that it will result in future gains; the problem is that the
single year time frame is not long enough to measure whether the investment pays off.
Unfortunately the unavailability of any more recent annual reports from the private actors on
the retail pharmaceuticals market until after this thesis is due hinders any future analysis.
104
Begg (2006) p. 105ff
51
Conclusions
At the beginning of this thesis I identified what I believed to be a methodological problem in
empirical studies of SOE privatization. I found that empirical evaluation of SOE privatization
efficiency was often carried out using a number of factors that were quite similar across
several studies and authors. I argued that these factors were insufficient in order to assess the
efficiency of an SOE privatization and set out to find a more suitable method of evaluating the
efficiency of SOE privatizations. The alternative method proposed in this thesis was that of
measuring goal fulfillment. I then carried out two evaluations of a recent case using both of
the previously described methodologies in order to display the ways these alternate
methodologies influence the evaluation results. Comparing the two methods illustrates the
importance of methodology when assessing evaluation results and has the potential to raise
questions regarding on what basis major decisions of SOE privatization are taken and the
need for more interdisciplinary empirical research into the effects of SOE privatization.
The empirical results of the evaluation using the methodology that measures efficiency in
terms of goal fulfillment showed that the first goal of increased consumer access to retail
pharmaceuticals was largely accomplished. Only one of the six hypotheses tested showed any
signs of any sign of decreased access, and that one only very slightly. Four of the other six
hypotheses showed improvements across the board while one hypothesis remains
inconclusive due to lack of available data. It can therefore be said that consumer access to
retail pharmaceuticals has improved post privatization as defined by the established goals.
The results relating to the second goal of the privatization, lower prices of pharmaceuticals,
were more ambiguous. The many difficulties associated with measuring the cost of
pharmaceuticals compared to a counterfactual non-privatization scenario as well as the
problems of relating cost fluctuations to the privatization as opposed to other factors such as
patent expirations caused both hypotheses to yield inconclusive results. What can be noted
however, is that had there been any dramatic changes in the cost of pharmaceuticals post
privatization this would likely have been easily observable.
In order to ascertain the overall efficiency of the privatization the fulfillment of these goals
should be put in relation to the input that was required to reach these results. The input
parameters previously described were for example an increased workload of government
bodies, costs associated with evaluations and other preparatory work and the income
52
generated by the sale of the SOEs assets. As is explained in more detail in the empirical
section it would appear that the financial gains generated by the sale far outweigh any
increase in costs generated by the privatization.
The empirical results of the evaluation using the methodology that measures efficiency in
terms of productivity, output, employment, investment and profitability yielded some
interesting results as well. Measurement of these factors showed that while output,
employment and investment improved these parameters did so at the expense of productivity
and profitability.
What is immediately obvious when comparing these results to each other is that they provide
information about two entirely different aspects of the privatization even though the purpose
of the evaluations was in principle the same. Megginson et.al, for example, describe the
purpose of privatization as achievement of socially beneficial aims105 which is well in line
with the definition used to select the goals that were used to assess the privatization using the
alternative efficiency definition. My interpretation is that the measurements of operational
efficiency, while purporting to measure the efficiency of the privatization, in fact primarily
measure the efficiency of the SOE and not of the privatization itself. And while the pre and
post privatization efficiency of the SOE may be of interest to the general success of the
privatization effort it hardly tells the entire story as is made evident by the discrepancy
between the results of the two evaluations. The importance of the discrepancy in the results is
highlighted further by the fact that use of the two methodologies in this case led to two widely
different assessments of the privatization’s efficiency.
While dissecting the previous evaluations of privatization efficiency I broached the subject of
construct validity. Construct validity, as previously explained, pertains to “the extent to which
the variables used to measure program constructs convincingly represent the constructs in the
program logic model”.106 In the case of the previously conducted evaluations described in this
thesis the variables used to measure program constructs were the ones defined as operational
efficiency. The question is then: do these variables convincingly represent the constructs in
the program logic model? In order to answer this question one needs to be familiar with the
program logic model of the privatization in question and it is here that the previous
105
106
Megginson et.al (1998) Page. 43ff
Rossi et.al ( )
53
evaluations of privatization detailed in this thesis run into problems. Evaluating a large
number of privatizations of SOEs in varying sectors and of varying magnitudes using a fixed
set of variables calls into question this connection of variables to program logic. In the case of
the privatization of Apoteket AB, the program logic model was that privatization would lead
to increased consumer access and lowered prices of pharmaceuticals. Judging from the results
of the analysis of my chosen case using the two disparate methods it seems clear that the
overlap between operational efficiency and the program logic in this case is virtually nonexistent. Based on this case alone it is of course difficult to draw conclusions about what
construct validity problems may be present in other evaluations of privatization. But what is
made evident is that using a fixed set of measurements in order to evaluate a broad range of
disparate privatizations will conflict with acceptable construct validity unless the goals in the
program logic model are identical in each of those privatizations. When evaluating as many as
61 separate former SOEs the idea that a fixed set of variables is able to accurately represent
all 61 program logic models seems extremely far-fetched. These construct validity issues
present in the empirical studies of privatization clearly call into question the ability to
synthesize the results of these evaluations into any sort of general assessment as to the
efficiency of privatization as a policy tool.
A likely reason for this evident methodological problem in evaluations of privatization is the
fact that some evaluators seem not to have realized that they are in fact evaluating. Based on
results presented in this thesis I can only speculate as to the possible consequences this
methodological shortcoming can cause but the potential dangers of basing major decisions
regarding SOE privatization on insufficient information can be severe. As the World Bank
report stipulates, large sections of the economies of many developing countries are dependent
on SOEs 107 and any poor decisions made regarding their future could have drastic
consequences. But SOE privatization, or non-privatization for that matter, does not only occur
in developing countries and while consequences may be less severe in developed economies,
poor decisions may still have dire consequences for communities and individual citizens.
I should also point out that I do not imply that simply taking goal fulfillment into account is a
universal solution to the problems of insufficient information in decision-making. The goal
fulfillment analysis of my case also suffers from some major shortcomings, not the least of
107
The World Bank, (1995). p. 7ff
54
which is the inability of the methodology to identify unexpected consequences. For example
the very process of redesigning markets and “organizing the conditions for self
organization”108 could have consequences for government power projection in society and
accountability.109
This thesis clearly points to the need for more interdisciplinary evaluation of privatization and
its consequences in order to provide as complete a picture as possible regarding the efficiency
of SOE privatization.
108
109
Jessop (2010) p. 65
Jessop (2010)
55
Reflections
Thoughts regarding my choice of case
For the purpose of this thesis my choice of case may have been somewhat flawed. The
primary reason that I draw this conclusion is the relative shortages of empirical data in certain
categories, the most important of which is the annual reports from 2011 that are due July 2012
while this thesis is due in May 2012. Had more time passed since the privatization more
empirical information may have been available which would have improved the quality of the
assessments. But while there were problems with the choice of case the case also featured
some interesting unique characteristics. It was for example decided that vertically integrated
chains of pharmaceutical producers and retailers would not be allowed to establish retail
pharmacies in Sweden after the re-regulation of the market. While it was acknowledged that
vertically integrated chains could help reduce the retail prices of pharmaceuticals, (one of the
goals of the re-regulation in the first place) it was decided that vertically integrated chains
posed a risk to the quality of the information received by citizens in pharmacies.110 It was
argued that citizens asking for advice in pharmacies owned by pharmaceuticals producers
would be directed to the producers’ own products even though a competing product may be
more appropriate. Through the prohibition of vertically integrated chains this perceived risk
was given precedence over the official goals of the privatization. This, again, points to the
need for additional factors to be considered when evaluating a privatization effort.
An interesting aspect of the case of Apoteket AB is that the privatization breaks with one of
the major incentives for privatization, namely what Shirley & Walsh call “(…) the SOE
record of failure and waste.”111 In fact, judging from the investigations carried out preprivatization regarding the merits of a future privatization of Apoteket AB there was little to
complain about. There was even the acknowledgement that the state monopoly carried out the
sale of retail pharmaceuticals with competence, safety and quality.112
The need for further research
There is clearly a need for more scientific study of privatizations across several fields, not just
in economics. Any major decision such as the privatization of an SOE will have ramifications
110
SOU 2008:4 p. 207f
Shirley & Walsh (2001) p. 3
112
SOU 2008:4 (2008) p. 23
111
56
that go far beyond the performance of the SOE itself and simply measuring the operational
efficiency of former SOEs does not adequately assess these impacts.
While the need for empirical work using alternative methodologies is clear another conclusion
can be drawn from this thesis. Conducting an evaluation based on goal fulfillment and other
additional parameters will invariably increase the evaluator’s total workload. This could prove
to be a significant problem when conducting large sample empirical evaluations such as some
of those described in this thesis. Megginson et.al for example study the pre- and postprivatization efficiency of no less than 61 former SOEs. A substantial increase in the
workload of every case would severely hamper the possibility of carrying out such large-scale
studies. If one was committed to analysis of previous evaluations of privatization it may also
be interesting to go back to the previously evaluated former SOEs and assess to what extent
the operationalizations made relate to the program logic for each individual case.
Another aspect that may warrant future research is the process of goal-setting in this context.
What strikes me personally as rather odd in terms of the way the goals are formulated is that
they are in some ways in direct opposition to each other. Reasonably, the number of citizens
per pharmacy in a country also has some impact on the average output sales value of those
pharmacies. If the customer base stays the same it would stand to reason that running a large
number of pharmacies would generate more expenses for the retailer than would running a
small number of pharmacies because of minimum staffing requirements and other factors.
This relationship would put increased access to pharmaceuticals in some degree of conflict
with reduced retail prices of pharmaceuticals assuming one also aspires to some level of
sustainability. Assuming that there is some intention of actually achieving these goals, setting
goals that to some degree oppose each other is an interesting phenomenon that may warrant
future research.
57
Summary
In the beginning of this thesis a problem regarding efficiency operationalization in previous
studies of SOE privatization was introduced. The purpose of the thesis was then established as
discerning in what ways various operationalizations of efficiency influenced the obtained
results and their validity.
After a brief introduction to the history of privatizations and the progression of ideas and
feelings regarding privatization previous empirical studies of privatization were detailed. It
was established that these previously conducted empirical studies of privatization were quite
similar in terms of their operationalizations of efficiency as well as their end results. From
these previous studies an operationalization was derived based on their similarities, which was
labeled operational efficiency. I then argued that these previous empirical studies were in fact
evaluations based on the definition of evaluation supplied by Vedung. From the basis of
evaluation theory I proceeded to question the operationalizations in the detailed evaluations of
privatization and, by extension, the validity of their conclusions. The basis of the criticism of
the operationalizations in the detailed evaluations of privatization was based primarily on the
lack of adherence to goals. In order to identify what the potential consequences of the
operationalizations in the detailed privatization evaluations could be two analyses of a recent
case, Apoteket AB, were carried out. One of the analyses was carried through the
identification and operationalization of the goals of the privatization; this process was labeled
goal fulfillment. The other was carried out using the previously defined operational efficiency.
The results of these two analyses were then compared. Despite the purposes of the analyses
being the same the results provided information regarding completely different aspects of the
privatization and the connection between operational efficiency and goal fulfillment was
nonexistent. I then argued that the construct validity of the measurement operational
efficiency was sorely lacking and that by extension the conclusions regarding the efficiency of
privatization as a policy tool are questionable.
58
References
Printed Sources
Begg, D 2006 Foundations of Economics, third edition. New York: McGraw-Hill Education
Bengtsson A. C. 2006 The Swedish retail monopoly on pharmaceuticals, Is it compatible with
EC-Law? Lund: University of Lund Faculty of Law.
Bryman, A. 2008 Social Research Methods, third edition. Oxford: Oxford University Press
Donaldson, S. I. and Lipsey, M. W. 2006 Roles for theory in contemporary evaluation
practice: Developing practical knowledge. In I. F. Shaw, J. C. Greene, & M. M. Mark (eds.),
The SAGE Handbook of Evaluation. London: Sage.
Dyck, A. 2001 Privatization and Corporate Governance: Principles, Evidence and Future
Challenges. The World Bank Research Observer, Volume 18, Number 1, Pages 59-84, Spring
2001.
Galal, A., Jones, L., Tandon, P., Vogelsang, I. 1994 Welfare Consequences of Selling Public
Enterprises, An Empirical Analysis: a Summary. World Bank Publications. Washington DC:
The World Bank
Jessop, B. 2010 Multi-level Governance and Multi-level Metagovernance, Changes in the
European Union as Integral Moments in the Transformation and Reorientation of
Contemporary Statehood. In: Bache I. and Flinders, M. (eds) 2010 Multi-level Governance,
New York: Oxford University Press.
Johansson, O. 2011 Lika men olika? – redovisning av effektivitet inom statliga myndigheter.
Förvaltningshögskolans rapporter nr.121 Göteborgs Universitet, Gothenburg.
Jordahl, H. 2009 Privatiseringar av statligt ägda företag, En litteraturöversikt med fokus på
OECD-länderna. Stockholm: Regeringskansliet
Lindgren, L. 2009 Utvärderingsmonstret, Kvalitets- och resultatmätning i den offentliga
sektorn. Lund: Studentlitteratur AB
McLaughlin, J. and Jordan, G. 1999 Logic Models: A Tool for Telling Your Program’s
Performance Story. In: Evaluation and Program Planning. Volume 22, Number 1, Febuary
1999.
Megginson, W., Nash, R. and Randenborgh M. 1998 The Financial and Operating
Performance of Newly Privatized Firms: an International Empirical Analysis. The Journal of
Finance Volume 53, Issue 3, Pages 1081-1110, June 1998.
Moïse, P. and Docteur, E. 2007 Pharmaceutical pricing and reimbursement policies in
Sweden. Paris: OECD Health Working Papers
59
Nellis, J. 2002 The World Bank, Privatization and Enterprise Reform in Transition
Economies: A Retrospecitve Analysis. The World Bank Operations Evaluation Department.
Washington DC: The World Bank
Odell, M. 2009 ”Förord”. In: Jordahl, H. 2009 Privatisering av statligt ägda företag, en
litteraturöversikt med fokus på OECD-länderna. Regeringskansliet. Stockholm.
Peters, G. and Pierre, J. 2010 Multi-level Governance and Democracy: A Faustian Bargain?
Multi-level Governance, Edited by Ian Bache and Matthew Flinders. Oxford: Oxford
University Press
Rossi, P.H., Lipsey, M.W. & Freeman, H.E 2004 Evaluation - A Systematic Approach, 7th ed.
Thousand Oaks, CA: Sage.
Shirley, M. and Walsh, P. 2001 Public versus Private Ownership: The Current State of the
Debate. World Bank Policy Research Paper. Washington DC: The World Bank
The World Bank. 1995 Bureaucrats in Business, The Economics and Politics of Government
Ownership. New York: Oxford University Press
Vedung, E. 2009 Utvärdering I politik och förvaltning, 3:e upplagan. Lund: Studentlitteratur
AB
Public Sources, Annual Reports and other economic data.
Admenta Sweden AB. 2011 Årsredovisning Doc Morris Apotek. Stockholm: Admenta
Sweden AB.
ApoPharm Holding AB. 2011 Årsredovisning och Koncernredovisning för ApoPharm
Holding AB. Solna: ApoPharm Holding AB.
Apoteket AB. 2011 Årsredovisning 2011. Stockholm: Apoteket AB
Apoteket AB. 2010 Årsredovisning 2010. Stockholm: Apoteket AB
Apoteket AB. 2008a Delårsrapport januari-september. Stockholm: Apoteket AB
Apoteket AB. 2008b Årsredovisning 2008. Stockholm: Apoteket AB.
Apoteket AB. 2007 Ett liv i hälsa, årsredovisning 2007. Stockholm: Apoteket AB.
Apoteksgruppen i Sverige AB. 2010 Årsredovisning 2010. Stockholm: Apoteksgruppen i
Sverige AB.
Apoteksgruppen i Sverige AB. 2011 Årsredovisning 2011 Stockholm: Apoteksgruppen i
Sverige AB.
Kronans Droghandel Apotek AB. 2011 Årsredovisning för räkenskapsåret 2010.
Stockholm: Kronans Droghandel Apotek AB
60
Lindhé, A., Andrén L., Gustafsson, L. and Nyman, H. 2011 Kostnadsutvecklingen för
läkemedel i Västra Götalandsregionen 2011-2012. Vänersborg: Västragötalandsregionen
Regionens Hus
Myndigheten för tillväxtpolitiska utvärderingar och analyser. 2011 WP/PM 2011:49,
Geografisk tillgänglighet till Läkemedel, Delrapport 1 En analys av omregleringen av
apoteksmarknaden. Östersund: Tillväxtanalys
Reinfeldt, F. & Hägglund G. 2009 Regeringens proposition 2008/09:145, Omreglering av
apoteksmarknaden. Stockholm: Socialdepartementet.
Socialdepartementet. 2006 Dir. 2006:136, Omreglering av apoteksmarknaden. Stockholm:
Socialdepartementet.
Socialstyrelsen. 2012 Läkemedel – Statistik för år 2011. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2011 Läkemedel – Statistik för år 2010. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2010 Läkemedel – Statistik för år 2009. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2009 Läkemedel – Statistik för år 2008. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2008 Läkemedel – Statistik för år 2007. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2007 Läkemedel – Statistik för år 2006. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2006 Läkemedel – Statistik för år 2005. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2005 Läkemedel – Statistik för år 2004. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2004 Läkemedel – Statistik för år 2003. Stockholm: Socialstyrelsen.
Socialstyrelsen. 2003 Läkemedel – Statistik för år 2002. Stockholm: Socialstyrelsen.
Statens Offentliga Utredningar. 2009 SOU 2008:4 Omreglering av apoteksmarkanden,
Huvudbetänkande av Apoteksmarknadsutredningen. Stockholm: Statens Offentliga
Utredningar.
Statistiska Centralbyrån. Invånare per kvadrat kilometer efter region, kön och tid (2012)
2012-03-22 13:27. Strömsund: Statistiska Centralbyrån
Tandvårds- och läkemedelsförmånsverket. 2011 Översyn av 2011 års modell för utbyte av
läkemedel på apotek, med modell och strategi för år 2012 och framåt. 2550/2011 Stockholm:
Tandvårds- och läkemedelsförmånsverket
Sveriges Apoteksförening. 2011 Två år med ny apoteksmarknad. Stockholm: Sveriges
Apoteksförening.
61
Web Based Sources
Alliansfritt Sverige. Stockholmsskogar privatiseras.
http://www.alliansfrittsverige.nu/2011/12/stockholmsskogar-privatiseras.html 02-03-2012
13:58
Corp Watch. What is Neoliberalism? A Brief Definition for Activists
http://corpwatch.org/article.php?id=376 02-03-2012 13:59
Läkemedelsverket. Ökad tillsyn av apotek under 2011. http://www.lakemedelsverket.se/Allanyheter/NYHETER-2011/Okad-tillsyn-av-apotek-under-2011-/ 2012-3-21 17:39
Riksrevisionen. Försäljningen av apotek.
http://www.riksrevisionen.se/GRANSKNINGAR/Pagaende-granskningar/Pagaendegranskningar/Forsaljningen-av-apotek/ 2011-03-08 14:46
Socialist Alternative. Neo-liberalism and the attack on public education internationally.
http://www.socialistalternative.org/publications/education/ch6.html 02-03-2012 14:02
62