Evaluation in the CMA
Our mission is to make markets work well for
consumers, businesses and the economy
Natalie Timan, Director of
Economics
19 November 2014
1
Contents
● Promoting competition: the benefits of competition and how they can
be realised
-
What is competition and why does it matter?
Benefits of competition
How benefits can be realised
● Evaluation in the CMA
-
Essential features
Positive impact in competition law enforcement, mergers, market investigations
and consumer policy
Ex-post evaluation
● Indirect benefits
2
Promoting competition: the benefits
of competition and how they can be
realised
3
What is competition and why does it matter?
Vigorous competition provides
firms with incentives to deliver
what consumers want as
efficiently and innovatively as
possible
Consumers play a key role in
activating vigorous competition
between firms
4
Benefits of competition
1. Lower prices
2. More choice
3. Better quality
4. Innovation
5
Competition, innovation and growth
● Link competition to innovation and growth key way of promoting
competition policy and reforms
● CMA’s missions statement: make markets work well for consumers,
businesses and the economy
● How competition increases innovation and growth
-
Output = f [natural resources, capital, labour] technology, management
-
Prioritise interventions that lift restrictions on e.g.:

Land (natural resource);

Finance (capital)

Markets associated with labour participation and higher education
(labour)

Innovation intensive sectors e.g. Pharmaceuticals

Tackling public restrictions on competition -> managers not
sheltered from impact of competition
6
But how do we get all these benefits?
Markets have to be competitive to realise benefits for
consumers
● Sufficient firms in the market
to ensure rivalry
● Ease of market entry and exit
● No barriers to expansion for
firms in the market
● Widespread availability of
information about the market
7
Why might markets not be competitive?
Competition problems…
…Leads to...
•
•
•
Capacity constraints
High concentration
Lack of substitutes
Market power
•
•
•
Intrinsic barriers
Strategic barriers
Regulatory barriers
Barriers to entry,
expansion & exit
•
•
•
•
Price fixing
Share markets
Allocate customers
Firms follow each other
Coordinated
conduct
•
•
Foreclosure
Dampened competition
Vertical
relationships
•
Limited shopping and
switching
Weak customer
response
…and ultimately risk of..
Poor outcomes
for consumers
8
Barriers to entry in the pharmacy industry
● In 2003 OFT recommended entry controls for community pharmacies
be abolished altogether
● In 2005 DH introduced a criterion of consumer choice and created
exemptions from control of entry test
● What impact?
-
£16.4m-£24.5m in travel time savings
-
£3.3m in reduced waiting times
-
£5m in consumer savings on medicines
-
increase of 8.8 per cent in the number of pharmacies in England
● What would have happened?
-
None of the above
-
Regulatory costs to businesses and the NHS of £12.5m
9
What roles can government play in
markets?
Government can play 4 roles in markets
…influencing the market
without participating
directly itself
through market
shaping/regulat
ions
through taxes and
subsidies
…directly participating in
the market
through acting
as a buyer
through acting
as a supplier
The impact government can have on firm incentives
is a theme that runs across these different roles
10
Evaluation in the CMA
11
The CMA uses a range of tools to make
markets work better for consumers
Unfair Contract
Terms
Anti-competitive
agreements
Abuse of
dominance
Market
Studies and
Investigations
Unfair
Commercial
Practices
Other
consumer
legislation
Merger
control
Advocacy
Guidance
CONSUMER
ENFORCEMENT
COMPETITION
ENFORCEMENT
As an independent authority we use our prioritisation principles
to determine where to focus our resources and tools
Education Guidance
Compliance
SOFTER NON-ENFORCEMENT TOOLS
12
CMA’s approach to evaluation
● Our approach to evaluation has three strands:
-
Estimating the impact of each project as it completes
Evaluating projects to identify the costs and benefits, and lessons learned
Exploring further how best to assess the CMA’s wider impact on economic growth
including through deterrence or improving the prospects for exit, entry and
innovation in markets
● The aim of our evaluation work is two-fold:
-
External accountability: to evaluate whether the CMA delivers its objectives and
does so cost effectively and to demonstrate this externally
Internal management and prioritisation: to inform future CMA work regarding how
we prioritise, conduct and follow-up our work to ensure we maximise our impact.
This is an assessment of the accuracy of our predictions about markets and the
effectiveness of our interventions. A separate internal evaluation of our
procedures and efficiency is carried out after each project
13
Essential features (i)
● Any evaluation or impact estimation will have the following common,
essential features:
-
A ‘Theory of harm’
Understanding what the action/reform was intended to do
Test changes in the market resulting from the action/reform to evaluate
the impact the action/reform has had on welfare
● An understanding of how competition works in a market is an
important step towards establishing these features
● CMA’s knowledge of the market stems from case teams’ in-depth
investigations
14
Essential features (ii)
● Identify those markets that are directly affected by a policy and
those indirectly affected through knock-on effects
● For each affected market determine:
-
the size of the market
who are the major players?
is the market local, regional, nation, or international?
are there any barriers to entry or exit in the market?
15
Essential features (iii)
● Explore the implications for firms by asking does the reform:
-
directly or indirectly expand the number or range of suppliers?
increase the ability of suppliers to compete?
increase supplier’s incentives to compete vigorously?
● Explore the implications for consumers by asking if the reform:
-
reduces barriers to accessing and assessing information?
makes it more/less obvious what other consumers are doing in a
market/that others could be getting a better deal?
improves the ability for consumers to switch providers?
● Consider how firms and consumers interact!
16
Positive Impact
17
Positive Impact Evaluation
Background
●
Positive Impact Evaluation aims to quantify the benefits delivered to consumers from CMA
work
●
As part of its Performance Management Framework with BIS, the CMA is required to
demonstrate the delivery of £10 of benefit to UK consumers for every £1 the organisation
costs. The target relates only to direct financial benefits to consumers
●
Calculations are made using assumptions formed after the case closes to estimate increases
in consumer welfare arising from CMA interventions
●
Figures are intended to be lower bound estimates and presented as three year rolling
averages due to uneven caseloads across years
●
Previously, where market studies/investigations or mergers were looked at by both the OFT
and the CC, the benefit was split 80/20
Why do it?
●
To demonstrate the value for money delivered to UK consumers
●
For the benefit of staff to see the results of hard work
●
To track performance and maintain our position as a world class competition authority
18
Historical Performance
Annualised Benefits (OFT)
Estimated average annual
consumer savings 2010-13
Mergers
£13m
Consumer Protection Enforcement
£77m
Market studies, reviews of orders and undertaking,
and market investigation references
Competition Enforcement
£195m
Total Benefits
£422m
Total OFT costs (averaged over 2010-12)
£48m
Ratio
8.7:1
£136m
19
Competition Law Enforcement
Method
●
Competition Law Enforcement relates to anything which remedies Chapter I (Anticompetitive agreements/Cartels) or Chapter II (Abuse of Dominance) infringements
●
The positive impact is estimated as set out in the formula below. Ideally price changes
and duration of infringement will have been calculated during the investigation or can
be calculated from information specific to the case. Otherwise we apply a rule of thumb
based on international evidence.
Rules of Thumb
•
•
𝑎 = annual impact
t = turnover of goods
p = price increase (%)
𝑎 = 𝑡𝑝
𝑎𝑡 =
𝑐
𝑎
𝑠=1
(1+𝜌)𝑠
𝑎𝑡
= total impact
c = no. years operational
𝜌 = social discount rate
•
𝟏
In absence of better information and
based on international best practice and
academic research
•
c = 6 years
•
𝜌 = 3.5%
•
Overcharge = 15% (fully
passed on)
●
This should represent a lower bound estimate and ignores any price increases which
may have arisen from competitors not involved in the anti-competitive behaviour
●
For Chapter I, the turnover is taken to mean the turnover of parties involved in the anticompetitive behaviour in the relevant market(s). For Chapter II, the case teams can
typically provide the appropriate turnover figures.
1
OECD recommend: c = 3 years, overcharge = 10%/5% (Cartel/AoD)
20
Competition Law Example – Napp Pharmaceuticals (2001)
●
Napp is a leading supplier of pain medication and held a dominant position in the
market for sustained release morphine to hospitals and communities
●
In order to maintain its dominant position in the larger community market, Napp was
offering large discounts (up to 90%) in the gateway hospital market
●
The OFT intervened and found Napp guilty of predatory pricing. They were fined
£2.2m (after appeal) and were limited to offering hospital discounts of less than 80%
●
To calculate the positive impact, case teams came up with an estimate
•
●
●
Assuming an overall price decrease of x%, a y year duration and a 3.5% discount rate
This calculation was followed up with ex post, in depth analysis looking at actual price
data from each market:
•
Community: competition improved and prices decreased by 25%. This was compared to a
counterfactual which assumed prices remained at pre intervention levels. This gave savings of £1.7
million per year or £15.1 million from 2001-2009
•
Hospital: due to the restrictions on discounts, the benefits from the community market had to be
offset against extra costs in the hospital segment. Hospital prices increased by £200k to £225k per
year
•
In the interests of calculating a conservative estimate, we took the £225k figure which gave overall
savings of £1.5m per year or £13m (2001-2009)
The estimate from the impact estimation was lower than the ex post evaluation. This
example underlies our belief that impact estimation is usually conservative
21
Mergers
Method
●
How much consumer detriment was avoided from the blocking of anti-competitive mergers?
●
Previously, we used Cournot or Betrand merger simulation. However, due to a large amount of
assumptions and the time involved in such methods, we now use a new approach
●
GUPPI (Gross Upward Pricing Pressure Index) gives a measure of the incentives of a merged
firm to rise prices (%)
●
Equation:
𝐷12 (𝑃2 −𝐶2 )
𝐷12 = pre-merger diversion ratio from Product 1 to Product 2 (
∆𝑄2
)
∆𝑄1
•
𝐺𝑈𝑃𝑃𝐼1 =
•
Multiplying this percentage by the revenues of the relevant products, gives the
gross price increase expected from an anticompetitive merger. This is based on
the assumed pass through rate of 100% (ie all upward pricing pressure is realised
in actual price increases).
𝑃1
P
C
= pre-merger price levels
= costs (which, with P, gives an idea of the margin)
●
Most case teams will have all required information for the calculation while some will have
estimated the ratio in the course of their work
●
Where insufficient information is available to calculate GUPPI ratios, we work closely with case
teams to make accurate assumptions about potential prise increases
●
In some cases, on the advise of case teams, we may adopt a more bespoke approach to
assessing the positive impact of a merger decision (see example 2 below)
22
Mergers Example 1 – Jewson / Build Centre (2012/13)
●
Jewson agreed to sell 22 branches as UiL for merger with Build Centre
●
OFT looked at GUPPI ratios and revenues to find impact for each branch before these figures were
added together to estimate total positive impact
●
Diversion ratios were calculated from survey data asking respondents who shopped at either Jewson or
Build Centre to indicate where they would purchase specific items if they were no longer available in their
preferred shop. These were then combined with information from company accounts on prices and costs
to establish the GUPPI percentage.
●
For example, area one:
•
•

Total revenue of Build Centre = £am

GUPPI (Build Centre to Jewson) = b%

Multiply together (assume 100% pass through) gives detriment = £ck

Total revenue of Jewson = £dm

GUPPI (Jewson to Build Centre) = e%

Detriment = £fk

Total positive impact for area one = £gk
This same method is applied to all 22 affected areas giving estimated total avoided detriment from the
UiL of £hm per year
Assuming discount rate of 3.5%, inflation of 3% and duration of 2 years; can work out Present Value
and NPV positive impact
23
Mergers Example 2 – Akzo / Metlac (2013/14, CC)
●
CC blocked the merger of Akzo and Metlac due to substantial lessening of
competition in the supply of metal packaging coatings for beer and beverage cans
in the UK
●
Through close work with the case team, the following conclusions were made
regarding the impact of the blocked merger
●
Customers complained that Metlac was an important price competitor to Akzo and
the rest of the market
●
The relevant market was estimated as the £xm B2E segment
●
The assumed price rise was set at α% which was conservative due to the fact that
Metlac was not seen as a constraint over the entire market
●
●
α% of £xm  £ym of avoided detriment to customers
With an assumed duration of β years and discount rate of 3.5%, NPV was
£y𝑚
calculated as £zm (£ym + 1.035 = £zm)
24
Market Investigations
Method
● Quantifying the positive impact from investigations into nonfunctioning markets is difficult and, due to the variety of
activities, there is no generic approach
● Case teams are crucial as their expertise on the case at hand
is key to forming realistic assumptions about the positive impact
of the investigations. In some cases, a detailed detriment
calculation will have been made during the investigation and
this information should provide a good basis for calculating the
positive impact.
● Often, this information is expanded with observations on
developments after the remedy and occasional in depth
evaluations some time after the intervention
25
Consumer Policy
Method
● Consumer policy covers a wide range of enforcement activity which
seeks to change trader behaviour which contravenes consumer
protection legislation and promotes competitive practices
● Formerly, the positive impact calculation was based on complaints
from consumers. Looking at the number of complaints received in a
12 month period before and after the intervention and assigning each
complaint a monetary value allows estimation of the positive impact
the CMA has had. Thus, complaints are equated to actual
infringements
● Now, the calculation is more similar to the Market Investigations and
Competition Enforcement methods listed above. We work closely
with case teams to establish key assumptions and often use the same
formula described under Competition Enforcement.
26
Ex-post evaluation
27
Ex-post evaluation
● What is ex-post evaluation?
-
An ex-post assessment of the actual impact of our interventions in enforcement
cases.
Primary purpose: measure the net benefits to consumers of any intervention.
Secondary purpose: examine wider benefits (usually unquantifiable), such as the
effect on consumer confidence and deterrence.
Typically compares the quantified net benefit to the cost incurred by the OFT in
the initial intervention.
● How do we evaluate?
-
Can involve: (i) desk-based research on prices and other market data, (ii) surveys
of consumers, businesses and other stakeholders.
Usually conduct two to three evaluations a year. Can be conducted immediately
after a case is completed, but typically there is a delay, potentially up to several
years, whilst recommendations are implemented.
28
Ex-post evaluation
● Formal techniques:
-
Simulations and structural models
Event studies
Difference in Difference techniques
● Reality:
-
Identify changes in market characteristics (price, quality, choice etc)
Identify changes in consumer behaviours (shopping habits, awareness
and understanding, product selection)
29
Indirect impact
30
Indirect impact
● Positive impact is measured as a direct impact but it underestimates overall
impact:
-
Indirect impacts exist beyond the scope of direct impact evaluation
Certain areas may be delivering higher (lower) benefits than this evaluation suggests
•
-
In particular, mergers seem to be undervalued which might be redressed if indirect
impacts are taken into account
This is important when it comes to prioritisation of cases
● Increases in enforcement activity are not always associated with the
effectiveness of a competition authority
-
In fact, a well functioning regime should be able to minimise its enforcement through effective
deterrence
● CMA has begun surveying relevant literature, focussing on the bottom up
method
-
In order to properly account for the impact of different competition tools, a bottom up
approach is preferable
Top down studies give a good estimate of the overall impact a competition authority is having
on macro variables but does little to apportion this between direct and indirect and even less
to split the impacts into different competition areas
31
Cartels and deterrence (I)
Population of Cases
Undeterred
Deterred
Undetected
Detected
Type II Errors
(unobservable)
Direct Impact
(observable)
Indirect Impact
(unobservable)
32
Cartels and deterrence (II)
●
Indirect Impact refers to the amount of deterred harm in an economy. The possibility of
Type II errors mean it is also worth looking at the amount of undetected harm as well.
●
In addition, deterred harm may not only come from reducing cartel formation but also
from reduced overcharge by existing but undetected cartels.
●
Deterred harm can be looked at in three ways:
●
●
●
-
Surveys
-
Looking at the effect of enforcement on profitability of cartels
-
Estimating the effect of policy on the population of cartels by drawing inferences from the population of
discovered cartels
The papers look at:
-
Ratios of latent to observed cartels
-
Magnitude of overcharge
-
Average duration
Methods include:
-
Conditional probability of detection
-
Natural (econometric) experiments
-
Profitability of price fixing
Key shortcoming
-
Unobservable population
-
Sample selection bias
33
Merger and deterrence
●
Literature on deterrence of mergers much more limited than that of cartels
●
Indirect benefits from merger control arise in two forms:
-
Frequency based deterrence- potential mergers do not go ahead due to the possibility of scrutiny by the
merger control regime
-
Composition based deterrence- mergers are modified to avoid the scrutiny of the merger control regime
●
Some literature has focussed on survey based methods
●
Other academics have focussed on empirical and theoretical work which looks at the
effect of merger enforcement on the number/composition of (future) mergers notified to
the competition authority
●
Most work assumes that merger enforcement has a greater deterrent effect on
anticompetitive (rather than pro-competitive) mergers and the literature suggests that
increasing numbers of enforcement decisions do lead to fewer notified mergers
●
There is, however, disagreement on the types of actions that have the greatest
deterrence effect
●
Type I and II errors are also worth considering
●
Deterrence is most likely at the bottom and top of the anticompetitive harm distribution;
mergers with small anticompetitive effects are likely to be deterred by the admin costs
whilst the most anticompetitive mergers are deterred by the prospects of being blocked
34
Questions?
35