The Role of Impact Assessment
in Shaping
‘Good Public Policy’
Jonathan B. Wiener
Perkins Professor of Law, Public Policy & Environmental Policy, Duke University
University Fellow, RFF
Past President, SRA, and Co-Chair, World Congress on Risk
ENA-UCL conference on “Regulatory Impact Assessment in Europe”
Paris, 10 June 2013
All slides © 2013 Jonathan B. Wiener; not for copying or redistribution without permission.
Impact assessment and benefit-cost analysis:
a continuing conversation – in Paris
Benjamin
Franklin
Joseph
Priestley
Jeremy
Bentham
Jules Dupuit
continuing today…
Benjamin Franklin, letter to Joseph Priestley, Sept. 19, 1772:
“In the Affair of so much Importance to you, wherein you ask my Advice, I
cannot for want of sufficient Premises, advise you what to determine, but if
you please I will tell you how. When those difficult Cases occur, they are
difficult, chiefly because while we have them under Consideration, all the
Reasons pro and con are not present to the Mind at the same time; but
sometimes one Set present themselves, and at other times another, the first
being out of Sight. . . .
“To get over this, my Way is, to divide half a Sheet of Paper by a Line into
two Columns; writing over the one Pro, and over the other Con. Then during
three or four Days Consideration, I put down under the different heads short
Hints of the different Motives, that at different Times occur to me, for or
against the Measure. When I have thus got them all together in one View, I
endeavour to estimate their respective Weights . . . and thus proceeding I find
at length where the Ballance lies . . .
“And, tho’ the Weight of Reasons cannot be taken with the Precision of
Algebraic Quantities, yet, when each is thus considered, separately and
comparatively, and the whole lies before me, I think I can judge better, and am
less liable to make a rash Step; and in fact I have found great Advantage from
this kind of Equation, in what may be called Moral or Prudential Algebra.”
Evaluating Impact Assessment (IA of IA)
• How well does Impact Assessment shape ‘good public policy’?
•
•
•
•
Can be helpful in creating demand for policies that increase social well-being
Ex ante IA for policymaking
Ex post IA for policy revisions, & for testing accuracy of ex ante IA
Need empirical evaluation of IA – compared to counterfactual (with/without),
and across variation in approaches and instititions
• Pros:
• Economic: Improving social net benefits, cost-effectiveness
• Cognitive: Framing decisions, avoiding heuristic errors (“rash steps”)
• Political: Transparency, oversight/control, reducing special interest politics
• Cons:
•
•
•
•
Delay
Low quality
Bias (e.g. overstate costs, understate benefits?)
Narrow scope; yields skewed analyses and policies
Problem: Narrow scope in IA
• Focusing on just one impact or one policy
•
•
•
•
•
Costs: just administrative, or just compliance
Benefits: just target risk reduction
Alternative policy options: just one (or none)
Comparative studies across countries: sampling bias
Tradeoff: precise but narrow vs. broad and comprehensive
• E.g.: Narrow scope in IA of costs: just Administrative cost
• Measurable (e.g. Standard Cost Method, SCM)
• Politically salient (“cut red tape,” “paperwork reduction”)
• But: focusing on just Administrative cost could be counterproductive
• Restricting information collection can prevent high quality IA, BCA
• Restricting information collection/disclosure can increase cost
• E.g. shift to prescriptive technology standards
Narrow scope in
IA of Risk Reduction
Target
Risk
• One ‘target’ risk at a time.
• One decision maker.
• Risk assessment: how serious is the risk?
• Risk management (regulation): what to do about the risk?
• Impact assessment: evaluate the pros & cons of the regulation
• on these terms (one target risk, costs of compliance)
More complex reality:
Multiple, Interconnected Risks
(TR = target risk, CR = countervailing risk)
TR
TR
CR
TR
CR
Examples:
• Medical care, drugs
• Food safety
• Air/water/waste
• Transportation safety
• Climate & Energy
• Financial risk
• Counterterrorism
CR
Challenges:
• Multiple risks
• Priority-setting (triage)
• Ancillary impacts, tradeoffs
(co-benefits, iatrogenesis)
• Interconnected, emergent
• Multiple decision makers
Risk-Risk Tradeoffs and
Risk-Superior Moves
Precaution
vs. TR
(target
risk)
Risk-Superior
Move
•Z
•Y
•X
Weigh the
Tradeoff
p(TR)
=1
p(AR) = 1
RPF
RPF2
Precaution vs. AR
(ancillary risk)
E.g. Graham & Wiener, Risk vs. Risk (1995); Wiener, “Managing the Iatrogenic Risks of
Risk Managenent” (1998); Wiener, “Precaution in a Multirisk World” (2002).
Narrow scope in both
Benefits (risk reduction) and Costs
• Focusing BCA on just the Target risk reduction, and on just the
Cost of compliance to the regulated industry
• Easier to assess; greater precision
• Focus on TR neglects ancillary impacts (countervailing risks)
• Focus on reducing industry compliance cost favors easy substitute
products/technologies, which can induce their own countervailing risks
• E.g.: ammonia to CFCs to HCFCs to HFCs; butter to margarine;
MTBE; coal (CO2) to natural gas (CH4); BPA to …?
• BCA of BCA: narrow scope can induce R-R tradeoffs
• It’s often a larger error to leave out an important impact, than to be less precise
on an included impact.
• Franklin: get the decision maker to consider “the whole” – all important impacts
• Keynes: “I’d rather be roughly right than precisely wrong”
• Need comprehensive IA: full portfolio impacts
Narrow scope in Comparative studies
of Regulation across Countries
• Focusing on just one or a few risks or regulations or
institutions
• Claims of fixed national cultures of risk
• Claims of fixed national styles of regulation
• Claims of a “reversal” in relative precaution, based on a narrow
sample
• Need: more comprehensive studies
Reversal over time relative precaution
in the US vs. EU ?
“More and More, Europeans Find
View espoused
Fault with US: Wide Range of
Events Viewed as Menacing” -by:
NY•Times
, 9 April
2000, p.A1
Genetic
Engineering,
EU
GMO foods
crops
“Americans
seem to/ be
pragmatic • EU officials
about new ideas and inventions.
• Hormones
in Beef,
Europeans
tend to worry.
…a
• NGOs
pervasive
technophobia
including
rBST … -- T.R.
Reid, Wash. Post, 2001
• News media
• Climate
Change
“Precaution
is for
Europeans” –
NY Times, April 2003
• Toxic Chemicals
“Europe is considered fairly risk• Guns
averse
… America, on the other
hand, is often seen as having a
• Antitrust
strong risk-taking culture” – The
Economist, 24 January 2004
US
“In the US they believe that
if no risks have been proven
about a product, it should
be allowed. In the EU we
believe something should
not be authorized if there is
a chance of risk.”
-- Pascal Lamy, EU Trade
Commissioner, 1999
• Scholars
E.g. David Vogel et al. (2000, 2001,
2003, 2012): “Reversal” or “flipflop” in relative US/EU precaution
between 1970-90 and 1990-2010.
Why: shifts in public opinion,
leadership, IA using BCA.
Cautions in Comparing Regulatory Systems
•
•
Broad claims may be overstated
– Claims of fixed risk cultures conflict with claims of dynamic regulation
• E.g., fixed “national styles of regulation” (Vogel 1986 vs. Vogel 2012)
• E.g., “Europeans are more risk-averse” while “Americans are
technological optimists” (an old stereotype) conflicts with “reversal”
(greater US precaution in 1970s, to greater EU precaution since 1990s)
– Heuristic exaggeration of inter-group contrasts (e.g. Henri Tajfel)
• US and EU not so different; both at high end of global relative precaution
• End of Cold War = rhetorical contest for leadership?
– Ambition vs. actuality: political rhetoric vs. policy reality
– Comparisons may vary by component: risk assessment, risk management,
review, enforcement; and across different laws, agencies, topics
– Sample selection bias: “availability” heuristic in comparative research
• Broad claims drawn from a few recent visible cases (e.g.: GM foods).
Sampling by convenience – cases under the streetlamp – cherry-picking.
We used two methods to test relative US-EU precaution:
– Case Studies – in-depth; a broad array
– Aggregate Data – a sample of 100 from a universe of 2878 risks
Findings from our case studies:
Parity and Particularity – selective
precaution
US
EU
1970s – 1990:
• Marine environment
• Guns
1990 - 2010:
• Hormones in Beef, rBST
• GM foods / crops
• Climate change
• Toxic Chemicals
1970s – 1990:
• New drug approval
• Strat. Ozone (CFCs)
• Nuclear power
• Endangered species
• Lead (Pb) in gas/petrol
1990 - 2010:
• BSE/vCJD in Beef, Blood
• Smoking tobacco
• Particulate Matter (PM) air
pollution
• Terrorism
Are some
societies “more
precautionary”
than others?
Key findings:
• Selective application
of precaution, in both
Europe and the USA.
We studied US and
Europe, 1970-2010:
• No strong US-EU
trend: < 6% shift.
• A dozen case
• Why: Not broad
shifts in public,
leaders, BCA. Trade
protection, crises.
studies.
• Quantitative
comparison of a
random sample
of 100 from a
universe of 2878
risks.
• Hybridization: much
legal borrowing: e.g.
PP, Better Reg., Impact
Assessment (IA).
• Explanatory
factors.
• Impacts.
(RFF Press / Earthscan / Routledge, 2011)
Plus new symposia in Reg. & Gov. (2013) and EJRR (2013).
• Precaution can yield
risk-risk tradeoffs.
Need IA, foresight,
optimal precaution.
An update from The Economist
(27 April 2013, p.32)
• A US Senator recently “grumbled about Europe’s ‘non
science-based regulations.’ That glanced at an old
philosophical dispute, with American regulations weighing
costs and benefits … while the European ‘precautionary
principle’ distrusts products or new technologies until they
are proved safe.”
• But recently “[r]egulatory approaches have also converged,
with American officials talking about ‘humanized’ costbenefit analysis, and Europeans fretting about the price of
too much precaution.”
• [My comments: not convergence, so much as parity &
particularity; and not just cost, but ancillary impacts too.]
Narrow scope in tools and countries?
• Diffusion of several regulatory tools around the world, e.g.:
•
•
•
•
•
Environmental Impact Assesment (EIA)
Market-based incentive instruments (cap & trade, taxes, information disclosure)
Precautionary Principle (PP)
Regulatory Impact Assessment (RIA)
Regulatory Oversight Bodies (ROBs)
• Across many countries, RIA and ROBs vary in functions &
powers, capacity & expertise, structure & location, …
• Strategic action: are agencies moving to evade ROBs ?
• What can we learn from these patterns for ‘better regulation’?
•
•
•
J.B. Wiener et al., eds., The Reality of Precaution (RFF Press/Routledge 2011)
J.B. Wiener & A. Alemanno, “Comparing US OIRA and EU IAB,” in Rose-Ackerman & Lindseth,
eds., Comparative Administrative Law (2011)
J.B. Wiener, “The Diffusion of Regulatory Oversight,” in Livermore & Revesz, eds., The Globalization
of Cost-Benefit Analysis in Environmental Policy (Oxford Univ. Press, 2013)
Source: OECD, RIA as a Tool for Policy Coherence 15 (2009).
Global diffusion of RIA and ROBs
• US
•
•
RIA since 1978
ROB: OIRA, since 1980 (‘Return’ power since 1981; ‘prompts’ since 2001)
• Australia
•
•
RIA since 1985
ROB: OBPR; Productivity Commission
• New Zealand
•
•
RIA / RIS: since 1990s
ROB: Ministry of Commerce, then Treasury; Productivity Commission (2010) ?
• EU
•
•
RIA since 2002
ROB: IAB, since 2006 (‘Return’ power since 2010; ‘prompts’ since 2006)
• OECD member states (including South Korea, Mexico, Chile)
•
•
RIAs: from 50% in 1998, to 100% in 2010
ROBs: from ~33-50% in 1998, to ~50-90% in 2010
• Developing countries: increasing (with varying institutions & analytic tools)
•
E.g. Bangladesh, Brazil, Bulgaria, China, Indonesia, Kenya, Philippines, Serbia, South
Africa, Tanzania, Uganda, Vietnam, …
Toward comprehensive IA
• Precise but narrow vs. broad and comprehensive
• Consider all important impacts – so “the whole lies before me”
• Full portfolio impacts
• Full social costs – not just administrative cost or compliance cost; general
equilibrium; and dynamic (accounting for innovation)
• Full social benefits – in a multirisk world, both target risk and ancillary
impacts (co-benefits, countervailing risks, substitutes); interconnectedness
• Optimal analysis of additional “ripples” (proportionate analysis)
• Broader comparative studies
•
•
•
•
•
•
•
IA of IA (counterfactual, and varying elements)
Across countries (beyond US-EU – go global), with hybridization/borrowing
Across risks and policies (avoid sampling bias)
Across types of policy instruments (e.g. EIA, RIA, ROB, MBI, PP)
Across types of RIA and ROBs (varying functions, powers, structure)
Across agencies (varying quality and influence of IA, and potential evasion)
Toward a “global policy laboratory”
Thank you.
www.law.duke.edu/fac/wiener