Theory and Practice of Regulatory Impact Assessments
in Europe: A Comparative and Interdisciplinary
Perspective
10 June 2013
LEARNING ABOUT QUALITY IN IMPACT
ASSESSMENT: METHODS AND RECIPES
Dr Claire A. Dunlop
University of Exeter, UK
Project: Analysing Learning in Regulatory Governance
(ALREG) sponsored by the European Research Council (ERC)
Assessing IA Quality: Menu of Methods
• UK National Audit Office (NAO) approach
• Scorecard approach (Hahn for US,
Renda/Schrefler for EU, Chittenden for UK)
• OECD method of ‘thick’ peer review and
documentary analysis
• UK Regulatory Policy Committee (RPC)
• Explore the different paths and sequences to
usage rather than quality via qualitative
comparative analysis (Exeter)
Think Usage, Not Quality!
Provide alternative analysis to standard mode
Why?
- costs of achieving ‘saturation’…
- practical difficulty…
- normative assumptions of what ‘quality ‘is…
Rather than focus on how ‘good’ an IA, we draw
on case studies to explore what it looks like on
the ground
How it is ‘used’ – politically (control);
instrumentally (evidence-based policy);
communicative (dialogue); perfunctory
(symbolic)
Research Questions
1. How is RIA shaped during the process of its
implementation?
2. What are the combinations of conditions that
lead to different usages?
3. How can this inform policymaking?
Data, Coding and Methods
31 cases studies (12 UK, 19 EU), 2003, 04, 05, 06
Qualitative case studies NOT actual impact assessments
Meta-analysis of case studies by coding 33 variables in 0-1
format
2 Methods:
• Univariate analysis used to identify four types of usages of RIA
– political, instrumental, communicative, and perfunctory
• Qualitative comparative analysis (QCA) used to explore the
different combinations of conditions that lead to different
usages of RIAs
Variables (1-11)
Variables (12-21)
Variables (22-30)
Variables (31-33)
Univariate Findings
Qualitative Comparative Analysis (QCA) Findings
Conclusions
Analysis of pathways to usages as a complement to
scorecard approach
Configurational analysis– helps uncover complexity
underpinning RIA implementation and usage
Equifinality! We can get to ‘quality’ in different ways
Appraisals are framed and reframed by communities of
practice
Policymakers role in managing expectations of what a RIA is
for
Governments, international organizations and audit bodies
should explore different combinations of conditions
leading to the usages they deem desirable