Linking modes of governance
and social-ecological outcomes in
environmental evaluation
Outline of an evidence-based approach
Edward Challies, Nicolas Jager, Jens Newig, Ana Adzersen
Outline
 Introduction: Participatory environmental governance
 Hypotheses: Participation in environmental policy-making
 Methodology: Meta-analysis
 Key Contributions
2
Participatory environmental governance
 Rise of participatory governance, 1970s →
 State failure, ‘inclusive government’ and ‘good governance’
 Instrumental and democratic/emancipatory value of participation
 Tensions in citizen participation
 Participatory environmental governance
 From social side-effects to social-ecological synergies?
3
Hypotheses: Participation in
environmental policy-making
 Hypothesised causal links between:
 participation and policy effectiveness (implementation, compliance)
 participation and environmental standards of the output
4
Conceptualising goals of participation
instrumental
quality of the decision
itself
incorporation of
local knowledge
emancipatory
quality of
implmentation
knowledge of
stakeholder
positions
acceptance and
identification
education
and capacity
building
pre-emptive
legal
protection
balancing of
interests
building
trust
transparency
and
accountability
democratic
policy
formulation
addressing
conflicts
ProceduralJustice-effect
5
Participation  policy effectiveness
+/Implementation & Compliance (outcomes)
- Institutional fit/ compatibility
- Knowledge for implementation
- Conflict resolution
- Representation of diverse interests (incl. veto players)
- ‘Win-win’ scenarios
- Discursive fairness
- Procedural justice
- Education of policy addressees
- Network creation
- Waking sleeping dogs
- Demand-institutional misfit
6
Participation  environmental standards of output
+/Environmental standards of the policy/decision (output)
- Representation of environmental actors
- Environmentally relevant knowledge
- ‘Common good’
- ‘Positive sum game’ solutions
- Innovative/creative solutions
- Disadvantaging environmental actors
- Co-optation of environmental interests
- Increased likelihood of veto
7
Example: Procedural justice effects
Participatory
process
Perceived
legitimacy &
fairness
Increased
acceptance of
the output
Improved
implementation
and compliance
8
Example: Negotiation effects
Participatory
process
Communication
& bargaining
Optimal
allocations &
positive sum
solutions
Higher
environmental
standards of the
output
9
Methodology
 Meta-analysis - case survey
 Systematic aggregation of single small-N case study data
 Precise coding according to theoretically informed scheme
 transforms qualitative into quantitative data
 combines richness of case material with scientific rigour of large-N
comparative analysis - seldom applied
 Screening of 2,871 texts for possible cases
 Population of 572 cases
 Random sample of 200 cases
10
Conceptual framework for cross-case analysis
C ontext
P ro b le m s tru c tu re , in s titu tio n a l a n d a c to r c h a ra c te ris tic s
P rocess
S ocial outcom es
F o rm o f
p a rtic ip a tio n ,
fa irn e s s
L e a rn in g , tru s t,
a c c e p ta n c e
S ubstantive output
S ubstantive
outcom es
Im p le m e n ta tio n a n d
c o m p lia n c e
E nviron m ental
im pact
C h a n g e s in
e n v iro n m e n ta l
q u a lity
E n viro n m e n ta l
s ta n d a rd o f d e c is io n
11
Case-survey analysis – variable coding
 Description of the case material through semi-quantitative
variables
DISC FAIR
s-q
[0..4]
(99)
Discursive fairness: Degree to which the DMP was executed through a process of fair
discourse.
Indicators include: all participants must be able to attend, make statements, participate
in the discussion, and participate in the decision-making (Webler & Tuler 2000: 569).
0 = DMP was not discursively fair, but highly discriminatory;
2 = DMP afforded participants limited opportunity to engage in fair discourse;
4 = DMP was characterised by fair discourse.
 Coding done by three coders
 Resulting data set allows for statistical analysis, QCA, etc.
12
Counterfactual reasoning
counterfactual situation with
lower intensity of factor 1
current state
counterfactual situation with
higher intensity of factor 1
Factor 1
↑
Factor 2
Factor 2
↑
+ causal relationship
Factor 1
↓
Factor 2
↓
Factor 1
↑
Factor 2
↓
- causal relationship
Factor 1
↓
Factor 2
↑
Factor 1
13
Key contributions
 Conceptual: Exploring mechanisms by which, and circumstances under
which, participation can improve decision/policy-making effectiveness and
environmental standard of policies.
 Policy: Evidence-based insights to support decision/policy-makers to achieve
environmental policy goals in a socially and environmentally sustainable way.
Aid decision-makers in characterising policy contexts (e.g. environmental
issue, stakeholder field, etc.) and selecting suitable modes of participation
given specific goals.
 Methodological: Large N meta-analysis; investigation of causality via
hypotheses and counterfactual reasoning.
 Proposition: Environmental policy evaluation should take stock of the role of
public participation in securing both environmental and social outcomes, and
not see the latter as mere side-effects.
14
EDGE Project
Evaluating the Delivery of Environmental Governance using an
Evidence-based Research Design (EDGE)
Project team:
Prof. Dr Jens Newig,
Dr Edward Challies
Nicolas Jager M.A.
Research Group on Governance, Participation and Sustainability
Leuphana University
Scharnhorststraße 1
21335 Lüneburg
Germany
Project timeframe: 2011-2016
http://www.leuphana.de/institute/infu/forschung/governance-nachhaltige-entwicklung.html
15
References












Bulkeley, H., & Mol, A. P. J. (2003). Participation and Environmental Governance: Consensus, Ambivalence and Debate.
Environmental Values, 12(2), 143-154
Cooke, B. (2001). The Social Psychological Limits of Participation? In B. Cooke & U. Kothari (Eds.), Participation: the new tyranny?
(pp. 102-121). London, New York: Zed Books.
Delli Carpini, M. X., Cook, F. L., & Jacobs, L. R. (2004). Public Deliberation, Discursive Participation, and Citizen Engagement: A
Review of the Empirical Literature. Annual Review of Political Science, 7, 315–344.
Fritsch, O., & Newig, J. (2009). Participatory governance and sustainability: Early findings of a meta-analysis of stakeholder
involvement in environmental decision-making. In E. Brousseau, T. Dedeurwaerdere & B. Siebenhüner (Eds.), Reflexive
governance for global public goods. Cambridge, Mass.: MIT Press.
Fung, A. (2006). Varieties of Participation in Complex Governance. Public Administration Review, 66 (Special Issue), 66-75.
Innes, J. E., & Booher, D. E. (2004). Reframing public participation: Strategies for the 21st century. Planning Theory and Practice,
5(4), 419-436.
Lind, E. A., & Tyler, T. R. (1988). The Social Psychology of Procedural Justice. New York, London.
Mitchell, R. B. (2008). Evaluating the Performance of Environmental Institutions: What to Evaluate and How to Evaluate it? In O.
R. Young, L. A. King & H. Schroeder (Eds.), Institutions and Environmental Change. Principal Findings, Applications and Research
Frontiers (pp. 79-114). Cambridge, Mass.: MIT Press.
Newig, J. (2007). Does public participation in environmental decisions lead to improved environmental quality? Towards an
analytical framework. Communication, Cooperation, Participation (International Journal of Sustainability Communication), 1(1),
51-71.
Rowe, G., & Frewer, L. J. (2005). A Typology of Public Engagement Mechanisms. Science, Technology, & Human Values, 30(2),
251-290.
Swyngedouw, E. (2005). Governance Innovation and the Citizen: The Janus Face of Governance-beyond-the-State. Urban Studies,
42(11), 1991-2006.
Webler, T., & Tuler, S. (2000). Fairness and Competence in Citizen Participation. Administration & Society, 32(5), 566-595.
16