Coproduction in ECoC evaluation
Developing an innovative monitoring and evaluation
method for impact assessment and learning
Utrecht School of Governance
Utrecht University
The Netherlands
Aline Bos | Albert Meijer | Daniëlle Fictorie
1
Contents
1. Background: Utrecht candidate ECoC 2018
2. Monitoring and evaluation (M&E): state of the art
3. Utrecht M&E system
a. Communities of knowledge
b. European dimension
c. Smart city and trust
4. Conclusion and discussion
2
1. Background




Utrecht: candidate European Capital of Culture 2018
Development M&E system
Building on existing systems (i.e. Liverpool, Turku)
Supplementing existing systems:
-
Participatory approach
European dimension
3
2. Monitoring and evaluation (M&E)
Purposes of M&E - in existing M&E of ECoCs:
 Impact assessment
 Public accountability
However, other purposes may be:
 Institutional learning/ organizational strengthening
 Understanding and negotiating stakeholder perspectives
(Swanborn 1999)
4
2. Monitoring and evaluation (M&E)
Coproduction: participatory M&E (PM&E)
Who
What
Conventional
Participatory
External experts
Community members, facilitator
Predetermined indicators (mainly
People identify their own
cost and production outputs)
indicators of success
‘Scientific objectivity’, distanced
How
evaluators, procedures, delayed
access to results
When
After program or mid-term
Why
Accountability
Self-evaluation, methods adapted
to local culture, immediate sharing
More frequent, small-scale
evaluations
Empower local people to initiate,
control and take corrective action
See also Guba & Lincoln (1989), Estrella & Gaventa (1998), Jackson (1995), McKenzie et al. (2006)
5
3. Utrecht M&E system
Participatory and conventional methods/principles combined
= significant innovations: systematic and effective learning
while enhancing accountability
(Gaventa & Blauert 2000, Guijt 1999)
Utrecht M&E: hybrid system
Conventional methods:
Accountability
Utrecht
M&E system
Participatory methods:
Learning
6
3a. Utrecht M&E system:
communities of knowledge
Utrecht M&E system:
- M&E in ‘communities of knowledge’, based on social
presence, motivation to share knowledge and
collaboration
- Collective learning, mutual sensemaking
- Capturing know-how not easily articulated
- Key communities and peripheral communities
Qualitative insights from the communities of knowledge will
be combined with more quantitative insights. Moreover,
communities of knowledge can also facilitate the access to
quantitative data.
7
3b. Utrecht M&E system:
European dimension
European dimension in impact assessment ECoCs:
1. M&E in Europe: European input in the M&E system
2. M&E with Europe: European learning
3. M&E for Europe: Contributing to ‘Europe in the making’
(see next slide)
8
3b. Utrecht M&E system:
European dimension
Identity approach
Client approach
-
People define themselves as European
-
People recognize European symbols
-
People can mention European values
-
People value the contribution of Europe to the
development of a cultural climate
-
People value the idea that countries influence
each others cultures
Citizenship approach
-
People define themselves as European citizens
-
People know their rights as European citizens
9
3c. Utrecht M&E system:
Trust and smart cities
Link the M&E system to the central ECoC theme.
In Utrecht: ‘trust’ and ‘smart cities’
(to be elaborated later on)
10
3. Conclusion and discussion
ECoC M&E can be complemented by:
- Strengthening the learning processes
- Strengthening the European dimension
Discussion:
- How to reconcile the tension between the different
approaches?
- How to compose the communities of knowledge in a
sustainable and useful way?
11