Guidelines for a comprehensive
Evaluation Plan
Efie Meletiou (Frosso.Meletiou@ec.europa.eu)
Unit 03 Evaluation and Impact Assessment, DG EMPL
Presentation to CY ESF Managing Authority
Nicosia, 26 October 2010
European Social Fund
A comprehensive Evaluation Plan
should
 Not be a list of planned evaluations!
 Show how exactly you will use evaluation as a management tool!
 Be based on The Intervention Logic!
…and on programme cycle
Be multi-annual, reviewed annually!
Identify information needs
…and link them to clear monitoring arrangements
Refer to management, timing, resources
Refer to dissemination and use of results
European Social Fund
A comprehensive Evaluation Plan
should be based on
The Intervention Logic
Society
Economy
Environment
Programme
Evaluation
Impacts
Needs
problems
issues
Objectives
Relevance
Results
Inputs
Outputs
Efficiency
Effectiveness
Utility
Sustainability
European Social Fund
A comprehensive Evaluation Plan
should be based on
the programme cycle
Diagnosis
Preparation
Ex-post
evaluation
Ex-ante
evaluation
Finalisation
Application
Implementation
Evaluation on-going
European Social Fund
A comprehensive Evaluation Plan
should identify information needs
 Assess evolution of programme in relation to objectives
themes (strategic prior. / regulatory requirement)
judgment criteria
specific evaluation questions
Information collection (what/when/how)
 links with monitoring
 additional fieldwork
…and definitions/thresholds on “important divergence”
 procedure for launching operational evaluation
An Evaluation Plan should make an explicit commitment
to all of the above
European Social Fund
A comprehensive Evaluation Plan
should identify information needs
 Think of addressing Added Value
 Volume effects: more/mirroring or boosting
 Scope effects: extending coverage, different from volume
 Role effects: novel actions/agenda setting
 Process effects: Community way of doing things/varies from
policy area to area
European Social Fund
A comprehensive Evaluation Plan
should identify methods of analysis
 How to make use of existing indicators
 How to make use of other tools
 How to combine and analyse
 Judgement criteria / baselines
 Participative evaluation?
 Meta-evaluations
European Social Fund
A comprehensive Evaluation Plan
should plan for resources
 Internal arrangements:
- Specific assignment of responsibilities?
 Contractual arrangements:
- Expert support to MA for whole period?
- Framework contract for evaluations?
- Contract for 2 evaluations per year?
 Steering committee:
- How inclusive?
European Social Fund
A comprehensive Evaluation Plan
should address
the dissemination and use of results
 Timing is crucial
-
policy cycle
budget review
stock taking need
opportunity for change
 Who is in charge?
 What are the procedures?
 A learning organisational culture (Sweden)
European Social Fund
Some final words
Think of evaluation as a resource, not as a way of looking
back and allocating blame; and not as an external
imposition.
Don’t carry out evaluations for their own sake. When
planning them think of how they can improve citizens’
lives. Identify all interest groups and users.
Ideally, the evaluation plan should be part of the
programming document
European Social Fund