DCAF-ISSAT Draft OGN on evaluation
Example of Evaluation Question
For a useful list of evaluation questions per evaluation criteria, see: Saferworld “Evaluating for security and
justice”, 2009, p. 29-30
Example of the ADE (for the European Commission) “Thematic Evaluation of the European Commission
support to JSSR”, 2011
EQ: To what extent has the Commission’s support to JSSR in partner countries been in line with its
policies and objectives in that field and with wider EU development objectives, and what has been the
Commission’s added value?
Rationale and coverage of the question:
This evaluation question addresses the dual nature of JSSR in EU external action: it is a prerequisite as well as a component of
economic and social development in countries and regions where the EU provides development assistance, and at the same
time it is a part of EU security and crisis management policies.
The question covers the issue of relevance from two angles:
a)
Firstly, it will be verified whether at the programming stage the Commission’s support to JSSR has reflected the
priorities and principles set out in the key EU concepts and policies that are relevant to JSSR, including A Concept for European
Community Support for Security Sector Reform, COM (2006)253, the Council Conclusions on a Policy Framework for Security
Sector Reform (2006), the EU Concept for ESDP Support to Security Sector Reform (2005) and Council Conclusions on Security
and Development (2007). As the key EU policy documents in the field of JSSR have only been issued recently, the alignment of
the Commission’s support to key internationally agreed policies and guidelines (such as those issued by the OECD DAC) will also
be assessed, in particular for interventions decided early on in the evaluation period.
b)
Secondly, the suitability of this support will be assessed against the broader development and security objectives of
the EU as set out in the Commission Communications on Conflict Prevention (2001) and on Governance and Development
(2003), the European Security Strategy (2003), the EU’s Development Policy Statement, and the European Consensus on
Development (2005).
Moreover, this question covers the Commission’s added value in JSSR as outlined in its main Communication on security system
reform (COM(2006)253, section 4.3). This Communication specifies a number of distinct features of the way that the
Commission operates which enable it to provide added value in support of JSSR processes (e.g. the Commission’s supranational
nature, its long-term presence on the ground, and its commitment to policy coherence for development).
Evaluation criteria / key issues and link with intervention logic:
The question addresses the relevance of Commission JSSR support to general Commission objectives and the coherence with
wider EU development objectives. It also examines the added value provided by the Commission in support of JSSR as outlined
in its main concept paper on JSSR
The question spans all the layers of the intervention logic.
Source of information and data
Sub-question 1
The Commission’s support to security and justice reform
collection tool
has been informed by EU and internationally-agreed
policies and concepts relevant to JSSR.
Indicator 1.1
Indicator 1.2
Indicator 1.3
Commission strategy and programming documents explicitly referred
to key EU strategy and policy documents on JSSR or to internationallyagreed policy statements on JSSR.
Commission interventions were designed with a view to their
contribution to strategic approaches to JSSR (e.g. based on a strategic
security and governance assessment, etc.)
Explicit or implicit reference to the principles guiding Commission
support for JSSR (as laid down in COM(2006)253 based on the OECD
DAC guidelines) in the programming documents of Commission
Offical Communications /
policies, CSPs/RSPs
Programming documents,
assessments, answers to
questionnaire
CSPs/RSPs, Programming
documents
Page 1 of 2
DCAF-ISSAT Draft OGN on evaluation
Indicator 1.4
interventions
Views of stakeholders about the alignment of the Commission’s JSSR
support with EU and internationally agreed JSSR policies
Sub-question 2
The Commission’s support to security and justice reform
was consistent with wider EU development objectives
Indicator 2.1
Commission strategy and programming documents explicitly referred
to the overarching EU policy commitments (e.g. EC Treaty, EU
Consensus on Development, etc.).
Commission interventions were designed with a view to fostering an
integrated approach to promoting security and development.
Indicator 2.2
Indicator 2.3
Indicator 2.4
Commission interventions explicitly took into account key crosscutting issues highlighted by overarching strategy documents (such as
gender, human rights, good governance, and environment) at
programming level and during implementation.
Views of EU stakeholders (Commission, Council and EUMS staff at HQ
and field level and EU Parliament) on the consistency of JSSR
interventions with the wider goals of EU external cooperation.
Interviews, answers to
questionnaire
Source of information and data
collection tool
Offical Communications /
policies, CSPs/RSPs, answers to
questionnaire
CSPs/RSPs, programming
documents, interviews,
answers to questionnaire
CSPs/RSPs, programming
documents
Interviews, answers to
questionnaire
Sub-question 3
The Commission had a particular role and strengths in
supporting security and justice reform
Source of information and data
collection tool
Indicator 3.1
Programming documents at intervention level identify the
Commission added value when intervening in the area of JSSR
The Commission’s long-term presence on the ground and its global
reach has facilitated its contribution to JSSR processes
The Commission’s supranational nature has enabled it to support JSSR
processes in ways which were not possible for other actors, including
bilateral donors
Evidence that the variety of the Commission’s financial and nonfinancial instruments is a critical advantage in supporting JSSR
compared to other donors
EUMS gave a specific mandate to the Commission to support JSSR
processes
CSPs/RSPs, interviews, answers
to questionnaire
CSPs/RSPs, interviews, answers
to questionnaire
Interviews, answers to
questionnaire
Indicator 3.2
Indicator 3.3
Indicator 3.4
Indicator 3.5
CSPs/RSPs, interviews, answers
to questionnaire
Interviews, answers to
questionnaire
Page 2 of 2