World Bank Country Office Kyiv, Ukraine
Monitoring & Evaluation for Results
M&E Training Course
World Bank Country Office
Kyiv, Ukraine
June 9-13, 2008
A. Core Session: June 9-11, 2008
B. Specialized/Advanced Topics Session: June 12-13, 2008
_________________________________________________________________
Background and Key Information
The World Bank Country Office in Kyiv, Ukraine, and the World Bank Institute Evaluation
Group will sponsor a course, Monitoring and Evaluation for Results, from June 9-13, in
Kyiv, Ukraine. The course will be broken into two sessions, with core sessions held from
June 9-11 and the specialized/advanced topics session on June 12-13. While the course
will draw participants largely from Ukraine, Moldova and Belarus, it is open to other
World Bank clients and staff who work in the Europe and Central Asia (ECA) region.
The internal and external demands for more regular, more rigorous, and better quality
M&E practices require program teams from government agencies to civil society to be
better equipped to plan, implement, and oversee a variety of M&E tasks. In some cases
the program teams will undertake all of this work. In other cases they will carry out
some of it, such as program monitoring, but not have responsibility for other parts, such
as evaluations carried out by an independent evaluation body. In order to make sound
choices that will result in well planned programs and M&E functions – ideally this should
be done at the program design stage – program teams should understand M&E methods
and options. Given this, the course aims to expose key M&E approaches to participants
who are expected to take a lead role in more systematic M&E use and improved
programs, projects, or policies.
Course instruction, led by two senior M&E experts from the World Bank, will be
conducted in English, with simultaneous interpretation into Russian throughout the
course. All materials will be available in Russian and English. CD ROMs of all training
materials (in English and Russian) will also be provided to participants. The course will
feature group work in the development of common M&E tools, techniques and
approaches.
Sessions
A. Core Session: Essentials of M&E Concepts and Methods
This three-day session is intended to provide an overview of M&E methods. The
workshop will provide participants with the basic M&E tools, techniques and resources
needed for planning, organizing, and/or managing programs and projects. Through
exercises and group works participants will learn the links between performance
measurement, monitoring and program evaluation. This session will serve as a
prerequisite to the specialized/advanced topics session. Note that persons who
completed the November 2007 M&E course at the World Bank in Kyiv are not required
World Bank Country Office Kyiv, Ukraine
to complete this session and may wish to complete only the specialized/advanced topics
session.
Topics
This session will feature the following topics:
 Common M&E terminology usage, tools, techniques and approaches
 Use of logic models for planning and program articulation
 Determination of objectives and evaluation questions
 Evaluation design strategies and options
 Indicator and measurement strategies
 Data collection methods
 Sampling methods
 Data analysis methods
 Reporting options
 Issues of utilization of M&E information
Group Projects – Using Common M&E Tools, Techniques and Approaches
The session features group work, allowing participants to gain experience in developing
and using some common M&E tools, techniques and approaches. Group work will
involve the development of the following tools: (a) logic models, with indicators and
measurement strategies; (b) monitoring plans, which involve the specification of
indicators, measurement strategies, baseline data, targets, data collection strategies and
data sources, assumptions and risks; and (c) evaluation plans, which contain most of
the elements of monitoring plans, but with more information about evaluation questions,
designs, sampling strategies, analysis strategies and other topics required in planning an
evaluation.
B. Specialized/Advanced Topics Session: Exploring More Tools and
Techniques
This two-day session provides in-depth presentations of specialized and more advanced
M&E topics. Time will be allotted for practical exercises and informal question and
answer sessions around common M&E problem areas. Resources for further study and
research in the M&E field will be provided. Participants who complete the core session
are expected to complete this specialized/advanced session. This session also targets
those who have previously completed the core sessions (e.g., at the November 2007
Kyiv course, or the same course offered at an earlier date in the ECA region).
Topics
This session will feature the following topics:
 Strengthening a monitoring system
 Managing M&E activities
 Terms of reference for evaluation
 Evaluation ethics
 Quantitative and qualitative methods
 Pros and cons of impact evaluation
Format
The course will include instructor presentations, case studies, individual and group
exercises.
World Bank Country Office Kyiv, Ukraine
Presenters
The course presenters from are:
Richard Tobin Lead Monitoring and Evaluation Specialist
Jaime Quizon, Senior Evaluation Officer
Maurya West Meiers, Evaluation Analyst
Sara Okada, Evaluation Analyst/Consultant
Participant Selection
Ideally, participants will have the following profiles:
 a basic understanding of M&E/research methods
 role/position in their respective agency where they have some
responsibility for M&E-related functions involving M&E in project
management, research, management, training, mentoring or advising;
 confirmation of availability to attend the full sessions in which they are
enrolled; and
 English proficiency is preferred, but candidates who otherwise meet the
above criteria and are non-English speakers are also encouraged to attend
(simultaneous interpretation and translation will be provided).
Schedule (Tentative)
The course will begin at 9:00 am and end at 5:30 pm each day. The morning
and early afternoon sessions will feature instruction on core modules and
instructor/participant ‘debriefing’ sessions. The mid-afternoon sessions will
feature group work. On June 11th, the organizers will host an early evening
reception. Certificates will be awarded at the conclusion of the course. A
tentative agenda is enclosed.
More Information
Further information about the content of this course can be obtained from Ms.
Sara Okada (sokada@worldbank.org) in Washington, DC, or Yulia Snizhko
(ysnizhko@worldbank.org) in Kyiv. For logistical matters, please contact Victoria
Yevseeva (vyevseeva@worldbank.org) in Kyiv.
Additional information about M&E training offered by the World Bank Institute
Evaluation Group is at www.worldbank.org/wbi/evaluation/training.
World Bank Country Office Kyiv, Ukraine
COURSE AGENDA
Kyiv, June 9-13, 2008
Monday, June 9, 2008: Essentials of M&E Day 1
Time Estimate
Topic
Welcome, Introductions and Overview of Series and Materials
 Ice-breaker (size appropriate)
 On note card: Why are you here, how does it relate to your job
9:00 – 10:00
and what would you like to take away from this training?
 Overview of course material
 Short presentation of previous workshop participant(s)
10:00 – 11:00 Module 1: Introduction to M&E Definitions and Basic Concepts
 What is evaluation? Definition and Examples
 What is monitoring? Definition and Examples
 Why is it important? Discussion and Suggestions
 What are some challenges and some enablers? Discussion and
Suggestions
11:00-11:15 Coffee Break
11:15-12:15 Module 2: Results Chain
 Results Chain, Elements and Causality
 Determining objectives
 Evaluation plan and evaluation questions
12:15-1:15 Lunch
1:15-3:00 Module 2: Case Exercises
 Create a Results Chain
 Develop Evaluation Questions
3:00-3:15 Coffee Break
3:15-5:30 Group Work (Participants’ own project)
 Create a Results Chain
 Create Evaluation Plan, Monitoring Plan or Results Framework
Tuesday, June 10, 2008: Essentials of M&E Day 2
Time Estimate Topic
9:00-9:15 Review of Day 1
9:15-10:45 Module 3: Indicators and Measures
 Basic concepts of indicators and measurement and their
characteristics
 Importance of defining key terms
 Key measurement issues – relevance, validity, reliability:
examples
10:45-11:00 Coffee Break
11:00-12:00 Module 3: Exercise
 Selecting indicators and measures for Results Chain
12:00–1:00 Lunch
World Bank Country Office Kyiv, Ukraine
1:00-3:00 Module 4: Evaluation Design
 Elements of Design
 Types of Design: pros and cons
 Common Designs for Program Evaluation
Module 4: Case Exercise
 Choose the most appropriate design for a given project
3:00-3:15 Coffee Break
3:15-5:30 Group Work (Participants’ own project)
 Create a Results Chain
 Create Evaluation Plan, Monitoring Plan or Results Framework
Wednesday, June 11, 2008: Essentials of M&E Day 3
Time Estimate
Topic
9:00 – 9:15 Review of Day 2
9:15-10:45
Module 5, 6 ,7: Introduction to Data Collection, Sampling, Data
Analysis
 Quantitative/Qualitative Data
 Data Collection methods: Exercise
 Targets, Baseline and Monitoring
10:45-11:00 Coffee Break
11:00-12:00 Module 5, 6 ,7: Cont’d
 Sampling: Examples
 Data Analysis methods: Exercise
12:00-12:30 Conclusions: Utilization and Reiteration of key messages
12:30–1:30 Working Lunch: Case Exercise Finalization
1:30-2:30 Group Presentations of Results Chain & Comments
2:30-2:45 Coffee Break
2:45-4:45 Group Presentations of Results Chain & Comments
4:45-5:30 Evaluation and Certificates
World Bank Country Office Kyiv, Ukraine
Thursday, June 12, 2008: Specialized Topics in M&E Day 1
Time Estimate
Topic
8:30 – 9:30
Welcome, Introductions, Overview and Materials
 Ice-breaker
 Refresher on Definition, Terminology, etc
9:30-10:30
Module 1: Strengthening a Monitoring Plan
 Setting Objectives
 Baseline Data
 Data Collection for Monitoring
 How to Use Monitoring Data for Project Management
10:45-11:00
Coffee Break
11:00-12:30
Module 1: Exercises
 Creating an Indicator Monitoring Plan for Your Project
12:30-1:30
1:30-3:30
Lunch
3:30-3:45
3:45-5:30
Coffee Break
Module 2: Quantitative and Qualitative Evaluations
 Strengths and Weaknesses of Quantitative and Qualitative
Evaluations
 Mixed Methods
Module 2: Exercises
Quantitative & Qualitative Data Collection & Data Analysis
 Assessing and Using Existing Data (Discussion)
 Observation (Examples and discussion)
 Surveys (Group work on survey questions)
 Focus Groups/Interviews (Role-play and discussion)
 Content Analysis (Individual/Group work)
Friday, June 13, 2008: Specialized Topics in M&E Day 2
Time Estimate
Topic
9:00-10:00
Module 3: Pros and Cons of Impact Evaluation
 Different Types of Evaluation
 What is Impact Evaluation? (Overview, types of different design
including natural experiments)
 Reality of Impact Evaluation (Cost, Time, Technical Issues)
 Keith MacKay’s Presentation video (National M&E System)
10:00-11:00
Module 3 (Cont’d)
11:00-11:15
Coffee Break
11:15-12:30
Module 4: Managing Evaluation
 Creating Good TORs
 Managing M&E Consultants
World Bank Country Office Kyiv, Ukraine


Utilizing Evaluation Results
Evaluation Association
12:30-1:30
1:30-3:30
Lunch
3:30-3:45
3:45-4:45
Coffee Break
4:45-5:30
Evaluation and Certificates
Module 4: Exercises
 Creating TORs
 Q&A
Module 4: TBD*