Monitoring and Evaluation
ECO-UNESCO’s experience
Elaine Nevin
National Director
ECO-UNESCO
NYCI Conference 2011
Who we are?
A National Environmental
Education and Youth
Organisation. Est. in 1986
environmental education
and environmental work with
young people.
Affiliated to the World
Federation of UNESCO Clubs,
Centres and Associations
(WFUCA)
Our Mission
To channel young
peoples passion into
environmental
protection and
conservation and to
promote the personal
development of
young people.
Our Aims…
•Promote environmental awareness, knowledge and
understanding in young people
•Promote environmental protection & conservation
•Promote the personal development of young people
•Promote UNESCO ideals
What we do…
Environmental Youth programmes
ECO-UNESCO Clubs
Young Environmentalist Awards
Programme
Environmental Workshops
Youth Forums, events and activities
ECO Choices programme
Youth for Sustainable Development
programme
Education and Training programmes
Short Trainings – Teachers, Youth
Leaders & Community Worker
FETAC accredited centre
Sustainable Development L5
ECO-Community Development L3
Educational Publications
Various interactive resources
(Print/Media)
Representation & Advocacy
What we do…
ECOUNESCO
Clubs
Youth for
Sustainable
Development
PEP
Young
Environmentalist
Awards
ECOChoices
programme
Childrens
Progs
Mission
Teacher
and Leader
Training
Accredited
trainings
Education
resources
General
training
The Scale of our Work
•Over 9500 people annually
•National Basis / YEA all-Ireland programme
•A team of 10 staff members
• 2 government work placements at any one time
• support 6 graduate interns
• over 100 volunteers
What we want to know…M&E

How many people/organisations/
schools are accessing our programmes and
services at any one time and over a period?

What are the demographics of these people
- age, gender, geographical spread etc?

What is the retention rate of our
programmes (for longer term direct contact
programmes) – do we have a high fall off
rate?
What we want to know…



What have people gained from participating in
an ECO-UNESCO workshop, programme or
training? - knowledge, skills developed, attitudes,
behaviour change, how much have they enjoyed
it etc.
What is the impact of our programme on the
participant?
(impact from a range of perspectives – due to the
nature of our work)
What we want to know…

Are we delivering programmes that address a
need?

As an organisation are we achieving our goals,
targets and expected outcomes?

Are we delivering quality programmes?
Why do we monitor & evaluate?

Out of our own desire to develop quality
programmes; review, learn, modify &
improve

Out of our own need to ensure our delivery to
greater numbers of young people

To ensure we are doing the job the way we
should be doing it, with the people we should
we doing it with, achieving the results
addressing the needs of young people with an
impact on our participants

Are we responding to needs of young people?

Is our M & E funder driven?
Some of our tools of Monitoring &
Evaluation


Performance Management System – Planning,
developing, monitoring and appraisal
Before and after questionnaires on knowledge,
attitude and behavior (baseline/endline)- for
programmes

Analysis of programme participant feedback on
evaluation forms

Specific programme evaluations e.g. YEA evaluations

Other tools –
 Youth worker self-evaluation forms post youth
programme/training /workshops
 people counter to count numbers of people visiting
the Greenhouse etc.
Samples of some of our tools of
Monitoring & Evaluation measure

PLANNING (Qualitative/Quantitative)
 Annual Implementation Plan

MONITORING (Quantitative/Quantitative)
 Direct Participant Analysis Information
 Direct Participant Feedback on Learning & on Satisfaction
 Evaluation forms as part of programmes e.g. YEA
Through YEA/Clubs/YSD, analyse what, how many and
the impact of environmental action projects;

EVALUATION
 Reviews carried out
 Impact Assessment
SAMPLE 1: PLANNING - Annual Implementation Plan –
Organisational Plan for a year
Programme - Strategic Aims
Objective
Actions
Projected
Output
KPIs (Key
Performance
Indicator)
Review on a quarterly basis
Outcome
Target
Group
Time
Frame/
Deadline
Result /
(Progress)
SAMPLE 2: MONITORING -Participant Analysis – Demographics etc.
Age range, gender breakdown & geographical spread of
participants
<10
DATE
10-14
22-24
NEW
PARTICIPANTS
>25
TOTAL
PARTICIPANTS
M
Mth
18-21
NAME
OF
EVENT
PROG
CODE
Day
15-17
F
M
F
M
F
M
F
M
F
M
F
Year
Monthly/annual programme -MONITORING
Program m e & Se rvice Outputs
Octobe r 2011
YOUTH; 12;
YSD; 6; 14%
29%
YEA; 9; 21%
YS D
TRAIN; 7; 17%
EYC; 8; 19%
YEA
EYC
TRAI N
YO U T H
CONTACT
HOURS
SAMPLE 3: MONITORING - Participant Feedback on
Satisfaction & Learning
ECO-UNESCO measures the following:
• Content relevance
• Process (quality of facilitation, presentation and group work/activities)
• Resource materials
• Learning
We analyse reactions and comments to enhance quality of future work
Drim nagh Cas tle Oct 09 to Jan 10
0%
2%
8%
1
2
46%
3
4
44%
5
46% - Excellent
44% - Very Good
8% - Good
2% - Fair
0% - Poor
3
SAMPLE 3: MONITORING - Participant Feedback on learning
Youth Leader Training Course
Participants carry out Self Assessment of Learning in terms of knowledge,
skills & attitudes – before & after each activity/ programme
Self-assessment of Your understanding,
knowledge
and Skills Related to:
Before Course
1
1
1
2
2
2
3
3
3
4
4
4
5
Q1. What Environmental Education
(Education for Sustainable Development) is
After Course
1
2
3
4
5
5
Q2. How to integrate interactive
Environmental Education activities into your
daily work
1
2
3
4
5
5
Q3. How to encourage critical thinking in
young people through Environmental
Education
1
2
3
4
5
Q4. How to foster Environmental Awareness
in Young People through Environmental
Education
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
Q5. How to carry out an Environmental Action
Project (Energy Audit) with Young People
1 = No understanding or skills 3 = Some understanding or skills
5 = A lot of understanding or skills
SAMPLE 4: Impact Assessment - PILOT INITIATIVE 2010-2011

In 2010, we wanted to look at how we could measure the impact of our
programmes; reviewed and decided to implement an Impact Assessment
mechanism

Impact Assessment Workshops (3 to 6 months post delivery)
Workshop used to measure two of our programmes:
 Youth for Sustainable Development Peer Education programme
 FETAC Level 5 Sustainable Development programme
The method/workshop measures in two ways:
 In relation to what use and in what ways the learning has been
applied in practice by the participant/learners themselves after
completing the ECO-UNESCO programme
 In a broader context how much the ECO-UNESCO programme has
impacted on them and is of use to society in general and on their
communities?


ECO-UNESCO took the role of facilitator
Former participants become the impact assessors
Sample Impact Assessment Grid
Sample Impact Assessment Stories
Story 1
Name: Frida Besong
“Thanks to my participation in ECO-UNESCO’s FETAC Level 5 Sustainable Development
course, my interest was aroused on Sustainable Development issues and I finally wrote my
M.A thesis on International cooperation regarding the management of water in Lake Chad
and other natural resources in the Lake Chad Basin. At this stage I have applied for a P.H.D
In Education for Sustainable Development and I am waiting for the outcome”
Story 2
Name: Julia Lane Barnes
“My update since I finished ECO-UNESCO’s FETAC Level 5 Sustainable Development course is
that I am involved with working with a local interfaith youth group with an emphasis on the
common theme of the inner spiritual garden. We use lots of the methodology that I learned
doing ECO-UNESCO’s FETAC Level 5 Sustainable Development course. Our long term aim is
to make a link between the restoration of a local walled garden and going to Mount Sinai
with the Makhad Trust to help mend the walled gardens of the Bedouim.”
Story 3
Name: Helen
“I helped organize the Greenhouse gig fashion show (recycled materials), which was
about organization – there was loads of organization in it. And it was about making
people aware about climate change & development, so that was Peer Education.”
(about 50 to 60 Peers)
Challenges

Time - Small organisation with limited capacity – want to do the
job and make sure we are doing the job well – but need to
balance this with getting the job done – can be a challenge.

Buy in from all involved - depending on people to collect data
and information and to input it into a system (Salesforce CRM)

Making sure what you are monitoring and evaluating is what you
need to be monitoring and evaluating – using the information you
get
M & E needs to focus on Quality as much as Quantity.



We gradually implemented the M & E system – but it is under
constant review
Specialist organisation with diverse funding streams with diverse
requirements in terms of outcomes and outputs
Future plans…2011 onwards
Review current M & E system – ensure it is useful
and informing our work in the way it should be
Ensure it is in line with and collecting necessary
information for the implementation of National
Quality Standards Framework - use this as a tool.
For further information please contact
ECO-UNESCO
'the greenhouse'
17 St., Andrew St.,
Dublin 2
T: 00 353 (0) 1 6625491
F: 00 353 (0) 1 6625493
E: info@ecounesco.ie
W: www.ecounesco.ie
ECO-UNESCO – Conserving the Environment; Empowering Young People
ECO-UNESCO
The Greenhouse, 17 Saint Andrew Street, Dublin 2,
tel +353 1 6625491
www.ecounesco.ie
email: info@ecounesco.ie