Project Evaluation Plan – Guidance and Templates This document

advertisement
Project Evaluation Plan – Guidance and Templates
This document has been developed to help you create an Evaluation Plan for your Nominet Trust
project. It should take about an hour to go through the steps outlined below and create.
Project evaluation. You’re working hard to get your project off the ground; surely worrying about
evaluation is something that could wait for a bit? Well, unfortunately, no it isn’t. The best project
evaluations are those that are planned at the very outset. If you have a clear vision of what success
looks like to you, planning for evaluation will enable you to capture all the relevant information
throughout the project lifecycle that you need to demonstrate the impact you have made.
There are a variety of ways that you may choose to evaluate your project, ranging from self
reporting to external evaluation. While the Nominet Trust welcomes its partner projects using
existing evaluation approaches, we are equally committed to exploring new and alternative ways to
understand social impact. Whatever the approach, we emphasize the importance of understanding
what changes have resulted from your work, and what we can learn about the model/approach you
have used to achieve these changes.
Background to Monitoring & Evaluation
Project monitoring and evaluation should have a clearly defined workstream in your plan, with
resource allocated to it. There should also be a line item in your project budget specifically relating
to monitoring and evaluation, with a minimum of 5% of your project budget allocated to it, so that
finance is ring-fenced to allow you to deliver against these requirements in your Funding Agreement
with us. If you intend to use external organisations to perform your end of project evaluation, you
may need to allocate additional funding to this workstream.
During your project, we expect you to complete our online Project Monitoring forms at an agreed
frequency (usually quarterly). At the end of your project, you will be asked to complete our online
Evaluation form. Time invested in developing your Evaluation Plan at the start of your project will
allow you to deliver your monitoring and evaluation obligations more effectively.
Your Evaluation Plan
Before your project starts, we ask you to submit a two-part evaluation plan that tells us:
 How you will measure the changes that result from your work (the outcomes)
 How you will reflect on the benefits of your approach
Practically this means:
1. Telling us what outcomes you are choosing to measure, and how you will measure them
during the course of your project
2. Identifying assumptions about your approach, which you will reflect on at the end of the
project
This guide offers some step by step guidance to creating your evaluation plan:
 Step 1 looks at how to measure outcomes.
 Step 2 looks at identifying assumptions.
At the end of each step is a template to help you create your plan.
We highly recommend getting in touch with Nominet Trust’s Development Research team to help
you create your evaluation plan. We are always happy to speak on the phone, online or in person. If
Page 1 of 5
you would like to discuss any aspect of your project evaluation please contact us on
developmentresearch@nominettrust.org.uk
To view a presentation and read more about our evaluation approach, follow this link to the
Nominet Trust website:
http://www.nominettrust.org.uk/how-to-apply/guidance-notes/evaluating-your-project
Step 1 – Measuring Outcomes
As a social investor we’re interested in measuring ‘outcomes’, meaning the changes that happen in
society as a result of your project. There’s more detail below about how to identify your outcomes.
Deciding how to measure your outcomes can seem daunting at first, so we suggest breaking it down
into 5 steps:
1.
2.
3.
4.
5.
Identify your outcomes
Decide what you will use to measure your outcomes
Choose how to measure them
Choose when to measure them
Decide who will measure them
1. Identifying your outcomes
We recommend that you identify around three to five outcomes for your work. To do this, think about
what the main changes are that you want to see take place as a result of your project. This might include a
change for project participants or beneficiaries (e.g. increased skills or confidence in an area), a change in
the community, or even a change in the behaviour or an organisation or nation.
To begin with, it is important that you distinguish between the things that your project will produce - for
example a website, teaching resources or training sessions - and the changes (or outcomes) that these will
effect in the world. The things your project produces tend to be called ‘outputs’ (or deliverables). It can be
helpful to start by identifying your outputs, as these drive the changes that are the outcomes.
For example, a project might support young people seeking employment by training them to find work
online. The ‘output’ might be “running 10 training workshops for a total of 100 young people”. The hoped
for ‘outcome’ might be that “young people who complete the course are more confident about searching
for work online”.
It is important to separate these out because evaluation is about measuring the changes that have
happened as a result of a project (the outcomes) not just the things it has produced (the outputs).
Someone may have run 1000 training workshops, but we only know it’s a successful project if they have
achieved the outcomes they identified at the outset.
We’ve provided a table below to help you identify outputs and outcomes. If you are finding it difficult to
think about outcomes, it can be useful to list all the major ‘outputs’ of your project. And then think about
the outcomes that these ‘outputs’ are trying to achieve. So, in our example project, there might be an
entry “running 10 training workshops with 100 young people” in the output column and an entry “young
people who complete the course are more confident about searching for work online” in the outcomes
column.
2. Identifying what you will use to measure your outcomes
Once you have identified your project outcomes, you need to think about what you will measure to know if
you have achieved them. In our example “young people who complete the course are more confident
about searching for work online” we will want to measure their confidence about searching for jobs online.
Alternatively, if your work is about internet safety, you might want to measure if people are demonstrating
Page 2 of 5
safer behaviours online, and so on. Look at each of your outcomes and decide what you will need to
measure to know if there has been a change.
3. Choosing a way to measure your outcomes
Now you know what you are measuring, you need to think about how you will measure it. This could be
done through a range of different approaches depending on your project; for example interviews,
questionnaires, or usage data from digital resources (such as the number of people using a website over a
set time period). You may decide that giving a questionnaire to each participant and asking them to report
on their progress is enough. Or you might decide to use other information to help, such as feedback from
other participants in the process e.g. the trainer running the workshop.
In our example of young people engaging in their job searching course, we might give them questionnaires
to find out if they feel more confident about finding work online when they have completed the course.
There are many scales and tools to choose from when measuring outcomes. Please contact the
Development Research team if you are unsure about what would be the best way to measure outcomes.
4. Choosing when to measure it
Now you know what you are measuring and how you are measuring it. The next decision is when you will
measure it. Remember, if you want to know whether a change has happened, you have to measure the
same thing at the start and at the end. In our example project, if we just measured young people’s
confidence in searching for jobs online at the end of the project it might be very high. Great, it seems - but
of course they may have been very confident at the start of the project and we’ve not made any
difference!
So, you may decide to just measure once at the start of the programme and once at the very end. Or if you
are using a different method of evaluation it may be that you are getting feedback throughout.
5. Deciding who will measure it
Lastly you need to decide who will be accountable for ensuring measurement takes place consistently. This
breaks down into 2 areas: who is collecting the outcomes information, and who is analysing it. Of course
this could be the same person if it is a small project; but may be different people if you have a bigger team.
In our example project, the tutor ‘owns’ the job of giving out and collecting the questionnaires from the
participants at the beginning and end of the project. The project manager ‘owns’ the job of analysing the
results and reporting them.
This is a really important step as it makes visible who has responsibility for each part of the evaluation
process.
Outcomes Template
The table below provides a template to help you document your project outcomes. Copy it into a separate file
so you can return the completed form to us as part of your project set-up process. You can list the project
outputs if it helps you, but don’t worry about writing these down in detail as these will already be on
your project plan.
Project Name:
List the project
outputs /
deliverables
(optional)
EXAMPLE:
Running 10
training
workshops with
100 young people
List the
intended
project
outcomes
What do you
intend to
measure?
How do you
intend to
measure these?
Approximately
when will this
be measured?
Who will
measure it?
EXAMPLE:
Young people
who complete
the course are
more confident
about searching
for jobs online
EXAMPLE:
Change in young
people’s
confidence in
searching for jobs
online
EXAMPLE:
Questionnaire
before and after
the programme
asking them to
rate their
confidence on a
EXAMPLE:
At the start and
end of every
training
programme
EXAMPLE:
Tutor hands out
and collects
forms; project
manager collates
them and reports
Page 3 of 5
scale
Step 2 – Identifying Assumptions
We are always very interested to understand what models work to address social challenges. A core
part of this involves understanding the assumptions that underpin a project.
We would like you to identify three to five assumptions that underpin your project. These are core
beliefs you have about why your project will work. In our example project, the programme creators
believe that young people who are out of work will be able to find jobs online. Another assumption
is that young people will be interested in searching for jobs online. A further assumption might be
that young people will have access to the internet after they finish the course. At the end of the
project we will ask you to reflect on whether these assumptions were borne out by your work.
The table below provides a template to help you document your assumptions. Copy it into a separate file so
you can return the completed form to us with your outcomes template as part of your project set-up process.
Project Name:
List of project assumptions
EXAMPLE: Young people being taught how to access job
sites will help them get into employment
Final Thoughts
We appreciate that coming up with an evaluation plan is not always straightforward, and sometimes
it just really helps to talk through your ideas. That’s why we recommend getting in touch with us to
help create your evaluation plan. We are always happy to speak on the phone, online or in person.
Just drop us an email on developmentresearch@nominettrust.org.uk
Page 4 of 5
Evaluation Plan: [project name]
Summary
Please summarise the approach
EXAMPLE:
Project Outcomes
List the project
outputs /
deliverables
(optional)
List the
intended
project
outcomes
What do you
intend to
measure?
How do you
intend to
measure these?
Approximately
when will this
be measured?
Who will
measure it?
EXAMPLE:
Running 10
training
workshops with
100 young people
EXAMPLE:
Young people
who complete
the course are
more confident
about searching
for jobs online
EXAMPLE:
Change in young
people’s
confidence in
searching for jobs
online
EXAMPLE:
Questionnaire
before and after
the programme
asking them to
rate their
confidence on a
scale
EXAMPLE:
At the start and
end of every
training
programme
EXAMPLE:
Tutor hands out
and collects
forms; project
manager collates
them and reports
Project Assumptions / Research Questions
List of project assumptions
EXAMPLE: Young people being taught how to access job
sites will help them get into employment
Page 5 of 5
Download