CDNOS evaluation framework steps

advertisement
Planning & Evaluation Framework for Community Development
In this paper we explore a story taken from a real Community Development (CD) project
to illustrate how anyone may use the CDNOS when planning and evaluating a project to
demonstrate the distinctive role of CD in ensuring that the project achieved its desired
outcomes. The story starts here:
Background to the project:
The project came about as a response to local residents groups wanting to green the alleyway. The local
Urban Regeneration Company (URC) had contracted Groundwork to facilitate a programme of community
led environmental improvements and a recognition of the significant levels of racial tensions and incidences
attracting national media coverage.
The community was highly diverse yet divided with mixed tenure residents mostly older white working class
alongside, orthodox Jews and residents with Pakistani, Eritrean, South African, Zimbabwe, Polish, Ukraine
and Irish heritage, new refugees and a more transient student, asylum seeker and migrant worker
population. The terraced streets had been alley-gated but they were not well maintained and used for
storing bins, as dogs’ toilet and collecting rubbish.
Step 1: Get other people involved
Evaluation has often been seen as the business of project managers and funders; a
Community Development evaluation process has to look wider and involve all
stakeholders in evaluation if it is to become a powerful process for learning and working
together. It is useful and vital to involve project stakeholders in defining what it is you are
all trying to achieve from the beginning; each stakeholder will bring a different
perspective on the project’s aims and by gaining a sense of ownership they will invest
more closely in the results of the project.
The local residents groups, the council neighbourhood management team and the community committee
were all involved from the onset however it was recognised that the diversity of the population was not
really represented within existing community organisations and therefore Refugee Action were brought in
as a key partner.
Challenge questions:
 How were the different stakeholders defined?
 How was the community defined?
 How was representation ensured?
Notes: Key area 1 might support this element of reflecting on how to implement a project,
as it is core to all Community Development practice.
Step 2: Define what it is you are doing (your aim)
a) Think about what it is you want to achieve and what the identified need is, this will
be your overall aim. It should directly reflect the impact you want the project to
have.
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
1
Challenge questions:
 Who identifies the needs?
 Consider that there might be different aims to different stakeholders
 If the need is not identified by the wider community, what is the commitment
to involve the wider community?
Notes: Key area 2 might be useful when looking at how to understand and engage with
communities
b) Examine your overall aim in light of the CDNOS key values, they are:
Equality and Anti-discrimination
Social Justice
Collective Action
Community Empowerment
Working and Learning Together
Challenge questions:
 Is the need based on the CDNOS key values?
 Does your project ensure that the work will seek to develop strengths in the
five dimensions of the key values?
 Do you need to pay attention to any specific dimension so that you are not
undermining one principle while seeking to fulfil another?
Notes: It’s a complex judgement and may feel like a dilemma, as our work example
demonstrates.
At this stage, you are being urged to consider how the CDNOS key values might guide
the project holistically
The local residents groups, the council neighbourhood management team and the community committee
were all involved from the onset however it was recognised that the diversity of the population was not
really represented within existing community organisations and therefore Refugee Action were brought in
as a key partner
The project aimed to bring people together to make a positive change in their community using the interest
in alleyway greening as a means of challenging discrimination and encouraging positive and sustainable
change.
This reflected the CDNOS key values in that it aimed to:
Tackle discrimination and exclusion (Key value reflected; Equality and Anti-discrimination)
Promote cohesion (Key value reflected; Working and Learning Together)
Create a sustainable environment (Key value reflected; Social Justice)
Support community action (Key value reflected; Collective Action)
Enable communities to work and learn together (Key value reflected; Working and Learning Together)
The same needs were not expressed by everyone but, all needs were addressed through the one project.
The community groups saw a need to improve the alleyways, the URC wanted to engage the community
and Groundwork and Refugee Action had recognised the needs of the less vocal asylum seeker, refugee
and migrant workers to feel safe and welcome in their neighbourhoods.
The specific aims reflect the CDNOS key areas in that it aimed to:
Understand and engage with communities (Key area reflected; 2)
Provide community development support to community groups to build skills and capacity (Key area
reflected; 6)
Support community learning from shared experiences (Key area reflected; 5)
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
2
Step 3: Think about how you are going to do it (objectives)
Once you have agreed your overall aim, it is time to focus on specific changes you hope
to make. Consider the activities (these will be your objectives) you will need to carry out
in order to achieve the aims and ensure that they are relevant to the aims and needs.
They should describe the work you intend to do and be realisable; the SMART
methodology suggests that each action needs to be Specific, Measurable, Achievable,
Resourced and Time-scaled.
Looking at the CDNOS may help prompt ideas of activity you need to do to successfully
achieve real change, and will help you to ensure that you look at each activity within an
over-arching framework so that one activity doesn’t contradict another. In the specific
example the focus is on the actual standards
Challenge questions:
 Who identifies the needs?
 Who identifies the resources within the community and the different stakeholders
 What level of support is required to support actively engagement from all
stakeholders?
Notes: Key area 3 might give an idea in how to approach a process of ensuring that all
stakeholders can move forward collectively.
The objectives were to:
► To develop relationships with key people and organisations to better understand
the needs and conflict within the community (Standard reflected; 6)
►To run a community grants scheme enabling groups to apply for up to £5,000 to
plan and implement their own action for change (Standard reflected; 14)
►To identify skills gaps and run 8 tailored learning workshops to enable people to develop skills and share
experiences within the community (Standard reflected; 18)
►Provide capacity building support to enable people to come together to identify common issues, plan
their action and achieve their common goals (Standard reflected; 9)
►Support the community to address current conflict using it as a catalyst for change through mediation
and facilitation (Standard reflected; 11)
►Support residents to organise at least 1 celebratory event per project (Standard reflected; 10)
►Help strengthen links between the groups and the council Neighbourhood Management Teams to
maximise long term sustainability through joint meetings signposting and information sharing (Standard
reflected; 13)
The example demonstrates that by using a process of the CDNOS, the project was able
to identify a range of activities which cumulatively added up to a consistent CD approach.
Step 4: Plan out the method or process your will use to deliver your
project
This is a way to identify how you will achieve your objectives – so how you will work and
it is likely to reflect the CDNOS sub standards and the CDNOS key values in practice.
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
3
Based on your identified activities in the previous section you should be able to identify
the method and process of how you will deliver you project.
Challenge questions:
 Who is involved in the basic action planning?
 How are different ideas being shared?
 How are resources identified?
Notes: It might be useful to keep in mind that different stakeholders might take different
actions and implement different approaches due to different needs and skills.
The project process was as follows:
1) Initial door to door awareness raising to establish levels of interest and promote the grant scheme
2) Participatory learning and awareness raising sessions along with hands on support to help people:
identify need, develop ideas; engage the wider community and prepare their grant application. A
key session was the myth busting session held by Refugee Action which challenged some of the
participants pre-conceptions and provided a space for honest discussion on their perceptions of
their changing community
3) A panel of partners and community representatives judged the applications and successful
applicants were awarded their grant.
4) Practical support and further participatory workshops to support the groups implement their
projects and manage their grants including support from Refugee Action to help their clients
understand what was happening and to get to know the other residents involved. Residents visited
garden centres and purchased materials and plants and made their own containers, involving
neighbours in all aspects of planning, purchasing, planting and making cups of tea
5) Celebration events brought everyone together to share success and learning
The example demonstrates how the process was planned and how the CDNOS
standards helped guide the thinking.
Step 5: Identify your outputs
The outputs are all the services you deliver (e.g. training sessions) or the physical
products (e.g. trees planted, publicity items etc) - these are often the numbers or
quantitive
Challenge questions:
 Who is identifying the information?
 What are the resources within the project to deliver different aspects?
Notes: The sub standards of the CDNOS can help you to examine how your plans and
methods can be enhanced by empowering approaches. They will result from the
objectives and the process so review what you are doing and how to help identify them.
8 capacity building workshops delivered
19 people engaged in workshops
7 community events
240 people attending community events
6 environmental improvement projects delivered
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
4
Step 6: Identify the outcomes
The outcomes are the specific changes that will occur if you achieve your aims and can
be the less tangible aspects like: gains in confidence, improved relationships etc.
The outcomes may result from the approach you have taken in working with the group
(i.e. the process) for example increased skills, stronger community group etc. and/ or the
end product e.g. the environment is improved.
The outcomes you choose to monitor may need to reflect the funding priorities that are
paying for the project, and you may have additional outcomes that you want to measure
to reflect your organisational aspirations or approach.
Once you have agreed your desired outcomes, it is vital to reality-check them so that you
are confident that they can be achieved within the scope of the project; the following
guidelines will help:
 It is best to limit the number of outcomes in relation to the size of your project.
 Check that the outcomes you are expecting to achieve can be achieved by the
activities you have described in your objectives – if not you will need to revise your
objectives or outcomes.
 Also consider that you may achieve unwanted or unexpected outcomes. Some of
these you may be able to predict but it is worth ensuring that your evidence
collection methods enable unexpected outcomes to be captured.
Challenge questions:
 Who is identifying the desired outcomes?
 How do your desired outcomes fit with the funding?
Notes: It might be useful to start identifying if the different stakeholders already are
collecting data around some of the desired outcomes or are familiar with different
methods.
►Increased community capacity – workers supported residents form a new group and the workshops
helped existing groups develop their skills and knowledge and building on the success of the barbecue day
the streets are now planning a world music and comedy evening on the alley. The groups also benefitted
from education about definitions of asylum, refugee, migrant workers, and illegal immigrants and
understood what each group were entitled to in the UK.
►Increased cross sector collaboration – working relationships have improved with the Local Authority’s
Environmental Services and the Neighbourhood Management Team and local support bodies such as
Refugee Action.
►Increased number of community led improvements and activities - The improved alleyways are
being used as a community resource giving residents a greater sense of community and feeling of safety
and nicer environment to live in. Residents are more aware of environmental sustainability and with lots of
effort and hard work the streets are growing fruit and vegetables alongside shrubs and flowers, they have
water butts, insect hotels and bird boxes and are working together to maintain the alley.
►Reduced isolation - as a result of the increased interaction people feel more confident to come out of
their houses and attend events in the alleys.
The CDNOS key areas guided our thinking
Step 7: Identify the outcome indicators and evidence you will need to
collect
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
5
Outcome indicators show that the outcome has been achieved; they can be qualitative
(assessing people’s perceptions or feelings) or quantitative (things you can count). You
will probably have more than one indicator for each outcome so just use the most useful.
Once you have chosen your indicators you will need to work out what evidence you need
to collect. This may be done through existing project monitoring processes or you may
need to set up additional data collection systems. Plan out when you need to collect the
evidence and who from: you may collect data from everyone or just a random sample.
Consider the appropriateness of the data collection method for the project participants
and the funder (is it robust?)
Your outcome indicators may reflect how the values of CD were put into practice.
Challenge questions:
 Who is gathering the information?
 Who “owns” the information?
 Have you collectively developed and implemented basic tools or methods for
collecting different types of information throughout the project?
Notes: It might be that different stakeholders have great success with implementing
different types of tools, methods for gathering information, so it might be that they will be
able to reuse and share these with the rest of the stakeholders within the project. For
more information on ideas of what tools to use, see: Paul Hamlyn Evaluation Resource
Pack www.phf.org.uk/downloaddoc.asp?id=243)
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
6
These are examples of outcomes that would result from taking a CD approach. Some of the indicators
might not reflect the CDNOS as they might fulfil the quantitative aspects of gathering information, so it is
important to ensure that the indicators also reflect aspects of the CDNOS, as to ensure that the project is
embedding CD. We have demonstrated this below:
Outcome
Indicator (quantitative or
qualitative) not necessarily
CD specific
Indicator (quantitative or
qualitative) embedding CD
Increased no. of people
from different backgrounds
working together
(Key area 3)
No. of people actively
engaging broken down by
ethnicity
No. of people feeling that they
are more able to work with
people from different
backgrounds
Increase level of collective action
(Key area 2)
No. of community initiatives
delivered through collective
action
Participants are actively involved
in identifying community needs
No. of people actively involved in
planning community initiatives
based on self identified needs
No. of people who feel they can
effect change by working
collectively
Increase in skills and knowledge
(Key area 5)
No. of training workshops
Participants reporting increased
skills and knowledge as a result
of the process
Increased collaborative working
(Key area 4)
No. of new formed
partnerships between
community groups and public
sector
No. of new or positive linkages
formed between community
groups and with public sector
No. of community activists who
feel more confident to work in
partnership, as a result of the
project
Community groups have
improved access to funding (Key
area 6)
No. of funding opportunities
accessed/ amount secured
Groups feel more confident to
apply independently for funding
based on self identified needs
Step 8: Identifying and measuring the overall impact
The impact is the effect the project has on a broader level over the longer term. The
impact is most likely to be achieved by the contribution of several initiatives rather than
your project alone. It is often beyond the scope of any single project to claim that it has
made the pivotal contribution to such a broad outcome, but it is possible to show that
your project has contributed towards such an outcome e.g. from the story by using the
CDNOS values as a guide, we were able to evidence how an inclusive project
contributed to community cohesion, which is a key ingredient in a Healthier Society
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
7
The project contributed to:
Increased cohesion and reduced discrimination (Key value reflected; Equality and Anti-discrimination)
A more sustainable environment (increased social justice) (Key value reflected; Social Justice)
Increased collective action (Key value reflected; Collective Action)
A more empowered community (Key value reflected; Community Empowerment)
The project cannot claim to have achieved these on its own, however it has contributed to them
Challenge questions:
 Did your project achieve what it set out to do?
 What would you do differently next time?
Notes: You can now demonstrate how your project outcomes link to these broader
impact themes – the ABCD (Achieving Better Community Development) pyramid is a
useful resource in mapping those links (Source:
http://www.proveandimprove.org/new/tools/ABCD.php)
Steps and project element broken down:
Impact
Overall
Aim
CDNOS KEY VALUES
CDNOS KEY VALUES
Outcomes
Specific Aim
CDNOS KEY AREAS
CDNOS KEY
AREAS
Indicators
Objectives
CDNOS STANDARDS
Method/process
CDNOS KEY
VALUES in
practice
Outputs
CDNOS SUB -STANDARDS
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
8
Project example
The project is focuses on using environmental action as a mechanism for bringing
disparate communities together, talking about discrimination and building cohesion
Overall aim
► Tackle discrimination and exclusion
(Key value reflected; Equality and Anti-discrimination)
► Promote cohesion
(Key value reflected; Working and Learning Together)
► Create a sustainable environment
(Key value reflected; Social Justice)
► Support community action
(Key value reflected; Collective Action)
► Enable communities to work and learn together
(Key value reflected; Working and Learning Together)
CDNOS
Key
values
Specific aim
► Take a community development approach to group work and understand and
engage with communities (Key Area reflected; 2)
► Provide community development support to community groups to build
skills and capacity (Key Area reflected; 6)
CDNOS
► Support community learning from shared experiences
key areas
(Key Area reflected; 5)
Objectives
► To develop relationships with key people and organisations to
better understand the needs and conflict within the community
(Standard reflected; 6)
► To run a community grants scheme enabling groups to apply for
up to £5,000 to plan and implement their own action for change
(Standard reflected; 14)
CDNOS
► To identify skills gaps and run 8 tailored learning workshops to enable
people to develop skills and share experiences within the community
standards
(Standard reflected; 18)
► Provide capacity building support to enable people to come together to
identify common issues, plan their action and achieve their common goals
(Standard reflected; 9)
► Support the community to address current conflict using it as a catalyst for
change through mediation and facilitation (Standard reflected; 11)
► Support residents to organise at least 1 celebratory event per project
(Standard reflected; 10)
► Help strengthen links between the groups and the council Neighbourhood
Management Teams to maximise long term sustainability through joint
meetings signposting and information sharing (Standard reflected; 13)
Process / Outputs
Process
Actively seek contacts with
refugee and Asylum seekers
(Sub standard reflected; 6.5)
Collect and record the
Outputs
No. of BME participants
No. of consultation events
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
9
perceptions of individuals (Sub
standard reflected; 6.10)
Identify and make contact with
people who are concerned about
their community (Sub standard
reflected; 9.2)
Support community to identify
the causes and effects of conflict
and (Sub standard reflected;
11.1)
Facilitate tailored learning
activities (Sub standard
reflected; 18.3)
Use a range of participative
methods in training delivery that
take into account different
learning styles (Sub standard
reflected; 18.7)
Depth of information from
diverse sources % who feel
that...
No. of people contacted to
engage in process
% of respondents who feel
they have reached a shared
understanding?
No. of facilitated workshops
% of people who feel satisfied
that...
CDNOS sub
standards
No. of training events
Range of methods used or
adapted?
No. of people participating in
training workshops
Outcomes / indicators
These are examples of outcomes that would result from taking a CD
approach. Some of the indicators might not reflect the CDNOS as they
might fulfil the quantitative aspects of gathering information, so it is
important to ensure that the indicators also reflects aspects of the
CDNOS, as to ensure that the project is embedding CD. We have
demonstrated this below:
Outcome
Increased no. of people
from different backgrounds
working together
(Key area 3)
Increase level of collective
action (Key area 2)
Indicator (quantitative or
qualitative) not necessarily CD
specific
No. of people actively engaging
broken down by ethnicity
No. of community initiatives
delivered through collective action
CDNOS key
areas
Indicator (quantitative
or qualitative)
embedding CD
No. of people feeling
that they are more able
to work with people from
different backgrounds
No. of people actively
involved in identifying
community needs
No. of people actively
involved in planning
community initiatives
based on self identified
needs
No. of people who feel
they can effect change
by working collectively
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
10
Increase in skills and
knowledge
(Key area 5)
No. of training workshops
No. of people reporting
increased skills and
knowledge as a result of
the process
Increased collaborative
working
(Key area 4)
No. of newly formed partnerships
between community groups and
public sector
No. of new or positive
linkages formed
between community
groups and with public
sector
Community groups have
improved access to funding
(Key area 6)
No. of funding opportunities
accessed/ amount secured
Impact
►Increased cohesion and reduced discrimination
(Key value reflected; Equality and Anti-discrimination)
►A more sustainable environment (increased social justice)
(Key value reflected; Social Justice)
►Increased collective action
(Key value reflected; Collective Action)
►A more empowered community
(Key value reflected; Community Empowerment)
DRAFT: Planning & Evaluation Framework for Community Development – January 2012
© CDNOS planning & evaluation working group
No. of community
activists who feel more
confident to work in
partnership, as a result
of the project
No. of groups who feel
more confident to
apply independently
for funding based on
self identified needs
CDNOS
Key
values
11
Download