Students’ Green Fund
First Support Day | 06 September 2013
Agenda
11am
Welcome, context and purpose of today
11.15am
Meeting your fellow projects
11.30am
Introduction to M&E and how things are going to work today
11.45am
Domain huddle
12.30pm
Lunch
1.15pm
Audiences huddle
2pm
Project framing huddle
2.30pm
Getting you up to speed – 1) Behaviour change, 2) Research
methods, 3) Carbon calculation
3.30pm
Next steps and questions
4pm
Close
Welcome
Context
Purpose of today
The successful projects
25 successful projects, 27 SUs / colleges
Lancaster, UCLan, Bradford, Sheffield,
Staffordshire, Exeter, Leicester,
Southampton, Northampton, Cumbria,
Gloucestershire, Bedfordshire,
Greenwich, Newcastle, Liverpool,
Leeds, City, Bristol, Birmingham City,
Worcester, Falmouth and Exeter,
Roehampton, Brighton, Wigan and
Leigh College, University of Sheffield,
Sheffield Hallam University and
Sheffield College
Meeting your fellow
projects
Find someone from a different
project, explain your project
to them as an ‘elevator pitch’,
hear their pitch and move on.
Try to fit in 3 different
projects!
Introduction to M&E, and how
things will work today
Monitoring and evaluating the SGF is
important…
• M&E is an essential part of an effective project
- Without it, you won’t know what you’re achieving
- It will help you learn what is/isn’t working (so you
can adapt)
- It will help others learn from what you are doing
- Finally, it will help others judge your effectiveness
(and resource you accordingly)
• In short, M&E is primarily there to enable learning…
and thereby deliver change:
Learning and change are intertwined
Argyris & Schon’s Double Loop
Learning (1978)
Consequences and
other changes
Acting
New mental
model
Discovering
Choosing
Previous
mental model
Collectively we’re aiming for…
Impact
• 4,000 tCO2/year saved
• Increase in LiFE and Green
League scores
• Student adoption of proenvironmental behaviours
continues beyond HE
• Institutional leaders are more
engaged in sustainability,
resulting in a more holistic and
mainstream approach to
sustainability across the
institution
• Student governors, course reps,
and academics become more
engaged in sustainability,
resulting in more courses with
embedded sustainability content
• Institutions become more
receptive and collaborative to
student opinion and demand on
sustainability issues, and act
accordingly
• Institutions integrate
sustainability into their graduate
attributes and core purpose
Awareness and
participation
• Increase in student
participation in proenvironmental
actions
• Increase in student
awareness of
sustainability
initiatives
• Increase of 1015% adoption of
pro-environmental
behaviours
• Graduate attributes
include skills and
knowledge to take
positive action on
sustainability
• Improve graduate
employability
Reach
•
•
•
•
•
125,000 unique
page views
50,000
students
engaged
20,000
followers on
social media
5,000 staff
engaged
100%
engagement of
HE SU’s with
SGF
How things will work today
• We want your help in developing the M&E strategy for the fund
– this will ensure you it is appropriate to your project
activities, and provides you with the information you need to
understand how you project is working
• Hopefully you’ve all completed Task 2 circulated with the
information about the event today.
• This sheet outlined three key areas of work, which we’re
calling huddles. These are:
• Domain
• Audience
• Framing of your project / Approach
Huddles
Domain
•
•
•
•
•
•
•
•
•
Energy
Water
Waste
Food
Transport
Biodiversity
Health
Wellbeing
Equality and Diversity
Audience
•
•
•
•
•
•
•
•
•
Students – the usual suspects (e.g. already
engaged)
Students – not the usual suspects
Students – accommodation type
Students – domicile
Students – location on campus
University staff
Local community
Landlords
Local business
Framing the project or activity / approach
•
•
•
•
•
•
Skills and employability
‘Green’ / pro-environmental
Diversity
Curriculum
Student-led projects
Competition
Example | Student Switch Off
Students –
accommodat
ion type
Energy
Skills
and
employ
ability
Competition
Students –
not the usual
suspects
Skills
and
employ
ability
Waste
Forming the huddles
• The next three sessions of the day you will be working in the
huddles you have identified to identify the areas of
questioning that are most important to your project
• If your project spans several domains, please try to visit
each one
• When you are at the huddle, please use the flip chart pads,
post-its etc. provided to brainstorm the questions or areas of
questioning that will help you track the performance of your
project
• There are example questions on each table to prompt you if
needed
• A member of the E&E team will also be visiting the huddles
to help guide your discussions
Huddle 1: Domain
Huddle 1 | Domain
• What are the key indicators that
will help you track the
effectiveness of your project?
• Are there particular behaviours
or attitudes you need to assess?
• Do you need to know levels of
awareness of these domains?
• What would project success in
this domain look like?
Domain
•
•
•
•
•
•
•
•
•
•
Energy
Water
Waste
Food
Transport
Biodiversity
Health
Wellbeing
Equality and Diversity
Anything we have
missed?
Huddle 2: Audiences
Thinking about audiences
First rule(s) of behaviour change:
“Identify the audience group and the target
behaviour” [AD, GSR Knowledge Review 2008]
“Identify the barriers and benefits to an activity from
the audience’s point of view” [McKenzie Mohr 2000]
“Always segment” [Andreasen 1995]
Key principles for segmentation
Segmentation is…
1. A practical tool
Developed to subdivide large audiences into well-defined
subgroups, in order to target them more effectively
If a behaviour change tool: segment on the behaviour(s)
in question, OR the most proximal determinants of those
behaviours
2. An iterative process
Make this as transparent as possible but keep it flexible
3. As much an art as a science
Focus on practical purposes of the model for diverse
stakeholders: more heads are better than one (steering
group, advisory group etc)
Potential uses of segmentation
models
Segmentation is good
for subdividing large
samples in order to…
• Build insight
• Develop strategy
• Design interventions
• Devise KPIs, and
evaluation frameworks
• Target subgroups more
effectively with mass
communications/interve
ntion
• Develop a common
language for audiences
(collaborate on all the
above)
Segmentation is less
good for…
• Accurately profiling
individuals
• Re-classifying
existing subgroups
– especially where
relevant
‘communities’
already exist which
can be targeted
effectively
(eg. localities; faith
groups; age bands)
Defra Pro-Environmental
Segmentation Model (2008-)
Segment willingness and
ability
Ability to act
3: Concerned
consumers
I think I do more than a lot of
people. Still, going away is
important, I’d find that hard to give
up..well I wouldn’t, so carbon offsetting would make me feel
better.
14%
5: Cautious participants
I do a couple of things to help
the environment. I’d really like
to do more, well as long as I
saw others were.
14%
Low
Willing
to Act
High
4: Sideline supporters
I think climate change is a big
problem for us. I know I don’t
think much about how much water
or electricity I use, and I forget to
turn things off..I’d like to do a bit
more.
14%
6: Stalled starters
I don’t know much about
climate change. I can’t afford
a car so I use public
transport.. I’d like a car
though.
10%
Low potential and
unwilling
High potential and
willing
1: Positive greens
I think it’s important that I do as
much as I can to limit my impact
on the environment.
18%
2: Waste watchers
‘Waste not, want not’ that’s
important, you should live life
thinking about what you are
doing and using.
12%
7: Honestly
disengaged
Maybe there’ll be an
environmental disaster, maybe
not. Makes no difference to me,
I’m just living life the way I want
to.
Low
18%
High
Defra Pro-Environmental
Segmentation Model (2008-)
Student
population (%)
(NUS GLF 2009)
General public
(%)
(Defra 2007)
+ or (%)
S1 Positive Greens
25.9
18
7.9
S2 Waste Watchers
4.5
12
-7.5
S3 Concerned Consumers
19.9
14
5.9
S4 Sideline Supporters
11.8
14
-2.2
S5 Cautious Participants
15.6
14
1.6
S6 Stalled Starters
4.7
10
-5.3
S7 Honestly Disengaged
17.6
18
-0.4
Segment
Defra Pro-Environmental
Segmentation Model (2008-)
17 Golden Questions…
•
•
•
•
•
•
•
•
•
I would only travel by bus if I had
no other choice
For the sake of the environment,
car users should pay higher taxes
People who fly should bear the
cost of the environmental
damage that air travel causes
I don't pay much attention to the
amount of water I use at home
People have a duty to recycle
We are close to the limit of the
number of people the earth can
support
The Earth has very limited room
and resources
If things continue on their current
course, we will soon experience a
major environmental disaster
The so-called 'environmental
crisis' facing humanity has been
greatly exaggerated
•
•
•
•
•
•
•
•
It would embarrass me if my friends
thought my lifestyle was purposefully
environmentally friendly
Being green is an alternative lifestyle
it's not for the majority
I find it hard to change my habits to
be more environmentally-friendly
It's only worth doing
environmentally-friendly things if
they save you money
The effects of climate change are too
far in the future to really worry me
It's not worth me doing things to help
the environment if others don't do
the same
It's not worth Britain trying to combat
climate change, because other
countries will just cancel out what we
do
Which of these best describes how
you feel about your current lifestyle
and the environment?
Huddle 2 | Audience
• What are the key indicators
that will help you track the
effectiveness of your project?
• Are there particular
behaviours or attitudes you
need to assess?
• Do you need to know levels of
awareness, attitudes and
behaviours amongst these
audiences?
• What would project success
each audience look like?
Audience
• Students – the usual suspects
(e.g. already engaged)
• Students – not the usual
suspects
• Students – accommodation
type
• Students – domicile
• Students – location on campus
• University staff
• Local community
• Landlords
• Local business
• Anything we’ve missed
Huddle 3: Framing of your project
Huddle 3 | Framing of your project
• What are the key indicators that
will help you track the
effectiveness of your project?
• Are there particular behaviours
or attitudes you need to assess?
• What would project success in
this framing / approach look like?
Framing the project or
activity / approach
•
•
•
•
•
•
Skills and employability
‘Green’ / pro-environmental
Diversity
Curriculum
Student-led projects
Competition
1) Introduction to behaviour change
What is behaviour change?
Definitions are scarce – but here are two alternatives...
• For policymakers:
An intervention to encourage individuals to change their behaviour in a
way that will help Government achieve its policy goals...incorporating a better
understanding of behaviour (NAO 2011)
•
For practitioners:
A way of working based on the understanding of behaviours and audiences
which results in learning and change (Darnton 2012)
 Implications include that we may not need to target individuals in order to
change behaviour
Spanning 3 Schools of Behavioural
Theory…
Behavioural
Economics
Social
Psychology
Sociology
…with 3 different views of people
Behavioural
Economics
Social
Psychology
Individual as
Rational Man
Individual as
Social Animal
Sociology
Individual as
Actor
Introducing ISM Individual Social Material
Each model/discipline suggests different avenues for intervention,
and when faced with complex problems, we need to draw on as
many as we can:
• AD’s GSR Review 2008: “There is no one winning model”
• BUT, faced with myriad models, practitioners ask ‘Which one
should I use?’
• ISM devised to bridge this divide: one single multi-disciplinary
model
• Predominantly a practical tool (also, theoretical objections)
• ISM argues that for substantive and lasting behaviour change
we must:
i) Work across multiple contexts
ii) Draw on multiple disciplines
iii) Involve multiple stakeholders
iv) Combine into a package of interventions
(or multiple at the same time)
The ISM Model (Darnton & Evans for
TSG 2013)
ISM applied to…recycling
ISM applied to…mobile phone driving
Implications from behavioural theory
for SGF M&E
•
•
Overall, measure success vs ‘reach, outcomes and impacts’
•
If using ISM as overarching evaluation framework, implication is that
multidisciplinary ISM requires multiple research methods, eg.
•
But to understand (and potentially replicate) effectiveness, also measure
change in the target factors and the target behaviour
o Costs & Benefits – experiments (inc. RCTs)
o Values, Beliefs & Attitudes – primary methods (quali; quant)
o Habits – (self) monitoring inc. pagers/diaries, SRHI
o Meanings – discourse analysis
o Infrastructure – (street) audits
o Time & Schedules – time use studies
Outcome: story of success across ISM contexts/labels
(qv. recycling), plus factor-specific insights
k
k
2) Research methods for M&E
Monitoring and evaluation is all
about…
• Understanding your achievements and the changes
occurring as a result of your project activities.
• Therefore we need to know:
• What the current situation is, including what your audiences
are thinking and doing (baseline research)
• What the situation is at the end of the funding period, what
your audiences are thinking and doing then (follow-up
research)
• Carrying out mid-point research provides an update half way
through, allowing you to adapt your work accordingly
Two main research techniques…
Qualitative
•
Quantitative
Concerned with ‘hard’
data, for example ‘what?’
and ‘how much or how
many?’
• Therefore often numerical
(QUANTity)
• Methods include face-toface questionnaires and
online surveys.
•
•
•
Concerned with the ‘how’ and
‘why’
Common methods include indepth interviews or focus
groups.
Particularly useful when
investigating why people
behave in certain ways.
Surveys / Questionnaires
• Generally considered a quantitative research technique
• But, open-ended questions can obtain qualitative data
• Online surveys are increasingly popular, there are
advantages and disadvantages with this method…
Online surveys
Advantages
•
•
•
•
•
•
Can reach large groups of
people in a short amount
of time.
Can gather facts, opinions,
feedback and attitudes
Easily repeatable
Can cover more
stigmatised or sensitive
topics.
Possibility of interviewer
bias is also reduced
They are also fast to
deploy and report upon
Disadvantages
•
•
•
•
•
•
•
Self-selection bias
Possible technical issues
‘Netiquette’
Survey fatigue
May be less suitable for
complex issues
Possibility for respondents
to misinterpret questions
May require follow-up
research to unpack
findings
Minimising disadvantages of online
surveys
• Don’t promote the survey as a
green/environmental/sustainability survey to
encourage those without a particular interest in this
area to take part
• Provide an incentive for completing the survey
• Test your survey before you send it out, making
sure instructions to respondents are clear and easy
to understand
Samples and populations
Total population
•
•
•
•
For example, distributing the
survey through an ‘all-student’
mailing list.
Sending to all students/staff
allows you to gather data on
people who have been taking
part in your projects as well as
those who haven’t.
Keep track of who is answering
the survey so you can carry out
extra promotion with sectors of
the population who might be
underrepresented.
If you’re just surveying your
total project population, consider
finding a control group to enable
you to assess wider changes.
Sampling
• Only surveying a particular
proportion of the population.
• Normally selected randomly.
• Need to remember that
smaller groups are less likely
to be selected.
• Should be representative of
the total population.
• Online tools are available to
help you calculate sample
sizes.
Focus groups
Advantages
•
•
•
•
Used to explore complex
ideas and behaviours
A great tool for collective
idea generation, evaluating
services and exploring
experiences.
Discover how different
groups think and interact
Save time and money
compared to individual
interviews.
Disadvantages
• Can be time consuming and
costly to run and analyse
• Some groups are difficult to
moderate so you may not
get the information you are
looking for
• Some participants may not
feel confident expressing
their opinions in a group
situation (though online
groups can be a solution to
this)
Focus group top tips
• Groups normally include 6-8 participants (thought
its always good to invite a couple more to account
for drop outs)
• Run groups across your project audiences, and
invite a range of people to get a cross-section of
views
• Ask open-ended, probing questions and try to
involve all members of the group (even the quiet
ones)
• The facilitator should remain neutral and simply
guide the discussion rather than controlling it
M&E Guidance | Handbook and ongoing support
• Monitoring and evaluation handbook distributed
next week with further top tips and guidance
• Review your reach, outcome and impact
statements and make sure you’re able to gather
data on all of these
• Get in touch if you have any queries
• Use the ‘how to’ research guides that are
available on NUS Connect
3) Carbon calculation
Carbon target
Its about more than carbon, but…
We need to collectively and quantifiably save
4,000 tCO2/year = 8,000 tCO2
Fyi average carbon for UK citizen c 11 tCO2/yr, so
target like 727 people
Electricity, gas, water, travel, (food)
Only 12 projects have given carbon targets in
bids. Total of just 3,000 tCO2. Need your help!
How to monitor? - utilities
k
k
Defra conversion figures
0.58982kg of CO2e per kWh of grid electricity
0.20435kg of CO2e per kWh of natural gas
0.3441kg of CO2e per m3 of water
This is updated every year so will change over the
course of the SGF
11/22/2013
8/14/2013
5/6/2013
1/26/2013
10/18/2012
7/10/2012
4/1/2012
12/23/2011
9/14/2011
6/6/2011
2/26/2011
11/18/2010
8/10/2010
5/2/2010
1/22/2010
10/14/2009
7/6/2009
k
Av kWh/day Elec
18.0
16.0
14.0
12.0
10.0
8.0
6.0
4.0
2.0
0.0
12/27/2014
8/14/2013
4/1/2012
11/18/2010
7/6/2009
2/22/2008
10/10/2006
k
Av kWh/day Gas
160.0
140.0
120.0
100.0
80.0
60.0
40.0
20.0
0.0
How much carbon has the Agombar
family saved over last two years?
1,100 kWh electricity x 0.58982kg = 0.7 tCO2
1,350 kWh gas x 0.20435kg = 0.3 tCO2
Total = 1 tCO2
How to monitor? - travel
gCO2/km
Agombar family does 14,000 km a year, has new
car that has better fuel efficiency by 10gCO2/km
(1,000g in a kg, 1,000 kg in a tonne)
So should save us c0.2tCO2/yr, 0.4tCO2 over two
years.
Better to displace car and plane travel
How to monitor? - food
How to monitor? - food
Local food: Each additional 1 percent of
your food that comes from the UK saves
1.51kg of CO2e per year.
Organic food: Each additional 1 percent
of your food that is produced organically
saves 2.25kg of CO2e per year
Composting: Each additional 1kg of
compost saves 0.45kg of CO2e per year
Food waste: Every 1kg reduction in food
waste saves 4.5kg of CO2e per year
Eating less meat: Each additional
vegetarian meal saves 1.1kg of CO2e per
year
Growing your own: Each additional 1m2
of growing space saves 0.9kg of CO2e per
year
www.fifediet.co.uk/carbon/fide-diet-foodprint/
Next steps
Finalising your proposal documents
• Re-submitted by 17th September
• Should include feedback provided by the SGF
team
• Must also include a review of your reach,
outcome and impact statements
Reporting | Requirements and
deadlines
• Requirement to submit monthly and quarterly
reports
• Templates will be sent out following today’s
meeting
• First monthly report is due 17 September 2013
• First quarterly report is due 15 November 2013
Questions time
Thank you!
rachel.drayson@nus.org.uk
07876 860 324