Research Brief – Inspiring Change Manchester
LOT 2: Service user/ volunteer experiences and changes
Success and Impact of Project specific elements
Background to the research
Inspiring Change Manchester is a new Manchester partnership, led by the housing and
homelessness charity Shelter, with an ambitious vision to help people improve their lives and
sustain these improvements by transforming Manchester’s systems and services. Inspiring
Change Manchester is one of 12 partnerships being funded by the National Lottery through
Fulfilling Lives, the Big Lottery Fund’s £112m initiative to support people with multiple needs.
Project delivery with beneficiaries will commence in July 2014.
The Big Lottery Fund has commissioned CFE Research to lead a national evaluation of the
Fulfilling Lives programme. This will largely focus on outcomes for service users through the
collection of a consistent set of data across all 12 local projects within the programme (data
such as the Homelessness Outcomes Star and NDT assessment).
Inspiring Change Manchester will also undertake a local-level evaluation, dovetailing with the
national evaluation to minimise duplication. The key focus for the local evaluation is to
understand what is working, for whom and under what circumstances. This will help the project
to be delivered more effectively over time in terms of its impact both on individual service users
and on local systems change.
The overarching aim for the local evaluation of Inspiring Change Manchester is to assess the
extent to which the project is achieving its four intended outcomes:




Manchester residents with three or more complex needs will have improved health, wellbeing, housing, employability, reduced re-offending, and these will be sustainable
Manchester residents with multiple and complex needs will have more opportunities for
involvement in the services they need, and influence on decisions that affect them
Services will share more information and better co-ordinate interventions for people with
multiple and complex needs
Commissioning of mainstream services will respond to project learning, funding costeffective evidence-based interventions for people with multiple and complex needs.
In order to provide an independent assessment that will carry weight with commissioners and
other organisations, the majority of local evaluation work will be commissioned from external
organisations with relevant expertise.
1
The evaluation will comprise five key components over 4 lots, as follows.
LOT 1: Systems change monitoring and analysis: to examine the extent to which the
project successfully influences systems change in Manchester – i.e. how wider services are
commissioned and delivered.
LOT 2a): Primary data collection with individual service users and volunteers to
understand experiences and changes: building on CFE’s national set of outcomes data to
understand more about client journeys and experiences and how these differ.
LOT 2b) Exploring the success and impact of unique project-specific elements: to
understand more about local success and to inform national debates on service delivery.
Elements to be researched may include the difference a Psychologically Informed
Environment makes.
LOT 3: Value for money/cost effectiveness exercise: to influence senior commissioners
by providing robust evidence, rooted in local data, showing the fiscal, economic and social
benefits of the project’s approach.
LOT 4: Ad hoc research activities: to provide for one-off research needs.
This research brief refers to Lot 2 (Lot 2a and Lot 2b). Contractors may wish to respond only
to Lot 2a) or to Lot 2b) or to both parts. We anticipate that a combined response to both Part
2a and Part 2b may be more cost-effective and greater value for money due to the potential
to explore the slightly different research avenues (ie Service user experiences and impact of
project specific elements) through the same interviews, for example. We will consider and
review approaches that respond only to one element however and assess the merits that
appointing two different research contractors would bring.
Inspiring change Manchester will plan to deliver its services and operate for 8 years. All local
evaluation partners appointed through this tendering process will be contracted for the first 2
years. All 4 research lots will then be re-tendered.
There is a combined total budget of £36,000 (excluding VAT) for Lot 2 over two years. It is
envisioned this will be equally split with £18,000 for Lot 2a and £18,000 for Lot 2b.
The content of the research under Lots 2a) and 2b) is described separately in more depth
below. You may wish to present your response to this tender as more of a combined
methodology which clearly states how the two disparate research aims will be met.
Content of the research
The contractor/s will initially attend an inception meeting at Shelter offices to finalise an agreed
approach to take forward. The contractor will also be expected to attend at least one learning
and data sharing workshop per year hosted by the project’s “innovation lab”. Finally, the
contractor may be required to attend a planning session with organisations delivering the other
evaluation lots. These requirements and additional project management should be factored in
to the overall total maximum budget per year.
Appointed contractors may be required to work closely with organisations leading the
evaluations of Lots 1, 3 and 4 throughout the evaluation. This is to reduce the burden placed
2
upon service users and other stakeholders involved in the research and to remove any
unnecessary duplication.
LOT 2a) Primary data collection with individual service users and volunteers to
understand experiences and changes
The aim of this element of the local evaluation is to explore:
 service user journeys throughout the Inspiring change project;
 service delivery effectiveness and differences within this;
 service user experiences of delivery organisations and how these experiences differ;
 any changes over the lifetime of the project (ie do experiences improve as service
delivery and systems change occurs?)
These areas relate to both the service users involved in the Inspiring Change Manchester
project and those engaged with partner and wider organisations.
This data collection with service users and volunteers will determine the achievement of the
following outcomes and indicators.
Outcome
Manchester residents with three
or more complex needs have
improved health, wellbeing,
housing, employability, and
reduced re-offending and these
outcomes are sustainable.
Manchester residents with
multiple and complex needs
have more opportunities for
involvement in the services they
need, and influence on decisions
that affect them
Indicators
 Number of individuals supported by the project
who engage in education, training and
employment

Increased number of individuals with complex
needs actively involved with services through
volunteering, employment, consultation and
evaluation activities
The specific questions to be answered through this primary data collection will include, but
are not limited to, those listed below. A full topic guide will be agreed with the appointed
evaluation partner following the tendering process.







Which aspects of the service / system are helpful and what needs to change?
What are the reasons why service users leave the project early?
Is the project reaching the full range of people it should, i.e. is there equality in who is
engaged (BME, gender etc)?
Do certain interventions work better for some groups of beneficiaries than others?
Is need being met as identified in local needs analysis?
What is the personal growth of service users? How much has the project “changed
lives”?
What is the enhanced value of service user involvement?
This element of the local evaluation will need to be carefully co-ordinated with:
 The service user interviews conducted by the national evaluators CFE;
3



The quantitative outcomes data to be collected for CFE;
Routine quantitative monitoring data collected by Shelter’s Evaluation Officer and
Engagement workers;
Research activities lead by the project’s Service User core group
This co-ordination will be in terms of the service users involved in research; evaluation timings/
frequency and crucially, the content of the research. Further details of these complementary
research elements are outlined in the Appendix. Please do read this carefully as it may have
a bearing on contractor’s suggested approaches.
Since in-depth investigation is required as part of this Lot, qualitative methodologies (such as
interviews or focus groups) are anticipated to be proposed by contractors. Shelter will
however consider different and innovative approaches which still comprehensively
meet the research aims. Inspiring Change Manchester also expects that service users may
be involved in this element of the evaluation (ie through peer researching). This is in addition
to the research conducted independently by the service user core group as outlined in the
appendix. The appointed contractor may therefore be involved in up-skilling service users in
research techniques and methodologies (note that the national evaluators, CFE, are also
proposing to do the same).
Role of the contractor
It is likely that the contractor’s activity will therefore comprise the following:







Comprehensive planning
Topic guide development
Research set-up (ie interview/ focus group set-up)
Training service users in interview skills
Conducting research
Analysis
Learning and service delivery review
Shelter welcome suggestions regarding the main output for the work (ie whether it be a report
or a series of presentations/ workshops).
Budget and timings
As stated, project delivery is expected to run for 8 years, with local evaluation partners being
commissioned for the first 2 years at which point contracts may be re-tendered. There is a
total budget of £18k (excluding VAT) available for this research element over two years. It is
anticipated this will be equally divided with £9k in Year 1 and £9k in Year 2.
Component part of evaluation
LOT 2a: Primary data
collection with service users
Year 1 budget available
£9k (excluding VAT)
Year 2 Budget available
£9k (excluding VAT)
A delivery partner/s for Lot 2 will be in place by July when local evaluation activity will
formally begin to coincide with the Inspiring Change Manchester project commencing
delivery directly to beneficiaries. Evaluation contracts will end in June/ July 2016.
4
The research may be spread out over the course of each year and will take place with
service users at different points in their journey to allow comparison. Some research may
also follow up with the same participants. Final stage research should take place before
June 2016 to allow time for learning and review going forward into the project’s 3rd year.
Responding to this brief
The deadline for responding to this brief is 5pm on Friday 27th June. Received briefs will
then be assessed and scored by the project team. Shelter may require potential contractors
to attend an interview.
Please outline as succinctly as possible:








Your understanding of the brief;
Your proposed methodology (Please indicate the number of service users you
envisage conducting research with and how you would include service users/ peer
researchers in your work. If you are also responding to Lot 2b please highlight where
there will be overlap in the proposed methodologies – ie following different lines of
exploration with the same service users);
Your relevant research experience – including qualitative research techniques
Your “subject” experience – ie research with individuals with multiple and complex
needs
Your capacity to deliver this brief over the two year period (including availability to
commence activity in July 2014)
A breakdown of your costs
The project team that would undertake the research
Any perceived risks and how these would be mitigated
Please email your proposals to Sarah Colston, Senior Evaluation and Research Officer, by
5pm on Friday 27th June sarah_colston@shelter.org.uk
If you have any questions Sarah can also be contacted on 0344 515 2129
LOT 2b) Exploring success and the impact of unique project-specific elements:
The aim of this part of the evaluation is to explore the impact of certain specific elements of
Manchester Inspiring Change project delivery in order to understand more about local
success and to inform national debates on service delivery.
Data collected under this lot will support the understanding of how all four project outcomes
are being achieved:


Manchester residents with three or more complex needs will have improved health, wellbeing, housing, employability, reduced re-offending, and these will be sustainable
Manchester residents with multiple and complex needs will have more opportunities for
involvement in the services they need, and influence on decisions that affect them
5


Services will share more information and better co-ordinate interventions for people with
multiple and complex needs
Commissioning of mainstream services will respond to project learning, funding costeffective evidence-based interventions for people with multiple and complex needs.
It has yet to be agreed which project specific elements will be researched. The number of
areas to be researched and in what depth will be dependent on the resources available
through the appointed contractor. The decision on which elements to research will be made
collaboratively between the Inspiring Change Manchester project and the contractor. This
will be based largely around what evidence is most required to influence decision makers/
improve service delivery and what is practically feasible.
Some suggestions for potential, interesting areas to investigate include:




The difference that a Psychologically Informed Environment (PIE) makes to effective
delivery (a PIE focusses on the psychological well-being of the client - good practice
techniques for delivering services in PIE include motivational interviewing). This
would involve primary research with service users, project staff and partner
organisations.
The impact of an agreement Inspiring Change has secured with local Jobcentres
(project service users will be exempt from sanctions). This agreement should
influence both employment opportunities and policy. Research may involve
qualitative and quantitative data collection through Jobcentres; policy-makers and
service users.
Impact of the project’s ‘flexible fund’ on service users (fund will provide e.g. access to
furniture or to pay for courses).This would involve interviews with service users and
project staff.
The difference that elements of the GROW campus make to service users (Service
users will be supported by the project’s ‘Getting Real Opportunities for Work’
Campus to enhance their employability or progress towards other meaningful activity
such as volunteering). This research may involve primary research with service
users, project staff, partner organisations and local businesses.
Role of the contractor
Inspiring Change Manchester are very much looking for a partner to deliver this element of
the evaluation. The contractor will be closely involved in discussions concerning the design
and planning of the research. Shelter are therefore seeking a contractor that is experienced
not just at delivering research but also in planning research in partnership/ collaboration. Time
for these discussions should be factored in to contractor’s proposals.
Although the elements to be researched have yet to be decided, the potential list above
suggests that the research will largely involve primary (mainly qualitative) research with
service users and relevant stakeholders. There may therefore be a degree of overlap with the
research to be conducted under Lot 2a).
The key outputs from the research will also be agreed in conjunction with the contractor. It is
likely to involve a report with clear recommendations. As above, the contractor will be required
to attend at least one learning and data sharing workshop per year hosted by the project’s
“innovation lab”.
To summarise, the contractor will:
6






Be involved in the design and planning of the research
Develop topic guides with support from Shelter
Conduct primary research (which may overlap with Lot 2a) above
Analyse the findings
Produce an output with clear recommendations for decision makers
Inform service delivery review and project learning
Budget and timings
As with Lot 2a) above, the project would like a contractor to be in place in July to coincide
with the commencement of project delivery. The appointed evaluator would then work
closely with Shelter during the two year contract period to both design, plan and deliver the
research activities.
There is an available budget of £18k (excluding VAT) for this piece of research. It is
anticipated that this will largely be delivered equally over both years:
Component part of evaluation
LOT 2b: Impact of unique
project specific elements
Year 1 budget available
£9k (excluding VAT)
Year 2 Budget available
£9k (excluding VAT)
Responding to this brief
Since the specific areas to be researched have yet to be agreed we would ask contractors to
outline indicative costs and day rates for a range of methodological options. For example,
that the contractor could deliver 20 qualitative interviews at a certain specified cost.
Please outline in your proposal:






Your understanding of the brief;
Your indicative costs and day rates, covering:
o Planning/ design
o Research delivery (outlining a range of options)
o Learning/ recommendations
Your relevant research experience – including research techniques and collaborative/
partnership research
Your capacity to deliver this brief over the two year period (commencing July 2014)
The project team that would undertake the research
Any perceived risks and how these would be mitigated
Please email your proposal to Sarah Colston, Senior Evaluation and Research Officer,
sarah_colston@shelter.org.uk by 5pm on Friday 27th June.
If you have any questions Sarah can be contacted on 0344 515 2129
7
APPENDIX:
Lot 2 of the local evaluation for Inspiring Change Manchester will need to be planned and
delivered with due consideration to and co-ordination with the following other research
activities:




The service user interviews conducted by the national evaluators CFE;
The quantitative outcomes data to be collected for CFE;
Routine quantitative monitoring data collected by Shelter’s Evaluation Officer and
Engagement workers;
Research activities lead by the project’s Service User core group
More detail is provided about these four elements below:
The service user interviews conducted by the national evaluators CFE
CFE have been appointed by BIG Lottery to conduct an evaluation of all 12 projects receiving
funding under the Fulfilling Lives Programme, of which Inspiring Change Manchester is one.
Under this national evaluation, Qa Research will be conducting interviews with service users
involved in all 12 projects in order to understand the changing impact of the different projects;
to explore what is working and to understand the needs and potential outcomes for individuals
with multiple and complex needs. Qa will be agreeing an interview schedule with each project.
This will likely involve follow-up interview with some service users. Qa will also seek to upskill/ train service users involved in the project in research techniques.
The quantitative outcomes data to be collected for CFE
CFE research will require the collection of a range of quantitative outcomes monitoring data
over the lifetime of the project at various intervals. Shelter will share the comprehensive
recording matrix/ spreadsheet with the appointed contractor. The data to be collected includes:





Demographic information
Record of previous and current interventions
Homelessness outcomes star
NDT assessment
Warwick-Edinburgh well-being scale
Routine quantitative monitoring data collected by Shelter’s Evaluation Officer and
Engagement workers;
The types of data to be collected are listed below. This list will be refined in agreement with
the Service User Core Group. The data will be collected at different points dependent on the
aims of the collection and the sort of information at hand. Exact frequencies of collection will
be established once the project has in place its data capture systems, and will reflect the
timings of CFE’s quantitative requirements. Certain data may be recorded by engagement
workers directly whilst the collection of other data (such as numbers in employment/
8
volunteering) may simply be co-ordinated by engagement workers/ evaluation consultants but
obtained via other agencies (to ensure the data captures the whole picture of volunteering/
employment and not just that of project clients)
 Frequency/ number of contacts with service users (recorded per appointment)
 Trends in when service users engage and disengage (recorded at engagement and
disengagement)
 Manchester-specific service user outcomes (in line with CFE’s outcomes collection, to be
collected at case opening, quarterly intervals, case closure, and post-closure follow-up)
 Record of interventions accessed by service users, both within the project and from wider
services (recorded on a case-by-case basis)
 Record of service user participation at all levels within the project and in wider services
(case-by-case basis)
 Numbers who are currently in employment as volunteers in partner organisations
 Increase in service users with multiple and complex needs volunteering in wider activities
of partner organisations
 Increase in service users with multiple and complex needs employed in wider services of
partner organisations
 GROW data - Individuals involved in volunteering; consultation; evaluation etc. Target: 20
as paid GROW Trainees; 40 as volunteer Peer Mentors; 48 as Service User Core Group
members (recorded when occurs).
Research activities lead by the project’s Service User core group
These activities will aim to present and illustrate the experiences and journeys of service users
and volunteers by using more creative techniques. It is envisioned these may include video
diaries, blogs, “before and after” pictures, stories and use of social media. Certain outputs may
be used for engagement purposes. Service users will receive training and appropriate
equipment to conduct some of the research tasks. For example, the Service User Core Group
has indicated that there would be value in attending training workshops in photography and
creative writing in order to be able to capture the stories and experiences of service users.
Alongside this, members of the Service User Core Group and other volunteers may
themselves be shadowed by a local photographer or storyteller, for example to hear their
experiences of “mystery shopping” after they have emerged from a service provider. It is likely
that the majority of this activity will take place towards the end of Year 1 and in Year 2. The
Service User Core Group will lead further planning of this activity in June/ July 2014.
9