Action for Prisoners’ Families
Seminar 25 June 2014
Quality services:
ticking all the boxes
Evaluation and family relationship
measures
Anna Kazimirski
EVALUATION AND FAMILY RELATIONSHIP MEASURES
ANNE KAZIMIRSKI, NPC
v
Action for Prisoners’ Families – 25th June 2014
TRANSFORMING THE UK CHARITY SECTOR
Charity
Increasing the
impact of charities
eg, impact-focused
theories of change
Funder
Increasing the
impact of funders
eg, effective
commissioning
Sector
NPC works at the
nexus between
charities and
funders
Consultancy
Think tank
Strengthening the
partnership
Eg, collaboration
towards shared
goals
4
WHY MEASURING IMPACT IS IMPORTANT
Raises profile
Motivates
staff
The Brandon
Centre provides
Helps secure
Influences the
counselling funding
and psychotherapy to
debate on
young people between the ages of 12 “what works”
and 21
Improved
services
Taken from: Rickey, B, Lumley, T and Ni Ogain, E . (2011) A Journey to Greater
Impact New Philanthropy Capital.
5
COMPONENTS OF NPC’S APPROACH TO
MEASURING SOCIAL IMPACT
Effective measurement
framework developed
Map your
theory of
change
Prioritise
what you
measure
Choose
your level
of evidence
Select your
sources and
tools
Strategic vision
Leadership
Case for impact measurement
6
THEORY OF CHANGE
Links activities intermediate outcomes  final outcomes
clarifies what the activities aim to achieve and how
provides a structure for identifying what can be measured
provides the case for why achieving intermediate outcomes is
important
A conceptual map of how activities lead to outcomes
7
EXAMPLE THEORY OF CHANGE
Offenders (and
families) feel
empowered to take
responsibility and
maintain gains
Strengthened family
ties & relationships
Reintegration into
(non criminal)
social & family
groups
Family therapy
Levels of conflict
within families
reduced
Observable
reduction in problem
behaviours during
process
Contribution to
reduced reoffending
8
OFFENDERS’ FAMILY RELATIONSHIPS
• Commissioned by NOMS to develop toolkit to measure intermediate outcomes
of work to improve family & peer relationships of offenders
• Project going through peer review, to be published later this year
• Parallel project: RAND Europe, with ARCS (UK) and University of Glamorgan,
focusing on arts and mentoring work with offenders
9
10
PROJECT PROCESS
Evidence review – identifying outcomes and toolkits
Consultation with providers & commissioners to select outcomes to focus on
Development and piloting of toolkit
Consultation with providers & commissioners to inform guidance
Analysis and reporting
DEVELOPMENT OF RELATIONSHIPS TOOLKIT
•
Desistance theory:
•
•
•
Link between creation/ investment in family relationships / reduction in peer
pressure with reduced offending
Consultation:
•
Outcomes differed depending on the type of intervention & beneficiary
•
Needed to reach a balance between being prescriptive and allowing for
flexibility
We researched existing scales and evaluations in each outcome area and
prioritised:
• good validity and reliability; recently developed; brevity.
Toolkit: 25 standardised scales,
to be used in paper questionnaires for offenders to complete.
For each outcome area: short general scales & longer specialised scales.
12
TOOLKIT STRUCTURE
Child
relationships
Partner
relationships
+
+
Peer
relationships
Wider family
relationships
+
Reduced
reoffending
+
+
+
RAND
personal
development
outcomes
Parenting
13
OUTCOME AREAS
Partner relationships
Increased satisfaction with partner relationship, reduced conflict and improved commitment
Child relationships
Quality of relationships with children, children’s well-being and behaviour
Wider family relationships
Family functioning, resilience, conflict and communication, satisfaction with relationships in
the family and quality of relationships
Parenting
Improved satisfaction with the parent-child relationship and parenting skills
Peer relationships
Satisfaction with peer relationships and reduction of negative peer relationships
14
CHARACTERISTICS OF TOOLKIT
• The questions can be used as monitoring or as part of an evaluation with a
‘before and after’ (pre/post) research design, with a comparison group where
possible
“Intervention” group
Before
measure
Programme / service
After
measure
“Comparison” group
Before
measure
After
measure
15
RESOURCES
http://www.thinknpc.org/
NPC’s Wellbeing measure: http://www.well-beingmeasure.com
NPC’s papers on
How to approach evaluation: http://www.thinknpc.org/publications/npcs-four-pillar-approach/
Theory of Change: http://www.thinknpc.org/publications/theory-of-change/
How to communicate your results: http://www.thinknpc.org/publications/talking-about-results/
Profile of six charities who radically improved their approach to impact measurement:
http://www.thinknpc.org/publications/a-journey-to-greater-impact/
http://inspiringimpact.org/
Code of Good Impact Practice
Online impact marketplace
anne.kazimirski@thinknpc.org
020 7620 4855
Twitter: @npcthinks, @AnneKazimirski
16
Action for Prisoners’ Families
Seminar 25 June 2014
Quality services:
ticking all the boxes
www.prisonersfamilies.org.uk
@Prisonerfamily