Whose outcomes are
they anyway?
Robin Miller
Health Services Management Centre
What are we going to cover?

Background to Sandwell Integrated Support
Service (SISS)
 Background to the Partnership Outcomes
Evaluation Tool (POET)
 Bringing the two together – POET in action
 Reflections on the process and the outcomes
 Next steps for POET
Sandwell Integrated Support Service

Developed as a result of a long-standing wish
to improve the co-ordination of services for
disabled children and their families
 Also hoped to address the difficulties
experienced by young people with a disability
as they left school
 Initially championed by Children’s Trust but
managed within Children’s Services
Sandwell Integrated Support Service

Early Years Support Service
 Child Development Centre
 Children with Disability Social Work Team
 Sensory Impairment Team
 Youth Services
 Learning Disability Nurses and Psychology
 Connexions Personal Advisors
 Adult Social Work Team
Sandwell Integrated Support Service

Single Operational Manager co-funded by
health, social care and education
 Co-located in a refurbished base (with space
for resource room and out-patients)
 Managed ‘disability’ purchasing budget for
children’s health and social care
 Won award (APSE Public-Public Partnership)
and praised in Joint Area Review report
We did have…….
‘Single’ framework for assessment & care coordination with weekly referral meeting
 Integrated structure with operational and
professional leads identified
 Monthly management meetings
 Whole service meetings every 2-3 months
 Accountability to Local Strategic Partnership
through Management Group (which included
family/carer rep)
 Family Fun Days

We also had…….

Different assessment and care co-ordination
process across all the teams
 Some teams working with a small number of
children with complex needs and others
working with a larger number with less
complex needs
 Teams sitting in discrete groups
 A management team with mixed views
 Some personality clashes!
 Tensions in wider partnership
The common sense of partnerships
‘To argue for the importance of partnerships
is like arguing for ‘mother love and apple
pie’. The notion of partnership working
has an inherently positive moral feel about
it and it has become almost heretical to
question its integrity’.
(McLaughlin, 2004: p. 103)
Six reasons why partnerships are created






Necessity – that is partnerships are mandated by law or
regulation.
Asymmetry – one party wishes to exercise control over
another.
Reciprocity – partners seek mutual benefit through
cooperation.
Efficiency – partners may gain more efficiency through
cooperation.
Stability – organisations can encounter less uncertainty
through interaction.
Legitimacy – organisations may obtain or enhance their public
image through cooperation.
(Oliver, 1990)
National policy context (in theory)
Integration
Better Services?
Better Outcomes?
(Do they? How? For whom? In what contexts?)
The negatives of partnerships?
Although collaboration may intuitively seem like a good
thing:



For Alex Scott-Samuel collaboration is often better
described as “putting mutual loathing aside in order to get
your hands on the money” (quoted in Powell and Dowling,
2006, p.308).
For Powell and Dowling (2006, p.305) partnership working
involves “the undefinable in pursuit of the unachievable”!
For Thompson and Perry (1998, p.409), “collaboration is
like cottage cheese. It occasionally smells bad and
separates easily.”
Lack of empirical evidence linking
collaboration and service user outcomes



Despite the interest and efforts which have gone into
collaborative activities internationally, there is little empirical
evidence clearly linking this to improved service user
outcomes.
Cameron & Lart (2003) Systematic review of 491 papers:
Dearth of evidence to support the notion that joint working
between the NHS and social services is ‘effective’ (see also
Dowling et al, 2004; Dickinson, 2008).
Evaluations have tended to focus more on the processes of
collaboration than the outcomes.
So is integration ineffective?
 Schmitt (2001) undertook a
Lack of evidence of
systematic review of 25 years of
effectiveness doesn’t
necessarily mean
collaborative working in the US,
integration doesn’t work, it
and concluded:
means we don’t know
whether they work.
‘What is missing from these studies
is the theoretical rationale for the
 Integration is incredibly
choice of outcomes and
difficult to evaluate.
measurement of the team
characteristics that link the team
 What outcomes should
intervention to improved functional
integration produce?
outcomes’ (ibid: 53).

Different forms of outcomes

Change outcomes improvements made in physical, mental or
emotional functioning. This includes improvements in symptoms of
depression or anxiety that impair relationships and impede social
participation, in physical functioning and in confidence and morale

Maintenance outcomes are those that prevent or delay deterioration in
health, well-being or quality of life. This can include low-level activities
such as living in a clean and tidy environment and having social contact.

Service process outcomes reflect the impact of the way in which
services are delivered. This might include the degree to which service
users are: treated as a human being; feel that their privacy and
confidentiality are respected; or, treated as someone with the right to
services.
Aims of POET project

Main question - do partnerships improve
outcomes for service users?

Design a generic evaluation toolkit that can
test this assumption

Strong theoretical underpinning
POET: Partnership Outcomes
Evaluation Toolkit
1. Online staff survey
- Tests the ‘health’ of
the partnership
- Identifies
developmental and
problem areas
- Highlights areas for
celebration
- Surfaces all the
underlying ‘partnership
theories’.
See Dickinson (2007) for further detail
2. User survey
- Tests whether the
‘correct’ outcomes have
been identified.
- To what extent have
these outcomes been
met?
- Retroductive
techniques identify
generative mechanisms
Why did we want to pilot in SISS?
Summative – enabling us to be clear about what
we were hoping to achieve and how successful
we had been to date
 Formative – helping us to understand and make
sense of our context, to highlight areas for
celebration that could be built upon and identify
areas that required strengthening

Agreement from management group
Workshop for all staff in service
On-line questionnaire
Workshop with staff
Family questionnaire & interviews
Personal Role

Role definition, management & development
 Structure of service & how they fit in within
this structure
 Culture & professional identity
 Communication, learning & innovation
 What is positive and what should be changed
Organisational Role

Aims & objectives of service
 Leadership & accountability
 Ownership & trust
 Push & pull
 What is positive and what should be changed
Outcomes for people using service

Current outcomes and those which should be
achieved in the future
 Service user outcomes
 Barriers to people accessing the service
 Service process outcomes
And a final question on ‘any concerns regarding
management support’?
Family questionnaire and interviews

Evaluate the services received against keyobjectives (e.g. treated with respect, services
are co-ordinated, they are enabled to be ‘in
control’)
 What it is important to you from services (e.g.
not repeating story, a welcoming building,
have a lead professional to co-ordinate)
 What makes it difficult to access support?
 What information would you like?
What did we learn about the service?

On the whole, staff were positive about
working in the integrated service and families
were positive about the support received
 A service ‘culture’ had not been developed
(but early days) and there was evidence of a
lack of ‘trust’ between disciplines
 Outcome targets had not been clearly set and
performance monitored
 Families agreed on the whole with outcomes
identified by staff
What did we learn about integration?

Staff find it hard to identify what outcomes
should be delivered (beyond processes)
 Families want better co-ordination and
integration across all of the services that they
receive
 Evaluation is time consuming but does help
to understand current position and set future
direction
 Integration is often fragile
What did I learn about ‘integrated
leadership’?

Ambiguity provides an opportunity to adapt
the message but also muddies vision
 Improving the lives of the families is what
inspires staff & overcomes barriers
 Transformational leadership can be
strengthened by clearer incentives and
rewards
 Succession planning needs to start the day
that you start
Next steps for POET

Streamlined a number of process based
questions and made it easier to articulate
outcomes
 Based on Q methodology – provide a list of
outcomes that integration should achieve and
ask participants to ‘sort’ these on basis of
which they agree/disagree with
 Process based report on how it feels to work
in integrated teams and overview of local
aims
References (1)





Cameron,A. & Lart,R. (2003) Factors promoting and obstacles hindering
joint working: a systematic review of the research evidence. Journal of
Integrated Care 11, 9-17.
Dickinson H. (2008) Evaluating outcomes in health and social care. Policy
Press, Bristol..
Dickinson, H., Glasby, J., Miller, R. & McCarthy, L. (2009) Whose
outcomes are they anyway? Journal of Integrated Care 17, 37-44.
Dowling,B., Powell,M., & Glendinning,C. (2004) Conceptualising
successful partnerships. Health and Social Care in the Community 12,
309-317.
McLaughlin,H. (2004) Partnerships: panacea or pretence? Journal of
Interprofessional Care 18, 103-113.
References (2)




Oliver,C. (1990) Determinants of inter-organisational relationships:
integration and future direction. Academy of Management Review 15,
241-265.
Powell,M. & Dowling,B. (2006) New Labour's partnerships: comparing
conceptual models with existing forms. Social Policy and Society 5, 305314.
Schmitt,M.H. (2001) Collaboration improves the quality of care:
methodological challenges and evidence from US health care research.
Journal of Interprofessional Care 15, 47-66.
Thomson,A.M. & Perry,J.L. (2006) Collaboration processes: inside the
black box. Public Administration Review 20-32.