Community Care Outcomes Framework
Review of the Framework 2006 - 2011
April 2012
Contents
Page
3
Executive Summary
Background
5
Methodology
6
The respondents
7
Summary of responses
8
Report of workshop to consider responses
10
Discussion / Recent developments
12
 Personal outcomes and integration
 A continued focus on personal outcomes (including the
role of staff)
 Integration of performance frameworks
 Linkage between national policy and local planning and
implementation – the role of Contribution Analysis
Recommendations
15
Response of Community Care Benchmarking Network
16
Conclusion
18
Appendix 1 – Community Care Outcomes Framework
December 2008
Appendix 2 – local use of Framework at May 2011
Appendix 3 - methodology November 2010
Appendix 4a - the Questionnaire
Appendix 4b – questionnaire respondents
Appendix 5 - The questionnaire responses: detailed analysis
Appendix 6 – Logic modelling
19
2
20
22
25
35
38
50
Review of the Community Care Outcomes Framework
Executive Summary
In December 2008, following 2 years of gestation, the Scottish Government
issued a set of national outcomes for Community Care – or the Community
Care Outcomes Framework (CCOF). After three years of local work
implementing the Framework it was agreed that 2011 was an opportune time
to re-examine it and test whether it was still fit for purpose.
The review was carried out by the Scottish Community Care Benchmarking
Network and the Joint Improvement Team. The group gathered data through
a variety of means:
 a questionnaire, generally administered locally in an interview setting
with key informants in local partnerships or with national stakeholders
(86 returned questionnaires were received– 66 as part of individual
interviews and 20 as part of groups; 139 respondents overall),
 five benchmarking projects carried out by Community Care
Benchmarking Network
 a survey of use of the Framework (28 partnerships responded)
 a policy review.
In terms of use of the framework, the large majority of local partnerships are
using the majority of the indicators in the Community Care Outcomes
Framework to monitor their own local performance.
While very few
partnerships make use of the full framework, most use 10 or more of the 16
measures, and work is being done at local level to ensure reporting against
the majority of measures that are not currently in use.
A number of key themes emerged from the questionnaires, and these were
developed at a workshop in Summer 2011. The resultant recommendations
were put on hold pending the announcement about the Integration of Health
and Care in Scotland (December 2011) and have now been reviewed in light
of that announcement. The Network members wish to support an approach
that moves to one single suite of outcomes and indicators for Integration, for
tracking the progress of the Reshaping Care initiative, and for understanding
our impact on the personal outcomes for adults using services or support, and
their carers.
To that end Community Care Benchmarking Network propose to bring the
Community Care Outcomes Framework in line with the emerging suite of
Outcomes for Integration and to build on the evident enthusiasm across
Scotland for sustaining and building an approach which links outcomefocussed practice at the front line with an understanding of performance in
improving personal outcomes.
The Community Care Outcomes Framework review identifies the focus on
personal outcomes as a major strength of the Framework which should be
further developed, whilst recognising that we have not cracked reporting on
3
outcomes in a comprehensive and consistent way. Rather within Scotland we
are at the stage of grasping what began as a grand ambition, and the
Community Care Benchmarking Network members want to see that through
in the context of the renewed focus on Integration of Health and Care.
The recommendations of the report seek to
 build on the learning from the Community Care Outcomes Framework
to inform the development and use of a single suite of outcomes and
indicators, for both reshaping Care for Older People and the Integration
of Health and Social Care, particularly in relation to personal outcomes,
 to continue to encourage and support local use of the Talking Points:
Personal Outcomes Approach to focus practice on personal outcomes
and to draw information from that process to inform performance
management and joint commissioning
 to support local partnerships to take a whole systems approach in their
use of the single suite of outcomes and indicators, and
 to promote the use of contribution analysis at local level to develop
logic models for use in devising local joint performance management
frameworks
The Community Care Benchmarking Network has accepted the
recommendations and sees an important future role in supporting their
implementation.
4
Background
In December 2008, the Scottish Government issued a set of national
outcomes for Community Care – or the Community Care Outcomes
Framework (CCOF). This was agreed following a considerable period of
consultation with stakeholders across Scotland. There are four high level
community care outcomes which challenge partner agencies to improve
health, well being, social inclusion and independence and responsibility.
Beneath this, sixteen measures are grouped under six themes to support local
partnerships to measure progress against each of these outcomes.
The framework was accompanied by guidance notes containing definitions for
each of the 16 outcomes in the framework. This was to ensure a consistent
approach to data collection and reporting across Scotland, for what was a
new approach to reporting on community care activity and impact.1
The applicability of the framework was tested from 2007 - 2009 by seven
pathfinder/ early implementer partnerships. A key part of this work was to
look at how partnerships would be able to implement the framework, the
enablers (e.g. IT, training) which might be required to support them and how
they might translate the overarching framework into active performance
management frameworks at local level.2 The report on this work highlighted
the fact that there were local differences in approach and that this had an
impact on the way in which the framework was being used across
partnerships. 3
From the launch in 2008 up until 2011, there were a number of developments
in community care and more widely relating to the way in which local
partnerships are expected to measure success and progress. These may be
summarised as follows:




Further development of the outcomes approach
All partnerships due to be compliant with National Minimum Information
Standards for Assessment, Care and Support Planning and Review for
people using services and for carers (2008), allowing them to report on
a new set of personal outcomes indicators.
Reshaping Care for Older People initiative which looked at the
sustainability of the current model for health and social care services
for older people given the demographic shift in the population in favour
of older people.
Introduction of the NHS Scotland Quality Strategy and associated
Quality Measurement Framework, with impacts on the wider
performance management for health and social care.
1
Community Care Outcomes Framework, Final definitions, Sept 2008, reference
EI/SEP2008/Paper 3
2 Early implementers’ network: role of early implementers, August 2007, reference
EA/SEP2007/Paper1
3 Report on the early implementer sites, Jane Mudd Partnership, XXXX
5
By this time, partnerships had had almost three years to work with the
Community Care Outcomes Framework and it was agreed that this was an
opportune time to re-examine that framework.
Vision
The review was informed by the following principles, agreed in advance by the
members of the Community Care Benchmarking Network:







The CCOF should provide a clear outcomes framework which will be
meaningful to the public and practitioners are well as those responsible
for managing performance and developing policies and strategies
The Framework should, first and foremost, be for use locally and not
for the purpose of accountability to central government
It should be a performance framework which would be outcomes
focussed, have improvement indicators and be based on a whole
systems approach
The framework should focus on the public’s experience of using
services and their views on what would make an impact on improving
their quality of life
It should extend the use of the Talking Points: Personal Outcomes
Approach or equivalent
It should develop a more holistic and joined-up practice base which
would focus on the individual by being person centred
It should aim at an integrated health and care quality framework which
is a balance of quantitative and qualitative measures.
Methodology
The review of the CCOF was designed to be as inclusive of stakeholder views
as possible. It was seen as critical that front line managers owned and could
use the performance information the framework brought together to improve
services. It was also agreed that the review should consider the following:



Focus on key aspects of practice, processes, systems and
organisational culture
Whether the Community Care Outcomes Framework should extend to
other client groups, particularly mental health and learning disability
groups
A focus on what works within the current framework rather than
seeking to create something entirely new
The review was overseen by a project team made up of the Executive Group
of the Community Care Benchmarking Network, representatives from the
Benchmarking Network and representatives from Scottish Government and
the Joint Improvement Team.
The Scottish Community Care Benchmarking Network is a member-led
collaboration of the health and social care partnerships in Scotland. Growing
6
out of the early implementers of the Community Care Outcomes Framework,
the network now boasts almost comprehensive membership from health and
social care partnerships across the country and works to its own constitution
which sets out an overarching aim of contributing to “the development of
national and local performance standards and good practice in Community
Care” across Scotland.
The Network has run five Benchmarking Projects, each focussing on a
different theme within the Community Care Outcomes Framework. These
have generated learning and insights about the existing measures and their
efficacy in supporting improvements in performance. A report was made
available in June 2011 and this fed into the present Review of the Community
Care Outcomes Framework.
The Review project team devised a clear project definition document, project
plan and timetable. The first phase of the review began in February 2011
based on a standardised questionnaire covering both current use, and future
development of, the Community Care Outcomes Framework.
The group gathered data through a variety of means.

A questionnaire for stakeholders on their experiences of using the
CCOF and how beneficial this had been in terms of supporting local
practice on outcomes.
The questionnaire consisted of three sections:
1. Questions 2- 9 considered the current use of the framework.
2. Questions 10 – 18 considered the future use of the framework
3. Question 19 invited respondents to provide more detail on any of
the issues already covered, or to discuss issues they felt were
important, but not covered in the main questionnaire.
A copy of the questionnaire is included as appendix 4a to this report.
Members of the benchmarking network were asked to engage with
local stakeholders in completing the questionnaire. A list of possible
stakeholders was distributed with the questionnaire. Members were
given discretion as to how best to use the questionnaire locally – this
could be done on a one-to-one basis or through focus groups.

The questionnaire was complemented by a survey on the local extent
of use of the 16 national measures (the Community Care Outcomes
Framework). This survey was administered electronically via the
Community Care Benchmarking Network.

A review of policy documents relating to community care. The main
aim of the review was to identify key themes contained in current policy
documents and to use this as the basis of a gap analysis in terms of
themes covered by the current CCOF.
7
The Respondents
28 (of 32) partnerships responded to the survey on the local extent of use of
the 16 national measures.
A total of 86 returned questionnaires were received– 66 as part of individual
interviews and 20 as part of groups. This involved 139 respondents. 15.1%
of respondents were described as ‘senior managers’; 14.4% were
performance managers and a further 9.3% were information officers. The
majority of respondents worked either in a local authority, CHP or CHCP, but
there were also a small number from representing particular interest groups
(such as carers) or national policy agendas.
In addition to the 28 partnership responses to the survey on local use of
Community Care Outcomes Framework, questionnaire returns were received
from 24 of the 32 partnerships operating across Scotland. In addition to
these, 15 responses were received from Scottish Government, including a
number of officers who hold positions as policy leads. There were also a
further 3 NHS, 2 scrutiny bodies and 4 third sector organisations (some
organisations submitted multiple questionnaires reflecting different
perspectives).
A full list of questionnaire respondents by position and organisation may be
found at appendix 4b to this report.
Summary of Responses
The large majority of local partnerships are using the majority of the indicators
in the Community Care Outcomes Framework to monitor their own local
performance. While very few partnerships make use of the full framework,
most use 10 or more of the 16 measures. The survey indicates that the work
is being done at local level to ensure reporting against the majority of
measures that are not currently in use, and that there has been significant
progress across Scotland in the use of the framework since 2009 when a
similar survey was carried out across the 32 partnerships.
There was a general endorsement of the outcomes approach and the
Community Care Outcomes Framework which was seen as supporting this.
There was continued support for a balanced approach combining input, output
and outcome measures, but a significant feeling that the outcomes approach
should be strengthened, particularly in favour of personal outcomes.
The crucial role and attitude of staff in ensuring a successful transition to
outcomes approaches which impacted on service user and carer experience
was noted.
The majority of respondents felt it was too early to say how effective the
framework had been in influencing local performance or in improving
outcomes for service users.
8
There was also a strong sense that the framework was not working as a
whole systems approach, and that further work was needed to support
partnerships in making links across individual themes and sets of measures.
National policy leads also indicated an interest in how the framework could be
used to reinforce the links between national policy and local planning and
implementation.
A final comment summarises the overall impression from the questionnaires
which is one of support, but recognising that it is still early days in terms of our
work to assess the impact of our work on securing better outcomes for service
users.
“Where would we have been without CCOF? – It has helped to move thinking
but lots further to go – we’re not yet at the tipping point.“
9
Workshop July 2011
Objectives of the Workshop
1. Agree a final set of outcomes, measures and a framework within which
they fit; a Community Care Outcomes Framework “mark II”
2. Agree a shared understanding of the strategic fit with the National
Performance Framework and other performance agendas
Participants agreed that:



There was value in continuing with the Community Care Outcomes
Framework
It should aim to capture outcomes for all care groups
It should retain a mix of input/process/output and outcome measures
Content
The participants agreed the CCOF should be meaningful, person-centred
toolkit that supports improvement and the objectives of the Framework
should:
1. Focus on the benefits for service users and for carers
2. Provide a consistent framework for local and national reporting
3. Drive improvement across and within community care partnerships
A Vision for the framework was agreed as:
“A framework to understand how well we work with people to achieve their
outcomes and how we could do it better”
PUTTING MEASURES INTO THEMES
Two broad outline versions were created and put into themes as follows;
INPUTS
PROCESSES
OUTPUTS
OUTCOMES
THEMES
A1
A2
A3
Q3
S1
S3
User
Outcomes
10
INPUTS
INPUTS
BC1
BC2
PROCESSES
PROCESSES
OUTPUTS
OUTCOMES
THEMES
A1
A2
A3
C1 / C2
S2
Talking
Points
measures
Carer
Outcomes
OUTPUTS
OUTCOMES
R1
R2
Housing
measure
Anticipatory
care
(replacing
R3)
Telecare
Re-ablement
Mobility
Dying where
you want
Living where
you want
BC3
THEMES
Supporting
independence,
safety, and
care at home
ACTIONS / NEXT STEPS
The following actions were agreed at the end of the workshop:
ACTION: Populate logic model with measures relating to the 3 themes
ACTION: Reword existing measures not currently included in NHS HEAT or
Statutory Performance Indicators
ACTION: Develop definitions for new measures based on existing models in
use in East Renfrewshire and Orkney
ACTION: Articulate links to NHS HEAT Local Delivery Plans and Community
Planning
11
Discussion / Recent developments
Personal outcomes and integration
On 12 December 2011 the Cabinet Secretary for Health Wellbeing and
Communities announced the Government’s plans for integration of Health and
Social Care for adults in Scotland. The approach has started with the key
questions about what matters most to people who use these services - what
are the improvements they want to see and what are the barriers in the
current system that prevent staff from using their skills and resources to best
effect. As a result of this, the Government has agreed with COSLA that new
Health and Social Care Partnerships will be accountable to Ministers, leaders
of local authorities and the public for delivering new nationally agreed
outcomes. These will initially focus on improving older people's care and are
set to include measures such as reducing delayed discharges, reducing
unplanned admissions to hospital and increasing the number of older people
who live in their own home rather than a care home or hospital
In the light of this development we understand that a single suite of health and
care outcomes will be presented as part of the engagement phase of the
Integration work in 2012. Those outcomes will be supported by a set of
indicators, and local health and care partnerships will be held accountable for
their progress against these indicators.
The Community Care Benchmarking Network has considered this
announcement and its implications for the review of the Community Care
Outcomes Framework. The Network members wish to support an approach
that sees one suite of outcomes and indicators for Integration, for tracking the
progress of the Reshaping Care initiative, and for understanding our impact
on the personal outcomes for adults using services or support, and their
carers.
To that end Community Care Benchmarking Network propose to bring the
Community Care Outcomes Framework in line with this and to build on the
evident enthusiasm across Scotland for sustaining and building an approach
which links outcome-focussed practice at the front line with an understanding
of performance in improving personal outcomes.
The Community Care Outcomes Framework (2008) includes 6 measures
which are explicitly based on asking individual people about their own
experience of the impact of services and support. Some local partnerships
have managed to collect that information from individual care and support
plan reviews and to aggregate it to provide useful performance information,
which in turn is used to inform future service commissioning. Community
Care Benchmarking Network wish to continue to support this approach and to
see it built into the outcomes and indicators used to drive the integration
process.
12
A continued focus on personal outcomes (including the role of staff)
The Community Care Outcomes Framework is perhaps unique in Scotland in
focussing largely on the personal experience of people using support or
services, and of carers. The Review has emphasised the perceived value of
that approach amongst the local and national stakeholders, and the
Community Care Benchmarking Network members have confirmed that they
wish to continue to promote this approach, within the context of the single
suite of outcomes for integration.
The Community Care Benchmarking Network has developed a stream of work
built around the Talking Points: Personal Outcomes Approach which seeks to
promote outcome-focussed practice at the front line of health and care. Staff
need to be supported to work in this way, and this fits well with the process
benchmarking approach adopted by the Network.
The Institute for Research and Innovation in Social Services has recently
carried out a review of the Talking Points: Personal Outcomes Approach.
One of the key findings is that more work needs to be done to articulate the
links between person-centred practice and performance management within
health and care partnerships. Community Care Benchmarking Network will
work with Talking Points leads to further develop this linkage.
One of the key challenges to including a focus on improving personal
outcomes in performance measurement at a national level has been the
requirement to meet the quality standards of national statistical releases. In
order to gain support for the inclusion of personal outcomes in the suite of
outcomes and indicators for integration of health and care, it will be necessary
to have robust data at local partnership level which allows comparison and
aggregation across Scotland. In order to achieve this, the work on the suite of
Outcomes and indicators for Integration includes a proposal for a survey of
the experience of people using care services or support. This might be
carried out nationally or locally but would require to contain a core set of
questions that are administered consistently across all partnership areas.
The Community Care Benchmarking Network supports this proposal in
principle and suggests that the Talking Points Outcomes Frameworks be used
to generate the core questions.
Community Care Benchmarking Network also emphasised the importance of
continuing to support local work to gather information from individual
assessments and reviews. These generate a rich source of quantitative and
qualitative data which can be analysed and used locally to inform
performance management and joint commissioning. This cannot be done so
effectively on the basis of surveys, so any nationally agreed care user survey
should augment rather than replace local work to link outcomes-focussed
practice with performance management and joint commissioning.
13
Integration of performance frameworks
Community Care Benchmarking Network has also agreed that it is not
appropriate now to promote a refreshed Community Care Outcomes
Framework as a separate entity. Rather, the experience and ongoing
resource of the Network should be used to promote and support a focus on
personal outcomes within the single suite of outcomes and indicators being
developed for the Integration of Health and Care.
The Community Care Outcomes Framework review identifies the focus on
personal outcomes as a major strength of the Framework which should be
further developed, whilst recognising that we have not cracked reporting on
outcomes in a comprehensive and consistent way. Rather within Scotland we
are at the stage of grasping what began as a grand ambition, and the
Community Care Benchmarking Network members want to see that through
in the context of the renewed focus on Integration of Health and Care.
The second perceived strength of the Community Care Outcomes Framework
is its presentation as an integrated suite of measures, to be seen as a whole
and not isolated for individual attention. The review suggests that there has
been less success in embedding this approach than there has been in
promoting the focus in personal outcomes. This is perhaps not surprising
since the Community Care Benchmarking Network has not adopted this as a
key work strand to date. Nonetheless it remains important and is another
strong message that the Community Care Benchmarking Network wishes to
play into the development of the performance approach associated with the
integration of Health and Care in Scotland.
Linkage between national policy and local planning and implementation
– the role of Logic Modelling
Community Care Benchmarking Network has responded to the findings of the
Review of Community Care Outcomes Framework about understanding the
links between local actions and outcomes and national outcomes by drawing
up initial logic models for the three core outcomes that emerged from the July
2011 workshop;
 Experience of people using service or support
 Experience of carers
 Supporting independence, safety and care at home
The products of this work are at Appendix 6 to this report.
The Network is building its experience of using this technique and believes
that it can be highly effective as a means of building local trust and consensus
around what needs to be done to improve the experience of people using
services and support, and of carers. This can be developed to support a local
joint performance framework.
14
Logic modelling can be seen as a strand of broader benchmarking activity and
the Network is willing to support peer working whereby experienced
partnerships work with others to develop local Contribution Analyses.
Recommendations
1
The Scottish Government should work with all relevant stakeholders
to agree a single suite of outcomes and supporting indicators for the
Integration of Health and Social Care, and Reshaping Care for Older
People, which clearly demonstrates the impact of these initiatives on
personal outcomes for people who use care and support, and their
carers.
2
Scottish Government should not promote the Community Care
Outcomes Framework as a separate entity; rather, the learning from
the Community Care Outcomes Framework should inform the
development and use of the single suite of outcomes and indicators,
particularly in relation to personal outcomes.
3(a)
Local partnerships should be encouraged and supported to use
the Talking Points: Personal Outcomes Approach to focus
practice on personal outcomes and to draw information from
that process to inform performance management and joint
commissioning.
3(b)
At the same time, a national care experience survey should be
developed to generate objective data about personal outcomes
for people using health and care services, to feed the single suite
of outcomes and supporting indicators, and to sit alongside
locally generated information.
4
Local partnerships should be encouraged to develop their
understanding of how the various indicators in the single suite
interact and influence each other. Support should be offered to
partnerships to do this, as appropriate.
5
Local partnerships should be encouraged to use contribution
analysis approaches to generate locally agreed logic models which
explain how personal outcomes and system outcomes will be
improved in the local area.
15
Community Care Benchmarking Network – proposed contribution to implementation of recommendations
Recommendation
Proposed Community Care Benchmarking Network contribution
1. Single suite of outcomes and
indicators for integration of health
and social care
The SCCBN will participate;
a) in the transition to an Outcomes based approach to performance
2. Community Care Outcomes
Framework to be integrated into a
single suite model
Assist in the transition from the use of the CCOF to implementing an agreed single
suite of measures.
3a. Measuring personal outcomes
- Talking Points
Collaborate in:
b) in the development of an integrated suite of health and social care measures
for all Partnership services that are both qualitative and quantitative,
including personal outcomes measures, suitable for benchmarking,
a) an examination of the theoretical and practical issues underpinning the capture,
analysis and use of information for adults and carer outcomes in health and social
care settings.
b) the development of ways to link recording, aggregation of data and feedback to
staff and through the outcome review process
16
Recommendation
Proposed Community Care Benchmarking Network contribution
3b. Measuring personal outcomes
- national survey
SCCBN to regularly support collection, collation and reporting on results.
Use Benchmarking activity and analysis as a step to improving results for people
who use services and their carers through better and more focused reporting of
joint performance
4. Whole system performance
Partner in promoting and supporting improvement
Facilitate the sharing and development of best practice and service excellence in
Community Care through benchmarking, the exchange of information and
identifying ‘what works’
5. Contribution analysis
Continue to promote and support the work of the Community Care Benchmarking
Network subgroup to develop this approach - and implementation as part of
Benchmarking
6.Implementing Recommendations
SCCBN is remitted to act as a resource to support the development and delivery of
the first 5 recommendations
17
Conclusion
The review of the Community Care Outcomes Framework concluded its work
on schedule in September 2011. Findings were documented and plans for
development discussed.
By that time it was clear that an announcement about integration was
imminent and would be of relevance to the recommendations and action plan
arising from the Review.
The Review Project Team has worked with the Community Care
Benchmarking Network membership to re-frame the recommendations of the
Review and to consider the potential contribution of the Network.
The role of Community Care Outcomes Framework and Community Care
Benchmarking Network in developing and promoting a focus on improving
personal outcomes for people using support or care services, and their carers,
has led to a significant pool of expertise and experience across Scotland.
Drawing on that for the next phase of integration remains critical.
The Scottish Community Care Benchmarking Network will do all it can to
continue to promote this focus, in line with the wishes and aspirations of those
who contributed to this review.
18
Appendix 1
Outcomes Framework for Community Care 2009-10
National Outcomes
Improved health
Improved social inclusion
Improved well-being
Improved independence and responsibility
Performance measures and data sources
Themes
User/Carer
Experience
Code
S1
Measure
% of community care service users feeling safe.
Type
Outcome
Data Source / Status
Data drawn from NMIS
E/N
N
S2
Outcome
Data drawn from NMIS
N
Outcome
Data drawn from NMIS
N
Output
HEAT Standard
E
Output
A2
% of users and carers satisfied with their
involvement in the design of care package.
% of users satisfied with opportunities for social
interaction.
No. of patients waiting in short stay settings, or for
more than 6 weeks elsewhere for discharge to
appropriate setting.
No. of people waiting longer than target for
assessment, per 000 population.
No. of people waiting longer than target time for
service, per 000 population.
Output
A3
% of carers who feel supported and capable to
continue in their role as a carer.
% of user assessments completed to national
standard.
% of carers’ assessments completed to
national standard.
% of care plans reviewed within agreed
timescale.
No. of emergency bed days in acute specialties
for people 65+, per 100,000 pop.
No. of people 65+ admitted as an emergency
twice or more to acute specialties, per 100, 000
pop.
Percentage of people 65+ admitted twice or
more as an emergency who have not had an
assessment.
Shift in balance of care from institutional to
‘home based’ care.
% of people 65+ with intensive needs receiving
care at home.
% of people 65+ receiving personal care at
home.
Outcome
Pending Implementation of
Lord Sutherland’s Review of
Free Personal Care
Pending Implementation of
Lord Sutherland’s Review of
Free Personal Care
Data drawn from NMIS
N
S3
A1
Faster
access
Support for
carers
C1
Q1
Quality of
assessment
and care
planning
Q2
Q3
R1
R2
Identifying
those at
risk
Moving
services
closer to
users
patients
R3
BC1
BC2
BC3
Measures:






6 (7) outcomes
7 output
Process
N
Output
Data drawn from NMIS and
local systems
Data drawn from NMIS and
local systems
Data drawn from NMIS
Outcome
HEAT target (T12)
E
Outcome
National Indicator reported in
Scotland Performs
E
Output
Measure administered
through ISD
E
Input
No overarching measure
Input/
Outcome
Output
(proxy)
HEAT target (T8). Measure
administered through ASD
Measure administered
through ASD
Process
2 process
2 (1) input
NMIS is National Minimum standards for assessment, shared care and support plans
and review (July 2008)
HEAT is the NHS Scotland suite of measures and targets on which NHS Boards base
their annual Local Delivery Plans. All references here are to HEAT 2009/10.
ISD is the Information Services Division of NHS Scotland
ASD is the Analytical Services Division of Scottish Government
E – existing - measure in place in 2008 or before
N – new - measure defined December 2008
19
N
N
E&N
E
Appendix 2
Local use of Community Care Outcomes Framework as at May 2011
20
Partnership
S1
S2
S3
A1
A2
A3
C1
Q1
Q2
Q3
R1
R2
R3
BC1
BC2
BC3
Dundee
No
No
No
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Orkney Islands
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Western Isles
Yes
Yes
Yes
Yes
In Progress No
In Progress In Progress Yes
Yes
Yes
Yes
Yes
West Dunbartonshire
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
In Progress Yes
Yes
Yes
Yes
Yes
Yes
Argyll and Bute
yes
yes
yes
Yes
Yes
Yes
yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Perth and Kinross
No
Yes
No
Yes
In Progress In Progress In Progress In Progress In Progress In Progress Yes
Yes
Yes
Yes
Yes
Yes
North Lanarkshire
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Fife Council/NHS Fife
No
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
CEC
In Progress In Progress In Progress Yes
No
No
In Progress
Yes
Yes
Yes
Yes
Yes
Yes
Midlothian
Yes
Yes
Yes
Yes
Yes
In Progress In Progress In progress Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Stirling
In Progress In Progress In Progress Yes
Yes
Yes
In Progress Yes
Yes
In Progress Yes
Yes
Yes
Yes
Yes
Yes
North Ayrshire
yes
Yes
Yes
Yes
In Progress Yes
Yes
Inprogress
Yes
Yes
Yes
Yes
Yes
Yes
East Ayrshire
In Progress In Progress In Progress Yes
Yes
No
In Progress In Progress In Progress In Progress Yes
Yes
Yes
Yes
Yes
Yes
East Dunbartonshire
In Progress In Progress In Progress Yes
Yes
Yes
In Progress No
No
No
Yes
Yes
Yes
Inverclyde
In Progress In Progress No
Yes
Yes
Yes
In Progress Yes
No
No
Yes
Yes
Yes
Yes
Yes
Yes
West Lothian
In Progress In Progress In Progress Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Clackmannanshire
In Progress In Progress In Progress Yes
Yes
Yes
In Progress Yes
Yes
No
Yes
Yes
Yes
In Progress Yes
Yes
Aberdeenshire
In Progress In Progress In Progress Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Aberdeen
In Progress In Progress In Progress Yes
Yes
Yes
Yes
Yes
Moray
Yes
Yes
Yes
Yes
Yes
Yes
City of Glasgow
In Progress No
Renfrewshire
No
South Ayrshire
In Progress In Progress In Progress Yes
Shetland
In Progress In Progress No
Yes
Yes
Yes
In Progress No
Scottish Borders
Yes
yes
yes
no
no
yes
Yes
Yes
East Renfrewshire
yes
Yes
No
yes
yes
Yes
In Progress Yes
No
Yes
In Progress Yes
No
Yes
In Progress No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
In Progress Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
In Progress In Progress In Progress yes
Yes
Yes
y
Yes
Yes
Yes
still to be finalised)
Yes
to be finalised)
guidance
progress
Instill
times
guidance
No (waiting
No (waiting times
Yes
In Progress No
Yes
Yes
In progress Yes
East Lothian
S Lanarkshire
Highland
Falkirk
21
No
No
In progress Yes
No
Appendix 3
Methodology
1. Scope of the Review
1.1 Performance Framework
In August 2010, a Benchmarking Network workshop considered where the review
should focus its energies from the range of outcome-focussed performance measures in
use in Scotland – see diagram below
National purpose
National Outcomes
National Indicators ~ NHS Quality Measures
Menu of Local Indicators ~ HEAT
Local Joint Performance Frameworks
Single Agency indicators
Departmental performance indicators
Local team performance indicators
Individual personal objectives
It was anticipated that The National Performance Framework would be reviewed in
2011 following the May 2011, Parliamentary elections.
The NHS Quality Strategy, announced earlier in 2010 proposed a Quality Measurement
Framework that will encompass three levels of measures, underpinning the three quality
ambitions. (See http://www.scotland.gov.uk/Resource/Doc/311667/0098354.pdf).
Analysis and consultation was taking place with a view to agreeing 12 national Quality
Outcome Measures (Level 1) by October 2010 for implementation alongside the related
set of HEAT targets (Level 2) for 2011/12.
It is clear that with 5 of the 16 measures refer explicitly to people over 65, the current
suite of measures is weighted towards older people. However, since 2006 work with
other areas of policy in Scottish Government and local partnerships has sought to
promote a model whereby the Community Care Outcomes Framework is genuinely an
overarching framework for all of community care, with specific client group measures or
service-based measurement fitting underneath this umbrella. This makes sense in trying
to align reporting within a single performance framework and representatives from the
Community Care Benchmarking Network, Scottish Government and the NHS Quality
Strategy Team had been discussing the potential for alignment of the two frameworks
and all were present at the workshop in August 2010.
22
1.2 Scoping and Planning
Given the investment going back to 2006 by Scottish Government, Health and Care
Partnerships and other stakeholders, the importance, complexity and the potential
longer term impact of the proposed review demanded an approach which was well
structured and resourced. This required skills of leadership and project management as
well as technical knowledge and expertise.
The workshop agreed to follow project management principles, based on Prince 2
methodology. This led to a Governance, Structure and Management framework being
agreed giving us;

A project board

A project team

A project manager

A reference group
This hierarchical structure should give clear lines of accountability with roles and
responsibilities agreed, as well as providing support and guidance for completing
specific tasks and/or objectives, including the need to have wider stakeholder
involvement and/or buy in to the review, e.g., COSLA, ADSW, and NHS Scotland.
The Benchmarking Network and Joint Improvement Team partners agreed an outline
Project Definition document to Review the Community Care Outcomes Framework as
follows:
Scope:
1. To review the Community Care Outcomes Framework (2008) through
evaluating its application and impact in all 32 Community Care
Partnerships.
2. To make recommendations on improving its content in order to facilitate
more effective, and wider application through local partnerships working to
support people to live full and positive lives in their own homes or in a
homely setting.
Objectives:





Identify and involve stakeholders in review process
Review and analyse linkages with all relevant policy initiatives and performance
frameworks in health and care, identifying synergies and gaps
Determine most appropriate model linking Community Care Outcomes
Framework to National Performance Framework and supporting systems
Review name “Community Care Outcomes Framework” and recommend new
name if appropriate
Evaluate work undertaken by local partnerships since April 2007
23



Evaluate benefits derived from use of individual measures and groups of
measures
Determine what measures should be removed, possible improvements to
existing measures and any new measures
Report to Health and Care Delivery Group by September 2011
Project start date: November 2010
Project Sponsor:
Project finish date: September 2011
Partnership Improvement and Outcomes Division
Project Managers: Co-ordinator of Community Care Benchmarking Network and Lead
on Community Care Outcomes
Project Board: Outcomes Approach to Community Care Co-ordinating Group
Project Team: from SCCBN, Scottish Government and ISD
1.3 Project Team
The Project Team met 4 weekly from November 2010 to October 2011 managing the
following project plan.
January 2011
PLAN - Preparation of tools to collect data and other intelligence from stakeholders
February
DO - Consult stakeholders. Conduct interviews and surveys
March
REVIEW (1) - Examine the work undertaken by local Partnerships in applying the
Community Care Outcomes Framework
April to May
REVIEW (2) – Carry out survey of all stakeholders
June
REVISE - Evaluate ‘what works’
- Agree a framework, a final set of outcomes and measures
- A shared understanding of the strategic fit with the National Performance Framework
and other performance agendas
July to August
REPORT - Feedback findings and recommendations to Stakeholders
September 2011
REPORT – Submit report to Health and Care Delivery Group
24
Appendix 4a
Questionnaire
Community Care Outcomes Framework Review
Stakeholder Questionnaire
1. Responder’s organisation :
Position (please mark an ‘x’ in the most appropriate boxes which reflects the
contribution from an individual response or from a group):
NHS
Local Authority
Politician / NHS Board member
Director of Social Work
CHP General Manager
Planning Lead
Performance Manager
Team leader
Carer
National Policy lead
Joint Working Manager
Senior Manager
Information Staff
Frontline practitioner
Service user
Focus Group
Other (please specify)
Section 1: Questions about the current use of the framework
(Please note: A one page summary of the Community Care Outcomes
Framework is appended as an aide memoire)
2. Which statement best describes how the framework has – or measures from the
framework – been used in your area?
a) To carry out a whole-systems analysis of performance of local community
health and care services
b) To investigate parts of the whole system using some of the measures
c) We haven’t used the framework
d) Other (please describe)
25
NB: If possible, can you provide us with copies of the relevant documents which
show how it has been used?
3. Have you used some of the measures or themes more than others? If so:
a) Which ones? (a copy of the framework appears at the end of this
questionnaire)
b) Why?
4. The framework includes hard measures (egg. A1, R1), and also soft measures
(S1-3 and C1) aimed at capturing user/carer experience.
a) Do you have systems in place to collect the measures related to quality of
assessment and planning (measures Q1-3)?
Yes
No
 If yes, what do you use the information for? Is it used on a regular basis?
 If no, what have been the barriers to data collection?
26
b) Do you have systems in place to collect the measures related to user/carer
experience (measures S1-3 and C1)?
Yes
No
 If yes, what do you use the information for? Is it used on a regular basis?
 If no, what have been the barriers to implementation?
5. Has your partnership found an effective way to report on the user/carer
experience measures (S1, S2, S3 and C1)? If so, please describe how you have
done this.
6. Has your partnership developed any measures locally that you have found to be
useful in measuring performance?
If so, please tell us what they are.
27
7. Are there any measures in the existing framework that you think need to be
reworded to make them clearer?
If so, please tell us which ones, and your suggested changes.
8. To what extent has the framework helped you to:
a) Understand the performance of your local community health and care
services and the interplay of the range of factors?
Please explain
b) Influence the performance of your local community health and care services
and the interplay of the range of factors?
Please explain
28
9. Overall, how effective has the framework been in improving personal
outcomes for people using community health and care services or support?
Please give a rating from 0 to 10, where 0 means not effective at all
or Don’t know
0
1
2
3
4
5
6
7
8
9
10
o Why you have given this rating?
o If your view is that the framework has been effective in improving personal
outcomes, please describe some examples of this happening?
29
Section 2. Questions about the future of the framework
10. What should the framework seek to achieve in future?
11. What groups of clients should the framework address?
12. Is the Community Care Outcomes Framework the right title for this purpose?
If no, please offer suggested alternatives.
30
13. How important is it that the framework continues to include a mix of quantitative
(‘hard’) and ‘soft’ measures?
o Please rate on a scale of 0 to 10, where 0 is not at all important and 10 is vital
or Don’t know
0
1
2
3
4
5
6
7
8
9
10
o Why you have given this rating?
14. The current framework includes measures on input, process, output and outcome
measures. Is the balance right?
31
15. How important is it for you that we have a framework of defined measures:
 For use in your locality (Please rate on a scale of 0 to 10, where 0 is not at all
important and 10 is vital)
or Don’t know
0
o
1
2
3
4
5
6
7
8
9
10
Why you have given this rating?
 For use across Scotland, to allow you to compare performance across areas?
(Please rate on a scale of 0 to 10, where 0 is not at all important and 10 is
vital)
or Don’t know
0
o
1
2
3
4
5
6
7
8
9
10
Why you have given this rating?
16. What would be useful to support the partnership in their use of the framework in
future?
 From internal sources
 From the Community Care Benchmarking Network
 From the Scottish Government
 From others?
32
Section 3. Implementation of a personal outcomes approach
17. On a scale from zero to ten, how would you rate progress within your partnership
area in implementing a personal outcomes approach?
 Zero would be “implementation has not begun”
 Ten would mean “fully implemented”
or Don’t know
0
o
1
2
3
4
5
6
7
8
9
10
Why you have given this rating?
18. If you have made progress in implementation, which of the following do you
currently use the personal outcomes approach for?



Assessment
Performance Management
Joint Commissioning
33
19. Please include any other comments relating to your experience of using the
Community Care Outcomes Framework if you feel these have not already been
covered.
Thank You
34
Appendix 4b
Questionnaire Respondents by position and organisation
Total Number of Responses
Organisation
Partnerships
NHS
Scottish Government
Scrutiny Body
3rd Sector
Number of responses
24
3
15
2
4
(Some organisations submitted multiple questionnaires)
Breakdown of respondents by organisation
Organisation
Respondents
Aberdeenshire Council
3 x Strategic Development Officer, Head of Service,
Information Officer
Aberdeenshire CHP
Planning lead
NHS Grampian
Performance manager, project manager older people
services, joint working manager, senior manager and
frontline practitioner
Care Commission
Regulator
Clackmannanshire
Director of Social Work
2 x Senior Manager
Information Staff
Policy & Planning Support
CHP General Manager
Dundee Partnership
CHP General Manager
Head of Community Care
Performance Manager Dundee CHP
Planning and Development Manager
Strategy & Performance Manager
Senior officer (Information - Community Care)
Senior Officer (Outcomes)
East Ayrshire
Director of Social Work, CHP Facilitator; Senior Manager,
Community Care: Planning Lead/Performance Manager
East Dunbartonshire Council
Information staff/ performance manager
East Dunbartonshire CHP
Planning / performance manager
East Renfrewshire CHCP
Performance manager, team leader, commissioning and
development co-ordinator, joint working manager, team
leader.
NHS Lothian
Strategic Programme manager (community planning)
Edinburgh
Senior manager (operations) , R & I manger, joint
programme manager & planning lead
Senior manager( performance and information)
Fife
Team leader, senior manager, information staff
Glasgow City Council Social
Planning Manager - Service-wide
Work Services
Performance Manager - Service-wide
Performance Manager - Adult Services
NHS Greater Glasgow & Clyde, CHP Sector Director
35
Glasgow City CHP
Health Scotland
Highland
Inverclyde CHCP
Midlothian
Moray CHSCP
Reshaping care for older
people
NHS Ayrshire and Arran
NHS Ayrshire and Arran and
North Ayrshire council
North Ayrshire Council
NHS Tayside
NHS Lanarkshire
North Lanarkshire Social work
services
Orkney CHCP
Scottish Excel
Scottish Care
VOCAL
Scottish Government (Joint
Outcomes Team – Integration
and Service Development
Division)
Scottish Government (Policy
for Carers)
Scottish Government
ISD
COSLA
South Ayrshire Council
CHP Planning Manager
Local Government Health Improvement Programme
Manager
Carer, senior manager,
Service Manager Performance and Planning ASM
Research
3 x senior manager , director of social work
Community Care Team (Elgin- . collective response from 6
Community Care Officers including Team Manager), head
of community care, NHS business manager
Lead officer, lead officer for Lanarkshire partnership for
Change Fund programmes
Planning lead/ performance manager
Joint working manager
2 x Performance Manager, team leader, 2 x Local
Authority officers
Senior manager
Senior manager, team leader
Head of social work department, performance manager
joint working manager, senior manager, information staff,
performance manager, planning lead
National policy Lead
Chief executive of Scottish care
National Policy lead and senior manger
National policy lead, team leader
Policy for carers team leader
Information Adviser
Lead for mental health benchmarking
Joint working manager
Improvement program manager
3 x National policy lead
Strategist/Facilitator (policy into practice)
JIT policy implementation
Strategic Lead, Quality Team in Health Directorate
Group
National Health and Care Policy lead
Chairperson
Manager, Strategic Service Planning (LA) CHP Partnership
Facilitator(NHS/LA) Manager, Older People’s Services (LA)
Research Officer (LA)
36
South Lanarkshire Carers’
Network
Stirling Council Social care
Service
SWIA
West Dunbartonshire
West Lothian CHCP
Western Isles Community
Health and Social Care
Partnership
JIT
NHS Performance Forum
South Lanarkshire CHP, NHS
Lanarkshire
South Lanarkshire Council
Renfrewshire Council
Dumfries and Galloway
Dumfries and Galloway social
work services
Chairperson
Performance and Quality Assurance
Business Support
Senior Practitioner, Occupational Therapy
Occupational Therapist
Team Manager, Direct Provisions
Manager, Carer’s Centre
Head of service – resources and performance
national policy lead, senior manager, frontline practitioner,
CHP general manager, planning lead, performance
manager, team leader carer
Information Staff , senior manager, performance manager
Performance manager
Senior Manager (planning lead)
Planning and performance manager (emphasis of job on
performance)
Performance Manager, input from both CHP and SW staff
Performance and improvement lead officer
Operations Manager
37
Appendix 5 – the Questionnaire Responses: detailed analysis
Question 2 asked partners to select a statement which best describes how the
framework, or measures from the framework, had been used in their areas.
There were 70 responses to this question from 25 partnerships. The majority
of respondents stated that their partnerships were using the framework to
investigate parts of the system using some of the measures. However, there
was a significant minority (13%of respondents) who felt that they were using
the framework to carry out a whole systems approach. However, a closer
look at responses shows that there is a difference in opinion even within
partnerships. When grouped, it is clear that no single partnership answered
consistently that the framework was used to carry out a whole systems
analysis. This would suggest that partnerships are using the framework to
investigate parts of the system using some of the measures.
Question 3 asked whether partnerships used some measures more than
others and for any reasons behind this use. There were 70 responses to this,
from 25 partnerships. This information was supplemented by the responses
to the survey on local use.
Overall, the measures most used at local level are:






A1: delayed discharges
BC2: Intensive homecare
R1: emergency bed days
BC3: personal care at home
R2: number of older people admitted twice or more as an emergency
R3: number of older people admitted twice or more as an emergency
who have not had an assessment
All of these measures either are or have been required as part of other
reporting systems such as the NHS HEAT targets (Health improvement,
efficiency, access, treatment) or local authority Statutory Performance
Indicators (SPIs).
Measures least used are:







S1: % community care service users feeling safe
S2: % of users and carers satisfied with their involvement in designing
their care package
S3: % of users satisfied with opportunities for social interaction
C1: support for carers
Q1: % user assessments completed to national standard
Q2: % carers’ assessments completed to national standard
Q3: % care plans reviewed within agreed timescales
These appear to be in progress across most partnerships suggesting that the
majority of partnerships are developing measures which will sit against the
national framework.
38
Overall, this suggests that most partnerships are either reporting or
developing mechanisms to report against the majority of measures in the
Framework. Responses also suggest that while many partnerships
acknowledge that the indicators they use is driven by the information that is
available, they also reported considerable developmental work in relation to
outcome focussed measures such as S1-S3 and C1 (support for carers).
Question 4 focused on the softer measures around the quality of assessment
and care planning and service user/ carer experience. There were 70
responses to this, of which 60 answered either ‘yes’ or ‘no’ (remainder were
‘Not applicable’).
Of these 60 responses, 45% answered ‘yes’ without any qualification; 30%
answered ‘no’ without any qualification; the remaining 25% responded that
they collected some but not others. The least likely to be collected was Q2,
with Q3 being the most likely to be collected. A review of comments
suggested that some partnerships which answered ‘no’ to this question, also
collect some of the data, particularly around Q3.
Respondent comments focussed on the Q1, Q2 and Q3 measures with some
reference to carers’ assessment (proxy for C1) but there were very few
references to the user/ carer experience set of measures. In one case,
reference was made to NMIS compliance, but this was not explored in terms
of use of data locally. Comments highlighting barriers pointed to issues
around definition and data reliability which made collecting and using this data
problematic. There was also evidence of work to develop systems to collect
this type of data in the future. This was also evident in the survey of
measures used locally. It should also be noted, that the survey highlights
considerable work on all user and carer experience measures.
Question 5 continued to focus on S1, S2, S3 and C1, this time asking whether
partnerships had found effective ways of reporting on them. A significant
proportion of respondents reported that they collect user feedback via
surveys, which may either be annual or one off. There is also evidence of
work to embed questions on user experience into assessment and review
processes – this has already been done in some areas. Fewer respondents
mentioned carer experience. Those that mentioned this noted work that was
frequently being done in partnership with voluntary sector organisations
working with carers to gather data on carer experience.
Question 6 asked partnerships to identify any local measures which were
useful in measuring performance. Some partnerships pointed out that they
did not develop indicators per se, but rather used indicators which already
existed in other contexts (e.g. HEAT). A number of partnerships highlighted
the fact that they were developing indicators to support service redesign,
reablement was mentioned in particular.
Almost all partnerships stated that they used a range of indicators and
measures to report local performance. Some partnerships gave examples of
these, while others merely summarised local arrangements such as “a whole
39
range of measures” and “performance framework based on a basket of
measures approach”.
Those indicators which have been identified can be grouped under a number
of headings. The largest single set of these related to assessment, possibly
not surprising given that the national outcomes measures least used related to
quality of assessment and care management. A number of measures
relating to reablement are also used reflecting the development of reablement
approaches since the completion of the CCOF. Other headings included
housing, technology (telehealth/ telecare), hospital discharge, care home, and
acute hospital measures. There were also some relating to the person such
as capacity issues, guardianship and employability. These local suites of
indicators were linked to a range of agendas at local level. These included
Single Outcome Agreements, Joint priorities for community care and HEAT.
There were also some references to local work on health improvement,
children’s services, primary care measures, and older people. The
relationship between these local suites of measures and the CCOF is not
made explicit in answers to this question.
Question 7 asked whether there were any measures in the existing framework
that should be reworded to make them clearer. Not all responses answered
this question. Those that did covered all 16 of the measures in the existing
framework, though there was a clear emphasis on some measures more than
others. The table below shows those measures which respondents felt should
be reworded, and the number of respondents which answered in this way:
Measure
S1
S3
Q2
S2
Q1
BC2
BC1
A2
A1
A3
Q3
R1
R3
% of community care services users feeling safe
% of users satisfied with opportunities for social
interaction
% carers assessments completed to national
standard
% users and carers satisfied with opportunities for
social interaction
% user assessments completed to national
standard
% of people 65+ with intensive needs receiving
care at home
Shift in balance of care from institutional to home
based care
Number of people waiting longer than target for
assessment, per 000 population
Number of patients waiting in short stay settings
or for more than 6 weeks elsewhere for discharge
to appropriate setting
Number of people waiting longer than target time
for service, per 000 population
% of care plans reviewed within agreed timescale
Number of emergency bed days in acute
specialities for people 65+ per 100,000 population
% of people 65+ admitted twice or more as an
Frequency
17
10
9
8
8
7
5
5
4
4
4
4
4
40
C1
R2
BC3
emergency who have not had an assessment
% of carers who feel supported and capable to
continue in their role as carer
Number of people 65+ admitted as an emergency
twice or more to acute specialities, per 100,000
% of people 65+ receiving personal care at home
3
1
1
Some respondents also suggested alternative wording, though the majority
did not. The majority of criticism related to definitions, and the need for
greater clarity around some of these, particularly where measures were either
subjective or broad. Other comments concentrated on the fact that the
measures were frequently proxies for outcomes, and that some measures
needed to be updated to reflect new approaches, such as reablement and
Self Directed Support.
Question 8 in the questionnaire asked partnerships to consider two related
issues:
 The extent to which the framework has helped them to understand the
performance of local health and care services and the interplay of
factors impacting on this
 The extent to which the framework has influenced this performance
71 of the 86 returns answered this question. Of these 71, 37% (23) felt that
the framework was helpful to some extent both in understanding local
performance and in influencing this performance. Comments from
respondents indicate that they have found the emphasis on outcomes has
been beneficial; other comments suggest that the framework has enabled
them to raise the profile of performance management for health and social
care and allowed partnerships to consider the interaction across a range of
areas which impact on the bigger picture.
However, a significant minority of 18% (16) felt that the framework was not
helpful when understanding the performance of local health and care services.
Comments on this highlight weaknesses due to lack of comprehensive
reporting against some measures, and the fact that many measures are single
agency specific and merely brought together in the reports rather than linking
together activity in a meaningful way. It is also noted the even among
respondents who were positive about the framework, they felt that other
factors influenced local performance. The main one of these is the set of
HEAT targets for the NHS. There were some suggestions that local
authorities might be more influenced by the CCOF than NHS partners.
A further 7% (5) respondents felt it was too early to say whether the CCOF
had been helpful or had influenced local performance. Issues raised here
included the need for further development on a number of indicators and the
lack of benchmarking across partnerships.
Question 9 asked respondents to consider how effective the framework had
been in improving personal outcomes for people using community health and
social care services. This was first using a rating from 0 (low effectiveness) to
41
10 (high effectiveness) and then to explain the answer. There were 64
answers to this question.



26 stated that they did not know and did not give a rating
2 responded that it was not applicable
36 responses provided a rating. These ranged from 0 – 10, with an
average of 4.8.
Many of the comments suggested that it was too early to tell whether the
framework had been effective, or that the ongoing development of many of the
indicators meant that the whole system which the framework attempted to
represent was not yet operating. There was also a sense that the framework
concentrated too much on outputs rather than personal outcomes. However,
those who gave high ratings of effectiveness largely felt that the Framework
reiterated the outcomes approach and helped to raise the profile of this way of
working in social and community care. It was suggested that it provided an
effective means for shifting local performance management, though this was
limited by how much practitioners understood the outcomes approach. The
following comments highlight this:
“Really ties into what matters to people. Good balance if the
assessment process is right and people have the correct assessment
skills – this is key to outcomes.”
“I don’t think it has been embedded enough in practice. Talking
Points has had more impact.”
“Framework has helped system focus much more on outcomes for
users/carers – not just inputs/processes and proxies for joint working.
But inevitably not all staff “get it” yet”.
Questions 10 – 16 were concerned with the future of the framework.
Question 10 asked “What should the framework seek to achieve in future?”
Three main issues were identified:



Achieve improvement in personal outcomes (26%)
Improved partnership working (14%)
Effective way of benchmarking (13%)
A small proportion also suggested that it should be used to demonstrate
national policy, become more outcomes focussed and include improved
support for carers.
Some of the comments made by respondents were clear that the current
framework does not focus sufficiently strongly on people’s experiences of
using services and the outcomes that they have achieved as a result. There
was also a strong sense that any future framework should be developed to
ensure consistency in data recording and reporting across agencies which
42
would allow different data sets to be matched, thereby reducing unnecessary
duplicate or repeat reporting of data in different formats.
For example, one respondent commented: “It should seek to achieve an
understanding of a service user’s pathway and show an evidence trail that
demonstrates progress towards person centred outcomes, based on quality.”
This idea of the pathway was also mentioned by a (different) respondent
under question 9. In both cases the comment underpins an idea of the whole
system approach which might be provided by the framework.
Responses to question 10 also show a difference in opinion as to the future
direction of the framework. While the emphasis is on a framework that is
centred on outcomes, and particularly personal outcomes, there is a
significant minority which seems to be suggesting a framework that will allow
consistent reporting across Scotland which will allow for benchmarking and
comparisons. This type of response is more common from respondents who
work in information management or performance.
Question 11 asked which groups of clients the framework should address. 81
of the 86 returns answered this question. The top four groups identified were:




All community care client groups (36%)
Learning disabilities (15%)
Older people (10%)
All adult community care groups (10%)
Comments varied from those who felt that the framework should not change,
to those who felt that it needed to be clearer about whether it was a generic
framework for community care, or go down a more specialist route for one
particular care group. A number of respondents suggested having an
overarching framework with different sections or tabs relating to different client
groups sitting beneath this. Examples of comments include:
“Looks like a framework of measures for the 65+ age group. Should
consider its relevance to reshaping care policy and/or develop a
generic overarching framework with bespoke measures for different
services.”
“The framework appears to be trying to do too much. A decision
needs to be made on the focus of the framework. If the focus is to
remain on older people, then outcomes can be clarified for this client
group. If the framework is to be generic for all community care
service users, then the specific 65+ measures should be removed.”
“No reason to change the way it is currently set out. It implies a bias
towards older people, this reflects the real pressures that health and
care services are under, and also the pressures on carers – many of
whom are older people.”
43
Question 12 then asked about the name of the future framework and whether
the Community Care Outcomes Framework was the right title. Of 86
responses, 43% of these said that this was not the right title; 36% said that it
was the right title and 21% did not answer. A number of alternative
suggestions were made. Many of these suggested alternative names based
on outcomes, but a number reflected the fact that some respondents felt that
the framework should be focussed on indicators and measures rather than
personal outcomes. A sample of suggestions taken from across the range of
suggestions gives a flavour of this variation:






Older people’s outcomes framework
National community care performance information framework
National community care benchmarking framework
Community support outcomes
Joint performance outcomes
Integrated health and social care outcomes framework
Question 13 asked how important it was that the framework should include
both quantitative and qualitative measures. This was based on a rating from 0
(not important) to 10 (very important) and reasons for giving this rating also
requested.
There were 74 responses to this question, and 87% of respondents rated this
at 8 or higher. The most common reason for this response was that a mixture
of measures is required, with 19 responses highlighting the need for ‘balance’.
Most of the comments made a case for including qualitative data. One
comment suggested that qualitative measures give the framework credibility;
another suggested the need to counterbalance the measures so they are ‘not
too HEAT driven.’ Many of the respondents thought that the qualitative
measures were necessary to add meaning, to ‘get beneath’ or ‘test
assumptions behind’ the quantitative measures.
A meaningful framework needs both. We need to understand ‘what’ we
are doing and ‘why’ we are doing it.
Several people highlighted that increased understanding of the outcomes for
people using services was necessary to improve service planning, workforce
planning and to increase productivity. There was a view that listening to
service users and carers could ‘tell you what really makes a difference.’ Eight
respondents felt that there should be an increased focus on personal
outcomes.
However, concerns about qualitative data management were raised by six
respondents. One commentator argued that there would be no point in
including softer measures ‘if we are unable to quantify them’, while another
thought that though the soft measures are challenging, they should remain
because they will encourage partners to be more creative. Two respondents
44
thought that the soft measures are too prescriptive and should be determined
locally. A minority of three respondents thought that the quantitative
measures should be given preference from a benchmarking perspective,
because easier to compare, more robust and less subjective. One thought
that the need for proxies and process measures would continue until
‘outcomes can be successfully measured’, and another that triangulation was
required to make soft data dependable.
Two respondents thought that the qualitative and quantitative data should be
gathered separately rather than forming one framework, although one of the
two thought the data should be interpreted together.
Question 14 explored the balance across measures further asking whether
the balance between input, process, output and outcomes measures in the
current framework is correct.
Of the 86 respondents, only 45 answered this question: 27 replied yes and 12
no. A further six said that they didn’t know. Most did make comments which
covered a range of views.
As with the comments for question 13, 8 respondents said that there should
be an increased focus on personal outcomes. Three argued that the
framework should be completely about outcomes, with no need for processes
and inputs at the national level. More often however, the view was expressed
that the outcomes measures needed to be balanced with other types of
measure. One respondent was concerned that all other measures might be
overlooked ‘in the current ideological discourse about “outcomes”.
Overall, there was a suggestion that the balance should be shifted to an
increased emphasis on outcomes, with fewer input and process measures in
the framework. It was also proposed that greater thought might be given to
input measures, identifying those that have a particular impact on achieving
specified outcomes.
Echoing an issue raised with regard to question 13, two respondents raised
concern about the costs and time involved in analysing qualitative data. A
more common view was that work was still required to make use of the data in
the round and to realise the potential to ‘see the whole picture.’ One
suggested that logic modelling could be applied to the framework to link inputs
and activities to intermediate and long term outcomes. Another suggested
developing a data logic structure to estimate outcomes. One person
commented that in its current form the framework is a long way from its
potential to improve services because of the skill and knowledge required to
drill down in each measure to get to the contributory factors to see what needs
to change.
Three respondents raised questions about the labelling of some of the current
measures, particularly highlighting whether the categorisations of R1 outcome, R2 - outcome, BC1 - input were correct.
45
Question 15 had two distinct parts. The first asked how important it was that
we have a framework of defined measures for use at locality level; part two
asked how important it was to allow partnerships to compare performance
across Scotland. Respondents were asked to rate these on a scale of 0 (low)
to 10 (high), and then give reasons for their rating.
In terms of the importance for localities, 81% of respondents rated this as
being very important (ratings of 8-10). The main reasons why it might be
beneficial for localities are as follows:









Beneficial for trends/ comparisons/ planning and target setting
Useful for internal and external benchmarking
Way of evidencing practice and competence
Useful to evidence effectiveness of policy and practice change
Provides structure clarity and direction re, what we are trying to achieve
Help to identify where performance is strong or needs to improve
Will help demonstrate improvement in whole system outcomes
Supports the development/analysis of other measures locally,
influenced by the CCOF
Needs to provide clear definitions to allow better performance
management and benchmarking
Some of these were also highlighted as being significant in terms of
comparing performance across Scotland. The main themes identified against
this are:








Useful for internal and external benchmarking Useful to evidence
effectiveness of policy and practice change
Help to identify where performance is strong or needs to improve
Supports the development/analysis of other measures locally,
influenced by the CCOF
Needs to provide clear definitions to allow better performance
management and benchmarking
Could allow better dialogue around data capture and consistent
reporting
Fundamental to good practice and sharing for improvement
Identify and tackle variation to improve performance
Needs to be valid benchmarking in terms of comparisons and context
With regard to national comparisons, a number of partnerships raised concern
that this should not take away from the local prerogative to target areas of
local concern, and that comparisons of performance across Scotland should
not be used as a form of scrutiny but rather in the context of sharing learning
and good practice.
Question 16 asked respondents to consider what support they might require in
the future use of the framework. Respondents were asked to choose from
four options:
 From internal sources
 From the Community Care Benchmarking Network
46


From the Scottish Government
From others
Many respondents selected more than one option and all four were identified
as being important. The Benchmarking Network and the Scottish Government
were identified most frequently by respondents (69% each) but internal
supports followed close behind with 56% of respondents selecting this option.
Respondents were also invited to add comments to support their answer. In
terms of the Benchmarking Network, a number of respondents qualified their
answers by stating the type of support that the Network might provide. This
included actual benchmarking, support for consistent approaches across
Scotland, and a forum to share good practice.
Other themes that were raised in comments include:
 Need for improved systems to collect data as per minimum information
standards
 Need to address duplication in reporting – this may mean a single set
of indicators that satisfy a number of different audiences
 Need to support data production with high quality analysis, including
whole system analysis.
 Need for stability in reporting requirements to measure improvement
over time
 Need for commitment from all parties
Question 17 asked respondents to rate progress within partnerships in terms
of implementing a personal outcomes approach from 0 (not begun, to 10 (fully
implemented). They were then asked to justify their rating.
There were 69 answers to this question, 10 of which were ‘don’t know’ and 59
of which provided ratings. 35 responses gave a rating of 4-6, the average
rating being 5.4. It is interesting to note, that where partnerships submitted a
number of individual responses, there was not necessarily consistency in the
rating provided. For example, one Partnership submitted 6 responses, three
from the Council alone, and 3 from the CHP. The responses from the Council
were 4, 4 and 6; the three responses from the CHP were ‘don’t know’, 3 and
6. This variation was evident in a number of other partners, though there was
no set pattern which suggested that Councils rated implementation more or
less highly than NHS partners. Variation in Scottish Government responses
was even more marked. Of nine responses, there were three ratings given: 1,
6 and 8.
Comments to justify the rating highlighted a number of themes. The most
commonly cited was a general difficulty in embedding a cultural change to a
focus on outcomes, or ‘doing with’ rather than ‘doing to’. Successes in
implementing the approach were felt to be prevalent in small pockets, aided
by a well articulated vision and a good understanding of outcomes and desire
to improve service provision at senior management level, No common barriers
to achieving the required culture shift were identified, though a professional
47
opposition to outcomes approaches, resource constraints and the poor
reputation of electronic Single Shared Assessment were all mentioned.
Several responses to justify a positive rating mentioned the link between the
implementation of the outcomes approach and the use of Talking Points, in
particular how well this had been embedded in assessment and review
processes. However, several references to Talking Points also noted that
implementation had been at small scale or pilot level and that more work was
needed to embed it more consistently across partnerships. The role of data
systems to support embedding the outcomes approaches in local information
systems was also a common theme. This was seen to be consistent with a
good understanding of the strategic importance of outcomes at senior
management level.
Question 18 asked respondents to identify which of three options their
partnerships currently used the personal outcomes approach for. These
options were:
 Assessment
 Performance management
 Joint commissioning
There were 59 responses to this question, 8 of which gave a N/A response.
The largest response was assessment, with 37 respondents stating that they
used the outcomes approach for this. Performance management was the
next largest group with 30 respondents identifying this; fourteen respondents
stated that they used the outcomes approach for joint commissioning. A
number of responses noted that they used the outcomes approach for more
than one option. Again there is some evidence of inconsistency in answers
provided by different respondents from particular partnerships. This is true
within single organisations (i.e. different respondents from CHP X) as well as
differences in response from different organisations within the partnership (i.e.
CHP X and Council X).
The final question invited respondents to consider any additional information
which they would like to note or issues relating to their experience of using the
framework that they would like to raise. As expected, this resulted in a wide
range of answers, and it is difficult to identify common issues. However, there
are a number of themes which recur.
Some of these reiterate comments already made in the questionnaire, for
example about the ratio of inputs-outputs and outcomes, and a number of
people raising the issue about consistency and priority for measures in the
CCOF across Scotland. It was also stated that the CCOF needs to avoid
being seen as an optional extra – one respondent mentioned its absence in
discussions on applying the Change Fund and the reshaping care for older
people agenda. The comment below captures some of this concern:
It is the groups’ view that the current Community Care Outcome
Framework is becoming less relevant as other policy directives gain
48
momentum. (Reshaping OP services, Mutual Care, Community
Capacity Building Life stage Planning, Equalities and Diversity agenda)
Overall, the comments are supportive of the current approach, with many
respondents feeling that the potential in the framework has not yet been
realised and that it provides a sound foundation for future development.
However, there were a number of comments that highlight the shortcomings
of the whole system approach which the framework was supposed to
promote. A number of respondents expressed a sense that the framework did
not link up processes across organisations. The following quotes are
representative of this type of comment:
The Framework has taken discrete parcels of info that are available
and presented them. What we need to do is support a person-based
approach locally by joining up health and social care data (e.g. social
care data and emergency admissions and deaths) so that there is a
good joint ‘information reservoir’ as put forward in the Interlink Paper
(February 2011)
The CCOF is a mix of measures applicable to ‘social’ care and ‘health’
but do not provide scope for measuring joint performance. You either
‘fit’ into a category that applies to your organisation or do not.
Therefore, the 16 measures are not easily applicable to all Community
Care parties and ‘slanted’ towards local authority reporting.
The framework is geared to 65+ but should be applied to all ages for
example Disability services and Mental Health Services.
Respondents also suggested potential areas for development. These might
include developmental measures and there should also be links into other
organisations, including the nascent SCSWIS. Again, there was a repeated
call for consistency in language and data collection, and the need to
streamline reporting mechanisms to avoid duplication or repeat reporting.
This was also seen as important given the financial constraints under which
partnerships would be working in the coming years.
Measures need to be expressed in a consistent manner across all
statistical reporting without repetition, overlap or differences in data
dictionaries
A final comment summarises the overall impression from the questionnaires
which is one of support, but recognising that it is still early days in terms of our
work to assess the impact of our work on securing better outcomes for service
users.
Where would we have been without CCOF? – It has helped to move thinking
but lots further to go – we’re not yet at the tipping point.
49
Appendix 6 – Logic modelling
The logic model tool has developed over the last quarter of a century
and has its origins in programme management and evaluation. Logic
modelling is a flexible approach that can be used for many scenarios
– from small-scale projects to national strategies. In using the logic
model approach we start with outcomes and the inputs, activities and
outputs that lead toward these outcomes. Feedback loops are built
into the model by way of evaluative performance measures. In logic
modelling it’s important to be specific about what we mean by
outcomes. Outcomes refer here to end impact, i.e., the changes that
occur or difference made for individuals, families, groups of people or
communities as a result of the implementation of certain plans. The
logic model approach also locates this within a context of
assumptions and external factors. One example is given below.
A sub-group of the SCCBN, involving City of Edinburgh, East
Renfrewshire and South Lanarkshire partnerships, used a logic
model approach in the review of the Community Care Outcomes
Framework. The approach was employed in considering health and
quality of life, independence and safety, and the carers themes. The
sub-group also considered dementia using the same approach.
Logic modelling helped in coming up with a representation of desired
outcomes, the resources and people involved in contributing to these
outcomes. At the same time it supports focused thinking about key
measures of process and outcome. The approach promotes
50
evaluation and benchmarking in examining how well intended
outcomes were delivered, change in behaviour or knowledge, the
extent of participation, the effectiveness of activities, and how these
potentially relate back to how resources are deployed.
Experience of using logic models in this limited way within the
SCCBN suggests wider benefits from the approach. There are
strengths in the approach related to linking issues to interventions
and to outcomes which could be applied more widely in the work of
SCCBN. An example of this would be work to support the
development of local frameworks to report performance on
Reshaping Care for Older People.
51