Predictive Models for Health and
Social Care: A Feasibility Study
Authors: Bardsley M, Billings J, Dixon J,
Georghiou T, Lewis GH, Steventon A (2011)
‘Predicting who will use intensive social care:
case finding tools based on linked health and
social care data’, Age and Ageing 40(2): 265-270
February 2011
© Nuffield Trust
Background
Key points
Evidence that admission to a nursing home can be delayed or
avoided by means of preventative ‘upstream’ interventions
Increasingly, public services will have to become more proactive in
identifying and managing risk in older people, in order to mitigate
them as much as possible
It will be important to identify and support not just those at highest
risk of these costs, but also those currently at lower risk who might
become higher risk in future
Current methods of assessing risk largely rely on face to face
assessments. In health care this approach has been shown to be
less accurate at predicting predicting future events (hospitalisation)
compared to statistical models.
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Predictive modelling
• BMJ in paper* in 2002 showed Kaiser Permanente in California
seemed to provide higher-quality healthcare than the NHS at a
lower cost. Kaiser identify high risk people in their population and
manage them intensively to avoid admissions
• Modelling aims to identify people at risk of high costs in future
• Use pseudonymous, person-level data
• In health sector a number of predictive models are available
e.g. PARR++ and the combined model.
*Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Predictive Models Identify who will be
where on next year’s Kaiser Pyramid
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Patterns in routine data to identify high-risk people next year
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Patterns in routine data to identify high-risk people
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Patterns in routine data to identify high-risk people
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Distribution of Future Utilisation is Exponential
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Can we predict costly social care
events in the same way?
• Do the data exist in local systems?
• Can we extract individual, person-level data?
• Can we link a wide range of health and social care data at the
person level?
• Are the data accurate and complete enough to use?
• Can we build a valid statistical model?
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Feasibility Project
• Funded by Department of Health Care Services Efficiency
Delivery (CSED) programme over 18 months
• Worked with 5 sites to extract data sets
• Extracted person-level health & social care data sets
• Linked data from GP records (2 sites); GP register (all sites);
Hospital (all sites); Social Care (all sites)
• Undertook exploratory analyses and ran a range of models
• Tested the impacts of different data sets on the models
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Using health and social care data to
predict health and social care use
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Using routine data
• Less labour intensive so they can stratify the population
systematically and repeatedly
• Avoid “non-response bias”
• Can identify people with lower, emerging, risk
• Important issues of confidentiality and consent to consider
• Linking data sources at individual level across health and
social care is problematic where there is no NHS number in
social care
• The tools are never 100% accurate
• Data may be missing from routine databases on certain
groups
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Predictive factors – examples
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Information flows
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Data collected
• From four ‘sites’ (~ PCT areas)
• Total seven organisations: 3 PCTs, 1 Care trust, 3 LAs
• Total 1.4M pop (range 100,000-700,000)
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Data linkage - approach
First instance: NHS number (encrypted) from LA
In absence of NHS number:
• Central ‘batch tracing’?
• Shared PCT/LA databases?
Ultimately:
•construction of ‘alternative IDs’
97% of individuals in one site
(population ~400,000) were found to
have unique ‘alternative ID’.
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Data linkage
Groups of people in social care data – how many are we able to
match to GP register list (of ages 75+)?
Varies, but better for those with > service use
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Data linkage
Social & secondary care interface
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Predictive value, sensitivity and specificity
of the model incorporating a £5,000 threshold
From: Predictive
for Health
and Social
Care: A Feasibility Study
Predicting
socialModels
care costs:
feasibility
study
© Nuffield Trust
Information on Social Care Needs
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Simplifying and sorting
Accommodation
Adaptation
Adult Placement
Advice Given
Carer Support
Counselling
DA1
Day Care
Direct Payments
Domiciliary
Employment/Training
Equipment
Fostering
Group Activity
Holiday Payment
Housing Advice Given
ILS
Independent Living
Individual
Informal Projects
Information & Advice
Lifeline 400
Lifeline 400 + Smoke Alarm
Mobility Training (DAY)
Mobility Training (NIGHT)
Nursing
Nursing (Block Contract)
Nursing Block Beds
Old Codes
OT Advice Given
OT Rehab Sessions
Other
Other Carer Support Services
Outreach
Permanent
Placements with Parents
Professional Support
R.O. Allowance
Referral to Carers Contact Line
Referral to Carers Information Service
Referral to Crossroads
Registration
Rehabilitation
Residential
Residential Block Beds
Respite
Respite (Carer)
Secure Accommodation
Sheltered
Social Development
Special Sheltered
Specialist Assessment and/or Treatment
Specialist Communication Equipment
Specific Approval
Support
Supported Living
Talking Book Machine
Time Allocation Vouchers
Training
Transport
Washer
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Individual histories
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Transitions in care (75+ in one site)
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Individual health and social care event timeline
over a three-year period
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Predictive Modelling
Attempting to predict:
• For over 75s
a) Admission to care home (or receipt of high intensity home
care) or £5,000 increase in social care costs in one year
b) Where person had no ‘significant’ costs in prior two years
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Original models predicting change at £5k threshold
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Diagram
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Distillation flask
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Trade off between PPV (blue line) and sensitivity
(red line) according to different risk cut-offs
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Which variables are important in pooled £1k model?
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Models using lower £1k thresholds
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Impact of adding new datasets
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Iteration with Sites - Application
• Several sites said they would like to run the Combined Model and
the social care model “side-by-side”
• Theographs thought to be very useful
• Differences between sites: some sites preferred small numbers of
clients others wanted large numbers for mail-shots
• Regarded concerns over invasion of privacy as “a non-issue if the
wording is right”
• Keen to use predictive models for existing or planned re-ablement
services and multidisciplinary teams
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Personal observations on social care data
We could access user level social care data but...
• Different systems
• Issues of standardisation /coding etc and the absence of standard,
structured coding schemes
• Some very detailed information collected but can be difficult to use
in models
• We found it harder to obtain information on descriptions of needs
• Local concerns about data quality
• Note the history of comparative benchmarking
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
Headlines
• We have created linked health/social care data sets in five sites
(overall populations about 2 million people aged 75+)
• We modelled intensive social care:
• Models had satisfactory PPV but low sensitivity at a risk score
threshold of 50 (although can be traded-off against each other)
• Mainly social care driven
• Though accuracy worse than PARR model, accuracy was
comparable with SPARRA and the combined model
• Discussion with sites – they were less concerned with model
performance than we were
• We tried dozens of ways to improve the basic model (5K)
• We modelled change in social care costs with lower thresholds:
1K models are much better but usefulness may be diminished
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
And the data have so much more to offer.....
• Cost modelling
• Iso-resource classification (individual budgets)
• Trajectories and transitions of care
• Evaluation (re-ablement)
• Information for
- Commissioners and managers
- Professionals
- Service users and carers
From: Predictive Models for Health and Social Care: A Feasibility Study
© Nuffield Trust
www.nuffieldtrust.org.uk
Sign-up for our newsletter
www.nuffieldtrust.org.uk/newsletter
Follow us on Twitter
(http://twitter.com/NuffieldTrust)
February
From: Predictive 2011
Models for Health and Social Care: A Feasibility Study
© Nuffield Trust