Analysis of support
needs data on CHAIN
December 2008
1
Analysis of support needs data on CHAIN
1
Background
CHAIN provides assessments by outreach workers of clients’ support needs around
substance misuse, physical health, mental health - forms can be updated as
frequently as required. The information is used in reporting from the CHAIN system
and is being used in the National Centre for Social Research/ Broadway CHAIN
research project.
Given the importance of this information, in November – December 2008 the CHAIN
team undertook a project to assess the use of support needs data fields and
accuracy of the data on CHAIN. This report presents the findings of
•
Analysis about how soon after first contact and how regularly support needs
forms are completed was undertaken. (Section3)
ƒ
An exercise to compare forms from CHAIN completed by outreach workers with
assessments made by hostels. Hostels are often able to make a more detailed
assessment of needs using various tools and measures such as the ‘outcomes
star’. (Section 4)
2
Conclusions
ƒ
Support needs forms are currently updated around once every four months. The
period between reviews has reduced in recent years. (Section 3)
ƒ
There is an 80% or higher match in the prevalence of drug and alcohol support
needs reported for individuals by hostels and by outreach. The match rate is
understandably lower for mental and physical health. (Section 4.2)
ƒ
Figures reported from CHAIN are more likely to underestimate than overestimate
the prevalence of support needs in the single homeless population.
ƒ
Broadway and others using CHAIN data can be more confident about how
accurate support needs information from CHAIN is. The debate about this data is
evidence based – prior to undertaking this work all CHAIN could assert was that
the data was the best top line data available about rough sleepers support needs
given the wide use of the system.
ƒ
Given that in the areas of drugs and alcohol the match between CHAIN and
hostel data is high we may be able to get more value from the information held.
For example by extrapolating data collected during this exercise to assess the
likely profile of severity of support needs amongst outreach clients. (Section 4.4).
ƒ
It would be possible to undertake follow up work exploring the reason for
mismatches – for example speaking with outreach teams and hostels about case
studies where information does not match. This will provide an indication of
whether the mismatches reflect fluctuation in needs or whether new tools for
outreach teams to assess mental and physical health needs in particular could be
useful.
2
3
Findings – use of support needs forms
The base for the CHAIN data used in the analysis in section 3 is data for all CHAIN
clients with 2 or more actions (15,526) and all forms completed for these clients
(12,810). Inevitably a proportion of client records had to be excluded from analysis
for example where there had been issues with the merging of duplicate records.
Timing and Frequency of assessment
On average, forms are reviewed just under once a year (every 327 days). However,
this time between forms has been decreasing in recent years. The average for
clients whose records were created after 2006 is 165 days elapsing between
assessments and from 2007 this decreases to every 121 days – so just less than four
months between forms on average.
Figure (a)
Average days between forms being completed
When client record was created
All Clients
Clients created 2006
Clients created 2007
Clients created 01/01 - 30/06/2008
Average days between assessments
327
254
157
85
The average number of days between the creation of the client record and the first
form is 165. But it is important to note that approximately 50% of first forms are done
on the day a the client record was created — if these are excluded the average rises
to 342 days for all clients and 124 days for clients created from June 2006 forward.
Figure (b)
Average days between a client record being created and a needs
assessment being recorded on CHAIN
All clients
Average days from client record being created and first needs
assessment being recorded
% of support needs assessments completed on day record created
165
52%
Average days to first form excluding those completed on day record
created
342
Clients created June 2006 forward
Average days from client record being created and first needs
assessment being recorded
% of support needs assessments completed on day record created
22
Average days to first form excluding those completed on day record
created
49%
124
3
In order to determine how recent form information is, the date of the last form was
compared to the date of the last action on CHAIN. The average time between the last
support needs form being completed and the last action was 204 – around 6.5
months. This reduces to 143 days from 2006 forward.
In some cases the date of the last form was after the date of the last action. This
may result from faulty merging of duplicate clients, or it may indicate that a client’s
needs are in some cases being reviewed in a period of inactivity. Unfortunately, we
cannot determine how many cases fall into which category
Figure (c)
Average days between last form and last action
All clients
Average days from last form to last action
% forms later than last action
Clients created June 2006 forward
Average days from last form to last action
% forms later than last action
204
14%
143
8%
In all case above, the timing and frequency of form use improves greatly for clients
created more recently.
4
4
Findings - comparing outreach support needs assessments with hostel
assessments
4.1
Methods
To compare outreach workers top line assessments of support needs recorded on
CHAIN with detailed assessments made by hostel staff 23 hostels were selected to
cover all boroughs and the range of hostel size.
Current client lists were sent to hostels on an Excel form which enabled them to rate
support needs for alcohol, drugs, mental health and physical health as high, medium,
low, none, or not known.
16 hostels replied, giving data on 227 clients. 32 clients had to be eliminated as they
had no form on CHAIN, leaving a sample size of 195. These replies were compared
to the most recent form data on CHAIN. In each of the 4 categories, responses
including ‘unknown’ or ‘not selected’ were excluded. The hostel ratings of high,
medium and low were converted to ‘yes’ or ‘no’ to facilitate comparison with CHAIN
data, but were retained for the level of support needs analysis.
4.2
Overall match rate
The overall match rate when comparing support needs for individuals from CHAIN
and from Hostels are shown in figure (d). A match is where:
o A hostel stated the client has a high, medium or low support need in a
given area and CHAIN states ‘yes’ against the same support need
o A hostel states that the client has no support need and the CHAIN record
also states ‘no’.
In some cases as mismatch will occur as a new support need has developed over
time or a support need has been fully addressed over time – for example where
someone uses alcohol to help them cope with elements of rough sleeping such as
the cold and then reduces or abstains once they are in a hostel. However support
needs are often of an enduring nature albeit with frequent variations in severity of
need. We therefore suggest that most mismatches would occur where an outreach
team has not been able to make the full and detailed assessment that would be
made by a hostel.
The match rates for drugs and alcohol are very high (84% and 80% respectively).
(see section 4.3 for more information about the mismatches). Match rates for
physical health and mental health are less high. This could be for the following
reasons:
o Assessments of mental health are subjective and complex - outreach
workers maybe hesitant to label a client as mentally ill without very firm
foundation
o Physical health problems are sometimes temporary or not mentioned by
the client while other matters are more pressing.
5
Figure (d)
% of 195 clients where forms on CHAIN matched assessment by
hostels by four support needs areas
84%
Drugs
80%
% match
Alcohol
Mental Health
65%
Physical Health
53%
0%
Drugs
Alcohol
Mental Health
Physical Health
4.3
20%
40%
60%
80%
100%
84%
80%
65%
53%
Mismatches – do outreach teams record higher or lower prevalence of
support needs?
As shown in figure (e) where the assessments do not match, it is much more
common for the outreach team to have rated the client as ‘none’ and the hostel to
have identified a need rather than the other way around.
So figures reported from CHAIN are more likely to underestimate than overestimate
the prevalence of support needs in the single homeless population.
Figure (e)
Mismatches - % of occasions where hostel and outreach
assessments do not match
Alcohol
Drugs
Mental Health
Physical Health
4.4
Outreach reports need but
hostel do not
5%
2%
3%
14%
Hostel reports need but
outreach do not
15%
14%
32%
33%
The level of support need for clients included in analysis
CHAIN provides a top line yes/no classification for support needs. Hostels use
various tools to rate the severity of support needs and through this track clients’
6
progression. Various inventories, scales and indicators are used for this purpose. We
asked hostels to simply rate each support need identified as high, medium or low.
The results give a rough and ready indication of the prevalence of different levels of
need across the support needs.
In the case of alcohol and drugs, amongst those that have a support need the split
between high medium and low is fairly even – around a third in each group. In the
case of mental and physical health lower levels of need are more prevalent than high
levels of need.
4.5
Level of Support Needs as reported by Hostels
Alcohol Drugs
High
Med
Low
None
Not known
Mental health
Physical health
21%
21%
11%
18%
22%
19%
19%
17%
24%
20%
34%
31%
28%
37%
33%
31%
2%
2%
2%
2%
7