CENTRE FOR
DISABILITY RESEARCH
AND POLICY
FACULTY OF
HEALTH SCIENCES
Roger J Stancliffe
Eric Emerson
1

Monitoring progress in achieving disability
equality: Fulfilling Potential

December
2011
Discussion document
September
2012
Discussions
So Far
F
P
September
2012
Next Steps
January
2013
Building understanding of disability
July
2013
Actions, outcomes and indicators
Obtain wide range of opinions and ideas to inform new strategy
Highlight current and planned activity and publish responses to discussion document
Outline strategic priorities and explain how specific actions will be developed
Co-production
Draw on statistics and research to explore nature of disability in the UK today
Outline specific actions and timelines and how progress will be monitored

R
http://odi.dwp.gov.uk/fulfilling-potential/index.php

D
& I
›
-
11 domains containing
1 or 2 headline indicators up to 11 supporting indicators
Education
Employment
Choice & control
Income
Social participation
Transport
Health & wellbeing
Information & access
Housing
Friends & family
0 5 10
Number of Indicators
15
5

K
M
1
› Co-production is key to balancing the interests of
Government
DPOs
(People with disabilities)
ONS Opinions Survey 2013
What one thing would make the biggest different to your quality of life?
Improved health
More money
More time with family
Job change/Flexibility
More choice/independence
More friends/leisure activities
Better housing
0% 20% 40% 60%

› Monitor the inequality gap
over time
and for ‘at risk’ groups
K
M
2

L
B
Monitoring Changes in the Wellbeing of Young Disabled Australians
› Framework
v1 (2009) UN Convention on the Rights of
Persons with Disabilities
v2 (2011 onwards) Australia’s Social Inclusion
Indicators Framework
› Data
Annual survey of Household Income & Labour
Dynamics in Australia (HILDA)
Indicators matched to 44% of Framework indicators
8

EFT
EHIND
› Between 2001 and 2012, the gap between young Australians with disabilities and their non-disabled peers has grown in 11
(of 22) areas including
Not being employed
Being long-term unemployed
Having low economic resources and financial stress
Having low subjective well-being
Not having someone to turn to in times of crisis
Not having a voice in the community
Experiencing entrenched multiple disadvantage.
› It has not narrowed in any area at all

10
EFT
EHIND
Risk of
Exposure to
Entrenched
Multiple
Disadvantage
Among of
Australian
Adolescents and Young
Adults:
2001-2012
1
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
10

Capturing the views of service users:
England’s Adult Social Care Survey

A
S
C
S
› Annual survey (began 2010/11)
› How effectively are services helping users to live safely and independently? What is impact of services on their quality of life?
› Cross-sectional stratified random sample from every Council with
Adult Social Services
Responsibilities (CASSR, n=154)
› 2012/13 sample > 50,000
› Service users aged 18 and over in receipt of services funded wholly or in part by Social Services
› Questions about
Subjective wellbeing
Social relationships
Leisure activities
Safety
Food quality
And more ………………….
http://www.hscic.gov.uk/article/2215/User-Experience-Survey-Adult-Social-Care-
Guidance-2012-13

S
R
1
Physical Disability, Frailty and Sensory Impairment
Mental Health
Intellectual Disability
30%
25%
20%
15%
10%
5%
0%
My life is bad or very bad
I have little social contact with people and feel socially isolated
I do not have enough control over my daily life
I do not feel safe

S
R
2
2010/11 2011/12 2012/13
80%
70%
60%
50%
40%
30%
20%
10%
0%
ID: I am very happy with the way staff help me, it's really good
ID: I have as much control over my daily life as I want
ID: I have as much social contact as I want with people I like

W
I
G
O
?
› The data are valid, English disability services (especially for people with intellectual disability) are excellent and getting better
› People with intellectual disability have low expectations
‘Concentrating on mental characteristics (such as pleasure, happiness or desires) can be particularly restrictive when making interpersonal comparisons of well-being and deprivation. Our desires and pleasuretaking abilities adjust to circumstances …. deprived people tend to come to terms with their deprivation …
[as such] ….. the deprivation of the persistently deprived may look muffled and muted ’
› Supported responding introduces significant positive bias

S
R
12%
9% Self-completion
10%
8%
6%
4%
2%
0%
Completion supported by careworker 56%
My life is bad or very bad
I have little social contact with people and feel socially isolated
I do not have enough control over my daily life
I do not feel safe

R
A
Implications
1.
Invest in annual data collections that can be used to monitor all key aspects of disability equality
2.
Use subjective measures of wellbeing with extreme caution (if at all)
3.
Invest in developing supports for survey completion that are independent of service provider agencies
4.
Develop alternative ways of capturing the voices of people who use disability services

US National Core Indicators
CENTRE FOR
DISABILITY RESEARCH
AND POLICY
FACULTY OF
HEALTH SCIENCES
18

1.
Describe National Core Indicators (NCI)
2.
Examples of policy-relevant NCI analyses
1.
Choice of Living Arrangements
2.
Wellbeing and Choice of Living Arrangements
3.
ASD Eligibility Policies
3.
How is NCI Data Used by US States?
1.
Example from Kentucky
4.
Possible Applications in Australia

1. National Core Indicators (NCI): Features, development, current use and future expansion

› Annual survey on a random sample of service users (400+ per state)
Longitudinal (multi-year) data on specific individuals not available by design (to avoid survey fatigue)
› States opt in
Some states add questions to the standard NCI instrument to investigate issues of specific local policy interest
Some states focus on different parts of their service system in different years by oversampling different subgroups
› Used only by the intellectual disability and developmental disability
(ID/DD) service system in each participating state

›
Benchmarking: For each indicator the NCI provides for:
state-by-state comparisons,
comparisons with the national average,
year-by-year comparisons within states
› Summary data publicly available via the NCI website http://www.nationalcoreindicators.org/
›
Individual state annual reports available via NCI website
› Deidentified NCI data shared with
(selected) university researchers for independent secondary analysis

Individual
Outcomes
Employment
Choice &
Control
Relationships
Community
Inclusion
Family
Indicators
Information &
Planning
Access to
Supports
Community
Connections
Choice &
Control
Health,
Welfare, &
Rights
Health &
Wellness
Safety
Respect &
Rights
System
Performance
Service
Coordination
Incidents &
Mortality
Staff Turnover
= Adult Consumer Survey

HI
CA
WA
OR
UT
AZ
NM
SD
TX
OK
MO
AR
WI
IL
MS
MI
IN
OH
KY
AL
NY
PA
GA
VA
SC
NC
LA
FL
NH
ME
MA
NJ
RI
CT
MD
DC
2010-11 24 States
2011-12 29 States
2012-13 35 States

2. Examples of policy-relevant analyses using NCI data

POLICY IMPLEMENTATION QUESTION
Overall
What percentage of adult service users living outside the family home choose where and with whom they live?

6778 adult developmental disabilities service users living in non-family-home service settings in 26 US states (Stancliffe et al., 2011)

CONCLUSION
› Most people have no choice of where to live (55%) or whom to live with (59%).
› Policies endorsing choice of living arrangements are not being implemented satisfactorily.

POLICY IMPLEMENTATION QUESTION
Does choice of living arrangements vary by residence type and level of disability?

Choosing Whom to Live With (person chose) by Level of Disability and Residence Type
80
60
40
20
0
Mild Moderate
Level of ID
Severe Profound
Institution
Group home
Apartment
Own home
Foster care
Nursing facility

› People with severe/profound intellectual disability had little or no choice of whom to live with, regardless of residence type.

Choosing Whom to Live With (person chose) by Level of Disability and Residence Type
80
60
40
20
Group home
0
Mild
Own home
Moderate
Level of ID
Severe Profound
Group home
Own home

CONCLUSIONS
› For people with mild and moderate intellectual disability, choice of living companions varies dramatically by residence type:
own home (73.5% and 57.3% chose)
group home (9.5% and 9.7% chose)
› These findings support policies promoting individualised settings , such as one’s own home or an agency apartment.
These settings do provide substantially more choice about living arrangements , as intended .

Wellbeing and Choice of Living Arrangements

POLICY IMPLEMENTATION QUESTION
Does exercising choice of living arrangements lead to greater wellbeing? (Stancliffe et al., 2009)

› Loneliness
› Feeling happy
At Home
› Feeling afraid at home
› Feeling afraid in your neighborhood
› Home staff nice and polite
› Liking home

Self-Report Data Only
› Well-being items come from Section I of the NCI
Consumer Survey, which may only be completed by interviewing the person receiving services. Due to communication difficulties, some service users could not take part in the interview.
› Only included participants who were judged by interviewers to have given valid and consistent interview responses.
› These selection criteria yielded predominantly people with mild or moderate ID.

Loneliness the most widespread problem
100
90
80
70
60
50
40
30
20
10
0
53.9
79,7
Percent with positive outcome
90,6
83,2
79,2
88,6
Lonely Scared
Home
Scared
N'hood
Happy
Well-being outcome
Staff Home Like
Home

Personal characteristics controlled statistically in all comparisons.
Item
Loneliness
Feeling happy
Afraid at home
Afraid in neighbourhood
Home staff nice
Like home
Chose Who to Live with




Chose Where to Live




› Choosing where to live and whom to live with each are associated with:
multiple wellbeing benefits and
no wellbeing detriments.

› Grouping states by common policies to evaluate the impact of these polices on service provision and client outcomes.
EXAMPLE
› Hewitt et al. (2011) compared the proportion of state ID/DD service users with and autism/ASD diagnosis by state autism/ASD service eligibility policies:
ASD Eligibility Policies
None
Related condition (RC)
RC + autism –specific HCBS
No. of
States
% service users with ASD diagnosis
6 6.6%
14 8.4%
5 9.3% x 2 (2, 12,382)=17.39, p <.001

3. How is NCI Data Used by US States?

› Overall quality management
Set priorities for quality improvement
Report evidence to federal funders
› Report results to stakeholders
Internal state staff
Quality councils/review committees
State legislatures
Providers
Individuals and families receiving services

Kentucky: Example of State Application
Kentucky Health and Wellness Initiative
› Quality Improvement Committee (QIC) convened in 2012
Identified health and exercise as target area
Initiative funded 8 pilot programs promoting inclusive physical fitness and healthy eating activities
› See Moseley, Kleinert, Sheppard-Jones & Hall (2013)

Community-based Employment
› 7% of respondents from
Kentucky and 14% of respondents across NCI
States were reported to be working in communitybased employment (settings where most people do not have disabilities)

Recommendation:
Increasing integrated community employment
Changes included:
› Large increase in hourly funding rate to providers of integrated supported employment.
› Small decrease (11%) in hourly funding rate to providers of day activity services.

› 72% of respondents from
Kentucky and 40% of respondents across NCI
States reported they feel lonely at least half of the time.

Recommendations: Loneliness and friendships.
Changes included:
› Increased hourly funding rate for all integrated services and decreased rates for all segregated services.
› Launched a ‘‘Community
Belonging ’’ initiative , starting with
10 agencies, to ensure that people are connected to their communities via unpaid relationships.

Researchers using NCI Data
Research policy and process for requesting data and/or tools
Formal process through NASDDDS Research Committee
Several university researchers and students approved and currently working with data (autism, ageing, health
– e.g.,
University of Minnesota)
Public use of NCI Data
› New website with chart generator feature – customisable summary data publicly available

Chart Generator www.nationalcoreindicators.org

4. Possible Applications in Australia

OVERALL CONCLUSION
› Regular assessment of outcomes experienced by people with disability facilitates evaluation of:
Benefits to service users
Benefits of different service types
Benefits to service users with different characteristics
Policy implementation and effects
Relative disadvantage compared to the general community (subject to availability of comparison data).
› Countries that do not currently have a national system for assessing outcomes should examine the NCI for its local applicability.

Application of the NCI in Australia
NCI items used extensively in 1999 Australian national survey
Issue NCI in USA Australia
Target groups ID/DD only All disability types
Level of analysis State Needs discussion
Compliance burden Costs borne by each state.
Sampling strategy means limited burden on individual service users and providers.
Data availability
Process for identifying indicators
Raw data : State officials, independent researchers
Summary data : Public
Consensus among participating states and organisations.
Needs discussion
Needs discussion
Needs discussion

Implications and actions for policy makers
• Building on international examples, develop, field test, implement and refine an Australian national system of outcomes monitoring for disability service users.
• Gather (some) outcomes data directly from disability service users.
• Use the outcomes data to evaluate and improve the effectiveness of disability services, funding and policy.
• Make deidentified data publicly available.
• Create a body, with relevant expertise, to manage the outcomes monitoring process.

Implications and actions for practice
• Develop a consumer outcomes focus when delivering and managing disability services.
• Use outcomes data to evaluate and improve the effectiveness of disability services and policy.
• Educate stakeholders about outcomes and outcomes data.

References
Bradley, V. J. & Moseley, C. (2007). Perspectives: National Core Indicators: Ten years of collaborative performance measurement. Intellectual and Developmental
Disabilities, 45 (5), 354-358.
Hewitt, A. S., Stancliffe, R. J., Johnson Sirek, A., Hall-Lande, J., Taub, S., Engler, J.,
Bershadsky, J., Fortune, J., & Moseley, C. (2011). Characteristics of adults with autism spectrum disorder who use adult developmental disability services: Results from 25 US states. Research in Autism Spectrum Disorders, 6 (2), 741-751.
Moseley, C., Kleinert, H., Sheppard-Jones, K., & Hall, S. (2013). Using research evidence to inform public policy decisions. Intellectual and Developmental
Disabilities, 51 (5), 412-422. doi: 10.1352/1934-9556-51.5.412
Sheppard-Jones, K., Hall, S., & Kleinert, H. (2011). Using large-scale data sets to inform state DD policy. Community Services Reporter, 18 (9), 4 –10.
Sheppard-Jones, K., Prout, H., & Kleinert, H. (2005). Quality of life dimensions for adults with developmental disabilities: A comparative study. Mental Retardation, 43 ,
281-291.
Stancliffe, R. J., Lakin, K. C., Larson, S. A., Engler, J., Taub, S., & Fortune, J. (2011).
Choice of living arrangements . Journal of Intellectual Disability Research, 55 (8),
746-762.
Stancliffe, R. J., Lakin, K. C., Taub, S., Chiri, G., & Byun, S. (2009). Satisfaction and sense of well-being among Medicaid ICF/MR and HCBS recipients in six states.
Intellectual and Developmental Disabilities , 47(2), 63-83.

Centre for Disability Research and Policy www.sydney.edu.au/health_sciences/cdrp/
Email: disabilitypolicy.centre@sydney.edu.au
eric.emerson@sydney.edu.au
roger.stancliffe@sydney.edu.au