Şule Çalıkoğlu, Ph.D.
Carol Christmyer, RN., MS
Bruce Kozlowski
Outline of the Presentation
 CMS Five‐Star Rating
 Summary of the Family Member/Responsible Party Survey
 Relationship between Five‐Star and Overall e a o s p be ee
e S a a dO e a
Family/Responsible Party Rating
Five Star Rating: www medicare gov
Five‐Star Rating: www.medicare.gov
Health Inspections
 3 most recent annual inspections – weighted in favor of the most recent surveys
 All complaint inspections – last 3 years
 Deficiencies are weighted by scope and severity
 State cut‐points, updated with every new survey State cut points updated with every new survey information:  Top 10% receive a five‐star rating
p
g
 The middle 70% receive a rating of two, three, or four stars
 The bottom 20% receive a one‐star rating
Staffing
 Nursing Staff:
 Total nursing hours per resident day (RN+LPN+nurse
g
p
y(
aid hours)
 RN hours per resident day
 Case‐Mix Adjusted
C
Mi Adj
d
 National cut‐points using freestanding facilities, fixed for two years:





Five‐Star : RN >= 0.550 hrs Total>= 4.080 hrs.
Four‐Star: >= 75 percentile of the distribution
5p
Three‐Star:>=Median
Two‐Star: >= 25th percentile of the distribution
One Star:<25th percentile of the distribution
One‐Star:<25
Quality Measures (QMs)
 Long‐Stay Prevalence g
y
measures
 Short‐Stay Prevalence y
measures
 ADL change
 Delirium
 Mobility change
 Pain
 High‐Risk Pressure Ulcers
 Pressure Ulcers
 Long‐term catheters
L
h
 Physical restraints
 Urinary Tract Infection
 Pain
Quality Measures
 3 most recent quarters of available data
 Risk adjustment for five of the QMs (mobility, catheter, the long‐stay pain measure, delirium, short‐stay pressure ulcers)  Percentiles based on national distribution for all QMs, fixed for two years, except for two ADL measures that are state specific, updated quarterly
t t ifi d t d t l
 ADL measures weighted 1.6667 times as high as the other measures
 Imputation of missing data using state average
Overall Rating
Example
/
MHCC Family/Responsible Party Survey
 Survey Population : Family member/Responsible party of residents who had been at the facility for t f id t h h d b
t th f ilit f 90 days or more in all licensed nursing homes in M l d
Maryland
 Survey Method: Mailed survey with follow‐up y
y
p
reminder post cards and phone calls
 2009 Survey: Sample size: 17,181 S
S
l i 8 Response rate: 57.7%
Areas Assessed by the Survey
• Staff and Administration
• Care provided C id d • Food and meals • Residents’ personal rights • Physical aspects of the nursing home Ph i l t f th i h
• Overall rating
g
Survey Respondent Characteristics
 48% of the respondents are resident’s children
 55% of the residents had stayed in the nursing home % f h id
h d d i h i h
more than 2 years, 24% stayed 1 to 2 years.
 55% of the respondents contacted the resident by % f h d
d h id b phone 0‐1 time in the last 6 months
 65% of the respondents visited the residents more 6 % f h d
i i d h id
than 20 times in the last 6 months
Research Methods
 Total number of NH in Maryland: 233
 Total number in the study : 208
 Excluded facilities:




10 Subacute facilities
12 Facilities with very small sample size (10 or less) and very low response rates (<30%)*
3 Facilities do not report to CMS
1 Facility is too new to have a rating, 1 closed
 Data Sources:
 Five‐Star Rating: Jan 2010
 MHCC Family/Responsible Party Experience Survey 2009
 Minimum Data Set 2009
 MHCC Bed Inventory, Long Term Care Survey and Occupancy y,
g
y
p
y
Reports
Multiple Linear Regression: Control Factors
Measurement
Mean
Percent of Total
Patient Days Paid
by MEDICAID
60.07%
Occupancy Rate
89.05%
Number of
Licensed Beds
Ownership Type:
For-Profit
Non-Profit
131
63.9%
36.1%
Std
Dev.
Min
21.08
Max
Lower
Upper
Quartile Quartile
0 95.3%
52.6%
7.54 40.8% 99.4%
86.1%
61.9
31
558
93
Data Source
MHCC
72.6% Occupancy
Report, 2008
MHCC
Occupancy
94.3%
Report, 2008
159
MHCC Bed
Inventory
MHCC Long
Term Care
Survey, 2008
F il /R
ibl P t O
ll R ti
f
Family/Responsible Party Overall Rating of Care Using any number from 1 to 10, where 10 is the best care possible and 1 is the
worst care possible, what number would you use to rate the care at this
nursing home?
40%
35%
30%
30%
26%
25%
20%
20%
15%
11%
10%
5%
0%
1%
1%
Worst
2.00
3.00
4%
5%
5.00
6.00
2%
0%
4.00
7.00
8.00
9.00
Best
i ib i
f
i
b i
Distribution of Nursing Homes by Five‐Star Ratingg
Average Family Score by Five‐Star Rating
Simple Linear Regression Results
HEALTH
INSPECTIONS
QUALITY
MEASURES
NURSE STAFFING
Estimate
Standard
Error
Estimate
Standard
Error
7.82
0.081
7.96
5 Stars
1.31 **
0.163
4 Stars
0.84 **
3 Stars
OVERALL QUALITY
Estimate
Standard
Error
Estimate
Standard
Error
0.102
8.42
0.137
7.80
0.079
0.85 **
0.208
-0.02
0.179
1.07 **
0.135
0.111
0.61 **
0.127
-0.16
0.159
0.80 **
0.110
0.62 **
0.110
0.31
*
0.135
-0.01
0.168
0.70 **
0.109
2 Stars
0.29 *
0.109
0.31
*
0.131
-0.15
0.168
0.24
*
0.116
1 Star
(Reference
Category)
** p value
<0.0001
* p value
<0.05
0.00
.
0.00
.
.
.
0.00
.
.
Variable
Intercept
.
R Square= .33
R-Square=
33
R Square= .13
R-Square=
13
0.00 .
R Square= .01
R-Square=
01
R Square= .33
R-Square=
33
Multiple Linear Regression Results: Five
Multiple Linear Regression Results: Five Stars
HEALTH INSPECTIONS
Estimate
Standard
Error
Intercept
8.33
0.131
5 Stars
QUALITY MEASURES
NURSE STAFFING
Estimate
Standard
Error
Estimate
Standard
Error
0.153
8.96
0.141
8.27
0.139
0.94 ** 0.169
0.53 * 0.193
-0.09
0.151
0.143
4 Stars
0.62 ** 0.113
0.31 * 0.123
-0.12
0.12
0.133
3 Stars
0.41 ** 0.108
0.18
0.118
-0.04
0.141
2 Stars
0.18
0.102
0.21
0.115
-0.09
0.142
0.78 *
*
0.51 *
*
0.52 *
*
0.21 *
1 Star
0.00
.
0.00
.
0.00
.
Variable
** p value <0.001
* p value <0.05
R-Square= .47
Estimate
Standard
Error
8.57
OVERALL QUALITY
R-Square= .38
R-Square= .35
0.119
0.106
0.108
0.00
R-Square= .45
.
Multiple Linear Regression Results: Other
Multiple Linear Regression Results: Other Factors
FAMILY RATING OF CARE
Variable
Standard
Error
Estimate
% Medicaid-High
-0.25 **
0.086
% Medicaid-Medium
-0.11
0.087
% Medicaid-Low
0.00
.
Occupancy-High
0.20
*
0.086
Occupancy-Medium
0.18
*
0.084
Occupancy-Low
Bed Size-High
Size High
0.00
-0.11
0.11
.
0.095
Bed Size-Medium
-0.07
0.098
0.00
.
Bed Size-Low
For-Profit
Non-Profit
-0.43
0.00
*
0.078
.
** p value <0.001
* p value <0.05
K
St ti ti
Kappa Statistics
Domain/Statistic
Value
ASE
95% Confidence Limits
HEALTH INSPECTIONS
Simple Kappa
0 158
0.158
0 041
0.041
0 079
0.079
0 238
0.238
Weighted Kappa
0.375
0.041
0.294
0.456
Simple Kappa
0.054
0.037
-0.02
0.127
Weighted Kappa
0.193
0.045
0.106
0.281
Simple Kappa
0.044
0.036
-0.03
0.115
Weighted Kappa
0.025
0.047
-0.07
0.116
0 128
0.128
0 04
0.04
0 05
0.05
0 207
0.207
0.37
0.042
0.288
0.453
NURSE STAFFING
QUALITY MEASURES
OVERALL RATING
Simple Kappa
Weighted Kappa
Outliers ? y
g y
Box‐Plots of Family Member Rating by Overall Five‐Star Ratings
Policy Implications
 Family member/Responsible party experience of care is associated with CMS Five‐Star Rating but would provide additional information in assessing the quality of care for nursing homes.
 The lack of relationship between quality measures and experience of care requires more detailed examination.
 A few facilities that scored very high/low in family survey pp
g
y
but received opposite ratings need further analysis to understand the relationship between family experience of care and technical quality measures.
Maryland Health Care Commission Nursing Home Guide website:
www mhcc maryland gov
www.mhcc.maryland.gov
email: sule.calikoglu@gmail.com
cchristmyer@mhcc.state.md.us
hit
@ h t t
d
Multiple Linear Regression Results: Other
Multiple Linear Regression Results: Other Factors
HEALTH INSPECTIONS
Variable
Estimate
Standard
Error
NURSE STAFFING
Estimate
QUALITY MEASURES
Standard
Error
Estimate
Standard
Error
OVERALL QUALITY
Estimate
Standard
Error
% Medicaid-High
-0.24 *
0.084
-0.28 *
0.092
-0.31 **
0.092
-0.25 **
0.086
% Medicaid-Medium
-0.11
0.085
-0.14
0.092
-0.16
0.094
-0.11
0.087
% Medicaid-Low
0.00
.
0.00
.
0.00
.
0.00
.
Occupancy-High
0.14
0.086
0.30 **
0.092
0.28 *
0.093
0.20 *
0.086
Occupancy-Medium
0.18 *
0.083
0.21 *
0.092
0.18
0.093
0.18 *
0.084
Occupancy-Low
0.00
.
.
0.00
.
0.00
.
Bed Size-High
-0.08
0.091
-0.26 *
0.095
-0.32 **
0.094
-0.11
0.095
Bed Size-Medium
-0.09
0.092
-0.20 *
0.1
-0.27 *
0.099
-0.07
0.098
0.00
.
.
0.00
.
Bed Size-Low
For-Profit
Non-Profit
-0.44 **
0.00
0.075
.
0.00
0.00
-0.50 **
0.00
.
0.082
.
0.00
-0.55 **
0.00
0.082
.
-0.43 *
0.00
0.078
.
** p value <0.001
* p value <0.05
S
R
d
Ch
i i
Survey Respondent Characteristic: y
p
What is your relationship to the resident?
60%
48%
50%
40%
30%
20%
11%
13%
12%
9%
10%
1%
0%
3%
2%
0%
Spouse,
Partner
Parent
Mother-in- Grandparent Aunt, Uncle
Law, Fatherin-Law
Sister,
Brother
Child
Friend
Other
Resident Characteristic: Length of Stay
g
In total, about how long has the resident lived in this nursing home?
60%
55%
50%
40%
30%
24%
20%
13%
10%
8%
0%
3 months to almost 6 months
6 months to 11 months
One or two years
More than two years
Respondent Contact Characteristic: Phone Contact
In the last 6 months, how many times did you talk with the resident by phone?
60%
55%
45%
30%
21%
12%
15%
7%
5%
0%
0 to 1 time
2 to 5 times
6 to 10 times
11 to 20 times
More than 20 times
Respondent Contact Characteristic: Visits
In the last 6 months, about how many times did you visit the resident in the
nursing home?
75%
65%
60%
45%
30%
12%
15%
9%
10%
2 to 5 times
6 to 10 times
4%
0%
0 to 1 time
11 to 20 times
More than 20 times