Standardizing Assessment
Data: Continuity Assessment
Record and Evaluation
(CARE) Item Set
Presented by:
Barbara Gage, PhD
Engelberg Center for HealthCare Reform/The Brookings Institution
Stella Mandl, BSW, BSN, PHN, RN
Center for Clinical Standards & Quality/CMS
Presented to:
The Long Term Care Discussion Group
Wednesday, November 6, 2103
Thinking Ahead: Data Element
Standardization
Stella Mandl, RN
Technical Advisor
Division of Chronic and Post Acute Care
Center for Clinical Standards and Quality
Center for Medicare & Medicaid Services
Stella.mandl@cms.hhs.gov
Data Assessment Elements Goal
When we keep in mind the ultimate goal of
and step back to look at the big picture of what’s
been done to prepare, it becomes clearer where
the work converges; how much of the work is
connected and has already been done to achieve
Achieving Uniformity to Facilitate Effective Communication for
Better Care of Individuals and Communities
3
CARE: Concepts
Guiding Principles and Goals:
Assessment Data is:
• Standardized
• Reusable
• Informative
Standardization:
• Reduces provider burden
• Increases reliability and validity
• Offers meaningful application to
providers
• Facilitates patient centered care,
care coordination, improved
outcomes, and efficiency
• Communicates in the same information across
settings
• Ensures data transferability forward and
backward allowing for interoperability
• Fosters seamless care transitions
• Evaluates outcomes for patients that traverse
settings
• Allows for measures to follow the patient
• Assesses quality across settings, and Inform
payment modeling
4
As Is
As Is: Multiple Incompatible Data Sources
Nursing
Homes
MDS
To Be
Transition
LTCHS
LTCH CARE
Data Set
Inpatient
Rehab Facilities
IRF-PAI
Physicians
Hospitals
Home Health
Agencies
No Standard
Data Set
OASIS
Outpatient Settings
No Standard
Data Set
No Standard
Data Set
GOAL:
Uniform Data Elements
Across Providers
Standardized
Nationally Vetted
To Be: Uniform Assessment Data Elements
 Enable Use/re-use of Data
 Exchange Patient-Centered Health Info
 Promote High Quality Care
 Support Care Transitions
 Reduce Burden
 Expand QM Automation
 Support Survey & Certification Process
 Generate CMS Payment
5
CMS Vision for Quality Measurement
• Align measures with the National Quality Strategy
and Six Measure Domains
• Implement measures that fill critical gaps within the
six domains
• Develop parsimonious sets of measures - core sets
of measures
• Remove measures that are no longer appropriate
(e.g., topped out)
• Align measures with external stakeholders, including
private payers and boards and specialty societies
• Continuously improve quality measurement over time
• Align measures across CMS programs whenever
and wherever possible
6
CMS Framework for Measurement
Clinical Quality
of Care
• Care type
(preventive, acute,
post-acute, chronic)
• Conditions
• Subpopulations
Person- and
Caregiver- Centered
Experience and
Outcomes
• Patient experience
• Caregiver experience
• Preference- and goaloriented care
Care Coordination
• Patient and family
activation
• Infrastructure and
processes for care
coordination
• Impact of care
coordination
Population/
Community Health
• Health Behaviors
• Access
• Physical and Social
environment
• Health Status
Function
Efficiency and
Cost Reduction
Safety
•
•
•
•
•
All-cause harm
HACs
HAIs
Unnecessary care
Medication safety
• Cost
• Efficiency
• Appropriateness
• Measures should
be patientcentered and
outcome-oriented
whenever possible
• Measure concepts
in each of the six
domains that are
common across
providers and
settings can form
a core set of
measures
7
Building the Future State
• Assessment Instrument/Data Sets use uniform and
standardized items
• Measures are harmonized at the Data Element level
• Providers/vendors have public access to standards
• Data Elements are easily available with national
standards to support PAC health information
technology (IT) and care communication
• Transfer of Care Documents are able to incorporate
uniform Data Elements used in PAC settings, if desired
• Measures can evaluate quality across settings and be
used for setting comparisons
8
Keeping in Mind, the Ideal State
• Facilities are able to transmit electronic and
interoperable Documents and Data Elements
• Provides convergence in language/terminology
• Data Elements used are clinically relevant
• Care is coordinated using meaningful information that is
spoken and understood by all
• Measures can evaluate quality across settings and
evaluate intermittent and long term outcomes
• Incorporates needs beyond healthcare system
9
Standardizing Assessment
Data: CARE
Presented by
Barbara Gage, PhD
E-mail bgage@brookings.edu
Phone 202-238-3571
11
State of the Art in Measuring Patients’ Health
Status in Medicare
• Acute Hospitals  no standard assessment tool
to admit and monitor patients
• Long-Term Care Hospitals  newly standardized
items for quality reporting
• Inpatient Rehabilitation Facilities  IRFPAI
required
• Skilled Nursing Facilities  MDS required
• Home Health Agencies  OASIS required
12
DRA of 2005 called for one uniform tool that
could be used to measure patient health status at:
• Acute hospital discharge
• Admission/discharge/interim times for cases
using:
• Long Term Care Hospitals
• Inpatient Rehabilitation Facilities
• Skilled Nursing Facilities
• Home Health Agencies
13
Standard Language important for:
• Improving coordination of care – one set of terms
to define pressure ulcer severity, functional
impairment, cognitive impairment across
providers
• Improving data exchangeability – can’t merge
inconsistent items; need standard language to
transfer information between providers treating
the case
14
Continuity Assessment Record &
Evaluation (CARE) Development
Sponsored by CMS, Office of Clinical Standards and Quality
•
Principal Investigator/RTI Team: Barbara Gage, Shula Bernard,
Roberta Constantine, Melissa Morley, Mel Ingber
•
Co- Principal Investigators: Rehabilitation Institute of Chicago,
Northwestern University
•
Consultants: Visiting Nurse Services of NY, University of
Pennsylvania, RAND, Case Western University
•
Input by pilot test participants, including participating acute
hospitals, LTCHs, IRFs, SNFs, and HHAs
15
Consensus Development
• Year 1 of CARE development: Gain input from the
providers/research community
» Review existing assessment tools (MDS, IRFPAI,
OASIS, LTCH tools, acute items)
» Technical Expert Panels
– Clinical communities from 25 associations, including
AHA, AMRPA, NALTH, ALTHA, NAHC, VNAA,
AHCA, AAHSA, APTA, AOTA, ASHA, ANA,ARN,
CMA, Discharge Planners, Joint Commission, to
name a few
– Research/case-mix communities, including DRG,
FRG, HHRG, RUG
16
Standardized Assessment Items Should:
• Build on current measurement science but also add
new instrument development methodologies, and
• Modify existing assessment instruments to develop a
standard assessment instrument that will:
» Measure health and functional status
» Assess service needs
» Evaluate treatment outcomes
» Guide payment policy
» Improve seamless transitions
17
Common Domains in Existing Assessment Tools
 Administrative Information
 Social Support Information
 Medical Diagnosis/Conditions
 Functional Limitations
» Physical
» Cognitive
18
Differences Across Tools
• Individual items that measure each
concept
• Rating scales used to measure items
• Look-back or assessment periods
• Unidimensionality of individual items
18
19
Functional Item Comparisons
Tools
No. of
Functional
Items
Scale Levels
Assessment Periods
IRFPAI
18
7
Past 3 days
MDS 3.0
12
8
Past 5 days
OASIS
8
varies
Assessment day
20
Differences in Item Details
• Bathing:
» IRFPAI and OASIS – bathing only
» MDS – bathing and transferring in/out tub/shower
• Dressing:
» IRFPAI and OASIS – 2 items (Upper/Lower)
» MDS – 1 item
• Toileting:
» IRFPAI – level of independence
» OASIS – ability to get to/from
» MDS – ability to use toilet, transfer, change pads
Source: Gage and Green, 2006. Chapter 2. The State of the Art: Current CMS PAC Instruments in Uniform
Patient Assessment for Post-Acute Care, CMS Report, Contract #IFMC 500-02IA03.
21
Incomparable Functional Scales
IRF-PAI
7= Complete independence
MDS
0= Independent
6=Modified (device)
1= Supervision
5=Supervision
2= Limited Asst. (guided
maneuvering)
4=Minimal Assistance 25%
3= Extensive Asst
(3+ times/week)
4= Total Dependence
3= Moderate Assistance
50%
2=Maximal Asst. 25%
1= Total Asst.
0= Activity NA
8= Activity NA
OASIS
0= bathe independent
tub/shower
1= with devices,
independent
2= with person
(reminders, access, reach
difficult areas
3= participates but req.
other person
4= unable, bathes in
bed/chair
5= totally bathed by other
Unknown
22
CARE Item Development
• Formed 4 Workgroups
» Medical acuity/continuity of care
» Functional impairment
» Cognitive impairment
» Social/Environmental support
23
Workgroup Charge:
• Identify critical areas/domains for measuring
case-mix acuity, resource use, or outcomes
• Review existing legacy tools (MDS, IRFPAI,
OASIS), other leading measurement tools
(PROMIS, COCOA-B, VA)
• Propose core set that can be used at hospital
discharge and across all PAC settings
24
Issues in Selecting Items
• Identify Standard –
» Measures that applied across severity groups but
capture the range of severity
» Scales that do not lead to ceiling or floor effects when
measuring severity
» Assessment windows that would allow severity
comparisons across settings
• Self-report/performance-based items
• Current Medicare payment methods
• Minimal burden on providers
• Varying technology options across providers
25
Medical Items
•
Primary Acute Care Diagnosis
•
PAC Diagnosis
•
Comorbidities/Complicating Conditions
•
Physiologic Factors
•
Treatments
•
Prognosis/Life Expectancy/Frailty
26
Social/Environmental Items
Physical Living Environment
Prior residence
Structural barriers
Social Support and Assistance
Prior lives with
Lives with after discharge
Type of caregiver
Frequency of Assistance
27
Function Items
•
Core Function Items
» All patients, all settings
•
Supplemental Function Items
» Based on patient’s functional status (e.g.,
bedfast, self-care, basic mobility, IADLs)
» Maximize range of patient ability captured (i.e.,
avoiding floor and ceiling effects)
» Provide sufficient variation to capture
improvement in function
28
Core
Function
Domains
•
Prior
Functional Status
•
•
Need for Assistance
»
Eating
»
Bed Mobility
»
Oral Hygiene
»
Toilet Hygiene
»
Transfer
»
Dressing - Upper Body
»
Locomotion
Function Modifiers
»
Weight-bearing
»
Sitting Unassisted
»
Swallowing
29
Supplemental Function Items
• Bedfast
» Sit to lying, roll left or right, sponge bath
• Self Care
» Lower body dressing, shower/bathe, get in/out
of car, curb/1 step, short ramp
• Basic Mobility
» 4 steps-exterior, long ramp exterior, walk
longer distance-interior, wheel longer distanceinterior, 12 steps-interior
30
Supplemental Function Items
• IADLs
» Laundry, light shopping, make light meal,
dishwashing-by hand, dishwashing-machine,
telephone-answering, telephone-placing call,
medication management-oral meds,
medication management-inhalers, medication
management injectables
31
Cognitive/Self Report Items
• Mental status
» Orientation
» Memory
» Screening for delirium
• Mood/Depression
• Behavioral symptoms
• Pain
• Sensory input
» Vision
» Hearing
32
Cognitive Items (cont’d)
• Communication
» Comprehension
» Expression
• Fatigue
33
Reliability of the Standardized CARE Items
• Most CARE items based on existing validated items
currently used in the Medicare program; but few
items had been used in multiple settings or across
different levels of care.
• Two types of reliability tests were conducted to
examine whether the items performed consistently
across settings and across disciplines
» Traditional Inter-rater Reliability (pairs of
assessors rate the same patient similarly)
» Video Reliability (cross disciplinary rating of
standard video patients)
33
34
Traditional Inter-rater Reliability Methods
• Paired staff, matched on credential, assessed the
same patient
• Tested in subsample of participating PACPRD
providers (n = 34)
• Each site collected data on 10-15 patients
• 455 pairs of assessments collected
34
35
IRR Methods: Item Selection and Analysis
• Analyses followed methods used to evaluate
existing CMS tools (MDS, OASIS, IRFPAI)
• Reliability scores at least equal to existing tools
» Categorical items: Kappa (for 2 levels),
Weighted Kappa (for > 2 levels, Fleiss-Cohen
weights)
– Range: 0 poor, 0.01–0.20 slight, 0.21–0.40
fair, 0.41– 0.60 moderate, 0.61–0.80
substantial, and 0.81–1 almost perfect
» Continuous items: Pearson Correlation
36
IRR: Results
• Overall, the vast majority of Kappas (weighted and
unweighted) found were above 0.60
• Prior functioning and history of falls: 0.69 - 0.863
• Skin integrity:
» Pressure ulcers – 7 of 8 categorical items 0.67 or
higher
» Length and width correlations – approx 0.6
» Major wounds – 0.64 and higher
» Turning Surfaces – 3 of 5 above 0.6, 4 of 5 above
0.5
37
IRR: Results
• Cognitive status and Mood: 26 of 29 above 0.6
• Pain:
» Interview items – all 5 above 0.6, 4 above 0.79
» Observational assessment – 4 of 5 above 0.6
38
IRR: Results
• Impairments:
» Bladder and bowel:
– Current status items higher than 0.7
– Prior status items higher than 0.65
» Swallowing signs and symptoms: NPO (0.97),
None (0.84), coughing (0.68), loss of liquids and
holding food in mouth all low prevalence ranged
0.46-0.56
» All hearing, weight bearing, respiratory and
endurance items had kappas above 0.6
39
IRR: Results
• Functional status:
(calculated with and without letter codes)
» Core (self-care and mobility including walk and
wheel): all above 0.6 except ‘tube feeding’ and
‘walk 150 feet’
» Supplemental Self Care: all above 0.63 except
‘roll left and right’
» Supplemental Mobility: all above 0.63 except
‘walk 50 feet with two turns’ ‘walk 10 feet on
uneven surface’ and the wheel long and short
ramps
40
IRR: Results
• Functional status (cont’d):
(calculated with and without letter codes)
» IADLs: all above 0.7 excluding letter codes except
‘laundry’ and ‘light shopping’; all above 0.6 when
including letter code, except ‘use public
transportation’
• Overall plan of care: above 0.6
4
41
IRR: Summary
• IRR results indicate substantial to almost perfect
agreement for the majority of items evaluated
• The few lower kappa scores tend to be for low
prevalence items
• IRR results for CARE items are in line with the
majority of IRR results available for equivalent
items on MDS, OASIS, and FIM
42
CARE Item Reports
•
2 website URLs
»
http://cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Post-Acute-CareQuality-Initiatives/CARE-Item-Set-and-B-CARE.html
»
http://cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Post-Acute-CareQuality-Initiatives/Functional-Measures-.html
•
The Development and Testing of the Continuity Assessment Record and Evaluation (CARE) Item
Set: Final Report on the Development of the CARE Item Set. Volume 1 of 3 [PDF, 8MB]
•
The Development and Testing of the Continuity Assessment Record and Evaluation (CARE) Item
Set: Final Report on Reliability Testing. Volume 2 of 3 [PDF, 2MB]
•
The Development and Testing of the Continuity Assessment Record and Evaluation (CARE) Item
Set: Final Report on the Development of the CARE Item Set and Current Assessment
Comparisons. Volume 3 of 3 [PDF, 2MB]
•
Continuity Assessment Record and Evaluation (CARE) Item Set: Additional Provider-Type Specific
Interrater Reliability Analyses [PDF, 902KB]
•
Continuity Assessment Record and Evaluation (CARE) Item Set: Video Reliability Testing. [PDF,
348KB]
»
ASPE Report: Analysis of Crosscutting Medicare Functional Status Quality Metrics Using the
Continuity and Assessment Record and Evaluation (CARE) Item Set. Final Report [PDF, 2MB]
43
On-Going Efforts with standardized CARE
Item Set
• Developing setting-agnostic quality measures
• Quality Reporting Programs for IRF, LTCH, and
hospice
• E-specification of “Best in Class” by CMS/ONC
standards &interoperability groups
44
On-Going Efforts with Standardized CARE
Item Set
• Developing Outpatient Therapy Payment
Alternatives
– CARE-C: community therapy
– CARE-F: NF therapy
• B-Care (subset of standardized items for
Bundled Payment Initiatives)
• LTSS Care Items (add items for long term social
support programs)
45
For More Information:
Barbara Gage
Bgage@Brookings.edu
or
Stella Mandl
Stella.Mandl@CMS.HHS.gov