Alternative Methods for
Practice-Based Evidence
Harnessing Natural Variation for
Effectiveness Research
Practice-Based Evidence for Clinical Practice Improvement
by
Susan D. Horn, Ph.D
Institute for Clinical Outcomes Research
699 East South Temple, Suite 100
Salt Lake City, Utah 84102
801-466-5595 (V) 801-466-6685 (F)
shorn@isisicor.com
www.isisicor.com
1
Outline of Presentation
• Brief description of PBE-CPI, a practice-based
evidence approach, and how it differs from other
study methodologies
• PBE- CPI examples showing breadth of
findings from comprehensive data sets
• Application to pressure ulcer prevention in LTC
2
Practice-Based Evidence for
Clinical Practice Improvement Study Design
Analyzes the content and timing of individual
steps of a health care process, in order to
determine how to achieve:
• superior medical outcomes for the
• least necessary cost over the
• continuum of a patient’s care
3
Practice-Based Evidence for
Clinical Practice Improvement Study Design
Improve/Standardize:
Process Factors
•Management Strategies
•Interventions
•Medications
Control for:
Patient Factors
•Psychosocial/demographic Factors
•Disease(s)
•Severity of Disease(s)
Measure:
Outcomes
•Clinical
•Health Status
•Cost/LOS/Encounters
› physiologic signs and symptoms
•Multiple Points in Time
4
Outcomes Research
Uses large administrative databases to
evaluate effectiveness of
• specific treatment methodologies
• medical technologies
• providers
5
Practice-Based Evidence for
Clinical Practice Improvement Study Design
• PBE-CPI goes beyond outcomes research by
identifying outcomes
- examining detailed process steps
- adjusting for severity of illness
-
6
Practice-Based Evidence for
Clinical Practice Improvement Study Design
• PBE-CPI goes beyond guidelines, which
often are
- not decidable: give a vague description of
patients
- not executable: give a menu of process
steps to follow
- not connected to outcomes
7
Example from Stroke Guideline
Stroke patients with diagnosed depression
should be offered a course of treatment with
antidepressant drug therapy.
8
Practice-based Evidence for Clinical Practice Improvement
(PBE-CPI) compared to
Randomized Controlled Trial (RCT)
PBE-CPI
I. Select Key Conditions
to Study
RCT
I. Define Study
9
Practice-based Evidence for Clinical Practice Improvement
(PBE-CPI) compared to
Randomized Controlled Trial (RCT)
PBE-CPI
RCT
II. Data Collection
II. Data Collection
A. Patient Variables
- Patient eligibility and
A. Patient Variables
- Patient eligibility and
stratification factors
- Use severity of illness to
measure:
- comorbidities
- disease severity
- All patients qualify
stratification factors
- Eliminate patients who could
bias results:
- comorbidities
- more serious disease
~ 15% of patients qualify
10
Practice-based Evidence for Clinical Practice Improvement
(PBE-CPI) compared to
Randomized Controlled Trial (RCT)
PBE-CPI
RCT
II. Data Collection
II. Data Collection
B. Process Variables
B. Process Variables
- Methods for Stabilization
- Measure all processes and use
analysis findings to develop
protocol associated with better
outcomes
- Treatment Protocol
- Specify explicitly every
important element of the
process of care for both
treatment and control
arms
11
Practice-based Evidence for Clinical Practice Improvement
(PBE-CPI) compared to
Randomized Controlled Trial (RCT)
PBE-CPI
III. Data Analysis
Outcome Variables
- Dynamic improvement
based on fact
IV. Result
-
Effectiveness research
RCT
III. Data Analysis
Outcome Variables
- Change based on fact
IV. Result
- Efficacy research
12
Practice-based Evidence for Clinical Practice Improvement
(PBE-CPI)
• PBE-CPI is a comprehensive analysis of
patient, process, and outcome variables
• PBE-CPI studies are based on everyday
clinical practice, not controlled
circumstances.
13
Practice-based Evidence for Clinical Practice Improvement
(PBE-CPI)
•
•
Led by trans-disciplinary team that
Develops and frames questions
Gathers data
Interprets findings
Implements findings
Results in more generalizable and transportable
findings
14
PBE-CPI vs. RCT
• RCTs are considered to be evidence of the
highest grade.
• Observational (CPI) studies are viewed as
having less validity because they
reportedly over-estimate treatment
effects.*
* New England Journal of Medicine 2000; (June 22, 2000) 342:1887-92.
15
PBE-CPI vs. RCT
Results from 2 NEJM studies
“Average results of the observational
studies were remarkably similar to those of
the randomized, controlled trials.”
* New England Journal of Medicine 2000; (June 22, 2000) 342:1878-92.
16
PBE-CPI vs. RCT
Results from JAMA Study
Comparing results on 45 topics with binary
outcomes, found "very good correlation …between
summary odds ratios of randomized and nonrandomized studies"
r = 0.75, p < .001 for all studies,
r = 0.83, p < .001 for prospective studies.
JAMA (Aug 2001) 286;7:821-830
17
PBE-CPI vs. RCT
Conclusions
Well-designed observational studies do not
systematically over-estimate the magnitude
of the effects of treatment as compared
with those in randomized, controlled trials
on the same topic.*
* New England Journal of Medicine 2000; (June 22, 2000) 342:1887-92.
18
Assigned vs. Assumed Causality
• Assigned causality: RCT
•
–Known confounders are eliminated
Assumed causality: PBE-CPI
–No added confounders cause the significant
association to disappear
–A change in outcomes follows a change in
treatment as predicted by the PBE-CPI model
–Repeated studies on the same topic yield the
same findings
19
PBE-CPI and RCT
RCT
Progenitor of
RCTs
Practice effects
of RCT results
PBE-CPI
20
Experimental Designs
Experimental designs are rarely
feasible to evaluate complex
interventions in the real world.
21
PBE-CPI
RCT
•Hypothesis many and vague
•Hypothesis clear
•Alternatives not discrete
•Alternatives discrete
•Local knowledge
•Not depend on local
contributes
knowledge
22
PBE-CPI
RCT
•Confounders affect
•Confounders not
outcomes and are
interesting
interesting
•Effects large
•Effects small
Dr. Don Berwick, HSR, April 2005
23
PBE-CPI offers opportunities to
• Focus on clinical reality
• Discover best practices
• Test clinical questions and intermediate
outcomes
• Gain some control over variable patient,
process, and outcome data
24
Practice-based Evidence for Clinical Practice Improvement
(PBE-CPI)
• Connects outcomes with detailed process
steps
• Adjusts for severity of illness
25
Criteria for Selection
of a Severity Indexing System
• Disease-specific
• Independent of treatments
• Comprehensive (i.e., all diseases)
• Clinically credible
• Able to measure severity at multiple points in the
care process
• Statistically valid in explaining costs/outcomes
26
Comprehensive Severity Index
®
(CSI )
Severity Systems
Diagnostic/Procedure Based
Systems
Physiologic/Clinically Based
Systems
•AIM by Iameter
•Apache
•Disease Staging by MedStat
•Atlas by Mediqual
•APR DRGs by 3m
•Patient Management
Categories
CSI®
27
Comprehensive Severity Index CSI®

Over 2,200 individual criteria subdivided into more than
5,500 disease-specific groups

No treatments used as criteria

Computes disease-specific and overall severity levels on a
scale of 0-4 and continuous

Fixed times for inpatient reviews
- Admission review--first 24 hours
- Maximum review--any time during stay
- Discharge review--last 24 hours
28
Pneumonia Criteria Set
480.0-486; 506.3; 507.0-507.1; 516.8; 517.1; 518.3; 518.5; 668.00-668.04; 997.3; 112.4; 136.3; 055.1
CATEGORY
1
2
3
Cardiovascular
pulse rate 51-100; ST
segment changes-EKG;
systolic BP  90mmHg
pulse rate 100-129;
41-50; PACs, PAT,
PVCs-EKG;
systolic BP 80-89mmHg
pulse rate  130; 31-40;
systolic BP 61-79mmHg
pulse rate 30;
asystole, VT, VF,
V flutter;
systolic BP 60 mmHg
Fever
96.8-100.4 and/or chills
100.5-102.0 oral;
94.0-96.7
102.1-103.9; 90.1-93.9
and/or rigors
 104.0
90.0
Labs
ABGs
pH 7.35-7.45
pH >7.46 7.25-7.34
pH 7.10-7.24
pH 7.09;
pO2 51-60mmHg
pO2  50mmHg
WBC 11.1-20.0K/cu mm;
2.4-4.4K/cu mm;
bands 10-20%
WBC 20.1-30.0K/cu mm;
1.0-2.3K/cu mm;
bands 21-40%
WBC 30.1K/cu mm;
1.0K/cu mm;
bands 40%
chronic confusion
acute confusion
unresponsive
9-11
6-8
 5
Radiology Chest
X-Ray or CT
Scan
infiltrate and/or
consolidation in 1
lobe; pleural effusion
infiltrate and/or
consolidation in >1 but
3 lobes;
infiltrate and/or
consolidation in >3
lobes; cavitation or
lung necrosis
Respiratory
dyspnea on exertion;
stridor; rales 50%/3
lobes; decreased breath
sounds 50%/3 lobes;
positive for fremitus;
stridor
hemoptysis NOS;
blood tinged or purulent
or frothy sputum
cyanosis present
dyspnea at rest; rales
>50%/ 3 lobes;
decreased breath
sounds >50%/ 3 lobes
apnea
absent breath sounds
>50%/ 3 lobes
pO2 61mmHg
Hematology
WBC 4.5-11.0K/cu mm;
bands <10%;
Neuro Status
Lowest Glasgow
coma score
 12
white, thin, mucoid
sputum
Copyright
1998. Susan D. Horn. All rights reserved. Do not quote, copy or cite without permission.
4
 frank hemoptysis
29
How Does PBE-CPI Differ?
• Severity adjustment methodology
• Three-dimensional, comprehensive
measurement framework: patient, process, and
outcomes
• Balance of rigorous science with a pragmatic
operational focus
30
Nursing Home Study (NPULS)
1996-1997
•
6 long-term care provider organizations
•
109 facilities
•
2,490 residents studied
•
1,343 residents with pressure ulcer; 1,147 at risk
•
70% female, 30% male
•
Average age = 79.8 years
Funded by Ross Products Division, Abbott Laboratories
31
NPULS Outcomes
• Developed pressure ulcers
• Healed pressure ulcers
• Hospitalization
• Systemic infections
32
Effects of Nutritional Support
in Long Term Care
N
Pressure
Ulcer
Develop Rate
134
21.6%
210
23.8%
Fluid Order
396
25.0%
Snacks, House Shakes
No Nutritional Risk -No Nutritional Treatment
At Nutritional Risk -No Nutritional Support
403
27.3%
195
27.2%
323
35.6%
Nutritional
Treatment Strategies
Oral Supplement / Standard Medical
Nutritional
Enteral Formula
33
Bladder Incontinence Management
in Long Term Care
Treatments
N
Incontinent-Use one or more of following treatments: 1,441
Briefs, disposable
501
Toileting program
549
Briefs, reusable
118
Topical Treatment
1,159
Bed pads, disposable
193
Bed pads, reusable
221
Use of catheter
195
Continent-No incontinence treatment
209
PU Develop Rate
34.2%
23.6%
23.9%
26.3%
29.1%
29.5%
32.1%
51.3%
26.3%
34
Long-Term Care Residents with Agitation in Dementia
Recommended Practice
• Use fewest number of medications possible
(OBRA 1987)
• Minimize use of benzodiazepines
• Use atypical over typical antipsychotics
• Use SSRIs over tertiary amine antidepressants
• Avoid combination therapy
35
Medications from NPULS Study
Optimal Medications
Dementia & Agitation n = 803
No Psych Meds
Anti-psychotics
Anti-depressants
Anti-anxiety
32.5%
31.5%
34.6%
34.9%
Combinations in 42% of treated residents
36
Medication Use and Outcomes for Elderly
with Dementia with Agitation
Medication
% Hospital +
ER
% Restraints
% Pressure
Ulcers
No Psych Medications
20.0
19.9
37.2
Monotherapy
17.2
24.0
24.0**
12.3*
12.6**
SSRI + Antipsychotic
9.9**
Monotherapy includes antipsychotic only, antidepressant only, or antianxiety only
SSRI + antipsychotic medications concurrently.
*p<.05 **p<.01
Horn, Drug Benefit Trends 2003; 15 (Supplement 1, December): 12-18
37
Long Term Care CPI Results
Outcome: Develop Pressure Ulcer
General
Assessment
+ Age  85
Incontinence
Interventions
Pressure Relief
Staffing
Interventions
Interventions
+ Mechanical devices for
the containment of urine
(catheters)
+Static pressure
reduction: protective
device
+ History of PU
- Disposable briefs
+ Dependency in
>= 7 ADLs
- Toileting Program
+Positioning: protective
device
+ Male
+ Severity of Illness
+ Diabetes
- RN hours per resident
day >=0 .25
- CNA hours per resident
day >= 2
-LPN hours per resident
day >=0.75
Medications
+ History of tobacco use
- SSRI + Antipsychotic
38
Long Term Care CPI Results
Outcome: Develop Pressure Ulcer
Nutritional
Assessment
+ Dehydration signs and
symptoms: low systolic blood
pressure, high temperature,
dysphagia, high BUN, diarrhea,
dehydration
+ Weight Loss: >=5% in last 30
days or >=10% in last 180 days
Nutritional
Interventions
- Fluid Order
- Nutritional Supplements
• standard medical
- Enteral Supplements
• disease-specific
• high calorie/high
protein
Horn et al, J. Amer Geriatr Soc March 2004
39
AHRQ Partnership for Quality
Real-time Optimal Care Plans for Nursing Home QI
 Integrate sustainable quality improvement into daily operations
• Incorporate practice-based evidence for pressure ulcer prevention
• Integrate into daily work versus ‘add-on’ project
 Focus on critical data elements and information flow
•
•
•
•
Eliminate redundant documentation
Reduce paperwork and streamline documentation
Improve accuracy of information
Improve communication among trans-disciplinary care teams
 Translate documentation into data & data into trans-disciplinary
clinical reports
40
Results
 Decrease Pressure Ulcer Development on average 33%
 Increase Adherence to Best Practices
 Increase Staff Accountability and Satisfaction
–
–
–
Inclusion of front-line workers in QI efforts
Comprehensive documentation at point of care more complete
Communication among care team improved
 Reduce Inefficiencies
–
–
–
–
# documentation forms for CNAs decreased 50% or more
CNA time looking for documentation book decreased
Time to compile reports for State Regulators and MDS decreased
Time for Wound RN to summarize and report data decreased
 Improve State Survey Process
 Establish a foundation for EMR
41
Impact On Pressure Ulcer QMs
The combined facilities’ average shows an overall reduction of 33% in the QM % of high risk
residents with pressure ulcer from pre-implementation to initial post-implementation time periods
Q4 03 (Pre-Implementation) to Q3 05 (Post-Intervention Review) Combined Facilities
Average
20.0
% High Risk Residents
National Norm
15.0
Combined Facilities
10.0
5.0
0.0
Q4 03 – Q3 05% Change = - 33%
Q3 03
Q4 03
Q1 04
Q2 04
Q3 04
Q4 04
Q1 05
Q2 05
Q3 05
Facilities Average
14.0
13.0
12.9
10.6
9.6
9.4
12.0
9.1
8.7
National Norm
14.0
14.0
14.0
13.0
13.0
13.0
14.0
14.0
13.0
Source: CMS Nursing Home Compare; Facility QM data reports
42
DEVELOP PRESSURE ULCER by RN Time
40%
35%
38.1%
30%
31.8%
25%
25.1%
20%
15%
10%
9.4%
5%
0%
<10 min
10 - <20 min
20 - <30 min
30 - <40 min
RN Time Per Resident Per Day
Chi-Square (6 df) = 50.86, p<.0001, n=1,376
43
Value of Nurses
Logistic Regression: DEVELOP PU-- RN/LPN/CNA Time and Other Effects
Parameter
ADLs_78
CSI Severity
MDS PU_hx
Wt loss
Oral_eat prob
Catheter
Entcalpr
Ent_dis
Fluid order
RN 10-20m
RN 20-30m
RN 30-40m
CNA >2.25h
LPN >=45m
Estimate
0.28
0.01
0.75
0.34
0.39
0.78
-0.55
-0.98
-0.43
-0.41
-0.62
-1.86
-0.64
-0.64
Chi-Square
Pr > ChiSq
4.68
18.19
15.00
6.04
9.33
16.98
6.77
6.00
8.43
0.0305
<.0001
0.0001
0.0140
0.0023
<.0001
0.0093
0.0143
0.0037
7.84
13.12
42.82
5.76
8.74
0.0051
0.0003
<.0001
0.0164
0.0031
C = 0.727
44
HOSPITALIZATION by RN Time
20%
18%
16%
14%
12%
10%
8%
6%
4%
2%
0%
18.4%
11.1%
9.6%
6.1%
<10 min
10 - <20 min
20 - <30 min
30 - <40 min
RN Time Per Resident Per Day
Chi-Square (4 df) = 35.17, p<.0001, n=1,376
45
Effects of RN Time
Horn SD, et el. Assoc. between registered nurse staffing time and outcomes of long-stay nursing home
residents. Amer J Nursing (Nov 2005 to appear)
RN time of 30-40 min/resident/day
is associated with
•
•
•
•
Fewer UTIs
Fewer catheterizations
Less weight loss
Less decline in ADLs
• More nutrition supplements
46
Societal Perspective
Economic Value of Nurses
Dorr DA, Horn SD, Smout RJ. Cost analysis of nursing home registered nurse staffing times. J American Geriatrics Society 2005; 53:656-661
Cost/Benefit Analysis of More RN Time
$ Per 100 at-risk residents per year (FY2001 dollars)
Savings in avoided PU treatment cost
Cost of additional 30 min
RN care per resident day
$242,426
Savings in avoided hospitalizations
$518,627
$472,814
Savings in avoided UTI costs
30,882
Net Savings $319,121
Assumptions: $1,727 wtd avg to treat PU across stages,
$8,523 avg for Medicare hospitalization, $53,900K RN salary & FB/yr
47
Post-Stroke Rehabilitation Study
STUDY QUESTIONS
1. Which patient characteristics are associated
with improved post-stroke outcomes?
2. Controlling for patient characteristics,
which treatment interventions or
combinations are associated with improved
outcomes?
48
Post-Stroke Rehabilitation Study
STUDY QUESTIONS
3. What is the optimal intensity and duration of
various post-stroke treatment interventions?
4. What staffing characteristics are associated with
better outcomes?
49
Post-Stroke Rehabilitation Study
Project Clinical Team
•
•
•
•
•
Physicians
Nurses
Social Workers
Psychologists
Physical Therapists
•
•
•
Occupational Therapists
Recreation Therapists
Speech/Language
Pathologists
50
Post-Stroke Rehabilitation Study
Patient Characteristics
1,161 U.S. Patients
52% Male: 58% White, 26% Black
66.0 Mean age (18.6 - 95.5 yrs)
51
Post-Stroke Rehabilitation Study
Examples of OUTCOME VARIABLES
• Change in FIM score
• Deep vein thrombosis
• Length of rehab stay
• Major bleeding
• Discharge disposition
• Pulmonary embolism
• Contracture
• Pressure ulcer
• Death
• Pneumonia
52
Post-Stroke Rehabilitation Study
Examples of PROCESS VARIABLES
•
•
•
•
•
Medications
Nutritional process
Pain management
Time to first rehab
Intensity, frequency, and
duration of PT
interventions
• Intensity, frequency, and
duration of OT
interventions
• Intensity, frequency, and
duration of SLP
interventions
• Other therapy
interventions and dosage
53
Post-Stroke Rehabilitation Study
DEVELOP WORKABLE INSTRUMENTS
MULTIPLE AXES
•
FUNCTIONAL ACTIVITIES
Task-specific approach
•
THERAPY INTERVENTIONS
Organized around core functional activities
•
TIME SPENT
Interventions in activities, formal assessment, home evaluation54
Post-Stroke Rehabilitation Study
PT FUNCTIONAL ACTIVITIES
•
•
•
•
•
•
Pre-Functional
Bed Mobility
Sitting
Transfers
Sit-to-Stand
Wheelchair Mobility
•
•
•
•
•
Pre-gait
Gait
Advanced Gait
Community Mobility
Intervention not related to
functional activity
55
Post-Stroke Physical Therapy Form
P hysical T herapy R ehabilitation A ctivities
6649
Patient ID:
Date of Therapy Session:
S a m p l e
/
/
Time session begins:
Therapist:
:
INTERVENTION CODES
D uration of A ctivity:
Neuromuscular Interventions:
Enter in 5 minute increments.
01. Balance training
02. Postural awareness
Pre-Functional Activity
03. M otor learning
04. PNF
Bed M obility
05. NDT
06. Gait with body weight support
07. Involved upper extremity addressed
Sitting
08. Constrained induced movement therapy
Musculoskeletal Interventions:
09. Strengthening
Transfers
10. M obilization
11. PROM /Stretching
12. M anual Therapy
Sit-to-Stand
13. M otor Control
Cardiopulmonary Intervention:
14. Breathing
W heelchair M obility
15. Aerobic/Conditioning exercises
Cognitive/Perceptual/Sensory Interventions:
16. Cognitive training
Pre-gait
17. Perceptual training
18. Visual training
G ait
19. Sensory training
Education Interventions:
20. Patient
Advanced Gait
21. Family/Caregiver
22. Staff
Equipment Interventions:
Community M obility
23. Prescription/Selection
24. Application
25. Fabrication
Intervention not related
26. Ordering
Modality Interventions:
27. Electrical Stimulation
28. Biofeedback
29. Ultrasound
Pet Therapy:
30. Use of dog
31. Use of other animal
Assistive Device:
32. Ankle dorsi flex assist
33. Cane - Large base
34. Cane - Small base
35. Cane - Straight
36. Crutches - Axillary
37. Crutches - Forearm
38. Crutches - Small base forearm
39. Dowel
40. Grocery cart
41. Hemirail
42. Ironing board
43. KAFO
44. Lite gait
45.
46.
M irror
Parallel bars
Interventions:
Enter one intervention code per group of boxes.
to functional activity
Intervention #2 not related
to functional activity
Platform (parallel bars
or FW W )
48. Standing frame
49. Steps (various heights)
50. Step ladder
51. Swedish knee cage
52. Swiss ball
53. Tray table
54. W alker - FW W
55. W alker - Hemiwalker
56. W alker - Rising Star
57. W alker - Standard
58. W heelchair
O ther:
59.
C o-T reat:
47.
Area Involved/non-functional:
60. Upper Extremity
61. Lower Extremity
62. Trunk
63. Head/Neck
Disciplines:
No. of minutes:
Patient A ssessm ent:
Formal Assessment (initial, re-evaluation, discharge):
minutes
Home Evaluation:
minutes
W ork Site Evaluation:
minutes
Physical T herapy T im e:
Physical Therapist
PT Assistant
PT Aide/Tech
PT Student
minutes
minutes
minutes
minutes
G roup Physical T herapy T im e:
PT Group/Dovetail:
minutes
Enter the number of each that participated in the Group PT:
Patients
Therapists
Assistants
Aides/Techs
Students
56
Predicting Outcomes
Severe Stroke - CMGs 112+114
Outcome
LOS
Patient Variables alone
R2 = .39
Discharge
Home
c = .70
Add Total PT&OT Time/LOS
R2 = .39
c = .70
Instead add PT&OT - Specific
Activity Time/LOS
R2 = .61
c = .85
57
Predicting Outcomes
Severe Stroke - CMGs 112+114
Outcome
Patient Variables alone
Discharge
Total FIM
R2 = .64
Discharge
Motor FIM
R2 = .58
Add Total PT&OT Time/LOS
R2 = .64
R2 = .59
Instead add PT&OT - Specific
Activity Time/LOS
R2 = .78
R2 = .76
58
Post-Stroke Rehabilitation Outcomes Project
Outcome: Increase in Motor FIM CMGs 112+114
General Care
General
Assessment
– Age
– Adm Severity of Illness
+ No dysphagia
+ Chronic confusion
General
Interventions
– Days onset to rehab
+ Rehab length of stay
+ Enteral feeding
OT
Interventions
PT
Interventions
+ Home mgt time in 1st 3 hrs
+ Gait time in 1st 3 hrs
+ Comm integr in 1st 3 hrs
+ Adv gait time in 1st 3 hrs
– Bed mobility time in 1st 3
hrs
– Bed mobility time in 1st 3
hrs
+ Monoplegia or Normal
Medications
– Wheelchair time in 1st 3
hrs
– Old SSRIs
– Anti-Parkinsons
– Modafinil
59
Post-Stroke Rehabilitation Study
12 papers published in
Supplement to Archives of Physical Medicine and
Rehabilitation
December 2005
Opening the “Black Box” of Stroke Rehabilitation
And What It Means for Rehabilitation Research
8 additional papers in other journals
60
CPI Model - Length of Stay
Bowel surgery
PROCESS OF CARE
Assessment
+ Admission
CSI
(severity)
+ COPD
Surgery
+ Bowel Prep- GoLightly
Pain
Management
Nutrition
+ Post-op TPN
Wound
Management
PT / RT
Discharge
+ Discharge
to SNF
+ Skin to skin
+ Drain - JP
+ Drain - Penrose
61
Abdominal Surgery Nutrition Study
Disease CSI Score
Intervention Subgroup
N
Mean
Early & Sufficient
Not Early & Not Sufficient
Not Early & Sufficient
Early & Not Sufficient
42
61
25
55
50.7
49.3
48.8
41.8
62
Abdominal Surgery Nutrition Study
Nutrition CSI Score (Deaths and Transfers Removed)
Intervention Subgroup
N
Mean
Early & Sufficient
Not Early & Not Sufficient
Not Early & Sufficient
Early & Not Sufficient
29
47
21
43
9.8
7.7
8.0
7.7
63
Abdominal Surgery Nutrition Study
Length of Stay (Deaths and Transfers Removed)
Intervention Subgroup
N
Mean
Not Early & Not Sufficient
Not Early & Sufficient
Early & Not Sufficient
Early & Sufficient
47
21
43
29
14.8
14.6
13.3
11.9
64
Abdominal Surgery Nutrition Study
Total Charges (Deaths & Transfers Removed)
Intervention Subgroup
N
Mean
Not Early & Sufficient
Not Early & Not Sufficient
Early & Not Sufficient
Early & Sufficient
13
35
35
20
39,883
38,578
36,542
34,602
Neumayer LA, et al. Journal of Surgical Research 2001;95:1:73-77
65
Intended and Unintended
Consequences of HMO
Cost-Containment Strategies
Results from the Managed Care Outcomes Project
American Journal of Managed Care
March 1996
Main Study Question
“When one looks across multiple managed care
organizations at a year’s worth of actual data on
the care of thousands of typical patients treated by
their regular doctors, how is the amount of health
care services used associated with costcontainment efforts by the HMO?”
67
Managed Care Outcomes Project
Patient Population:
Nearly 13,000 patients
were included in the
study:
•1,309 - 3,938 patients
for each disease group
•1,876 - 2,663 patients
studied at each HMO site
This represented more
than:
•99,000 office visits
•480 emergency room visits
•1,000 hospitalizations
•240,000 prescriptions
Length of study period:
•One year
68
Managed Care Outcomes Project
Study controlled for patient, cost-containment practice, and HMO
site variables
Patient variables
•Severity of patient
illness
•Age and gender
•Time in study
•Number of
physicians seen by
patient
Cost-Containment
Practice Variables
•Second-opinion
requirements
•Strictness of site’s
gatekeeper
•Strictness of case mgt.
•Drug and visit co-pays
•Restrictions of
formulary
•Extent of generic
drug use
HMO Site Variables
•Physician payment
method
•HMO profit status
•Geographical
location
69
Managed Care Outcomes Project
Findings
With increased formulary restrictiveness, the study found:
• More patient visits to physicians
• More emergency room visits
• More hospitalizations
• Greater estimated cost of prescriptions per year
• Greater number of prescriptions per year
70
Managed Care Outcomes Project
Number of Prescriptions Per
Patient Per Year
Number of Prescriptions for Asthma
50
48.4
45
40
39.1
35
35.8
30
29 28.3
25
24
26.3
20
16.9
15
10
34.6
8.4
11.4 10.9
18
% Formulary
Limitation
46.1
Site 1
(0%)
Site 2
(65.2%)
Site 3
(65.2%)
Site 4
(75%)
Site 5
(76.1%)
8.8
5
0
Low Severity
Medium Severity
High Severity
71
Managed Care Outcomes Project
Cost of Prescriptions
Per Patient Per Year
Cost of Prescriptions for Arthritis
%
Formulary
Limitation
$1,400
1285
1236
$1,200
$1,000
1010
$800
858
726
$600
501
$400
$200
587 604
332 344
132 159
628
486
245
Site 1
(0%)
Site 2
(42.5%)
Site 3
(47.5%)
Site 4
(55%)
Site 5
(62.5%)
$0
Low Severity
Medium Severity
High Severity
72
Managed Care Outcomes Project
Number of Visits Per Patient
Per Year
Number of Visits for Ulcers
12
11.7 11.9
11.0
10
9.5
8.4
8
6.8
6
6.2
5.9
5.0 5.0
4
3.0 3.3
2
1.9
3.9
3.6 3.8
2.3
2.9
% Formulary
Limitation
Site 1
(0%)
Site 2
(12.5%)
Site 3
(12.5%)
Site 4
(25%)
Site 5
(25%)
Site 6
(37.5%)
0
Low Severity
Medium Severity
High Severity
73
Cost-Containment Measure
Formulary Limitation
RxCount
+0.280 (0.15)
+0.270(0.79)
+1.460 ***
Epigastric
Pain/ Ulcer
n=1511
+1.600 **
DisGpRxCt
+0.270 (0.19)
+2.390
+1.125 ***
+1.000 (0.09)
+0.550 (0.17)
RxCost
+0.780
*
+2.590(0.16)
+1.420 *
+2.280 *
+0.420 (0.49)
DisGpRxCst
+2.840
***
+2.650(0.84)
+4.800 ***
+3.790 **
+0.530 (0.55)
Visits
+0.320 (0.35)
-3.740 (0.09)
+0.740(0.39)
+2.130
ED Visits
+0.125 ***
-1.280
***
+0.290 ***
+0.287 ***
+0.289 ***
Hosp Adms
+0.419
-2.350
***
+1.020 ***
+0.490
+0.431 **
Outcomes
Hypertension
n=3477
***
Otitis Media
n=3862
Disease Group
Arthritis
n=2632
**
**
***
Asthma
n=1290
+0.800 *
+1.110
(0.31)
Shaded cells represent unexpected direction
*** indicates p <0.001
** indicates 0.001<p<0.01
* indicates 0.01<p<0.05
+ indicates increased utilization
- indicates decreased utilization
74
Summary of Findings
Curtailing access to medications via
cost-control mechanisms can adversely
affect other healthcare utilization:
• Additional office visits for dose titration/
monitoring
• ER/hospital visits
• Concomitant medications
and increase total healthcare costs.
75
Limitations of PBE-CPI Studies
•
Do not scientifically
prove causality of
underlying relationships
- “Association is not
causation.”
•
Incidence and type of test
ordering and availability
of information is not
uniform across sites
•
Accuracy and
completeness of current
documentation
•
Complexity of analysis
•
Labor-intensive manual
data abstraction
76
Limitations of CPI Studies
•
The strength of an observational study depends
on the study’s ability to control for patient
differences that would otherwise be addressed
through randomization. There is always the
chance that some unknown critical variable
may have been overlooked
77
Comprehensive Approaches to Care Save
30% to 50% of Health Costs
Improve/Standardize
Process Factors
•Management Strategies
•Interventions
•Medications
Control for:
Patient Factors
•Psychosocial/demographic Factors
•Disease(s)
•Severity of Disease(s)
Measure:
Outcomes
•Clinical
•Health Status
•Cost/LOS/Encounters
› physiologic signs and symptoms
•Multiple Points in Time
78