Quality and Technology
N9205
Oct. 17, 2000
Assessing the quality of
care or services

Was the right thing done?

Was it done done right?

Did it yield the right results?
Columbia University
School of Nursing
M6920, Fall, 2000
Donabedian framework

Structure/input
• capital investment
• staffing
• relationships

Process
• content
• sequence

Outcome
Columbia University
School of Nursing
M6920, Fall, 2000
Assessing quality
Person seeks care
Provider
Primary
Preventio
n
Outreach
Activities
Case Finding
Screening
Diagnosis
Diagnosis
Management
Evaluation of
Presenting
History,Physical
Complaint
Other Diagnostic
Procedures
Patient Education
Referrals
Therapy
Monitoring
Followup
Columbia University
School of Nursing
Desired effects
Office of Technology
Assessment, 1988
M6920, Fall, 2000
Critical issues

Selection of domain

Selection of measures

Identification of data source
Columbia University
School of Nursing
M6920, Fall, 2000
A special case :
technology assessment



Generally includes "machines"
Would also cover
pharmaceuticals?
Other possible "hidden"
technologies
• scheduling
• staffing patterns
• access systems
Columbia University
School of Nursing
M6920, Fall, 2000
Use of technologies?
• clinical excellence
• technological preeminence
• profit maximization
• in a fee-for-service system
• in a capitated or global budget
system
Columbia University
School of Nursing
M6920, Fall, 2000
Assessing technology




Is this safe?
Efficacious?
Effective?
Efficient?
• speed of outcome
• quality of outcome
• cost of outcome
Columbia University
School of Nursing
M6920, Fall, 2000
Renal dialysis




introduction-late 60's/early 70's
use of screening committees
ESRD Medicare policy
US compared to GB
Columbia University
School of Nursing
M6920, Fall, 2000
Heart transplant

early 70’s
• everybody try one
• few
centers
persist
procedure

with
mid 80's
• introduction of anti-rejection drugs
Columbia University
School of Nursing
M6920, Fall, 2000
CABG surgery


what are the trade-offs in
quality of life?
what about skill/competence
• limitations on facilities performing
in NY state
Columbia University
School of Nursing
M6920, Fall, 2000
BC/BS Technology
Assessment Agenda for 1997

Cost Effectiveness Analyses
• Cervical Cancer Rescreening
Methods
• Electron beam computed
tomography for CHD
Columbia University
School of Nursing
M6920, Fall, 2000

Clinical Effectiveness Analyses
•
•
•
•
•
fetal febrnectin
functional sterotactic radiosurgery
genetic testing for colon cancer
neurostimulation for tremor
non-coronary intravascular
ultrasound
Columbia University
School of Nursing
M6920, Fall, 2000
Critical policy problems

who is "disinterested observer"
to conduct assessment?
• use of consensus panels
(NIH/RAND models)
• one discipline? inclusion of "doers"?
• OTA elimination;
sizing
AHCPR
defining "experimental"?
 appeal to the courts
Columbia University
down-

School of Nursing
M6920, Fall, 2000
Critical research questions


use/role of public opinion
professional opinion and
practice
• too rapid adoption
• delayed adoption


financial incentives to use/not
use
short and long-term outcomes
Columbia University
School of Nursing
M6920, Fall, 2000
Hamilton & HO



Objective: understand the
relationship between volume
and quality
Reason: Is it “practice makes
perfect” or selective referral
patterns?
Method: regression analysis of
3 years of data
Columbia University
School of Nursing
M6920, Fall, 2000
Hamilton & Ho, Cont.



Result: negative relationship
between volume and length of stay
But: fluctuations in volume had no
effect on LOS or mortality
Conclusion: high volume = high
quality for reasons other than
practice makes perfect
Columbia University
School of Nursing
M6920, Fall, 2000
Meehan et al.

PRO study to
• assess quality of care for
Medicare patients with pneumonia
• determine whether process of
care performance is associated
with lower mortality

multi-center retrospective
cohort study (14,069 patients;
3555 hospitals in US)
Columbia University
School of Nursing
M6920, Fall, 2000
Mehan et al, cont.

Definition of process of care
• time from arrival to antibiotic
administration
• blood culture before initial antibiotics
• blood culture within 24 hours of hospital
arrival
• oxygenation assessment within 24
hours
Columbia University
School of Nursing
M6920, Fall, 2000
Mehan et al, cont.

Sample Selection
• decision on ICD-9-CM codes
• exclusion criteria (primarily
clinical confounders such as HIV)

Data collection
• training of medical records
abstractors
Columbia University
School of Nursing
M6920, Fall, 2000
Mehan et al, cont.


1/4 of elderly patients do not receive
antibiotics until at least 8 hrs post
admission; doing so is associated with
15% lower odds of mortality
1/3 of elderly patients do not have a
blood culture drawn within 24 hours;
doing so associated with 10% lower
odds of mortality
Columbia University
School of Nursing
M6920, Fall, 2000
Mehan et al, cont.


high rate of unconfirmed
pneumonia diagnoses when
clinical criteria were included
Intriguing query: did presence
of DNR orders limit therapy for
some patients?
Columbia University
School of Nursing
M6920, Fall, 2000
Mezey et al


Cross sectional telephone survey
Sample of 1016 from 1452 calls
•
•
•
•

over 18
English or Spanish speaking
medical or surgical admission
no nursing home pre or post stay
Instrument?
Columbia University
School of Nursing
M6920, Fall, 2000
Mezey et al


Forced choice answers?
Findings
• Racial, language and economic
differences
• Level of education most significant
Columbia University
School of Nursing
M6920, Fall, 2000
Zinn et al


Objective: identify contextual
attributes that influence TQM
adoption
Data: survey of licensed nursing
home administrators,
certification files and ARF
Columbia University
School of Nursing
M6920, Fall, 2000
Zinn et al, Variables
Variable
Definition
Source
Nsg. Home has adopted
TQM survey
Perceived competition
Admin. Perception
TQM survey
Herfindal index
Nsg home market concentration
MMACS
Excess capacity
Average # empty beds/county
MMACS
Hospital-based substitutes
# hospitals providing LTC
ARF
Nursing home size
# beds in facility
MMACS
M’care market penetration
Proportion of discharges
Medicare
ARF
HMO membership
Proportion of residents in HMO
ARF
Proportion Medicare
Proportion of NH residents with
Medicare coverage
MMACS
Per capita income (log)
Average per capita income in
county
ARF
Dependent Variabl e
TQM Adoption
Independent Variables
Columbia University
School of Nursing
M6920, Fall, 2000
Zinn et al, cont.



1: more competitive markets lead to
adoption--Partial support
3: facilities in areas with higher
M’care discharges more likely to
adopt--support
4: facilities in areas with greater
HMO penetration are more likely to
adopt--significant support
Columbia University
School of Nursing
M6920, Fall, 2000
Zinn et al, cont


2: Larger facilities are more likely
to adopt--no support
5: Facilities with grated proportion
of M’care recipients in total
census are more likely to adopt-no support
Columbia University
School of Nursing
M6920, Fall, 2000
Keeler et al



How can a good case mix
method be developed?
Combination of birth certificate
and hospital discharge data
Retrospective model building
effort
Columbia University
School of Nursing
M6920, Fall, 2000
Keeler et al

Factors ruled out
• race and management decisions

Factors had to have
•
•
•
•
consistent coding practices
unequivocally risk not outcome
prevalence consistent with clinician view
recorded variable associated with
outcomes
Columbia University
School of Nursing
M6920, Fall, 2000
Keeler et al




Merged data better than only
one source
Simple model explains 30% of
variance among hospitals
Best model explains 37%
Is the remainder practitioner
choice???
Columbia University
School of Nursing
M6920, Fall, 2000