Quality, Patient Safety &
Paying for Performance
Call for Papers
Leaders & Outliers: How Should We Measure
Quality & Safety?
Chair: Carol Haraden, Institute for Healthcare Improvement
Sunday, June 26 • 8:30 am – 10:00 am
●Development of the Medicare Patient Safety Monitoring
System
Susan L. Abend, M.D., FACP, David R. Hunt, M.D., FACS,
Nancy R. Verzier, MSN, RN, Nancy Safer, RN, Janet P. Tate,
MPH, Gaston Mbateng, Ph.D.
Presented By: Susan L. Abend, M.D., FACP, Qualidigm, 100
Roscommon Drive, Middletown, CT 06457; Tel: (860)6326395; Fax: (860)632-6326; Email: sabend@pol.net
Research Objective: Since the Harvard Medical Practice
Study 15 years ago, numerous studies have assessed the
magnitude of patient injuries occurring in US hospitals, and
have estimated the impact of adverse events on health care
costs and patient longevity. Assessing the true magnitude and
impact of these unintended events has been complicated by
variations in study design, event definitions, and data sources.
The larger studies use clinician opinion to confirm the
presence of an adverse event, (a method demonstrated to
have low inter-rater reliability, with correlation coefficients of
0.24-0.34) or use administrative data to identify diagnoses
presumed to be unintended consequences of care. The
Medicare Patient Safety Monitoring System (MPSMS),
developed by CMS under the leadership of the DHHS Patient
Safety Task Force, is a cost-effective, standardized surveillance
method for assessing hospital-related adverse event rates over
time in the Medicare population. The system will be used to
provide baseline information for CMS quality improvement
initiatives such as the Surgical Care Improvement Project, and
to provide annual data to the National Quality Report. The
MPSMS can also be used as a tool for hospitals to measure
and test changes that create safer therapeutic environments.
Study Design: The MPSMS team, supported by Qualidigm
(the Connecticut QIO), develops standard definitions of
hospital-related adverse events in close collaboration with a
Technical Expert Panel (TEP) comprised of major public and
private stakeholders, academic experts, and representatives of
AHRQ, CDC, FDA and VA. Data elements present in medical
records and/or Medicare administrative data, indicating the
presence of the adverse event and a closely-associated,
antecedent healthcare exposure, are then identified. Adverse
events are chosen from AHRQ- developed patient safety
measures, FDA reporting systems, CDC recommendations,
and information from peer-reviewed literature. Events must
meet the following criteria: 1) having easily-acquired data
elements, 2) clear association with specific healthcare
exposure, 3) high frequency, 4) high likelihood of morbidity or
mortality, 5) and potential for preventability, are chosen for
development. Algorithms are then created that indicate the
presence or absence of a specific event and antecedent
healthcare exposure. Algorithms are rigorously tested in an
alpha phase for data element findability and ambiguity, in a
beta phase for accuracy and interrater reliability, and are then
reviewed for face-validity by clinicians from across the country.
Population Studied: A random sample of 25,000- 40,000
Medicare fee-for-service patients discharged from general
acute-care hospitals are studied annually. The data sources
are charts reviewed for Medicare’s Hospital Payment
Monitoring Program, as well as related Medicare Part A and B
claims data. Eighty charts per month are reabstracted to
check reliability.
Principal Findings: The kappa statistic, (a measure of
reliability that corrects for chance agreement), from variables
collected in production, year 1, was 0.94 (95% CI 0.92-0.96).
Accuracy of data abstraction against gold standard records
was 99.2 + 0.1%. Mean cost of abstracting 10 production
measures was $74/record.
Conclusions: The MPSMS, the product of a robust publicprivate collaboration, is a reliable, cost-effective tool for
measuring adverse events in hospitalized patients, and is the
first system able to provide rates of specific, U.S.-hospitalrelated adverse events over time.
Implications for Policy, Delivery, or Practice: Because it is
standardized and has excellent inter-rater reliability, the
MPSMS can be used as a tool by hospitals to assess
performance and test process redesign initiatives. While
currently designed to find data from conventional medical
charts, its data elements can be easily organized into modules
within Electronic Health Records. It can be used to assess
changes retrospectively, or data elements can be noted
prospectively to trap errors or change processes leading to an
adverse event.
Primary Funding Source: CMS
●Comparative Rankings of Hospital Quality – Does the
Data Source Matter?
Anne Elixhauser, Ph.D., Bernard Friedman, Ph.D.
Presented By: Anne Elixhauser, Ph.D., Senior Research
Scientist, Center for Delivery, Organization, and Markets,
Agency for Healthcare Research and Quality, 540 Gaither
Road, Rockville, MD 20850; Tel: (301)427-1411; Fax: (301)4271430; Email: aelixhau@ahrq.gov
Research Objective: To assess the extent to which
information on Medicare discharges can be extrapolated to
the general inpatient population when comparing hospital
quality.
Study Design: Cross-sectional analysis using hospital
administrative data from the 2001 Healthcare Cost and
Utilization Project (HCUP) Nationwide Inpatient Sample
(NIS), a sample of 986 hospitals from 33 states. Measures
were drawn from the AHRQ Quality Indicators (QIs), which
use hospital administrative data: 15 Patient Safety Indicators
(PSIs) and 12 inhospital mortality measures. Risk adjustment
for PSIs used gender, age, comorbidities, and modified DRGs
(collapsed across the comorbidity/complication splits); for
mortality we used APR-DRGs. To avoid the influence of small
numbers of discharges on rankings, we dropped hospitals in
the bottom decile based on the frequency of Medicare cases
for each measure. For each measure, using Medicare
discharges only, we ranked hospitals, ordered them into
deciles, and assessed the percentage of hospitals that
changed from the highest or lowest rank by at least two
deciles when using all discharges.
Population Studied: Hospital inpatients from short-term,
non-Federal, acute care hospitals.
Principal Findings: PSIs: For hospitals in the first decile (i.e.,
lowest rate of patient safety events) using Medicare
discharges, 40% were in the third decile or lower using all
discharges for four conditions: postoperative hip fracture,
physiologic/metabolic derangement, respiratory failure, and
wound dehiscence. About a third of the first decile hospitals
based on Medicare patients were in the 3rd decile or lower
based on all discharges for: anesthesia complications, death
in low mortality DRGS, and foreign body left after procedure.
For five PSIs, about one quarter of the first decile hospitals
based on Medicare discharges were in the third decile or lower
with all patients. Fewer than 20% of the first decile hospitals
fell to the third decile or lower for the remaining three PSIs.
Improvements in rank were less dramatic than declines. For
all but three indicators, fewer than 10% of lowest decile
hospitals moved two deciles or more to better relative
performance. Inhospital mortality: For one mortality measure
(craniotomy), over 30 percent of first decile hospitals based on
Medicare patients were in the third decile or lower for all
patients. For three of the inhospital mortality measures, 1219% of the first decile hospitals based on Medicare discharges
were in the third decile or lower for all payers. For all other
inhospital mortality measures, fewer than 10% of first decile
hospitals shifted ranking. Improvement by at least two deciles
from the worst inhospital mortality ranking was seen for
CABG (21.1% of hospitals), craniotomy (24.0%), PTCA
(14.3%), and AAA repair (10.3%).
Conclusions: We found loose overlap of the top-ranked and
worst-ranked hospitals across a number of measures of
quality when comparing Medicare-only and all-payer analyses.
Depending on the measures used, comparative rankings of
hospitals could vary for different populations of discharges.
Implications for Policy, Delivery, or Practice: It is critical to
pre-test various ranking approaches before they are used to
inform patients and clinicians, or to reward high performing
hospitals.
Primary Funding Source: No Funding
●The Role of Information in Medical Markets: An Analysis
of Publicly Reported Outcomes in Cardiac Surgery
Mary Beth Landrum, Ph.D., Robert Huckman, Ph.D., David
Cutler, Ph.D.
Presented By: Mary Beth Landrum, Ph.D., Assistant Professor
of Biostatistics, Health Care Policy, Harvard Medical School,
180 Longwood Avenue, Boston, MA 02115; Tel: (617)432-2460;
Email: landrum@hcp.med.harvard.edu
Research Objective: Increased provider profiling has been
urged by many as a fundamental step in medical care reform.
Despite the building momentum for this strategy, the
underlying premise of profiling – that disseminating
information about provider quality will lead to improved
quality of care – has yet to be established. We analyze the
effect of public reporting using data from the longest-standing
effort to measure and report health care quality the Cardiac
Surgery Reporting System (CSRS) in New York State.
Study Design: We consider how report cards affect the
behavior of individual providers, specifically providers who are
publicly identified as being significantly better or worse than
their peers. Our analysis addresses three intended goals of
provider profiling. First, we examine whether report cards lead
poor-performing providers to improve their performance
relative to their peers, adjusted for changes in patient severity.
Second, we consider whether report cards lead to increased
volume for highly rated providers and lower volume for poorly
rated providers. Finally, we consider how report cards
influence the allocation of patients to particular physicians, in
particular whether report cards lead to more severe patients
being operated on by higher quality physicians. We use
longitudinal regression models to examine patterns in
volume, average severity and mortality following surgery. To
examine how behavior changes after a hospital is publicly
labeled as better or worse than their peers, we incorporate a
change-point in the model of provider-level outcomes over
time. These models hypothesize that a sharper change in
hospital behaviors is observed for hospitals publicly labeled as
unusual, compared to the general secular trends occurring in
hospitals. We compare fixed and random effect models and
also examine the use of propensity score models to match
hospitals and examine differential trends in the matched
samples.
Population Studied: Over 100,000 patients who underwent
coronary artery bypass surgery (CABG) in one of 34 hospitals
in New York State over a 9-year period (1991-1999).
Principal Findings: We find no evidence that patient severity
changed following a good or bad rating. We find some
evidence that public reporting affected both the volume of
cases and future quality at hospitals identified as poor
performers. Poor performing hospitals lost relatively healthy
patients to competing facilities and experienced subsequent
improvements in their performance as measured by riskadjusted mortality. However, these results were sensitive to
model specification.
Conclusions: We show that public reporting of quality
information can change provider behavior. We also conclude
that researchers need to be careful about how comparison
groups are formed for analyses such as these.
Implications for Policy, Delivery, or Practice:
Understanding the true effects of report cards on access and
quality of care becomes increasingly important as more states
and other organizations publicly release quality report cards
for an increasing number of conditions and as insurers start
to use provider-specific quality in determining appropriate
reimbursement.
Primary Funding Source: NIA
●The Summary Quality Index (SQuId): A Summary
Measure for Multiple Quality Indicators in Primary Care
Paul Nietert, Ph.D., Ruth G. Jenkins, MS, Andrea M. Wessell,
PharmD, Sarah T. Corley, M.D., Steven M. Ornstein, M.D.
Presented By: Paul Nietert, Ph.D., Assistant Professor,
Biostatistics, Bioinformatics, and Epidemiology, Medical
University of South Carolina, 135 Cannon Street, Suite 403,
Charleston, SC 29425; Tel: (843)876-1204; Fax: (843)876-1201;
Email: nieterpj@musc.edu
Research Objective: Primary care quality indicators are
helpful tools for translating research into clinical practice.
Providing feedback to clinicians on these indicators has been
shown to help them make system changes that improve the
quality of the care they provide. Currently our research team is
providing a geographically diverse group of clinicians with
quarterly reports on 78 different quality indicators. All of the
health care providers use a common electronic health record,
which facilitates data extraction and feedback. We wanted to
develop a single composite summary measure that would
help the research team evaluate the overall progress of the
practices’ quality improvement efforts. The summary
measure needed to be both statistically robust and clinically
meaningful.
Study Design: A total of 92 primary care practices are being
followed prospectively from Oct 2002 through Sept 2006. All
practices receive an intervention aimed at improving the
quality of care provided in the following domains: prevention
and management of hypertension, heart disease, stroke,
diabetes, and respiratory/infectious disease; cancer screening;
immunizations; substance abuse and mental health; nutrition
and obesity; and inappropriate prescribing in elderly patients.
A summary quality measure was created and calculated for
each patient, and patient summary scores were averaged to
obtain an overall summary quality index (SQuId) for each
practice. The practice level SQuId was tested for normality
using a Shapiro-Wilk test.
Population Studied: The population studied included over
350,000 patients in these 92 primary care practices.
Principal Findings: On average, the patient-level quality
measure rose from 25.0% to 30.3% during the first 2 years of
the study. A large majority (84.7%) of the practices
demonstrated improvement in their SQuId over time. The
SQuId was normally distributed across the practices and
would serve as an excellent dependent variable in any
regression model. Early feedback from clinicians indicates that
the SQuId is meaningful and informative.
Conclusions: The SQuId is an effective tool to aid in the
evaluation of multiple quality indicators.
Implications for Policy, Delivery, or Practice: In future
research studies and quality improvement efforts that include
multiple quality indicators, the SQuId should be considered as
a measure of overall quality.
Primary Funding Source: AHRQ
●Validating AHRQ PSIs with NSQIP Postoperative
Adverse Events
Dennis Tsilimingras, M.D., MPH, Patrick S Romano, M.D.,
MPH, Shukri F Khuri, M.D., William G Henderson, Ph.D.,
Anne Elixhauser, Ph.D., Amy K Rosen, Ph.D.
Presented By: Dennis Tsilimingras, M.D., MPH, Research
Health Scientist, Health Services, Bedford VA Hospital, 200
Springs Road, Bedford, MA 01730; Tel: (781) 687-2938; Fax:
(781) 687-3106; Email: dtsilimi@bu.edu
Research Objective: The Agency for Healthcare Research and
Quality (AHRQ) Patient Safety Indicators (PSIs) were
developed to identify potential inpatient adverse events using
administrative (billing) data. We sought to (1) validate the
AHRQ PSIs with postoperative adverse events in the National
Surgical Quality Improvement Program (NSQIP), which
employs trained nurse data collectors to prospectively collect
preoperative patient characteristics and 30-day postoperative
outcomes on most major operations in Department of
Veterans Affairs (VA) hospitals; and (2) test alternative PSI
definitions based on clinical information from NSQIP for the
purpose of future refinement of the PSIs. The overall study
examines five AHRQ PSIs. As an example, we present results
regarding the validation of AHRQ’s “Postoperative Respiratory
Failure” definition with NSQIP’s measures of “Unplanned
Intubation for Respiratory Failure or On Ventilator >48 hours
Postoperatively”.
Study Design: VA administrative records of inpatient
hospitalizations (from the Patient Treatment File) with valid
operating room procedures, excluding cases admitted with
respiratory/pulmonary disease, were matched with NSQIP
chart-abstracted records based on patient identifiers, facility
codes, and dates of service. A hospitalization had a PSI if it
met AHRQ’s definition that includes ICD-9-CM codes 518.81
(acute respiratory failure) or 518.84 (acute and chronic
respiratory failure). It had a NSQIP event if it met NSQIP’s
clinical definition (unplanned intubation or on ventilator >48
hours postoperatively). Using NSQIP as the “gold standard”,
we measured the sensitivity and positive predictive value of
the PSI using both the current AHRQ PSI definition and
alternative ICD-9-CM based definitions.
Population Studied: Inpatient discharges from 130 VA
hospitals from 10/01/00 through 9/30/01.
Principal Findings: From 561,229 inpatient hospitalizations in
the Patient Treatment Files, 56,419 were matched with NSQIP
data to identify 24,284 elective surgical discharges with valid
operating room procedures. From the 24,284 hospitalizations
used for the analysis: 1) the sensitivity and positive predictive
value for “unplanned intubation or on ventilator >48 hours
postoperatively” using AHRQ’s definition were 17% and 74%,
respectively; and 2) the sensitivity and positive predictive value
for “unplanned intubation or on ventilator >48 hours
postoperatively” using NSQIP’s clinical definition based on a
combination of ICD-9-CM procedure (codes for intubation/on
ventilator) and diagnostic codes were 63% and 66%,
respectively.
Conclusions: We found a higher sensitivity and a similar
positive predictive value when using a modified version of
AHRQ’s “Postoperative Respiratory Failure” PSI that includes
both diagnostic and procedures codes, rather than only
diagnostic codes. This suggests that combining procedure
with diagnostic codes may enhance the validity of
administrative-data-based indicators, such as the PSIs, as
identifiers of potential surgical patient safety events.
Implications for Policy, Delivery, or Practice: An efficient
screening tool for identifying inpatient safety events (that
warrant further investigation) can be constructed when using
clinical definitions that incorporate procedure and diagnostic
codes. The next release of the AHRQ PSI software will
incorporate the findings of this study.
Primary Funding Source: VA
Call for Papers
Crowded House: Crowding, Staffing & Patient Safety
Chair: Marie Bismark, Harvard School of Public Health
Sunday, June 26 • 10:30 am – 12:00 pm
●A Cluster Randomised Controlled Trial of the Medical
Emergency Team System
Ken Hillman, Jack Chen, Michelle Cretikos, Rinaldo Bellomo,
Daniel Brown, Gordon Doig, Simon Finfer, Arthas Flabouris
Presented By: Jack Chen, MBBS, Ph.D., Senior Research
Fellow, The University of New South Wales, Simpson Centre
for Health Services Research, Locked Bag 7103, Liverpool
BC/NSW 1871/Australia, Sydney, 1871; Tel: 612-98286662; Fax:
612-98286111; Email: jackchen@unsw.edu.au
Research Objective: It is well documented that patients
suffering cardiac arrests or dying on general wards have often
received delayed or suboptimal care. We conducted a cluster
randomised controlled trial to test the effect of the Medical
Emergency Team (MET) system on the incidence of cardiac
arrests, unplanned ICU admissions and unexpected deaths in
a group of 23 hospitals in Australia.
Study Design: We randomised 23 hospitals to either continue
operating as normal (n=11) or to introduce a MET system
(n=12). The primary outcome was the composite outcome of
cardiac arrest, unexpected death or unplanned intensive care
unit admission during the six-month study period following
MET activation.
Principal Findings: The two groups of hospitals were
comparable at baseline. From baseline to study period there
was a reduction in the rate of cardiac arrests (p=0.003) and
unexpected deaths (p=0.01) across both groups combined
over time. Introduction of the MET increased the overall
calling rate for an emergency team during the intervention
period (3.1 vs. 8.7 per 1000 admissions, p< 0.001). The MET
was called to 30.4% of patients who satisfied the calling
criteria who were subsequently admitted to the ICU. During
the study period the incidence of primary outcome was similar
in the two groups (5.86 vs. 5.31 per 1000 admissions, p=0.64).
Conclusions: The introduction of a MET system did not
significantly affect the primary outcome but the rate of cardiac
arrests and unexpected deaths decreased across both groups
of hospitals during the study. Many eligible patients were not
attended by the MET and further research is needed to
determine whether better implementation of the MET system
can improve outcomes.
●Variation in Volume-Outcome Relationships for
Hospitals and Surgeons Performing CABG Surgery
Christopher P. Gorton, M.D., MHSA, Jayne L. Jones, MPH,
Christopher S. Hollenbeak, Ph.D., David B. Campbell, M.D.,
Diane L. Arke, MS
Presented By: Christopher P. Gorton, M.D., MHSA, Senior
Physician Consultant, Pennsylvania Health Care Cost
Containment Council, 225 Market Street, Harrisburg, PA 17101;
Tel: (717)232-6787; Fax: (717)232-3821; Email:
kgorton@phc4.org
Research Objective: To describe the variation in risk-adjusted
outcomes for individual surgeons and hospitals with similar
case volumes; To examine the association between case
volume and two outcomes, 30-day mortality rates and 7-day
readmission rates, for surgeons and hospitals; To describe
trends in utilization of coronary vascular interventions as they
relate to volume and outcome.
Study Design: Retrospective analysis of administrative and
medical record abstracted data reflecting statewide experience
from Pennsylvania. Outcomes were modeled using a
generalized estimating equations (GEE) framework to account
for the correlations between patients treated by the same
surgeons and at the same hospitals. Our main outcome
measures were: Risk-adjusted thirty-day mortality rates for
hospitals and for surgeons; Risk-adjusted seven-day
readmission rates for hospitals and surgeons.
Population Studied: Data was obtained from the 62 nonfederal Pennasylvania hospitals in which adult isolated CABG
surgeries were performed during 2002. CABG surgery was
performed by 190 surgeons. Outcomes from 16,435 patients
were modeled.
Principal Findings: Wide variation was seen between and
within outcome measures at all volume levels. In looking at
individual hospitals and surgeons with outcomes statistically
higher or lower than expected, outliers were observed at both
extremes of the volume spectrum. Hospital volume was not
independently related to likelihood of patient death after
accounting for surgeon volume. Likelihood of patient death
within thirty days decreased as surgeon case volume increased
(adjusted risk ratio [RR] for each 25 case increment, 0.93; 95%
confidence interval [CI], 0.87-0.99; P=0.031). Neither hospital
nor surgeon case volumes were associated with seven-day
readmission rates.
Conclusions: Substantial outcome variations at all provider
volume levels render isolated case volume inadequate as a
basis for selective referral. The identification of a significant
association between case volume and only one of four studied
outcome measures shows the importance of multidimensional performance measurement and the need to
understand how processes of care affect outcomes.
Implications for Policy, Delivery, or Practice: We are
convinced that there is no simple proxy measure that will
permit appropriate steering of consumers towards certain
providers and away from others. We believe the correct way to
assess the performance of individual surgeons and hospitals
is to measure it, hospital by hospital and surgeon by surgeon.
Stakeholders should focus their energies on improving
measurement methodologies and standardizing
measurement systems. These developments will permit
sustained and systematic improvements in the quality of care
that cannot be achieved with simple proxies such as surgical
case volume.
Primary Funding Source: PA Health Care Cost Containment
Council
●Hospital Overcrowding is Associated with Increased
Seven Day Emergency Admission Mortality: A New
Imperative for Patient Safety
Peter Sprivulis, MBBS, Ph.D., Julie-Ann Da Silva, B Sc, Ian
Jacobs, RN, Ph.D., Amanda Frazer, MBBS, LLB, George
Jelinek, MBBS, DM
Presented By: Peter Sprivulis, MBBS, Ph.D., Harkness Fellow
in Healthcare Policy, General Medicine, Brigham and
Women's Hospital, Brigham Circle, Boston, MA 02215; Tel:
(617)283 0076; Fax: (617)301 4848; Email: psprivulis@ihi.org
Research Objective: Hospital overcrowding causes
emergency department dysfunction due to impaired access to
beds for emergency hospital admissions. This study examines
the relationship between hospital occupancy, emergency
department occupancy and emergency admission mortality.
Study Design: Analysis of hospital and death records for
62,495 emergency admissions at three tertiary metropolitan
hospitals between July-2000 and June-2003. Two, seven and
30 day mortality were evaluated against an Overcrowding
Hazard Scale (OZS) based upon hospital and emergency
department admitted patient occupancy, after adjustment for
age, diagnosis, referral source, urgency and mode of transport
to hospital.
Population Studied: 68437 emergency hospital admissions at
three tertiary metropolitan hospitals in Perth, Western
Australia.
Principal Findings: A linear relationship between the OZS
and 7-day mortality was demonstrated (r = 0.98, 95%CI 0.791.00). An OZS>2 was associated with an increased Day-2,
Day-7 and Day-30 death hazard of 1.3 (95%CI 1.1-1.6), 1.3
(95%CI 1.2-1.5) and 1.2 (95%CI 1.1-1.3) respectively. Thirty day
deaths associated with an OZS>2 compared to OZS<3 were
undifferentiated with respect to age, diagnosis, urgency,
transport mode, referral source or hospital length of stay but
had longer emergency department durations of stay (RR per
hour of ED stay, 1.1, 95%CI 1.1-1.1, P < 0.001) and longer
physician waiting times (RR per hour of ED wait, 1.2, 95%CI
1.1-1.3, P = 0.01).
Conclusions: Hospital and emergency department
overcrowding is associated with increased mortality and may
represent a remediable cause of fatal iatrogenic injury. The
Overcrowding Hazard Scale may be used to assess the risk to
patients posed by hospital and emergency department
overcrowding. Reducing overcrowding may improve outcomes
for patients requiring emergency hospital admission.
Implications for Policy, Delivery, or Practice: Hospitals
should examine strategies to improve acute patient flow as
well strategies to improve demand and capacity matching for
emergency hospital admissions.
Primary Funding Source: Australian Health Ministers
Advisory Council Grant
●The Relation of Crowded Working Conditions to Patient
Safety in Hospitals
Joel Weissman, Ph.D., Jeff Rothschild, M.D., Fran Cook, Sc.D.,
Eran Bendavid, M.D., Melissa Bender, M.D., Peter Sprivulis,
M.D., Joann David-Kasdan, RN, Jenya Kaganova, Ph.D, Les
Selbovitz, M.D., Scott Evans, Ph.D., Peter Haug, Ph.D., Jim
Lloyd, David Bates, M.D.
Presented By: Joel Weissman, Ph.D., Associate Proffessor,
Institute of Healh Policy, MGH/Harvard, 50 Staniford Street,
Boston, MA 02114; Tel: (617)724-4731; Fax: (617)724-4738;
Email: jweissman@partners.org
Research Objective: There has been a trend towards
increasing patient workload for available staff time, skill-mix
and physical resources (e.g. accommodation, equipment) in
U.S. hospitals. Potential impacts include hospital
overcrowding, emergency department inpatient boarding,
ambulance diversion, and industrial action by nursing staff to
fix patient nurse work load ratios. This study examines
whether workload is associated with an increased risk of
adverse events (AEs) for patients by evaluating whether the
daily rate of AEs vary by workload, controlling for patient
complexity and nurse staffing.
Study Design: Administrative records were screened for
indicators of potential adverse events using previously
validated criteria. Trained nurse abstracters reviewed 3446
medical charts (including readmissions) for 2504 screenpositive cases, and prepared case summaries. The
summaries were assessed by dual independent physician
reviewers in order to determine the presence or absence of
AEs, the date on which the event occurred, severity,
preventability and the factors contributing to AEs. In a parallel
data collection effort, we obtained workload and information
on patient complexity for each day of the 365-day study period,
including midnight census, throughput (including number of
admissions and discharges) RN staff, non-RN staff, patient
DRG, nurse assigned acuity, days of diversion, and
sociodemographic characteristics. After preliminary analysis,
the data for weekends and weekdays were analyzed separately.
The day was the unit of analysis. The relationships between
the workload variables and the daily rate of adverse events
(AEs per patient per day) were assessed using standard
correlation techniques.
Population Studied: All adults aged 18 to 64 admitted to a
major Boston teaching hospital between October 2000 and
September 2001, of which n = 8499 cases were selected for
study.
Principal Findings: The rate of adverse events for adult
medical-surgical patients was positively correlated with census
(p=.06), number of new admissions to the unit (p=.06),
patients per RN staff (p=.04), and patients per non-RN staff
(p=.02). Rates of AEs were 12-40% higher in the highest
quartile compared with the lowest quartile of days, depending
on the workload measure. Similar trends occurred when
examining patients in ICUs and non-ICU settings, but fewer
differences were significant.
Conclusions: High hospital occupancy, high patient
throughput, as evidenced by high patient admission rates per
day, and high patient:nurse ratios may be associated with
higher risks to patients, possibly due to an increased tension
between available and required staff time, skill-mix and
physical resources.
Implications for Policy, Delivery, or Practice: Hospital
overcrowding may be a significant contributor to patient harm.
Hospitals may need to develop strategies to better match the
supply of staff, skill and physical resources to demand in order
to optimize patient outcomes and avoid adverse events
associated with hospital overcrowding. In addition, as
researchers investigate new methods to reduce adverse
events, it may be important to consider effects during
crowded conditions.
Primary Funding Source: AHRQ
●Hospital Financial Performance, Nurse Staffing, and the
Outcomes of Patient Care
Mei Zhao, Ph.D., Gloria J. Bazzoli, Ph.D., Richard C.
Lindrooth, Ph.D., Askar Chukmaitov, MPH
Presented By: Mei Zhao, Ph.D., Assistant Professor, Public
Health, University of North Florida, 4567 Street Johns Bluff
Road, South, Jacksonville, FL 32224-2673; Tel: (904)620-1444;
Fax: (904)620-2848; Email: mzhao@unf.edu
Research Objective: Hospitals during the late 1990s
experienced many financial pressures, which led to declining
financial performance and efforts to shore up declining
margins by reducing costs. The objective of this paper is to
evaluate the relationship between changes in hospital finance
performance, nurse staffing, and patient outcomes.
Study Design: A panel study design is applied to data from
1995-2000 to examine the effect of hospital financial and
staffing incremental changes on patients’ outcome
incremental changes. Four risk-adjusted mortality rates and
five complication rates generated from AHRQ Inpatient
Quality Indicators (IQIs) and Patient Safety Indicators (PSIs)
were chosen as the patient outcome. Fixed-effects models and
first difference models were used to examine these effects.
Population Studied: All nonfederal short term general
medical-surgical hospitals in operation between 1995 and
2000 from 11 states that participated in the AHRQ’s HCUP
State Inpatient Databases (SID) program.
Principal Findings: We found that there is a weak association
between nurse staffing and patient outcomes and an
insignificant association between hospital financial condition
and patient outcomes. This weak correlation or insignificance
implies that incremental changes in RN and LPN staffing (or
in hospital financial condition) are not large enough to create
substantial change in inpatient quality of care or patient safety.
We did find significant negative relationships, however,
between hospital volume within a major diagnostic category
and associated mortality rates after accounting for patient
case mix and other factors. Additionally, hospitals treating
higher levels of patient severity had lower mortality rates.
Conclusions: The findings of this panel study are consistent
with recent literature that suggests hospital quality of patient
care is only weakly related to changes in hospital nurse
staffing, even though the prevailing belief is that increased
nurse staffing leads to improved quality of care. In addition,
the insignificant effect of hospital financial performance on
patient outcomes suggests that hospitals experiencing a
decline in financial performance may not be compromising
the care they provide patients.
Implications for Policy, Delivery, or Practice: In recent
years, hospitals have implemented many changes in hospital
staffing in an effort to reduce costs and improve financial
performance. Our results indicate that these efforts did not
result in a deterioration of patient quality of care or safety. At
most, our analysis indicates that the impact of hospital
staffing changes and financial changes on patient outcomes is
limited. Our results indicate that patient outcomes are
influenced more by the volume of care a hospital provides and
its experience in treating severely ill patients. As such, health
policy efforts focused on improving hospital quality should
focus more on regionalizing care of certain patients than on
implementing minimum staffing levels or enhancing hospital
reimbursement. However, even if most of the patient
outcome indicators were not significantly affected by staffing
levels, it is important to investigate how staffing reductions
may affect employee morale and productivity because these
may have long term effects on the health system.
Primary Funding Source: AHRQ
Call for Papers
Financing & Quality of Care
Chair: Paul Shekell, RAND
Sunday, June 26 • 5:30 pm – 7:00 pm
●Hospital Financial Performance and Patient Safety
Laurence Baker, Ph.D., David Classen, M.D., Kelly Dunham,
MPP, Jeffrey Geppert, JD, Shoutzu Lin, MS, Sara Singer, MBA
Presented By: Laurence Baker, Ph.D., Associate Professor,
Health Research and Policy, Stanford University School of
Medicine, HRP Redwood Building, Room 110, Stanford, CA
94305-5405; Tel: (650)723-4098; Fax: (650)725-6951; Email:
laurence.baker@stanford.edu
Research Objective: Hospitals in the United States face
financial pressures from declining reimbursements and rising
costs. One concern that arises is that financial pressures may
negatively influence quality of care. For example, financially
strained institutions may limit the resources committed to
adopting approaches that could prevent adverse events from
occurring such as improved staffing ratios, information
systems, technology, training, and creating a culture that
promotes patient safety. In addition, hospitals attempting to
increase revenue or decrease costs might introduce
production pressures on staff that adversely impact safety
culture and make errors more likely. The objective of this
project is to measure the relationship between hospital
financial performance and adverse events for patients.
Study Design: Longitudinal analysis of the relationship
between changes in financial performance and changes in
rates of patient safety events. We measure patient safety
outcomes at hospitals using the AHRQ Patient Safety
Indicators (PSIs), which define a range of adverse patient
events identifiable in discharge data, for 1997 and 2000. We
use Medicare cost reports and Dun and Bradstreet financial
measures to create measures of hospital financial
performance and the amount of financial pressure facing
hospitals in these years. Measures of hospital financial
performance include operating margin (net income/total
revenue), capital investment per discharge (total assets/total
discharges), profitability (net income/total assets), and Dun
and Bradstreet indices for risk of severe financial stress over
the next 12 months (financial stress class) and risk of severe
payment delinquency over the next 12 months (credit score
class). We estimate hospital-level regressions in which the
dependent variables are rates of occurrence of adverse events
defined by the AHRQ PSIs, and key independent variables are
measures of financial performance. Models include hospital
fixed effects to control for characteristics of hospitals, year
fixed effects to control for time trends, and other controls for
hospital characteristics that may change over time like beds
and technology. Effects are thus identified by changes in
financial performance over time within hospital.
Population Studied: Hospitals in 18 states covered by the
AHRQ HCUP data, including approximately 1,100 hospitals
and 15 million discharges.
Principal Findings: On average, hospitals experience a
dramatic decline in financial performance between 1997 and
2000. Hospitals that experience a decline in financial
performance by our measures also experience statistically
significant declines in performance on several Patient Safety
Indicators. There are a range of different events in the Patient
Safety Indicators. We find that the indicators that are
considered the most sensitive to the quality of nursing care
experience the greatest decline when financial performance
declines.
Conclusions: Deteriorating hospital financial performance
appears to have a negative impact on patient safety and
quality of care. The results also suggest that in recent years
hospitals have implemented few efficiency gains that have not
had an impact on quality.
Implications for Policy, Delivery, or Practice: Policies that
tend to increase the financial pressures on hospitals may
harm patients, and should be carefully evaluated. Efforts
specifically aimed at helping hospitals facing financial
pressure to monitor and maintain patient safety may be
valuable.
Primary Funding Source: AHRQ
●What’s the Return? Assessing the Effect of “Pay-forPerformance” Initiatives on the Quality of Care Delivery
Stephen Grossbart, Ph.D.
Presented By: Stephen Grossbart, Ph.D., Vice President,
Clinical Performance Measurement & Improvement, Quality
Management, Catholic Healthcare Partners, 615 Elsinore
Place, Cincinnati, OH 45202; Tel: (513) 639-2784; Fax: (513)
639-2762; Email: srgrossbart@health-partners.org
Research Objective: Identify the impact of “pay-forperformance” initiatives on the effectiveness of care delivery in
a multi-hospital system. This study tests the question: Did
hospitals participating in the Center for Medicare and
Medicaid Services’ (CMS) “Premier Hospital Quality Incentive
Demonstration Project” have significantly different rates of
improvement in the quality of care delivered compared to
similar hospitals within the same system that did not
participate in the voluntary demonstration project?
Study Design: The study uses a quasi-experimental design to
assess the impact of the Premier Demonstration project. A
test group of four acute care hospitals within a single
healthcare system that are participating in this demonstration
project was compared to a control group of seven hospitals
that were not participating in the project. The study limited
analysis to three clinical areas that are included in the Premier
Demonstration: Acute Myocardial Infarction (AMI), Heart
Failure, and Pneumonia. Performance for the first year of the
demonstration (federal fiscal year 2004) was compared to a
baseline year measured in federal fiscal year 2003 for both
participating and non-participating hospitals to determine if
the rate of improvement from the baseline year was different
between the two cohorts of hospitals. The study used
composite score methodology developed by Hospital Core
Performance Measurement Project for the Rhode Island
Public Reporting Program for Health Care Services and
modified by Premier, Inc. for use in their demonstration
project.
Population Studied: Patients discharged between October
2002 and September 2004 with a diagnosis of Acute
Myocardial Infarction, Heart Failure, or Pneumonia from 11
hospitals within Catholic Healthcare Partners. Catholic
Healthcare Partners is organized into 10 regional service areas
in five states and includes 30 hospitals, several long-term care
facilities, housing sites for the elderly, home health agencies,
hospice programs, outreach services, medical groups,
wellness centers and other organizations that operate
diversified healthcare activities. The system is the largest
healthcare provider in Ohio and also has hospitals located in
Kentucky, Pennsylvania, and Tennessee. The study was
limited to 11 hospitals with the highest levels of service within
the system.
Principal Findings: Hospitals participating in the pay-forperformance demonstration project had significantly higher
composite quality scores in each of the three clinical areas
studied. For AMI, the composite score for the cohort of
participating hospitals was 94.2% vs. 90.7% for nonparticipants (p-value <=.01). For heart failure, the composite
score for the cohort of participating hospitals was 86.9% vs.
84.9% for non-participants (p-value <=.01). For pneumonia,
the composite score for the cohort of participating hospitals
was 87.3% vs. 80.3% for non-participants (p <=.01). More
striking was the pace of improvement among the two cohorts.
In two of the clinical areas, participating hospitals saw
significantly greater improvement in the first year of the
demonstration project relative to their baseline performance
than the cohort of non-participating hospitals. In the area of
AMI, the improvement from baseline to the first year for
participating hospitals was 4.3% vs. 3.2% for non-participating
hospitals (p-value <= .01). The differential rate of
improvement in the area of heart failure was dramatic and
significant. The improvement from baseline to the first year
for participating hospitals was 19.1% vs. 10.7% for nonparticipating hospitals (p-value <= .01). Only among
pneumonia patients did non-participating hospitals slightly
outpace the pay-for-performance cohort. In the area of
pneumonia, the improvement from baseline to the first year
for participating hospitals was 7.2% vs. 8.2% for nonparticipating hospitals (p-value <= .01).
Conclusions: Outside of participation in the pay-forperformance demonstration, both cohorts of hospitals were
exposed to similar incentives to improve the quality of care for
their respective populations. The system shares a common
mission and set of values. Public reporting pressures are
identical. All hospitals in Catholic Healthcare Partners joined
the National Hospital Quality Initiative in October 2003 and
therefore each has a common interest to excel in the ten
publicly reported measures that are currently published on the
CMS web site. Furthermore, in 2004, Catholic Healthcare
Partners adopted a corporate strategic goal to perform in the
top quartile for all ten publicly reported measures in the areas
of AMI, heart failure, and pneumonia. Increased consumer
focus, public reporting initiatives, the activities of regulatory
and accreditation agencies, payer interest, as well as the
commitment of providers to their patients and communities
have all contributed to improve the quality of care delivered.
Despite facing similar conditions, however, those hospitals in
the pay-for-performance project experienced significantly
higher rates of overall quality and significantly higher rates of
improvement in two of three areas relative to the control
group of non-participating hospitals. The alignment of
financial incentives with performance has increased the rate of
quality improvement resulting in an overall higher quality of
care.
Implications for Policy, Delivery, or Practice: This analysis
provides strong evidence that pay-for-performance initiatives
significantly improved performance among a cohort of
participating hospitals. The study provides evidence to
support the conclusions regarding payment incentive found in
1) Institute of Medicine. _Crossing the Quality Chasm: A New
Health System for the 21st Century_. Washington, DC,
National Academy Press, 2001 and 2) MedPAC. _Report to the
Congress: Variation and innovation in Medicare_. June 2003.
The study helps to demonstrate support for policies that call
for the alignment of incentives with performance.
Primary Funding Source: No Funding
●Achieving High Quality-Low Cost Hospital Performance:
The Effects of Market and Organizational Characteristics
H. Joanna Jiang, Ph.D., Bernard Friedman, Ph.D., James W.
Begun, Ph.D.
Presented By: H. Joanna Jiang, Ph.D., Social Scientist, Center
for Delivery, Organization and Markets, Agency for Healthcare
Research and Quality, 540 Gaither Road, Rockville, MD 20850;
Tel: (301)427-1436; Fax: (301)427-1430; Email: jjiang@ahrq.gov
Research Objective: Hospital performance historically has
been measured one-dimensionally. Recent efforts at payingfor-performance have yet to embrace the goal of achieving
both higher quality and lower cost. The purpose of this study
is twofold: 1) to develop methods to identify hospitals based
on performance in both quality and cost of care, and 2) to
examine market and organizational characteristics that affect
quality and cost performance.
Study Design: Hospitals were classified into performance
quartiles for both quality and cost. Quality was measured by a
weighted composite of 10 risk-adjusted morality indicators
covering common medical conditions and surgical
procedures. Cost per discharge was computed by applying
cost-to-charge ratios to hospital discharge abstracts and
adjusting for principal diagnosis, severity of illness, and the
area wage index. The likelihood of achieving high quality-low
cost performance in relation to market and organizational
characteristics was examined through logistic regression for
the year 1997 and the year 2001, respectively.
Population Studied: Over 1350 nonfederal, general acute
hospitals in 10 states (AZ, CA, CO, FL, GA, IA, IL, NY, TN, WI)
that have organizational data available from the American
Hospital Association and discharge data available from the
Healthcare Cost and Utilization Project. Data on market
characteristics were extracted from the Area Resources File
and InterStudy HMO County Surveyor.
Principal Findings: Only a small proportion of hospitals
obtained high quality-low cost performance – being in the
lowest quartiles for both mortality and cost (8.04% in 1997
and 8.14% in 2001). Hospital competition and number of
HMOs in the market were positively associated with the
likelihood of achieving high quality-low cost performance
(p<.01 for both years). HMO penetration also had a positive
effect but only was significant for 1997 (p<.01). Among
hospital characteristics, for-profit status and system
membership showed positive relationships with the likelihood
of having high quality-low cost performance (p<.05 for both
years). No effects were found for service specialization and
nurse staffing. (Higher RN proportion was significantly
associated with lower mortality rate but higher cost, resulting
in minimal impact on achieving high quality-low cost
performance.) Significant interactions (p<.05) were found
between market and organizational characteristics. The
positive effect of HMO penetration on having high quality-low
cost performance was stronger for large, non-teaching
hospitals; likewise, the positive effects of hospital competition
and HMO penetration were greater for public hospitals; but
for system hospitals, these effects were negative.
Conclusions: Both market and organizational characteristics
have significant impact on a hospital’s likelihood of obtaining
high quality-low cost performance. Strong market forces
appear to have positive effects on hospital performance, and
hospitals with certain characteristics may have distinct
advantages to achieve high quality-low cost performance.
Implications for Policy, Delivery, or Practice: Payment
reforms are usually designed to encourage improvements in
either quality or cost separately. If the aim were to encourage
hospitals to attain high performance in both quality and cost,
realistic expectations should recognize the market forces and
organizational differences that are conducive or counteractive
to the goal.
Primary Funding Source: AHRQ
●The Effect of Palliative Care on Hospital Costs of
Terminal Hospitalization
Joan Penrod, Ph.D., Partha Deb, Ph.D., Carol Luhrs, M.D.,
Cornelia Dellenbugh, MPH, Tsivia Hochman, MS, Sean
Morrison, M.D.
Presented By: Joan Penrod, Ph.D., Research Health Scientist,
TREP/GRECC, Bronx VA Medical Center, 130 Kingsbridge
Road, Bronx, NY 10468; Tel: (718)584-9000 x3814; Fax:
(718)741-4211; Email: joan.penrod@mssm.edu
Research Objective: Hospital-based interdisciplinary
palliative care (PC) consultation is increasingly being used to
meet the growing demand for high quality care for patients
with chronic and life-limiting illnesses. PC focuses on
management of pain and other symptoms, identification of
goals of care, psychological support and care coordination.
Despite the prevalence of hospital-based PC teams, only a few
studies have examined their effect on hospital costs. We
examine total direct, ancillary, pharmacy and ICU costs
associated with receipt of PC compared to usual care (UC)
during terminal hospitalization.
Study Design: This is a retrospective, observational study of
hospital costs of 314 decedents at two urban VA hospitals. We
use data from VHA’s cost system and from review of patients’
medical records. A generalized linear model (GLM) with the
gamma distribution and identity link was used in the analyses.
Age, gender, co-morbidity, complications, principal diagnosis,
and whether the patient was medical or surgical were included
as covariates in all the models. To control for selection bias,
we also used propensity scores to match patients for whom
the probability of receiving PC was similar. We used a Probit
to generate a propensity score for receiving PC as a function of
covariates above and nearest neighbor and stratification
matching to compare average treatment costs. We also used a
two-part model (Probit followed by GLM) to examine ICU
costs.
Population Studied: Veterans who died from non-trauma
causes at two urban VA hospitals in 2003. We identified 82
decedents who received PC consultation and 232 who did not.
Principal Findings: PC and UC patients did not differ by age,
gender, hospital length of stay, or number of hospitalizations
in the prior year. PC patients had fewer complications, slightly
fewer comorbidites and were more likely to have cancer
compared to UC patients. After controlling for covariates, PC
decedents’ average direct total and ancillary costs were $192
(95% CI -$300 to - $84, p < .0001) and $97 (95% CI -$142 to $51, p < .0001) less per day, respectively, compared to UC
patients. Pharmacy costs were not significantly different
between the groups ($1.41, 95% CI -$20 to $22, p = .90). PC
decreased the likelihood of ICU use by .23 percentage points
(95% CI -36 to - .09). ICU costs for users were not
significantly different for PC compared to UC patients ($138,
CI -$166 to $442, p = .37). The results were similar using a
variety of propensity score matching methods.
Conclusions: PC consultation was associated with lower
hospital costs during terminal hospitalization. Costs may be
lower because PC teams help patients and families establish
clear and realistic goals of care and assist them to select
treatments that meet those goals, including less use of ICU.
Implications for Policy, Delivery, or Practice: A recent metaanalysis found PC care is associated with better symptom
control and higher family satisfaction. Our results suggest
that PC is also associated with lower costs. Taken together,
the findings suggest positive consequences with expansion of
PC for hospitalized patients.
Primary Funding Source: VA
●Delegation of Care to Support Personnel in Outpatient
Physical Therapy: Implications for Quality and Efficiency
Linda Resnik, Ph.D., PT, OCS, Zhanlian Feng, Ph.D., Dennis
Hart, Ph.D., PT
Presented By: Linda Resnik, Ph.D., PT, OCS, Assistant
Professor, Community Health, Brown University, 2 Stimson
Avenue, Providence, RI 02912; Tel: (401)863-9214; Fax:
(401)863-3489; Email: Linda_Resnik@Brown.edu
Research Objective: There is a growing trend in healthcare to
supplement or substitute cheaper health care personnel for
more highly skilled and costly health care providers As an
example, there is a trend in nursing to use licensed practical
nurses (LPNs) and on-the-job-trained support personnel such
as nurses aides in place of more highly trained registered
nurses (RNs). Similar staffing trends have occurred in physical
therapy where physical therapist assistants (PTAs) and therapy
aides are used in place of licensed physical therapists (PTs).
To our knowledge, no prior research has examined the impact
of utilization of support personnel on outcomes or efficiency
of physical therapy. The purpose of this study was to examine
the relationship between care delegation of care to support
personnel, utilization of physical therapy services, and patient
outcomes.
Study Design: Patients completed Health Related Quality of
Life (HRQL) surveys at intake and discharge. Clinicians
recorded number of visits and reported percentage of time
spent with each provider, ie, aide, PTA and PT. Three
independent variables representing delegation of care were
tested: More than 50 percent time with PTA (yes/no), percent
time, and percent time with an aide. Dependent variables
were: efficiency of care, measured by the number of visits per
treatment episode, and patient outcome, measured by
discharge HRQL. Two multilevel linear regression models
were developed. The first model predicted number of visits in
the treatment episode. The second model predicted the HRQL
score. In the multilevel models patients were nested within
therapists, therapists nested within facility, and facilities
nested within states. Models controlled for confounders at the
patient level (gender, age, impairment type, OHS intake score,
acuity, surgical history, payer type, employment), at the facility
level (ratio of full-time equivalent PTAs and PTs on the staff of
the clinic), and at the state level (4 types of regulation
governing PT).
Population Studied: The sample was 63,900 patients treated
by 2466 therapists, from 39 states in 2000 and 2001 drawn
from 351 clinics who participated with the Focus On
Therapeutic Outcomes, Inc. (Knoxville, TN).
Principal Findings: After controlling for patient, therapist,
clinic and state factors, lower percent time seen by a PT and
greater percent time seen by a PTA or aide was associated
with more visits per treatment episode. Patients seen less
than 75% time by a PT, more than 50% time with a PTA and 125% time with an aide had 4.5 more visits as compared to
patients seen primarily by a PT. Patients seen by the PT less
than 25% of the time had OHS score almost 2 points lower
than those seen by PTs 76-100% of the time.
Conclusions: Patterns of care delegation are associated with
efficiency of services. More time with the PT and less time
with PTA and aides is more efficient. Patterns of care
delegation impact patient outcomes. Better patient outcomes
occurred when the PT spent more time with the patient.
Implications for Policy, Delivery, or Practice: Although
delegation of care to physical therapy support personnel may
extend the productivity of PTs, it appears to result in less
efficient and effective services. More research is needed to
confirm these findings, and to explore the impact of care
delegation patterns in other settings.
Primary Funding Source: AHRQ
Call for Papers
Culture Change, System Change & Quality Improvement
Chair: Susan Edgman-Levitan, Massachusetts
General Hospital
Tuesday, June 28 • 8:30 am – 10:00 am
●Adverse Event Reporting Laws and Medical Errors
Laurence Baker, Ph.D., Kelly Dunham, MPP, Jeffrey Geppert,
JD, Michelle Michalek, BA
Presented By: Laurence Baker, Ph.D., Associate Professor,
Health Research and Policy, Stanford University School of
Medicine, HRP Redwood Building, Room 110, Stanford, CA
94305-5405; Tel: (650)723-4098; Fax: (650)725-6951; Email:
laurence.baker@stanford.edu
Research Objective: Improving patient safety is an important
goal for the U.S. healthcare delivery system. One specific
issue is the rate of preventable errors, and resulting harm to
patients. In an effort to reduce the incidence of preventable
errors, twenty-two states passed legislation requiring that
hospitals report the occurrence of preventable errors. The
objective of this project is to study the relationship between
state legislation and medical errors to determine if these laws
influence the incidence of preventable medical errors that
result in patient death across the United States.
Study Design: Longitudinal analysis of the relationship
between the adoption of adverse event reporting laws and
changes in rates of adverse events. We use multiple cause-ofdeath data from 1984 to 1998 to create four adverse event
indicators measuring death from a preventable error, based
on the methodology used to create the AHRQ Patient Safety
Indicators (PSIs). We use data from the Institute of Medicine
and state legislative records to identify states that passed
reporting laws, the timing of passage and/or implementation,
and other law characteristics. We estimate state-level
regressions in which the dependent variables are population
rates of mortality from the four preventable errors and the key
independent variable is the existence of adverse event
reporting legislation. We examine the overall effects of laws as
well as short term (1-3 years after passage) and long term (4+
years after passage) effects. Work in progress will model the
effects of laws with different degrees of specificity and different
penalty structures, and possibly other characteristics.
Population Studied: Since we use population rates for
persons aged 1-85 in all 50 states and the District of Columbia,
the population studied is implicitly the entire population of the
United States in this age range. Our data contain 26,747
deaths where the proximate or immediate cause was one of
the AHRQ preventable errors.
Principal Findings: Summing deaths from all four causes,
regression point estimates suggest that passage of adverse
event reporting laws is associated with a decline in mortality
rates from the studied events. Taking individual measures
separately, 2 of the 4 measures show declines in mortality
rates with passage of reporting laws and 2 show increases.
Statistical significance is often limited in these preliminary
analyses, so conclusions need to be drawn with caution.
Further work to improve model precision is ongoing. Some
results also suggest that the effect was largest in the years
immediately following adoption of the reporting law, and
remained but declined somewhat in subsequent years.
Conclusions: Legislation requiring the reporting of adverse
events may influence mortality rates from some preventable
errors. Drops in mortality rates may be due to actual
reduction in the occurrence of medical error, or to the coding
of medical error, as the proximate or contributing cause of
death. The increases in mortality rates with law passage that
we observe are harder to understand and will require further
analysis.
Implications for Policy, Delivery, or Practice: Continued
attention to adverse event reporting laws is warranted. Efforts
to raise awareness and promote reporting may be beneficial.
Primary Funding Source: AHRQ
●Improving Complex Systems: Top Performing Hospitals
in Door-to-Balloon Times for Patients with Acute
Myocardial Infarction
Elizabeth Bradley, Ph.D., Leslie A. Curry, Ph.D., David Berg,
Ph.D., Robert L. McNamara, M.D., MHS, Martha J. Radford,
M.D., Harlan M. Krumholz, M.D., HS
Presented By: Elizabeth Bradley, Ph.D., Associate Professor,
Epidemiology and Public Health, Yale School of Medicine, 60
College Street, New Haven, CT 06520; Tel: (203)785-2937; Fax:
(203)785-6287; Email: elizabeth.bradley@yale.edu
Research Objective: Hospitals are under increasing pressure
to meet clinical guidelines with public reporting of
performance indicators by the Centers for Medicare &
Medicaid (CMS) and the Joint Commission for Accreditation
of Healthcare Organizations (JCAHO). Recent research has
demonstrated substantial hospital-level variation in
performance on quality indicators; however, little is known
about how top performing hospitals are achieving their
success. We sought to describe common strategies in
hospitals with top performance in a key quality indicator for
acute myocardial infarction: door-to-balloon time. This quality
indicator reflects a complex clinical process that requires
coordination among several departments and disciplines, is
time-sensitive, and is critical to patient survival.
Study Design: We conducted a qualitative study using indepth interviews of clinical and administrative staff at
hospitals with exceptional performance in door-to-balloon
time for patients with ST-segment elevation myocardial
infarction (STEMI) treated with PCI. This method is well
suited for investigating complex interactions among people
and processes that are common in performance improvement
and organizational change. We used the constant comparative
method of qualitative data analysis facilitated by NUD-IST
software.
Population Studied: We included hospitals participating in
the National Registry of Myocardial Infarction, a large registry
of hospitalizations for myocardial infarction in the U.S., that
provided at least 50 patients with STEMI with PCI between
January 1, 2001 and December 31, 2002 (n = 151). Among
hospitals with median door-to-balloon times within the 90
minute clinical guidelines (n=35), we ranked hospitals by their
improvement during the previous 4 years and visited those
with the greatest improvement. We performed site visits and
in-depth interviews until we reached theoretical saturation,
which occurred after 11 hospitals and 122 interviews.
Principal Findings: Several key themes emerged to
characterize the top performing hospitals. First, top
performing hospitals had implemented substantial process
redesign efforts to eliminate unneeded steps and delays in the
clinical process. Second, top performing hospitals developed
rapid data feedback mechanisms both to monitor
performance and to identify areas for further improvement.
Third, top performing hospitals demonstrated effective
management of several key paradoxes: 1) standardized
protocols yet flexibility in adapting these protocols to fit
individual patient and changing organizational circumstances;
2) strong clinical champions along with extensive
interdisciplinary teamwork; 3) broad organizational goals yet
narrowly-focused performance targets; and 4) formal, explicit
channels of communication as well as informal, implicit
methods of communication. Fourth, even these top
performing hospitals struggled with interdepartmental and
interdisciplinary conflicts, which challenged improvement
efforts; participants worked through these “speed bumps” by
regularly and explicitly reminding groups and individuals of
their common goal of improving patient care.
Conclusions: We find a set of organizational features and
strategies that are common to hospitals with top performance
in a complex clinical process. Systems improvement, with
rapid data feedback, within an organizational culture that
promoted common goals and fostered multiple strategies to
achieve those goals were dominant themes among these
institutions.
Implications for Policy, Delivery, or Practice: Policy and
management efforts to improve hospital quality must involve
not only innovative system design and data feedback but also
sophisticated understanding of organizational culture.
Achieving top performance may require effective management
of paradoxical processes inherent in transforming
organizations.
Primary Funding Source: National Heart, Lung, Blood
Institute; Patrick and Catherine Weldon Donaghue Medical
Research Foundation
●Are Difference in Quality Between Physician Groups
Explained by Organizational Characteristics or Use of
Quality Improvement strategies?
Ateev Mehrotra, M.D., MPH, Meredith Rosenthal, Ph.D.,
Arnold M. Epstein, M.D., MA
Presented By: Ateev Mehrotra, M.D., MPH, Research Fellow,
Department of Health Care Policy, Harvard Medical School,
180 Longwood Avenue, Boston, MA 02115; Tel: (617)548-7517;
Email: amehrotra@partners.org
Research Objective: Despite continuing evidence
demonstrating large variations in health care quality, little is
known about what explains these variations. In this study we
looked at variations in the quality of primary care across
physician groups. We examined the association of
organizational characteristics (e.g., size, use of an electronic
medical record (EMR)) and the use of quality improvement
strategies (e.g., disease registries, reminder systems) with the
quality of primary care delivered to beneficiaries of PacifiCare,
a large California health maintenance organization.
Study Design: We obtained information on organizational
characteristics and quality improvement strategies through
structured interviews with PacifiCare-contracted physician
groups. Quality data were obtained from PacifiCare´s audited
quality performance reports (Quality Index and Provider
Profile), which are based on administrative data. Quality was
measured by determining what percentage of eligible patients
received the following: mammography, Pap smear, Chlamydia
screening, childhood immunizations, diabetic eye exam,
inhaled steroid for asthmatics, beta-blocker after myocardial
infarction, and ACE inhibitor for CHF. An overall quality score
for each physician group was created by adding standardized
individual scores. We tested whether 16 organizational
characteristics and 12 quality improvement strategies were
associated with quality scores. Those significant in univariate
analyses were included in a multivariate linear model. A model
was created for each of the 8 individual quality measures as
well as the overall quality score.
Population Studied: 119 physician groups in California who
care for over 1.5 million PacifiCare enrollees.
Principal Findings: In univariate analyses, medical groups
had statistically significant (p < 0.05) higher quality scores on
4 of the 8 individual quality measures as well as in overall
quality as compared with IPAs. In multivariate regression,
medical groups had significantly higher scores in overall
quality and in 3 of the 8 individual quality measures as
compared with IPAs (19.5% higher mammogram rate, 20.4%
higher Pap smear rate, and 11.4% higher diabetic eye
screening rate). The other covariates in the models were not
consistently associated with quality scores. Increasing fraction
of physician salary based on productivity was predictive of
higher overall quality. Higher volume of patients was
predictive of higher mammogram and childhood
immunization rates and older age of group was predictive of
higher Chlamydia screening rates. The use of an EMR, quality
bonuses for individual doctors, reminder systems for patients,
or feedback of quality performance to individual physicians
were not statistically significant predictors of quality in the
multivariate model. Medical groups were more likely to use an
EMR (36.8% vs. 1.6%) and have quality improvement
programs (e.g., contact patients to remind them of
mammograms, 73.7% vs. 27.8%). Yet less than 25% of the
quality differences between medical groups and IPAs were
explained by organizational characteristics and use of quality
improvement strategies.
Conclusions: As compared to IPAs, medical groups have
significantly higher scores on many quality measures.
Although medical groups are more likely to use electronic
medical records and have quality improvement strategies,
these characteristics and strategies explain only a fraction of
the differences in quality.
Implications for Policy, Delivery, or Practice: If these
findings are confirmed in other studies, policymakers should
implement laws and regulations that encourage the formation
of medical groups.
Primary Funding Source: CWF
●Do Quality Improvement Organizations Improve the
Quality of Hospital Care for Medicare Beneficiaries?
Claire Snyder, MHS, Gerard Anderson, Ph.D.
Presented By: Claire Snyder, MHS, Doctoral Candidate,
Health Policy & Management, Johns Hopkins Bloomberg
School of Public Health, 624 North Broadway, 6th Floor,
Baltimore, MD 21205; Tel: (410)732-7113; Fax: (410)955-0470;
Email: csnyder@jhsph.edu
Research Objective: This study explored whether Quality
Improvement Organizations (QIOs) improve the quality of
hospital care for Medicare beneficiaries. Hospitals participate
with the QIOs on a voluntary basis, and this study
investigated (1) the characteristics of hospitals that participate
with the QIOs and (2) whether hospitals that participate with
the QIOs improve the quality of care for Medicare
beneficiaries more than hospitals that do not.
Study Design: This was a retrospective comparative study
that assessed the performance of hospitals on 15
dichotomous quality of care indicators associated with
improved outcomes in the prevention or treatment of 5 clinical
areas (acute myocardial infarction, atrial fibrillation, heart
failure, pneumonia, stroke). The quality of care was evaluated
using data from a cross sectional sample of Medicare
beneficiary medical records abstracted in 1998 (baseline) and
a separate sample of medical records abstracted in 2000-2001
(follow-up). First, the characteristics (bedsize, profit status)
and baseline performance of participating hospitals were
compared to non-participating hospitals. Then, the
improvement in performance on the 15 quality indicators by
hospitals that actively participated with the QIOs was
compared to the improvement by hospitals that did not
participate with the QIOs while controlling for hospital
bedsize and profit status and patient age, sex, and race.
Population Studied: Four QIOs with responsibility for five
states (Maryland, New York, Nevada, Utah, Washington) and
the District of Columbia (DC) provided their data for this
study. Approximately 700-800 medical records per state in
each of the 5 clinical areas were abstracted at baseline (1998)
and at follow-up (2000-2001).
Principal Findings: Hospitals that voluntarily participated
with the QIOs were more likely to be not-for-profit and larger
than non-participating hospitals (p<0.05). At baseline, in
analyses controlling for hospital and patient characteristics,
there were statistically significant (p<0.05) differences
between participating and non-participating hospitals on 5/15
quality indicators, with participating hospitals performing
better on 3/5. There were no statistically significant
differences in change from baseline to follow-up between
participating and non-participating hospitals on 14/15 quality
indicators controlling for hospital and patient characteristics.
Participating hospitals improved more on the pneumonia
immunization indicator than non-participating hospitals
(p=0.005). For the 14 non-statistically significant indicators,
participating hospitals improved more than non-participating
hospitals on 8, and non-participating hospitals improved
more than participating hospitals on 6.
Conclusions: There are some important differences in
hospital characteristics and baseline performance of hospitals
that participated with the QIOs compared to those that did
not. Participating hospitals improved statistically significantly
more than non-participating hospitals on only 1/15 quality
indicators. Thus, hospital participation with the QIOs was not
associated with greater improvement in the quality of care for
Medicare beneficiaries.
Implications for Policy, Delivery, or Practice: The Medicare
program invests $200 million annually in the quality
improvement activities of the QIOs. This study does not
support the effectiveness of the QIOs in improving the quality
of care in the inpatient setting. Further research to assess the
QIOs’ effectiveness is needed, and the QIOs’ quality
improvement efforts may require modification.
Primary Funding Source: AHRQ
●Organizing for Quality: Journeys of Improvement at
Leading Healthcare Organizations in the US & UK
James L. Zazzali, Ph.D., MPH, Peter Mendel, Ph.D., Paul Bate,
Ph.D., Glenn Robert, Ph.D.
Presented By: James L. Zazzali, Ph.D., MPH, Assosiate Policy
Researcher, RAND, 1776 Main Street, Santa Monica, CA
90407; Tel: (310)393-0411; Email: zazzali@rand.org
Research Objective: To present cross-site and cross-national
findings of a recently completed 18-month international
research project to understand the ability of healthcare
organizations to sustain quality improvement (QI), and to
identify best practices in change management related to the
introduction and implementation of QI programs and
processes in large healthcare systems in the US and UK.
Study Design: Mixed methods with a multilevel approach.
Within each organization we conducted interviews with senior
leaders (i.e., a “macro-system” perspective) and used this
data to construct a survey that was administered to the clinical
and administrative staff in one high performing department
(i.e., a “micro-system” perspective). The survey measured
perceptions of the importance of key factors related to
sustaining QI efforts, the degree to which these factors were
met in the organization, the organizational culture of the
micro-system, and the respondents’ level of QI training and
QI team experience.
Population Studied: Ten leading health care organizations in
the US & UK (6 US sites and 4 UK sites). Survey of microsystem staff: 477 respondents in the US & UK (48% response
rate).
Principal Findings: Respondents from the two countries were
in agreement as to the two top most important factors related
to sustaining QI efforts: “training and time so we can improve
the quality of patient care and service,” and the need for
“senior management to make improving the quality of patient
care a priority.” In general, the staff in the US micro-systems
generally rated the various factors related to sustaining QI
efforts as significantly more important than the UK
respondents. Further, the staff in the US sites reported
significantly higher levels of QI training (51.7% for the US and
15.3% for the UK) and higher levels of QI team experience
(36.1% served on a QI team in the US, whereas 11.3% reported
this in the UK). The cultures of the micro-systems were
somewhat dissimilar across the two countries, with the US
sites exhibiting a more “task-oriented” culture than the UK
sites.
Conclusions: Respondents from the two countries were in
agreement at to some of the top factors related to sustaining
QI efforts, but there exist cross-national differences in how
these two countries organize for quality.
Implications for Policy, Delivery, or Practice: There are
significant opportunities for intra-national as well as crossnational learning for health care organization in the US and
the UK with regard to how they organize and sustain their QI
efforts.
Primary Funding Source: The Nuffield Trust, RAND
Corporation
Call for Papers
Quality & Safety for All: Caring for Vulnerable Populations
Chair: Paul Shekelle, RAND
Tuesday, June 28 • 10:30 am – 12:00 pm
●Racial Differences in Impact of HMO Coverage of
Diabetes Blood Glucose Monitors on Initiation of SelfMonitoring
Connie Mah, MS, Dennis Ross-Degnan, Sc.D., Alyce S.
Adams, Ph.D., Stephen B. Soumerai, Sc.D.
Presented By: Connie Mah, MS, Pre-Doctoral Research
Fellow, Department of Ambulatory Care and Prevention,
Harvard Medical School and Harvard Pilgrim Health Care, 133
Brookline Avenue, 6th Floor, Boston, MA 02215; Tel: (617)
509-9955; Fax: (617) 859-8112; Email: cmah@fas.harvard.edu
Research Objective: Despite being at higher risk for poor
health outcomes, black diabetic patients receive lower quality
of care and face greater barriers to self-management
compared to white patients. These differences exist even in
managed care settings, where system-related differences in
access to care are diminished. Insurance coverage of patient
self-management technologies like blood glucose selfmonitoring equipment may help to reduce race-related
barriers to effective care. We examine whether providing free
home glucose monitors in a health maintenance organization
(HMO) had greater impacts among black compared to white
diabetic patients.
Study Design: Using five years (two years pre-policy and three
years post-policy) of computerized medical record and claims
data, we used segmented longitudinal survival analysis to
examine racial differences in rates of initiation of selfmonitoring blood glucose (SMBG) following free insurance
coverage of self-monitoring equipment in 1994. To control for
fixed (age, gender, SES, BMI, pre-policy HbA1c test) and timedependent covariates (monthly diabetes medication use,
monthly glycemic levels), a piecewise extended Cox model
produced estimated hazard ratios comparing rates of SMBG
initiation between blacks and whites, before and after the
change in coverage. We modeled time in months until first
SMBG (1+ test strip dispensed), censoring at the end of the
study period or at time of first insulin use.
Population Studied: 2,275 continuously enrolled adult (18+
years) Boston-area diabetic patients with black or white racial
identification receiving care within 14 multi-specialty health
centers of Harvard Vanguard Medical Associates and insured
by Harvard Pilgrim Health Care.
Principal Findings: There were racial differences in rates of
initiating SMBG among patients using oral medications at
baseline, but not among insulin users. Among oral
medication users, unadjusted Cox models showed that SMBG
initiation rates prior to policy among black diabetic patients
were not statistically different from white patients (Hazard
Ratio=0.864, p=0.271). However, blacks had nearly 30%
higher post-policy initiation rates compared to whites (HR:
1.296, p=0.023). Insulin use and prior month HbA1c level were
strongly associated with rate of SMBG initiation (HR: 1.436,
p<0.0001; HR: 1.457, p<0.0001, respectively). Controlling for
these time-dependent covariates and fixed effects, blacks were
as likely to initiate SMBG as whites prior to the coverage policy
(HR: 0.980, p=0.931) but more likely after the policy (HR:
1.546, p=0.075). However, the pre-post change in these
hazard rates was not significant (p=0.152).
Conclusions: Our findings suggest that black diabetes
patients may have increased rates of initiating glucose selfmonitoring more rapidly than white patients following
insurance coverage of home monitors, but that this change is
explained in part by higher average HbA1c values and greater
insulin use. Although race may not be an independent
predictor of the effects of covering glucose self-monitors in
this setting, the policy appeared to be effective in triggering
patients to self-manage, particularly among blacks who may
potentially face larger barriers to quality care.
Implications for Policy, Delivery, or Practice: Coverage of
diabetes self-monitoring devices is currently mandated in over
38 state Medicaid programs. These results show the potential
of these policies narrowing racial gaps in self-management
and quality of care among racial minority groups.
Primary Funding Source: AHRQ, Harvard Pilgrim Health
Care Foundation
●Is JCAHO Accreditation Associated with Better Patient
Outcomes in Rural Hospitals?
Laura Morlock, Ph.D., Lilly Engineer, MBBS, MHA, Cyrus
Engineer, MHA, MHS, Maureen Fahey, MLA, Rebecca Clark,
BA, Andrew Shore, Ph.D.
Presented By: Laura Morlock, Ph.D., Professor, Health Policy
and Management, Johns Hopkins Bloomberg School of Public
Health, 624 North Broadway, Baltimore, MD 21205; Tel: (410)
955-5316; Fax: (410) 955-6959; Email: lmorlock@jhsph.edu
Research Objective: Fewer than 60% of rural hospitals seek
accreditation from the Joint Commission on Accreditation of
Healthcare Organizations (JCAHO) in contrast to 95% of
urban hospitals. In a recent survey of rural hospitals, 79% of
respondents indicated that the cost of accreditation was a
major deterrent. It is not known whether JCAHO accreditation
is associated with quality indicators based on patient
outcomes in rural hospitals. The objective of this analysis was
to examine relationships among JCAHO accreditation,
financial status and selected quality indicators based on inhospital mortality.
Study Design: This analysis utilizes data for a three year time
period for a random sample of approximately half of the rural
hospitals in five states. Information on the most recent
accreditation status was obtained from the JCAHO. Indicators
of financial performance were derived from the Medicare Cost
Report Files; total margins averaged over fiscal years 19992001 were used to measure financial status. The AHRQsponsored Healthcare Cost and Utilization Project (HCUP)
State Inpatient Databases for five states were used to
construct Inpatient Quality Indicators. We used three of these
risk-adjusted indicators to assess clinical performance,
including In-hospital Mortality for Acute Myocardial Infarction
(AMI), In-hospital Mortality for AMI patients (excluding
transfer cases), and In-hospital Mortality for Congestive Heart
Failure (CHF). We selected these indicators because the
majority of both JCAHO accredited and non-accredited rural
hospitals met a volume threshold of at least 30 patients with
these conditions over the three year time period. Our
analyses were based on multivariate regressions using
Generalized Estimating Equations to adjust for the clustering
of hospitals within states.
Population Studied: Patients discharged from 106 rural
hospitals in Arizona, Colorado, Florida, Iowa and Wisconsin
during 1999-2001.
Principal Findings: Among our sample of 106 rural hospitals,
54 (51 percent) were accredited by the JCAHO. Rural hospital
size as measured by number of beds (p<.0001), and better
financial status as measured by financial margin (p<.01), were
both associated with the likelihood of having sought and
obtained JCAHO accreditation. After controlling for financial
status, JCAHO accreditation was significantly associated with
lower in-hospital mortality rates for AMI (p<.01), for AMI
excluding patient transfers (p<.05), and for CHF (p<.0001).
Because accreditation was associated with hospital size, these
relationships were examined separately for hospitals larger
than 50 beds. Among these rural hospitals, accreditation was
significantly associated with in-hospital mortality for AMI
patients excluding transfers (p<.01) and for CHF patients
(p<.01).
Conclusions: JCAHO accreditation was associated with better
outcomes for patients with AMI and CHF treated in rural
hospitals after taking into consideration hospital size and
financial status.
Implications for Policy, Delivery, or Practice: These
findings, if corroborated by other studies, suggest that
accreditation status may be a useful indicator of the quality of
hospital clinical performance for rural health care consumers,
payers and regulators. These findings may also be
encouraging to the many rural hospitals with resource
constraints who are considering the costs and benefits of
embarking on the process of JCAHO accreditation.
Primary Funding Source: AHRQ
●The Usual Source of Care for Medicare Beneficiaries and
Whether they Receive Preventitive Services
Hoangmai Pham, M.D., MPH, Debra Schrag, M.D., MPH,
Marie Reed, MSH, J. Lee Hargraves, Ph.D., Peter B. Bach,
M.D., MAPP
Presented By: Hoangmai Pham, M.D., MPH, Senior Health
Researcher, Center for Studying Health System Change, 600
Maryland Avenue SW, Suite 550, Washington, DC 20002; Tel:
(202) 554-7571; Fax: (202) 484-9258; Email:
mpham@hschange.org
Research Objective: Rates of preventive services remain
below national goals. Quantifying the association between
characteristics of patients´ usual source of care, and variation
in receipt of these important quality indicators can help focus
quality improvement efforts. We examine the association
between characteristics of a nationally representative sample
of physicians and their practices, and the likelihood of patients
they treated receiving six necessary preventive services.
Study Design: Data on 8,517 physicians from the 2000-01
Community Tracking Study (CTS) Physician Survey were
linked to Medicare ambulatory care claims data on 110,201
beneficiaries they treated in 2001. Beneficiaries 65 years and
older were eligible if the physician with whom they had the
plurality of visits for evaluation and management services was
a CTS primary care physician. Outcome variables reflected
whether eligible beneficiaries received six recommended
preventive services. Independent variables reflected physician
sex; training and experience; practice type and revenue
sources; and reported access to information technology for
generating preventive care reminders and accessing
guidelines. Multivariable logistic regressions adjusted for
beneficiary race, sex, zip code income, local level of education,
urban/rural location, percentage of visits that were with their
usual source of care (USOC) physician, and Klabunde
comorbidity score, taking into account beneficiary clustering.
We used the attributable risks calculated from these
multivariable models to estimate how changes in physician
and practice characteristics would affect the probability that
beneficiaries receive these six necessary preventive services.
Population Studied: Nationally representative sample of
physicians and a representative sample of Medicare
beneficiaries they treated.
Principal Findings: 3,360 CTS primary care physicians served
as the USOC for 24,581 beneficiaries. Eligible beneficiaries
received the necessary services at rates of 47.9% for diabetic
eye exams, 55.9% for HbA1c monitoring, 46.7% for
mammography, 5.8% for colon cancer screening, 46.5% for
influenza vaccination, and 8.0% for pneumococcal
vaccination. Receipt of services was suboptimal for
beneficiaries at all income levels. Beneficiaries with board
certified physicians as their USOC were more likely than those
treated by non-certified physicians to receive HbA1c
monitoring (57.1% vs. 48.8%, p<0.05), mammograms (48.5%
vs. 36.5%, p<0.01), colon cancer screening (6.1% vs. 4.0%,
p<0.001), and influenza (47.4% vs. 41.7%, p<0.05) and
pneumococcal (8.3% vs. 6.5%, p<0.01) vaccinations. Similarly,
beneficiaries cared for by physicians who graduated from
U.S./Puerto Rican/Canadian medical schools had higher rates
of receipt of five services, than those treated by graduates of
other international medical schools (p<0.05 for all
comparisons). The advantage for beneficiaries with general
internist vs. family/general practitioner USOCs was seen in
diabetic eye exams (50% vs. 44.9%, p<0.05), mammograms
(50.5% vs. 42.4%, p<0.001), colon cancer screening (6.6% vs.
4.9%, p<0.01), and pneumococcal vaccination (8.5% vs. 7.4%,
p<0.05). Beneficiaries with USOCs in group practices were
more likely to receive all six necessary services than those in
other types of practices, as were beneficiaries cared for in
practices with lower relative Medicaid revenues (p<0.05 for all
comparisons). Physician sex, years in practice, and access to
information technology had limited associations with receipt
of services. Results were robust in multivariable analyses, and
under different definitions of usual source of care.
Conclusions: Although delivery of necessary preventive care
to Medicare beneficiaries does not meet national targets,
linkage of physician survey and patient data reveal provider
characteristics consistently associated with superior care.
Such information may help tailor quality improvement efforts
to different physician populations and care settings.
Implications for Policy, Delivery, or Practice: Such
information may help tailor quality improvement efforts to
different physician populations and care settings, and inform
quality assurance efforts and choice of physician.
Primary Funding Source: RWJF
●Perceptions of Specific Clinician Behaviors Linked to
Health Care Quality
Karen Shore, Ph.D., Roger Levine, Ph.D., Judy Mitchell, MS,
Margarita Hurtado, Ph.D., Steve Garfinkel, Ph.D., Kristin
Carman, Ph.D.
Presented By: Karen Shore, Ph.D., Sr. Research Scientist,
Health Program, American Institutes for Research, 1791
Arastradero Road, Palo Alto, CA 94304; Tel: (650)843-8121;
Fax: (650)858-0958; Email: kshore@air.org
Research Objective: To develop a comprehensive taxonomy
of the components of quality ambulatory health care, based on
both patient and provider perspectives.
Study Design: We conducted in-depth telephone interviews
with clinicians (physicians, nurse practitioners, and physician
assistants) and patients to elicit examples of specific
behaviors that represent good or poor quality care based on
actual office visits. We reviewed and analyzed clinician and
office staff actions reported to affect the quality of care in
these encounters, and then we produced a taxonomy of
behaviors that constitute quality care. This work is part of a
larger study to develop measures of health care quality in
ambulatory settings.
Population Studied: We collected over 3,000 “critical
incidents” (specific behaviors) from a total of 37 clinicians and
170 patients in two different geographic areas of the US.
Patient respondents at each of the two sites were divided
approximately equally among four racial/ethnic groups.
Principal Findings: We identified nine major categories of
incidents/behaviors, including: clinical skills (representing
34% of all incidents); rapport (28%); health-related
communication (21%); office practices, office and ancillary
staff (8%); ensuring that patients get needed care (5%);
accessibility/availability (3%); information seeking (2%);
“above and beyond” (behaviors that represented more effort
than expected) (1%); and ethical behavior (<1%). Preliminary
results show that Asian respondents and those less than 50
years old were more likely to report incidents related to
treating patients with courtesy and respect (e.g., seeing
patients on time, giving patient undivided attention).
Hispanic respondents were more likely to describe incidents
related to receiving care from ancillary staff (nurses,
therapists, technicians, etc.) and incidents related to
accessibility/availability (e.g., having weekend hours, returning
phone calls or emails, encouraging patients to come in or call
for follow-up care). Females were more likely to report
incidents related to ensuring that patients get needed care
(e.g., contacting other providers on behalf of patients) and
information seeking (e.g., obtaining relevant medical
information about patient, obtaining information needed to
treat patient). Further analyses are being conducted on these
data, and related analyses will be conducted comparing
responses of patients to those of providers.
Conclusions: Individuals with different demographic
characteristics appear to value different aspects of health care
quality. To ensure patient satisfaction with the quality of care
they receive, providers may need to modify their behaviors or
emphasize different aspects of the encounter based on the
demographic characteristics of the patient.
Implications for Policy, Delivery, or Practice: This study
presents critical information regarding what patients and
providers value in a health care encounter. It also contributes
to health care providers’ knowledge and understanding of the
extent to which the concept of health care quality differs
according to patients’ demographic characteristics. This
information can be used in quality improvement efforts, and
in developing policies and practices that effectively address
the areas of greatest concern to patients.
Primary Funding Source: AHRQ
●Prescribing Rates of Drugs to be Avoided in the Elderly in
Managed Care
Lok Wong, M, Russell Mardon, Ph.D., Arlene Bierman, M.D.,
Philip Renner, MBA
Presented By: Lok Wong, M, Senior Health Care Analyst,
Quality Measurement, National Committee for Quality
Assurance, 2000 L Street NW, Suite 500, Washington, DC
20036; Tel: (202)955-1784; Fax: (202)955-3599; Email:
wong@ncqa.org
Research Objective: To evaluate the extent to which Medicare
enrollees in managed care received drugs to be avoided in the
elderly. Drugs to be avoided in the elderly are defined in the
study as drugs classified as never or rarely appropriate by
Zahn (2001).
Study Design: Retrospective pharmacy claims data analysis in
9 health plans in the U.S. Percentages of elderly enrolled
throughout the year who received at least one drug or at least
two different drugs considered high-risk and harmful to the
elderly, reported by plan, age, gender and across the study
population were calculated.
Population Studied: Over 433,000 Medicare enrollees ages
65 and older in 9 health plans across the United States in
2002 and 2003. The number of enrollees enrolled throughout
a year in each plan ranged from 7,500 to 187,000.
Principal Findings: Nearly half a million elderly enrollees
received more than 500,000 prescriptions for drugs
considered never or rarely appropriate in the elderly. About
one third of Medicare enrollees (35.2%) received at least 1
prescription for a drug to be avoided in the elderly, classified
as never and rarely appropriate by Zahn (plan range 13.2 % to
53.2%). Less than a third (27.5%) of enrollees got at least one
rarely appropriate drugs (plan range 10.8% to 44.1%). About
7.7% of elderly enrollees got at least one never appropriate
drug (plan range 2.4% to 9.1%). Overall women were more
likely than men to receive at least one never appropriate drug
(8.3% vs. 6%) or rarely appropriate (drugs 28.6% vs. 25.7%).
A subset of Medicare enrollees (average 6%, plan range 1.1%
to 9.3%) received at least 2 different drugs to be avoided in
the elderly. Overall women were more likely than men to get
two or more different drugs to be avoided (6.7% vs. 4.3%).
Elderly enrollees ages 85 and older were less likely than
enrollees ages 65-74 years (4.2% vs 6.1%) to receive two or
more prescriptions. Differences are statistically significant.
Conclusions: Previous literature had suggested lower (about
1-20%) prescribing rates for drugs to be avoided in the elderly
in managed care. This study shows variable and high rates of
potentially harmful prescribing in the elderly managed care
population which warrants quality improvement and
monitoring. A HEDIS performance measure can highlight this
patient safety issue and encourage health plans to develop
interventions to prevent the elderly from receiving these drugs.
The extent of the quality problem is further highlighted by
patients receiving multiple types of drugs to be avoided.
Additional attention on women and younger enrollees aged
65-74 is warranted by higher prescribing rates in these
populations.
Implications for Policy, Delivery, or Practice: This study
highlights the need to monitor and reduce rates of harmful
drug prescribing in the elderly enrolled in managed care in
order to ensure patient safety and avoid adverse drug events.
Primary Funding Source: CMS
Related Posters
Poster Session B
Monday, June 27 • 6:15 pm – 7:30 pm
●Comparison of Weighted versus Dichotomous
Thresholds for Diabetes Performance Measurement in
Veterans Health Administration (VHA) Facilities
David Aron, M.D., MS, Leonard M. Pogach, M.D., MBA,
Mangala Rajan, MBA
Presented By: David Aron, M.D., MS, Director, Cleveland VA
Medical Center 14(W), VA HSR&D Center for Quality
Improvement Researach, 10701 East Boulevard, Cleveland,
OH 44106; Tel: (216)421-3098; Fax: (216)231-3427; Email:
david.aron@med.va.gov
Research Objective: To compare health care system
performance using weighted versus dichotomous measures.
Study Design: We obtained de-identified VHA chart
abstraction data for all veterans identified as having diabetes
between October 1, 2000 and September 30, 2001 with one or
more visits in the previous 12 months. We constructed 3
dichotomous performance measures based upon 2001
American Diabetes Association guidelines: <8% for A1c,
<140/80 mm/Hg for blood pressure (BP), and <130mg/dl for
Low density lipoprotein cholesterol (LDL-C). We utilized
published estimates from the Centers for Disease Control and
Prevention to calculate Quality Adjusted Life Years (QALYs)
gained from reduction in A1c from 8 to 7%; BP from 154/86
to 144/82 mm/Hg; and LDL-C from 192 to 142 mg/dl using
the following age strata: 25-34; 35-44; 45-54; 55-64; 65-74; 75-84;
and 85 and above. Similar to dichotomous performance
measurement, values above the upper limit for any parameter
received no QALYs gained; values below the lower limit
received maximal QALYs gained. QALYs gained for each risk
factor were calculated for each patient and expressed as a
percent of maximal QALYs possible. These results were
aggregated by facility for each risk factor. In addition, a
summary measure combining all three risk factors was
calculated. Similarly, a summary measure combining the three
dichotomous measures was calculated as the average of the
three measures. If a test was not performed it was counted as
above the upper limit for both the dichotomous and weighted
measure.
Population Studied: all veterans identified as having diabetes
between October 1, 2000 and September 30, 2001 with one or
more visits in the previous 12 months.The population was
largely male (97.5%) and older (53.7% > 65).
Principal Findings: There was substantial variation among
the 141 facilities for both weighted and dichotomous
measures. Mean performance as assessed by dichotomous
measures was 62% {range 48-75%} for A1c, 56% {37-70%} for
BP, 82% {66-91} for LDL-C, and 67% {55-76%} for the
summary measure. By comparison mean performance as
assessed by weighted measures (% maximal QALY gained)
was 45 {31-60%} for A1c, 73 {55-84%} for BP, 86 {68-97%} for
LDL-C, and 75 {59-84%} for the summary measure.
Correlation (Spearman rank) between the two types of
measures for summary score was very good (R= 0.75). Of the
14 poorest performers (bottom 10%) assessed by
dichotomous measures, when assessed with weighted
measures 12 remained in the bottom 10% and 2 moved to the
3rd quartile (below median). Of the 14 best performers (top
10%) by dichotomous measures, when assessed with
weighted measures, 6 remained in the top 10%, 2 moved to
the 2nd quartile, 5 moved to the 3rd and 1 moved to the 4th.
Conclusions: Mean A1c adherence was overestimated by 17%
and BP underestimated by 17% with the dichotomous
measure relative to the weighted measure. Both methods
similarly identified the worst performing facilities. However,
there was marked discordance in the rankings of best
performers.
Implications for Policy, Delivery, or Practice: Since the use
of QALYs may more accurately reflect population health, we
propose that performances measures be based upon QALYs
rather than dichotomous thresholds. As a continuous
measure, QALYs may be perceived as fairer because “partial
credit” can be given. Such measures may be more effective in
organizational improvement efforts.
Primary Funding Source: VA
●The Quality Olympics: A System-Wide Approach to
Improving Diabetes Care
David Aron, M.D., MS, Scott Ober, M.D., MBA, Aleece Caron,
Ph.D., Michelle Davidson, MEd, David C. Aron, M.D., MS
Presented By: David Aron, M.D., MS, Director, Cleveland VA
Medical Center 14(W), VA HSR&D Center for Quality
Improvement Researach, 10701 East Boulevard, Cleveland,
OH 44106; Tel: (216)791-3800x4947; Fax: (216)231-3427;
Email: david.aron@med.va.gov
Research Objective: Unacceptable levels of adherence to
recommended preventative care have prompted physician
managers to search for interventions to improve outcomes,
eg; clinical reminders, audit and feedback. Key components
necessary for successful continuous quality improvement
(CQI) include a strong and innovative quality leadership
together with top hospital management and a current
reporting system that can be accessed by local managers. The
competition among Community-based Outpatient Clinics
(CBOCs) termed the “CBOC Olympics,” was designed to
provide local managers with such a tool. This competition
provided broad dissemination of real-time data and was tied
directly to network performance measures, as well as
providing positive incentives to improve adherence and
enhancing multidisciplinary teamwork. The goal of this project
was to develop and evaluate a CQI intervention to improve
adherence among CBOC primary care providers to clinical
reminders for their patients with diabetes mellitus(DM).
Study Design: We created a inter-clinic competition to
provide continuous feedback to clinicians and staff about
every CBOCs level of adherence to clinical reminders.
Specifically, we tracked 6 DM measures: eye exam, foot exam,
lipid screening, hypertension screening, hemoglobin A1c
monitoring, and hemoglobin A1c control. VHA network
leadership determined target measures for satisfactory and
exceptional performance.
Population Studied: Data from primary care physicians from
13 VA CBOCs in Northeast Ohio were reviewed monthly from
June 2002 through November 2004 to determine guideline
adherence to diabetes clinical reminders. CBOCs were divided
into 3 tiers in order to adjust for patient volume.Data was
analyzed using Statistical Process Control (SPC). Site visits
were performed monthly to all clinics by an interdisciplinary
senior management team. Monthly Olympic winners from
each tier were disseminated to all employees and were based
solely on adherence to clinical reminders. Characteristics of
successful CBOC Olympic winners are included.
Principal Findings: Baseline results were high; every CBOC
achieved satisfactory adherence in 2002. There was an overall
trend of improvement; the average overall initial adherence to
clinical reminders was 81% and increased to 83%. This
increase has been maintained. There was also a decrease in
overall variation to adherence rates. Between June 2002 and
May 2003, the Upper and Lower Control Limits (UCL and
LCL) were 0.955 and 0.693 respectively. This variation was
reduced between July 2003 and October 2004: the UCL was
0.95 and the LCL was 0.71. However, there was considerable
variation among the CBOCs. Of the 13 CBOCs, 7 showed
significant, maintained improvement. The average adherence
rate for the first and second years ranged from 76.6% to
85.0% and from 80.9% to 89.7% respectively. Examination of
tier results indicate that the smallest and largest tiers made
improvements, but the middle tiers adherence declined.
Characteristics of successful CBOCs include enthusiastic
physician managers, multiple opinion leaders, good
interdisciplinary collaboration, and ownership of the data.
Conclusions: Successful implementation the CQI initiative
depended on continuous reinforcement, clarity of objectives,
and effective opinion leaders. The system as a whole reduced
variation in adherence to clinical reminders, but the level of
adherence desired was not achieved.
Implications for Policy, Delivery, or Practice: System
redesign may be necessary to achieve optimal adherence rates
to DM clinical reminders.
Primary Funding Source: VA
●Leaping the Chasm: Reducing Inpatient Mortality at
Academic Medical Centers
Raj Behal, M.D., MPH
Presented By: Raj Behal, M.D., MPH, Medical Director,
Clinical Effectiveness & System Redesign, University
HealthSystem Consortium, 2001 Spring Road Suite 700, Oak
Brook, IL 60523; Tel: (630)954-4892; Fax: (630)954-5879;
Email: rbehal@uhc.edu
Research Objective: To reduce risk adjusted inpatient
mortality at academic medical centers and to understand
organizational factors associated with higher than expected
mortality rates.
Study Design: Sequential quality improvement projects were
conducted at 3 US academic medical centers. Key steps
included: 1) analysis of risk adjusted comparative data to
identify populations with higher mortality; 2) evaluation of
coding and documentation practices and their impact in risk
adjustment; 3) interviews with physicians, nurses,
pharmacists, house officers, quality improvement staff,
hospital and clinical leadership; 4) assessment of
standardization of care using evidence-based practices; 5)
study of systems factors and clinical processes; 6)
identification of best practices at other centers; 7) presentation
of findings and recommendations to senior leadership.
Population Studied: Inpatient adult and pediatric inpatients
at 3 academic medical centers in the United States.
Principal Findings: The following contributory factors were
identified: hand-offs with inadequate information exchange;
incidence of preventable complications; failure to recognize
and act upon early signs of patient deterioration; impeded
patient flow into and out of intensive care units; suboptimal
selection and optimization of operative cases; and, variations
in care and use of evidence-based therapies. Follow up review
of outcomes showed that the first hospital reduced the
standardized mortality rate from 167 to 110; the second
hospital reduced the SMR from 108 to 92; the third hospital
reduced mortality for sepsis and cardiac bypass surgery by
over 50%.
Conclusions: Systematic identification of factors contributing
to inpatient mortality and subsequent interventions targeting
these factors can effectively reduce excess mortality.
Leadership focus on quality of care, effective use of data,
physician leaders' participation in improvement, and effective
structures for quality improvement were critical for success.
Implications for Policy, Delivery, or Practice: Redesign of
care processes and systems is important for radical
improvement in mortality in academic medical centers.
Assigning accountability based on organizational goals is
critical for sustaining results. Interventions targeting mortality
should be encouraged by payers and policy makers.
Primary Funding Source: No Funding Source
●Use of Incentives to Stimulate Physician Behavior
Change to Improve Quality: Physician Reactions and
Potential Barriers
Sandra Berry, MA, Stephanie Teleki, Ph.D., Cheryl L. Damberg,
Ph.D., Chau Pham, MA, Sandra Berry, MA
Presented By: Sandra Berry, MA, Senior Behavioral Scientist,
Health, RAND, 1776 Main Street, Santa Monica, CA 90407;
Tel: (310)393-0411 x7779; Fax: (310)393-4818; Email:
sandra_berry@rand.org
Research Objective: To understand physicians’ opinions of
and experiences with financial incentives for quality, and
identify barriers to using incentives to drive quality
improvements.
Study Design: Structured telephone interviews and focus
groups with Blue of California (BCC) Preferred Provider
Organization (PPO) physicians who are part of BCC’s
Physician Quality and Incentive Program (PQIP). PQIP is a
RWJ Rewarding Results demonstration project. All BCC PPO
physicians who met measurement criteria (n=11,000) were
scored on a set of clinical, generic prescribing, and
administrative measures and provided a comparative
performance report card. A subset (n=1,200) in Northern
California was also eligible for a financial bonus. One-hour
telephone interviews were conducted with 25 physicians (15
men, 10 women; 16 pilot, 9 non-pilot; majority aged 50-59).
Additionally, two focus groups (n = 24 doctors) were held in
the pilot area (18 men, 6 women; all pilot; majority aged 4059). A wide range of specialties was represented. Participants
were asked about their opinions of health care quality and
financial incentives, the quality of care they provide and help
needed to improve, and experiences with financial incentives
for quality, especially PQIP.
Population Studied: A convenience sample of BCC PPO
physicians.
Principal Findings: Physicians reported that their primary
objective in practicing medicine is “doing what is right” for the
patient. However, many obstacles were cited: limited time,
coding challenges, reimbursement delays/reductions, noncompliant patients, lack of systems to track care, unrealistic
expectations, and personal sacrifice. They agreed selfmonitoring is important and stated they did this by
talking/comparing themselves to colleagues, attending
conferences, reading journals, and following guidelines.
Overall, physicians felt financial incentives are a good idea
because “physicians are underpaid;” however, they did not
believe it possible to accurately measure quality and tie money
to quality performance, feel the bonus is money already owed,
and sponsors’ motives are suspect. Most feel that opinions of
and comparisons to colleagues are more important than
money and that they would respond to comparative
information; they noted that money will only change behavior
if measures are reasonable and easy to implement, and the
amount offered represents minimally 5-10% of their annual
salary. Many said they are overwhelmed with instructions
from the many plans with which they contract; their reaction is
to “tune out” the information and “just practice the best I
can.” Nearly all physicians were against public reporting,
citing inaccurate measurement and misinterpretation by
patients, which could damage physician careers. Physicians
felt that EMRs and reminder systems are key to improving
quality, and that measurement by medical specialty societies
would be more accepted. Awareness of any quality incentive
program was minimal, with some physicians accidentally
discarding bonus checks.
Conclusions: While many physicians indicated that they
would readily accept financial incentive payments, they remain
skeptical about measurement accuracy, the intentions of
program sponsors, and the likelihood of these programs to
modify their behavior.
Implications for Policy, Delivery, or Practice: Pay-forperformance programs must take physicians’ perspectives
into account, given that their buy-in is a crucial component of
success.
Primary Funding Source: RWJF, California HealthCare
Foundation
●Are Hospitals that Refuse to Participate in Adverse
Events Studies Different from Those that Accept?
Regis Blais, Ph.D., Marie-Laure Boursiquot, M.D.
Presented By: Regis Blais, Ph.D., Professor, Department of
Health Administration, University of Montreal, PO Box 6128,
Station Centre-ville, Montreal, H3C3J7; Tel: (514) 343-5907;
Fax: (514) 343-2448; Email: regis.blais@umontreal.ca
Research Objective: Patient safety has become a major
preoccupation in health care systems. One common indicator
of patient safety is the rate of adverse events (AE) among
hospital patients. Researchers who want to conduct AE
studies have to get permission from hospital authorities to
access patient charts. Some hospitals may refuse to give that
permission. If those that refuse provide poorer quality care
than those that accept, study results may not be valid. The
objective of this study was to compare hospitals that refused
to participate in one AE study to those that accepted, in terms
of hospital structure and services, patients' characteristics and
some patient outcomes.
Study Design: A sample of hospitals from Quebec (Canada)
were invited to participate in an AE study: 18 accepted and 5
refused. The two groups of hospitals were compared on their
structure and services (number of beds, services and
admissions per year, average length of stay, proportion of day
surgery), patients' charateristics (age, sex, case severity) and
outcomes (in-hospital death, 30-day and 90-day readmission).
Data mainly came from discharge abstract databases.
Population Studied: Population studied included 14 large
community hospitals and 9 teaching hospitals located within
260km of Montreal, and the 293000 patients who were
hospitalized in these hospitals over a year.
Principal Findings: The two groups of hospitals did not differ
in terms of structure or services. The patients of hospitals that
refused to participate in the AE study were nearly 5 years older
on average, but the difference did not reach statistical
significance. The proportion of females and the average
patient case severity did not differ. In-hospital mortality was
higher (significant) in non participating hospitals, as was
readmission (non significant). Multivariate regression showed
that after adjusting for patients' characteristics, refusing to
participate in the AE study was not associated with in-hospital
mortality or readmission.
Conclusions: Hospitals that refuse to participate in AE
studies may have different patients' case mix and worse
outcomes than those that accept. These could contribute to
their refusal. AE studies with high refusal rates could be
biased toward better outcomes than the reality.
Implications for Policy, Delivery, or Practice: Health care
settings sampled in AE studies should be representative of all
settings. To gain collaboration from potentially low health care
performers, the purpose of AE studies should be quality
improvement and not punishment.
Primary Funding Source: Quebec ministry of Health
●Methods to measure the Impact of Nurse-to-patient
staffing ratios at the micro level
Terry Capuano, RN, MSN, MBA, Joanna Bokovoy, RN, DrPH,
Nancy Davies-Hathen, RN, MSN, CCRN, Elizabeth Karoly,
RRT, MBA, Martin Everhart, BA, MPA, Paulette Kennedy, RN,
MSN, Carol Cyriax, RN, BSN, Tina Dalessandro, RN, Stephen
Bogar, BS, Lisa Romano, RN, MSN, Stephanie Pacelli, BS,
Patricia Matula, RN, MSN
Presented By: Joanna Bokovoy, DrPH, Director, Healthcare
Research, Clinical Services, Lehigh Valley Hospital, 1245 South
Cedar Crest Boulevard, Allentown, PA 18049; Tel:
(610)402.1813; Fax: 610.402.1408; Email:
joanna.bokovoy@lvh.com
Research Objective: To Develop methods that will allow
researchers to effectively study the impact of nurse staffing on
nurse and patient satisfaction and patient outcomes at a
micro level in a Magnet Hospital setting.
Study Design: Prospective, longitudinal study of four similar
medical-surgical units – one with a capped nurse-to-patient
staffing ratio and the other three with usual staffing ratios.
Methods and processes were developed to effectively evaluate
the unit level the impact of a capped nurse-to-patient staffing
ratio on nurse and patient satisfaction and nurse-sensitive
patient outcomes.
Population Studied: Staff and patients on four similar
medical-surgical units.
Principal Findings: Since many factors besides a capped
nurse patient staffing ratio can impact quality patient and staff
outcomes, those factors had to be identified and addressed.
Choosing a Magnet Hospital environment was one method,
and a second method was to build on the improvements
defined from our use of a validated model to evaluate the
impact of our nursing delivery a year before our study began.
An effective process was developed to maintain nurse to
patient ratios on the intervention medical-surgical unit at 1:4-5,
not to exceed 6 patients on days/eve shifts, and 1:7, not to
exceed 10 on nights. A variety of measures were developed or
refined to determine the effects of this staffing model on
nurse-sensitive patient outcomes, patient/family and staff
satisfaction, LOS, as well as other quality of care measures.
Included in these were relevant contextual information,
detailed staffing profiles (includes numbers, Benner stage of
clinical competence, qualifications, etc.), available workload
data (includes dependency/complexity), reliable and valid
quality assurance/outcome measures and comprehensive
costing data. Our overall approach appears methodologically
robust, reliable, and replicable.
Conclusions: Nurse staffing ratio issues can be effectively
measured on the micro level.
Implications for Policy, Delivery, or Practice: Defining the
role that nurse-patient ratios do or do not play in quality of
care is important because of current policy to mandate nursepatient ratios.
Primary Funding Source: No Funding Source
●Impact of Personal Digital Assistant Devices on
Medication Safety in Primary Care
James D. Bramble, Ph.D., Kimberly Galt, Pharm.D., Ann Rule,
Pharm.D., Wendy Taylor, BBL, Mark Siracuse, Ph.D.,
Bartholomew Clark, Ph.D., Wayne Young, Pharm.D.
Presented By: James D. Bramble, Ph.D., Associate Professor,
Creighton Health Services Research Program, Creighton
University Medical Center, 2500 California Plaza, Omaha, NE
68178; Tel: (402)280-4129; Fax: (402)280-4809; Email:
jbramble@creighton.edu
Research Objective: This study examines the impact of
personal digital assistants (PDAs) use on potential prescribing
errors in primary care office-based practices. PDAs offer great
potential for improving quality at the point-of-care. The PDA
offers the opportunity to access drug information, enter data,
and print information pertaining to the patient including the
prescription. PDA use allows for easy access to electronic
information at the point-of-care that has the potential to
reduce prescribing errors. Additionally, computer generated
prescriptions reduce legibility problems. Specifically, this study
will measure the occurrence of potential preventable
prescribing related errors.
Study Design: Using a randomized controlled trial we
examined the impact of PDAs on potential prescribing errors.
The intervention group received training to use the PDA for
prescription generation and drug information retrieval. After
successful completion of initial training of the intervention
group, they were encouraged to use the PDA in their
prescribing process. However, use was voluntary, resulting in
prescriptions being generated by PDA and traditional methods
based upon providers’ daily behaviors. Control group
physicians maintained their traditional method of prescription
generation. We assessed the face of the prescription and
compared baseline and post-intervention data (500
prescriptions each) on errors of legibility, commission,
omission, and interpretation.
Population Studied: We recruited 78 primary care physicians
(85% family practitioners, 15% internal medicine) from 30
primary care offices in the Omaha metropolitan statistical area
and one rural clinic. The average age was 42 with 33% female
and the majority family practitioners (85%).
Principal Findings: The intervention group generated 43% of
their post intervention prescriptions using the PDA. Potential
errors associated with legibility decreased from 9.1% to 2.7%.
The remaining legibility error we attribute to the less than
100% use of the PDA in the prescribing process. We also
observed significant reduction in errors of all categories of
omission such as failing to include the patient’s age
(including the patient’s age increased from 4.5 to 40.8%). Use
of abbreviations on the face of the prescription, often not
interpretable by the patient, also decreased. Use of
abbreviations for the route of administration decreased from
63.2% to 36.7% while use of abbreviations for frequency of
administration decreased from 85.8% to 50.7%. Two types of
errors increased including the vagueness of instructions and
identifying the wrong technique for drug administration.
Conclusions: This project introduced the use of PDAs and
corresponding applications that have the potential to enhance
patient safety by reducing prescription errors (e.g., legibility
errors). Additionally, the study showed that these
improvements are achievable within the typical small to
medium primary care office based practice. This work also
reveals the existence of barriers to achieving 100% success
(i.e., not all prescriptions were written on the PDA by the
intervention group). Despite, not having full adoption of using
the PDA, there was an overall improvement in patient safety.
Implications for Policy, Delivery, or Practice: This research
generated empirical evidence demonstrating the contribution
of PDAs to potentially reduce medical errors and improve
patient safety in the primary care physician office-based
practice environment. These findings inform policy makers
and administrators of the benefit of adopting hand held
technology.
Primary Funding Source: AHRQ
●Beneficial Effect of Physical Therapy in Lower Extremity
Trauma
Renan Castillo, MS, Ellen J. MacKenzie, Ph.D., Michael J.
Bosse, M.D., LEAP Study Group
Presented By: Renan Castillo, MS, Doctoral Student, Health
Policy and Management, Johns Hopkins University Bloomberg
School of Public Health, 624 North Broadway, Room 544,
Baltimore, MD 21205; Tel: (410) 614-4024; Fax: (410) 6142797; Email: rcastill@jhsph.edu
Research Objective: Despite the fact that physical therapy
services are routinely prescribed following lower extremity
trauma, there is little evidence that these services are
beneficial to the patient. The current study examines the effect
of physical therapy use on a range of measures of physical
impairment in a cohort of lower extremity trauma patients.
Study Design: Patients (N=382) from a larger study of severe
leg trauma were interviewed by a research coordinator and
examined by an orthopedic surgeon and a physical therapist
during the initial hospitalization, and at 3, 6, 12 and 24
months post discharge. Two measures of unmet need for
physical therapy were defined – based alternatively on need as
assessed by the orthopedic surgeon and the physical
therapist. Specifically, the orthopedic surgeon and a physical
therapist independently evaluated each patient during follow
up and were asked whether they felt the patient would benefit
from physical therapy between that time point and the next
follow up visit. Patients who were classified as needing
physical therapy services over the given time period and who
reported receiving no physical therapy at the end of that time
period (next follow up time point) were classified as having
unmet need as for that follow up segment (from the
independent perspectives of both the orthopedic surgeon and
the physical therapist). Multivariate regression techniques
were used to compare improvement in measures of physical
impairment between the groups with met and unmet need
over the 3-6, 6-12 and 12-24 month periods.
Population Studied: Patients in the current analysis
constitute a subgroup of a larger study conducted to assess
the outcomes of amputation or reconstruction following limbthreatening lower extremity trauma below the distal femur.
Only patients treated by reconstruction were selected for this
analysis.
Principal Findings: A range of measures of physical
impairment were used, including: percent impairment in knee
and ankle range of motion, reciprocal stair climbing pattern,
gait deviations when walking, self selected walking speed
greater than 4 feet per second, and the Functional
Independence Measure (FIM)... Patients with unmet need for
physical therapy as assessed by the physical therapist were
statistically significantly less likely to improve in all five of the
selected domains of physical impairment than patients whose
physical therapy need was met. These results held true even
after adjustment for patient socio-demographic, personality,
and social resources, as well as injury and treatment
characteristics, reported pain intensity, and impairment level
at the beginning of the study time period. Patients with unmet
need for physical therapy as evaluated by an orthopedic
surgeon were generally worse off than patients with met need,
but in only one of the five selected impairment measures
(range of motion)was the difference statistically significant.
Conclusions: The results are consistent with a beneficial
effect of physical therapy following lower extremity trauma.
Furthermore, the results suggest that physical therapists may
be more likely to identify patients who could benefit from such
services than orthopedic surgeons.
Implications for Policy, Delivery, or Practice: The results
point to a need for improved standards for the prescription of
physical therapy services, and highlight the importance of
involving a physical therapy professional in the prescribing
process.
Primary Funding Source: National Institutes of Health
●Using Patient Safety Indicators to Identify Outlier
Hospitals: A Picture is Worth 1000 Statistics
Cindy Christiansen, Ph.D., Peter Rivard, MHSA, Shibei Zhao,
MPH, Susan Loveland, MAT, Dennis Tsilimingras, M.D.,
MPH, Amy K. Rosen, Ph.D.
Presented By: Cindy Christiansen, Ph.D., Associate Professor,
Health Services, Boston University, 200 Springs Road,
Building 70, Bedford, MA 01730; Tel: (781)687 2915; Fax: (781)
87 3106; Email: cindylc@bu.edu
Research Objective: Patient Safety Indicators (PSIs),
developed by the Agency for Healthcare Research and Quality
(AHRQ), are useful for identifying potential in-hospital patient
safety events. However, characteristics inherent to indicators
make it difficult to interpret PSI results when trying to
understand rates from multiple indicators. Our objectives
were to 1) compare Bayesian and average-ranking
methodologies for selecting hospitals that have extremely high
or low rates and 2) improve presentation of hospital-level
information from PSI analyses.
Study Design: We used a retrospective one-year cohort to
calculate hospital-level PSI counts (numerators), acute-care
hospitalizations (denominators), observed, expected, and
AHRQ-smoothed rates (estimates of rates that are both riskadjusted and smoothed across multiple years of data) for 16
PSIs using Department of Veterans Affairs (VA) administrative
data from fiscal year 2001 (FY’01) and AHRQ PSI software
(version 2.0). From observed (O) to expected (E) ratios and
Bayesian models, distributions representing “true” O/E ratios
for each indicator at 118 hospitals were determined. For the 6
most frequent PSIs, we used simulation methods to obtain
hospital-level posterior densities for the six-indicator
combination of O/E ratios. We compared rankings of median
ratios from the posterior densities to average rankings of
smoothed rates for hospitals that ranked in the top 10 (or
bottom 10) by either method. Graphical methods were
developed to facilitate communication of results.
Population Studied: Patients discharged from Veterans
Affairs Medical Centers (VAMCs) between October 1, 2000
and September 30, 2001. The sample consists of 430,552
hospitalizations incurred by 281,423 patients at 118 VAMCs.
Principal Findings: Both methods selected 7 of the same
hospitals in the top 10 and 7 in the bottom 10. Six hospitals
chosen by the Bayesian but not by the average-ranking
method had rankings that varied widely across indicators. By
averaging ranks, the strength of evidence for having extreme
O/E ratios is lost; hospitals with ranks that vary widely, i.e.,
those having some very high and some very low ranks for the
6 indicators, are not identified as extreme. The Bayesian
method retains the evidence by accounting for the certainty in
the estimates of the individual indicators (using the posterior
densities) and incorporates known correlation across PSIs.
Using the Bayesian method we can conclude, with more
certainty than not, that the top 10 hospitals had 23% to 46%
fewer PSIs than expected, and the bottom 10 had 39% to 92%
more than expected. Posterior density graphs demonstrate
how the certainty of the estimates affects the combination
ratio and add another dimension when translating the results
to policy-makers.
Conclusions: Bayesian analyses provided more information,
appropriately recognized the uncertainty in estimates of
performance, and, with the help of graphics, were as easy to
understand as results from selection based on average
rankings.
Implications for Policy, Delivery, or Practice: Translating
patient safety analyses into concise, useable information is
both science and art. Bayesian models and graphics are tools
that all health care systems should consider when conducting
patient safety research and when translating results to
consumers and policy-makers.
Primary Funding Source: VA
●Consumer Satrisfaction with Primary Care Provider
Choice and Associated Trust
Ming Ying Lisa Chu-Weininger, Ph.D., Rajesh Balkrishnan,
Ph.D., Lu Ann Aday, Ph.D., Mark A. Hall, JD
Presented By: Ming Ying Lisa Chu-Weininger, Ph.D., Doctoral
Student, Management, Policy, and Community Health,
University of Texas School of Public Health, 2120 El Paseo,
Unit 408, Houston, TX 77054; Tel: (713) 797-1795; Email:
Ming.Ying.L.Chu-Weininger@uth.tmc.edu
Research Objective: Trust is important in medical
relationships and for the achievement of better health
outcomes. Developments in managed care in the recent years
are believed to affect the quality of healthcare services delivery
and to undermine trust in the healthcare provider. Physician
choice has been identified as a strong predictor of provider
trust but has not been studied in detail. Consumer
satisfaction with primary care provider (PCP) choice includes
having or not having physician choice. The study specific aims
were: (1) to determine variables related to the factors:
consumer characteristics and health status, information and
consumer decision-making, consumer trust in providers in
general and trust in the insurer, health plan financing and plan
characteristics, and provider characteristics that may relate to
PCP choice satisfaction; (2) to determine if the factors in aim
one are related to PCP choice satisfaction; and (3) to analyze
the association between PCP choice satisfaction and provider
trust, controlling for potential confounders.
Study Design: Cross-sectional survey
Population Studied: Analyses were based on secondary data
from a random national telephone survey in 1999, of
residential households in the United States which included
respondents aged over 20 and who had at least two visits with
a health professional in the past two years. Among 1,117
eligible households interviewed (response rate 51.4%), 564
randomly selected to respond to insurer related questions
made up the study sample.
Principal Findings: Analyses using descriptive statistics, and
linear and logistic regressions found continual effective care
and interaction with the PCP beyond the medical setting most
predictive of PCP choice satisfaction. Consumer
characteristics, information and decision-making, provider
characteristics and plan characteristics were factors found to
predict consumer’s report of enough PCP choice. The
information and decision making factor and provider
characteristics factor were also found to predict provider trust
and renamed ‘continuity of care related’ and ‘provider related’
in the revised framework. All in all, PCP competencies were
significant in predicting PCP choice satisfaction and provider
trust.
Conclusions: It appeared that PCP choice satisfaction as a
predictor of provider trust overlapped with other predictors of
provider trust, such as communication between the provider
and consumers. Although it may sound an aggressive task, to
examine all the known predictors of provider trust in one
study in the future. It may provide a holistic view of what’s
relatively important in predicting provider trust and in
sustaining trust in the long term. Findings alluded to the
importance of PCP competency training to include
interpersonal (which also entails cultural) competency.
Implications for Policy, Delivery, or Practice: This study has
implications for the development of health services
information. In the mist of growing consumerism, relevant,
quality, cognitive, and accessible provider specific information
would be helpful to the consumer. The goal of information is
for consumers to make a fair choice, without too much
complication to the decision making process. That is why it
makes sense to study ‘trust in provider specific information.’
Provision of useful information may be a better business
strategy compared to hard core media marketing.
Primary Funding Source: RWJF
●Variations in Quality Outcomes Among Hospitals in
Different Types of Health Systems, A Panel Study
Askar Chukmaitov, M.D., MPA, Gloria J. Bazzoli, Ph.D., Mei
Zhao, Ph.D.
Presented By: Askar Chukmaitov, M.D., MPA, Research
Associate/Ph.D. Candidate, Health Administration, Virginia
Commonwealth University, 1008 East Clay Street, P.O. Box
980203, Richmond, VA 23298; Tel: (804) 827-1811; Fax: (804)
828-1894; Email: achukmaitov@vcu.edu
Research Objective: Although prior research has found
differences in costs and financial performance across different
types of hospital systems, there has been no systematic study
of variations in patient quality of care or safety indicators
across different systems. Our study examines whether five
main types of health systems - centralized, centralized
physician/insurance, moderately centralized, decentralized,
and independent - as well as other hospital characteristics are
associated with differences in quality of patient care.
Study Design: Data were assembled for 6 years (1995 –
2000) from multiple sources. We used 4 AHRQ risk adjusted
inpatient quality indicators (IQIs) and 5 risk-adjusted patient
safety indicators (PSIs) as dependent variables. In addition to
examining hospital system type, we also examined hospital
characteristics including compliance with selected JCAHO
standards. The type of system to which a hospital belongs
may influence its ability to meet these standards as well as
influence its quality of care. Random effects models were
used in the analysis.
Population Studied: All short term general medical-surgical
hospitals from 11 states (AZ, CA, CO, FL, IA, MD, MA, NJ, NY,
WA, and WI) in operation between 1995 and 2000 that
participated in the AHRQ’s HCUP State Inpatient Databases
(SID) program.
Principal Findings: Overall, our findings have not identified a
clear pattern (neither worsening nor improvement) in quality
performance across different types of health systems.
However, selected quality and patient safety indicators show
significant variation for selected system types. Hospitals in
moderately centralized health systems may provide better care
for AMI patients. Hospital membership with centralized
physician/insurance system type may be associated with
worsening outcomes for post-operative pulmonary embolism
or deep vein thrombosis. Independent hospital systems are
worse performers in terms of decubitus ulcer in comparison
with all other types of health systems. In addition, hospital
compliance with the JCAHO standard for availability of patient
specific information is associated with better outcomes for
pneumonia patients.
Conclusions: Structural differences in health systems have
not affected quality performance in a definite way during the
1995 – 2000 period. Organizational characteristics of
moderately centralized health systems, such as moderate
centralization of authority at the system level and high
differentiation of services at the hospital level, may be
contributing to better inpatient quality performance. Low-tomoderate centralization combined with little differentiation of
services, as in independent hospital systems, may be resulting
in worsening of patient safety performance. Hospital
compliance with selected JCAHO standards may lead to
improvements in the process of care delivery and better
clinical outcomes.
Implications for Policy, Delivery, or Practice: Hospital
systems hold much potential for hospitals in improving
patient quality of care and safety because they provide a
laboratory for studying the health care process and sharing
lessons across multiple institutions. Our findings suggest
that hospital and health system managers may have not
systematically reaped the benefits of this potential in
improving quality performance of their organizations. Based
on our findings, we recommend that future studies use a
combination of IQIs and PSIs when examining institutional
quality of care because both provide different and
complementary information.
Primary Funding Source: AHRQ
●Racial and Ethnic Disparities in Patient Safety: Findings
from the 2004 National Healthcare Disparities Report
Elizabeth Dayton, MA, Ernest Moy, M.D., MPH, Roxanne
Andrews, Ph.D., Anna Poker, MS, RN
Presented By: Elizabeth Dayton, MA, Junior Service Fellow,
Department of Health and Human Services, Agency for
Healthcare Research and Quality, 540 Gaither Road, Rockville,
MD 20850; Tel: (301)427-1320; Fax: (301)427-1341; Email:
edayton@ahrq.gov
Research Objective: Our objective is to describe racial/ethnic
disparities in patient safety in American healthcare, drawing
on research from the 2004 National Healthcare Disparities
Report, a congressionally mandated annual report analyzing
differences in healthcare by race/ethnicity and several other
factors.
Study Design: Data come from two nationally representative
sources: the Healthcare Cost and Utilization Project (HCUP)
State Inpatient Databases (SID) and the Medical Expenditure
Panel Survey (MEPS). A sample of 7 million all-payer hospital
discharge abstract data from 22 SID (states with good quality
race data) applied to the AHRQ Patient Safety Indicator
software was used to develop national estimates for hospital
patient safety events. The MEPS collects annual household
data via computer-assisted surveys. Each MEPS respondent
undergoes five rounds of interviews over a two and a half year
period, and a sample of respondents’ data is linked to
information from their employers and medical and insurance
providers. The MEPS sample size is about 10 thousand
households. Twenty patient safety measures are analyzed.
Data for non-Hispanic blacks, non-Hispanic Asians and Pacific
Islanders, and Hispanics are compared with data for nonHispanic whites revealing components of patient safety where
minorities receive better, about the same, or worse care than
whites.
Population Studied: The national U.S. population (MEPS)
and discharges from US community hospitals (HCUP).
Principal Findings: For a quarter of measures minorities have
poorer patient safety than non-Hispanic whites. For over a
quarter, minorities have better patient safety than whites.
However, for nearly half of measures minorities and whites
have fairly equal patient safety. More specifically, rates of
iatrogenic pneumothorax (puncture of the lung) are lower
among Hispanics than whites, deaths from complications
potentially resulting from care are higher among Asians and
Pacific Islanders (APIs) than whites, rates of deep vein
thrombosis (blood clots forming in the legs following surgery)
and postoperative pulmonary embolus (blood clots traveling
from the legs to the lungs following surgery) are higher
among blacks and lower among APIs than whites, and rates of
postoperative septicemia (life-threatening invasion of the
bloodstream by microorganisms) are higher among blacks
and Hispanics than whites.
Conclusions: There are significant disparities in patient safety
between minorities and whites in the United States. For some
measures minorities have worse patient safety, for others they
have better patient safety, and for many measures no
racial/ethnic difference is observed. Analyzing and
understanding differences in patient safety may prove a
valuable tool for improving healthcare.
Implications for Policy, Delivery, or Practice:
Understanding patient safety differences may guide
improvement efforts and inform policy: differences
emphasize the best and poorest performing components of
healthcare, suggesting best practices and components of care
that would most benefit from intervention. Promoting
awareness of disparities is an important first step towards
their elimination.
Primary Funding Source: AHRQ
●Effect of Short Stay Patients on the Measurement of
Nursing Sensitive Quality Indicators
Nancy Dunton, Ph.D., Byron Gajewski, Ph.D.
Presented By: Nancy Dunton, Ph.D., Research Associate
Professor, School of Nursing, University of Kansas Medical
Center, 3901 Rainbow Boulevard Mail Stop 4043, Kansas City,
KS 66160; Tel: (913) 588-1456; Fax: (913) 588-4531; Email:
ndunton@kumc.edu
Research Objective: The valid measurement of nursing
sensitive quality indicators is fundamental to quality
improvement activities in acute care hospitals. Short stay
patients (e.g., same day surgery patients and observation
patients) add to nurses’ workloads, but are not captured in
midnight censuses. Further, the growth of managed care and
improvements in medical procedures have increased the
prevalence of short stay patients. This presentation
establishes the effect of including short stay patient hours in
two quality performance indicators endorsed by the National
Quality Forum: nursing hours per patient day (NHPPD) and
the patient fall rate.
Study Design: The National Database of Nursing Quality
Indicators (NDNQI), a project of the American Nurses
Association, has collected quarterly data on nursing sensitive
quality indicators since 1998. Until the fourth quarter of 2003,
the number of patient days was measured by the midnight
census. Since that time, hospitals have had the opportunity to
report on care hours for short stay patients. This analysis was
based on a correlational design comparing five specifications
of patient census, used in the denominators of NHPPD and
the fall rate: midnight census, midnight census plus actual
short stay hours, midnight census plus an estimate of short
stay hours, actual hours for all patients, and the average of
multiple censuses throughout the day. The various indicator
specifications were examined for five unit types: critical care,
step down, medical, surgical, and combined medical-surgical.
Population Studied: The analysis was based on NDNQI data
for the third quarter of 2004 from nearly 500 hospitals across
all 50 states. The patient care unit was the level of analysis
and over 4,700 units were included in the analysis.
Principal Findings: Nearly half (48%) of the units used
midnight census, 14% used midnight census plus actual
hours, 5% used midnight census plus an estimate of short
stay hours, 6% used actual hours for all patients, and 9%
used an average of multiple censuses throughout the day.
The interaction between unit type and patient day method was
significant for NHPPD (F=4.198). The simple main effects for
patient day method were significant for surgical and combined
med-surg units. The interaction between unit type and patient
day method for fall rates was not significant. The main effect
for patient day method on fall rates was significant (F=14.55).
Conclusions: Incorporating the presence of short stay
patients significantly altered the NHPPD and fall rate
indicators, reflecting an improved measurement of nursing
workload. The four data capture methods for incorporating
short stay patient hours produced different results. In
particular, the midnight census plus estimate of short stay
hours method and the average of multiple daily census
measures may not be as valid as methods that incorporate
actual hours.
Implications for Policy, Delivery, or Practice: Standardizing
the measurement of actual patient load in acute care hospitals
will improve the validity of national nursing quality indicators,
providing hospitals with improved tools for targeting quality
improvement activities. Moreover, improving the validity of
patient day measurement will enable improved specification of
multivariate models used to examine the influence of nurse
staffing on patient outcomes.
Primary Funding Source: American Nurses Association
●Instruments for Measuring Outcomes in End-of-Life
Care: An Evidence-Based Review
Sydney Dy, M.D., MSc
Presented By: Sydney Dy, M.D., MSc, Assistant Professor,
Health Policy & Management, Johns Hopkins Bloomberg
School of Public Health, Room 609, 624 North Broadway,
Baltimore, MD 21205; Tel: (410)614-4047; Fax: (410)955-0470;
Email: sdy@jhsph.edu
Research Objective: (for the Southern California EvidenceBased Practice Center) Assessing end-of life care requires
accurate, reliable, and comparable measures of patient and
caregiver experience. Our aims were to explore psychometric
properties of available instruments for evaluating end-of-life
care and to describe the use of these measures in the end of
life literature.
Study Design: As part of an evidence-based review of end-oflife care, we identified measurement instruments used to
define outcomes in interventions studies. We also identified
interventions related to advance care planning, continuity,
symptoms, caregiver burden, and satisfaction, and described
the use of measures in these studies.
Population Studied: Sources for our systematic review
included Medline, the Cochrane Database of Reviews of
Abstracts of Effects (DARE), the National Consensus Project
for palliative care, and several recent systematic reviews
including the Toolkit of Instruments to Measure End-of-Life
Care (TIME) and reviews from Health Canada and the
National Institute for Clinical Excellence (NICE), United
Kingdom. Searches were limited to published articles in the
English language involving adult human subjects.
Principal Findings: From 24,423 total citations, we identified
one high-quality recent systematic review that recommended
35 measures across 11 domains (TIME), and we identified an
additional 48 measures with published psychometric data.
Few of the instruments had been developed or evaluated in
palliative care populations; most were developed specifically
for cancer. Few were validated across different settings
important for end-of-life care (e.g. hospitals, nursing homes,
and hospices). Almost none have been compared across
different populations. Eighty-one unique intervention studies
met the inclusion criteria and underwent a detailed review.
Twenty-one were related to advance care planning, 20 to
continuity, 18 to symptoms, 13 to caregiver concerns, and 12 to
satisfaction. These studies used 97 separate instruments to
measure outcomes. There was little overlap in the
instruments used across studies: 80/97 were used in only one
study, and only 5 were used in 4 or more studies.
Conclusions: Instruments used in end-of-life intervention
studies lack sufficient evaluation across diseases, stages of
illness, settings, and populations applicable to end-of-life care.
Many intervention studies in end-of-life care did not use highquality instruments to measure outcomes. We identified little
overlap in measurement instruments across studies.
Implications for Policy, Delivery, or Practice: The lack of
appropriate and uniform measures for evaluating
interventions in end-of-life care limits the ability to synthesize
this literature and its implications for im-proving health care.
Further research should emphasize improving measures and
standards for evaluating outcomes in end-of-life intervention
studies, as well as further psychometric evaluation of existing
measures.
Primary Funding Source: AHRQ, National Institute for
Nursing Research
●Risky Concomitant Medication Dispensing in Ambulatory
Care
Jennifer Elston Lafata, Ph.D., Lonni Schultz, MS, Ph.D., K.
Arnold Chan, M.D., Sc.D., David H. Smith, MHA, Ph.D.,
Marsha A. Raebel, Pharm.D., Marianne Ulcickas Yood, MPH
Presented By: Jennifer Elston Lafata, Ph.D., Director, Center
for Health Services Research, Henry Ford Health System, 1
Ford Place, Suite 3A, Detroit, MI 48202; Tel: (313)874-5454;
Fax: (313)874-7137; Email: jlafata1@hfhs.org
Research Objective: To estimate rates of risky concomitant
medication dispensing among ambulatory care patients.
Study Design: We used automated membership, prescription
drug and medical claims data to compile information on
patient demographics, comorbidities and dispensings of
medications of interest during baseline (07/01/1999 –
12/31/1999) and follow-up (01/01/2000 – 06/30/2001).
Identification of risky medication pairs was based on FDA
warning labeling, Hansten and Horn Ratings and input from
practitioners practicing within the 10 participating health
plans. Two definitions of concomitant use were used: same
day dispensings and dispensings with overlap according to
the dispensed “days supply.” Among enrollees with a
dispensing during follow up for warfarin, digoxin,
cyclosporine, lovastatin/simvastatin, other statins, and
nitrates, we report the proportion and number with a
concomitant dispensing of a risky medication pair.
Population Studied: National cohort of 953,889 health plan
members aged 18 years or older who were continuously
enrolled with prescription drug coverage during baseline.
Principal Findings: Among these health plan members,
110,802 were dispensed at least one of the 6 object
drugs/drug classes of interest. The mean age of those
dispensed a medication of interest ranged from 49 years
(cyclosporine users) to 72 years (digoxin users), and males
represented between 51% (digoxin users) and 100% (nitrate
users) of the cohort. The proportion (number) of object
drug/drug class users who received a risky concomitant
dispensing was 1.3% (N=257) nitrate users, 8.9% (N=2,553)
other statin users, 19.0% (N=10,321) lovastatin/simvastatin
users, 24.3% (4,660) warfarin users, 26.4% (N=3,958) digoxin
users, and 29.4% (N=210) cyclosporine users when
concomitant use was defined using days supply overlap.
When only same day dispensings were considered the
proportions (numbers) were 0.3% (N=53), 2.7% (N=784),
9.2% (N=4,988), 8.7% (N=1,675), 15.8% (N=2,373) and 18.5%
(N=132), respectively. Thus, although the rate of risky
dispensing is greatest among patients receiving cyclosporine,
the greatest absolute number of risky dispensings occurred
among those dispensed lovastatin or simvastatin. For
warfarin users, the most frequent risky concomitant
dispensing was with NSAIDs (days supply/same day:
55.1%/46.1%). Among those dispensed digoxin, 68.4%/76.5%
were also dispensed verapamil/ditiazem/bepridil.
Verapamil/diltiazem was also the most frequent risky
concomitant dispensing among those receiving cyclosporine
(64.3%/69.7%) and lovastatin or simvastatin (52.8%/69.6%).
Conclusions: Most research in the area of risky concomitant
medication use has focused on co-prescribing in the hospital
setting. We found evidence of risky concomitant medication
dispensing among ambulatory care patients. In some cases,
this occurs despite the availability of low risk alternatives (e.g.,
Tylenol as opposed to NSAID use among warfarin users, and
other blood pressure medication as opposed to verapamil or
diltiazem use among digoxin, cyclosporine and lovastatin or
simvastatin users).
Implications for Policy, Delivery, or Practice: Because using
prescription claims data to monitor concomitant prescribing
in the outpatient setting is relatively simple and inexpensive,
an opportunity exists to augment current benchmarking and
other quality improvement efforts to better understand how
risky concomitant medication dispensing translates into
adverse events and ultimately to improve medication safety in
the outpatient setting.
Primary Funding Source: AHRQ
●Inappropriate Prescribing for Medicare Managed Care
Beneficiaries in the Last Year of Life
Cheryl Fahlman, BSP, MBA, Ph.D., Danielle Doberman, MPH,
M.D., Joanne Lynn, M.D.
Presented By: Cheryl Fahlman, BSP, MBA, Ph.D., Senior
Research Associate, Health Research and Educational Trust,
325 7th Street NW, Washington, DC 20004; Tel: (202)6262365; Fax: (202)626-2689; Email: cfahlman@aha.org
Research Objective: The primary aim of this study is to
assess violations of the Beers’ criteria for inappropriate
prescribing in the elderly for a group of Medicare beneficiaries
in their last year of life (LYOL) in a large national managed
care organization (MCO) using a pharmacy benefits manager.
This research provides the first claims-based data on Beers’
violations by disease and demographics at the end of life.
Study Design: This research used drug claims and enrollment
data collected between January 1998 and December 2000,
supplemented by the Medicare denominator file. We based
the analysis on the 2002 list of Beers’ criteria. We analyzed
the relationship between socio-demographic descriptors,
insurance characteristics, cause of death, and violations of the
Beers’ criteria. Logistic regression techniques were used to
estimate the determinants of Beers’ criteria violations.
Population Studied: Four thousand six hundred and two
beneficiaries qualified for the study, based on dying between
1999-2000, being continuously enrolled for two years before
death and having at least one prescription filled in the LYOL.
Principal Findings: Two thousand three hundred and forty
five beneficiaries (52%) had at least one claim in the LYOL
that violated a Beers’ criterion. Fully 13% experienced more
than one unique Beers’ violation, and 5% have three or more
violations during the study period. The most common
violations were the use of propoxyphene (13.5%), followed by
doxazosin (2.8%), and amitriptyline (2.1%). Based on total
claims, cancer patients were most likely to receive
propoxyphene (35.3%) followed by patients with a heart
condition (29.6%). For amitriptyline and doxazosin, patients
with a heart condition receive the greatest number of
prescriptions (35.7% and 32.3% respectively) and cancer
patients received the second highest amount (25.2% and
23.8% respectively). A large proportion of the potentially
inappropriate prescribing involves psychoactive drugs (ten out
of the most frequent seventeen). Sometimes, what meets
criteria for a violation may in fact be appropriate treatment,
such as promethazine for cancer patients. The logistic model
showed fewer Beers’ criteria violations associated with one or
fewer comorbidities (31.9%), being male (35.1%), being nonwhite (51.7%), and having died in 2000 (18.1%). Beers’
violations were more common if the beneficiary has five or
more comorbidities (36.7%) or has employer sponsored
retiree benefits (27.7%).
Conclusions: This study showed that many beneficiaries have
prescriptions that violate a Beers’ criterion, though most
received only one inappropriate agent. Most violations
involved psychoactive medications. With more prescriptions
comes a higher risk of receiving inappropriate medications.
Implications for Policy, Delivery, or Practice: The overall
rate, 52%, is higher than most other studies of community
dwelling elderly and inappropriate prescribing, probably
reflecting this particularly high-utilizing portion of the
population, those sick enough to die. Given their frail health
status, the level of prescription use, and the presence of a
pharmacy benefits manager, the level of inappropriate
prescribing may be surprising. Assessing whether prescribing
of these drugs reflects errors or is actually appropriate is worth
investigating.
Primary Funding Source: RWJF
●Assessing Outpatient Quality of Care
Brian Gallagher, MS, Edward Hannan, Ph.D., Liyi Cen, MS
Presented By: Brian Gallagher, MS, Senior Healthcare
Analyst, Health Policy, Management and Behavior, School of
Public Health, University at Albany, One University Place,
Rensselaer, NY 12144; Tel: (518)402-4091; Fax: (518)402-0414;
Email: bxg04@health.state.ny.us
Research Objective: Examine the quality of care in the
outpatient setting by examining complication rates of selected
procedures.
Study Design: Patients who had an outpatient procedure in
New York State during 2002 were matched to New York State
inpatient records. Patients who were admitted to the inpatient
setting within 30 days with a complication relating to the
outpatient procedure were identified. Rates of complications
by selected procedure were calculated.
Population Studied: Patients who had an outpatient
procedure in New York State during 2002 and who were
reported to the New York State Department of Health in the
outpatient hospitalization administrative data system, the
Statewide Planning and Research Cooperative System
(SPARCS).
Principal Findings: Of the 1,138,913 outpatient discharges in
New York State during 2002, 28,784 (.025%) were admitted to
inpatient care within 30 days of the outpatient discharge. Of
these readmitted patients, a subset can be identified as
suffering complications. The rate of complications varies by
procedure performed in the outpatient setting, but were all
relatively low. Outpatient procedures which had the highest
complication rates included tonsillectomy (ICD-9-CM
procedure code 28.2) with a 2.3% complication rate,
arteriovenosomy (ICD-9-CM procedure code 39.27), with a
1.9% complication rate, and insertion of devices (ICD-9-CM
procedure code 86.06, 86.07), which had a 1.3% complication
rate. High volume procedures including cataract (ICD-9-CM
procedure codes 13.1, 13.2, 13.3, 13.4, 13.5, 13.6), with 95,991
procedures, had a complication rate of .11%, knee (ICD-9-CM
procedure code 80.6), with 36,676 procedures, .36%
complication rate, and non-diagnostic intestine (ICD-9-CM
procedure codes 45.0, 45.3, 45.4, 45.5, 45.6, 45.7, 45.8, 45.9),
with 62,606 procedures, .5% complication rate, had low rates
of complications.
Conclusions: The rate of complications from outpatient
procedures as measured by inpatient treatment is low. It does
not appears that there is a major problem with the quality of
surgical care delivered in the outpatient setting.
Implications for Policy, Delivery, or Practice: The
complication rate for selected outpatient procedures in New
York State as measured by readmission to the inpatient
setting is low, but lessons can be learned from examining the
types of complications which do occur and the type of patients
who are most at risk for post operative complications.
Primary Funding Source: AHRQ
●Measuring the Use of Spirometry in Affirming a
Diagnosis of Chronic Obstructive Pulmonary Disease
(COPD)
Min Gayles Kim, MPH, Fernando Martinez, M.D., Russ
Mardon, Ph.D., Phil Renner, MBA
Presented By: Min Gayles Kim, MPH, Senior Health Care
Analyst, Quality Measurement, National Committee for
Quality Assurance, 2000 L Street NW Suite 500, Washington,
DC 20036; Tel: (202)955-1731; Fax: (202)955-3599; Email:
gayles@ncqa.org
Research Objective: To explore the potential for a
performance measure to assess the appropriate use of
diagnostic instrument (spirometry) to confirm airway
obstruction and a diagnosis of COPD. Despite numerous
clinical guideline recommendations to incorporate spirometry
in the assessment and diagnosis of COPD and staging of
disease severity, health plan data shows that most members
with COPD do not undergo spirometry.
Study Design: Observational study conducted in five health
plans. Using administrative data, an incident case of COPD
was specified using a negative diagnosis history period of two
years. Spirometry use rates assessed use within the two year
negative diagnosis period and the six months following the
first incidence of COPD diagnosis in administrative claims as
an indicator for confirmation of airway obstruction and
presence of disease.
Population Studied: Five health plans participated in the
study by providing patient-level administrative and medical
record data. The enrollments of these plans included
commercial, Medicare, and Medicaid product lines across
several geographical regions of the U.S., and ranged in size
from 52,000 to over 820,000 members.
Principal Findings: Using administrative records, the planspecific frequency of new cases of COPD ranged from 0.56 per
1000 to 3.52 per 1000 commercial plan members, and
averaged 28.41 per 1000 Medicare members. Of members
with a new COPD diagnosis, the average plan rate for
spirometry use was 32% (range 26% to 37%). Spirometry use
did not vary across product lines but showed lower rates in
men versus women and in the older age groups. A higher
percentage of spirometry tests (60%) occurred in the
physician office compared to a pulmonary function lab (39%).
The denominator validation in medical record averaged 65%
with a range of 30% to 100%. Validation was higher in the
Medicare population (73%). Contrary to perceived notion that
many spirometry tests happen in the physician office without
a claim generated, no spirometry tests were found in the MR
only without a corresponding administrative claim in two
plans and relatively few in the remaining two plans indicating
administrative data are reliable for capturing spirometry tests.
Conclusions: This study showed low rate of overall
spirometry use with greater use in younger population.
Administrative data provides reliable information on
spirometry use in COPD diagnosis. The low rate of spirometry
use may reflect ongoing controversy over the usefulness of
this diagnostic tool despite consensus in guidelines for its
use.
Implications for Policy, Delivery, or Practice: Measurement
of spirometry use is feasible and may encourage greater
research into its role in diagnosis and management of COPD.
Primary Funding Source: Other, Supported in part by
educational grant from Boehringer-Ingelheim & Pfizer
●Do Malpractice Claims Influence Physician Outcomes
and Practice Patterns?
Gilbert Gimm, Ph.D., MBA
Presented By: Gilbert Gimm, Ph.D., MBA, Health Care
Systems, University of Pennsylvania, 110 Sandra Road,
Wilmington, DE 19803; Tel: (267)252-8797; Email:
gilber20@wharton.upenn.edu
Research Objective: This paper examines how accurately the
tort liability system is able to identify physicians with adverse
outcomes, and whether malpractice claims have an influence
on physician behavior. Specific variables of interest are the
number of adverse outcomes, volumes, C-section rates, and
Medicaid patient mix.
Study Design: Using a panel dataset that links inpatient
deliveries between 1992-2000 in the state of Florida with
malpractice claims, I examine a series of OLS regressions that
control for observable patient and physician characteristics,
and for fixed effects.
Population Studied: Individual physicians in the state of
Florida between 1992-2000.
Principal Findings: The tort liability system does a relatively
poor job of identifying physicians with a higher percentage of
adverse outcomes, controlling for volumes. I find strong
evidence that physicians experience a 10% lower volume, on
average, in response to a malpractice claim. And I find
evidence that physician have fewer adverse outcomes in
response to injuries, not to the claim itself. No significant
effects were found with respect to C-section rates or Medicaid
patient mix.
Conclusions: The tort liability system is not very effective at
improving the quality of care. Also, malpractice claims have a
real effect on individual physician behavior in the form of a
lower volume of deliveries, which may be due to a supply-side
factors (i.e. greater cautiousness of practice, referring more
difficult cases to others, or time spent complying with an
investigation), or a demand-side factor (i.e. patient demand
and reputation effects).
Implications for Policy, Delivery, or Practice: The
malpractice liability system, which is designed to deter
negligent care, may not be very effective as a policy tool for
improving the quality of care. Futher research is needed to
determine the overall welfare effects associated with the
reduction in delivery volume, and whether substitution of
cases is occuring between physicians.
Primary Funding Source: AHRQ
●Implementation of Infection Control Guidelines in U.S.
Hospitals:A Pilot Study
Karen Goldsteen, MPH, Ph.D., , , Julie A. Jefferson, RN, MPH,
Raymond L. Goldsteen, MS, DrPH, , , , , ,
Presented By: Karen Goldsteen, MPH, Ph.D., Clinical
Associate Professor, Graduate Program in Public Health, State
University of New York, Stony Brook, Health Sciences Center,
Level 3, Room 071, Stony Brook, NY 11794; Tel: (631) 4446658; Fax: (631)444-3480; Email:
karen.goldsteen@stonybrook.edu
Research Objective: To indicate the extent to which evidencebased infection control guidelines are implemented in U.S.
hospitals.
Study Design: We interviewed Infection Control Practitioners
(ICPs) in 20 acute care hospitals in the U.S. We selected five
hospitals each from the Northeast, Southeast, Midwest, and
West. One of the five hospitals in each region was chosen for
its excellent reputation in infection control. Of the remaining
four hospitals, two were selected for their very high score and
two for their very low score on a measure of overall hospital
performance developed by HealthGrades. A telephone
interview, using a structured questionnaire developed by the
authors, was conducted in February-March 2004. The main
outcome measure was the Infection Control Performance
Indicator developed by the authors and based on reported
implementation of Category I guidelines in thirteen areas of
infection control: surgical site infections, intravascular
catheter-related infections, construction and environmental
risks, isolation precautions, disinfection and sterilization,
health care-associated pneumonia, nosocomial transmission
of methicillin-resistant Staphylococcus aureus (MRSA) and
vancomycin-resistant enterococcus (VRE), emerging
antimicrobial resistance, hand hygiene, Legionnaire’s Disease
and dialysis unit, bloodborne pathogens, surveillance, and
employee health policies.
Population Studied: U.S. teaching and community hospitals
with more than 200 beds.
Principal Findings: No hospital was 100% compliant with the
infection control guidelines measured by our Infection Control
Performance Indicator. The highest rate of compliance was
88%, and the lowest compliance rate was 53%. Just onequarter of the hospitals we interviewed were 80-88%
compliant. About half were 70-79% compliant.
Conclusions: The findings suggest that the evidence-based
guidelines for infection control are not fully implemented in
most hospitals, and many hospitals fall very far short of
achieving full implementation.
Implications for Policy, Delivery, or Practice: The U.S.
might significantly reduce the hospital-acquired infection rate
by ensuring that current, evidence-based infection control
guidelines are fully implemented. Further work is needed to
determe how to: (1) reduce internal barriers to full
implementation of guidelines; and (2) develop policies in the
external environment that facilitate infection control within
hospitals.
Primary Funding Source: Long Island Community
Foundation – Lilo and Gerard Leeds Fund
●Patient Characteristics and the Impact of Mortality
Indicator Definition on Provider Performance
Measurement
Christopher Gorton, M.D., MHSA, Jayne L. Jones, MPH, Diane
L. Arke, MS
Presented By: Christopher Gorton, M.D., MHSA, Senior
Physician Consultant, 225 Market Street, Harrisburg, PA 17101;
Tel: (717)232-6787; Fax: (717)-232-3821; Email:
kgorton@phc4.org
Research Objective: To determine differences in patient
characteristics (including cause of death) between subpopulations included in two commonly used mortality
measures, in-hospital mortality and 30-day mortality.
Study Design: Retrospective analysis of a Pennsylvania
statewide inpatient dataset in which thirty-day mortality and
cause of death were identified by linking Pennsylvania Health
Care Cost Containment Council inpatient data with
Pennsylvania Department of Health death certificate data.
Risk factor selection and expected rates were calculated using
multivariate logistic regression models to determine hospital
risk-adjusted mortality rates. We divided Pennsylvania
residents who died after CABG surgery into three distinct
subpopulations (Group 1: death during index hospitalization
<=30 post-operative days; Group 2: death during index
hospitalization >30 post-operative days; Group 3: death after
discharge from index hospital <= 30 post-operative days) and
studied patient characteristics including cause of death.
Population Studied: In 2002, 14,830 Pennsylvania residents
had isolated CABG in 62 non-federal Pennsylvania hospitals.
Principal Findings: 289 individuals died during the index
admission (in-hospital mortality rate 1.95%). 347 individuals
died within thirty days of their procedure (30-day mortality rate
2.34%). This difference is statistically significant (p=0.01). Inhospital mortality aggregates deaths in the index
hospitalization that occur both <= 30 post-operative days
(Group 1) and >30 post-operative days (Group 2), while 30day mortality aggregates deaths <=30 post-operative days that
occur both during (Group 1) and after (Group 3) discharge
from the index hospitalization. Group 1, the largest subpopulation, contains 257 Pennsylvania residents. These
individuals make up the majority (74.1%) of the deaths
captured in both measures. Differences in observed rates for
in-hospital and 30-day mortality arise from differing patient
characteristics of individuals in Groups 2 and 3.
We observed highly significant differences in patient
characteristics when comparing Groups 2 and 3: Atlas™
Probability of Death on admission (7.50% vs. 3.73%; p=0.003);
the incidence of heart failure diagnosis before/during the
index admission (59.38% vs. 3.33%; p<0.0001); the rate of
cardiac causes of death (37.50% vs. 65.55%; p=0.003); and the
rate of renal causes of death (9.38% vs. 0.0%; p=0.002). We
also observed highly significant differences when Group 3 was
compared to Group 1: Atlas™ Probability of Death on
admission (3.73% vs. 5.72%; p<0.0001); the incidence of heart
failure diagnosis before/during the index admission (3.33% vs.
41.25%; p<0.0001); and the rate of pulmonary embolism as
cause of death (5.56% vs. 0.78%; p=0.003). Finally, we
observed a number of moderately significant differences
(p<0.05) between Group 2 and Group 1.
Conclusions: Definition of outcome indicators is critical in
the measurement and reporting of provider performance.
Seemingly minor changes in definition produce significant
changes in apparent provider performance. These changes
can arise from significant differences in the patient
characteristics of the sub-populations that are aggregated to
produce the different mortality measures. Outcomes in
distinct mortality sub-populations may or may not be related
to the research objectives of particular investigations or
initiatives.
Implications for Policy, Delivery, or Practice: Researchers,
policy makers, clinicians and public health practitioners must
carefully choose outcome measure definitions that relate
directly to their public health policy objectives and quality
improvement goals.
Primary Funding Source: PA Health Care Cost Containment
Council
●Geographic Variation and Chronic Illness in Pediatric
Pneumonia Admissions
Christopher Gorton, M.D., MHSA, Jayne L. Jones, MPH
Presented By: Christopher Gorton, M.D., MHSA, Senior
Physician Consultant, Pennsylvania Health Care Cost
Containment Council, 225 Market Street, Harrisburg, PA 17101;
Tel: (717)232-6787;Email: kgorton@phc4.org
Research Objective: To compare county rates of hospital
admissions for pediatric pneumonia; To assess the
contribution of chronic conditions to county and statewide
pediatric pneumonia admission rates.
Study Design: Retrospective analysis of a Pennsylvania
statewide inpatient dataset containing 5,429 pediatric
pneumonia admissions during the last three quarters of 2003
and the first quarter of 2004,of which 4,948 (91.1%) were
included in the study. We divided the admissions into two
universes (all pneumonia and pneumonia excluding coded comorbid chronic conditions) and calculated pediatric
pneumonia admission rates for each Pennsylvania county. We
excluded transfers from other hospitals. We obtained county
child population demographics from US Bureau of Census
data and county Medicaid enrollment from PA Department of
Health data. We adjusted raw county pneumonia admission
rates using indirect standardization to account for differences
in mean age in months; percent male; percent black race; and
percent Medicaid enrollment.
Population Studied: All PA-resident children aged two
months through 17 years who were admitted to acute care
hospitals for a primary diagnosis of pneumonia (DRG 081 or
091) during the 12-month study period.
Principal Findings: The average total charges per admission
were $10,171. The Pennsylvania statewide admission rate for
all pneumonia was 156.3 admissions per 100,000 children
(adm/100kkids). There was substantial county-to-county
variation: County admission rates for all pneumonia ranged
from 77.0 adm/100kkids to 457.6 adm/100kkids. 5 of 6
regional health districts had one or more counties in the
highest admission rate quintile; 4 of 6 had one or more
counties in the lowest quintile. Six of 13 highest quintile
counties were geographically contiguous with one or more
lowest quintile counties. Pennsylvania’s two largest urban
counties fell in the middle quintile, as did several sparsely
populated rural counties.
Similar geographic patterns were seen among the 2,851
admissions that remained in the second universe after the
exclusion of 2097 records (42.4%) coded for co-morbid
chronic conditions. The Pennsylvania statewide admission
rate for pneumonia without chronic conditions was 90.0
adm/100kkids. County admission rates for pneumonia
without chronic conditions ranged from 18.3 adm/100kkids to
350.3 adm/100kkids. 62 of 67 counties (93%) remained in
the same or an adjacent admission rate quintile after children
with chronic conditions were excluded. On average, the
county admission rates for pneumonia without chronic
conditions were 58.1% of their admission rates for all
pneumonia (95% CI: 31.5 – 84.6%).
Conclusions: County pediatric pneumonia admission rates
vary widely, even among geographically contiguous and
demographically similar counties. Excluding children with comorbid chronic conditions to control for varying community
disease burden does not substantially alter county rank order
or the pattern or degree of variation in admission rates in our
study.
Implications for Policy, Delivery, or Practice:
Hospitalization is an intrusive intervention that has
substantial physical and emotional impacts on children and
their families and significant costs to public and private
payors. Researchers, policy makers and decision makers
should explore the sources of unnecessary variation in
pediatric pneumonia admission rates and seek mechanisms
to eliminate it.
Primary Funding Source: PA Health Care Cost Containment
Council
●Associations Between Public Reporting of Health
Outcomes and In-Hospital Mortality
Christopher Gorton, M.D., MHSA, Ying P. Tabak, Ph.D.,
Christopher S. Hollenbeak, Ph.D., Jayne L. Jones, MPH, Karen
G. Derby, BA, Richard S. Johannes, M.D., MS
Presented By: Christopher Gorton, M.D., MHSA, Senior
Physician Consultant, Pennsylvania Health Care Cost
Containment Council, 225 Market Street, Harrisburg, PA 17101;
Tel: (717)232-6787; Email: kgorton@phc4.org
Research Objective: There are currently few studies that have
attempted to determine whether outcomes for patients in
public reporting (PR) of health outcome states differ from
those in non-public reporting (non-PR) states. We report here
on the first phase of a multi-part study to address this
question. The objective of this study was to determine
whether in-hospital mortality rates for six important principal
diagnoses differed between patients from hospitals in
Pennsylvania (PA), a PR state, and those from hospitals in
non-PR states.
Study Design: We identified six high-frequency, high-mortality
principal diagnoses for which risk-adjustment models have
been previously reported. We used a large clinical database to
analyze discharges from 200 hospitals from PA and 42
hospitals from other non-PR states (non-PA). Using clinical
data on admission (history, physical findings, vital signs,
laboratory results) imported electronically or abstracted from
chart review, we used logistic regression models fit to 20002001 data and validated with 2002-2003 data to identify 17 to
32 variables for each condition predictive of mortality at the
p<0.05 level. In order to minimize selection bias and to
balance distribution of covariates, we used these models to
calculate in-hospital mortality propensity scores for each
discharge and then created propensity-score matched PA and
non-PA cohorts for each condition. We then computed the
odds ratio for in-hospital mortality between the matched
cohorts.
Population Studied: The data set included a total of 757,947
adult admissions in 2000-2003 for acute myocardial infarction
(AMI; n=107,280), congestive heart failure (CHF; n=261,349),
hemorrhagic stroke (n=17,414), ischemic stroke (n=98,361),
pneumonia (n=208,237), and sepsis (n=65,306) from 242
acute care hospitals. Propensity score matching yielded
62,947 matched pairs for analysis.
Principal Findings: Overall, the median age for the six
conditions ranged from 75 to 77. The crude mortality was 3.9%
(CHF), 4.5% (pneumonia), 6.3% (stroke), 10.6% (AMI), 17.9%
(sepsis), and 31.2% (hemorrhagic stroke). After propensity
score matching, PA patients showed significantly lower
mortality for AMI (OR: 0.74, CI: 0.66-0.84), CHF (OR: 0.73, CI:
0.65-0.81), hemorrhagic stroke (OR: 0.73, CI: 0.55-0.98),
ischemic stroke (OR: 0.78, CI: 0.67-0.90), and pneumonia
(OR: 0.75, CI: 0.67-0.83). For sepsis, mortality among PA
patients was not significantly different from non-PA patients
(OR: 0.98, CI: 0.84-1.2). The predictive power for all models
was excellent with C-statistics ranging from 0.78-0.91 for the
propensity-score-matched cohorts.
Conclusions: These preliminary findings suggest that public
reporting of health outcomes is associated with
improvements in in-hospital mortality in at least one state that
provides public reports of health outcomes.
Implications for Policy, Delivery, or Practice: Policy makers
should consider enhancing public reporting within current PR
states and funding research to evaluate its effectiveness.
Future research should further examine the impact of public
reporting by assessing additional conditions; outcomes such
as 30-day mortality, readmissions and length of stay; and
outcome trends over time.
Primary Funding Source: PA Health Care Cost Containment
Council and Cardinal Health
●Statewide Experience in In-Hospital Complications of
Bariatric Surgery
Christopher P. Gorton, M.D., MHSA, Jayne L. Jones, MPH
Presented By: Christopher P. Gorton, M.D., MHSA, Senior
Physician Consultant, Pennsylvania Health Care Cost
Containment Council, 225 Market Street, Harrisburg, PA 17101;
Tel: (717)232-6787; Email: kgorton@phc4.org
Research Objective: To determine rates of in-hospital
complications occurring during admissions for bariatric
surgery.
Study Design: Retrospective analysis of a Pennsylvania
statewide inpatient dataset containing administrative data on
all individuals who underwent bariatric surgery (principal
procedure 43.7, 43.89, 44.31, 44.39, 44.69, or 45.91) for a
diagnosis of obesity (principal diagnosis 278.00 or 278.01 with
DRG of 288, 292 or 293) during 2003. We developed a list of
324 candidate complication ICD-9 diagnosis and procedure
codes and grouped them into organ system based
complication groups. We performed descriptive statistical
analysis of the demographics of the population undergoing
bariatric surgery and calculated rates of individuals
experiencing specific complication codes and falling into
various complication code groups.
Population Studied: In 2003, 7,231 patients underwent
bariatric surgery in Pennsylvania general and specialty acute
care facilities. The mean age of the patients was 43.0 years
(median 43 years; range 15 to 78 years; inter-quartile range 35
to 51 years). The patients were 83% female.
Principal Findings: The majority of patients underwent Rouxen-Y Stomach Bypass (N=6,795; 94.0%), while the remainder
underwent either Gastric Banding/Vertical Banded
Gastroplasty (N=401; 5.5%) or Biliopancreatic Diversion
with/without Duodenal Switch (N=35; 0.5%). Fourteen
individuals died during the index hospitalization (0.2%). Over
twenty percent of discharges (N=1491; 20.6%) were coded
with one or more of 107 ICD-9 diagnosis or procedure codes
indicative of a complication. Patients experienced the
following rates of complication groups during or after surgery:
Procedure/Medical Care-related, N=483, 6.68%; Respiratory,
N=342, 4.73%; Cardiovascular, N=330, 4.56%; Metabolic
Disturbances, N=262, 3.62%; Digestive System, N=230,
3.18%; Hernias, N=184, 2.54%; Infection, N=124, 1.71%; and
Neurological, N=14, 0.19%. 331 individuals (4.58%) had
complications from 2 or more complication groups, including
10 of the 14 individuals who died. Patients 65 years and older
had significantly more complication groups coded than those
under 65 (35.0% vs. 20.6%; p=0.0008). Post-operative length
of stay for patients without coded complication groups
averaged 3.0 days, compared to 3.9 days for individuals with
complications from 1 complication group and 8.7 days for
individuals with complications from 2 or more complication
groups. Charges for patients without coded complication
groups averaged $31,744, compared to $39,971 for individuals
with complications from 1 complication group and $76,107 for
individuals with complications from 2 or more complication
groups. All these differences were significant at the level of p<
0.0001. Neither hospital nor surgeon bariatric case volume
was significantly correlated with numbers of patients having
complications from 1 or more complication groups.
Conclusions: Bariatric surgery was widely performed in
Pennsylvania in 2003. While the in-hospital mortality rate was
quite low, significant numbers of individuals experienced
complications. Individuals who experienced complications
had significantly longer post-operative lengths of stay and
significantly higher charges. Each individual who died had
complications from at least 1 complication group and the
majority of decedents had complications from 2 or more
groups.
Implications for Policy, Delivery, or Practice: Patients
considering bariatric surgery need to be aware of the risks of
complications as well as the potential benefits.
Primary Funding Source: PA Health Care Cost Containment
Council
●Medication Errors in Nursing Homes: A State's
Experience Implementing a Reporting System
Sandra B. Greene, DrPH, Charlotte E. Williams, MPH, Richard
Hansen, Ph.D., Kathleen D. Crook, MPA, Roger Akers, MSIS,
Timothy S. Carey, M.D., MPH
Presented By: Sandra B. Greene, DrPH, Research Associate
Professor, Health Policy and Administration, Cecil G. Sheps
Center for Health Services Research, University of North
Carolina at Chapel Hill, 725 Airport Road, CB # 7590, Chapel
Hill, NC 27599; Tel: (919) 966-0993; Fax: (919) 966-1634;
Email: SandraB_Greene@unc.edu
Research Objective: Medication errors are prevalent in
nursing home settings due in part to the large number of
medications prescribed and the at-risk nature of the
population. In an effort to improve patient safety North
Carolina passed a Medication Error Quality Initiative requiring
nursing homes to annually report errors occurring in the
facility, including information on causes, the type of staff
involved, types of drugs and impact on patients. This paper
describes the implementation process for the first annual
report and summary findings.
Study Design: We developed a medication error reporting
form that nursing homes were encouraged to use throughout
the reporting period to capture individual error events. At the
end of the reporting period the nursing homes were directed
to a secure web site where they entered aggregate information
concerning their facility’s errors. A medication error was
defined as any preventable medication-related event that
adversely affects a patient in a nursing home and is related to
professional practice, or healthcare products, procedures, and
systems, including prescribing, prescription order
communications, product labeling, packaging and
nomenclature, compounding, dispensing, distribution,
administration, education, monitoring and use.
Population Studied: All licensed nursing homes in North
Carolina, excluding nursing home beds within hospitals, are
subject to the legislation.
Principal Findings: All 384 facilities successfully reported,
with over 96% using the web-based tool. The remainder faxed
their reports. A total of 10,920 medication errors were
reported for this first report, which included 9 months of data.
Three facilities reported no errors, while one reported 1,648
errors. The median number of errors was 15. The most
frequent type of error reported was a dose omission (55.6%).
Twenty-five percent involved other dosage related errors:
wrong dose (8%), extra dose (7.5%) under dose (5.4%) and
overdose (4.5%). The vast majority of errors (79%) occurred
among patients who were cognitively impaired and not able to
direct their own care. In 5.9% of the errors, the patient was
affected in some way. The most predominant effect of the
medication error was “inadequate effect of the product” which
is consistent with the large number of dose omissions. With
respect to patient outcomes, 91.1% of the errors did not reach
the patient or reached the patient and caused no harm; 8.1%
required further monitoring; 0.5% of errors caused temporary
harm. Twenty-three patients (0.2%) required emergency
treatment because of a medication error, and for one patient
the medication error resulted in permanent harm.
Conclusions: Self reported medication errors vary widely
among nursing homes. Facility variations may be due to
differences in a facility’s comfort level in reporting errors, a
facility’s interpretation of what is an error as well as actual
variations in errors incurred in the medication use process.
Implications for Policy, Delivery, or Practice: Mandatory
reporting of medication errors should be viewed as a way to
encourage education and process changes at the facility level,
to reduce subsequent errors and improve patient safety. We
will work with facilities over the next years to improve
reporting.
Primary Funding Source: North Carolina Department of
Health and Human Services, Division of Facilities Services
●Hospital Factors Associated with Uterine Rupture and
Neonatal Birth Trauma in Women with History of Prior
Cesarean Delivery
Kimberly D. Gregory, M.D., MPH, Lisa M. Korst, M.D., Ph.D.,
Ida R. Shihady, MPH, Moshe Fridman, Ph.D., Arlene Fink,
Ph.D., Linda Burnes Bolton, Dr.Ph., RN, FAAN
Presented By: Kimberly D. Gregory, M.D., MPH, Director,
Maternal Fetal Medicine, Obstetrics & Gynecology, CedarsSinai Medical Center, 8700 Beverly Boulevard, Suite 160W,
Los Angeles, CA 90048; Tel: (310) 423-5420; Fax: (310) 4230140; Email: gregory@cshs.org
Research Objective: Nationally, the risk of uterine rupture is
0.7% for women attempting vaginal birth after cesarean
(VBAC), and the risk is associated with patient-specific clinical
factors. This study attempts to identify hospital level risk
factors for uterine rupture and newborn birth trauma and
specifically evaluates clinical policies that might be associated
with low risk of adverse maternal and neonatal outcomes.
Study Design: We conducted structured interviews of nurse
managers at hospitals in California reporting greater than 50
deliveries in 2002 regarding their hospital resources and
clinical policies. We developed a “willingness to intervene”
construct to define hospitals with practices that supported
nurses calling physicians for specific clinical scenarios. This
construct was based on responses to 3 questions using a 4point Likert scale (1=Very Likely to Call Physician, 4=Very
Unlikely to Call Physician). We obtained maternal and
newborn linked hospital delivery discharge and birth certificate
data for 2002 from the Office of Statewide Health Planning &
Development. We report the odds ratio for uterine rupture
and newborn birth trauma based on hospital structural
(hospital ownership, teaching status, delivery volume, 24 hour
anesthesia) and clinical variables (allow VBAC Y/N,
“willingness to intervene”), and case mix adjustments (age
group, medical complications, route of delivery) for singleton
liveborn deliveries using hierarchical logistic regression
techniques. This study was IRB approved.
Population Studied: Hospitals in California providing care to
women with a history of prior cesarean (and their newborns)
during 2002.
Principal Findings: The response rate was 84% (225/268
hospitals completed the interview), and 74% of hospitals
allow VBAC. There were 400,702 singleton linked records, of
which 14% (55,154) were to women with a history of prior
cesarean delivery. The VBAC attempt rate was 26%, the VBAC
success rate was 17%. The uterine rupture rate was 0.3%.
Among hospitals that allow VBAC, the following variables were
associated with uterine rupture on univariate analyses:
hospital ownership, teaching status, VBAC allowed, and
“willingness to intervene.” After controlling for patient clinical
factors, hospital type, VBAC allowed, and “willingness to
intervene” were still significantly associated with uterine
rupture. State hospitals had increased risk and churchaffiliated hospitals had decreased risk relative to for-profit
hospitals. The odds of uterine rupture in hospitals with
clinical practices that support “willingness to intervene”
(median=2; range 1-4) were 1.46 times lower (95% confidence
region, 1.11-2.12) per unit increase in the “willingness to
intervene” factor. After excluding births with neonatal
anomalies/malignancies, 51,638 deliveries were used to model
neonatal trauma. The odds of neonatal trauma in hospitals
with clinical practices that support “willingness to intervene”
were 1.13 times lower (95% confidence region, 1.01-1.26) per
unit increase in this factor.
Conclusions: Previous studies have focused on patientspecific risks for uterine rupture including obstetric
management. Current recommendations have emphasized
availability of hospital resources to maximize patient safety for
women attempting VBAC.
Implications for Policy, Delivery, or Practice: Our study
suggests that there may be specific nursing practices that
contribute to VBAC safety, risk of uterine rupture, and
neonatal birth trauma.
Primary Funding Source: AHRQ, RO1HS11334
●Factors Affecting Patient Outcomes in Ambulatory
Surgery
Brian Harahan, BA, Maureen Smith, M.D., MPH, Ph.D., Ann
Hundt, MS, Ph.D., Scott Springman, M.D., Pascale Carayon,
Ph.D.
Presented By: Brian Harahan, BA, Trainee, Population Health
Sciences, University of Wisconsin-Madison Medical School,
610 Walnut Street, 707 WARF Building, Madison, WI 53726;
Tel: (608)263-2880; Fax: (608)263-2820; Email:
bjharahan@wisc.edu
Research Objective: Over 60% of surgical procedures in the
U.S. are performed in ambulatory settings. This growing
trend places increasing amounts of responsibility on patients
to self-manage their follow-up care as more complex
procedures are performed and more complicated patients get
these procedures. Increased age and comorbidity may
predispose a patient to adverse outcomes. Our objective is to
explore the effect that age and comorbidity have on patient
response to post-surgical symptoms.
Study Design: Data were obtained from patients, N=342,
undergoing ophthalmic, open joint, otolaryngological, or intraextra-abdominal ambulatory surgery from four outpatient care
centers in a mid-sized midwestern community. Medical
record abstraction was used to verify eligibility and collect data
on patient comorbidities. Nurses queried patients by
telephone at least seven days after discharge on the incidence
of twenty-one possible symptoms and whether the patient
contacted a provider for each symptom (Hundt et al., 2004).
Symptoms were categorized prior to analysis into high risk
(e.g., shortness of breath, chest pain), moderate risk (e.g.,
eating problems), and low risk (e.g., coughing) based on
expert opinion. Data on comorbidities were summarized
using the Charlson Comorbidity Index. The dependent
variable was, “Did you attempt to contact your regular doctor,
surgeon's office or the outpatient surgery center, for each
symptom.” Analysis was conducted at the symptom level
accounting for clustering within patients. Logistic regression
was used to examine the relationship of the dependent
variable to symptom risk, age and the Charlson index. The
final model controlled for gender, education, type of
procedure, and presence of a permanent caretaker. Variables
examined but not included in the final model (P-value > 0.20)
were self-reported health status, ASA status, patient
information and preparedness for the surgery, and type of
anesthesia.
Population Studied: 342 patients aged 18-95, 39% greater
than 65, responded to the survey. 150 patients, 30% of whom
were greater than 65 years old, were selected for analysis
because they contacted a health provider for a new or
worsening symptom. 284 new or worsened symptoms were
reported at the surgery center or after discharge.
Principal Findings: Twenty-four percent of symptoms were
reported by respondents older than 65 years. The probability
of provider contact increased significantly for high risk
compared to low risk symptoms (odds ratio=3.76, 95%
confidence interval=1.53-9.28). The data are suggestive that
patients over the age of 65 years are less likely to contact their
provider for high risk symptoms, but this study lacked the
statistical power to permit a firm conclusion. There was no
evidence that patients with higher comorbidity levels were
more likely to contact a provider for high risk symptoms.
Conclusions: High-risk symptoms were more likely to lead to
contact with a provider when compared to low risk symptoms,
with suggestive evidence that older patients were less likely to
respond appropriately to high risk symptoms.
Implications for Policy, Delivery, or Practice: Older patients
are likely at higher risk for adverse events related to postsurgical complications and further research should evaluate
whether older patients are less likely to respond appropriately
to high risk symptoms.
Primary Funding Source: AHRQ
●Mitigating Factors in Near Miss Events Reported to a
Patient Safety Reporting System: A Qualitative Study From
the ASIPS Collaborative
Daniel Harris, Ph.D., Wilson D. Pace, M.D., Douglas H.
Fernald, MA
Presented By: Daniel Harris, Ph.D., Senior Project Director,
Center for Healthcare Research, The CNA Corporation, 4825
Mark Center Drive, Alexandria, VA 22311; Tel: (703) 824-2283;
Fax: (703) 824-2511; Email: harrisd@cna.org
Research Objective: Medical error reporting systems are
advocated as strategies for improving healthcare quality and
safety; however, a growing concern exists regarding how
analysis of reports collected by these systems can be
translated into improved healthcare practices. We analyzed
mitigating factors described in near miss events reported to
one such system to see if focusing on “what went right” in
these events can identify potential intervention strategies to
break the chain of events that lead to patient harm.
Study Design: The three-year Applied Strategies for
Improving Patient Safety (ASIPS) demonstration project
developed a patient safety reporting system (PSRS) to collect
anonymous and confidential narrative medical error reports
voluntarily submitted by clinicians and staff in two PracticeBased Research Networks (PBRNs) in Colorado. We
qualitatively coded factors contributing to and mitigating the
cascade of events in the 608 incident reports submitted
during the first two years of the project, and how these factors
affected whether the incident did or did not result in patient
harm. Using Atlas.ti software, we analyzed mitigators by
locus, agent, and agency in different types of reported events,
and compared harmful events with no-harm near miss events
to identify how mitigators intervened in the event process to
protect patient safety.
Population Studied: Over 600 error reports submitted by
primary care providers, other clinical staff, and non-clinical
staff from 34 primary care practices affiliated with two PBRNs
serving a largely Medicaid patient base in urban and rural
communities in Colorado.
Principal Findings: We identified three major categories of
mitigating factors: actions deliberately undertaken by specific
actors with the intent of mitigating an incident, systems
factors whereby the operation of existing procedures or
technological systems mitigates the incident, and serendipity
that leads to mitigation by chance without deliberate action or
system performance. These factors are not mutually exclusive
and can build on each other. We also identified primary
subcategories of each mitigating factor, in which types of
incidents they typically occur, and where in the chain of events
they typically occur. We found that communication chains
and transition points in long event chains are particularly rich
opportunities for error and mitigation of error. Finally, we
found that the existence of various mitigating and contributing
factors differentiated between no-harm and harmful incidents,
respectively, that were otherwise similar and shared attributes
in common. Comparing mitigators and contributors revealed
system vulnerabilities and suggested interventions.
Conclusions: A wide range of factors acted to mitigate the
cascade of events surrounding medical errors and to prevent
errors from reaching or harming patients. While some
mitigators appear across event types, others tend to be more
associated with specific types.
Implications for Policy, Delivery, or Practice: Medical error
reporting systems provide opportunities for taking advantage
of “the gift of failure.” Focusing on the mitigating factors of
what went right in near miss events rather than exclusively on
what went wrong expands the opportunity to identify
interventions that can address system vulnerabilities that lead
to patient harm.
Primary Funding Source: AHRQ
●Reorganizing Community Leg Ulcer Care: A Pre-Post
Implementation Study
Margaret B. Harrison, RN, Ph.D., Ian D. Graham, Ph.D., Karen
Lorimer, RN, MscN, Elaine Friedberg, RN, MHA, Tadeusz
Pierscianowski, MBBS, FRCPC, FAAD, Tim Brandys, MD,
FRCSC
Presented By: Margaret B. Harrison, RN, Ph.D., Associate
Professor, Faculty of Health Science, Schoolf of Nursing &
Community Health and Epidemiology, Queen's University, 78
Barrie Street, Kingston, Ontario, Tel: (613) 533-6000 ext 74760;
Fax: (613) 533-6331; Email: harrisnm@post.queensu.ca
Research Objective: The objective was to determine the
health outcomes and efficiencies of the impact of
implementating a nurse-led service supporting guidelinedriven practice for the popoulation with leg ulcers.
Study Design: A prospective, one year pre-post design was
used to evaluate the new service model which involved both
organizational and clinical changes. Outcomes included
healing rates at 3 months, and resource use (nursing visits,
supplies).
Population Studied: Individuals newly referred to homecare
for management of their leg ulcer in one large homecare
authority (mixed urban-rural setting of approximately one
million people) were eligible to participate in the study. Study
inclusion criteria included: English or French speaking, having
an ulcer (open wound) below the knee of venous or mixed
etiology (venous-arterial). Community leg ulcer care was
delivered in either the individual’s home or in one of two
nurse clinics.
Principal Findings: There was a statistically significant
improvement in 3-month healing rates increasing overall from
23.1% in the pre-period (18/78) to 55.6% (100/180) in the postperiod. The median number of nursing visits declined from 37
to 25 over the period(z -2.047, p = 0.041) while the median
supply cost per case went from $1923 to $406 per case (z 2.828, p = 0.005).
Conclusions: Reorganization of care for the population with
leg ulcers was associated with improved healing and efficient
use of nursing visits. More people could be cared for with the
same allocation of home care dollars.
Implications for Policy, Delivery, or Practice: The significant
impact that the intervention had was due to two major
elements; using a quality guideline for clinical care and
importantly re-organization with a service model to support
evidence-based practice with community leg ulcer care. As an
exemplar of a population requiring complex community care,
the approach we describe using population-based planning
and evidence-based practice, may have broader application.
Primary Funding Source: Ontario Ministry of Health and
Long Term Care
●Improving Patient Safety in Hospitals: Contributions of
High Reliability Organizations and Normal Accident
Theory
Michael Harrison, Ph.D., Michal Tamuz, Ph.D.
Presented By: Michael Harrison, Ph.D., Senior Research
Scientist, Center for Delivery, Organization, and Markets,
Agency for Healthcare Research & Quality, 540 Gaither Road,
Rockville, MD 20850; Tel: (301)427-1434; Fax: (301)427-1430;
Email: mharriso@ahrq.gov
Research Objective: To identify the distinctive contributions
of two leading organizational approaches to safety in highhazard industries. To specify conditions under which these
theories apply to hospitals.
Study Design: A qualitative, interview-based study of learning
from medication errors in hospitals provides examples for our
review of empirical and theoretical studies of high reliability
organizations (HRO) and normal accident theory (NAT) in
hospitals and high-hazard industries.
Population Studied: Three tertiary care teaching hospitals
(n=342 healthcare professionals and administrators)
Principal Findings: (1)Developing Culture: HRO emphasizes
that organizational culture influences safety. Researchers
advocate developing a “culture of mindfulness” through
noticing hazards. Unfortunately, planned cultural change is
usually slow, hard to manage, and risky. Interviews with
clinicians illustrate how turnover, employment arrangements,
and professional education undermined development of
mindful cultures. (2)Designing Organizational Structures and
Technologies: NAT highlights how organizational structure
and technology contribute to errors by intensifying interactive
complexity and coupling. Hospitals are characterized by
interactive complexity, but traditionally their clinical processes
have been loosely coupled. Loose coupling leads to
inefficiency, but provides opportunities to catch errors before
they cause harm. Tradeoffs in loose coupling were evident in
interviews with providers who order, dispense, and administer
medications. When adverse events do occur to individual
patients in loosely-coupled health systems, they do not usually
spread. Recent technological developments, such as
computerized order entry, tighten links among units and
functions (e.g., ordering and dispensing medication).
Although these information technologies improve efficiency
and routine safety, they may also increase the chances of
errors spreading across patients and units. (3)Specifying
Conditions for HRO Solutions: HRO advocates continuous
training, but training cannot solve problems from poorly
designed systems. NAT suggests that other HRO solutions
might actually reduce hospital safety. For instance, adding
redundancies by increasing human or technological checks
can make processes harder to understand and reduce
clinicians’ vigilance. In interviews, pharmacy managers
observed that redundancy could lead to complacency and
medication errors. (4)Implementing Changes: NAT and HRO
agree on some safety solutions, but differ on how to
implement them. Both perspectives recognize the importance
of rewarding safety-promoting activities. HRO assumes that
safety cultures readily create appropriate reward systems. NAT
sees developing reward systems as problematic. In practice,
employees are subject to conflicting expectations and rewards.
As NAT anticipates, in interviews nursing managers reported
being subject to conflicting incentives for reducing errors and
encouraging error reporting.
Conclusions: NAT provides a more realistic image of hospital
operations than HRO and reveals barriers to implementing
HRO solutions. The cultural changes envisioned by HRO may
slowly yield safety benefits; NAT highlights the immediate
political impact of competing goals and conflicting reward
systems.
Implications for Policy, Delivery, or Practice: HRO
advocates the long-term advantages of promoting a culture of
mindfulness and immediate benefits of conducting safety
audits. NAT underscores the potential contribution of safety
monitoring and promotion by external stakeholders (e.g.,
JCAHO) to tip the internal balance towards patient safety.
NAT cautions against adding redundancies and warns of
possible risks of tightly coupling operations through
information technology or efficiency measures.
Primary Funding Source: AHRQ, Aetna Foundation's Quality
Care Research Fund
●Developing and Validating an Algorithm to Identify
Pregnancy Episodes in an Integrated Health Care Delivery
System
Mark C. Hornbrook, Ph.D., Evelyn Whitlock, M.D., MPH,
Cynthia Berg, M.D., MPH, William Callaghan, M.D., MPH,
Carol Bruce, MPH, Rachel Gold, Ph.D., MPH
Presented By: Mark C. Hornbrook, Ph.D., Cnief Scientist,
Center for Health Research, Kaiser Permanente Northwest,
3800 North Interstate Avenue, Portland, OR 97227-1110; Tel:
(503) 335-6746; Fax: (503) 335-2428; Email:
mark.c.hornbrook@kpchr.org
Research Objective: To validate a computerized algorithm for
grouping appropriate ambulatory and inpatient care
encounters into pregnancy episodes. The algorithm was
compared with the gold standard of mothers medical records.
Study Design: We developed a computerized algorithm to
identify key parameters of all types of pregnancy episodes,
including live births, stillbirths, therapeutic abortions,
spontaneous abortions, ectopic pregnancies, and
trophoblastic disease. To assess the algorithm’s accuracy, we
applied it to a sample of automated medical records of
women of childbearing age who were members in 1998-2001
of an integrated health care delivery system. The algorithm
searches for diagnosis, procedure, and Diagnosis Related
Groups as well as test results associated with pregnancy. It
uses a logical hierarchy to define the episode parameters: start
and stop dates, pregnancy outcomes, outcome procedure,
gestational age, fetal count, and phases of pregnancy. We
validated the algorithm’s assessment of the pregnancy
histories of a sample of subjects with assessments obtained
by blinded medical records abstractors using actual medical
records.
Population Studied: Members of Kaiser Permanente
Northwest, a prepaid group practice HMO with about 455,000
members, constituted our study population. Within this
population, the algorithm searched for evidence of pregnancy
episodes among all women of childbearing age, 12 to 55 years
old, who were KPNW members for at least 42 continuous
days at any time during the study period, January 1, 1998,
through December 31, 2001. The sample size was 511 cases.
We included only those episodes fully contained within the
study period and the mothers health plan eligibility period.
Principal Findings: The tested version of the algorithm
correctly identified 97 percent of pregnancy episode end dates
and achieved exact or close agreement for 99 percent of
pregnancy outcome types. It achieved 96 percent agreement
on gestational age among episodes agreeing on both end date
and outcome type. Using the results obtained from the
validation study, we modified the algorithm and improved the
classification results.
Conclusions: Using comprehensive medical record and
administrative data to retrospectively identify the beginning
and end of pregnancy regardless of pregnancy outcome allows
for defining a window to explore maternal pregnancy
complications. Understanding these complications may
facilitate design and improvement of obstetrical care in health
care systems. Our pregnancy episode grouper makes use of
more information than various commercial episode grouper
software.
Implications for Policy, Delivery, or Practice: While more
than six million women in the US become pregnant each year,
efforts to assess the extent and nature of morbidities
experienced during pregnancy have been limited to analyzing
hospitalization data. Hospitalization data likely capture only
severe and intrapartum morbidities and do not identify
whether morbidities appeared gradually over pregnancies.
These shortcomings are salient because recent changes in
medical practice have resulted in increased treatment and
better outcomes for maternal complications in outpatient
settings. A pregnancy episode framework is needed to
measure morbidity profiles in the prepartum, intrapartum,
and postpartum phases.
Primary Funding Source: CDC
●Identification of Actionable Items for Systems
Improvement through Executive WalkroundsÓ in a
Pediatric Hospital
Lisa Horowitz, Ph.D., MPH, Melissa McClay, MPH, Allan
Frankel, M.D., James Mandell, M.D., Eileen Sporing, MSN,
RN, Don Goldmann, M.D.
Presented By: Lisa Horowitz, Ph.D., MPH, Quality
Improvement Consultant, Department of Patient Safety and
Quality, Children's Hospital Boston, 300 Longwood AvenueWolbach 001, Boston, MA 02115; Tel: (617)355-7447; Fax: (617)
730-0632; Email: lisa.horowitz@childrens.harvard.edu
Research Objective: To use comments elicited during Patient
Safety Executive Walkrounds to identify themes and actionable
items in order to resolve potential safety concerns.
Study Design: As part of a multi-center study, the Children’s
Hospital Executive Walkrounds Team (EWT) visited a different
unit in a large pediatric teaching hospital for one hour three
times per month. Three rotating executives conducted semistructured interviews with groups of staff including staff
physicians, physician trainees, nurses and pharmacists, using
standardized questions and explicitly encouraging staff to
share concerns free of retribution. To identify key themes and
actionable items, all staff comments were recorded verbatim
by a scribe during the rounds and entered as text into a
database. Comments were then numbered and coded by two
observers into thematically distinct categories, and
summarized in an Action Plan – a key tool for ensuring follow
up of actionable items. One week later a Debrief Session
including the EWT and clinical leaders from the specific unit
was convened to discuss each item in the Action Plan,
determine actionability, and assign responsibility for resolving
the item. 12 weeks later, staff who participated in Executive
Walkrounds received feedback and progress-to-date in the
form of the updated Action Plan. Each comment was then
labeled “completed,” “in progress,” or “tracked in database,”
depending on the action taken.
Population Studied: Three rotating senior hospital executives
and 550 multidisciplinary staff members, including staff
physicians, physician trainees, nurses, and pharmacists, of a
large urban teaching hospital participated from February
through December 2004.
Principal Findings: Twenty-seven Executive Walkrounds were
conducted in unique clinical areas. Attendance at Executive
Walkrounds ranged from 8-47 (mean 12) staff per round. Of
384 items, 14 thematically distinct categories were identified
by 2 observers, with an agreement rate of 90%. The top five
themes were: Workload/Resources (89% of Executive
Walkrounds), Communication (78%), Facilities (78%),
Information Systems (63%), and Documentation (52%).
Pediatric-specific concerns, such as child safety in medical
exam rooms and sterilizing toys in the waiting area, were
present in 15% of the comments. In a sub-sample of the first
12 Executive Walkrounds, 83% of items incorporated into the
database were considered “actionable” and assigned to
clinical leaders for resolution. At the end of 12 weeks, 32% of
the actionable items were deemed “completed.” Efforts to
resolve the remainder of the items are ongoing.
Conclusions: Concerns regarding workload, resources and
communication were identified frequently by staff as
impacting patient safety. Actionable items were identified and
addressed through the Action Plan. One third of the items
were resolved comprehensively.
Implications for Policy, Delivery, or Practice: Conducting
Patient Safety Executive Walkrounds is feasible and effective in
a large tertiary care teaching hospital, and allowed this
institution to recognize and address systems issues with the
potential to compromise high quality healthcare delivery. The
Action Plan is an important tool that helps in assigning
accountability for resolving staff concerns, allows for ease in
feeding back information to staff, and enables the Executive
Walkrounds Team to track what would have been otherwise
unmanageable data.
Primary Funding Source: No Funding Source
●First Look at a Composite Safety Measure
Robert Houchens, Ph.D., Chinliu Zhan, M.D., Ph.D., Rosanna
Coffey, Ph.D., Edward Kelley, Ph.D.
Presented By: Robert Houchens, Ph.D., Director, Statistical
Informatics, Medstat, 5425 Hollister Avenue, Suite 140, Santa
Barbara, CA 93111-2348; Tel: (805) 681-5867; Fax: (805) 6815899; Email: bob.houchens@thomson.com
Research Objective: To develop a composite (aggregate)
safety index using as components the Agency for Healthcare
Research and Quality’s (AHRQ) Patient Safety Indicators
(PSIs). This composite is designed as a test for developing
composite measures for the National Healthcare Quality
Report and the National Healthcare Disparities Report.
Study Design: We applied AHRQ’s PSI software to five years
of the Health Care Utilization Project (HCUP) Nationwide
Inpatient Sample (NIS), 1998 – 2002. We used estimates of
excess mortality and excess length of stay (LOS) attributable
to patient safety events, similar to those developed by Zhan
and Miller (JAMA, 2003), as relative weights for different types
of safety events to produce a composite measure of inpatient
safety. We also developed standard error estimates. We used
these estimates to assess trends in inpatient safety
nationwide, and to statistically compare patient safety
performances across various demographic groups.
Population Studied: Inpatients discharged from short-term
community hospitals in the United States between 1998 and
2002.
Principal Findings: The patient safety composites have
sound statistical properties and they are easy to understand by
broad audiences. The composite indices showed substantial
variation in inpatient safety over time and across patient
groups, in terms of both excess mortality and excess LOS.
“Drill-down” analyses further revealed many important
component PSIs driving the overall safety differences.
Conclusions: Composite indices are feasible, meaningful, and
useful in inpatient safety assessment, especially for trend
analyses and for comparing the level of inpatient safety among
demographic groups.
Implications for Policy, Delivery, or Practice: It is difficult to
use the numerous existing patient safety measures to describe
inpatient safety in the United States. Composite measures
represent important communication tools. Such indices
could help to assess overall progress in safety improvement,
to identify vulnerable subgroups, and to evaluate safety
improvement strategies.
Primary Funding Source: AHRQ
●Advance Care Planning: Does it Make a Difference?
Ronda Hughes, Ph.D., MHS, RN, Anne Wilkinson, Ph.D., Karl
Lorenz, M.D., Joann Lynn, M.D.
Presented By: Ronda Hughes, Ph.D., MHS, RN, Senior
Health Scientist Administrator, Center for Primary Care,
Prevention, & Clinical Partnerships, AHRQ, 540 Gaither Road,
Rockville, MD 20850; Tel: (301) 427-1578; Fax: (301) 427-1595;
Email: rhughes@ahrq.gov
Research Objective: To examine the research literature on
end-of-life care to determine the impact of advance directives
(including living wills and do not resuscitate orders) on those
at the end-of-life, using studies involving advance-care
planning.
Study Design: As part of a systematic review of end-of-life
care, we reviewed articles that evaluated advance directives
and other aspects of advance care planning at the end of life.
Population Studied: Sources for our review included
Medline, the Cochrane Database of Reviews of Abstracts of
Effects (DARE), the National Consensus Project for palliative
care, and several recent systematic reviews from both Health
Canada and National Institute for Clinical Excellence (NICE),
United Kingdom. The searches were limited to published
articles in the English language (1990 – 2004), involving adult
human subjects. Research was included if it targeted patients
or families, not only reporting about clinicians.
Principal Findings: From 24,423 total citations, 1,274 articles
were reviewed, including 95 systematic reviews, 134
intervention, and 682 observational studies. We identified 10
systematic reviews, 12 intervention studies, and 17
observational studies on patient or caregiver satisfaction. The
effectiveness of advance directives and advance care planning
is supported by little reliable scientific evidence of improving
outcomes. Targeted communication and planning can
improve patient and family satisfaction. However, high quality
research designs have not been used consistently and, when
applied, have shown quite modest effects, even upon
increasing the rate of making decisions in advance. Whether
improved advance care planning actually improves the
experience for patients and their families has only thin and
equivocal evidence. The evidence provides several key
insights involving advance care planning. First, advance care
planning should reflect changing preferences and
circumstances because patient’s preferences can change over
the course of their illness. Second, when clinicians and
families understand and agree with a patient’s preferences
and prognosis, patients are more likely to experience preferred
outcomes. Third, physical and psychosocial support for
patients and their families is needed and can improve
communication and decision-making among clinicians,
patients, and families. Fourth, interventions limited to one
type of strategy and one site of care as well as having few
study subjects are not likely to change care patterns or have
long term impact; which describes about half of the studies
reviewed.
Conclusions: This systematic review identified a very large,
diverse literature reflecting the tremendous growth and
importance of the field of end of life care over the last decade.
This review illuminates strengths of the field as well as
opportunities for research. We identified equivocal evidence
supporting the use of advance directives. Advance care
planning requires a well-informed patient or their surrogate to
make decisions about future care so that treatments
undertaken during a future period of decisional incapacity will
still be in accord with the patient’s preferences.
Implications for Policy, Delivery, or Practice: Studies
examining the effect of advance care planning on subsequent
health care utilization. Inconsistent evidence supports
advance directives, but evidence does support advance care
planning to support patient’s preferences and as a means to
improve patient-clinicians communication.
Primary Funding Source: No Funding Source
●Valued Aspects of Ambulatory Health Care: Comparing
Patients and Health Care Providers Based on Office Visit
Accounts
Margarita Hurtado, Ph.D., MHS, Karen Frazier, BA, Anna
Levin, BA, Erica Eisenhart, BA, Karen S. Shore, Ph.D.
Presented By: Margarita Hurtado, Ph.D., MHS, Principal
Research Scientist, American Institutes for Research, 10720
Columbia Pike, Suite 500, Silver Spring, MD 20901; Tel: (301)
592-2215; Fax: (301) 593-9433; Email: mhurtado@air.org
Research Objective: To compare patient and provider
perceptions regarding high quality ambulatory care based on
office visit accounts, and examine if the quality aspects
identified differ between good and poor visits. This is part of a
larger study to identify quality improvement levers based on
specific provider behaviors and CAHPS survey data.
Study Design: We conducted semi-structured telephone
interviews with patients and health care providers to obtain
detailed personal accounts of specific office visits they
considered of good or poor quality. After transcription, we
used Atlas.ti to code the text regarding aspects of quality care,
stage of encounter, provider type and nature of the experience
(positive or negative). We defined the initial coding scheme
based on processes of care in ambulatory settings and
previous research, and refined it based on the narratives.
Intercoder reliability was 89%. Using content and frequency
analysis, we examined the relative saliency of quality-related
behaviors at three levels: office visit, individual behaviors and
respondent.
Population Studied: 82 adult patients and 16 health care
providers (physicians and nurse practitioners) described 536
office visits (417 and 91, respectively, mean = 5.5). Office visit
accounts included 2505 specific behaviors.
Principal Findings: Both patients and health care providers
described office visits where behaviors related to clinical skills
(e.g., problem resolution, skill in performing procedures,
history-taking, appropriate prescribing) were the most
frequently mentioned aspect of quality (73% of visits).
Providing health-related information was also mentioned by
patients and providers in equal proportions, but less often
(43% of visits). For patients, visit accounts included behaviors
related to interpersonal and communication skills (e.g., careful
listening, clear explanations, courtesy, respect) almost as
frequently (64% of visits) as clinical skills, but for providers,
they were among those less frequently mentioned (16% of
visits). In contrast, aspects related to coordination of care
were mentioned more frequently by providers (41% of visits)
than by patients (29% of visits). Access-related aspects (e.g.,
waiting time, visit duration, financial access) were mentioned
in about half of the visit accounts by patients and one-third of
those by providers. Aspects related to shared decision-making
were seldomly mentioned by either patients (15% of visits) or
providers (10% of visits). Rarely mentioned were
administrative and office environment aspects. Most
behaviors mentioned occurred during the consultation phase
of the visit. We did not find any major differences in the
aspects of care mentioned in accounts of good quality visits as
compared to poor ones.
Conclusions: Both patients and providers place a high and
equal emphasis on clinical skills when describing good and
poor quality office visits. To a lesser degree, both also value
the provision of health-related information. The largest gap
between the two is with respect to the provider’s interpersonal
and communication skills, which were mentioned 4 times
more frequently in patient accounts than in provider accounts.
Implications for Policy, Delivery, or Practice: This study
provides further evidence that quality improvement of
ambulatory care should emphasize provider interpersonal and
communication skills. Increased awareness of the importance
of shared decision making among providers and patients is
also needed.
Primary Funding Source: AHRQ
●Patient Safety in Primary Care: the Relationship Between
Human Error Consequence and Physician Discretion
Sylvia Hysong, Ph.D, Richard Best, Ph.D., Frank Moore, Ph.D.
Presented By: Sylvia Hysong, Ph.D, Research Health
Scientist, VERDICT, Audie L. Murphy Veterans Memorial
Hospital, 7400 Merton Minter Boulevard -11C6, San Antonio,
TX 78229; Tel: (210)617-5300 x5229; Email:
shysong@verdict.uthscsa.edu
Research Objective: This research is part of a larger project,
the prime objective of which is to systematically analyze the
work of primary care health delivery in the VA. Key among the
objectives is to create optimization models to demonstrate
potential staffing mix redesigns for increased efficiency,
employee satisfaction, and patient satisfaction. The present
study specifically examines the relationship between the
consequence of error in task performance and the degree of
discretion in performing the task.
Study Design: We used Functional Job Analysis (FJA Fine &
Cronshaw, 1999), to empirically examine task statements as
the smallest identifiable units of work analysis. Task
statements collectively describe the work of primary care, and
individually can be analyzed for levels of complexity across
multiple dimensions (e.g., data, reasoning, autonomy, human
consequence for error (HEC), etc). Primary care personnel
from five VA medical centers participated in two-day focus
groups to generate task statements descriptive of primary care
in the VA. After editing and rating the statements per FJA
protocol, statements were distributed to all primary care
personnel in 9 sites (including the original five) for
verification; participants indicated the frequency and duration
of each task that they personally performed.
Population Studied: We analyzed the jobs of physicians,
advanced practitioners, RNs, LVNs, health technicians, clerks,
and pharmacists. Each focus group contained 6-8 subject
matter experts from each job title across 5 sites. Surveys were
then distributed to the entire complement of primary care
employees across the 9 sites.
Principal Findings: Approximately 65% of the tasks
performed by physicians have potentially serious
consequences if errors are made, while only 1 – 4% of the
tasks performed by RNs, LVNs, Health Techs, and clerks carry
the same severity of error consequences. These data suggest
the most consequential tasks also emphasize the importance
of decision-making processes: HEC, reasoning, and data
requirements of physician tasks were significantly correlated
(r’s = .65, .51, and.74 respectively). On average, tasks with very
high HEC ratings were also below the midpoint on discretion,
thus suggesting that the most consequential physician tasks
also leave little room for judgment. This relationship pattern
also generally held true for RNs, LVNs, and Health Techs.
Conclusions: Physicians have a high amount of work with
potential for harm compared to other primary care personnel,
thus potentially increasing error probability. The VA has
addressed this concern via strategies that minimize subjective
decision-making. For example, practice guidelines are
promoted as prescriptions for ensuring high quality of care
with maximum patient safety. Our research supports this
observation, though results are preliminary at present.
Implications for Policy, Delivery, or Practice: The inverse
relationship between worker discretion and human error
consequence is reflective of a trend towards regulating clinical
practice through guidelines and pathways as decision-making
tools. However, health care settings are characteristically
complex, adaptive systems that co-evolve with their
environment. Thus, algorithms or practice guidelines may be
helpful but insufficient strategies for healthcare quality and
patient safety. Initiatives that help cultivate good clinical
decision-making may be a complementary strategy for
improving patient safety. Task analyses can help clinics make
better, evidence-based decisions about specific strategies and
initiatives.
Primary Funding Source: VA
●Resident Physicians Report on Adverse Events and Their
Causes
Reshma Jagsi, M.D., DPhil, Barrett T. Kitch, M.D., MPH,
Debra Weinstein, M.D., Eric Campbell, Ph.D., Matthew
Hutter, M.D., Joel Weissman, Ph.D.
Presented By: Reshma Jagsi, M.D., DPhil, Resident Physician,
Radiation Oncology, Massachusetts General Hospital, Cox 3,
100 Blossom Street, Boston, MA 02114; Tel: (617)470 2091;
Fax: (617)726 3603; Email: rjagsi@partners.org
Research Objective: Resident physicians are front-line
providers of medical care in inpatient and ambulatory settings,
and thus have a unique vantage point from which to comment
upon patient safety. This study explored residents’
experiences regarding patient safety and perceived causes of
adverse events.
Study Design: A questionnaire assessed residents’
experiences with adverse events (AEs), mistakes, and nearmisses, as well as opinions regarding the causation of these
events. Data were gathered about most recent event, and the
number of events in the last week. Data also were gathered
on post-graduate year, specialty, chronic fatigue, and
characteristics of the last week of practice including setting of
rotation, patient load, and work hours.
Population Studied: Responses were obtained from 821
residents and fellows in the 76 accredited programs at the
Massachusetts General Hospital and Brigham and Women’s
Hospital in 2003 (response rate = 57%). Analyses were
limited to the 689 responses from trainees in a primarily
clinical year of training.
Principal Findings: Over half (55%) of the trainees in this
study reported ever having a patient under their care who
suffered an AE. The most common types of AE were
procedural and medication-related. Over two-thirds were
considered serious. Respondents in surgical specialties who
reported events described them as significant or worse 83% of
the time, compared with 66% for medical and 67% for
hospital-based specialties (p=.002). Of the most recent AEs,
24% were attributed to mistakes, and trainees felt at least
partially responsible for 77% of these mistakes. The most
common reasons for the mistakes as perceived by trainees
were working too many hours (19% of mistakes), inadequate
supervision (20%), and problems with hand-offs (15%)
In the last week, 114 respondents (18%) reported having a
patient under their care suffer an AE, 42 (6%) reported AEs
involving a mistake for which they felt responsible, and 141
(23%) reported near-miss incidents for which they felt
responsible. In multivariate analyses, significant predictors of
AEs in the past week were inpatient rotation, procedural
specialty, and duty hours > 80 in the past week (all p<.05).
Predictors of near-miss errors were inpatient rotation, PGY-1
status, and days of fatigue in the past month (all p<.05).
Conclusions: Information obtained from front-line trainees
suggest that adverse events are commonly encountered and
often associated with errors. By showing a direct correlation
between fatigue and errors in a broad spectrum of specialties
other than medicine, and in settings outside of the ICU, these
findings add to evidence from studies by Landrigan, et al, and
Lockley, et al, that suggested that long work hours may lead to
increased attentional failures and errors.
Implications for Policy, Delivery, or Practice: Eliciting
residents’ perspectives is important because residents may
perceive events, actions, and causal relationships that chart
reviewers or observers cannot. This study provides essential
data for designing patient safety interventions and provides
evidence supporting recent policies aimed at reducing
resident work hours.
Primary Funding Source: Anonymous Donor
●U.S. Providers Reported Use of the Human
Papillomavirus (HPV) Test for Recommended and NonRecommended Indications: Results from a Nationally
Representative survey, 2004
Nidhi Jain, M.D., MPH, Kathleen Irwin, M.D., NPH, Rheta
Barnes, MSN, MPH, Daniel Montano, Ph.D, Danuta Kasprzyk,
Ph.D, Crystal Freeman, Ph.D., MPH
Presented By: Nidhi Jain, M.D., MPH, Medical
Epidemiologist, Division of STD Prevention, Centers for
Disease Control and Prevention, 1600 Clifton Road, Mailstop
E-80, Atlanta, GA 30333; Tel: (404)639-1823; Fax: (404)6398607; Email: ncj0@cdc.gov
Research Objective: Human papillomavirus (HPV) infection
is associated with genital warts, cervical intraepithelial
neoplasia (CIN) and anogenital cancers. In the U.S., 20
million people are infected with HPV and 5.5 million new
infections are diagnosed annually making it the most
common sexually transmitted infection at an economic
burden of 4 billion dollars. Recently, CDC and several national
clinical organizations recommended a recently licensed test
for HPV for providers to test patients for two reasons: to guide
the management of women with borderline abnormal (“ASCUS”) Pap test results and to use as an adjunct to Pap testing
for women aged =30 years to monitor persistent HPV
infection which may lead to cervical cancer. How U.S.
providers are currently using the HPV test is not known.
Study Design: Providers received self-administered mail
surveys May-August, 2004 addressing their HPV testing
practices. Percentages were weighted for disproportionate
sampling and non-response by specialty.
Population Studied: U.S. physician and mid-level provider
specialties who provide primary care or are likely to manage
patients with genital warts were selected randomly from
national registries and master files. Sampled physician
specialties included internal and adolescent medicine,
family/general practice, obstetrics/gynecology, urology, and
dermatology as well as certified nurse-midwives, physician
assistants, and nurse-practitioners (n=6906).
Principal Findings: The adjusted response rate was 81%. Of
all respondents, 17% were unaware of the new HPV test; of
those aware of the test, 47% reported ever testing their
patients for HPV. Obstetricians/gynecologists (92%) and
nurse-midwives (88%) were more likely to report HPV testing
compared to primary care providers (31%-56%) and
specialists (5-10%). Some clinicians reported HPV testing for
uses not recommended by guidelines including: upon request
of female (58%) or male (32%) patients; for female (32%) or
male (21%) patients with anogenital warts, for female (34%)
or male (18%) patients with non-HPV sexually transmitted
infections, and to check infection status of female (30%) and
male (13%) patients as a primary screening test for HPV
infection. Among the subset of providers who performed Pap
tests for cervical cancer screening (n=2930), 74% reported
HPV testing women with “ASC-US” Pap results for
management (recommended use), 48%-67% reported testing
patients with higher-grade Pap abnormalities (nonrecommended use), and 21% reported HPV testing women as
an adjunct to Pap tests with 12% of providers usually/always
testing women >30 years (recommended use) and 15% of
providers usually/always testing women <30 years (nonrecommended use). Obstetrician/gynecologists and nursemidwives most commonly reported HPV testing for nonendorsed reasons.
Conclusions: In this nationally representative sample of
clinicians, most U.S. providers are not currently testing their
patients for HPV. Of those who do, many report using the
HPV test for reasons consistent with recent guidelines, but
some providers are testing for other non-recommended
reasons. Testing practices for HPV varied by clinician specialty
and non-recommended testing uses were most common in
those specialties more likely to report HPV testing.
Implications for Policy, Delivery, or Practice: Survey
findings should be used to develop/update clinical training
materials, decision support tools and patient educational
materials to encourage HPV testing that is consistent with
national guidelines.
Primary Funding Source: CDC
●Failure of ICD-9-CM Coding to Identify Chronic Kidney
Disease in Patients with Diabetes
Elizabeth Kern, M.D., Donald R. Miller, Sc.D., Chin-Lin Tseng,
DrPH, Miriam Maney, MS, David Aron, M.D., MS, Leonard
Pogach, M.D., MBA
Presented By: Elizabeth Kern, M.D., VA Quality Scholar,
Education, Cleveland Department of Veterans Affairs Medical
Center, 10701 East Boulevard, Cleveland, OH 44106; Tel: (216)
368-6129; Fax: (216)368-5824; Email: eok@case.edu
Research Objective: Diabetes and chronic kidney disease
(CKD) are increasingly prevalent co-morbid diseases that are
independent risk factors for poor outcomes, especially in
patients with cardiovascular disease. There is evidence that
CKD is often under-diagnosed. However, methods of casemix adjustment for quality performance measures frequently
depend on ICD-9-CM diagnosis codes in administrative
records to identify co-morbid disease. The objective of this
study was to determine how frequently and accurately ICD-9CM diagnosis codes in administrative records identify CKD in
individuals with diabetes.
Study Design: We used the 4-variable (age, sex, race, serum
creatinine) Modification of Diet in Renal Disease predictive
equation to estimate glomerular filtration rate (eGFR) for a
national, retrospective cohort of veterans with diabetes. We
calculated eGFR from two or more serum creatinine
measurements over the span of one year beginning
9/30/1998. Using eGFR, we classified individuals for presence
or absence of CKD according to National Kidney Foundation
guidelines. Over 70% of veterans were Medicare
beneficiaries. Searching linked VA and Medicare inpatient and
outpatient administrative records, we determined the extent to
which individuals with CKD were recognized by various
algorithms of ICD-9-CM diagnosis codes.
Population Studied: The population studied was 561,088
veterans with diabetes who were alive as of 9/30/1998,
without a diagnosis of end-stage renal disease or dialysis for
two years prior.
Principal Findings: We were able to determine eGFR for
263,730 individuals (47.0%). The crude prevalence of CKD was
32% (83,338/263,730), declining to 29% when adjusted to the
US diabetic population over age 40. CKD rose sharply with
age, from a crude rate of 3.8% for those under 40 years old, to
46.1% in those 70 years and older. Using an algorithm of
nine ICD-9-CM codes indicating kidney failure, only 20.2%
(16,864/83,338) of individuals with CKD were able to be
identified by diagnosis codes in both inpatient and outpatient
records over the time span of one year (true positives). The
false positive rate was 1.2% (1,034/180,392). Expanding the
algorithm to include any of 79 renal-related diagnosis codes
increased identification of true positives to 42.4%
(35,318/83,338), while the false positive rate increased to 6.8%
(12,317/180,392).
Conclusions: CKD is a common co-morbidity in older
veterans with diabetes. Despite having two or more laboratory
encounters for serum creatinine tests within a one year time
span, individuals with CKD generally failed to be coded for
their disease in either VA or Medicare reimbursed encounters.
However, if present, ICD-9-CM codes were highly specific for
the presence of true CKD.
Implications for Policy, Delivery, or Practice: The failure to
code for CKD in populations of patients with diabetes and
CKD may adversely affect the validity of quality performance
measures that depend on ICD-9-CM diagnosis codes in
administrative records to identify co-morbid disease.
Primary Funding Source: VA
●Quality and Patient Safety in Telephone Medicine
Shersten Killip, M.D., MPH, Carol L Ireson, Ph.D., RN, Steven
T Fleming, Ph.D., Margaret M Love, PhD, Carol M McLay,
BSN MPH, Whitney J Katirai, BA
Presented By: Shersten Killip, M.D., MPH, Associate
Residency Director FP, Family and COmmunity Medicine,
University of Kentucky, K 302 KY Clinic 0284, Lexington, KY
40536; Tel: 859-323-5917; Fax: 859-323-6661; Email:
skill2@email.uky.edu
Research Objective: The purpose of this project was to
analyze the telephone medicine systems at an academic
health center from a systems perspective, looking for potential
threats to patient safety. The research questions included: (1)
Are there currently gaps in our telephone care system which
may contribute to patient adverse events? and (2) Can a
systems based evaluation method be developed to identify key
components in patient safety as it relates to telephone
medicine?
Study Design: This study used both quantitative and
qualitative design elements. Telephone interviews were
conducted with a sample of 64 patients over 10 weeks.
Analyses of the outcomes from each telephone call, including
systems-level outcomes (was the patient called back, did the
message get to the primary care doctor, etc) and patient-level
outcomes (checking that the patients felt better/followed
instructions/did not suffer any adverse events) were done.
Interviews were conducted with all residents and attendings at
the University of Kentucky Family Practice in person. Process
data from these interviews were analyzed for existing and
actual weaknesses in the system.
Population Studied: The populations studied in this project
are UKFP faculty, the UKFP residents, and the English,
Spanish, or French-speaking adult patient population of the
UKFP Clinic. The University of Kentucky Medical Center is an
academic health center that draws on a wide area with the
principal catchment area being the eastern half of the state.
The Family Practice Residency is staffed with 9 faculty and 18
family practice residents.
Principal Findings: Preliminary findings identified significant
gaps in our telephone system, as well as patient satisfaction
issues. Almost 10% of phone calls went unanswered, and
nearly a third of patients rated their experience as poor. The
majority of patients understood and followed the physician’s
recommendations which were a visit to the emergency room
or an over the counter medication. Less than a third of
patients sought no further care after their telephone call. An
unexpected finding was that the majority of callers were
proxies for the actual patients (i.e. mothers calling for
children), adding yet another layer of complexity to telephone
medicine.
Conclusions: The telephone medicine system at the UKFP
has system and patient satisfaction problems. Problems with
patient safety were implied by failure to answer calls, but few
actual adverse events were reported. This is not surprising
given the small sample size and the level of intervention
usually done over the phone; however, the level of potential
harm was quite high.
Implications for Policy, Delivery, or Practice: Telephone
medicine is a large part of ambulatory patient care, and it has
rarely been investigated for patient safety issues as they relate
to the telephone systems. This study will allow us to redesign
our telephone system to better care for patient safety. A future
project will evaluate the results of our intervention, then
disseminate it to several related sites. Should this intervention
be successful, and reproducible at other sites, the implications
for the practice of telephone medicine could be enormous.
Primary Funding Source: AHRQ
●Gestational Pyelonephritis as an Indicator of the Quality
of Ambulatory Maternal Health Services
Lisa Korst, M.D., Ph.D., Carolina Reyes, M.D., Moshe
Fridman, Ph.D., Michael C. Lu, M.D., MPH, Calvin J. Hobel,
M.D., Kimberly D. Gregory, M.D., MPH
Presented By: Lisa Korst, M.D., Ph.D., Associate Professor,
Pediatrics, USC Keck School of Medicine, 4650 Sunset
Boulevard, MS #30, Los Angeles, CA 90027; Tel: (323)
671.7623; Fax: (323)906.8043; Email: LKORST@chla.usc.edu
Research Objective: Inpatient conditions that might be
avoided through improved outpatient services are called
Ambulatory Care Sensitive Indicators, and they include
pyelonephritis in non-pregnant adults. No such indicators
have been developed for pregnant women. Because early
screening and treatment of asymptomatic bacteriuria should
prevent most cases, the purpose of this study is to examine
whether hospital-specific rates of gestational pyelonephritis
may serve as a measure of the quality of ambulatory maternal
care.
Study Design: The California Office of Statewide Health
Planning and Development provided a data set linking the
birth certificate to maternal and newborn delivery records for
the 1997 calendar year. Antepartum records of hospital
admissions for these women were included. For clarity, a
“low-risk” study population was created, and patients with
maternal, fetal and placental morbidities were largely
excluded, as were women without first trimester prenatal care.
We generated hospital-specific infection rates using a
Bayesian hierarchical logistic regression model.
Population Studied: All low-risk women and their newborns
who delivered in California in 1997.
Principal Findings: We identified 281,031 low-risk women, of
whom 1,853 (0.66%) had at least one inpatient admission for
gestational pyelonephritis. The model suggested only two
significant risk factors: MediCal as a payer (OR = 1.62
compared with all other payers), and African-American race
(OR = 1.24 compared with Caucasian race). Women with
pyelonephritis were more than twice as likely to deliver
preterm. Adjusted rates of gestational pyelonephritis for the
291 hospitals in the sample ranged from 0.25% to 2.34%.
Conclusions: These findings suggest that because of its
preventability and its consequent related morbidity,
gestational pyelonephritis meets clinical requirements as a
quality indicator for ambulatory maternal care. Hospitalspecific pyelonephritis rates also meet technical requirements
for quality indicators.
Implications for Policy, Delivery, or Practice: The use of
such rates would provide an opportunity for hospitals to
improve patient outcomes through partnership with admitting
obstetricians in the development of policies and procedures
for women at risk.
Primary Funding Source: Maternal-Child Health Branch of
the California Department of Health Services, and the Agency
for Healthcare Research and Quality
●The National Database of Nursing Quality Indicators:
Developing Measures of Pediatric Nursing Quality
Susan R. Lacey, Ph.D., RN, Susan F. Klaus, RN, MSN, Janis B.
Smith, RN, MSN, Karen S. Cox, Ph.D., RN
Presented By: Susan R. Lacey, Ph.D., RN, Associate
Professor, College of Nursing, University of Alabama in
Huntsville, Sparkman Drive, Huntsville, AL 35899; Tel: (256)
824-2438; Email: laceys@uah.edu
Research Objective: This research sought to identify and pilot
test nurse sensitive pediatric indicators for inclusion in the
National Database of Nursing Quality Indicators (NDNQI),
the American Nurses Association's repository for national
unit-based benchmarking that measures nursing's direct
impact on quality linked to staffing.
Study Design: 1) Exploratory, descriptive approach including
literature review, consultation with experts, and comment
from clinical and administrative leaders in pediatric care.
2) National multi-site pilot test to examine the reliability of
data collection tools.
Population Studied: This study included pediatric patients
from 26 units across the United States; 13 pediatric, medicalsurgical, 6 pediatric intensive care, and 7 neonatal units.
Principal Findings: This study found the indicators developed
- 1) pain assessment/intervention/reassessment cycle and 2)
peripheral intraveneous infiltration assessment - nurse
sensitive and feasible. Full scale nation-wide implementation
is now in progress for all participating NDNQI sites.
Conclusions: This work, coupled with other quality initiatives,
helps establish comparison benchmarks for pediatric patients
and supports evidence of nursing’s impact on quality patient
outcomes. ANA and NDNQI continue to lead the way in
nurse sensitive quality initiatives.
Implications for Policy, Delivery, or Practice: These new
pediatric indicators will 1) widen the effort to protect our most
vulnerable patients from neonates to adolescents and 2)
further the scientific study between patient outcomes and
staffing.
Primary Funding Source: This study was performed under
contract to the American Nurses' Association
●Comparing and Assesing Risk Adjustment Methods for
Quality Report
Yue Li, Ph.D. Candidate, MS, Andrew W. Dick, Ph.D., Dana B.
Mukamel, Ph.D.
Presented By: Yue Li, Ph.D. Candidate, MS, Community and
Preventive Medicine, University of Rochester Medical Center,
601 Elmwood Ave., Box 644, Rochester, NY 14642; Tel:
(585)260-8195; Fax: (585)461-4532; Email:
yue_li@urmc.rochester.edu
Research Objective: Quality report-cards provide information
about the relative performance of providers based on riskadjusted health outcomes. The validity of these performance
reports depends on the ability of statistical modeling
techniques to correctly adjust for differences in patient mix,
and on the correct specification of quality measures. Current
practice is to model risk assuming an additive provider effect
(quality) on patient risks and outcomes, rather than allowing
for interaction between them. Quality measures are defined
arbitrarily as either additive (O-E difference: difference
between Observed and Expected outcomes) or multiplicative
(O-E ratio). The difference measure is consistent with the
additive statistical risk adjustment model while the ratio
specification is not. Furthermore, the exact relationship
between quality, patient risks and patient outcomes is usually
unknown. This study was designed to examine the
consequences of misspecified risk-adjustment model and
quality measures. We investigated whether different
specifications of risk-adjustment models and quality measures
return consistent quality rankings, eg, identification of high or
low quality-outliers.
Study Design: We developed hospital quality measures based
on mortality by estimating logistic and linear probability
regression models on the 2002 New York State Cardiac
Surgery Reporting System (CSRS) data. This dataset includes
detailed clinical information for patients undergoing isolated
coronary artery bypass graft (CABG) surgeries. We assumed
either additive or multiplicative fixed effects of quality in model
estimation, and constructed corresponding additive or
multiplicative quality measures based on mortality rate or
odds of mortality. Kappa statistics were used to test for
agreement between these measures. We also performed
computer simulations, based on the empirical data, to explore
how the commonly used risk-adjustment method (based on
conventional logistic regression) performs for varied patterns
of quality effects.
Population Studied: All 16120 patients undergoing isolated
CABG surgeries at 36 New York State hospitals in 2002.
Principal Findings: All risk-adjustment models calibrated
approximately equally well to the data (Hosmer-Lemeshow
statistics = 8.2-10.0, P = 0.27-0.41), and had excellent
discrimination (c-statistics > 0.80). Among the 36 hospitals
offering the surgery, four were designated as quality outliers by
all quality measures. Three others were flagged as such by
some, but not all, measures. Kappa statistics showed fair
(kappa = 0.2 – 0.4) to substantial (kappa > 0.8) agreement
between measured quality of the hospital. In the simulation
we found that when the risk-adjustment was performed on a
small sample (eg, number of patients in a hospital <100), only
60-70 percent of the true quality-outliers can be identified for
some simulated specifications of quality effects; when the
sample size was relatively large (eg, >500 per hospital), over
90% of the true outliers can be identified for all simulated
quality effects.
Conclusions: Existing risk-adjustment methods potentially
oversimplify the patterns of the interaction between quality of
care and patient characteristics and thus cannot estimate
quality accurately. This problem is mitigated when quality
measures are based on a large sample of patients.
Implications for Policy, Delivery, or Practice: The
effectiveness of health care quality reports depends on the
accuracy of the performance information they convey.
Appropriate statistical modeling is essential to ensure valid
translation of outcomes data, especially of small sample, into
meaningful quality measures.
Primary Funding Source: No Funding Source
●Nursing Factors and Surgical Outcomes in VHA
Elliott Lowy, Ph.D., Sales, Anne E, Ph.D., RN, Nancy Sharp,
PhD, Gwendolyn Greiner, MPH, Yu Fang Li, Ph.D., RN
Presented By: Elliott Lowy, Ph.D., HSR&D, VA, 1100 Olive
Way, Seattle, WA 98101; Tel: (206)277-4789; Fax: (206)7685343; Email: elliott.lowy@med.va.gov
Research Objective: We report on the association between
nursing factors, including staffing levels and skill mix, and
patient outcomes after surgery, including overall in-hospital
mortality and in-hospital failure to rescue. Failure to rescue
measures mortality of patients who experience serious,
possibly nursing-sensitive, complications after surgery.
Study Design: We present an observational, cross-sectional
study using archival data. We used the DSS department
budget and cost report (ALBCC) for number of nursing
personnel hours per month by type of nurse, and we used
responses to a survey of all VHA nursing personnel in 124
facilities, mailed 2/03-6/03. Patient data, for surgeries in VHA
facilities between February and June 2003, come from the
National Surgical Quality Improvement Program, and include
patient characteristics, pre-existing conditions, complications,
and outcomes of surgery. Data on nursing units come from
the DSS TRTIPD files, a new DSS extract file that can link
inpatients to nursing units. All analyses were conducted at the
nursing unit level.
Population Studied: All patients who stayed on an acute
med-surg unit at any of 124 VHA facilities between February 1,
2003 and June 30 2003.
Principal Findings: With 510 inpatient units in the sample,
overall mean unit-level mortality rate was 4.0% (SD 12.3%);
mean failure to rescue rate was 20.1% (SD 27.5%). RN
nursing hours per patient day (RNHPPD) are positively and
significantly correlated with unadjusted overall mortality,
probability of experiencing one of the serious complications
after surgery, and failure to rescue. Non-RN hours per patient
day (OTHPPD) are negatively and significantly correlated with
overall mortality, probability of experiencing one of the serious
complications, and conditional mortality. A likelihood score
for complications was created, based on age, gender, race,
Charlson index, and ICU stay; after risk adjustment for this
likelihood, RNHPPD and OTHPPD are no longer significantly
correlated with either mortality or failure to rescue.
Conclusions: Risk adjustment is critical. Conducting patientlevel analyses is essential for power, but on unit-level analysis,
there may be no significant relationship between nursing
hours and patient outcomes.
Implications for Policy, Delivery, or Practice: Final results
are pending, but this is the first large scale study of nursing
factors and patient outcomes with unit-level data. A finding of
no significant relationship might call into question results of
prior studies suggesting that nursing hours are an important
factor in patient mortality outcomes.
Primary Funding Source: VA
●Organizational Factors and Human Factors Related to
Harmful Medical Event Outcomes in 23 Academic Medical
Centers – Evidence using Electronic Medical Error-Event
Reporting Systems for Targeting Patient Safety Programs
Sandra Magnetti, MS, DrPH, Raj Behal, M.D., MPH
Presented By: Sandra Magnetti, MS, DrPH, Post doctoral
fellow, Clinical Practice Advancement Center, University
HealthSystem Consortium, 2001 Spring Road, Oak Brook, IL
60523; Tel: (630) 954-1719; Fax: (630) 954-5879; Email:
magnetti@uhc.edu
Research Objective: To identify human and organizational
factors, which are significantly related to harmful vs. nonharmful medication events. In response to the IOM ‘s report
that at least half a million preventable injuries and 44,000 to
over 98,000 preventable deaths occur in hospitals each year,
at a cost exceeding $30 billion due to medical errors, we
utilized a web-based hospital event reporting system to
identify human organizational factors that are related to
patient harm.
Study Design: We analyzed medication error-event data
(representing approximately 25% of total medical events)
reported over a three-year period submitted by 25 academic
medical centers, which participated in a collaborative, on-line,
web based event reporting system. Front-line clinical staff
entered information on the events and scored the level of
harm using a progressive 10-point scale ranging from near
miss to patient death due to the event. Unit managers had the
option to review the report and comment on factors that
contributed to the event. The input screen format allowed
them to select one or more of 36 human and organizational
factors, depending on their perception of the event. We used
the nurse unit manager’s report to correlate these human and
organizational factors with a harm/no harm score. The score
was developed by dichotomizing the progressive 10-point
scale into those events that caused harm to the patient
ranging from temporary harm requiring extra intervention to
patient death versus those events that caused no harm to the
patient. Chi-Square statistics were used to evaluate these
relationships.
Population Studied: Medication error-event data from
inpatients aged 18 or older were selected from 23 academic
medical center hospitals’ web based reporting systems for
years 2001-2004.
Principal Findings: We found that of the 24,809 reported
medication events, 10.49% were related to some degree of
harm or discomfort to the patient. The remaining 90.51% of
the reported events did not reach the patients or require
additional treatment. Twelve of the 36 Human and
Organizational factors were significantly related to the harmful
events; a p value <= 0.001 was used to determine significance
for these factors using Chi Square analysis. These events were
significantly related to factors centering around each of six
areas: TEAM FACTORS: “communication problem between
providers”, WORK ENVIRONMENT:
“distractions/interruptions”, “high noise level”; STAFF
FACTORS: “staffing insufficient”, “system for covering patient
care”; TASK FACTORS: “inexperienced staff”, “inadequate
resident supervision”, “order entry system problem”,
“emergency situation”, “cardiac/respiratory arrest situation”;
ORGANIZATION/MANAGEMENT: “bed availability”; and
PATIENT CHARACTERISTICS: “patient understanding”.
Conclusions: This study reveals that there are specific human
and organizational factors related to medication error-events,
which, either necessitated additional treatment measures,
were life threatening, or resulted in death.
Implications for Policy, Delivery, or Practice: This project
provides information as to which events are more associated
with harm to the patient and could be used to target errorevent prevention programs. This study also provides empirical
evidence as to the use of and importance of the development
of standardized web-based reporting systems.
Primary Funding Source: No Funding Source
●Functional Performance and the Continuum of
Rehabilitation Care
Chetan Malik, MBBS, Carl V. Granger, M.D., Carol M.
Brownscheidle, Ph.D.
Presented By: Chetan Malik, MBBS, Clinical Instructor, Dept.
of Rehabilitation Medicine, University at Buffalo, 4351
Chestnut Ridge RD #8, Amherst, NY 14228; Tel: (716)8177867; Email: cmalik@buffalo.edu
Research Objective: Analyze the relationships among
admission FIM(TM) ratings, etiological diagnosis and
utilization of home-based rehabilitation services.
Study Design: The FIM(TM) instrument allows a variety of
health care professionals to assign a rating to a patient’s level
of function in 18 motor and cognitive tasks that represent
basic activities of daily life. FIM(TM) ratings range from 0-126
with higher values representing more independence or fewer
hours of help in performing daily functions. Based upon the
admission FIM(TM) ratings patients were divided into 4
groups: Group 1: <=60 (N=56), Group 2: >60 to <= 90
(N=177), Group 3: >90 to <=100 (N=208) and Group 4: >100
(N=424) respectively representing >=4, <4 to >=1, <1 to >0
and zero hours of help needed. The FIM(TM) database was
searched for records of patient receiving home-based
rehabilitation. Admission FIM(TM) ratings of patients
receiving rehabilitation services at home were examined to
determine potential relationships to utilization of physical
therapy (PT), occupational therapy (OT), and home health
aide (HHA) services. Net rehabilitation charges were also
compared among the four groups. Kruskall-Wallis test was
used to compare mean PT, OT and HHA visits among the
four FIM(TM) groups. Five etiological groups (based on ICD
codes) were represented in large enough numbers to permit
statistical analysis: musculoskeletal (N=254), circulatory,
including cardiac and stroke (N=199), poisoning and injuries,
including motor vehicle accidents (N=194), infection (N=44),
and nervous system disorders (30).
Population Studied: 865 patients receiving home-based
rehabilitation were studied. Mean age was 75 years (SD=12;
range 13 to 98 years); 66% were females, 33% were males.
Data on gender was not available for 1% of patients.
Principal Findings: Higher FIM(TM) ratings were associated
with fewer PT visits (17, 15, 12 and 11 respectively for Groups 1
to 4 p<0.001), OT visits (12, 7, 3 and 2 respectively for Groups
1-4, p<0.001), HHA visits (21, 10, 6 and 3 for Groups 1-4
respectively, p<0.001) and lower mean net rehabilitation
charges ($ 3883, 2581, 1592 and 1324 for Groups 1-4
respectively, p<0.001). Patients with circulatory and
neurological disorders required more OT visits (7 and 9
respectively) as compared to patients with infections (4 visits),
poisoning and injuries (3 visits), and musculoskeletal diseases
(1 visit). Number of HHA visits was 1 for patients with
musculoskeletal disorders and 2 in the remaining four
impairment groups. Number of PT visits was 13 for patients
with circulatory disease and poisoning and injuries and 12 for
the other three groups.
Conclusions: Higher FIM(TM) ratings are associated with
decreased utilization of home-based rehabilitation services,
and lower rehabilitation charges. Functional performance at
the onset of home-based rehabilitation care appears to be a
better determinant of future in-home rehabilitation services
than etiological diagnosis.
Implications for Policy, Delivery, or Practice: In 2002, the
FIM(TM) instrument was adopted by the Centers for Medicare
and Medicaid Services (CMS) for inpatient rehabilitation
assessment under the prospective payment system. The
results of the present study indicate the value of the FIM(TM)
instrument as an assessment tool in the home care venue of
the continuum of post-acute care settings.
Primary Funding Source: No Funding Source
●Towards a Framework for Long-Term Maintenance of
Quality Measures: Results of a Survey of Leading
Measures Developers
Soeren Mattke, M.D., D.Sc., Soeren Mattke, Cheryl Damberg,
Ph.D.
Presented By: Soeren Mattke, M.D., D.Sc., Scientist, RAND,
1200 South Hayes Street, Arlington, VA 22202; Tel: (703)4131100; Fax: (703)413-8111; Email: mattke@rand.org
Research Objective: In recent years, quality measures have
gained increasing importance not only for research but also
for policy and practice, such as in pay-for-performance or
public reporting initiatives. Their new role requires developers
and users to ensure that measures continue to meet rigorous
standards in the long run. However, little is known about
effective and efficient approaches to measures maintenance.
Study Design: We conducted a survey of ten nationally
recognized measures developing organizations and selected
researchers about their current polices and procedures as well
as desirable properties for a comprehensive system for
measures maintenance. We consolidated their input into a
briefing document for panel discussions with all respondents
to arrive at consensus recommendations for a measures
maintenance framework.
Population Studied: Five measures developers, two provider
and three purchaser organizations. Six were private sector
organizations; two governmental agencies and two selfregulatory institutions.
Principal Findings: All organizations had procedures for
measures maintenance, but the degree of formalization of the
procedures varied. Three key functions for a measures
maintenance system emerged: Ad hoc review to deal with
unexpected problems with a measure, annual maintenance to
incorporate changes in coding conventions and regular reevaluation to thoroughly review measures in pre-defined
intervals. Importance, scientific soundness, feasibility and
usability were universally used as evaluation criteria. The panel
discussions yielded a consensus set of recommendations for
relationships between maintenance functions, evaluation
criteria and measures disposition.
Conclusions: A sufficient degree of implicit consensus was
found among leading measures developers to arrive at a
consensus framework for policies and procedures for
measures maintenance.
Implications for Policy, Delivery, or Practice: This
framework offers a starting point for the development of
measures maintenance systems. While different organization
may choose to implement it in a way that is most consistent
with their mission and structure, it provides guidance as to
which components should be included.
Primary Funding Source: CMS
●Challenging Accepted Wisdom on Rules, Blame and
Safety Culture: a Hospital Operating Theatres Case Study.
Ruth McDonald, Ph.D., Justin Waring, Ph.D., Stephen
Harrison, Ph.D.
Presented By: Ruth McDonald, Ph.D., Research Fellow,
National Primary Care Research and Development Centre,
University of Manchester, Williamson Building, Oxford Road,
Manchester, M13 9PL; Tel: +44 (0) 161 275 3535; Fax: +44 (0)
161 275 7600; Email: ruth.mcdonald@man.ac.uk
Research Objective: The current orthodoxy within patient
safety research and policy is characterised by a faith in rulesbased systems, which limit the capacity for individual
discretion, (and hence fallibility) and the eradication of ‘blame
cultures’, which are assumed to discourage clinicians from
reporting errors. This study examines how different groups of
hospital staff (doctors and managers) view guidelines and
errors, in order to assess both the extent to which this
orthodoxy is accepted by hospital staff and the likely barriers
to implementing these policy solutions.
Study Design: An ethnographic study of a hospital operating
theatres department in an acute care hospital. In addition to
observation of activities and documentary analysis, 39 semistructured interviews were conducted. Interviews were taperecorded and transcribed verbatim.
Population Studied: Clinicians and managers working within
the operating department of a large teaching hospital in the
North of England. Interview participants were consultantgrade surgeons from a range of specialties (general surgery,
urology, ENT, maxillofacial, cardiothoracic, gastrointestinal,
ophthalmic, orthopaedic), consultant anaesthetists and
departmental managers.
Principal Findings: We identified two different and distinct
sets of narratives, with the accounts given by medical staff
challenging the belief of mangers that surgical work can be
governed by formal rules and procedures. Whilst doctors
rejected written rules, they adhered to the unwritten rules of
what constitutes acceptable behaviour for members of the
medical profession. This includes eschewing guidelines and
refusing to pass judgement on other medical colleagues. In
contrast to ‘cognitive bias’ based explanations of physician
behaviour, which depict doctors as being overoptimistic, we
found that doctors expected to make mistakes and accepted
them as inevitable. The fear of blame was less of a barrier to
reporting than the belief that since mistakes were inevitable
and all patients and contexts are different, then the ability to
learn from mistakes is negligible. The manager narrative
views mistakes as preventable, places faith in formal rules and
processes and lays blame at the door of unruly medical staff.
Reducing this propensity to blame individuals is a remote
prospect and is by no means guaranteed to increase error
reporting, particularly if doctors see mistakes as inevitable and
do not recognise failure to comply with guidelines as an error
or violation.
Conclusions: Current ‘solutions’ intended to improve patient
safety may understate the impact and persistence of different
norms, values and attitudes amongst different groups of staff
on behaviour in the hospital setting. Rather than a safety
culture with shared values and norms, it may make more
sense to talk of different identities (managerial and medical)
and the different attitudes to risk, blame and error associated
with these.
Implications for Policy, Delivery, or Practice: Solutions
based on standardisation, systematisation and developing a
safety culture are likely to prove difficult to implement and
may fail to deliver the benefits anticipated.
Primary Funding Source: Department of Health (England)
●Physician Attitudes Toward Pay-for-Performance
Programs: Development and Validation of a Measurement
Instrument
Mark Meterko, Ph.D., Gary J. Young, Ph.D., Bert White, MBA,
James F. Burgess, Ph.D., Dan Berlowitz, M.D., Matthew R.
Guldin, MPH
Presented By: Mark Meterko, Ph.D., Manager, Methodology
& Survey Unit, Center for Organization, Leadership &
Management Research, VHA HSR&D, VA Medical Center
(152M), 150 South Huntington Avenue, Boston, MA 02130;
Tel: (617)278-4433; Fax: (617)232-6140; Email:
mark.meterko@med.va.gov
Research Objective: Financial incentive programs for
physicians have become increasingly prevalent as a
mechanism for reducing care process variation, improving
clinical outcomes and reducing costs. Empirical support for
the efficacy of such programs is scarce, however, due in part
to the lack of a conceptual model of the features of incentive
programs that are salient to physicians, and to the lack of a
psychometrically sound measure of physician attitudes toward
those features. The goal of the present study was to develop,
test and evaluate such a measure.
Study Design: The study involved three phases: (a) the
development of a conceptual model of the factors relevant to
physicians’ perceptions of and reactions to financial
incentives; (b) the pilot testing of a preliminary questionnaire
representing those constructs through focus groups with
physicians at three medical practices; and (c) the large-scale
mail administration of the revised survey and thorough
psychometric assessment of the proposed measures.
Population Studied: Focus groups were conducted at three
eastern Massachusetts medical practices selected to represent
a variety of relationships between physician groups and payers
involved in incentive programs. Each group involved 13-15
primary care physicians and lasted 40-60 minutes.
Information from this phase was used to modify a draft
questionnaire in several crucial ways. The revised instrument
was subsequently distributed to 2497 private and group
practice physicians in New York State and Massachusetts who
were affiliated with two of the seven sites participating in a
larger national evaluation study of financial incentives.
Principal Findings: A total of 798 physicians (32%)
responded. Of these, 632 (79%) indicated that they were
aware of a particular quality target and associated incentive
and subsequently answered the key core survey questions.
The aware respondents were randomly split into a derivation
sample (n=316) and a validation sample (n=316). Exploratory
factor analysis in the derivation sample suggested that six
underlying dimensions could account for 57% of the variance
in the 26 core items: awareness, credibility, control,
cooperation, lack of unintended consequences, and impact.
Multi-item scales representing these six constructs were then
computed in the validation sample, and these data were
subjected to a multi-trait analysis. Ninety percent of the itemto-hypothesized scale correlations demonstrated convergent
validity, and 92% of item-to-other scale correlations
demonstrated discriminant validity. Internal consistency
reliabilities of the hypothesized scales ranged from .69 to .80.
and were substantially greater than any of the correlations
among the scales themselves. Only one scale (lack of
unintended consequences) showed any substantial ceiling or
floor effects (14% of respondents at the theoretical
maximum).
Conclusions: Six dimensions – awareness, credibility, control,
cooperation, lack of unintended consequences, and impact –
appear to be important for understanding physician reactions
to quality target and financial incentive programs. Physician
perceptions of these dimensions can be measured with
reliability and validity using a relatively brief self-report
questionnaire.
Implications for Policy, Delivery, or Practice: There is a
clear need for further systematic empirical research regarding
the efficacy of linking financial incentives to quality targets as a
way to influence physician practice behavior. The existence of
a conceptual model of the salient features of targets and
incentives, and of a valid and reliable measure of physician
attitudes toward those features and of their self-reported
impact, should contribute to the advance of that research
agenda.
Primary Funding Source: AHRQ, RWJF
●Beyond Staffing: Variation in Administratively Mediated
Variables in Hospitals
Ann F. Minnick, Ph.D., RN, FAAN, Lorraine Mion, Ph.D., RN,
FAAN, Mary Johnson, Ph.D., RN, Catharine Catrambone,
DNSc, RN
Presented By: Ann F. Minnick, Ph.D., RN, FAAN,
Independence Professor and Associate Dean-Research,
College of Nursing, Rush University, 600 South Paulina Street,
1062A AAC, Chicago, IL 60612; Tel: (312) 942-6990; Fax: (312)
942-3038; Email: Ann_ F_Minnick@rush.edu
Research Objective: To describe administratively mediated
inputs (capital, employment, organizational, and labor) that
comprise US hospital nurse working conditions.
Determination of the interactive impact of working conditions
on outcomes has been limited because most data sets
contain only staffing data or patient variables. This has made
it impossible to study the interrelationship of these inputs and
to examine their overall impact patient outcomes. The extent
inputs other than labor quantity varies by unit and by hospital
has not been demonstrated. Such determination is needed to
guide safety and quality efforts.
Study Design: Descriptive. 130 work place inputs suggested
by literature review, a national panel and the Minnick and
Roberts clinical behavior production model were studied
based on potential(1) explanatory power for specific outcomes
and (2) impact on multiple outcomes. Capital inputs included
those which may influence (1) the development of unsafe
patient behavior (2) ability to observe patients (3) staff
knowledge about the patient (4) visual and auditory stimuli.
Examples include bed monitors, distances from bedside to
supplies, computerization and room configuration.
Employment terms included temporal terms (e.g. shift length)
and work models (e.g. nursing, physician and support staff
deployment). Organizational variables included worker
empowerment and work process design. Labor variables
included work force characteristics and labor quantity. The
investigators conducted all observations, physical
measurements, reviews of staffing data and interviews with
unit personnel on-site in 2003-5
Population Studied: General acute care hospitals (mean
occupancy 100 or more) regardless of ownership, teaching or
profit status were included in the randomization procedure
that encompassed the New York , Chicago, Houston, DallasFt. Worth, Denver and Phoenix metropolitan areas. Within
each of the 40 hospitals, a general ICU, a medical and a
surgical floor were studied .
Principal Findings: Variation in administratively mediated
variables exceeded variation in staffing quantity. ICU and nonICUs varied from each other on almost every variable; within
hospital variation on non-ICU units equaled that noted across
hospitals. There was little variation in characteristics of
registered nurse assignment and use of ancillary nursing
personnel on day shift in ICU. Large within unit variation was
evident; e.g. distance to clean supplies varied on one unit
from 3 feet to more than 200. The relationship of some
inputs’ presence or absence suggests a potential unit resource
categorization.
Conclusions: (1) Most ICUs possess labor saving and safety
capital inputs (2)Major improvements on non-ICU units are
needed. (3) Assuming that administratively mediated
variables within a hospital are similar is unwarranted.
Implications for Policy, Delivery, or Practice: (1) Outcome
production studies, including those determining the role of
nursing labor, must account for these inputs and how they
vary by unit. It is possible that these other inputs may mask
the true impact of nursing labor and/or that nursing labor is a
proxy for some input combination. (2) Determining the role
of these inputs in producing outcomes could provide
direction for input allocation. (3)As the US hospital system
embarks on another cycle of bed replacement, avoiding many
of the observed design flaws may be possible. (4) New
typologies to categorize the work models are needed.
Primary Funding Source: NIA
●Failure to Rescue or Failure to Measure What's
Important? No Shortcuts to Assessing Quality
James Moriarty, BS, BA, James Naessens, MPH, Dawn Finnie,
MPA, Matthew Johnson, BA
Presented By: James Moriarty, BS, BA, Data Analyst, Health
Care Policy & Research, Mayo Clinic, 200 1st Street SW,
Rochester, MN 55905; Tel: (507) 538-3973; Fax: (507)284-1731;
Email: moriarty.james@mayo.edu
Research Objective: To investigate the usefulness of the
Failure to Rescue indicator defined by the Agency for
Healthcare Research and Quality (AHRQ) Patient Safety
Indicators (PSI’s) for monitoring hospital quality. Failure to
Rescue has been defined as the death of patients who
experience certain complications during hospitalization. Each
complication has specific criteria which exclude a patient from
the population at risk.
Study Design: A retrospective cohort study was used to
assess issues of interest. AHRQ PSI’s were applied to hospital
administrative data. Failure to rescue rates were compared
between complication categories. Analyses were also
performed based on whether the condition putting the patient
into the eligible population was present upon admission or
was acquired during care. Trends over time of the Failure to
Rescue population were assessed.
Population Studied: The patients included in this study were
all inpatients discharged from January 1, 1996 to December 31,
2003 at St. Marys Hospital (SMH) and Rochester Methodist
Hospital (RMH), the two facilities of Mayo Clinic Rochester,
both of which provide a full range of hospital care. Patients
aged 75 or greater, who are excluded in the AHRQ measure,
were kept in this study. The number of inpatients per year
ranged from 52,793 in 1996 to 64,705 in 2003.
Principal Findings: There were a total of 21,432 patients over
the eight year timeframe. The AHRQ Failure to Rescue
indicator is composed of six subgroups of complications:
acute renal failure, deep vein thrombosis or pulmonary
embolism (DVT/PE), pneumonia, sepsis, shock and/or
cardiac arrest, and gastrointestinal (GI) hemorrhage and/or
acute ulcer. The mortality rates for the six subgroups were:
acute renal failure, 23.0%; DVT/PE, 9.3%; pneumonia, 13.5%;
sepsis, 20.4%; shock/cardiac arrest, 58.0%; and GI
hemorrhage/acute ulcer, 10.4%. The proportion of patients
falling into each of the six subgroups at RMH differed from
SMH with higher proportions of DVT/PE at RMH (25.1% vs.
18.1%) and higher proportions of pneumonia at SMH (35.1%
vs.23.3%). Whether or not the condition is pre-existing or
acquired can also have an effect on mortality. The mortality
rate for patients diagnosed with acute renal failure classified
as having the condition upon admission was 20.8%, but those
patients that acquired the condition after admission had a
mortality rate of 27.3% (p <.001).
Conclusions: The results of this study show that the AHRQ
Failure to Rescue measure could be improved to give a more
accurate representation of the quality of patient care. These
results should also be applied to the Failure to Rescue
measure used by the National Quality Forum for their Core
Measures of Nursing Care Performance which includes five of
the six complication subgroups on major surgical patients
with similar exclusion criteria.
Implications for Policy, Delivery, or Practice: Hospitals with
a greater percentage of patients in these higher mortality
groups or with more patients with pre-existing complications
may affect overall mortality rate for the Failure to Rescue PSI.
This may lead to bias in comparing institutions on the
measure. More study needs to be performed to attain reliable
and valid quality measures.
Primary Funding Source: No Funding Source
●The Scope of the End of Life: a Systematic Review of the
Literature
Richard Mularski, M.D., Joanne Lynn, M.D., MA, MS, Ronda
Hughes, Ph.D., MHS, Karl Lorenz, M.D., MSHS, Sally Morton,
Ph.D., Paul Shekelle, M.D., Ph.D.
Presented By: Richard Mularski, M.D., Staff Physician, Fellow
in Health Services Research, Medicine, VA Greater Los
Angeles Healthcare System, 11301 Wilshire Boulevard,
mailcode 111G, Los Angeles, CA 90073; Tel: (310) 478-3711;
Fax: (310) 268-4933; Email: Richard.Mularski@med.va.gov
Research Objective: Improving end-of-life care requires a
definition of the population to whom we apply interventions
and assess patients’ and caregivers’ experiences. A definition
of the end-of-life population across the myriad of chronic
progressive diseases has eluded consensus in the field. We
aimed to systematically review literature that might inform a
conceptual and operational definition of the end-of-life
population.
Study Design: As part of an evidence-based review on end-oflife care, we systematically reviewed articles that evaluated
prognosis or trajectories of illness at the end of life. Inclusion
criteria included English language, publication period from
1990 to 2004, and adult human subjects. Articles were
selected using pre-determined criteria to inform the definition
of end of life or prognosis at the end of life.
Population Studied: Sources for our review included
Medline, the Cochrane Database of Reviews of Abstracts of
Effects (DARE), the National Consensus Project for Palliative
Care, several recent systematic reviews from Health Canada
and National Institute for Clinical Excellence (NICE - United
Kingdom), and input from a technical expert panel. Our
exclusions included articles addressing survival estimates for
acute hospitalizations or disease processes, articles enrolling
patients already in hospice or palliative care programs, and
articles evaluating isolated prognostic factors or the natural
history of a cohort.
Principal Findings: From 24,423 total citations, we identified
348 potentially relevant articles, 63 of which ultimately met
inclusion / exclusion criteria. We identified four themes for
specification of the end-of-life period: (1) active dying, (2)
patient readiness or acceptance of treatment limitations, (3)
severity of illness, and (4) poor prognosis. The literature base
failed to provide a uniform definition of the scope the end of
life.
Conclusions: In general, there are multiple and varied
definitions of the end-of-life population. None of the
constructs identified had substantial empirical validation of
their defining characteristics. Prognostication models and
clinician estimates lack precision and generalizability for
counseling patients and families or for discriminating
appropriate health care interventions. The prognostication
literature does not support a clear definition for the end-of-life
nor does it provide sufficient estimates for a statistical
likelihood of dying within a specific time period.
Implications for Policy, Delivery, or Practice: Our review
identified several constructs and varied definitions of the
scope of the end-of-life experience. The absence of a
consensus definition or rigorous validation of prognostic
methods poses a significant barrier to advancing the field of
end-of-life care and evaluating interventions to improve
patients’ and families’ experiences at the end of life. Research
is needed to characterize the implications of alternative
conceptual and operational definitions of the end of life. Novel
approaches should consider methods other than attempting
to determine a natural divide between everyday existence and
the end-of-life period or attempting to determine a statistical
divide based on the likelihood of dying within a certain time
period. With increased use and testing of theoretical models
for the end-of-life experience and consensus about the end-oflife scope, the field can be further advanced and the quality of
care for the dying improved.
Primary Funding Source: AHRQ, NINR
●Development of a Risk Adjustment Methodology for ICU
Length of Stay Using Administrative Data
Helen Neikirk, MA, Raj Behal, M.D., MPH, Liping Lu
Presented By: Helen Neikirk, MA, Program Director, Analytic
Services, Clinical Informatics, University HealthSystem
Consortium, 2001 Spring Road, Suite 700, Oak Brook, IL
60563; Tel: (630) 954-2462; Fax: (630) 954-5879; Email:
neikirk@uhc.edu
Research Objective: To develop a risk adjustment
methodology for Length of Stay (LOS)in the intensive care
unit (ICU) using administrative data.
Study Design: An administrative data base comprised of
patient level data for all inpatient admissions from 106
academic medical centers and teaching hospitals was used to
select patients who were admitted to an ICU during 2003.
Cases with ICU LOS greater than 300 days were excluded
from the study resulting in a total sample of 141,301 cases. A
log transformation was performed to normalize the
distribution of ICU LOS and linear regression analysis was
conducted on the log-transformed dependent variable.
Independent variables included age, sex, race, admission
source transfer from another acute care hospital, admission
source transfer from skilled nursing facility (SNF) or long term
care facility (LTC), emergency admission status, mechanical
ventilator use, tracheostomy, 30 specific comorbid conditions
defined by AHRQ, and other diagnoses associated with
increased severity of illness. The data set was randomly split
into two equal subsets and a model was generated on one half
and validated on the other half.
Population Studied: In-patients aged 18 and older from 106
academic medical centers and teaching hospitals who were
admitted to an intensive care unit in 2003.
Principal Findings: R-square value of the final model was 0.33
with a maximum VIF of 1.79. Variables found to be highly
significant predictors of increased ICU LOS included
tracheostomy, small bowel surgery, sepsis, craniotomy,
mechanical ventilation, weight loss/malnutrition, lymphoma,
congestive heart failure, chronic anemia, hemorrhagic stroke,
acute respiratory failure, and psychosis. Variables found to be
highly significant predictors of decreased ICU LOS included
diabetes, shock, coma, emergency admission, and transfer
from SNF or LTC. The model results were applied to the
entire data set to generate an expected ICU LOS for each case
and an observed to expected ICU LOS ratio was calculated for
each hospital. The hospital-specific bserved to expected ratios
ranged from 0.57 to 1.53.
Conclusions: This study demonstrates the usefulness of
administrative data in risk adjustment for ICU LOS. The
regression technique was able to account for a significant
amount of variation with clinically meaningful variables
derived from ICD-9 diagnosis and procedure codes and other
administrative data elements. Application of the model
revealed considerable variation in risk adjusted ICU LOS
among hospitals.
Implications for Policy, Delivery, or Practice: Widespread
concern about quality, cost, and efficient utilization of health
care resources indicate the need for cost effective methods for
outcome evaluation. ICUs account for approximately 7
percent of hospital beds and 20 to 30 percent of hospital costs
in the U.S. Clinically meaningful methodologies for risk
adjustment that use administrative data will be important
tools for evaluation of efficient use of ICUs in hospitals.
Primary Funding Source: No Funding Source
●Making it Safe: The Effects of Leadership Inclusiveness
and Professional Status on Psychological Safety and
Learning in Health Care Teams
Ingrid Nembhard, MS, Amy C. Edmondson, AM, Ph.D.
Presented By: Ingrid Nembhard, MS, Ph.D. Candidate,
School of Business Administration, Harvard University,
Soldiers Field Road, Morgan Hall T20, Boston, MA 02163; Tel:
(617) 495-1488; Email: inembhard@hbs.edu
Research Objective: Most hospitalized patients require care
from a range of disciplines, making teamwork in the routine
delivery of care essential. Moreover, hospitals attempting to
improve the quality of care often rely on interdisciplinary
teams to carry out improvement projects. The present study
investigates factors that promote team engagement in quality
improvement (QI) work, despite the salience of a welldocumented status hierarchy in medicine. We aim to
contribute to the understanding of the role of status in
shaping perceptions of psychological safety (the belief that it
is safe to speak up with questions, concerns or suggestions),
and how these effects may be locally modified by leader
behavior to facilitate team engagement in QI work.
Study Design: We surveyed staff in 23 neonatal intensive care
units (NICUs) that were members of a quality improvement
collaborative. The NICUs were recruited by phone and email
directed to the leader of each improvement team. We then
invited all team members to participate in our survey via an
invitational letter distributed in accordance with procedures
outlined in their hospital IRB approval (i.e. email or staff
mailboxes). We offered two versions of the survey (paper and
web-based) to allow participants to choose their preferred
format. Participants responded to items that measured their
impressions of psychological safety, leadership inclusiveness
(words and deeds exhibited by leaders that invite and
appreciate others’ contributions) and team engagement in QI
work, and reported their profession and other demographic
information.
Population Studied: Multidisciplinary health care teams from
NICUs pursuing quality improvement. In all, 1,440 health care
professionals (46% percent of team members contacted)
from 23 NICUs in the United States and Canada completed
the survey, for a NICU response rate of 58%. Of the
respondents, 1,229 persons declared their profession: 100
physicians, 998 nurses, and 131 respiratory therapists.
Principal Findings: Not only do high status individuals have
higher psychological safety than low status individuals, but
also cross-disciplinary teams differ in mean safety. Moreover,
there is an interaction between status and team, which
appears related to the degree of leadership inclusiveness in
the team. Greater inclusiveness minimizes the effect of status
on psychological safety, and vice versa. With more leadership
inclusiveness comes greater psychological safety, which in
turn predicts greater team engagement in QI work.
Conclusions: Status does predict psychological safety, but it
need not be deterministic. Aspects of team membership,
especially the degree of leadership inclusiveness influence
individuals' perceptions of safety as well. This is noteworthy
because we can alter teams to create safety, a precursor of
staff engagement in QI work.
Implications for Policy, Delivery, or Practice: Our results
provide reassurance that traditional status differences in
health care may be overcome - to the benefit of quality
improvement efforts. We identify a strategy for improving the
climate within cross-disciplinary teams in health care (i.e.
leadership inclusiveness), and provide an initial specification
of the antecedents for QI work in health care. Our findings
imply training leaders to be inclusive to foster psychological
safety may be a critical antecedent of effective quality
improvement.
Primary Funding Source: Harvard Business School Division
of Research
●Improving Quality of Care in Addiction Treatment
Maria Orlando, Ph.D., Suzanne Wenzel, Ph.D., Pat Ebener,
MA, James Dahl, Ph.D., Donna Farley, Ph.D., Wallace
Mandell, Ph.D.
Presented By: Maria Orlando, Ph.D., Behavioral Scientist,
Behavioral and Social Sciences Group (BHS), The RAND
Corporation, 1776 Main Street, Santa Monica, CA 90401; Tel:
(310)393-0411, 6604; Fax: (310)260-8150; Email:
orlando@rand.org
Research Objective: Measurement of quality of care and
targeted Quality Improvement interventions has received little
attention in addiction treatment. The RAND Corporation and
Phoenix House, the largest provider of therapeutic community
treatment in the nation, have collaborated on developing
methods to monitor and improve quality of care in therapeutic
communities. This collaboration has produced the
Dimensions of Change Instrument (DCI) to assess treatment
process, and a study of how programs can use DCI
information in a Continuous Quality Improvement (CQI)
framework to enhance quality of care. This presentation
reports principal findings on the measurement of treatment
process, and how quantitative data can be used by program
staff to plan targeted interventions.
Study Design: The DCI was administered to an entry cohort
of 993 clients (519 adults in 10 programs; 474 adolescents in 8
programs) who were assessed longitudinally throughout
treatment. Retention and post-treatment outcomes data were
also collected. Six treatment programs were subsequently
selected for participation in a quality improvement
demonstration. Staff of the 6 programs were presented with
DCI scores and retention rates for clients in their programs
compared with norms based on other programs’ data. Staff
of 4 selected experimental programs received instruction in
CQI and help in preparing and implementing quality
improvement action plans based on the longitudinal client
process data from their own program. Experimental
programs are being monitored via regular telephone calls and
site visits. Clients in all 6 programs are completing the
process instrument pre and post demonstration period,
allowing for observation of the impact of the CQI process.
Population Studied: Adult and adolescent clients and
programs represent diverse regions of the U.S. and of the
population of clients in Phoenix House programs. Study
participants are primarily members of ethnic minority groups
(African American or Hispanic), and a majority has been
referred to treatment from the criminal justice system.
Principal Findings: Analyses to-date support the validity of
the DCI (e.g., measurement of process is invariant across
client subpopulations; the instrument is sensitive to change in
residents during treatment; scores are associated with
retention). Staff acceptance of the ongoing quality
improvement activities demonstrates that DCI data can be
understood by program staff with researcher-facilitated
discussion groups, the data can be used by staff to develop
targeted quality improvement action plans, and staff
enthusiastically carry out these plans within their own
programs. Site visits now underway are yielding detailed
information on the implementation of quality improvement
action plans within each of the programs.
Conclusions: This study represents an innovative step
forward in enhancing quality of care in addiction treatment. It
furthermore is demonstrating that health services researchers
and addiction treatment providers can collaborate to ensure
that complex data can be applied to informed program-level
decision-making to improve quality of care.
Implications for Policy, Delivery, or Practice: The chief
implication for addiction services delivery and practice is that
researchers and providers can collaborate successfully to
enhance understanding of quality of care and to improve
quality of care. Our development of a new process instrument
and its successful application to a quality improvement effort
should prove valuable to the addiction services field generally.
Primary Funding Source: NIDA
●Patient Safety Attitudes of Trainee Physicians: A cross
sectional survey
Gareth Parry, BSc, MSc, Ph.D., Lisa Horowitz, Ph.D., J Bryan
Sexton, Ph.D., Donald Goldmann, M.D.
Presented By: Gareth Parry, BSc, MSc, Ph.D., Reader in
Health Services Research, Health Services Research,
University of Sheffield, Regent Court, 30 Regent Street,
Sheffield, S1 4DA; Tel: 44 114 2220798; Fax: 44 114 2724095;
Email: g.parry@sheffield.ac.uk
Research Objective: To measure the patient safety attitudes
of trainee physicians at an academic pediatric center.
Study Design: Cross sectional survey of Trainee physicians’
self-reported patient safety attitudes as measured using the 51
item Safety Attitudes Questionnaire (Inpatient Version) and a
specific 23 item Trainee Survey. The Safety Attitudes
Questionnaire can be summarised into six domains:
perceptions of management, safety climate, teamwork
climate, working conditions, stress recognition and job
satisfaction. The Trainee Survey can be summarised into six
domains: clinical supervision and support, communication
with senior clinical staff, handoffs and multiple services,
orientation of new personnel, role identification during codes
and support following an adverse event.
Population Studied: 295 trainee physicians based at an
academic pediatric health center in January 2004.
Principal Findings: From 295 trainee physicians surveyed,
responses were obtained from 209 (71%); residents and
fellows accounted for 29.2% and 70.8% of responses
respectively. From the Safety Attitudes Questionnaire trainee
physicians rated most positively job satisfaction (58.8%
indicating a very positive response), safety climate (51.3%) and
working conditions (51.5%). They rated less positively
teamwork climate (only 16.1% indicating a very positive
response), and stress recognition (14.1%). From the Trainee
Survey trainee physicians rated most positively clinical
supervision and support (47.7% indicating a very positive
response) and communication with their immediate senior
physician (28.9%). They rated less highly support following an
adverse event (8.2% indicating a very positive response)
handoffs and multiple services (7.7%) and role identification
during codes (4.0%).
Conclusions: Trainee physicians are relatively comfortable
with their supervision and ability to care independently for
patients but may be less confident about their communication
and teamwork with other health care providers and thus their
ability to care interdependently for patients.
Implications for Policy, Delivery, or Practice: Traditionally
resident and fellow programs aim to train physicians to
become independent care providers. To improve patient
safety, physicians may also need to work interdependently
with fellow health care professionals within their own and
other departments. Consideration could be given to
enhancing training programs so that they produce physicians
who can work not only independently but also
interdependently. Senior role models will be important in
convincing trainees that quality improvement and patient
safety are as high a priority as learning traditional medical
skills. Arguably, most junior and senior faculty in academic
institutions do not have these skills themselves (i.e., they are
not formally trained and evaluated on collaboration), and it is
difficult to see how they will serve as role models and teachers
without considerable education. Continuing education
programs, possibly involving experts from outside the
healthcare industry, are needed. Initiatives to improve patient
safety attitudes of trainee physicians must be accompanied by
real change in the systems in which they work, so that
interdependent working is encouraged and rewarded. When
trainee physicians make errors, hospitals must provide clear,
blame-free reporting mechanisms and transparent systems to
support the involved clinicians. Accordingly, clinicians
involved in successful interdependent recovery from errors
should be celebrated as local heroes.
Primary Funding Source: CWF
●Linking Data to Decision-Making: The Use of Outcomes
Management Systems in Child/Adolescent Mental Health
Treatment Settings
Vaishali Patel, MPH, Ph.D. Candidate, Anne W. Riley, Ph.D.
Presented By: Vaishali Patel, MPH, Ph.D. Candidate, Ph.D.
Candidate, Health Policy and Management, Johns Hopkins
Bloomberg School of Public Health, 6 Char Street, Edison, NJ
08820; Tel: (732)261-8323; Email: vapatel@jhsph.edu
Research Objective: The IOM’s ‘Crossing the Quality Chasm’
report cites the need to harness information technology to
inform care and measure performance as a means of driving
quality improvement. Yet, little is known about the “human
factors” involved in the implementation of Outcomes
Management Systems (OMS) and how outcomes data can
inform decision-making at various levels of the organization.
We conducted a qualitative study to identify organizational
conditions that support and hinder the use of outcomes data,
and to describe the experiences of different types of staff using
OMS in child and adolescent out-of-home care agencies in
Maryland.
Study Design: Using a multiple case study design we
purposefully selected 2 Residential Treatment Centers (RTC)
and 2 Treatment Foster Care (TFC) programs from 10
programs representing 4 different organizations that were
using the same Internet-based OMS that collected the
following data: demographic, family history, behavioral/social
issues, psychiatric diagnoses, treatment history, services, and
functioning rating scale. Within each case, clinicians,
managers/clinical supervisors, quality improvement personnel
and executive directors were interviewed using tailored semistructured field guides. Quality improvement and treatment
team meetings were observed to describe decision-making
processes. Grounded theory and case study methods guided
the analysis of the transcribed interviews and field notes
describing the meetings.
Population Studied: Thus far, 23 interviews with staff
members have been conducted across 3 programs (2 TFC and
1 RTC), representing 10 clinicians/case managers, 4 quality
improvement managers, 7 Unit Directors/clinical supervisors,
and 2 Executive Directors (one of whom is responsible for 2 of
the 3 programs). Interviews are being conducted at a 4th
program, an RTC, to confirm initial findings. Six meetings
(both quality improvement and treatment team meetings)
have been observed.
Principal Findings: All agencies struggled with a key step in
their efforts to use an OMS to inform decision-making:
generating meaning from the data. Organizational efforts
emphasized data collection and entry of outcomes data rather
than analysis and interpretation of the data, resulting in a
repository of data that was not used, and was considered
meaningless by staff. Staff from all levels of the organization
reported experiencing: difficulties interpreting the data and
applying it to their work; frustration regarding the limited
amount of feedback provided; and concerns regarding how
the data might be misused. Issues related to poor
implementation of OMS also impacted staff buy-in and
subsequent use of outcomes data for decision-making. Other
types of data were more valued and used to inform group
decision-making processes.
Conclusions: In order for outcomes data to be useful for
decision-making, a needed initial step is to work with staff to
assess how outcomes information can guide current planning
and clinical decision-making. This critical aspect of data
utilization is no less challenging than implementing the data
entry and collection of outcomes data but involves different
skills and resources.
Implications for Policy, Delivery, or Practice: As states and
accreditation organizations seek to mandate the use of OMS
within child mental health treatment settings measures
should be taken to ensure that adequate infrastructure and
knowledge level are in place within organizations so that the
data can be actively used for quality improvement.
Primary Funding Source: NIMH
●Using Quality Improvement Techniques to Improve
Nursing Engagement and Bedside Care
Marjorie Pearson, Ph.D., MSHS, Lynn Soban, Ph.D., RN,
Valda V. Upenieks, Ph.D., RN, Patricia H. Parkerton, Ph.D.,
MPH, Jack Needleman, Ph.D.
Presented By: Marjorie Pearson, Ph.D., MSHS, Social
Scientist, RAND Corporation, 1776 Main Street, Santa Monica,
CA 90401; Tel: (310) 393-0411x7566; Fax: (310) 260-8155;
Email: mpearson@rand.org
Research Objective: To assess the change implementation
activities of hospitals participating in the pilot phase of the
Transforming Care at the Bedside (TCAB) initiative, a
collaborative-based effort to improve the work environment on
medical/surgical units and thereby improve the quality of
patient care and nurse engagement.
Study Design: An observational study of implementation data
reported by the hospitals participating in a pilot quality
improvement (QI) collaborative led by the Institute for
Healthcare Improvement and funded by The Robert Wood
Johnson Foundation. Linking patient and nurse outcome
objectives, the TCAB QI framework encourages multiple
change strategies to improve four major elements of
medical/surgical operations: 1) workplace vitality, 2) reliability
and safety, 3) patient-centeredness, and 4) value and
efficiency. Participating organizations are encouraged to test
new ideas as well as proven practices, and suggested
strategies are characterized by the quality of the existing
evidence base (as applied to medical/surgical settings). The
UCLA/RAND TCAB evaluation team is utilizing this framework
to code all change activities reported in the hospitals’ monthly
collaborative reports and designing key study variables to
measure implementation intensity and framework fidelity.
These variables will be used to assess the feasibility of
changes at the medical/surgical unit level, variation in such
changes, and their relationship to both nursing and patient
outcomes.
Population Studied: Thirteen hospitals, geographically and
organizationally diverse, selected as innovators and early
adopters (based on criteria such as Magnet hospital status
and Baldridge Awards) to participate in the 2-year pilot phase
of the TCAB initiative.
Principal Findings: Preliminary findings suggest that most
organizations were able to initiate numerous changes early in
the collaborative. They undertook a total of 152 different
change activities during the first 3 months of the collaborative.
Changes per individual hospital ranged from 0 to 26. The
organizations emphasized two of the four TCAB framework
elements in their initial QI efforts. 42% of the change
activities they undertook in the first three months of the
initiative were directed toward improving the patientcenteredness of care and 38% towards efficiency and value.
Considerably less initial emphasis was placed on changes to
increase reliability (12%) and workforce vitality (9%). The vast
majority of changes tested were new ideas or strategies not yet
fully tested in medical/surgical settings; 7% fit the TCAB
framework designation of medical/surgical unit best practices.
Conclusions: As participants in this ambitions initiative to
change the work environment as well as the quality, safety,
and patient-centeredness of care, TCAB pilot hospitals initially
have emphasized two of four TCAB priorities (patient
centeredness and increased value) and relied primarily on new
ideas not yet fully tested. Subsequent research will assess
their ability to expand into the areas of reliability and vitality,
as well as the relationship of these activities to changes in
medical/surgical care processes and patient and nurse
outcomes.
Implications for Policy, Delivery, or Practice: The TCAB
pilot collaborative offers promising potential for contributing
to the evidence base for medical/surgical management and
increasing our understanding of whether and how QI
techniques can be applied to changing the work environment
as well as care processes.
Primary Funding Source: RWJF
●Can Pharmacogenomics Improve Drug Safety?
Kathryn Phillips, Ph.D., David Bates, M.D., Jennifer Haas,
M.D., Brian Alldredge, PharmD, Amalia Issa, Ph.D., Haiden
Huskamp, Ph.D.
Presented By: Kathryn Phillips, Ph.D., Professor, Clin
Pharm/Institute for Health Policy Studies, UCSF, 3333
California Box 0613, San Fransisco, CA 94143; Tel: (415)5028271; Fax: (415) 502-0792; Email: kathryn@itsa.ucsf.edu
Research Objective: The recent, highly publicized safety
issues with antidepressants and Cox-2 NSAIDS have
prompted calls for greater use of genetic information to
individualize drug therapies (“pharmacogenomics”).
Pharmacogenomics has the potential to improve drug safety
both by enabling the development of safer drugs and by
tailoring prescribing for individuals so that they may receive
therapies that optimize the likelihood of benefit while
minimizing the risk of adverse effects. Yet, there have been
many questions raised about if and when pharmacogenomicbased drugs and diagnostics will be of value and how the FDA
can facilitate the use of pharmacogenomics to improve drug
safety. We examine how pharmacogenomics can be used to
improve drug safety and review current and proposed efforts
to achieve this goal.
Study Design: We conducted a review of academic, industry,
and government literature (N=187); reviewed key FDA
guidance documents; conducted interviews with key
academic, industry, and government thought leaders through
our role as advisors to the FDA (N=15); and reviewed the
published literature
Population Studied: NA
Principal Findings: We analyzed the following issues:
(1) What data are (a) available and (b) needed to evaluate the
use of pharmacogenomics to improve drug safety? (2) What is
the (a) current and (b) proposed role of the FDA in using
pharmacogenomics to improve drug safety? We then
specifically evaluate these issues as they relate to
antidepressants and Cox-2 NSAIDS. We found an alarming
lack of existing data about how to evaluate the use of
pharmacogenomics to improve drug safety, including a lack of
data on the incidence of adverse drug events (ADEs) and how
these are associated with genetic variation. These limitations
include the lack of a national, systematic, and comprehensive
ADE database; the lack of relevant and comparable data in
drug package inserts; and the lack of data on ADEs for
individual drugs. More fundamentally, few data are available
suggesting to what extent ADEs are due to genetic variation.
Thus, simply obtaining more data on ADEs will not resolve
this problem. The FDA is taking a proactive approach to
exploring how to use pharmacogenomics, including the
development of several key guidance documents that will be
released in the near future. These documents include
guidance on pharmacogenomic data submissions; guidance
on the co-development of pharmacogenomic based drugs,
biological products and diagnostic tests; and an initiative on
addressing the drug “pipeline problem”.
Conclusions: As illustrated by the recent Cox-2 cases,
improving the safety of drugs is a vital priority. As illustrated
by the recent drug safety cases, improving the safety of drugs
is a vital priority. Pharmacogenomics has the potential to
improve drug safety by providing more tailored regimens and
thus improving health care outcomes and decreasing costs;
however, many challenges remain to achieve these goals.
Adequate data are not presently available regarding ADE rates,
common pathophysiologies amongst adverse events, and
genetic determinants of ADEs to permit an evaluation of the
potential for pharmacogenomics to improve drug safety or
impact healthcare costs.
Implications for Policy, Delivery, or Practice: Regulatory
policies implemented by the FDA may directly impact the
availability and accessibility of data by creating incentives or
disincentives for example to share data or to collect certain
types of data. The current FDA initiatives are expected to
increase the amount of PGx data that are used and to make
such data more widely available; however, more remains to be
done. The challenges in using pharmacogenomics to improve
drug safety will increasingly require interdisciplinary efforts
and require health services researchers to play a more visible
role.
Primary Funding Source: NCI
prevalence of error at specific process-level steps for ED
management of allergy data
Study Design: A prospective observational study of
parent/child dyads presenting to an urban tertiary care ED.
Consecutive dyads were identified at triage; parents were
subsequently approached to complete a structured gold
standard interview (GSI) to judge the allergy history. Parents
were asked "Has your child ever had a problem or reaction to
a medication that he/she was given?" A yes or not sure
answer was further explored to determine if an immune-based
reaction was causative. Records inclusive of nursing forms
and medication order sheets were abstracted. The main
outcome was the rate of false positive (FP) and false negative
(FN) data specific to medication allergies documented at
nursing triage and on medication order forms.
Population Studied: English and Spanish-speaking parentchild dyads presenting for ED care at an urban tertiary care
hospital.
Principal Findings: We completed 256 observations over 23
days in summer 2004; 211 of 256 (82.4%) parents completed
the GSI. 33 of 59 medication problems/reactions were judged
immune-based; 20/33 were Type I hypersensitivity reactions.
At triage, twenty-seven of 33 instances of allergy were correctly
documented (18.2% FN rate). 8 of 186 patients without
allergies to medications were incorrectly noted to have an
allergy (4.3% FP rate.) 111 subjects had at least one medication
ordered. 3/111 patients with no true allergy had incorrect data
recorded on an order form (2.7% FP rate) and 4/111 patients
with a true allergy had no allergy recorded (3.6% FN rate). No
medication contraindicated by allergy history was ordered.
Parents reported that 23/201 (11%) physician interviewers did
not review allergies. 43 patients received a medication prior to
GSI; parents reported 6/43 (14%) treating nurses did not
review the allergy history prior to giving medication.
Conclusions: Current ED practice produces both false
negative and false positive data for the topic of allergies to
medications. These errors may perpetuate based on limited
subsequent interviews with parents and incorrect actions
during the process of ordering and delivering medications.
Implications for Policy, Delivery, or Practice: A "front-end"
health information technology solution that captures allergy
data directly from parents could improve the valid capture and
dissemination of critical data that influences safe prescribing.
Primary Funding Source: AHRQ, Local hospital award
●Getting the Data Right: Evaluating Information Accuracy
in Emergency Medicine
Stephen Porter, M.D., MPH, MSc, Shannon Manzi, PharmD,
Diana Volpe, RN, Anne Stack, M.D.
●A Comparison of Formal Consensus Development
Methods Used for Developing Clinical Guidelines
Rosalind Raine, BSc, MBBS, MSc Ph.D. FFPH, Andrew
Hutchings, BSc, MSc, CPFA, Colin Sanderson, MA, MSc, PhD,
Nick Black, M.D., FFPH, FRCS, FRSA, DRCOG, DCH
Presented By: Stephen Porter, M.D., MPH, MSc, Assistant
Professor, Division of Emergency Medicine, Children's
Hospital Boston, 300 Longwood Avenue, Boston, MA 02115;
Tel: (617)355-6624; Fax: (617)730-0335; Email:
stephen.porter@childrens.harvard.edu
Research Objective: Information management in the
emergency department (ED) setting is prone to risk. Allergies
to medication represent a critical element of a patient's
medical history that must be accurately captured and
communicated to ensure safety. Our objectives were: 1) to
identify gaps in ED system performance for capture and
integration of patients allergy histories, and, 2) to identify the
Presented By: Rosalind Raine, BSc, MBBS, MSc Ph.D. FFPH,
MRC Clinician Scientist/senior lecturer in health services
research, Public Health and Policy, London School of Hygiene
& Tropical Medicine, Keppel Street, London, WC1E 7HT; Tel:
(0) 207 927 2038; Fax: (0) 207 580 8183; Email:
rosalind.raine@lshtm.ac.uk
Research Objective: To compare two consensus
development methods commonly used for developing clinical
guidelines (Delphi and nominal group technique) in terms of
the judgments produced, closeness of consensus,
concordance with research evidence and reliability.
Study Design: Groups rated the appropriateness of four
treatments (cognitive behavioural therapy, behavioural
therapy, brief psychodynamic interpersonal therapy and
antidepressants) for three conditions (irritable bowel
syndrome, chronic fatigue syndrome and chronic back pain),
in the context of various clinical and social cues. Main
outcome measures were the mean differences in group
ratings and extent of agreement about those ratings, odds
ratios for concordance of group ratings with research
evidence, and weighted kappa statistics for reliability of ratings
between groups. A group's rating for a scenario was the
median of individual ratings and extent of agreement was the
mean absolute deviation from the median.
Population Studied: 213 general practitioners and mental
health professionals in four Delphi and four nominal groups.
Participants were randomly sampled from UK national clinical
databases.
Principal Findings: The effect of consensus method on final
ratings varied with therapeutic intervention. Within-group
agreement (consensus) was closer in the nominal than in the
Delphi groups in both rounds. Except for brief psychodynamic
interpersonal therapy (p<0.001) there was no difference in the
improvement in agreement. There was no difference between
groups in their concordance with research evidence (odds
ratio 1.13, 95% confidence interval 0.79-1.61). The Delphi
method was more reliable (Kappas 0.88 and 0.89 compared
with 0.41 and 0.65 for nominal groups), largely due to the
difference in group size.
Conclusions: The advantages of nominal groups (more
consensus; greater understanding of reasons for
disagreement) could be combined with the greater reliability
of the Delphi approach by developing a hybrid method.
Implications for Policy, Delivery, or Practice: Clinical
practice guidelines are extensively used to improve the quality
of health care. The methods used to develop guidelines vary
widely but there is a shift towards basing them on a
combination of the best available scientific evidence and
consensus judgements obtained by formal, explicit methods.
Little is known about the relative advantages of the Delphi
method, which is conducted by mail, compared with the
Nominal Group Technique which includes a facilitated
meeting. Our study is the first to compare directly the
reliability of Nominal and Delphi groups for developing
clinical guidelines. Our results are being used to inform the
future development of guidelines by the UK National Institute
of Clinical Excellence, whose aim is to develop guidelines to
reduce disparities by improving quality of care for all.
Primary Funding Source: Medical Research Council UK
●Switching Health Plans and the Role of Perceived Quality
of Care
Julie Rainwater, Ph.D., Patrick S. Romano, M.D., MPH, Jorge
S. Garcia, M.D., MPH
Presented By: Julie Rainwater, Ph.D., Research Scientist,
Center for Health Services Research in Primary Care,
University of California Davis, 2103 Stockton Boulevard,
Sacramento, CA 95817; Tel: (916)734-5265; Fax: (916)734-2732;
Email: jarainwater@ucdavis.edu
Research Objective: Pay-for-performance initiatives should
foster value-based health plan choices as plans compete on
both cost and quality of care. While several studies find that
increases in costs drive consumers to switch plans, the role of
other factors, including beliefs about quality of care are less
clear. This study prospectively evaluates factors related to
health plan switching.
Study Design: An observational study linked to the 2002
Open Enrollment (OE) period of California’s largest
purchasing organization was conducted. Members were
surveyed before Open Enrollment. After OE, health plan
selection data was supplied by the employer and respondents
were resurveyed to find out why they switched.
Population Studied: 2,500 members of the California Public
Employees Retirement System (CalPERS). Pre-OE
questionnaires were completed by 69% of the total sample;
81.3% of the pre-OE respondents completed the post-survey.
Principal Findings: Overall, 6% of members surveyed
voluntarily switched health plans during OE. The most
frequently cited reasons for switching were to lower costs, copayments, or out-of-pocket expenses (40%), or wanting a
better choice of doctors (34%). Switch rates varied from 11.1%
among members who were facing monthly premium
increases of $25-$49 to 2.8% among those facing smaller
increases. Switch rates varied from 9.5% among persons
aged 40 years or younger to 1.1% among those over 70, and
from 8.7% among persons with annual household income
between $50,000 and $75,000 to 1.2% among those whose
household income was less than $30,000. Respondents with
a history of switching plans (12.1%) or medical groups (9.5%)
in the past three years, or with an obstetric-related
hospitalization in the past year (16.7%), were especially likely
to switch. Kaiser members were significantly less likely to
switch plans than members of other HMOs (1.0% versus
11.1%). Controlling for expected premium increase, pre-OE
plan satisfaction was significantly associated with switching.
Switchers were more likely to be dissatisfied with their care or
plan, to have had problems getting needed care, and to have
had treatment delays while waiting for approvals from their
current plan. Switchers were more likely to report feeling
susceptible to poor care if they stayed with their current plan,
indicating that, if they stayed with their current plan, they
anticipated problems with having to find a new doctor or with
obtaining referrals or tests. Respondents who reported
hearing negative stories about their plan in the media over the
past year were significantly more likely to switch plans.
Conclusions: Costs are the main driver of health plan
switching, but perceived susceptibility to continued poor care,
negative media reports, and dissatisfaction with providers and
plans also play a role.
Implications for Policy, Delivery, or Practice: Mediareported negative information about health plan quality did
stimulate increased switching, suggesting a role for public
reporting in driving the health care system toward higher
quality. However, respondents' own experiences were the
dominant quality-related predictors of switching. Health plans
and providers that operate in the pay-for-performance
environment should continue to be responsive to consumer
experiences and their beliefs about what constitutes quality of
care.
Primary Funding Source: AHRQ
●The Relationship of Hospital Organizational Culture to
Patient Safety Indicators
Peter Rivard, MHSA, Cindy L. Christiansen, Ph.D., Shibei
Zhao, MPH, Susan A. Loveland, MAT, Amy K. Rosen, Ph.D.
Presented By: Peter Rivard, MHSA, Research Associate,
Center for Healthcare Quality, Outcomes, and Economics
Research, Veterans Administration, 200 Springs Road (152),
Bedford, MA 01730; Tel: (781)687-3573; Fax: (781)687-2227;
Email: rivardp@bu.edu
Research Objective: There are increasing efforts to
understand, measure, and improve the organizational culture
of patient safety. While numerous studies have examined the
relationship between organizational culture and quality
improvement, little attention has been focused on the
relationship between culture and hospital patient safety. This
paper examines the association between one aspect of
hospitals’ organizational culture, as indicated by shared
values, and rates of potential patient safety events. There are
three competing hypotheses from organization theory: a
culture where hierarchy and bureaucracy are important is (1)
positively associated with safety because it supports the
necessary integrating role of formal structures; (2) negatively
associated with safety because it inhibits timely response to
new threats to patient safety and other aspects of
organizational learning for safety; or (3) not directly associated
with safety.
Study Design: Potential patient safety events were measured
using the Agency for Healthcare Research and Quality
(AHRQ) Patient Safety Indicators (PSIs). Bayesian models
were used to estimate observed-to-expected ratios for the six
most frequent indicators; ratios were then combined, using
simulation methods and posterior densities, to create a single
combined indicator for each hospital. We used the Zammuto
& Krakower (1991) Competing Values instrument, which
indicates shared values in terms of the relative dominance of
each of four archetypes: hierarchical/bureaucratic, group,
entrepreneurial, and rational cultures. Data were obtained
from a previously administered employee survey. To examine
the relationship between the shared-values measure and
combined PSI rates, we ran bivariate correlations and linear
regressions; control variables included hospital bed size and
teaching status.
Population Studied: We studied 115 Veterans Administration
inpatient facilities. PSI rates were calculated using
administrative data from 2001. Culture surveys were
administered to hospital employees in 2000.
Principal Findings: Results support hypothesis 3: we found
no relationship between culture and patient safety measures.
Bivariate correlation between the hierarchical/bureaucratic
shared values measure and the combined patient safety
indicators was r = -0.05 (p = 0.58); multivariate linear
regression of the combined patient safety indicators on the
hierarchical/bureaucratic shared values measure showed a
small, non-significant relationship (Adjusted R-square = 0.016; p = 0.76).
Conclusions: There appears to be no direct association
between the importance of hierarchy and bureaucracy in a
hospital’s culture and that hospital’s performance on patient
safety indicators. Based on organization theory, we speculate
that this relationship may be mediated by the extent to which
a given hospital’s structures of hierarchy and bureaucracy can
be characterized as enabling, as opposed to coercive.
Implications for Policy, Delivery, or Practice: While a great
deal has been done to apply knowledge about safety culture
from other industries to healthcare settings, this study
suggests that we have more to learn about the role of
hierarchical and bureaucratic values in a hospital
organizational culture of patient safety. It may be more
important for patient safety leaders to focus on the particular
functions of hierarchy and bureaucracy rather than on their
extent or overall importance in the hospital setting.
Primary Funding Source: VA
●Practice Pattern Analysis – Quality Improvement Though
Intervention
Deborah Rogers, BS, Soyal Momin, MS, MBA, Alan Denham,
Raymond Phillippi, Ph.D., Judy Slagle, RN, MPA, Ken Patric,
M.D.
Presented By: Deborah Rogers, BS, Senior Research Analyst,
Health Services Research, BlueCross BlueShield of Tennessee,
801 Pine Street, Chattanooga, TN 37402; Tel: (423)755-5889;
Fax: (423)755-5100; Email: Deborah_Rogers@bcbst.com
Research Objective: The primary objective of this study was
to develop a Practice Pattern Analysis addressing the quality of
care delivered to the Medicaid population. The secondary
objective was to improve the practice patterns of the providers
through intervention.
Study Design: The Practice Pattern Analysis (PPA) was
designed to address three issues: access to care, utilization,
and quality of care. Access to care measures consisted of PCP
Visits and Specialist Visits per member per year. Utilization
measures included were Admissions/1,000, Emergency Room
Visits/1,000, Outpatient Surgeries/1,000, Presciptions/1,000,
Percentage of Generic Drugs Dispensed, and Out-Of-Network
Referrals. Quality of care measures incorporated percentages
of Mammography Screenings, Cholesterol Screenings, and
Well Child Visits where applicable. Primary Care Practitioners
(PCPs) were provided a comparison of their panel access,
utilization, and quality of care to that of their peer group for
the Medicaid population, with the peer group being the
physician specialty. The PPA also provided the Regional
Medical Directors and Regional Clinical Network Analysts with
a snapshot of practitioner performance within his/her region.
Data was examined on a quarterly basis with a three-month
run out to complete the data. Outliers, (i.e. those with Out-OfNetwork Referrals, Specialist Visits, or Emergency Room
Visits/1,000 above the 90th percentile,) were then provided
with a detailed listing of members for that measure. The
Regional Clinical Network Analyst or Regional Medical
Director then contacted the practitioner to discuss the practice
pattern. Additionally, the Practice Pattern Analyses were
subdivided between Adult and Pediatric populations to
provide an accurate median for each specialty. Specialty
Medians for each measure, presented in graphical format,
allowed the provider to quickly gage his/her status, as well as
make a comparison to his/her peers within his/her specialty.
A pre-post study design was utilized to evaluate the impact of
the PPA.
Population Studied: To be included in the study, a physician
must have had a panel size of 50 or more members for the
quarter. The population was approximately 850 practitioners
overseeing the care of approximately 465,000 Medicaid
members.
Principal Findings: A comparison of second quarter 2003
and second quarter 2004 PPAs was conducted. Favorable
declines were noted in the Outpatient Sugeries/1,000,
Admissions/1,000 and Out-Of-Network Referrals.
Advantageous increases were noted in the PCP Visits Per
Member Per Year, as well as the percentage of Well Child
Visits and Mammography Screenings. Adverse findings were
noted as increases in the Emergency Room Visits/1,000 and
the Specialist Visits Per Member Per Year, and the decline in
Cholesterol Screenings.
Conclusions: The PPA continues to be an effective tool for
monitoring provider performance and improving quality of
care for the Medicaid population. Negative behaviors and
patterns can be positively identified and addressed by the
appropriate staff in an ongoing improvement process
Implications for Policy, Delivery, or Practice:
1. PPAs can be used to decrease the variability in provider
behavior. 2. PPAs can be used to support the Network
Development Process. 3. PPAs should be implemented using
a multi-disciplinary approach.
Primary Funding Source: BlueCross BlueShield of Tennessee
●Violation of Patients' Trust Expectations: What
Determines Disrupted Trust in Physicians and Plans
Marsha Rosenthal, MPA, Ph.D.
Presented By: Marsha Rosenthal, MPA, Ph.D., Center for
Gerontology and Health Care Research, Brown University, 2
Stimson, Providence, RI 02912; Tel: (401)863-3401; Email:
marsha_rosenthal@brown.edu
Research Objective: This study examines how older patients
lose trust following a problem with health services. Although
researchers have examined patients’ trust in doctors and
health plans, they have not studied what causes patients to
lose trust, nor what restores trust if it has been disrupted. The
subject takes on added significance with managed care, as
patients navigate a system that relies on their informed
decisions about quality of care. This paper examines patients’
expectations of physicians and plans, and how a violation of
those expectations is evaluated by patients to decide if a
doctor or plan is trustworthy. The research further aims to
identify how patients’ loss of trust is influenced by 1)
enrollment in a managed care plan; 2) plan ownership
(nonprofit vs. for-profit status); 3) problems with care,
coverage or access to health services.
Study Design: A national telephone survey of consumer trust
in health care was conducted in 2002. Multivariate analyses
(OLS, logistical regression, ordered logit) were used to analyze
responses about patients’ expectations and experiences with
physicians and plans. In 2003, semi-structured interviews
were conducted with volunteers, expanding on the questions
in the survey. These responses were examined for repeated
categories and themes, and for insights into patients’
perspectives.
Population Studied: Three hundred eighty-one Medicare
beneficiaries were studied from a national survey of 5000
adults. In-depth interviews were conducted with 107 Medicare
beneficiaries at six sites in New Mexico and New Jersey.
Principal Findings: Survey and interview responses provided
new insights into how patients make decisions about whether
physicians or plans are trustworthy. Qualitative data revealed
that patients have “hidden expectations” beyond those
typically stated on surveys. If these are violated, there is loss of
trust in doctors and plans. However, patients feel restored
trust if doctors or plans communicate effectively and work to
resolve the issue. Survey data showed that patients in
managed Medicare and traditional Medicare lose trust for
similar reasons. Patients who have problems with care do not
lose trust in their doctors, but patients who have problems
with coverage lose trust in their plans (p<.01). However,
patients who believe they are in a nonprofit plan have greater
expectations that they can trust their doctors (p<.05) and
plans (p<.01); patients are more likely to maintain trust when
they have an adequate choice of plans (p<.01).
Conclusions: Managed care per se does not influence
patients’ decisions about trusting doctors or plans. However,
plans’ nonprofit status connotes more trustworthy behaviors
to patients. Among such behaviors, communication matters
most in maintaining patient trust, along with having a choice
of plans.
Implications for Policy, Delivery, or Practice: Patients’ loss
of trust can affect their evaluation of quality of health services,
plan selection, health care utilization and compliance. 1) As
the Medicare Advantage program moves forward, federal
policy should take account of the importance that adequate
choice of plans plays maintaining patient trust. 2) Physicians
and plans should take account of the value of communication
and problem resolution in maintaining patient trust.
Primary Funding Source: AHRQ, Aspen Institute
●The Severity of Medication Errors Identified by a Bar
Code Point of Care Administration System
Julie Sakowski, Ph.D., Jeff Newman, M.D., MPH, Tim Schiro,
PharmD, Beverly Michaelsen, MBA, Patricia Zrelak, Ph.D., RN
CNRN, Krystin Dozier, RN
Presented By: Julie Sakowski, Ph.D., Senior Health Services
Researcher, Institute for Research and Education, Sutter
Health, 345 California Street Suite 2000, San Francisco, CA
94104; Tel: (415)296-1808; Fax: (415)296-1844; Email:
sakowsj@sutterhealth.org
Research Objective: Bar code point of care medication
administration (BPOC) systems are effective at mitigating
medication administration errors. Previously, we found that
the BPOC implemented in one hospital system prevented
errors in 1.1% of administrations, but an additional 2.8% of
administrations deviated from the order written. However,
the clinical impact these errors could have on patients is
unclear, making it difficult to draw conclusions about the
impact of BPOC systems on patient safety. The purpose of
this study is to assess the potential clinical significance of the
prevented and observed errors identified by a BPOC system.
Study Design: This study utilized a previously validated
technique to estimate the potential severity of medication
errors without knowing patient outcomes. A panel of
pharmacists, nurses, and physicians reviewed medication
administration error scenarios and rated the potential severity
of the errors between 0-10, where 0 signified no effects and 10
indicated probable death. A severity index was calculated for
each scenario using the average rating from all reviewers. The
medication error scenarios were compiled from BPOC
generated error reports. Errors included early and wrong
doses and administration of medications not ordered or
discontinued (N=127,117, 134, 46). Both observed and
prevented errors were included in the sample, and reviewers
were blinded to prevent potential bias. Descriptive statistics
including the number of error events receiving each rating
score and the mean severity were calculated. Reduced form
linear regressions were performed to estimate the
associations among prevented and observed errors, type, and
severity.
Population Studied: Medication errors occurring in adult
inpatients at 6 community hospitals affiliated with a large nonprofit health system in California.
Principal Findings: Among our 424 error scenarios (N=304
observed, 120 prevented) we found that only 10% (95% CI =
7% - 12%) might potentially result in any adverse effects
(N=32 observed, 10 prevented errors). The most severe errors
involved attempted administration of the wrong drug (N=2)
and the wrong dose (N=1). Regression analyses indicate that
‘dose early’ errors are less severe than no order in system,
order discontinued, and wrong dose errors. We were not able
to detect any difference in severity or type classification
breakdown between prevented and observed errors.
Conclusions: While 90% of the medication errors detected by
our BPOC system were minor, the frequency of more serious
errors was similar among those prevented by and observed
despite BPOC. Applying our findings to the 3,000,000
medication administrations delivered via BPOC in our network
suggests 33,000 errors were prevented by BPOC; 3,300 of
which would have resulted in moderate to severe adverse
events.
Implications for Policy, Delivery, or Practice: Our findings
support the potential benefits of BPOC. However, the number
and severity of errors occurring despite its implementation
suggests that there are opportunities to improve the use of
these systems. We recommend users concentrate on
developing processes to prevent occurrences of no order in
system, wrong dose, or order discontinued errors, especially
for the high-alert drugs most likely to cause harm including
anticoagulants, insulin, opiates and narcotics, injectable
potassium chloride, and sodium chloride (solutions greater
than 0.9%) to maximize the impact on patient safety.
Primary Funding Source: No Funding Source
●Nursing Factors and Patient Outcomes in VHA
Anne Sales, MSN, Ph.D., Nancy Sharp, Ph.D., Gwendolyn
Greiner, MSW, MPH, Yu-Fang Li, Ph.D., Elliott Lowy, Ph.D.
Presented By: Anne Sales, MSN, Ph.D., Research Scientist,
Health Services Research and Development, VA Puget Sound
Health Care System, 1660 South Columbian Way, Seattle, WA
98108; Tel: (206)764-2068; Fax: (206)764-2935; Email:
ann.sales@med.va.gov
Research Objective: We report on the association between
nursing factors, including staffing levels and skill mix, and
patient outcomes, including overall inpatient mortality and
conditional mortality, similar to failure to rescue for a wider
group of patients among inpatients in Veterans Health
Administration (VHA) hospitals. Conditional mortality
measures mortality of patients who experience serious
complications that may be sensitive to inpatient nursing care.
Study Design: Retrospective observational study. We used
VA cost accounting data for number of nursing personnel
hours per month by type of nurse and skill mix, at the nursing
unit level, in 125 VA medical centers. Data on nursing process
are from a survey of all nursing personnel in these 125 VA
medical centers. Data on patient characteristics, pre-existing
conditions, complications, and outcomes of admission for all
patients admitted to VHA inpatient care from 2/03 to 6/03
come from the VA inpatient Patient Treatment Files. All
analyses were conducted at the patient level, including nursing
unit and facility level variables. We stratified by ICU and nonICU admitting unit and estimate the model in two steps, first
predicting patient probability of experiencing a major
complication, then probability of death conditional on
experiencing a complication, controlling for probability of
complication. Results are preliminary with analysis
continuing, including correction for the complex multi-level
error pattern.
Population Studied: We included 14,719 VHA patients first
admitted to an ICU and 78,193 patients first admitted to an
acute medical/surgical unit.
Principal Findings: Overall mortality for ICU patients was 6%
with conditional mortality 16%. Overall mortality for acute
med/surg patients was 3% with conditional mortality 7%. In
the multivariate regression predicting probability of death after
experiencing a complication, several nursing process and
facility variables were significant with predicted probability of
complication increasing the risk of death very significantly in
both ICU and acute patients. Among ICU patients, increased
total nursing hours per patient day were related to increased
risk of death (OR 1.31, p=0.03) while increased RN hours per
patient day were related to decreased risk of death (OR 0.71,
p=0.01). Among acute patients, the inverse was found: total
nursing hours were related to decreased risk of death (OR
0.79, p=0.01) while RN hours were related to increased risk of
death (OR 1.30, p=0.01). Skill mix was significantly
associated with decreased risk for acute patients but not for
ICU patients.
Conclusions: Preliminary results appear to indicate different
patterns of association of nurse staffing with patient
outcomes for ICU compared to non-ICU acute patients. Prior
large-scale studies have not distinguished between patients
initially admitted to ICU vs. non-ICU care. These results may
indicate different approaches to nurse staffing patterns in
intensive vs. acute care.
Implications for Policy, Delivery, or Practice: This is the first
large-scale study of nursing factors and patient outcomes
using unit level rather than facility level data and is the first
opportunity to explore differences among these two settings.
Nurse staffing continues to be a critical factor in providing
optimal patient care. Understanding the effects of different
settings is essential.
Primary Funding Source: VA
●Coverage of Asthma Care in Employer-Sponsored Health
Insurance Plans
Marissa Scalia, MPH, Seymour Williams, M.D.
Presented By: Marissa Scalia, MPH, Fellow, Air Pollution and
Respiratory Health Branch, Centers for Disease Control and
Prevention, CDC/NCEH/APRHB, 1600 Clifton Road MS E17,
Atlanta, GA 30333; Tel: (404)498-1879; Fax: (404)498-1088;
Email: mscalia@cdc.gov
Research Objective: Providing quality asthma care may
reduce excess medical expenditures and increase productivity
for plan enrollees with asthma. Our study evaluated the
coverage of quality asthma care in a convenience sample of
employer-sponsored health insurance plans.
Study Design: We identified the standards of asthma care as
defined by the National Asthma Education and Prevention
Program (NAEPP) and assessed the coverage of the
associated activities found in health insurance benefits
summaries received from three employers. Health benefits
managers clarified information missing from or ambiguous in
the summaries. We also compared the smoking cessation
benefits to the benefit design recommendations made by the
Centers for Disease Control and Prevention’s (CDC) Office on
Smoking and Health.
Population Studied: Summaries of 22 plans were provided by
two Fortune 500 companies and one government agency.
Our review identified eight preferred provider organizations
(PPOs), seven fee-for-service with PPO option plans, six health
maintenance organizations, and one consumer-driven plan.
Principal Findings: All of the plans cover most of the
recommended interventions. In two of the plans, neither
spacers nor chambers are covered by either the pharmacy
benefit or as medical supplies. Disease management (DM)
programs for asthma are available in fourteen of the plans.
Education as a stand-alone service, including asthma
education, is excluded by fourteen of the plans. Fifteen of the
22 plans cover nicotine replacement therapy and Bupropion
for smoking cessation as part of the pharmacy benefit or
smoking cessation benefit. Each of six plans has a smoking
cessation program. One of these six offers counseling alone;
four offer counseling combined with nicotine replacement
therapy and Bupropion; and one offers Bupropion and
nicotine replacement but no counseling.
Conclusions: All of the 22 plans reviewed cover most of the
recommended equipment and activities of quality asthma
care; however, some activities of care are inconsistently
covered by these health insurance plans. Asthma education is
not covered in more than half of the plans because education
is excluded, although some education may be delivered while
providing another service (e.g., during an appointment).
Although over half of the plans include Bupropion and
nicotine replacement therapy, less than a quarter of the plans
offer a smoking cessation program that includes counseling,
and fewer offer a multifaceted smoking cessation program.
Implications for Policy, Delivery, or Practice: Our review of
these plans reveals at least three areas where improvement in
coverage would satisfy NAEPP recommendations. First,
NAEPP recommends providing spacers and chambers, which
if covered will improve the delivery of medications, especially
for certain high-risk groups (i.e., children). Second, since the
guidelines emphasize a need for self-management, covering
asthma education as a stand-alone service should be
considered. Third, smoking cessation should be the foremost
priority for coverage in these health insurance plans since the
health improvements and savings for both enrollees and
employers would be substantial. Although not addressed in
the NAEPP guidelines, improved asthma care might also be
achieved by increasing the number of plans with asthma DM
programs if these programs meet all the NAEPP guidelines.
Primary Funding Source: CDC
●Regulation and Quality Improvement in Invasive
Cardiology Facilities
Charles Schade, M.D., MPH, John G. Brehm, M.D., FACP,
David Lomely, BS
Presented By: Charles Schade, M.D., MPH, Medical
Epidemiologist, Scientific Support, West Virginia Medical
Institute, 3001 Chesterfield Place, Charleston, WV 25304; Tel:
(304) 346-9864; Fax: (304) 346-9863; Email:
cschade@wvmi.org
Research Objective: (1) To determine whether invasive
cardiology procedures can be safely performed in facilities
without on-site cardiac surgery serving patients of substantial
cardiac risk in a rural state; and (2) to validate the reliability of
routine clinical data reporting in support of safety monitoring.
Study Design: Time series
Population Studied: Three newly-licensed invasive cardiology
acilities in West Virginia performing angioplasty without onsite cardiac surgery, each performing procedures (85%
angiography, 15% percutaneous transluminal angioplasty) on
about 1,000 patients per annum. Under contract with the
West Virginia Health Care Authority, we monitored data these
facilities submitted to the American College of Cardiology
National Cardiovascular Data Registry (NCDR) in 2003 and
2004. We audited 30 records per facility submitted following
the third quarter of 2003, using a trained nurse abstractor to
collect NCDR data elements from the original patient charts.
Each facility received a detailed report of the results of its first
year of operation and audit findings, with recommendations
for improvement.
Principal Findings: Results are now available for 6
consecutive quarters (all prior to facility feedback), and results
will be available for the first post-feedback quarter before the
meeting. Patients seen in the facilities were generally of high
cardiac risk (21.5% obese [BMI>35]; 28.9% with history of
diabetes; 63.1%, hypertension; 64.2%, hypercholesterolemia;
22.9%, current smoking). Complications occurred
sporadically, with total significant adverse events remaining
less than the relevant ACC benchmark (4.5%) statewide in
every quarter, and in each facility for all quarters but one.
Complication rates have not varied significantly among
facilities, though there are insufficient data to assess long
term trends. Audits confirmed unexplained variations in
certain demographic items (e.g., smoking prevalence) among
hospitals, and demonstrated generally reliable submission of
complication data (most agreed 100% of the time).
Conclusions: ?Experience to date with invasive cardiology
procedures in facilities without onsite cardiac surgery in a
rural state with a high prevalence of cardiac risk factors has
demonstrated complication rates not higher than expected
based on comparable NCDR data. Facility reports reliably
reflect events actually documented in medical records.
Implications for Policy, Delivery, or Practice: Feedback to
facilities following audits may present an opportunity to
improve documentation and care quality.
Primary Funding Source: West Virginia Health Care
Authority
●Incorporating Health Care Quality within Health
Antitrust Law
Helen Schneider, Ph.D.
Presented By: Helen Schneider, Ph.D., Economist, Petris
Center, UC-Berkeley and Los Alamos National Lab, D-3, Mail
Stop F607, Los Alamos, NM 87545; Email: hc72@cornell.edu
Research Objective: To examine the impact of hospital
mergers and competition on the post-merger quality of care.
Study Design: Regression analysis is used to examine
independent associations between hospital mergers and
health care outcomes. We use Coronary Artery Bypass Graft
(CABG) mortality to measure health care quality. In our further
analysis, broader quality indicators of patient experience and
of hospital quality rankings will be considered.
Population Studied: California general acute care hospitals
that report to the Office of Statewide Health Planning and
Development (OSHPD). The study separately examines the
effect of mergers on nonprofit hospitals.
Principal Findings: Preliminary findings show no post-merger
quality improvements. This result does not vary by hospital
ownership.
Conclusions: There is no significant association between
hospital mergers and competition and health care quality as
measured by the CABG mortality rates.
Implications for Policy, Delivery, or Practice: Antitrust
policy usually treats price as proxy for consumer welfare
although this claim is not supported by empirical evidence.
Therefore, quality should be incorporated into the antitrust
analysis. If mergers lead to higher prices and lower quality,
thus lower social welfare, the case for the antitrust challenge
of hospital mergers is strengthened.
Primary Funding Source: No Funding Source
●Patient and Caregiver Satisfaction with Care at the End of
Life: A Review of the Evidence
Lisa R. Shugarman, Ph.D., Karl Lorenz, M.D., Joanne Lynn,
M.D., M.S., Richard A. Mularski, M.D.
Presented By: Lisa R. Shugarman, Ph.D., Associate Health
Policy Researcher, RAND Corporation, 1776 Main Street, PO
Box 2138, Santa Monica, CA 90407-2138; Tel: (310) 393-0411
x.7701; Fax: (310) 393-4818; Email: Lisa_Shugarman@rand.org
Research Objective: Both patient and caregiver satisfaction
are key metrics of healthcare performance at the end of life. In
order to evaluate progress in the field of end-of-life care and
clarify research priorities we examined the research literature
on end-of-life care to determine the association of other
processes and outcomes of care with patient and caregiver
satisfaction.
Study Design: As part of a systematic review of end-of-life
care, we reviewed articles that evaluated patient and caregiver
satisfaction with care provided at the end of life.
Population Studied: Sources for our review included
Medline, the Cochrane Database of Reviews of Abstracts of
Effects (DARE), the National Consensus Project for palliative
care, and several recent systematic reviews from both Health
Canada and National Institute for Clinical Excellence (NICE),
United Kingdom. The searches were limited to published
articles in the English language (1990 – 2004), involving adult
human subjects.
Principal Findings: From 24,423 total citations, 1,274 articles
were reviewed, including 95 systematic reviews, 134
intervention, and 682 observational studies. We identified 10
systematic reviews, 12 intervention studies, and 17
observational studies on patient or caregiver satisfaction. The
preponderance of the literature supports the effectiveness of
palliative care for improving patient and caregiver satisfaction.
Subjective measures of the end-of-life care experience include
satisfaction and quality-of-care measures that overlap
significantly. Compared to global or nonspecific measures,
satisfaction instruments that assess focused aspects of care or
that were specifically developed for palliative care settings
have been most useful in demonstrating intervention effects.
Possibly for that reason, effects of interventions on
satisfaction have been somewhat inconsistent. Measures of
satisfaction that are more specific and strongly related to
explicit intervention aims or processes (e.g., communication,
pain control, practical support and enhanced caregiving) have
demonstrated greater sensitivity to change and support a
process-outcome relationship among these variables. The
relationship of other processes or attributes of care (e.g.,
treatment of symptoms other than pain, spiritual support,
continuity and coordination of care) to satisfaction is less
evident in the literature, although such relationships are
supported qualitatively. The ability to demonstrate
relationships between these aspects of care and satisfaction
may be partially related to challenges in defining spiritual
support as an intervention and measuring spiritual support
and continuity of care.
Conclusions: Our systematic review identified a very large,
diverse literature reflecting the tremendous growth and
importance of the field of end of life care over the last decade.
This review illuminates strengths of the field as well as
opportunities for research. We identified evidence supporting
the association of satisfaction with pain management,
communication, practical support, and enhanced caregiving;
however, little is known about the association of satisfaction
with non-pain symptoms, spiritual support and continuity.
Implications for Policy, Delivery, or Practice: Studies
evaluating satisfaction should use specific measures that
reflect processes of care, and studies are needed to examine
the relationship of satisfaction to less studied processes such
as non-pain symptoms, spiritual support, and continuity.
Primary Funding Source: AHRQ, National Institute of
Nursing Research
●Creating a Culture of Safety in Hospitals
Sara Singer, MBA, Anita Tucker, DBA
Presented By: Sara Singer, MBA, Doctoral Student, Health
Policy, Harvard University, 303-1 Sherman Hall, Boston, MA
02163; Tel: (617)495-5047; Fax: (617)496-4397; Email:
ssinger@hbs.edu
Research Objective: We aimed to provide a framework of
actions through which healthcare leaders can instill strong
safety cultures in hospitals. We also offer case studies of two
hospitals to illustrate the framework and to enable leaders to
evaluate their own efforts to create a culture of safety.
Study Design: Drawing on relevant findings from
organizational, leadership, and high reliability organization
literature, we developed a managerial framework for creating a
culture of safety. To illustrate our framework, we present
qualitative and quantitative data from two hospitals. We
gathered qualitative data through semi-structured interviews
with senior leadership and front-line staff as well as on-site
observation of safety-related meetings. In addition, to provide
a quantitative measure of our framework, we used survey data
from 6538 respondents from 44 hospitals participating in a
nationwide study of patient safety climate. Exploratory factor
analysis of survey questions related to senior leaders’ role in
patient safety resulted in a 7-item measure of patient safety
leadership (Cronbach’s alpha = 0.89). We analyzed the
average levels of problematic responses, i.e. respondents who
reported strongly disagreeing or disagreeing with items that
asked about positive safety practices, to compare the two
hospitals’ patient safety leadership.
Population Studied: We studied senior leaders and other
personnel from two hospitals that differed in size, location,
organization, and safety leadership. Survey participants at
each hospital included 100% of senior managers and
physicians and a 10% random sample of all other personnel.
Principal Findings: Strong safety culture leadership required
five actions: 1. setting and communicating a clear, compelling
safety vision, 2. focusing on system issues, 3. leading by
example, 4. valuing and empowering personnel, and 5. being
dissatisfied with the status quo. In addition, leaders needed
to apply these rules clearly and consistently. The case studies
demonstrated that safety vision, a systems approach, and a
willingness to act in support of safety were necessary but
insufficient for achieving a strong safety culture. In addition to
those three actions, strong safety leadership required
consistently empowering others to act on behalf of safety and
a constant quest for process improvement. The cases
highlighted the benefits of storytelling as a means of engaging
personnel and relationship training for encouraging
individuals to speak up. Survey results supported qualitative
findings.
Conclusions: Creating a compelling vision of safety and being
charismatic are important characteristics of safety leadership.
However, creating a strong safety culture also requires leaders
to empower others to act on behalf of safety and to overtly
value front line employees’ expertise and contributions in this
area. In addition, safety leadership demands a constant
quest for process improvement, even when admitting the
need for change poses a legal risk.
Implications for Policy, Delivery, or Practice: Developing
leadership skills to achieve strong safety culture is particularly
challenging in an industry that enforces hierarchy, individual
skill, and individual accountability. As financial pressures on
hospitals create starker tradeoffs between safety and
productivity, the need for the leadership skills explored in this
paper may become even more acute.
Primary Funding Source: AHRQ, Fishman Davidson Center
●Emergency Room – Next Frontier for Performance
Measures
Kurt Stevenson, M.D., MPH, Karen Benson-Huck, Ph.D.,
ARNP
Presented By: Kurt Stevenson, M.D., MPH, Lead Medical
Director, Healthcare Improvement and Research, Qualis
Health, 720 Park Boulevard, Suite 120, Boise, ID 83712; Tel:
(208) 389-5021; Fax: (208) 343-4705; Email:
kurts@qualishealth.org
Research Objective: To assess the feasibility of measuring
and reporting emergency room (ER) care for selected clinical
conditions by utilizing retrospective medical record
abstraction.
Study Design: Data were abstracted from ER medical records
for patients seen for 1) Acute myocardial infarction (AMI) 2)
Asthma 3) Closed head injury with subdural hematoma 4)
Traumatic pain 5) Visceral pain 6) Penetrating extremity
trauma and 7) Pulmonary embolus (P.E.). The conditions were
prioritized for study by experts using a modified Delphi
consensus process from a list of 56 potential areas for quality
improvement in ER care. Demographic data/information
about general care (such as documenting allergies) as well as
data for elements of care for a specific clinical condition were
collected and analyzed, with reports provided to the Centers
for Medicare & Medicaid Services (CMS) and to participating
ER’s for their use in quality improvement.
Population Studied: 978 patients age 18 and over from 15
states (Arizona, California, Florida, Georgia, Illinois, Indiana,
Iowa, Maine, Michigan, North Carolina, New Hampshire,
Pennsylvania, Texas, Washington, and Wisconsin) with ER
visits in critical access, rural, and urban hospitals from January
1, 2001 through July 30, 2004.
Principal Findings: Opportunities for improving care were
identified for each area: *AMI - Average time from ER arrival
to an EKG was 84 minutes (median time 19.5 minutes), while
the recommendation is 10 minutes. Only 38% of AMI
patients diagnosed in the ER eligible for Beta blockers received
the drug in the ER. *Asthma - Beta2 agonists were used only
76% of the time; yet this is a drug that can best provide
symptomatic relief for most patients. *Closed Head
Injury/Anticoagulant status - Elderly patients on anticoagulant
drugs because of conditions such as atrial fibrillation can have
unrecognized intracranial bleeding from closed head injuries.
Yet, an assessment of anticoagulant status was documented
in only 75% of patients who were diagnosed with a subdural
hematoma. *P.E. – This condition may be missed in the ER.
Even so, tools that assess for P.E. risk factors were used in
only 7 individuals out of 119 P.E. patients. *Pain - For patients
with hip fractures, the mean time from ER arrival to pain
medication was 80 minutes; the time to medication for
urinary tract calculi patients was 57 minutes.
Conclusions: It is possible to assess the quality of care in the
ER using medical records data to identify gaps in care,
opportunities for improving care, and to make changes in ER
processes and treatment that can improve patient outcomes.
Additional efforts to standardize ER terminology,
documentation, and reporting are needed in order to be able
to compare care across settings.
Implications for Policy, Delivery, or Practice: CMS is
implementing performance measures for ERs in critical access
hospitals starting in August, 2005. It is expected that
performance measures will eventually include all ERs in the
U.S. that treat Medicare patients, which will impact care
delivered in the ER and reimbursement policy across the
country.
Primary Funding Source: CMS
●Quality Improvement in Rural Hospitals: Results from
the Idaho Critical Access Hospital Collaborative
Kurt Stevenson, M.D., MPH, Sharon Wilson, RN, BS, Jane
Burgman, RN, James W. Moore, MS, Janel Galbraith, RN,
Mary Sheridan, RN
Presented By: Kurt Stevenson, M.D., MPH, Lead Medical
Director, Healthcare Improvement and Research, Qualis
Health, 720 Park Boulevard, Suite 120, Boise, ID 83712; Tel:
(208)389-5021; Email: kurts@qualishealth.org
Research Objective: Determine the impact of group learning,
as outlined by the Institute for Healthcare Improvement (IHI)
Breakthrough Series Collaborative model, on healthcare
performance improvement in a cohort of rural critical access
hospitals (CAH) in Idaho.
Study Design: Following the IHI model, CAH in Idaho were
recruited to participate in three learning sessions followed by
action periods and a final outcomes conference. Hospitals
were trained on evidence-based measures for acute
myocardial infarction (AMI) and community-acquired
pneumonia (CAP) and were provided tools for measurement
in the first learning session. They developed plans for testing
specific interventions and shared successes and failures at
each subsequent learning session. All data were self-reported
and assessed in a pre-post intervention study design.
Population Studied: Fourteen CAHs participated in this
demonstration project that was conducted from January 2004
to October 2004. All patients with either AMI or CAP were
included. Additionally, all eligible patients >65 years were
assessed for administration of pneumococcal vaccination.
Principal Findings: There was 100% participation by CAHs in
all aspects of the project. All teams shared their work, ideas,
intervention tools, lessons learned, and barriers to
improvement. Among required measures, the proportion of
AMI patients who received timely administration of aspirin
increased from 78.1% to 92.4%; the proportion of CAP
patients who received antibiotics within 4 hours improved
from 77.9% to 90.7%; and the proportion of CAP patients with
blood cultures within 24 hours improved from 15.1% to
40.0%. Among optional measures, the proportion of CAP
patients who received initial antibiotic therapy consistent with
current guidelines improved from 89.7% to 100%; the
proportion of CAP patients with arterial oxygenation
assessment during the first 24 hours improved from 98% to
100%; and. the proportion of smoking patients who were
counseled about smoking cessation improved from 0% to
53.7% for AMI patients and from 8.9% to 100% for CAP
patients. The total proportion of eligible patients >65 years
who received the pneumococcal immunization improved from
29.1% to 53.8%.
Conclusions: There were significant quality gaps in the
baseline performance for many of the quality indicators
examined in this cohort of rural hospitals. The hospitals,
however, were motivated to participate in a shared learning
collaborative model and demonstrated significant
improvements in care for almost indicators tested.
Implications for Policy, Delivery, or Practice: Rural
hospitals, although representing a significant portion of
healthcare delivery in the US, are often excluded from research
studies due to low patient census and small numbers of
practitioners. Based on the positive outcomes in this study,
rural hospitals should be considered in future research studies
and performance improvement projects. Assessments of
ideal performance improvement methodology may be needed
to address the unique characteristics of these hospitals.
Primary Funding Source: CMS
●Improving Patient Safety and Quality of Care Through
Bar Coding Solutions
Patsy Sublett, MSN, RN,-BC
Presented By: Patsy Sublett, MSN, RN-BC, Clinical Systems
Manager, Danville Regional Health System, 142 South Main
Street, Danville, VA 24541; Tel: (434)799-4517; Email:
sublettp@drhsi.org
Research Objective: Alarmed by the reports regarding the
increased number of patients that die every year from medical
errors, the staff at Danville Regional Health System (DRHS)
organized a multidisciplinary team to address the specific
issue of medication errors and analyze the organization’s
medication administration process enterprise-wide. The team
identified three objectives: 1) Streamline workflow; 2.) Reduce
medication errors; and 3.) Improve patient safety.
Study Design: Using Siemens Medical Solutions medication
management solution (Med Administration Check (MAC)),
DRHS set out to achieve its goals. MAC is an application that
helps nurses verify and document medication administration
at the point of care. Using wireless computers containing
electronic medical records and wireless bar code scanners,
nurses scan the bar code on a patient’s ID badge, which
displays an online medication administration work list on the
computer. The bar code on the medication is scanned, and
the system automatically checks the scanned medication vs.
the medication ordered. The computer then verifies that the
five “rights” of administration are accurate – right medication,
right dose, right time, right route, and right patient. If there is
a discrepancy—for example, the medication that was scanned
is the incorrect medication for the patient about to receive it—
an alert appears on the computer screen, warning the nurse
that one of the five rights is being breached, preventing a
potentially fatal error. Implementation began in October 2002.
Population Studied: DRHS implemented the MAC solutions
from Siemens throughout its 350-bed facility.
Principal Findings: Results were immediate. DRHS
documented an average of 116 potential medication errors
avoided each month, and an average annual savings of
$800,000 due to increased efficiency. In addition the facility
has eliminated all hard copy paper charting because all
medication distribution is charted online.
Conclusions: DRHS achieved the goals outlined in its original
program. Implementing healthcare information technology
(IT) solutions within the hospital arena to help manage
medication distribution can be successful. There has also
been improvement in workflow and the working relationships
between nurses and pharmacy staff has improved. Overall,
DRHS demonstrated that patient controversy (death, injury)
caused by medication errors can be controlled and potentially
eliminated.
Implications for Policy, Delivery, or Practice: The future of
the healthcare industry is indeed based on the ability to
implement healthcare IT solutions such as bar coding for
more efficiency, better patient care and cost effectiveness. Bar
coding and medication management are key for reaching
these goals. The implementation and success of the project at
DRHS can be used as an example for the industry.
Primary Funding Source: No Funding Source
●Toward a Clinically Plausible and Cost-Effective Risk
Adjustment System for Pay-for-Performance: Development
and Validation of Mortality Predictive Models Using
Automated Clinical Data
Ying Tabak, Ph.D., Karen Derby, BA, Yadong Yang, DS
Presented By: Ying Tabak, Ph.D., Senior Biostatistician,
Research, CardinalHealth (MediQual), 500 Nickerson Road,
Marlborough, MA 01752; Email: ying.tabak@cardinal.com
Research Objective: Pay-for-performance has been promoted
by many stakeholders. One obstacle to adopt this approach is
the lack of a clinically plausible risk adjustment system that
can be implemented on a large scale at low cost. Methods
using billing data lack precision and are poorly accepted by
physicians. Methods using clinical data from chart abstraction
are too costly to be practical. The objective of this study was
to develop and validate admission-based severity adjustment
models for heart failure, pneumonia, ischemic stroke, acute
myocardial infarction, and septicemia mainly using automated
laboratory data (LAB) for clinical severity measures
supplemented with electronically collected UB92 data for
demographics, discharge status, and comorbidities. The
values of vital signs and altered mental status, that were not
currently automated, were also analyzed.
Study Design: In-patient mortality, a widely used indicator of
quality, was used as the outcome measure. Demographics,
LAB, vital signs, and comorbidities (identified by 6th digit ICD9 coding) were covariates. Multiple levels of LAB and vital
signs were crafted per literature and empirical examinations.
Five logistic regressions models, one for each disease group,
were derived from 170,825 (11,129 deaths) admissions in
2000-2003 across 39 teaching and 39 non-teaching hospitals
that electronically imported LAB data to AtlasTM (MediQual).
C-statistic assessed model fit. Bootstrapping validated each
model internally. Manually abstracted data (n=569,708; 37,573
deaths) from 80 teaching and 124 non-teaching hospitals
validated the models externally.
Population Studied: A total of 740,533 adult admissions in
2000-2003 for heart failure (n=261,349; 10,088 deaths),
pneumonia (n=208,237; 9,354 deaths), ischemic stroke
(n=98,361; 6,221 deaths), acute myocardial infarction
(n=107,280; 11,334 deaths), and septicemia (n=65,306; 11,705
deaths) were analyzed.
Principal Findings: The median age ranged from 75 to 77.
The crude mortality was 3.9% (HF), 4.5% (pneumonia), 6.3%
(stroke), 10.6% (AMI) and 17.9% (septicemia). The most
significant predictors (p<.0001) entered into all five disease
groups included: age, albumin < 2.5 g/dl, (odds ratio range:
1.8-2.1), BUN > 40 mg/dl or creatinine > 3.0 mg/dl (OR range:
1.6-3.6), pH arterial < 7.3 or > 7.48 (OR range: 1.7-3.0), systolic
BP < 90 mm Hg (OR range 1.6-2.9), moderate/severe altered
mental status (OR range:2.3-11.3), and metastatic cancer (OR
range 2.0-3.4). The 95% confidence intervals for the predictors
were narrow. Other significant predictors included
hypo/hyperglycemia, hypo/hyperthermia, and other abnormal
hematologic parameters and electrolytes. The c-statistics for
all the derivative and validation models ranged from .82 to .85
with average of .835.
Conclusions: Laboratory findings provide objective and
precise measures of acute pathophysiologic conditions.
Admission LABs indicating acidosis/alkalosis, malnutrition,
hepatic or renal dysfunctions predict mortality for all five
clinical conditions. Vital signs and altered mental status are
highly significant and should be automated. Pathophysiologic
variables commonly measured on admission can generate
parsimonious and clinically plausible models with excellent
accuracy.
Implications for Policy, Delivery, or Practice: This study
evaluated five major diseases across a large number of
patients and hospitals. Based mainly on automated data,
these models are cost effective to implement as a risk
adjustment method for pay-for-performance. With increasing
automation of clinical data, a valid and cost-effective risk
adjustment system is feasible.
Primary Funding Source: No Funding Source
●Medicare Physician Group Practice Demonstration:
Design of Quality Incentives
Michael Trisolini, Ph.D., MBA, John Pilotte, MHS, Gregory C.
Pope, MS, John Kautter, Ph.D., Bela Bapat, MS, Heather
Grimsley, MBA
Presented By: Michael Trisolini, Ph.D., MBA, Senior Health
Services Researcher, Program on Health Care Quality and
Outcomes, RTI International, 411 Waverley Oaks Road, Suite
330, Waltham, MA 02452; Tel: (781) 788-8100; Fax: (781) 7888101; Email: mtrisolini@rti.org
Research Objective: To design and implement financial
incentives for quality improvement for physician group
practices (PGPs) serving Medicare fee-for-service (FFS)
beneficiaries.
Study Design: The Medicare PGP Demonstration Project
provides financial incentives for quality improvement if a PGP
can also demonstrate cost savings. Cost savings are
generated if a participating PGP holds its rate of growth in
average Medicare cost per beneficiary below the rate in its
surrounding service area. Those cost savings are then
allocated to two bonus pools, one for quality improvement
and one for cost containment. A goal of the demonstration is
to promote broader use of disease management and
coordination of care, as these efforts may provide for both
quality improvement and cost containment by reducing
complications and progression of chronic diseases that are
suffered by many Medicare beneficiaries. The quality portion
of the bonus pool is earned by meeting targets for quality
indicators. Quality indicators include 32 measures in four
modules: 1) diabetes measures; 2) congestive heart failure
measures; 3) coronary artery disease measures; and 4)
hypertension and cancer screening measures. Seven of them
are measured using Medicare claims data; 25 are measured
using medical record abstracts. For example, one claimsbased quality measure is the rate of HbA1c testing done
annually for diabetics. A medical-records based measure is
the level of HbA1c control achieved for diabetics. Two types of
targets can be met to earn the quality bonuses for each
measure. Threshold targets can be met if the PGP achieves a
higher rate of performance than a pre-determined threshold.
An improvement target is satisfied by demonstrating a 10%
improvement. Meeting either the improvement or the
threshold target means the PGP will earn the bonus payment
for that quality measure. Total quality bonus payments earned
each year will depend on the number of quality measures for
which either improvement or threshold targets were reached.
Population Studied: Medicare beneficiaries
Principal Findings: This presentation reports on the design
of quality incentives for this Medicare pay for performance
demonstration project. The demonstration is slated to begin
on April 1, 2005 and run through March 30, 2008. This
presentation also includes comparisons with the design of
other quality improvement incentive programs for physicians.
Conclusions: A quality incentive program that has the
potential to improve the quality of care for Medicare
beneficiaries can be designed for physician group practices
treating fee-for-service Medicare beneficiaries.
Implications for Policy, Delivery, or Practice: The PGP
Demonstration is a new reimbursement mechanism for
Medicare, and may be expanded beyond the PGPs
participating in this three-year demonstration if it proves
successful.
Primary Funding Source: CMS
●Overcoming Bias in Estimating the Volume-Outcome
Relationship
Alexander Tsai, Mark Votruba, Ph.D., John F.P. Bridges, Ph.D.,
Randall D. Cebul, M.D.
Presented By: Alexander Tsai, Department of Epidemiology
and Biostatistics, Case Western Reserve University School of
Medicine, 2511 Overlook Road, Suite 6, Cleveland Heights, OH
44106-2459; Tel: (216) 321-5496; Email: act2@case.edu
Research Objective: To examine the effect of hospital volume
on 30-day mortality for patients with congestive heart failure
(CHF) using administrative and clinical data in conventional
regression and instrumental variables (IV) estimation models.
Study Design: The patient was the primary unit of analysis.
We fit a linear probability model to the data to assess the
effects of hospital volume on patient mortality within 30 days
of admission. Both administrative and clinical data elements
were included as risk adjusters to illustrate the importance of
appropriate risk adjustment. Linear distances between
patients and hospitals were used to construct the instrument,
which was then used to assess the endogeneity of hospital
volume.
Population Studied: The primary data consisted of
longitudinal information on comorbid conditions, vital signs,
clinical status, and laboratory test results for 21,439 Medicareinsured patients aged 65 years and older hospitalized for CHF
in northeast Ohio in 1991-1997.
Principal Findings: When only administrative data elements
were included in the risk adjustment model, a statistically
significant inverse volume-outcome relationship was identified
(P=0.045). The estimate was markedly attenuated when
clinical data were added to the model as risk adjusters
(P=0.28). IV estimation shifted the estimate in a direction
consistent with selective referral, but we were unable to reject
the consistency of the linear probability estimates.
Conclusions: Use of only administrative data for volumeoutcomes research may generate spurious findings. The IV
analysis further suggests that conventional estimates of the
volume-outcome relationship may be contaminated by
selective referral effects.
Implications for Policy, Delivery, or Practice: Taken
together, our results suggest that efforts to concentrate
hospital-based CHF care in high-volume hospitals may not
reduce mortality among elderly patients.
Primary Funding Source: AHRQ
●Adverse Events in Acute Care: Pressure Ulcer Risk &
Occurrence Over 10 Years
Elizabeth VanDenKerkhof, RN, DrPH, Margaret B. Harrison,
RN, Ph.D., Elaine Friedberg, RN, MHA
Presented By: Elizabeth VanDenKerkhof, RN, DrPH, Assistant
Professor, Faculty of Health Science, School of Nursing &
Community Health, Queen's University, 92 Barrie Street,
Kingston, Ontario, K7L 3N6; Email: vandenke@kgh.kari.net
Research Objective: To analyze the risk and occurence of
pressure ulcers in a large acute care facility over 10 years.
Study Design: Yearly risk and point prevalence surveying and
population profiling was conducted between 1994 and 2003.
The data was collected prospectively with a dedicated RN
team in one large teaching hospital. Two phases of analysis
were undertaken: Descriptive, observational Population Risk
and prevalence Trends over 10 years Multivariable modelling
Identification of factors associated with pressure ulcers (stage
II or greater)
Population Studied: From 0600-1800 all patients on service
or admitted during the day to medical, surgical or critical care
units were included in the yearly studies. In total 95% of
eligible patients (8143/8562) particapted in the surveys.
Principal Findings: Of the 8,143 patients surveyed over the
10-year period, 885 were found to have stage II or greater
ulcers. Severity of pressure ulcers was variable over the period
in terms of staging and proportions with multiple ulcers. In
multivariable analysis increasing age and being male sex,
length of time in hospital, service, and Braden activity and
friction/shear deficits were associated with the presence of
pressure ulcer of stage II or greater. After controlling for age,
sex, time in hospital, service, and year, the odds of pressure
ulcer in patients with a friction/shear problem dropped from
14 (95% CI 11, 18) to 4.6 (95% CI 3.1, 6.9), and in bedfast
patients from 12 (95% CI 8.6, 17) to 3.5 (95% CI 2.2, 5.6).
Conclusions: Over a 10 year period pressure ulcer prevalence
estimates in one large hospital remained fairly constant in
spite of the risk profile increasing. Activity and friction/shear
deficits are associated with pressure ulcers of stage II or
greater, and indicate continuing risk with these patients.
However, demographic and clinical characteristics also play an
important role in describing the characteristics of patients
with pressure ulcers.
Implications for Policy, Delivery, or Practice: Pressure ulcer
occurrence continues to be a significant problem in acute
care. Current emphasis on adverse events and patient safety
should bring a renewed focus on the personal, environmental
and system factors related to this longstanding problem.
Primary Funding Source: Canadian Institutes of Health
Research Personnel Award
●Strategies for Assessing Health Plan Performance on
Chronic Diseases: Selecting Performance Indicators and
the Application of Health-Based Risk Adjustment
Ann Volpel, MPA, Jonathan Weiner, DrPH, Steven Johnson,
Ph.D., John O'Brien, MPA, David Idala, MA, Hoon Byun, MA
Presented By: Ann Volpel, MPA, Senior Research Analyst,
Center for Health Program Development and Management,
University of Maryland, Baltimore County, 1000 Hilltop Circle,
Social Sciences 3rd Floor, Baltimore, MD 21250; Tel: (410)4556518; Fax: (410)455-1309; Email: avolpel@chpdm.umbc.edu
Research Objective: As Medicaid managed care programs
mature, states are looking to refine their methods of
measuring and improving the performance of health plans.
Along with other private and public health insurers, states are
developing strategies to pay for performance and implement
value-based purchasing. Many states are also increasing their
scrutiny of the care provided to enrollees with chronic
diseases. This research, funded by the Center for Health Care
Strategies, piloted an approach to measure health plan
performance using administrative data to evaluate the care
provided to enrollees with chronic diseases.
Study Design: Using encounter data from Maryland’s
Medicaid program, a series of performance measures were
developed and calculated for enrollees with four chronic
diseases: asthma, diabetes, HIV/AIDS, and schizophrenia.
The focus on chronic diseases sets the research apart from
many performance measurement approaches that concentrate
on preventive services such as well child visits. A unique
feature of our approach is the application of health-based risk
adjustment to performance measurement, a relatively new
field in Medicaid. Health-based risk adjustment provides the
ability to reduce the confounding effects of case-mix
differences between health plans. The results of our analysis
were used to profile health plans to identify differences in
performance. Selected performance indicators allowed us to
evaluate patterns of service utilization across all four diseases,
as well as to measure disease-specific performance for each
plan. This study also identified performance indicators that are
associated with desired outcomes. Logistic regression models
were used to identify factors associated with a decreased risk
for hospitalization for each disease studied. The regression
model controlled for health status, eligibility category, and
demographic factors including sex, age, race, and geographic
region. Our analysis provides a foundation identifying
performance indicators that a state may consider including in
a performance measurement program.
Population Studied: We analyzed administrative data from
the Maryland Medicaid program in CY2002. We selected
individuals with at least 320 days of enrollment in one of six
health plans who met criteria for one of the following four
conditions: asthma n=16,836, diabetes n=7,121, HIV/AIDS
n=2,162, and schizophrenia n=5,038.
Principal Findings: The research yielded three important
findings: 1-Outcome measures such as inpatient admissions
are highly sensitive to risk and should be risk-adjusted if used
to assess performance; 2- Process measures, such as
preventive services, are relatively insensitive to risk and do not
need to be risk-adjusted to be considered valid performance
indicators; 3-Regular access to ambulatory care services
significantly reduces the likelihood of a hospital admission.
These findings held across all four chronic conditions
examined.
Conclusions: Administrative data can be a rich source of
information for states to assess health plan performance for
chronically ill populations across a number of parameters.
Administrative data, when of high quality, can also support
sophisticated risk adjustment techniques.
Implications for Policy, Delivery, or Practice: State policy
makers may use our techniques to evaluate potential or
existing pay for performance and value based purchasing
strategies. This paper provides a detailed template and
specific examples regarding how to design and execute such
evaluations.
Primary Funding Source: RWJF, Center for Health Care
Strategies
●Impact of State Tort Reforms on Medical Malpractice
Payments
Teresa Waters, Ph.D., Peter P. Budetti, M.D., JD
Presented By: Teresa Waters, Ph.D., Associate Professor,
Preventive Medicine, Center for Health Services Research,
University of Tennessee Health Science Center, 66 North
Pauline Street, Suite 463, Memphis, TN 38163; Tel:
(901)448.5826; Email: twaters@utmem.edu
Research Objective: State tort reform is a high priority for
many states as they struggle to respond to rising medical
malpractice premiums. Very little empirical evidence exists on
the impact of various state-level reforms. The purpose of this
study is to investigate the impact of state-level tort reforms on
the number and size of medical malpractice payments.
Study Design: Cross-sectional, time series data on the
number of physician paid claims and average physician claim
payment were constructed for each state for the years 19912003. A comprehensive review of state laws was conducted to
create Likert scales rating state tort laws in 13 separate areas.
Multivariate fixed effects models were used to explore the
impact of the 13 tort area scales on number and size of claims
payments.
Population Studied: Medical malpractice payments for US
physicians were identified using the National Practioner Data
Bank public use file. Number of practicing physicians were
identified using AMA data.
Principal Findings: Total number of paid claims and number
of paid claims per 1,000 practicing physicians (per state, per
year) were negatively related to strong expert witness
requirements (p<0.01). These measures were also negatively
related to stronger frivolous suit penalties (p<0.05), caps on
damages (p<0.10), and ad damnum clauses (p<0.10).
Average paid amounts for physician claims and physician
medical malpractice payment dollars per 1,000 practicing
physicians (per state, per year) were also negatively related to
expert witness requirements (p<0.01). Dollars per 1,000
physicians were also negatively related to caps on damages
(p<0.10) and periodic payment of damage awards (p<0.10).
Conclusions: Our results suggest that expert witness
requirements and frivilous suit penalties may have a negative
effect on number and size of paid physician medical
malpractice claims. We find weaker evidence that stronger
caps on damages, ad damnum clauses and periodic payment
of damage awards may also have a negative impact. A number
of other tort reforms did not appear to have an effect,
including: caps on attorney's fees, immunity provisions,
pretrial screening, shorter statute of limitations, and joint and
several liability.
Implications for Policy, Delivery, or Practice: Our study
provides some of the first comprehensive evidence on the
relative impact of various state-level reforms using a national
database of physician medical malpractice payments. When
seeking to identify tort reforms that might lower the number
and level of physician paid claims, state and national
lawmakers should consider the evidence on the relative
'efficacy' of the available tort reform measures.
Primary Funding Source: Foundation, Kaiser Family
Foundation
●Practice Pattern Analysis: Improving the Efficiency of
Care
William Westerfield, MA, Ray Phillippi, Ph.D., Ken Patric,
M.D., Judy Slagle, RN, MPA, Steven Coulter, M.D.
Presented By: William Westerfield, MA, Research Scientist,
Health Services Research, BlueCross BlueShield of Tennessee,
801 Pine Street 3E, Chattanooga, TN 37402; Tel: (423) 7556260; Email: William_Westerfield@bcbst.com
Research Objective: The primary objective of this study was
to describe the Practice Pattern Analysis (PPA) methodology
of a single-state health plan in the Southeastern United States.
The secondary objective of this study was to measure the
impact of the PPA on the efficiency of network practitioners.
Study Design: The PPA employed by the health plan was
designed to reveal differences in the efficiency of care of an
individual practitioner compared to his/her specialty. The
main measures reported on the PPA were based on an
episode of care model. An episode of care is a unit of analysis
that includes all the services a patient received to treat a
particular illness. Each episode of care was assigned to one of
nearly 600 Episode Treatment Groups (ETGs). Some of the
measures reported included the case mix index, efficiency
ratio, episode volume, patient satisfaction, and efficiency by
service category (ancillary, facility, management, pharmacy
and surgery). The PPA includes a detailed report of the
practitioners Top 5 ETGs that compares total and service
category costs per episode. A pretest-posttest design was
used to measure the impact of the PPA on the efficiency of
network practitioners. The efficiency of practitioners was
measured using the PPAs produced in the 2nd quarter of 2003
and the 1st quarter of 2004. Efficiency Ratios from 2003 and
2004 were compared for all practitioners and practitioners
identified as outliers. Expected costs for each ETG were
calculated and compared to a practitioner’s actual costs to
calculate the Efficiency Index. The Efficiency Index was
adjusted for specialty differences by dividing the Practitioner
Efficiency Index by the Specialty Efficiency Index to create the
Efficiency Ratio. The 2003 Efficiency Ratios were adjusted for
medical inflation using the Consumer Price Index for the
Southeast.
Population Studied: The study population included 8,902
practitioners who received a PPA in 2003 and 8,857
practitioners who received a PPA in 2004. The study
population also included a subset of 573 outlier practitioners
who received a PPA in 2003. Outliers were practitioners
whose performance was in the lowest 10 percentiles in 2003
for all practitioners.
Principal Findings: The methodology employed by the health
plan significantly impacted the efficiency of the network
practitioners. Preliminary findings indicate the mean
efficiency score of all practitioners improved 8% between 2003
and 2004 (difference of means p<.05). Only 26% of
practitioners who were outliers in 2003 remained outliers in
2004 (117 out of 573). The mean efficiency score of
practitioners who were outliers in 2003, but not in 2004
improved an astounding 80% in single year (difference of
means p<.05).
Conclusions: The PPA methodology employed by the health
plan had a positive effect on the efficiency of the all
practitioners. More importantly, PPA methodology had a
major impact on the efficiency of outlier practitioners helping
these practitioners substantially improve their efficiency.
Implications for Policy, Delivery, or Practice: : Efficiency
based PPAs are an effective method to improve the efficiency
of care delivered by practitioners. The PPA employed by this
health plan is portable and could easily be employed by other
health plans.
Primary Funding Source: BlueCross BlueShield of Tennessee
●Culture Schlock: Is Safety Culture Measurable or
Relevant?
Arthur Williams, Ph.D., Todd Huschka, BS BA, Timothy Beebe,
Ph.D., Rosa Cabanela, Ph.D., Erin McMurtry, BS, Sandra
Bruggeman, BA
Presented By: Arthur Williams, Ph.D., Chair, Division of
Health Care Policy & Research, Health Care Policy & Research,
Mayo Clinic, 200 1st Street SW, Rochester, MN 55905; Tel:
507-284-3356; Email: williams.arthur@mayo.edu
Research Objective: Evaluate the IHI Safety Climate Survey to
determine whether it provides an accurate descriptive
measure of “safety culture.” Use findings to comment upon
the “safety culture” concept in terms of interpretability and
administrative relevance. Comment upon a review of the
“safety culture” literature.
Study Design: Survey questionnaires containing the 19 IHI
Safety Climate Survey questions and a Single Safety Climate
Question (SSCQ) were distributed via intranet to more than
3000 nurses, physicians, and residents. The overall response
rate was 45.4%, resulting in 1,451 useable survey responses.
Exploratory (EFA) and confirmatory factor analyses (CFA) were
done using Stata version 8.2 and AMOS version 5.0. The CFA
were fit with different sets of items and factors to model
underlying relationships. Standard criteria for EFA (loadings
>.40, eigenvalues > 1.00, no cross-loading, etc.) and CFA (CFI
>.900, RMSEA < 0.05, CMIN/DF 1.00 to 3.00, etc) were used.
Item Response Theory (IRT) was applied to assess survey
questions against several constructs. Cronbach’s alphas were
calculated, and correlation analyses were done among latent
constructs and items.
Population Studied: 1,451 clinical survey respondents at a
large mid-western hospital.
Principal Findings: The underlying correlations between the
IHI Safety Climate Survey score and the SSCQ was 0.60.
Several models were generated using EFA and CFA with
anywhere from 4 to 10 items from the IHI survey loading on
factors. In no case was more than 30.7% of the sample
variance accounted for on a factor. CFA indicated that the 19
IHI questions are not consistent with a “safety culture”
construct. Correlations from many combinations of items with
the SSCQ ranged from 0.51 to 0.61. Combinations of items
did not have distinct relationships with the SSCQ; almost any
combination could be selected to obtain a similar correlation.
The simplest model of 4 items (Cronbach’s alpha = 0.70) had
a correlation of 0.51 with SSCQ and 0.91 with the IHI 7
question scores. IRT indicated that 9 of the 19 IHI questions
had an alpha <0.90 implying low discrimination. Deleting
these showed minimal information loss. No total information
was provided at the upper ends of trait distribution (>1std).
Conclusions: There are a multitude of seemingly random
collections of items that could be used to model factors and
correlations with the IHI scores. Many of the questions
included in the various models did not include the word
safety, nor did they seem to be related to safety. In general,
one could randomly select between 4 and 10 questions and
create his or her own purported measure of “safety culture.”
Implications for Policy, Delivery, or Practice: Increasingly
healthcare organizations are becoming aware of the
importance of transforming organizational culture in order to
improve patient safety. This growing awareness has created
an interest in developing assessment tools focused on the
culture of safety. However, in the desire to create bigger or
better tools researchers run the risk of measuring that which
cannot be easily measured, drawing conclusions unrelated to
the question at hand, removing resources from direct patient
care (opportunity costs), and making dysfunctional
administrative prescriptions.
Primary Funding Source: Internal
●Extent of Potentially Inappropriate Antibiotic Prescribing
for Healthy Adults with Acute Bronchitis in Managed Care
Lok Wong, MHS, Jennifer Lis, MBA, MHSA, Barbara Souder,
Ph.D., MPH, RN, Mahil Senathirajah, MBA
Presented By: Lok Wong, MHS, Senior Health Care Analyst,
Quality Measurement, National Committee for Quality
Assurance, 2000 L Street, NW, Suite 500, Washington, DC
20036; Tel: (202)955-1784; Fax: (202)955-3599; Email:
wong@ncqa.org
Research Objective: To investigate the extent to which
antibiotics are inappropriately prescribed to healthy adults
with acute bronchitis in managed care.
Study Design: Retrospective claims analysis of data from four
commercial and one Medicaid managed care plan;
percentages of healthy enrollees with acute bronchitis who
received an antibiotic prescription were calculated. A
validation study was also conducted with a sample of 150
patient records per plan to determine accuracy of
administrative data.
Population Studied: 2.6 million managed care enrollees ages
18 to 64, of which 1.6 million were identified with an acute
bronchitis episode in 2003, excluding unhealthy enrollees with
comorbid conditions or acute competing diagnoses, i.e.
bacterial where antibiotics may be appropriate, or prior
treatment with antibiotics.
Principal Findings: Consistently high rates of inappropriate
antibiotic prescribing were found across the five plans,
average 70%; plan range 64% to 74. These rates of prescribing
were found regardless of age cohort - 18-49 and 50-64 years of
age, location of treatment - ER vs. non-ER - and type of
bronchitis diagnosis - acute bronchitis vs. bronchitis not
otherwise specified. Similar but slightly lower rates of
prescribing were found in patients with comorbidities, i.e.
COPD or other competing diagnoses, where antibiotics may
be appropriate.
Conclusions: Overall high rates of antibiotic prescribing in
patients without clinical characteristics that warrant antibiotic
treatment found in the study, i.e. comorbid condition or
competing diagnosis highlights the extent to which physicians
in managed care are inappropriately prescribing antibiotics for
healthy adults with acute bronchitis. Managed care plans
should be encouraged to develop quality improvement
activities to reduce inappropriate antibiotic prescribing in
clinical practice through the introduction of a standardized
clinical performance measure targeting inappropriate
antibiotic prescribing in healthy adults with acute bronchitis.
Restricting a measure to patients identified with acute
bronchitis - ICD-9 diagnosis code 466.0 - will improve the
clinical validity of the measure denominator, even though
results indicate all patients regardless of the type of bronchitis
diagnosis - acute vs. not otherwise specified - are receiving
antibiotics at similar rates. Patients with chronic bronchitis
were considered to have a comorbid condition and excluded
from the study.
Implications for Policy, Delivery, or Practice: A HEDIS
Effectiveness of Care performance measure highlighting the
extent of inappropriate antibiotic prescribing in adults with
acute bronchitis will help to drive quality improvement
activities in managed care organizations and improve overall
awareness of the widespread overuse and misuse of
antibiotics that contributes to antibiotic drug resistance.
Similar measures addressing inappropriate antibiotic
prescribing in pediatric populations have been introduced into
pay for performance schemes to drive quality improvement in
medical groups and among physicians. In addition,
dissemination of these study findings will improve physicians’
awareness of poor compliance with treatment guidelines for
managing otherwise healthy patients with acute respiratory
conditions such as acute bronchitis. Further research is
needed to understand therapeutic decision-making by
physicians for acute bronchitis patients with comorbid
conditions and why antibiotic prescribing rates may be lower
than in healthy adults.
Primary Funding Source: CDC, Council for Affordable Quality
Healthcare, National Committee for Quality Assurance
●Presence and Correction of Regression to the Mean
When Providers’ Quality of Care is Evaluated by Changes
in Beneficiaries’ Health Status Over Time
Ning Wu, Ph.D., M.D., Jason Roy, Ph.D., Vincent Mor, Ph.D.,
Susan C Miller, Ph.D., Kate Lapane, Ph.D.
Presented By: Ning Wu, Ph.D., M.D., Abt Associates, Inc, 55
Wheeler Street, Cambridge, MA 02138; Tel: (617)520-2588; Fax:
(617)349-2675; Email: Ning_Wu@abtassoc.com
Research Objective: To evaluate the probable impact of
regression to the mean on the evaluation of quality of care
provided by Medicare + Choice Organizations (M+CO's).
Study Design: Simulation
Population Studied: Medicare beneficiaries who enrolled in
Medicare + Choice Organizations (M+CO's).
Principal Findings: We used the data from the first cohort of
Health Outcome Survey (HOS), a study evaluating the health
care quality of Medicare + Choice Organizations (M+COs). In
the HOS, Medicare beneficiaries’ health status is measured by
SF-36 at baseline and two years later. Beneficiaries’ health
transition status, e.g., mental health “being the same or
better” or otherwise, is evaluated by comparing the changes in
the SF-36 scores over a two-year period to a pre-defined cutoff. The health transition status is then aggregated at the
provider level to evaluate M+COs’ performance. In simulation,
we demonstrate that due to RTM, beneficiaries with low
baseline scores are more likely to be misclassified as mental
health being the same or better although they experience
deterioration, and vice versa for beneficiaries with high
baseline scores. As a result, health transition rates aggregated
at the plan level may be based either downwards or upwards
conditional on the distribution of baseline scores in the
M+CO. That is M+CO’s with high baseline scores may appear
to perform worse than M+CO’s with low baseline scores
because of RTM. We also propose a method to correct the
bias introduced by RTM and demonstrate via simulation that
estimations of transition rate has been improved by the
correction method, so has the provider profiling.
Implications for Policy, Delivery, or Practice: Because
improving individuals’ health outcomes is the ultimate goal of
health care, we expect more and more health services
researchers evaluate provider performance by measuring the
changes in patients’ health status. The problem of RTM has
not been discussed or examined in such longitudinal studies.
Using simulation, we demonstrated that RTM may bias the
comparison of provider performance and lead to unwarranted
reward or sanction of providers. Further research is needed to
examine RTM using real data. The knowledge gained from
further research on RTM will improve the validity of HOS
study results, and provide insight or even solutions to other
health services researchers facing similar problems.
Primary Funding Source: No Funding Source
●How Much Does Medicare Pay Hospitals for Medical
Injuries?
Chunliu Zhan, M.D., Ph.D., Bernard Friedman, Ph.D., Andrew
Mosso, MS, Peter Pronovost, M.D.
Presented By: Chunliu Zhan, M.D., Ph.D., Service Fellow,
AHRQ, 540 Gaither Road, Rockville, MD 20850; Tel: (301)4271225; Fax: (301)427-1341; Email: czhan@ahrq.gov
Research Objective: Previous studies indicate that medical
injuries during hospitalization significantly extend length of
stay and incur significant cost, in addition to increasing a
patient’s risk of dying in the hospital. But the extent to which
hospitals are reimbursed by payers for treating medical
injuries is not clear. This study assesses how much Medicare,
the biggest payer in the US, is paying hospitals for medical
injuries and explores the implications.
Study Design: The Agency for Healthcare Research and
Quality Patient Safety Indicators (PSIs) were used to screen
selected types of medical injuries, based on ICD-9-CM codes
in discharge summaries for Medicare beneficiaries in the 2002
Health Care Utilization Project’s Nationwide Inpatient
Sample. Diagnosis-Related Groups (DRGs) were assigned to
the claims with medical injuries based on the ICD-9-CM codes
using 3M’s DRG Grouper Software, and payments were
calculated based on published Medicare payment formulae
and payment determination data such as DRG relative weights
and other adjustment factors. Second, the ICD-9-CM codes
indicating the medical injuries were removed from the claims,
and DRGs were re-assigned and payments recalculated. The
changes in DRGs and payments were examined.
Population Studied: Medicare beneficiaries aged 65 or over.
Principal Findings: Of the Medicare claims that had
iatrogenic pneumothorax (IP, n=2,466), postoperative sepsis
(PS, n=1,399) and decubitus ulcer (DU, n=34,115), the
percentages of claims changed DRGs after removing PSIindicating ICD-9-CM codes are 2.11%, 0.79% and 2.38%. With
few exceptions, these changes were from DRGs with
complication to DRGs without complication. Medicare extra
payments were at $1369 (Confidence Interval, CI=57) per IP
event, $8807(CI=296) per PS event, and $735(9) per DU
event, which account for 6.75%, 19.62% and 4.60% of the
average Medicare payment. Of the extra payment, 92%, 99%
and 91% respectively were due to changes in the outlier
payment for cases with unusually high costs. National
Medicare payment for IP, PS and DU events were estimated at
$16.4 million, $59.2 million and $155.0 million respectively.
Conclusions: The Medicare DRG-based inpatient payment
system pays a substantial amount to hospitals for treating
medical injuries, but the amount appears to be a small
fraction to the cost incurred to hospitals.
Implications for Policy, Delivery, or Practice: Medicare
could achieve savings by investing in patient safety
improvement. These improvements could also yield savings
for hospitals that serve Medicare patients and for private
payers by reducing payment for treating medical injuries and
avoiding cost-shifting from Medicare.
Primary Funding Source: No Funding Source