Quality & Patient Safety
Call for Papers
Patient Safety 2004: Connecting the Dots to Reduce Harm
Chair: Daniel Stryer, M.D.
Sunday, June 6 • 11:30 a.m.-1:00 p.m.
• Adverse Patient Safety Events: Costs of Readmissions
and Patient Outcomes Following Discharge
Didem Bernard, Ph.D., William Encinosa, Ph.D.
Presented by: Didem Bernard, Ph.D., Economist, CFACT,
AHRQ, 540 Gaither Road, Rockville, MD 20850; Tel:
301.427.1682; Fax: 301.427.1276; E-mail: dbernard@ahrq.gov
Research Objective: Most people think that patient safety
events are isolated. While it is known that adverse patient
safety events result in longer lengths of stay, it is not known
whether they lead to excess risk of readmissions and whether
they have an effect on patient outcomes following discharge.
We examine long-term costs and patient outcomes associated
with potentially preventable adverse medical events: costs of
subsequent hospitalizations within 30 days after the initial
index admission, probability of readmissions, probability of inhospital death within 30 days after discharge, and probability
of discharge to long-term care facility for major surgery
patients.
Study Design: The newly-released AHRQ Patient Safety
Indicators (2003) were used to identify 14 types of potentially
preventable adverse events among major surgery patients.
The data source is the Healthcare Cost and Utilization Project,
State Inpatient Database for Florida. Audited hospital cost
reports from the Florida Agency for Health Care
Administration were used to compute hospital costs. For each
major surgey index admission, we constructed hospital cost
measures based on the index admission as well as hospital
readmissions within 30 days of discharge following major
surgery. We then conducted multivariate regression analyses,
controlling for market characteristics, hospital characteristics
and the patient’s risk of adverse outcomes, to predict the
hospital costs attributable to the potentially preventable
adverse medical event, as well as to predict the excess risk of
readmission, excess risk of discharge to a long-term care
facility, and excess risk of in-hospital death within 30 days after
discharge.
Population Studied: The population is all elderly Medicare
major surgery patients discharged from general acute-care
hospitals in Florida in 1995 and 1996. Unique patient
identifiers in this data enable us to examine readmissions.
Principal Findings: Out of 195,049 adult major surgery
discharges, 2.8 percent experienced at least one of the 14
potentially preventable adverse medical events. Of those
patients with such patient safety events, 14.3 percent died in
the hospital, 17.8 percent were readmitted within 30 days after
discharge, and 22.5 percent were discharged to a long-term
care facility. Mean hospital costs for index hospitalizations for
patients with adverse medical events were $19,391 versus
$9,253 for patients who did not have adverse medical events.
Total hospital costs for the index hospitalization and 30 day
readmissions for patients who had a potentially preventable
adverse medical event were $21,176 versus $10,359 for
patients who did not have adverse medical events. Controlling
for covariates, we estimate that 65 percent of this difference
($7,010) was attributable to the adverse patient safety event.
Among patients who did not die during the index admission,
those who had potentially preventable adverse medical events
were 34 percent more likely to be readmitted within 30 days
following discharge. Patients who had potentially preventable
adverse medical events were twice as likely to die during a
readmission within 30 days following discharge. Patients who
had potentially preventable adverse medical events were 32
percent more likely to be discharged to a long-term care facility
following the index admission.
Conclusions: The excess costs and adverse outcomes of
potentially preventable adverse patient safety events continue
to occur even after a patient leaves the hospital. Controlling
for covariates, we estimate that total hospital costs within 30
days of index admission are 67 percent higher for patients
who experience potentially preventable adverse medical events
($17,460) compared to patients who do not experience such
events ($10,450).
Implications for Policy, Delivery or Practice: A reduction in
adverse patient safety events would not only improve quality
of care and reduce Medicare’s cost for that hospitalization but
also reduce costs of readmissions and long-term care.
Primary Funding Source: AHRQ
• Medication Safety in the Primary Care Physician’s Office
Kimberly Galt, Pharm.D., Bart Clark, Ph.D., Ann Rule,
Pharm.D., James Bramble, Ph.D., Kevin Moores, Pharm.D.
Presented by: Kimberly Galt, Pharm.D., Associate Professor
of Pharmacy Practice, Center for Practice Improvement and
Outcomes Research, Creighton University, 2500 California
Plaza, Boyne Room 143, Omaha, NE 68178; Tel: 402.280.4259;
Fax: 402.280.4809; E-mail: kgalt@creighton.edu
Research Objective: Medication safety is a primary concern
to society. However, we know little about the magnitude of
medication related safety issues in the ambulatory care
environment. The Institute of Medicine suggests that hospital
patients represent a fraction of the total population at risk of
experiencing a medication-related error; the majority of
medication prescribing and use occurs in the ambulatory
environment. Knowing how physician offices concern
themselves with safe medication practices in the environment
may be useful to future decisions about office environment
and the processes that take place related to patients and
medication management. This study examines the safety
aspects of the medication use process in ambulatory primary
care offices.
Study Design: We developed a 154-item written survey to
assess medication safety practices governing the medication
use process in primary care offices. Identified safety domains
provided the framework around which survey items were
developed. Two physician clinic offices piloted the survey to
assure content and face validity, and reduce item ambiguity.
We administered the survey using the interviewer-assisted
technique to 31 primary care office managers in the Nebraska
and Iowa region. Using direct observation and on-site
interviews we assessed the environment, facilities,
technologies, and office behaviors related to the medication
use process.
Population Studied: We surveyed thirty-one primary care
offices/clinics.
Principal Findings: Almost half of the practices (44%) report
no specific procedure to respond to a serious medication
error, 56% report no established procedure for providing
prescription drug samples to patients, 36% of practices report
that pharmacists repeat back the prescription when they
telephone prescriptions in to minimize errors associated with
verbal transmission, only 33% of the practices report updating
the patients chart when they renew medications by phone, and
24% of the practices report dismissing individuals from
employment because of errors. We examined the prescribing
process itself to determine areas for practice improvement
that may optimize the opportunity for safer prescribing.
Additionally, the study reports on office practices related to
maintaining current patient charts, collecting timely
medication histories from patients, generating new
prescriptions, prescription renewal processes, methods of
prescription transmission, and prescription clarification once
received by the pharmacists.
Conclusions: This data provides evidence of suboptimal and
sometimes unacceptable practices related to medication
safety. Improving the medication use process in primary care
offices is a critical step to improving medication safety for the
public.
Implications for Policy, Delivery or Practice: The findings of
this study support evidence-based decisions about
improvement practices, and helps define where policy-makers,
practitioners, and inter-professional efforts between primary
care physicians and pharmacists are necessary. The study also
illuminates where these efforts add the greatest value and
have the greatest impact on improving patient medication
safety in the local community.
Primary Funding Source: AHRQ
• A Comparison of Medical Error Reports Submitted to a
Voluntary Patient Safety Reporting System by Different
Classes of Reporters: A Report from the ASIPS
Collaborative
Daniel Harris, Ph.D., Wilson Pace, M.D., Douglas Fernald,
M.A.
Presented by: Daniel Harris, Ph.D., Senior Project Director,
Healthcare Operations and Policy Research Center, The CNA
Corporation, 4825 Mark Center Drive, Alexandria, VA 22311;
Tel: 703.824.2283; Fax: 703.824.2264; E-mail: harrisd@cna.org
Research Objective: Medical error reporting systems are
advocated as strategies for improving patient safety; however,
limited evidence exists regarding the nature of reports
submitted to such systems by various classes of reporters,
especially in ambulatory primary care settings. We analyzed
and compared characteristics of reports to one such system to
ascertain “who reports what” and to determine how that may
impact the effectiveness of reporting systems to improve
patient safety.
Study Design: The Applied Strategies for Improving Patient
Safety (ASIPS) demonstration project developed a patient
safety reporting system (PSRS) to collect narrative medical
error reports voluntarily submitted by clinicians and staff in
two Practice-Based Research Networks (PBRNs) in Colorado.
Of the 608 reports submitted during the first two years of the
project, 522 (85.8%) contained sufficient information to allow
us to classify the reporter as a provider (physician, physician
assistant, or nurse practitioner), other clinical staff, or nonclinical staff. After coding the narrative reports into a multiaxial taxonomy, we compared reports by reporter type using
cross tabulations, analysis of variance, and discriminant
analysis.
Population Studied: Over 500 error reports submitted by
primary care providers, other clinical staff, and non-clinical
staff from 33 primary care practices affiliated with two
Colorado PBRNs.
Principal Findings: Although we promoted the reporting
system among all members of participating practices, we
received fewer reports from non-clinical staff (6.9%) than
from providers (68.6%) or other clinical staff (24.5%); this
differential increased between year one and year two of the
project despite efforts to decrease it. Non-clinical staff
reported events in less detail than did other classes of
reporters. Although all types of reporters reported events that
involved patient harm, other clinical staff were less likely to
report harm overall, providers were more likely to report
clinical harm, and non-clinical staff were more likely to report
non-clinical harm. Similarly, although all reporter types
reported events that involved each other, each type was more
likely to report events involving participants in their own class.
There were no differences in likelihood of reporting events
involving patients or third parties from outside the practice.
Compared to other reporters, providers reported more errors
involving disclosure to patients and delays in diagnosis, and
fewer involving system use or malfunction; other clinical staff
reported more errors involving communication within the
practice and documentation in the patient record; and nonclinical staff reported more errors involving patient
management and distractions and fewer involving diagnostic
testing and delays in using information. Discriminant analysis
yielded functions that differentiated between reporter types
and correctly classified the majority of reporters.
Conclusions: Different classes of reporters submitted
different kinds of reports in differing levels of detail to a
voluntary PSRS. Members of each reporter class tended to
report events involving the types of activities and participants
they could be expected to observe.
Implications for Policy, Delivery or Practice: Medical error
reporting systems need to assure that all classes of reporters
participate in order to be certain that a full range of error types
are reported.
Primary Funding Source: AHRQ
• The Impact of the Leapfrog Group on Hospital Patient
Safety Practices
Dennis Scanlon, B.A., M.A., Ph.D., Jon Christianson, Ph.D.,
Eric Ford, Ph.D.
Presented by: Dennis Scanlon, B.A., M.A., Ph.D., Assistant
Professor of Health Policy & Administration, The Pennsylvania
State University, 119B Henderson Building, University Park, PA
16802-6500; Tel: 814.865.1925; Fax: 814.863.2905; E-mail:
dpscanlon@psu.edu
Research Objective: To assess the impact that the Leapfrog
Group is having on hospital patient safety practices in its
Regional Rollout (RRO) Communities. To assess the views of
Leapfrog members regarding the use of incentives for
encouraging rapid adoption of recommended hospital safety
practices.
Study Design: Comparative case studies of seven of the
twenty-two Leapfrog Regional Rollouts. As part of the case
study design, the authors conducted in depth and in person
interviews of multiple stakeholders in each of the seven RROs,
including hospital administrators, employers and public
purchasers, health insurers and health plans, benefits
consultants and brokers, and leaders of physician and hospital
trade organizations. Over 100 interviews of key stakeholders
have been conducted across the seven markets, with 12-18
interviews completed in each of the seven markets. In
addition, we conducted a web based survey of Leapfrog's 140
members regarding opinions on and plans for using
incentives to encourage hospitals to adopt patient safety
practices.
Population Studied: The Leapfrog membership and
stakeholders in its regional rollout markets.
Principal Findings: The Leapfrog Group has clearly had an
impact on hospitals' awareness of patient safety generally and
the recommended Leapfrog patient safety practices
specifically. The principal metric used by Leapfrog to measure
hospital performance has been willingness to report results to
the Leapfrog online survey. While many of the RROs have
been successful in acheiving hospital reporting, hospital
adoption of Leapfrog standards, or plans to adopt these
standards, varies widely. Hospitals cite a variety of positive
and negative affects of the Leapfrog initiatives, and the degree
to which Leapfrog's efforts are received in a community varies
significantly depending on local Leapfrog leadership and
purchasing activity. Many hospitals believe that Leapfrog's
future success hinges on the ability to align payment and nonfinancial incentives to performance to valid measures of
quality and safety performance. However, while many
purchasers acknowledge the importance of aligning incentives
to performance expectations, most agree that the Leapfrog
standards are not currently amenable to incentives programs
such as tiered benefit designs based on Leapfrog standard
compliance. Hence, the ability to acheive desired outcomes in
the near future is uncertain in the current health care
environment
Conclusions: While Leapfrog has generated quite a bit of
interest and attention nationally, its success seems to vary by
community based on market characteristics and purchaer
activity. Our analysis identifies examples of more and less
successful communities and factors related to Leapfrog's
success in certain RRO communities. We also identify
practical barriers in health markets preventing the success of
quality and patient safety efforts.
Implications for Policy, Delivery or Practice: While employer
and purchaser market based approaches to addressing
hospital quality and safety deficiences sounds appealing, there
can be significant barriers to implementing such initiatives
and to acheiving rapid adoption of recommended hospital
patient safety practices. Policymakers should be aware of
these limitations as they seek to address safety and quality
issues. Our findings help policymakers, purchasers and
providers understand when market based or regulatory based
approaches are appropriate.
Primary Funding Source: RWJF
• Patient Safety Problems in Adolescent Medical Care
Donna Woods, Ed.M., M.A., Ph.D., Eric Thomas, M.D.,
M.P.H., Edward Ogata, M.D., MM, Jane Holl, M.D., M.P.H.
Presented by: Donna Woods, Ed.M., M.A., Ph.D., Fellow,
Institute for Health Service Research and Policy Studies,
Northwestern University, Feinberg Medical School, 339 E.
Chicago Avenue, Chicago, IL 60611; Tel: 847.832.9650; Fax:
312.503.2936; E-mail: woods@northwestern.edu
Research Objective: To better understand and describe the
types of patient safety problems identified in a hospital-based
(inpatient and outpatient) medical care of adolescents.
Study Design: Mixed method, two part study:
(A) quantitative data from the Colorado and Utah Medical
Practice Study population-based study of adverse events
(AE’s) and preventable adverse events (PAE’s) used to
estimate rates of AE’s and PAE’s in hospitalized adolescents
and
(B) This part of the study uses the critical incident analysis
methodology that is widely applied to the study of human
factors and system design to determine factors associated
with system failure risks. Pediatric clinicians (attending
physicians, residents, nurses, and pharmacists) were asked to
participate in an in-person, audio-taped interview to describe
medical cases “where something did not go quite right or did
not go as planned in the medical care of a child.” A
standardized interview protocol was used. A classification of
problem types and child-specific-factors related to adolescence
was developed and relative frequencies of these factors were
assessed.
Population Studied: Study (A): Adolescent hospital records
of a representative sample of hospitalized children (3,719) in
Colorado and Utah in 1992; Study (B) In-patient and
outpatient pediatric patients at an urban tertiary care teaching
hospital.
Principal Findings: (A) Adolescents (12-20 years) AE and PAE
rates were 3.41% and 0.95% respectively, compared with
0.63% and 0.53% respectively for infants (0-1 yr.), and 0.92%
and 0.22% respectively for children (1-11 years old). Thus,
adolescents had a PAE rate nearly 2 times the rate for infants
and nearly 3 times the rate for toddlers and school age
children.
(B) Of 157 identified hospital-based critical incidents, 21%
described patient safety problems in the medical care of an
adolescent – the resulting injuries ranged from no harm to
death. Problems in the context of surgical and non-surgical
related procedures (25%) and medication processes (25%)
were most common. Problems were also described in the
context of diagnostics (16%) and clinical communication with
the patient or family (13%). Problem types included problems
of execution (36%), problematic communication (26%) and
problematic decisions (26%).
In 72% of the incidents, an adolescent oriented child-specificfactor contributed directly to the problem occurrence.
Developmental factors contributed most often (47%) followed
by factors related to physical characteristics (35%) and factors
associated with minor status (17%). Evident latent conditions
that contributed included, problems in the “systems”(49%) of
medical care provision and the “culture” (35%) of medical
care.
Conclusions: Adolescents experience the highest rate of AE’s
and PAE’s among children. Problems were shown to occur in
all aspects of medical care. The high rate of AE’s and PAE’s
among adolescents and the high level of adolescent oriented
child specific contributing factors described in the critical
incidents suggest that adolescents are at particular risk for
patient safety problems.
Implications for Policy, Delivery or Practice: To date there
have been no other studies focused on adolescent patient
safety. Further study is needed to improve the understanding
of the patient safety risks in adolescent medical care and
develop focused interventions to address the specific risks
adolescents experience to improve adolescent patient safety.
Primary Funding Source: Children’s Memorial Institute for
Evaluation and Research, Children’s Memorial Hospital,
Chicago, IL
Call for Papers
Measurement & Improvement: Where Are We Now
& Where Are We Headed?
Chair: Jeroan Allison, M.D., M.S.
Sunday, June 6 • 5:00 p.m.-6:30 p.m.
• The Effect of Using Rules Technology with Computerized
Provider Order Entry (CPOE) in Medication Error
Reduction in an Outpatient Setting
Andy Steele, M.D., M.P.H., Sheri Eisert, Ph.D., Patricia Gabow,
M.D., Joel Witter, M.D., Mike Jones, Pharm.D., Eduardo Ortiz,
M.D., M.P.H.
Presented by: Sheri Eisert, Ph.D., Director, Health Services
Research, Denver Health, 777 Bannock Street, Denver, CO
80204; Tel: 303.436.4072; Fax: 303.436.5952; E-mail:
Sheri.Eisert@dhha.org
Research Objective: CPOE systems with clinical decision
support have been shown to prevent medication errors and
decrease adverse drug events. However, the majority of this
research has focused on the acute inpatient setting. We
assessed the impact of rules technology on medication errors
related to drug-laboratory interactions in an outpatient primary
care setting utilizing CPOE.
Study Design: In collaboration with commercial knowledge
content and information technology vendors, rules were
developed to address a set of drug-laboratory interactions.
Using CPOE, providers entered medication orders
electronically, and alerts were generated if a potentially
important drug-laboratory interaction was detected. A nonrandomized 4-month pre- and 5-month post-comparison was
done from August 2002 through April 2003. During the preintervention period, rules were turned on, but messages were
not displayed to the providers. During the post-intervention
period, alerts were displayed to the providers. Provider
ordering behavior was monitored focusing on the number of
medication orders not completed and the number of ruleassociated laboratory test orders initiated after alert display.
Population Studied: This study was conducted at four Denver
Health outpatient primary care clinics that collectively see
about 80,000 visits annually. All provider staff that were
eligible to enter medication orders participated in the study.
All registered patients were eligible for the intervention.
Principal Findings: The rule processed 16,291 times during
the study period; 7,017 during the pre-intervention period and
9,274 during the post-intervention period. During the postintervention period an alert was displayed 12.2 percent (1,093
out of 9,274) of the times the rule processed - 48 percent were
for “missing laboratory values,” and 52 percent were for
“abnormal laboratory values” representing potential druglaboratory interactions. There was a significant increase in the
percentage of time the provider followed the recommendation
and did not complete the medication order when an alert for a
drug associated abnormal laboratory result was displayed, 5.6
percent during pre-intervention versus 10.9 percent during
post intervention (p-value = 0.03). The number of times a
provider ordered an alert recommended laboratory test
increased from 39 percent during the pre-intervention period
to 51 percent during the post-intervention period (p
value<.001).
Conclusions: In our study, providers responded positively to
automated alerts and significantly increased their adherence
to institutional, consensus-based guidelines for improved
drug-laboratory monitoring. This resulted in a 95% increase in
cessation of the medication ordering process for a potentially
harmful drug-laboratory interaction, and a 31% increase in the
ordering of appropriate laboratory tests, which may result in
fewer medication errors and improved quality of care.
Implications for Policy, Delivery or Practice: The use of
automated alerts embedded in a computerized order entry
system can change provider ordering behavior, resulting in
decreased medication errors and possibly fewer adverse drug
events. Although our study only looked at drug-laboratory
interaction alerts, decision support systems can be configured
to provide other types of automated alerts, which have broad
implications for improving many aspects of patient safety and
quality of care.
Primary Funding Source: AHRQ
• Measure, Learn, and Improve: Have Physicians Begun to
Engage in the Quality Improvement Cycle?
Anne-Marie Audet, M.D., M.Sc., SM, Stephen Schoenbaum,
M.D., M.P.H., Michelle Doty, Ph.D., M.P.H., Jamil Shamasdin
Presented by: Anne-Marie Audet, M.D., M.Sc., SM, Assistant
Vice President, Quality of Care, The Commonwealth Fund,
One East 75th Street, New York, NY 10021; Tel: 212.606.3856;
E-mail: ama@cmwf.org
Research Objective: Payers and regulatory and oversight
organizations have shown interest in using quality
improvement principles and physician performance measures
to improve health care. But little is known about how
physicians themselves use data to monitor and improve the
care they deliver.
We conducted a national survey of physicians with the
following objectives: 1) to explore whether physicians have
adopted basic quality improvement principles (i.e., measure,
learn, improve); 2) to identify the quality-of-care data to which
physicians say they have access; 3) to describe physicians’
willingness to share these data; and 4) to determine whether
physicians engage in quality improvement activities.
Study Design: Mail survey completed by a national random
sample of 3,598 physicians caring for adult patients. The
survey was conducted between March and May 2003. The
survey sample was randomly selected from a national list the
American Medical Association (AMA) that includes both AMA
members and non-members. All physicians in the sample
were involved in the direct care of adult patients and had been
in practice at least three years post-residency.
Population Studied: National random sample of US
physicians caring for adult patients.
Principal Findings: The response rate was 52.8% (1,837
physicians). Forty-three percent of respondents said they have
easy access to data about their patients’ clinical profile. Seven
of eight find it difficult or impossible to identify patients who
have abnormal laboratory results (84%) or take specific
medications (85%). One-half of physicians do not have
access to any data about the quality of the care they deliver.
Physicians in group practices with more than 50 members are
much more likely than solo physicians to have access to these
data (adjusted OR=2.14, p<0.001). Specialists are less likely to
have data on their quality compared with primary care
physicians (OR = 0.38, p <0.05). Health plans are the most
common source of quality-of-care data (25%). Only 14% of
physicians generate their own data. One-third of physicians
participate in quality improvement efforts. Physicians in
groups larger than 50 are more likely to participate than solo
physicians (OR = 2.38, p <0.05). Thirteen percent “definitely
agree” that performance data should be shared with their own
patients, while 45% disagree. Seventy percent said these data
should “probably not” or “definitely not” be shared with the
public.
Conclusions: The results of the survey suggest that
physicians are not making full clinical use of available data
about their own practice to guide their care. Although they no
doubt strive to provide care of the highest quality, many
physicians do not have adequate systems in place to ensure
that they consistently do so. Clinical practice in the United
States, for the most part, is not data-driven, nor is it
transparent even within the context of the physician-patient
compact.
Implications for Policy, Delivery or Practice: The
implications of these findings are important and deserve the
attention of physicians, professional organizations, and
policymakers alike. It is hard to imagine how, in the absence
of quality-of-care information, effective solutions to some of
the pervasive problems with health care quality can be
generated. Although quality improvement is an essential
component of professionalism, most practicing physicians do
not appear to be participating in it. Physicians should be
taught the knowledge and skills to participate in quality
improvement activities, and the acquisition and application of
these skills should be rewarded.
Primary Funding Source: CWF
• Gaps in Quality of Care for HIV-Infected Veterans
Todd Korthuis, M.D., M.P.H., Steven Asch, M.D., M.P.H.,
Henry Anaya, Ph.D., Samuel Bozzette, M.D., Ph.D.
Presented by: Todd Korthuis, M.D., M.P.H., Assistant
Professor of Medicine, General Internal Medicine and
Geriatrics, OHSU, 3181 S.W. Sam Jackson Park Road, Portland,
OR 97239-3098; Tel: 503.494.8044; Fax: 503.494.0979; E-mail:
korthuis@ohsu.edu
Research Objective: While the veterans administration (VA)
has improved quality of care for many chronic illnesses, the
quality of HIV care has not been assessed. We sought to
assess the overall quality of care among HIV-infected veterans
and identify gaps in quality of care.
Study Design: Cross-sectional abstraction of medical records.
Quality indicators were measured as the percent eligible
persons receiving highly active antiretroviral therapy (HAART),
prophylaxis for Pneumocystis carinii pneumonia and
mycobacterium avium complex, and screening for syphilis,
toxoplasmosis, hepatitis A, hepatitis B, hepatitis C, and
dyslipidemia.
Population Studied: 5071 HIV-infected veterans receiving
medical care in 2001 at 16 Veterans Administration (VA)
facilities
Principal Findings: The proportion of eligible persons
receiving indicators of quality of care ranged from 49 to 70%.
69% of eligible veterans received HAART. In adjusted logistic
regression models, African Americans were less likely to
receive HAART (OR=.78, 95% CL .62-.99) and more likely to
receive screening for syphilis (OR=1.58, 95% CL 1.32-1.89),
hepatitis A (OR=1.35, 95% CL 1.15-1.59), hepatitis B (OR=1.45,
95% CL 1.22-1.72), and hepatitis C (OR=1.36, 95% CL 1.131.63). Persons with intravenous drug use (IVD) as their mode
of HIV exposure were less likely to receive HAART (OR = 0.61,
95% CL .47-.79) and lipid screening (OR=.56, 95% CL .39-.80),
but more likely to receive screening for hepatitis B (OR=1.23,
95% CL 1.01-1.51) and syphilis (OR=1.40, 95% CL 1.12-1.74).
Persons with 10 or fewer visits were less likely to receive
HAART (OR=.17, 95% CL .07-.26), PCP prophylaxis (OR=2.5,
95% CL .09-.67), or screening for hepatitis A (OR=.47, 95% CL
.29-.77), B (OR=.43, 95% CL .25-.76), C (OR=.50, 95% CL .27.92), or dyslipidemia (OR=.25, 95% CL .09-.67).
Conclusions: Though overall quality of care was fair for most
quality indicators, disparities in the quality of HIV care persist
for African Americans, women, and persons with IVD or
unknown mode of HIV transmission.
Implications for Policy, Delivery or Practice: Interventions
to close gaps in quality of care for African Americans, women,
and persons with IVD and unknown HIV exposure mode are
indicated.
Primary Funding Source: VA
• New Directions in Quality Measurement: Moving
Towards Standardized Performance Measurement for
Physician Offices
Joachim Roski, Ph.D. M.P.H., Donna Pilliterre, M.S., Sarah
Scholle, Dr.PH
Presented by: Joachim Roski, Ph.D. M.P.H., Vice President,
Performance Measurement, NCQA, 2000 L Street, N.W.,
Suite 500, Washington, DC 20036; Tel: 202.955.5139; Fax:
202.955.3599; E-mail: roski@ncqa.org
Research Objective: Recent evidence has demonstrated that
patients may on average only receive about 50% of
recommended ambulatory care services. Partially in response
to these findings efforts are underway to measure and report
on performance of physician offices around the US. In the
absence of national standards several larger regional
coalitions or single entities are implementing differing
measurement and reporting efforts. This presentation will
describe, contrast and compare ten of the most expansive of
these physician measurement projects.
Study Design: A survey was completed in
November/December 2003 by representatives of large
regional/local implementation coalitions for measurement at
the physician-level. Additional face-to-face interviews were
conducted to clarify survey responses and gather additional
information. Survey content focused on projects’ drivers and
specific objectives (including pay-for-performance efforts),
details on selected performance measures, data collection
methods and measurement approaches, data sources,
projects’ initial experiences, specific challenges encountered,
and opportunities for standardization.
Population Studied: Survey representatives were comprised
of lead representatives from ten large coalitions in California,
Washington, Nevada, Minnesota, Massachusetts, Missouri,
and New York. Coalitions’ membership ranged from multistakeholder groups (health plans, medical groups/physicians,
employers) to those that are dominated by a single
stakeholder group. In total these measurement efforts address
ambulatory care efforts received by more than 40 million
residents in those markets.
Principal Findings: Common drivers for implementing
physician level measurement efforts included lacking
transparency of performance at the physician level, the need
to identify “high value” providers, introduction of pay-forperformance programs, as well as institutionalisation of
community-wide (all payers) health care quality measurement
efforts. Few programs had been in operation for more than 1-2
years. Units of analysis range from individual physicians, to
practice sites (clinics), to medical groups. HEDIS®
effectiveness of care measures build the core of most
measurement activities augmented by select service utilization
measures, experience-of-care ratings, structural quality
elements (e.g., IT functionality), and others. Due to lack of
access to other economically viable data sources most efforts
rely exclusively on administrative electronic information that is
already available in calculating performance rates. The
majority of respondents are looking to identify nationally
available standards for performance measurement and
benchmarking.
Conclusions: There is significant overlap in purpose,
effectiveness-of-care measure selection and data sources
accessed for performance measurement between the
described efforts. Significant differences exist in the chosen
unit of analysis (individual physicians vs. groups),
methodological detail (attribution of patients) and deployment
of measures outside of HEDIS®.
Implications for Policy, Delivery or Practice: Implementing
reliable and standardized performance measurement systems
at the physician level remains a major challenge. Building on
budding current measurement implementation efforts offers
an opportunity to define consensus around measurement
standards and move towards implementation of an eventually
nation-wide physician-level performance measurement effort.
A similar implementation course allowed HEDIS® to become
the most prominent and widely implemented performance
measurement effort for managed care organizations in the US
Primary Funding Source: CWF
• Evaluation of the CDC Guideline for the Prevention of
Surgical Site Infection
Sujha Subramanian, Ph.D., Lucy Savitz, Ph.D., Linda Pucci,
M.S., Shula Bernard, Ph.D., Michele Pearson, M.D.
Presented by: Sujha Subramanian, Ph.D., Research Health
Economist, Division of Health Economics Research, RTI
International, 411 Waverley Oaks Road, Suite 330, Waltham,
MA 02452; Tel: 781.788.8100; Fax: 781.788.8101; E-mail:
ssubramanian@rti.org
Research Objective: Surgical Site Infections (SSIs) remain a
significant cause of health-care-associated morbidity and
mortality despite the availability of evidence-based prevention
guidelines. The objectives of this study were to measure
adherence to the CDC’s SSI Guideline and identify barriers
and enablers to adherence.
Study Design: We assessed adherence to recommended
practices for antimicrobial prophylaxis (AP) and perioperative
glucose (POG) management among four procedure
categories: coronary artery bypass grafting, prosthetic joints,
vascular surgery, and general surgery. For AP, we assessed
the antimicrobial used, and the timing and duration of
administration. For POG management, we assessed whether
serum glucose levels were obtained prior to surgery and
whether levels were maintained at or below 200mg/dl. Data
was collected via chart abstraction at four hospitals
representing a variety of settings. Key informant interviews
were also performed with infection control and perioperative
personnel to obtain qualitative information on policies and
practices.
Population Studied: The study population included patients
who were 18 years or older who underwent procedures in the
four categories. Based on sample size calculations, 800
records, equally allocated at 50 records per procedure per
hospital (200 records total per hospital) were randomly
selected from procedures performed during 2002.
Principal Findings: Preliminary analysis of 400 records
indicates that the proportion of patients who received AP
within 1 hour of surgical incision and whose AP was
discontinued within 24 hours of surgery are generally low with
adherence rates of less than 50% for the majority of the
procedure categories. Conversely, there is high degree of
adherence to guideline recommendation for the type of agent
given to the patients across all procedures (88% - 100%).
Testing for serum glucose level was performed in 18%-30% of
the cases when all patients are considered and in 70%-100%
when only patients with diabetes are included. Hyperglycemia
was detected in 12-20% of patients. Factors identified from
qualitative investigation that can foster improved compliance
with guidelines include implementing orientation and
refresher training for staff, multifaceted dissemination of
recommendations, infection control data feedback
mechanisms, integration of prevention practices into job
functions, nurse/physician champions, surgical team stability,
and outreach to surgeons practicing at multiple locations.
Conclusions: Adherence to recommended practices for SSI
prevention varied by the type of measure; improvements are
needed in the timing and duration of AP, and in the
maintenance of normoglycemia. Modified vehicles for
guideline dissemination and providing timely feedback from
surveillance activities could lead to improved adherence to
evidence-based recommendations.
Implications for Policy, Delivery or Practice: SSI process
measures have been recently included in Center for Medicare
and Medicaid Services’ quality indicators and the Joint
Commission on Accreditation Healthcare Organization’s core
measures. This study will provide practical information for the
implementation and interpretation of these indicators and
assist in identifying strategies for improving the quality of
surgical care.
Primary Funding Source: CDC
Call for Papers
What about Structure? Organizational & Financial Effects
on Quality
Chair: Joachim Roski, Ph.D., M.P.H.
Monday, June 7 • 4:00 p.m.-5:30 p.m.
• Health Care Market Structure and the Quality of Care
Provided by Safety Net Hospitals
Jeannette Rogowski, Ph.D., Jose Escarce, M.D., Ph.D.
Presented by: Jose Escarce, M.D., Ph.D., Professor of
Medicine, Internal Medicine, David Geffen School of Medicine
at UCLA, 911 Broxton Avenue, Los Angeles, CA 90024; Tel:
310.794.3842; Fax: 310.794.0726; E-mail:
Jose_Escarce@rand.org
Research Objective: Safety net providers are providing an
increasingly disproportionate share of uncompensated care as
a result of increased financial pressures in health care
markets. The consequences for the quality of care provided by
these hospitals is unknown. In fact, there is very little
knowledge regarding the effect of health care markets on the
quality of hospital care in general. The goal of this paper is to
study how health care markets influence the quality of hospital
care, with a focus on quality provided by hospitals in the safety
net.
Study Design: Data for this study consist of linked discharge
abstract and vital statistics data for the state of California for
1994-1999. Six medical conditions are studied: myocardial
infarction, stroke, gastrointestinal hemorrhage, congestive
heart failure, hip fracture and diabetes. These conditions were
chosen because they exhibited low variation in populationbased rates of admission and thus admission rates would not
be affected by health care market structure. Outcomes are
measured by 30-day mortality.
A broad range of safety net definitions are considered,
including dichotomous classifications such as public
ownership, high DSH percentages and a continuous measure
of safety net status. Market structure is defined by managed
care penetration (as determined from Interstudy data) and
hospital competition measured by the Herfindahl index based
on a variable radius definition of the hospital market that
includes 90% of the patients treated (based on data from the
AHA). Risk adjustment is based on condition specific risk
adjusters as derived from the discharge abstract data and
include demographics, comorbidities, and condition-specific
measures of severity.
Population Studied: The sample consists of all adults (aged
25 and older) treated in 340 hospitals between 1994 and 1999.
Principal Findings: For AMI, safety net hospitals have higher
risk adjusted mortality rates. For instance the odds ratio for
mortality among public hospitals compared to non-profit
hospitals is 1.49 (p<0.001). For non-public high DSH hospital
the odds ratio is 1.10 (p<0.05). However, safety net status is
associated with lower mortality for congestive heart failure and
diabetes and is unrelated to mortality for stroke,
gastrointestinal hemorrhage and hip fractures. Hospital
competition is associated with lower mortality for some
conditions (congestive heart failure, stroke, gastrointestinal
hemorrhage and hip fractures) and has no significant effect
for the other conditions. Managed care penetration is
associated with lower mortality for some conditions
(congestive heart failure and gastrointestinal hemorrhage) and
has no significant effect for the other conditions. There is little
evidence of interactions of safety net status and hospital
competition or managed care penetration.
Conclusions: Safety net hospitals provide lower quality of care
for AMI, but not for all medical conditions. The quality of care
provided by safety net hospitals does not appear to be affected
by competition or managed care penetration, despite the
increased price competition associated with higher
competition and managed care presence in the market. For
hospitals in general, there is evidence for some medical
conditions that hospital competition and managed care
penetration are associated with lower risk adjusted mortality
rates.
Implications for Policy, Delivery or Practice: Safety net
hospitals provide lower quality of care for some medical
conditions but not for others. It is important for policymakers
to understand the reasons why safety net hospitals have lower
quality for some conditions.
Primary Funding Source: AHRQ
• Do Increases in Managed Care Market Share Influence
Quality of Cancer Care in the Fee-For-Service Sector?
Nancy Keating, M.D., M.P.H., Mary Beth Landrum, Ph.D.,
Ellen Meara, Ph.D., Patricia Ganz, M.D., Edward Guadagnoli,
Ph.D.
Presented by: Nancy Keating, M.D., M.P.H., Assistant
Professor of Medicine and Health Care Policy, Department of
Health Care Policy, Harvard Medical School, 180 Longwood
Avenue, Boston, MA 02115; Tel: 617.432.3093; Fax:
617.432.0173; E-mail: keating@hcp.med.harvard.edu
Research Objective: Increases in managed care market share
are associated with decreases in expenditures in the fee-forservice sector. Whether these decreases in expenditures are
associated with changes in the quality of care delivered is not
well understood. We assessed whether increases in managed
care market share were associated with stage at diagnosis and
quality of cancer care for elderly patients with cancer in the
fee-for-service sector.
Study Design: Retrospective cohort study using cancer
registry and Medicare claims to assess the proportion of
patients in the fee-for-service sector diagnosed with advanced
stage cancer and the proportion with care in accordance with
accepted quality indicators. Fixed effect models were used to
assess the association between increases in managed care
market share and stage or receipt of quality indicators.
Population Studied: Population-based sample of fee-forservice Medicare beneficiaries aged 66 and older residing in 11
Surveillance, Epidemiology, and End Results (SEER) areas and
diagnosed with breast or colorectal cancer during 1992-1999.
Principal Findings: Stage at diagnosis. As managed care
market share increased, the proportion of women diagnosed
with early-stage breast cancer in the fee-for service sector also
increased (P=0.02), although there was no association
between managed care market share and stage at diagnosis
for patients with colorectal cancer (P=0.22).
Quality indicators. Increases in managed care market share
were not associated with receipt of surveillance
mammography after diagnosis for women with breast cancer
in the fee-for-service sector (P=0.95), nor were they associated
with receipt of radiation after breast-conserving surgery
among women who underwent breast-conserving surgery
(P=0.29). For patients with colorectal cancer, increases in
managed care market share were not associated with receipt
of adjuvant chemotherapy for patients with stage III colorectal
cancer (P=0.89) nor they associated with surveillance
colonoscopy (P=0.47). Increases in managed care penetration
were associated with increased rates of surveillance CEA
testing for patients in areas with the highest rates of managed
care only (P=0.001).
Conclusions: In this population-based sample of elderly
patients diagnosed with breast or colorectal cancer, increases
in managed care market share were associated with earlier
diagnosis of breast cancer but had limited or no effect on
performance on indicators of quality.
Implications for Policy, Delivery or Practice: These results
suggest that the spillover of effect of managed care with
respect to expenditures in the fee-for-service sector may be
due to more efficient utilization of resources rather than to a
reduction of needed services. Fears that increases in managed
care would have large negative spillover effects on the quality
of cancer care appear to be unfounded.
Primary Funding Source: NCI, Doris Duke Charitable
Foundation
• The Relationship between Nosocomial Bacteraemia and
Organizational and Structural Factors in a Random
Sample of 54 UK Neonatal Intensive Care Units
Gareth Parry, Ph.D., Janet Tucker, Ph.D., William TarnowMordi, M.D.
Presented by: Gareth Parry, Ph.D., Harkness Fellow in Health
Care Policy, Pediatric Health Services Research Program,
Children's Hospital Boston, 333 Longwood Avenue, LO-240,
Boston, MA 02115; Tel: 617.355.6646; Fax: 617.730.0174; Email: gareth.parry@childrens.harvard.edu
Research Objective: To assess the relationship between
organizational and structural factors of UK neonatal intensive
care units to risk-adjusted nosocomial bacteraemia.
Study Design: Prospective observational study.
Population Studied: 13,334 infants concurrently admitted to
54 randomly selected UK Neonatal Intensive Care Units
between March 1998 and April 1999.
Principal Findings: There were 402 cases (2.97%) of infants
with nosocomial bacteraemia, defined as bacteraemia or
septicaemia of probable nosocomial origin more than 48
hours after birth (excluding vertical transmission. The median
unit-level percentage of infants with nosocomial bacteraemia
was 2.48% or 8/322 (minimum = 0% in three units or 0/59,
0/100 and 0/120 maximum = 8.65% or 23/266 in one unit).
Using generalized estimating equations, the risk-adjusted
odds of probable nosocomial bacteraemia were: increased
when admitted within 7 days of another infant being admitted
with vertical transmission of bacteraemia by 1.37 (95%
confidence interval 1.09, 1.72) compared to admitted at > 7
days; increased by 1.13 (1.07, 1.20) for each additional level 1
cot per hand washbasin; decreased by 0.53 (0.35, 0.79) in
infants admitted to units with an infection control audit nurse
dedicated to the neonatal intensive care unit, compared to
units without; not related to an increase in the floor space of
the unit per cot, odds ratio 0.99 (0.98, 1.00) per M2; not
related to the quality of hand-washing and infection control
protocols, odds ratio 1.03 (0.96, 1.11) per increase in score.
Conclusions: There is widespread variation in the rates of
nosocomial bacteraemia in UK neonatal intensive care units.
Lower rates of nosocomial bacteraemia are associated with an
infant being admitted in the one week period post the
admission of an infant with vertically transmitted bacteraemia,
the presence of an infection control nurse dedicated to the
neonatal intensive care unit, and with increased availability of
hand washbasins. There was no association with rates of
nosocmial bacteraemia and current UK neonatal intensive
care floor space and the quality of hand-washing and infection
control protocols.
Implications for Policy, Delivery or Practice: In addition to
on-going educational initiatives to reduce rates of nosocomial
bacteramia and the variation in rates between units we
recommend that the Joint Commission on Accreditation of
Healthcare Organizations considers including in their
National Patient Safety Goals that: 1) Health care settings
employ a clinician charged with ensuring current best
practices in the audit of infection control and 2) Hand
washbasins and alcohol rub are available at every cot or
bedside.
To reduce nosocomial bacteraemia rates to zero a complete
systems-based re-think is required so that it is impossible for
clinical staff to move from one patient to another without
washing their hands. This may require greater empowerment
of nursing and junior clinical staff to speak up if they see
another member of the clinical staff fail to clean their hands,
but may also require that patients and their families are
empowered to speak up too.
Primary Funding Source: CWF, NHS Executive
• Mortality Associated with Physician Volume and
Speciality in Myocardial Infarction Patients
Hude Quan, M.D., Ph.D., Andrew Fong, B.Comm., Bibiana
Cujec, M.D., Yan Jin, M.A., David Johnson, M.D.
Presented by: Hude Quan, M.D., Ph.D., Assistant Professor,
Community Health Sciences, University of Calgary, 3330
Hospital Drive, N.W., Calgary, T2N 4N1; Tel: 403.944.8912;
Fax: 403.944.8950; E-mail: hquan@ucalgary.ca
Research Objective: We assessed if mortality after acute
myocardial infarction (AMI) was associated with physician
volume and specialty.
Study Design: This cohort study analyzed the association
between mortality (in-hospital, 30 days and 1 year from
admission) and physician factors in AMI patients discharged
from Alberta hospitals, Canada. The volume of AMI patients
treated for the most responsible hospital physician during
1994/5-1999/2000 was categorized into quartiles. Most
responsible physician specialty was divided into general
practitioner (GP), internist, cardiologist and other specialties.
The GPs were further grouped into GPs with or without
internist or cardiologist consultation during hospitalization.
Multilevel model was employed to adjust for physician as a
random effect and age, sex, AMI anatomical location,
comorbidities and procedures as risk factors.
Population Studied: Newly diagnosed AMI patients were
identified from Alberta, Canada hospital discharge
administrative data during 1994/95 and 1999/2000.
Principal Findings: Of 19,928 incident AMIs during the study
years, 19 cases were excluded due to a missing physician
unique identifier. The risk adjusted in-hospital, 30 day and 1
year mortalities were significantly different across quartiles of
physician volume and the 5 categories of physician specialty.
For example, the adjusted odds ratio for 30-day mortality was
0.75 (95% confidence interval: 0.63-0.89) for the 2nd quartile,
0.73 (0.60-0.91) for the 3rd and 0.94 (0.72-1.2) for 4th relative
to the 1st quartile volume. For 1-year mortality, the odds ratio
decreased to 0.82 (0.68-0.98) for the 4th quartile relative to
the 1st quartile. The odds ratio for 1 year mortality was 0.77
(0.65-0.90) for GP with internist/cardiologist consultation,
0.79 (0.66-0.94) for internist, 0.59 (0.50-0.69) for cardiologist
and 0.95 (0.74-1.21) for other specialist when the baseline was
the GP without internist/cardiologist consultation.
Conclusions: This study demonstrated that physician volume
and specialist certification for patients admitted to hospital for
AMI were related to mortality. The higher physician volume
was associated with a decrease in long-term mortality but not
short-term mortality. When internists and cardiologists
provide primary patient care or consultative services, survival
significantly improves.
Implications for Policy, Delivery or Practice: Physician
factors in improving the quality of care are important for policy
makers and consumers. This study noted a diversity in one
dimension of AMI care (mortality) in Alberta, Canada
dependent upon physician volumes and specialty designation.
Low volume physicians or GPs providing care without
specialist consultation for AMI patients was associated with
worse one year survival.
Primary Funding Source: Alberta Center for Health Service
Utilization Research.
Call for Papers
Empowering Patients: The Impact of Public Reporting
& Direct Patient Involvement
Chair: Julie Brown
Tuesday, June 8 • 9:15 a.m.-10:45 a.m.
• Hospital Ratings: Quality Measures or Mere Puffery?
Donna Havens, Ph.D., R.N., Victoria Kellogg, Ph.D., R.N.,
Sherigo Page, B.S.N., Joseph Vasey, Ph.D.
Presented by: Donna Havens, Ph.D., R.N., Academic Division
Chair, The School of Nursing, The University of North
Carolina at Chapel Hill, Carrington Hall, CB #7460, Chapel
Hill, NC 27599; Tel: 919.966.4269; E-mail: dhavens@unc.edu
Research Objective: To explore what is known about three
highly advertised hospital recognition programs that purport
to identify “high performing” hospitals that deliver quality care
to patients.
Study Design: Descriptive statistics, spatial mapping, and
synthesis of the research literature were used to (1) describe
three programs purporting to identify hospitals that are better
than the rest; (2) examine geographic dispersion and variation
of the rated hospitals, (3) identify hospitals with multiple
designations and group overlap; (4) synthesize what is
empirically known about performance of these groups; and (5)
explore policy and research implications.
Population Studied: All hospitals designated in 2001 as
“Best” by the US News and World Report, as “Top” by
Soluscient/HCIA and as “Magnets” by the American Nurses
Credentialing Center. All research publications reporting
evaluations of these three designations were reviewed.
Principal Findings: The designation processes use multiple
frameworks and measurement models--no two designations
use the same criteria. Variation exists in the geographic
dispersion by state and designation type. There is little
overlap—few hospitals are recognized by more than one
rating system. Examination of JCAHO evaluation scores
revealed no statistically significant differences between
designated hospitals and the rest. Logistic regression showed
that higher HCAHO scores only slightly and not significantly
predict being designated. There is a paucity of research
focused on the “Best” and the “Top” and those studies report
conflicting results. Only slightly more research has been
conducted on the magnet hospitals.
Conclusions: Quality health care is a prime concern for all
stakeholders. When it comes to hospital care, “The public has
less information about selecting a hospital than they do when
purchasing a toaster or a car.” The number of hospital ratings,
purporting to identify “high performing” hospitals is growing,
and these designations are considered “PR bonanzas”.
However, little research has been conducted to validate the
ability of these hospital ratings to identify hospitals that deliver
better care than the rest.
Implications for Policy, Delivery or Practice: Quality health
care is a prime concern for all stakeholders. When it comes to
hospital care, “The public has less information about selecting
a hospital than they do when purchasing a toaster or a car.”
The number of hospital ratings, purporting to identify “high
performing” hospitals is growing, and these designations are
considered “PR bonanzas”. However, little research has been
conducted to validate the ability of these hospital ratings to
identify hospitals that deliver better care than the rest.
Primary Funding Source: , The Elouise Ross Eberly
Professorship
• Report Cards and Consumer Choice in Kidney
Transplantation
David Howard, Ph.D., Bruce Kaplan, M.D.
Presented by: David Howard, Ph.D., Assistant Professor,
Department of Health Policy and Management, Emory
University, 1518 Clifton Road, N.E., Atlanta, GA 30322; Tel:
404.727.3907; Fax: 404.727.9198; E-mail:
dhhowar@emory.edu
Research Objective: This study was undertaken to examine
quality the responsiveness of consumers’ choice of kidney
transplant center to quality using a unique dataset consisting
of the universe of registrants for cadaveric kidney
transplantation. There are three principal study questions: 1)
Does quality influence patients’ choice of transplant center? 2)
Do report cards for transplant centers influence choice? 3)
Does the mode of report card dissemination (Internet versus
hardcopy) matter?
Study Design: Patient choice was modeling using a
conditional logit model with random coefficients. Transplant
center attributes – quality (as measured by graft failure rates
one year post-transplant), distance from patients’ home,
lagged market share, and hospital bed size – were interacted
with patient characteristics, including age, primary diagnosis,
education, and insurance type. Coefficients from the model
were used to compute predicted choice probabilities, and
standard errors computed via the delta method. The impact of
report cards was assessed by comparing observed to expected
ratios of the number of patients choosing hospitals that
experienced an improvement in outcomes from one report
card to the next versus hospitals that experienced a decline in
outcomes versus those that experienced no change.
Population Studied: The universe of patients registering for a
cadveric kidney transplant in the continental United States
between January 1, 1998 and June 30, 2001 (N = 47,125).
Principal Findings: The conditional logit model shows that
quality matters; a center that experienced an increase in its
actual one-year graft failure rate of one standard deviation
(0.05) from the sample average (0.09 to 0.14) could expect a 5
percent decline in patient registrations. Transplant centers
with large improvements in outcomes between report cards
experience larger than expected gains in patient registrations.
The converse is true for hospitals that experience a decline in
performance. The placement of report cards on the Internet in
September of 1999 was not associated with a shift in choice
patterns.
Conclusions: 1) At least some patients and their physicians
consider quality when choosing transplant centers. Transplant
centers with high survival rates will attract more patients. 2)
Choice patterns are responsive to the information contained
in report cards. 3) The placement of report cards on the
Internet had no detectable impact on choice patterns.
Implications for Policy, Delivery or Practice: While, is
impossible to gauge from the estimates whether demand is
sufficiently responsive to differences in outcomes to induce
hospitals to make socially optimal investments in quality,
policymakers should not assume that patient welfare is
diminished by increased competition in health care and use of
selective contracting by managed care. Report cards lead
patients to choose hospitals with lower graft failure rates, but
dissemination of report cards on the Internet only may be
insufficient for reaching the types of patients with end stage
renal failure.
• Public Reporting Formats That Motivate Older
Consumers to Compare Medicare Health Plan Options
Nancy Mitchell, B.A., Elizabeth Frentzel Jael, M.P.H., Jennifer
Uhrig, Ph.D., Peyton Williams, B.A., Philip Salib, B.A., Lauren
Harris-Kojetin, Ph.D.
Presented by: Nancy Mitchell, B.A., Policy Analyst, Program
on Aging, Disability, and Long Term Care, RTI International,
1615 M Street, N.W., Washington, DC 20036-3209; Tel:
202.728.2082; Fax: 202.728.2095; E-mail: nmitchell@rti.org
Research Objective: To develop and cognitively test print and
web-based information explaining Medicare and comparing
Medicare health plan options for a pre-Medicare target
audience. The goal was to develop materials for employers to
distribute to employees/retirees about to turn 65 to help them
choose a Medicare plan.
Study Design: Based on findings from previous research with
Medicare beneficiaries, we determined our target audience to
be in the pre-contemplation or contemplation stage within
Prochaska and DiClemente’s Stages of Change model. We
developed three products to help move them to the action
stage. The first product was a bookmark to alert the target
audience and increase their interest in information comparing
Medicare health plans. The second product was a printed
booklet that introduced Medicare and explained how it worked
with their employer’s current health insurance offerings. The
third product was developed as a printed booklet and an
interactive website, and provided comparative information on
available health plans. It also included a summary worksheet
to aid consumers in selecting a plan. Considering Bettman’s
Cognitive Information Processing Model, we attempted to
keep the information concise, and selected the content based
on what consumers reported as most helpful for choosing
health plans. Each product was reviewed by consumer
reporting experts, and cognitively tested with members of the
target audience. We conducted a total of 48 cognitive testing
interviews across 5 rounds of testing. All products were
revised based on the expert review and cognitive testing
findings.
Population Studied: Employees and retirees ages 58-64.
Principal Findings: Participants liked the idea of the
bookmark arriving prior to the other products to alert them
that they would need to think about Medicare soon.
Participants did not like the photos that were initially in the
products because the people looked too old; the photos were
replaced with people who were in the target audience’s age
range. Participants had difficulty understanding generic
descriptions of different health plan options (fee-for-service,
HMO, PPO, etc.). Participants had difficulty understanding
differences in cost; cost information was revised to only key
elements. All participants liked the look and layout of the
products, and reported that they were easy to use.
Participants understood the quality data. Most reported that
they would want information about whether their doctor
participated in the plans before making a final decision. All
participants were able to use the worksheet to track factors
that influenced their decision, and most reported that they
would use the worksheet in real life. Participants trusted the
information as it did not come directly from the health plans.
Conclusions: After multiple rounds of testing and subsequent
revisions, participants understood the material presented and
were able to use the products to choose a Medicare plan.
Implications for Policy, Delivery or Practice: Remove
descriptions of generic plan types (FFS, HMO, PPO, etc), and
only present key elements of cost data such as total estimated
out-of-pocket costs per month. Keep the information concise,
and do not present non-essential details. Use photos that
accurately represent the target population to help them relate
to the materials.
Primary Funding Source: AHRQ
• Consumers’ Use of Quality Information When Selecting a
Health Plan
Julie Rainwater, Ph.D., Jorge Garcia, M.D., Patrick Romano,
M.D. M.P.H.
Presented by: Julie Rainwater, Ph.D., Research Sociologist,
Division of General Medicine, University of California, Davis,
4150 V Street, Suite 2400, Sacramento, CA 95817; Tel:
916.734.5265; Fax: 916.734.2732; E-mail:
jarainwater@ucdavis.edu
Research Objective: For health care markets to operate
efficiently, consumers must review and use information about
quality. Few studies have prospectively evaluated factors
related to the use of health plan quality information during
open enrollment (OE). This study investigates pre-OE
predictors, and post-OE correlates, of consumers’ use of an
employer-sponsored report card on health plan quality.
Study Design: In a prospective longitudinal cohort study, we
examined report card utilization with descriptive analyses that
included demographic variables, health status, pre-OE
satisfaction with health plan (using CAHPS and other
measures), health care utilization, contemplation of a plan
switch during OE, and forced choice due to discontinuation of
the current plan. Multivariate logistic regression was used to
identify independent predictors and elucidate causal
pathways.
Population Studied: In 2001, we randomly sampled 2,500
members of the California Public Employees Retirement
System (CalPERS), and analyzed responses from 1,591
individuals (64% of the total sample) who completed two
mailed questionnaires—one prior to and one immediately
following Open Enrollment.
Principal Findings: Overall, 20.2% of respondents said they
read or reviewed the quality report card during OE. Forcedchoice consumers were more likely to use quality information
than optional choice consumers (38% vs. 16%, p < 0.001).
Among optional choice consumers, report card users were
more likely to have single coverage, be long-time CalPERS
members (5 or more years), and be non-Kaiser HMO
members (rather than Kaiser or PPO members). Report card
users had poorer self-reported general health (p < 0.02), but
were not more likely to report multiple chronic health
conditions, than non-users. Health services utilization
(including the intent to use obstetric services), time in the
current plan, and pre-OE CAHPS satisfaction ratings were not
significantly related to report card usage. However, all CAHPS
satisfaction ratings were lower for those who said they were
“considering switching” health plans, and individuals who
contemplated switching were significantly more likely to
review the report (28% vs. 18%, p < 0.01). Report card users
were also more likely than non-users to relate problems with
having to change doctors in the past (22% vs. 15%, p < 0.02),
experiencing delays while waiting for plan approval (22% vs.
16%, p < 0.04), hearing negative stories about their health
plan (26% vs. 19%, p < 0.05), and anticipating that they might
experience a medical error if they remained with their current
plan (8.4% vs. 4.2%, p < 0.04). Finally, consumers who
voluntarily switched during OE were significantly more likely to
have read the report (32% vs. 15%, p < 0.001).
Conclusions: Consumers who contemplate switching health
plans during OE, or who are required to select a new plan,
make the greatest use of quality information provided in an
employer-sponsored report card.
Implications for Policy, Delivery or Practice: Quality
information has the greatest salience for individuals who
contemplate changing their health plan, due to poor health or
dissatisfaction, or who are in a forced-choice situation.
Producers and disseminators of report cards should consider
efforts to identify and target these individuals.
Primary Funding Source: AHRQ
Related Posters
Poster Session A
Sunday, June 6 • 6:45 p.m.-8:00 p.m.
• Electronic Technologies: When Will They Make It into
Physicians' Black Bags?
Anne-Marie Audet, M.D., M.Sc., SM, Stephen Schoenbaum,
M.D., M.P.H., Michelle Doty, Ph.D., M.P.H., Jamil Shamasdin,
Jordon Peugh, M.S., Kinga Zapert, Ph.D.
Presented by: Anne-Marie Audet, M.D., M.Sc., SM, Assistant
Vice President, Quality of Care, The Commonwealth Fund,
One East 75th Street, New York, NY 10021; Tel: 212.606.3856;
E-mail: ama@cmwf.org
Research Objective: Physicians in the United States are
adopting electronic technologies at a slow rate, despite
studies demonstrating their clinical benefits. Compared with
what is known about technology use in hospitals and other
institutions, little is known about how physicians in private
practices use such technology.
Objectives: To investigate physicians’ current use, future
plans, and barriers in adoption in regard to: electronic medical
records (EMRs), electronic prescribing and ordering, clinical
decision support systems (CDSs), and electronic
communication with other physicians and patients.
Study Design: Mail survey completed by a national random
sample of 3,598 physicians caring for adult patients. The
survey was conducted between March and May 2003. The
survey sample was randomly selected from a national list the
American Medical Association (AMA) that includes both AMA
members and non-members. All physicians in the sample
were involved in the direct care of adult patients and had been
in practice at least three years post-residency.
Population Studied: National sample of US physicians caring
for adult patients.
Principal Findings: Physicians most commonly use electronic
technologies for billing (79%). The most frequent clinical use
is to access laboratory results (59%). Other uses are less
prevalent: 27 percent of physicians report using EMRs
routinely or occasionally; 27 percent use electronic ordering
tools to prescribe or order tests; and 12 percent of physicians
receive electronic alerts about drug prescribing problems.
Physicians in groups larger than 50 are 9.5 times more likely to
be using an EMR compared with those in solo practice
(adjusted OR, p <0.001), and twice as many physicians in
large practices plan to adopt an EMR in the next year
compared with physicians in solo practices (22% vs. 13% p
<0.05). The top three barriers to adoption reported were startup costs (56%), lack of standards within the IT industry
(44%), and lack of time (39%).
Conclusions: Most physicians have yet to embrace electronic
technologies as a clinical management tool. Adoption is
uneven, and depends on the practice environment.
Implications for Policy, Delivery or Practice: Cost remains
the major barrier to diffusion. Attention needs to be given to
policies and business models that will make electronic
technologies affordable to all physicians.
Primary Funding Source: CWF
• Organizational and Institutional Determinants of Quality
Control in Physician Office Laboratories
George Avery, M.P.A.
Presented by: George Avery, M.P.A., Instructor, Psychology,
University of Minnesota-Duluth, 1207 Ordean Court, BohH
320, Duluth, MN 55812; Tel: 218.726.7364; E-mail:
aver0042@umn.edu
Research Objective: This study examines compliance by
physician office laboratories (POLs) with regulations such as
those promulgated under the Clinical Laboratory
Improvement Act of 1988 (CLIA ’88) to assure the quality of
the data produced in the POL. Current research findings
indicate that deficiencies exist in laboratory quality control
systems. Specifically, the project examines whether:
1.
Increasing regulatory requirements under CLIA ’88
are related to more frequent use in POLs of quality control
tools commonly used to assure data quality.
2.
Different regulatory environments represented by
private sector laboratory accreditation agencies such as the
College of American Pathologists (CAP), Joint Commission on
the Accreditation of Healthcare Organizations (JCAHO), etc.
are related to stronger or weaker quality systems in POLs.
The project proposes that factors in organizational structure,
learning, communications, and values has a direct effect on
both the selection of an appropriate level of regulation by the
POL and interact with the regulatory environment to
determine the strength of the laboratory quality system. By
controlling for a set of such factors, the project proposes to:
3:
Model how the environments created by CLIA
regulations and private sector accreditation programs interact
with these organizational factors to stimulate use of quality
control tools in the POL.
Study Design: A two-stage multilevel LOGIT model is built to
model determinants of laboratory choice of regulatory level,
with predicted regulatory level used as an instrumental
variable to model the interaction of regulatory institutions and
organizational climate on the use of quality control in
physician office laboratories.
Population Studied: A national sample of 633 tests
performed by 121 physician office laboratories studied by the
Arkansas Laboratory Medicine Sentinel Network (ALMSN) is
examined. Data on organizational climate and quality control
usage from the ALMSN is matched to sturctural data from the
CMS On-line Surveillence, Certification, And Reporting
(OSCAR) dataset.
Principal Findings: 1. Laboratories regulated by private-sector
accrediting bodies such as JCAHO or CAP are more likely to
perform quality control tests than those regulated by publicsector regulatory bodies.
2. The stronger the role profit maximization plays in
organizational culture, the less likely the laboratory will
perform quality control measures.
3. Organizational committment to quality is a predictor of
quality control usage.
4. An organizational perception of resource adequacy is a
predictor of non-performance.
Conclusions: Private sector accrediting agencies operating in
a market for regulation are more effective in producing quality
behavior in physician office laboratories than public regulatory
agencies. This is likely due to incentives for private sector
bodies to reduce the opportunity cost of regulation, reducing
disincentives to comply with regulatory requirements.
Implications for Policy, Delivery or Practice: These findings
suggest that certification by public agencies plays little role in
improving quality. This does not imply that CLIA plays no role
in quality improvement, as the requirements for certification
and/or accreditation have been shown to limit the scope of
tests in laboratories least likely to adopt a stringent quality
control program, and have facilitated the decision of
laboratories to seek accreditation. It does imply, however, that
private accreditation agencies may be better at gaining
laboratory quality improvements than the public agencies, and
that future improvements in the CLIA program might be
obtained by using these private agencies, rather than public
agencies, to accredit and inspect laboratories.
Primary Funding Source: CDC
• Higher Case Volumes of Elderly Patients is not
Associated with Safer Medication Use in This Population
Raj Behal, M.D., M.P.H.
Presented by: Raj Behal, M.D., M.P.H., Medical Director,
Clinical Systems Effectiveness, Clinical Practice Advancement
Center, University HealthSystem Consortium, 2001 Spring
Road, Suite 700, Oak Brook, IL 60523; Tel: 630.954.4892; Fax:
630.954.5819; E-mail: rbehal@uhc.edu
Research Objective: Use of long-acting benzodiazepine
medications among older patients is associated with a high
risk of adverse effects. It is not known whether hospitals
experienced in caring for a larger volume of elderly patients
are less likely to use these medications.
Study Design: Pharmacy charge files linked with inpatient
discharge data (UB92) were used to identify patients
discharged with general medical DRGs from twenty-six
academic medical centers between July 1 2002 and June 30
2003. Revised Beers criteria for potentially inappropriate
medications were applied to estimate use of long acting
benzodiazepine (LA-BDZ, defined as diazepam or
chlordiazepoxide) in two distinct age groups: a) non-elderly,
age 18 to 64 years; and b) elderly, age 65 or older. For each of
the twenty-six hospitals, proportion of general medicine cases
that were 65 years or older was determined. Correlation
between volume of elderly cases and prevalence of
benzodiazepine use among elderly was assessed.
Population Studied: General medicine patients over the age
of 65 discharged from twenty six academic medical centers
over 12 months period.
Principal Findings: The elderly accounted for 21.5% of the
overall LA-BDZ use. Overall, prevalence of LA-BDZ usage
among elderly patients ranged from 0% to 5% (median 2%);
prevalence of LA-BDZ use among non-elderly patients was 1%
to 7% (median 3%). Elderly cases accounted for 18% to 50%
of the study hospitals (median 30%). There was no correlation
between hospital volume of elderly cases and prevalence of
use of benzodiazepines (R2 = 0.0015).
Conclusions: Among general medicine patients in academic
medical centers, patients older than 65 years account for over
one-fifth of usage of long-acting benzodiazepine –
medications considered potentially inappropriate regardless of
diagnosis. Prevalence of use of these medications was similar
among non-elderly and elderly patients, possibly indicating a
lack of systemic controls to prevent use of unsafe
medications. Overall prevalence of LA-BDZ use among elderly
patients is not affected by the hospital experience in caring for
older patients.
Implications for Policy, Delivery or Practice: Volume alone
may not be a good proxy for safe medication use.
• Organizational Structure and Processes Affect
Performance on Patient Safety Outcomes
Raj Behal, M.D., M.P.H.
Presented by: Raj Behal, M.D., M.P.H., Medical Director,
Clinical Systems Effectiveness, Clinical Practice Advancement
Center, University HealthSystem Consortium, 2001 Spring
Road, Suite 700, Oak Brook, IL 60523; Tel: 630.954.4892; Fax:
630.954.5879; E-mail: rbehal@uhc.edu
Research Objective: To develop a tool for evaluation of
patient safety programs in acute care hospitals and to
determine organizational features that affect patient safety
outcomes.
Study Design: Survey study of a convenience sample of
academic medical centers. Survey design: The recent patient
safety literature was reviewed for recommendations for
implementing safety programs. These recommendations were
organized in a framework adapted from the Burke-Litwin
model for organizational performance. The framework was
then used to develop questions for evaluation of a safety
program in the following areas: leadership, mission and
strategy, culture; structures, systems, managerial; task
requirements and skills; and individual needs and values. In
addition, questions regarding proactive evaluation of ten highhazard clinical processes were developed. Failure to Rescue, a
patient safety indicator developed by the Agency for
Healthcare Research and Quality, was selected as the
outcome measure.
Population Studied: Ten academic medical centers in the
United States
Principal Findings: 9 of 10 medical centers scored less than
75% on the tool. Medical centers with specific structures,
systems, and managerial practices in place were more likely to
perform proactive assessment of high hazard care processes
and had better performance on failure to rescue indicator.
Conclusions: Presence of specific organizational features is
associated with better performance on measures of patient
safety. Leadership, mission and strategy and culture taken in
aggregate do not appear to correlate with performance.
Findings of this pilot study are limited due to small sample
size and require confirmation with a larger sample.
Implications for Policy, Delivery or Practice: Leaders in
academic medical centers should guide implementation of
evidence-based structures, systems and managerial practices
in order to improve patient safety. Public and private funding
to support organizational development in this manner should
be encouraged.
• Do Ambulatory Care Sensitive Conditions Affect
Beneficiaries’ Experience and Satisfaction with Health
Care?
Shulamit Bernard, Ph.D., R.N., Erica Brody, M.P.H.
Presented by: Shulamit Bernard, Ph.D., R.N., Program
Director, Healthcare Quality Program, RTI International, 3040
Cornwallis Road, Research Triangle Park, NC 27709; Tel:
919.485.2790; Fax: 919.990.8454; E-mail: sbernard@rti.org
Research Objective: To examine whether the incidence of
ambulatory care sensitive conditions are associated with self
reports of poor access to care or dissatisfaction with health
care services.
Study Design: We used data from the 2000 National
Medicare Fee-for-Service (MFFS) CAHPS survey, conducted
for the Centers for Medicare and Medicaid Services (CMS).
The sample of beneficiaries, drawn from a sampling frame
constructed from the CMS Enrollment Data Base (EDB),
resulted in 103,551 (64%) completed surveys. MFFS survey
data were merged with 1999-2001 Medicare claims data.
Using ICD-9 codes, and data from the twelve months
preceding the survey response date, we constructed a
measure of Ambulatory Care Sensitive Conditions (ACSC). In
addition, we estimated case-mix adjusted means for 5 CAHPS
measures using the CAHPS Macro version 3.4. The case-mix
adjusted means were stratified by the ACSC indicator and twosample tests for mean differences were performed. Due to
large sample sizes the Gaussian distribution was assumed.
Population Studied: Beneficiaries enrolled in the Medicare
Fee-for-Service program in 2000.
Principal Findings: We found that Medicare beneficiaries
who had a claim for an inpatient, observation stay or
emergency room visit for an ACSC were more likely to report
problems with getting needed medical care as measured by
the CAHPS Needed Care Composite. We did not find any
relationship between ACSC and the likelihood of problems
with the Getting Care Quickly or the Communication
Composites. There was no relationship between having an
ACSC and ratings of satisfaction with Medicare or health care.
Conclusions: Our findings suggest that Medicare
beneficiaries who report access problems with getting needed
care also experience higher rates of ACSC. The lack of a
similar finding for the Getting Care Quickly composite is
puzzling. In addition, beneficiaries who had an ACSC do not
differ in their reports of satisfaction from beneficiaries who did
not have an ACSC, suggesting that ACSC do not impact
negatively on more attitudinal measures of quality or that
beneficiaries do not attribute an ACSC to the plan
performance.
Implications for Policy, Delivery or Practice: The findings
suggest that patients reporting barriers to obtaining needed
care do in fact have a higher rate of adverse medical events.
Further, this analysis suggests that while some reports are a
direct result of the quality of care received and may be used to
monitor clinical quality, satisfaction with care is not associated
with this clinical measure of quality and access.
Primary Funding Source: CMS
• The Impact of Quality Improvement Organizations on
Acute Myocardial Infarction Care: Views of Hospital
Quality Management Directors
Melissa Carlson, M.P.H., M.B.A., Elizabeth Bradley, Ph.D.,
William Gallo, Ph.D., Jeanne Scinto, Ph.D., M.P.H., Miriam
Campbell, Ph.D., M.P.H., Harlan Krumholz, M.D.
Presented by: Melissa Carlson, M.P.H., M.B.A., Doctoral
Candidate, Epidemiology and Public Health, Yale University,
60 College Street, New Haven, CT 06520; Tel: 917.434.7293;
E-mail: Melissa.Carlson@yale.edu
Research Objective: Extensive resources are committed to
the Centers for Medicare and Medicaid Services’ (CMS)
Quality Improvement Organization (QIO) program; however,
little is known about how hospitals perceive the QIO’s effect
on quality of care. This study’s objective is to determine how
hospital quality management directors view the impact of
QIOs on the care delivered to patients admitted for acute
myocardial infarction (AMI) at their institutions.
Study Design: We interviewed hospital quality management
directors regarding the extent of their interaction with the
QIO, the existence and helpfulness of QIO interventions, the
degree to which QIO interventions helped or hindered quality
efforts at their hospital, and their recommendations for
improving QIO effectiveness.
Population Studied: A national random sample of 105
hospital quality management directors interviewed between
January and July 2002.
Principal Findings: The majority (73%) of quality
management directors reported that the amount of contact
with the QIO (mean and median = 3 contacts in the last year)
was appropriate; 27% preferred more contact. The most
commonly reported QIO interventions were educational
programming (99/105) and the provision of benchmark data
(93/105) and hospital performance data (82/105) and more
than 60% of respondents found these interventions to be
“very helpful”. Many hospitals also reported that QIOs helped
develop quality indicators for the hospital and 85% found this
intervention to be “very helpful.” One-quarter of respondents
(27/105) believed quality of care would have been worse
without the QIO interventions. Recommendations to QIOs
for improving their effectiveness included: appealing more
effectively to both physicians and senior administration in the
hospital, and enhancing and maintaining the perceived validity
and timeliness of data used in quality indicators.
Conclusions: Our study demonstrates that the QIOs have
largely overcome their previously adversarial relationship with
hospitals and have effectively become partners with hospitals
in promoting health care quality. This is evidenced by the
frequent and positive contacts between the QIO and the
hospital quality management staff, the many interventions in
which QIOs are actively involved, and the prevalent view by
hospitals that these interventions are helpful. The generally
positive view among most hospitals concerning the QIO
interventions suggests that QIOs are poised to take a leading
role in promoting quality of care. However, to fulfill this
leadership role effectively, QIOs need to integrate physicians
and administrators more fully into their initiatives.
Implications for Policy, Delivery or Practice: The success of
the QIO model in improving the quality of AMI care
suggested by this study implies that future partnerships
between providers and QIOs, if forged with a collaborative
spirit, have the potential to improve the quality of health care
in multiple settings. However, the full potential of QIOs will
likely not be realized until QIOs are able to get greater
physician and hospital administration engagement.
Primary Funding Source: Qualidigm staff (JS) assisted in the
preparation of this material under contract number 500-99CT02 modification #7 with the CMS. The contents presented
do not necessarily reflect CMS policy. Pub. # 6SOW(AMICASPRO,POPS-2003).
• Women Veterans’ Satisfaction With Inpatient
Hospitalization
Carron Cherrie-Benton, Ph.D. Candidate
Presented by: Carron Cherrie-Benton, Ph.D. Candidate,
Project Manager, VISN 8 Measurement Team, VA, 11605
North Nebraska Avenue, Tampa, FL 33612; Tel: 813.558.3955;
Fax: 813.558.3994; E-mail: Carron.cherrie@med.va.gov
Research Objective: Understanding women veterans’
satisfaction is one means to improve the quality of health
services in the VHA system. A cross-sectional survey was
conducted to determine women veterans’ satisfaction with
inpatient care in VISN 8.
Study Design: The multi-site descriptive survey included 329
women from six VAMCs. A modification of the Boston Picker
Institute Survey, which consists of the nine Veterans
Healthcare Service Standards, was used. A mixed method
approach incorporated open-ended questions to assess
perceptions of safety and privacy. Site coordinators recruited
women being discharged from inpatient care. Surveys were
mailed from a central location with a self-addressed stamped
envelope. Data analysis included descriptive statistics, T-tests,
one-way analysis of variances, and qualitative analysis.
Comparisons were made with 2002 survey results and VHA
national data.
Population Studied: Women veterans discharged from
inpatient facilities in VISN 8.
Principal Findings: The response rate was 55.52%,
comparable to the VHA national survey response rate of
55.97%. Results indicated that a majority of women veterans
(86.5%) were more satisfied than the national sample
(76.54%) with the quality of care in VISN 8 facilities. Women
were informed about their medical condition (87.6%),
included in treatment decisions (83.7%) and had enough
privacy (87.6%). Further analyses of VISN compared to
national data will be presented.
Conclusions: Overall, women veterans are satisfied with inpatient care in VISN 8 facilities. Opportunities for
improvement are in the availability of caregivers, discharge
instructions, follow-up, and out-patient care coordination.
Implications for Policy, Delivery or Practice: As a vulnerable
minority in the VHA system, it is imperative for future health
service planning, policy and quality improvement to
understand women veterans’ satisfaction with VA health
services.
Primary Funding Source: VA
• Mortality Following Admission for Acute Myocardial
Infarction in Military Health System Facilities: Factors
Associated with 30 Day and One Year Survival
Stanley Chin, M.S., Lanna Forest, Ph.D., Thomas Williams,
Ph.D.
• Quality Transformation: Assessing the Impact of
Computerized Order Sets on Asthma Quality Indicators
for Inpatients
Deena Chisolm, Ph.D., Sofia Veneris, M.H.A., Ann
McAlearney, Sc.D., Kelly Kelleher, M.D., M.P.H.
Presented by: Stanley Chin, M.S., Senior Health Scientist,
ACS Government Services, 5270 Shawnee Road, Alexandria,
VA 22312; Tel: 703.310.0034; E-mail: stanley.chin@acs-inc.com
Research Objective: To test for differences in risk-adjusted
mortality rates following Acute Myocardial Infarction between
military hospitals, or Medical Treatment Facilities (MTFs), as
compared to civilian hospitals within the Military Health
System (MHS).
Study Design: A retrospective cohort from administrative
data, using logistic regression to predict 30-day and one-year
mortality in MTFs and civilian hospitals. Rates are adjusted for
comorbid conditions, gender, age, and utilization patterns
using longitudinal administrative data.
Population Studied: Patients who were hospitalized for AMI
in either FY01 or FY02 (n=11,895) in either an MTF or a civilian
hospital, following exclusions for rule-out admissions,
transfers, and other factors.
Principal Findings: There is a significant difference in
mortality outcomes across the two systems. Risk-adjusted 30day mortality rates were 10.9% (10.3-11.5%) for civilian sources
of care and 8.2% (7.0-9.5%) for MTF care. Additional
explanatory factors include age and gender: younger women
were found to have higher mortality rates than men with
similar characteristics (age 45-55 comparing women to men:
OR 0.43, 0.26-0.73), although there were no significant gender
differences at higher ages; having had contact with the
healthcare system in previous twelve months is associated
with higher survival (OR 1.29, 1.08-1.53). Survival rates are
significantly lower for patients without a previous AMI and an
unspecified infarction site (OR 0.42, 0.30-0.59). Lack of
documented contact with the healthcare system in the year
prior to the AMI admission was also found to be a significant
predictor of mortality (in thirty-day mortality, odds ratio 1.29,
95% CI 1.08-1.53). When comparing risk-adjusted mortality
outcomes in MTFs only, no significant difference was found
between major medical centers and community hospitals.
Conclusions: Care for AMI at civilian hospitals has
significantly higher mortality than care for similar patients
delivered at military hospitals. Additional uncontrolled factors
other than quality of care may contribute to these differences,
including time to treatment, type of hospital, timing of
procedures and drug therapies, and clinical condition at
arrival. Further analyses, using these clinical data and
advanced techniques (instrumental variables, hierarchical
modeling) is indicated.
Implications for Policy, Delivery or Practice: Additional
quality assurance monitoring may be indicated in specific
civilian facilities. Continued monitoring of outcomes and
further examination of other, uncontrolled causal factors is
warranted to test if problems in access to care, transport time
to hospitals, or other sources of mortality can be found and
reduced.
Primary Funding Source: Dept. of Defense
Presented by: Deena Chisolm, Ph.D., Post-Doctoral Research
Fellow, Center for Health, Outcomes, Policy, and Evaluation
Studies, The Ohio State University, 320 West 10th Avenue, A333, Starling-Loving Hall, Columbus, OH 43210; Tel:
614.438.6869; Fax: 614.438.6859; E-mail:
chisolm.1111@osu.edu
Research Objective: Condition-specific order sets within
computerized physician order entry--CPOE--systems are
designed to decrease unnecessary practice variation and to
promote best practices. These sets build evidence-based
treatment recommendations into the normal flow of patient
care, making the use of evidence an integral part of the care
process. Most researchers and policy-makers, including the
influential LeapFrog Group, agree that this integration will
improve quality of care. Columbus Children’s Hospital, Inc.-CCHI--has recently implemented a CPOE system with diseasespecific order sets to reach this goal. Our research objective
for this project was to determine the impact of CCHI’s asthma
order set on indicators of asthma care quality.
Study Design: Three indicators of asthma care quality were
measured: use of systemic corticosteroids, use of metereddose inhalers, and use of pulse oximetry. Rates for each
indicator were calculated for three patient groups: those
admitted prior to order set roll-out--pre-set; those admitted
after roll-out without order set used--no set; and those
admitted after roll-out with order set used--set. Utilization
rates for the three groups were compared using raw
frequencies and logistic regression. Length of stay, total
charges, and pharmacy charges were calculated to measure
the impact of order set use on cost efficiency. Group
differences for these variables were tested using ANOVA. All
analyses for this study were conducted using data from the
CCHI Decision Support System and the Eclypsis CPOE
system.
Population Studied: The study population included patients
with a primary ICD-9 diagnosis code of 493.xx admitted to
CCHI between November 1, 2001 and November 30, 2003.
Patients admitted directly to the pediatric intensive care unit
were excluded. The population analyzed included 261 pre-set
patients, 63 no set cases, and 466 set patients.
Principal Findings: There was a significant positive
relationship between asthma order set use and selected
quality indicators. Multivariate logistic regression showed that
order set patients are significantly more likely to have systemic
corticosteroids used during the admission—odds ratio of
6.08, significantly more likely to have an inhaler used--odds
ratio of 1.41, and significantly more likely to have pulse
oximetry--odds ratio of 2.62. No set patients did not differ
significantly from pre-set patients on any indicator. The
increase in appropriate treatment use did not lead to any
increase in cost. No significant differences were found in total
charges, pharmacy charges, or length of stay between the
three study groups.
Conclusions: Use of a disease-specific order set within a
CPOE system can significantly improve healthcare quality
across a variety of quality indicators with no associated
increase in charges. Consistent use of order sets based on
best practices can, therefore, improve the overall quality of
care provided by a healthcare institution without increased
cost.
Implications for Policy, Delivery or Practice: Transforming
the delivery of healthcare through the use of information
technology can be an expensive, complex process. However,
this major investment can lead to measurable and meaningful
improvements in healthcare delivery. These results give
further credence to policy makers’ calls for expanded use of
CPOE systems with condition-specific order sets.
• Assessing Organizational Climate of Patient Safety:
Baseline Survey
Judith Colla, Sc.D., Ann Bracken, M.D., Ph.D., Diana Luan,
RN, M.S., William Weeks, M.D., M.B.A.
Presented by: Judith Colla, Sc.D., Research Associate,
Community & Family Medicine, Dartmouth Medical School,
Center for Evaluative Clinical Studies, Hanover, NH 03755; Tel:
802.295.9363 Ext. 5806; Fax: 802.291.6286; E-mail:
judith.colla@dartmouth.edu
Research Objective: We wanted to provide baseline
measurements for determining if training in patient safety
improves senior managers’ perceptions of the patient safety
climate at their respective organizations. We also wanted to
provide feedback to participants at their next training, identify
limitations that might be addressed before the second
administration of survey, and clarify analytical methods.
Study Design: The Veterans Healthcare Administration’s
(VA)National Center for Patient Safety (NCPS) recently
initiated training in patient safety for teams of senior
managers from fifteen different states and, in collaboration
with researchers at the VA in White River Junction, VT and
Dartmouth Medical School, administered a cross-sectional
survey measuring participants’ perceptions of the patient
safety climate at their respective organizations.
The Patient Safety Questionnaire is a validated instrument
with 4 demographic questions, 54 items using a Likert Scale
from 1 to 5, and 9 previously established domains. Items
were coded so that higher scores reflect more favorable
responses with respect to a climate of patient safety. Analysts
calculated the prevalence of agreement (4 or 5 on Likert Scale)
and disagreement (1 or 2 on Likert Scale) for each item as well
as mean scores for each domain, stratifying results by
previous training in Root Cause Analysis (RCA) or Failure
Mode and Effects Analysis (FMEA).
Population Studied: The population studied consisted of
senior managers from fifteen different states participating in
patient safety training recently initiated by the NCPS of the VA.
Of 50 participants, 47 (94%) completed the baseline survey.
Of respondents, 25 (54%) worked for the state and 21 (46%),
a healthcare organization such as a hospital. Forty-one (91%)
reported their primary work to be administrative and 3 (6.7%),
direct patient care. Twenty-five (53.2%) reported having had
training in RCA; 19 (40.3%), in FMEA; and 28 (59.6%), in
either RCA or FMEA.
Principal Findings: Even at baseline, respondents agreed
(66.4%) more often than they disagreed (19.6%) with
statements reflecting climates that enhance patient safety.
Also, for eight of the nine domains, mean scores were
between three and four, also reflecting climates that enhance
patient safety. Perceptions of patient safety climate were less
favorable for participants in this training than reported
previously for top management from hospitals purposely
selected for their initiative in promoting patient safety and
more favorable for those previously trained in patient safety,
significantly so for RCA training.
Conclusions: Senior managers’ perceptions of the patient
safety climate at their respective organizations are positive at
baseline. Prior patient safety training was associated with
participants reporting enhanced patient safety climates.
Implications for Policy, Delivery or Practice: Training in
patient safety appears to improve perceptions that senior
managers have of patient safety climate at their respective
organizations, especially with regard to norms about reporting
errors. Administering this survey to employees at all levels in
their respective healthcare organizations should be useful in
validating the perceptions of these top managers, detecting
differences by organizational level, distinguishing regional
differences, and overcoming the limitations of small sample
size.
Primary Funding Source: AHRQ,
• Rural Patient Safety: Needs Assessment in Rural
Hospitals
George Demiris, Ph.D., Suzanne Austin Boren, MHA, Kathryn
Nelson, MHA, Leslie Hall, M.D., Wilbert Meyer, M.H.A., Shari
Riley, J.D., M.H.A.
Presented by: George Demiris, Ph.D., Assistant Professor,
Health Management and Informatics, University of MissouriColumbia, 324 Clark Hall, Columbia, MO 65211; Tel:
573.882.5772; Fax: 573.882.6158; E-mail:
DemirisG@health.missouri.edu
Research Objective: Several initiatives are addressing patient
safety enabling an electronic voluntary reporting of adverse
events within academic medical centers in urban settings.
Such initiatives are lacking in the rural context and it remains
unknown whether the same challenges and solutions apply to
rural hospitals.The study purpose is to provide insight into the
organizational culture and level of readiness to adopt patient
safety strategies in a rural setting as well as to identify critical
issues pertaining to the rural context that need to inform the
design of such strategies.
Study Design: We developed two interview protocols, one for
the interviews conducted with administrators (chief executive
officers, quality officers etc.) and one for the interviews with
health care providers (physicians, nurses, physical therapists
etc.) The interview protocols consisted of 11 items for the
administrators’ sessions and 13 items for the health care
providers’ sessions and had a Flesch Reading Ease of 50.1 and
42.4 respectively. The Flesch Kincaid Grade Level for both
instruments was 10. The Health Sciences Institutional Board
Review of the University of Missouri approved the interview
protocols. Letters were mailed to the chief executive officers of
eight rural hospitals explaining the purpose of the study and
asking them to identify administrators and health care
providers in their institution who could be contacted for an
interview. The sample of rural hospitals was a convenience
one, allowing for a diverse representation of rural hospitals in
the state of Missouri. The individuals were then contacted by
phone. After explaining the purpose of the study and the
mechanism to ensure the anonymity of their responses, they
were asked to provide consent to participate. If individuals
agreed to participate, an interview session was scheduled.
Interviews were conducted over the phone. The taped
sessions were transcribed and analyzed. The content analysis
software QSR-N6 was used.
Population Studied: Eight rural hospitals in the state of
Missouri were selected as a convenience sample.
Six of these hospitals are accredited by the Joint Commission
on Accreditation of Healthcare Organizations (JCAHO). All
eight hospitals have a surgical unit; four of them have an
intensive care unit and only two have a rehabilitation unit. Two
of the hospitals are non-profit, five are government owned and
one is a for-profit institution.A total of 16 administrators and
14 health care providers were interviewed. Of the 16
administrators, two stated their title as Chief Executive
Officers, five as Quality Improvement Managers, six as
Directors of Clinical Services, one as Chief Operating Officer
and two as Risk Management Services. Of the 14 health care
providers, nine were physicians (no specialties were recorded)
and five nurses.
Principal Findings: The interview sessions lasted an average
of 10.4 (± 0.08) minutes. Based on both the administrators’
and care providers’ responses, four mechanisms of error
reporting were identified:
A) The individual who experienced or participated in the event
filled out the event report. These forms are then coded based
on a script provided by the liability insurance. Information is
forwarded to the insurance and after it has been processed,
the hospital receives a form displaying aggregate data.
B) Errors are reported to the Quality Improvement Manager.
These reports can be formal (using the report forms) or
informal (conversation). The Quality Improvement Manager
documents the reports and decides if a root cause analysis
needs to be performed for any of them.
C) A committee of staff members reviews all report forms and
decides if any action needs to be taken.
D) A medication error hotline is in place so that health care
providers can call and report a medication error. All other
events can be reported to the Risk Manager.
These mechanisms are not exclusive so that staff members
could report that their organization had two mechanisms
(e.g., A and C) in place.
Only three of the administrators (19%) felt that there was a
timely response to adverse event reports. Half of the
administrators (50%) stated that the current mechanism is an
appropriate and adequate outlet to ensure patient safety.
Three administrators reported that errors and adverse events
are reported properly, while one respondent felt that these
were being over-reported and the remaining 12 (75%) believe
that they are being under-reported. The majority of the
administrators (88%) perceived the ability to describe trends
and patterns within reported data sets that would promote
systemic resolution or error prevention as very important.
While half of the administrators rated the current reporting
mechanism as appropriate and adequate, only 21% of the
health care providers agreed with this statement. None of the
health care providers found errors and adverse events to be
over-reported; the majority (93%) believe that they are being
under-reported. Only 36% of the care providers interviewed
have themselves reported an error or adverse drug event
during their tenure with their organization. However, 71% of
the providers believe that there is no culture of blame that is
regularly placed on individuals involved in medical errors.
Health care providers stated that an electronic reporting
system would be useful and could have the potential to
shorten the reaction time after an event has been reported.
Issues of usability, training and infrastructure were raised as
possible implementation challenges.
Conclusions: The study findings indicate the need for a
medical error reporting system that will enable an effective
outlet for ensuring patient safety in rural clinical settings. The
fact that the majority of both administrators and health care
providers stated that there is no timely response to medical
error reports, is a definite call for improvement of the current
infrastructure. The great majority of health care providers
stated that the current mechanisms do not lead to identifying
best practices and do not provide appropriate and adequate
outlets to ensure patient safety. The paper- based systems that
are currently in place, are perceived mostly as simple and easy
to use but not as effective interventions that could lead to
resolution. On the other hand, while an electronic reporting
system appears to be useful, concerns have been voiced about
the current lack of infrastructure and staff members’ familiarity
with computers in many of the rural sites.
The study findings indicate that the current patient safety
approaches in the rural settings under study are unlikely to
lead to a comprehensive understanding of patient safety or
promote effective targeting of unsafe practices. The
improvement of patient safety can be accomplished effectively
with a focus on primarily prevention. Such an undertaking
relies on a comprehensive understanding of the
environmental and behavioral circumstances surrounding
adverse events. In addition, an analysis of the epidemiology of
such events can lead to the development of systemic,
environmental, and behavioral efforts to prevent their
recurrence. Several patient safety interventions have been
designed for academic urban medical centers; the
environmental and behavioral circumstances that impact
patient safety, however, might differ between urban and rural
settings. Therefore, the implementation of an adverse event
reporting system in a rural setting needs to be customized to
address the needs and expectations of administrators and
health care providers in the rural context.
Implications for Policy, Delivery or Practice: The
implementation of an adverse event reporting system in a
rural setting needs to be customized to address the needs and
expectations of administrators and health care providers in the
rural context. This study provided an insight to patient safety
culture and awareness of several rural hospitals pointing out
the importance of interventions designed specifically for rural
settings. This study also calls for action in the areas of rural
care providers education and policy implications for error
reporting in rural settings.
• Receipt of Routine Surveillance Care Following Cancer
Treatment with Curative Intent
Jennifer Elston Lafata, Ph.D., Greg Cooper, M.D., Janine
Simpkins, M.A., Lonni Schultz, Ph.D., Gary Chase, Ph.D., Lois
Lamerato, Ph.D.
Presented by: Jennifer Elston Lafata, Ph.D., Director, Center
for Health Services Research, Henry Ford Health System, 1
Ford Place, 3A, Detroit, MI 48202; Tel: 313.874.5454; Fax:
313.874.1883; E-mail: jlafata1@hfhs.org
Research Objective: Many medical societies recommend
routine surveillance to detect recurrent disease among cancer
survivors. Yet, one of the least studied aspects of cancer
control is the care delivered to survivors. We estimate the
proportion of cancer survivors receiving surveillance care and
compare care received to established clinical practice
guidelines.
Study Design: Information on outpatient care receipt and its
indication was abstracted from medical records for potential
surveillance care received up to 5 years following initial
treatment. Kaplan-Meier product limit estimates were used for
time to surveillance care receipt (routine office visits, and
recurrence and metastatic disease testing delivered for
screening purposes) as well as to attainment of care that met
or exceeded published guideline recommendations.
Population Studied: Cohorts of patients aged 30 and older
diagnosed with breast, colorectal, endometrial, lung, or
prostate cancer between 1990-1995 in an integrated delivery
system who received treatment with curative intent were
identified via tumor registry (N=100 per cancer type).
Principal Findings: Service receipt was common among
these cancer survivors. With the exception of colorectal
cancer patients, over 90% had 1 and over 88% had 2 routine
office visit(s) within 18 months of treatment. Colorectal
patients were least likely to have received testing for new
primaries (only 61% had such exams by 30 months, compared
to over 80% for those with other cancers). Regardless of
cancer site, median time to receipt of first year guideline
recommended care was less than 10 months. Yet, while over
88% of lung, endometrial and prostate patients received year
one guideline recommended care by 18 months, only 78% of
colorectal and 69% of breast cancer patients had done so.
Care that exceeded guideline recommendations was also
common. This was particularly true for routine office visits,
where over two thirds of all patients had received 4 or more
routine visits by 18 months. Metastatic disease testing, often
not recommended by published guidelines, was common
regardless of site. By 30 months post treatment, 74% of those
with lung cancer and 99% of those with prostate cancer had
received such testing. Likewise, approximately a third of
colorectal and prostate cancer patients exceeded guideline
recommendations for routine laboratory testing by 18 months
and almost 25% of breast cancer patients had exceeded the
recommended number of mammograms by 18 months. By
30 months post-treatment, these numbers had risen to 51%
and 61%, respectively.
Conclusions: Regardless of cancer site, the receipt of routine
follow-up care is common among cancer survivors. In all
cases the median time to guideline recommended care after
curative treatment was less than 10 months. Similarly, first
year care exceeding published clinical practice guidelines was
not uncommon. Yet, also of note is a non-negligible
proportion of patients who fail to receive recommended care.
Implications for Policy, Delivery or Practice: Compared to
established guidelines, there appears to be both under- and
over-use of surveillance care among cancer survivors. Whether
there are clinical reasons for such variation, and the cost and
health implications of them remains to be understood.
Primary Funding Source: NCI
• Colorectal Cancer Screening in Primary Care
Jennifer Elston Lafata, Ph.D., George Divine, Ph.D., Tamir BenMenachem, M.D., M.S., L. Keoki Williams, M.D., M.P.H.,
Christina Moon, M.A.
Presented by: Jennifer Elston Lafata, Ph.D., Director, Center
for Health Services Research, Henry Ford Health System, 1
Ford Place, 3A, Detroit, MI 48202; Tel: 313.874.5454; Fax:
313.874.1883; E-mail: jlafata1@hfhs.org
Research Objective: Despite evidence based
recommendations for colorectal cancer (CRC) screening,
reported screening rates remain low and the factors
associated with them are not well understood. Using
automated data available within an integrated delivery system,
we report 5-year rates of CRC screening among a primary care
(PC)population as well as evaluate the factors associated with
screening use.
Study Design: Retrospective cohort analysis in which
automated clinical and administrative data available within a
large Midwestern integrated delivery system were used to
determine patients’ 5-year CRC screening receipt, sociodemographic characteristics, PC visit history, comorbidities
(using the Deyo adaptation of the Charlson Comorbidity
Index), and PC physician and clinic affiliation. Per evidence
based guidelines, CRC screeners were defined as those
receiving at least 1 sigmoidoscopy, colonoscopy, or barium
enema, or 3 or more fecal occult blood tests (FOBTs) within
the prior 5 years. Generalized estimating equation (GEE)
approaches were used to test for adjusted and unadjusted
associations with screening receipt.
Population Studied: Patients aged 55-70 with an office visit to
one of 23 PC clinics in 2002. Eligible patients were those
without history of CRC or related disease, and continuously
enrolled in an affiliated health plan for the 5-year period ending
12/31/2002 (N=8,256).
Principal Findings: Average age of the cohort was 61 years;
59% were female, 34% African American, and 72% married.
In 2002, 48% of the cohort met the criteria for CRC screening
receipt. 6.4% met this criterion by having had a colonoscopy,
17.1% by sigmoidoscopy, 1.9% by barium enema, 8.4% by 3
FOBTs, and 14.2% by using a combination of these services.
Although we found the proportion of the cohort receiving
FOBT to have increased steadily over time (from 16.8% in
1998 to 23.2% in 2002), on average individuals received <1
test during the 5-year period. In fact, only 19.0% of the cohort
received 3+ FOBTs during the period.
In models that accounted for the non-independence of
patients receiving care from the same PC physician and that
adjusted for patient age, sex, race, marital status, income, PC
visit frequency, Charlson comorbidity index, and clinic
affiliation, we found CRC screening was significantly (p<0.05)
associated with increasing age (OR=1.03), being married
(OR=1.15), household income (OR=1.07 per $10,000), PC
visits (OR=1.06), and decreasing Charlson score (OR=0.98),
as well as with the clinic in which care was received.
Conclusions: Even among an insured population, less than
half of patients presenting in a PC clinic had been screened for
CRC. In particular, we found younger, unmarried persons with
relatively more comorbidities and infrequent PC visits to be
less likely than their counterparts to have been screened. We
also found those residing in low income neighborhoods to be
less likely to have been screened.
Implications for Policy, Delivery or Practice: Addressing the
CRC screening needs of both individuals with potentially
competing comorbidities and those with relatively limited PC
contact presents a distinct challenge, particularly without
overwhelming already busy PC practices.
• Educating Patients about Health Care Safety: An Analysis
of Campaign Messages
Vikki Entwistle, M.A., M.Sc., Ph.D., Troyen Brennan, M.D., JD,
M.P.H., Michelle Mello, JD, Ph.D., David Studdert, LLD, ScD,
M.P.H.
Presented by: Vikki Entwistle, M.A., M.Sc., Ph.D., Harkness
Fellow in Health Policy, Health Policy and Management,
Harvard School of Public Health, 677 Huntington Avenue,
Kresge 429, Boston, MA 02115; Tel: 617.432.6448; Fax:
617.432. 4494; E-mail: ventwist@hsph.harvard.edu
Research Objective: Numerous campaigns have been
launched to raise public awareness about the problems of
health care errors and to advise patients about efforts to
improve the safety of their health care. We set out to identify
the various roles advocated for patients within these
campaigns, and to investigate the practical and ethical
implications of these.
Study Design: 1. Analysis of the content of educational
materials developed for national campaigns advising patients
about safety in health care.
2. Key informant interviews with people responsible for the
development, distribution or evaluation of campaign
materials.
For each campaign, we documented a) how the problem of
patient safety was introduced, b) which efforts to improve
patient safety were described, and c) what actions patients
were encouraged to take in what kinds of health care
circumstances. We considered how and to what extent
patients’ adoption of each action might impact upon the
safety and other aspects of the quality of their care, and what
system features and/or provider behaviours would be needed
to render patients’ actions effective.
Population Studied: 1. English language versions of leaflets,
newsletters and posters associated with campaigns from
federal agencies (AHRQ, FDA CDER, VHA) and other
nationally active organisations (ISMP, JCAHO, NPSF).
2. Staff members from the above agencies and organisations
(these interviews will be ongoing through April 2004).
Principal Findings: Our preliminary analyses indicate that:
The campaign materials introduce the ‘problem’ of health care
errors in various ways. They suggest various actions that
patients can take to make their health care safer, but
sometimes without explaining why, how or in what
circumstances these actions might be effective. The following
types of role are explicitly or implicitly advocated for patients
within at least some of the materials studied:
1.
Select a high quality health care provider
2.
Communicate to ensure that treatment decisions are
based on accurate, up-to-date assessments of your health
problems and current treatment
3.
Communicate to facilitate co-ordination of care from
different providers
4.
Co-operate with health care providers’ safety
improvement efforts
5.
Encourage health care providers to adopt safety
improving behaviours
6.
Watch for technical problems or errors in the delivery
of planned care and intervene before they cause harm
7.
Monitor your health status and the effects of
interventions
8.
Provide safe self-care
9.
Ensure you are adequately informed to carry out the
above functions.
The specific actions that are recommended imply various
degrees and senses of responsibility for patients. There have
been few empirical investigations of patients’ responses to the
campaign messages.
Conclusions: National campaigns encourage patients to play
various roles to enhance their safety, but the effects of their
messages are poorly understood.
Implications for Policy, Delivery or Practice: Attention
should be paid to patients’ perceptions of the roles they and
others should be asked to play to help ensure the safety of
their health care. The ethical issues arising from these
campaigns warrant further debate.
Primary Funding Source: CWF, Chief Scientist Office,
Scottish Executive Health Department
• Oregon Patient Safety Efforts: Lessons Learned
Mary Ann Evans, Ph.D., M.P.H., Jim Dameron, MA, MSA, Joel
Young, M.A., Grant Higginson, M.D., M.P.H.
Presented by: Mary Ann Evans, Ph.D., M.P.H., Lead Research
Analyst, Health Systems Planning Office, Oregon Department
of Human Services, 800 NE Oregon Street, Portland, OR
97232; Tel: 503.731.4017; Fax: 503.731.4078; E-mail:
maryann.evans@state.or.us
Research Objective: Oregon Patient Safety Efforts: Lessons
Learned
It’s been four years since the IOM published the seminal work
on medical errors and patient safety. States responded in a
variety of ways to the call to increase patient safety.
Oregon passed legislation establishing the Patient Safety
Commission in August 2003. The Commission is a semiindependent state agency charged with improving patient
safety by reducing medical errors. The Commission is
authorized to establish a confidential, voluntary, nonpunitive,
adverse even reporting system for the state. The reporting
system will allow participating organizations to share
information about errors and learn from each other’s mistakes
and successes. The Commission is funded through fees paid
by hospitals, birthing centers, pharmacies, ambulatory surgery
centers, and nursing facilities. While entities can choose
whether or not to participate in reporting, all are required to
contribute funding. The state provides in-kind contributions.
The purpose of our paper is to briefly describe the process
that helped bring about the patient safety legislation in the
hope that others can translate the lessons we learned.
Lesson #1: Involve many stakeholders
Many Oregon agencies have roles in improving safety
including: Department of Human Services, Boards of Medical
Examiners, Nursing, and Pharmacy, and Department of
Consumer and Business Services. In September 2002, the
Health Systems Planning office brought together government
agencies along with health care providers, insurers,
consumers, health care purchasers, and university faculty to
discuss reducing medical errors and improving patient safety.
Over the course of several months, the group crafted Oregon
House Bill 2349 establishing the Oregon Patient Safety
Commission.
Lesson #2: Work with political partners and seek creative
solutions
HB 2349 originally proposed a patient safety corporation
rather than a commission. When it became clear that a
corporation was not viable, negotiations between the governor
and stakeholders resulted in the creation of a commission.
Lesson #3: Leverage successful activities to attract partners
and funding
The following activities are a direct result of the Patient Safety
legislation:
Patient Safety Workshop. In collaboration with AHRQ and
the National Academy for State Health Policy, Oregon is
convening state and national experts to discuss patient safety
and to consider ways to help launch the Oregon Patient Safety
Commission. Attendance at the workshop includes key health
care organizations, purchasers, and patient safety
stakeholders. The workshop is co-sponsored by: Kaiser
Permanente, Oregon Association of Hospitals and Health
Systems, Oregon Coalition of Health Care Purchasers, Oregon
Health and Science University, Oregon Healthcare
Association, Oregon Medical Association, Oregon Nurses
Association, Oregon State Pharmacists Association, and
Regence BlueCross BlueShield of Oregon.
Federal Challenge Grant. Oregon was the only state agency in
the country to win a competitive challenge grant from AHRQ
in this year’s grant cycle. The goal of our project is to develop
new approaches for reducing medication errors in long-term
care and assisted living facilities in partnership with key
stakeholders.
Patient Safety Improvement Corps. Oregon has a team that is
working with the Veterans Administration and AHRQ to
transfer best patient safety practices to Oregon.
Primary Funding Source: AHRQ,
• Can Admission Rates for Ambulatory Care Sensitive
Conditions Be Used to Screen for Quality of Care
Problems in the California Medicare HMO Population?
Feng Zeng, Ph.D., Nasreen Dhanani, Ph.D., Elizabeth Sloss,
Ph.D., June O’Leary, Ph.D., Glenn Melnick, Ph.D.
Presented by: Feng Zeng, Ph.D., Doctoral student, RAND,
1700 Main Street, PO Box 2138, Santa Monica, CA 90407-2138;
telephone (310) 393-0411 ext 7320; fax (310) 393-4818 or (310)
451-7061 (e-mail: feng_zeng@rand.org).
Research Objective: To estimate rates of admission for
several ambulatory care sensitive conditions for the entire
California Medicare HMO population and by plan.
Study Design: Given timely and appropriate outpatient care,
admissions for ambulatory care sensitive conditions (ACSCs)
are largely considered preventable. As a result, rates of
admission for ACSCs are being used as quality of care
screening tools. Rates for 15 ACSCs were estimated: (1)
asthma/chronic obstructive pulmonary disease, (2) congestive
heart failure, (3) seizure disorder, (4) diabetes mellitus, (5)
hypertension, (6) hypoglycemia, (7) urinary tract infections,
(8) cellulitis, (9) dehydration, (10) hypokalemia, (11) gastric or
duodenal ulcer, (12) bacterial pneumonia, (13) severe
ear/nose/throat infections, (14) influenza, and (15)
malnutrition. While many more ACSCs have been identified,
these 15 conditions were selected because of their relevance to
the over age-65 population. This research compares ACSCs of
different HMOs and studies whether the differences of ACSCs
across HMOs can be explained by age and sex differences.
Population Studied: California Medicare enrollees
continuously enrolled in the same HMO from January 1996
through December 1996.
Principal Findings: Among a total of 853,494 FTE
beneficiaries continuously enrolled in an HMO in 1996, there
were 34,196 admissions for ACSCs. Depending on age and
gender, rates of admission for the 15 ACSCs combined ranged
from 0.06 to 10.57 admissions per 1,000 FTE enrollees. The
largest total number of discharges was found for
asthma/chronic obstructive pulmonary disease (4,380),
congestive heart failure (9,104), urinary tract infection (2,995),
and bacterial pneumonia (8,628). Male age 85 and above had
the highest rates of ACSC admissions (107.6 per thousand
enrollees).
The 853,494 beneficiaries were enrolled in 175 HMOs. Of the
175 HMOs, only 27 HMOs had more than 1000 beneficiaries.
The 27 HMOs are responsible for more than 99% of the total
beneficiaries and ACSCs. Overall, within HMOs, the rates of
admission for acute and chronic conditions are highly
correlated (0.91) but vary by individual ACSCs: Correlations
among admission rates for asthma/chronic obstructive
pulmonary disease, congestive heart failure, urinary tract
infection, and bacteria pneumonia range between 0.42 and
0.82. Across HMOs, there were large differences in
admission rates across conditions. These differences remain
even after adjusting for age and sex.
Conclusions: Correlations of ACSC admission rates within
and across HMO plans may provide the first step to
monitoring quality of care in Medicare HMOs and identify
plans with potential problems. Comparing rates across
HMOs provides better benchmarking than comparing to Feefor-Service rates without adjustment for selection.
Implications for Policy, Delivery or Practice: Rates of
admission for ambulatory care sensitive conditions have been
developed as screening tools to identify quality of care
problems. The California Medicare HMO population serves
as an important test case to determine the ability of these
measures to meet their intended purpose. It may be
important for policy makers to monitor the ACS rates for
HMO plans to control for the quality of care. More research is
needed to understand the differences of ACSC rates across
different HMOs.
Primary Funding Source: Robert Wood Johnson Foundation
• Disparities in Glycemic Control for Patients with Mental
Illness: Protective Effect of Primary Care
Susan Frayne, M.D., M.P.H., Hai Lin, M.D., Jewell Halanych,
M.D., M.S., Fei Wang, Ph.D., Dan Berlowitz, M.D., M.P.H.,
Katherine Skinner, Ph.D., Erica Sharkansky, Ph.D., Terence
Keane, Ph.D., Leonard Pogach, M.D., Donald Miller, Sc.D.
Presented by: Susan Frayne, M.D., M.P.H.,Center for Health
Care Evaluation, VA Palo Alto Health Care System, 795 Willow
Road, Menlo Park, CA 94025; Tel: 650.493.5000; Fax:
650.617.2690; E-mail: sfrayne@stanford.edu
Research Objective: Recent studies have begun to document
that patients with mental health conditions (MHC) may
receive less intensive care for medical conditions. We
examined whether disparities in glycemic control were seen
for diabetics with vs. without MHCs, and if so, whether
enrollment in primary care had a protective effect.
Study Design: Using Veterans Health Administration (VA)
administrative files and FY99 lab data from a central VA
registry, we compared diabetics with vs. without MHC on the
Diabetes Quality Improvement Project measure of poor
glycemic control (HbA1c greater than or equal to 9.5 or test
not done) in univariate and multivariable analyses (adjusting
for age, sex, race and physical comorbidities, in aggregate and
in several major MHCs). We then stratified patients based on
whether VA records indicated they had received primary care,
and repeated multivariable analyses.
Population Studied: National sample of all 339,559 VA
patients with diabetes (based on ICD9 codes and antiglycemic
prescriptions) whose facility submitted lab data to a central
registry; based on ICD9 codes in FY97-98 VA administrative
data, 83,473 had any MHC, 52,839 depression, 19,977
substance abuse, 17,238 psychosis, and 6,050 personality
disorder. Of those with and without any MHC, 87% and 86%,
respectively, were enrolled in VA primary care.
Principal Findings: Comparing diabetics with versus without
MHC, 29% vs. 24% had poor glycemic control (p < .0001).
With poor glycemic control as the outcome and patients with
no MHC as the reference group, adjusted odds ratio for any
MHC was 1.19 (1.17, 1.21), for depression was 1.19 (1.16, 1.22),
for substance abuse was 1.37 (1.32, 1.42), for psychosis was
1.26 (1.21, 1.31), and for personality disorder was 1.38 (1.30,
1.46). Disparities were exaggerated among those not
receiving primary care: for any MHC, adjusted odds ratio was
1.17 (1.14, 1.19) for those receiving primary care, and 1.34 (1.25,
1.45) for those not receiving primary care. The effect was even
more striking for those with substance use disorders: adjusted
odds ratio was 1.32 (1.27, 1.37) for those enrolled in primary
care, and 1.63 (1.45, 1.84) for those who were not.
Conclusions: Patients with diabetes and MHC are more likely
than those without MHC to have poor glycemic control;
disparities persist after adjusting for demographics and
comorbidity, and are greater for certain specific MHCs, such
as substance abuse, psychosis and personality disorders.
While patient-level or provider-level barriers may contribute to
this effect, the observation that disparities are exaggerated for
patients with MHC who are not enrolled in primary care
suggests that system-level factors may also play a role,
especially for those with substance use disorders.
Implications for Policy, Delivery or Practice: National
diabetes quality improvement efforts should pay special
attention to the mentally ill, especially those with substance
abuse, psychosis or personality disorders. Systems-level
interventions, such as efforts to improve access to primary
care for the mentally ill, may advance efforts to optimize care
for patients with comorbid medical and mental illness.
Primary Funding Source: VA
• Enhanced Post Marketing Surveillance Approach to Drug
Adverse Event Reporting: The Case of Lipid Lowering
Drugs
Karen Friery, B.S., Beatrice Golomb, M.D., Ph.D.
Presented by: Karen Friery, B.S., Medicine, University of
California, San Diego, 3366 North Torrey Pines Court, Suite
110, La Jolla, CA 92037-1025; Tel: 858.558.4950; Fax:
858.558.4960; E-mail: kfriery@ucsd.edu
Research Objective: : To develop a novel Post-Marketing
Surveillance (PMS) approach for drug Adverse Events (AEs),
targeted to patients, with the aim to reduce time to detection
of potential new AEs, and increase understanding of AE
characteristics. Lipid lowering drugs served as the
demonstration case.
Study Design: A pilot PMS survey project was designed to 1)
solicit AE reporting directly from patients, who may have time
and interest in completing surveys related to their AEs; and 2)
elicit expanded information, including (but not limited to)
timecourse to AE onset after initiation (initially and on
rechallenge), to AE recovery after discontinuation,
completeness of recovery , AE co-occurrence, quality of life
impact, physician response, other AE characteristics, and
factors used in assessment of drug AE causality. Medical
records were used for validation in a subset.
Population Studied: Persons with AEs on lipid-lowering
drugs recruited through doctor referrals, media-reports, and
the internet.
Principal Findings: Patient-targeted recruitment was
successful: 450 persons completed IRB-approved PMS
surveys in this pilot effort. Additionally, this study validated the
ability of this approach to provide pivotal information. First,
new information was secured regarding known AEs. E.g. for
myopathy, data on timecourse showed that even late-onset
reported AEs were drug related (as these too abated off drug
and recurred with rechallenge – with a significantly shorter
mean onset time on reinitiation, p < .01); and a dose-response
relationship was upheld: 95% of those resuming an equal or
higher potency drug/dose combination experienced
recurrence (providing potent internal validation for the
approach), while only 55% who rechallenged with a lower
potency drug had recurrence, suggesting that the potency of
the drug – rather than factors like lipophilicity – primarily
determined AE occurrence in susceptible individuals.
Information on physician response, and medical care given,
when patients presented with presumptive AEs, yielded
insight on physician recognition of known AEs and on health
care utilization in the setting of unrecognized AEs. Second,
several new presumptive AEs were indeed identified that met
drug AE causality criteria. (Two have been subsequently
corroborated in other classes of study.) Furthermore, strong
correlations among certain AEs led to testable hypotheses
regarding common mechanisms.
Conclusions: Targeting patients was an effective approach to
securing PMS data. An enhanced PMS survey design was
verified to provide vital new and timely information of direct
relevance to patients and physicians, and to accelerate
identification of presumptive new AEs that met specific
causality criteria.
Implications for Policy, Delivery or Practice: Expansion of
this approach through internet-based recruitment, and to
other classes of drugs, may accelerate identification of
unacknowledged drug AEs for all classes of drugs, thereby
potentially reducing patient suffering and speeding
information accrual for policy determinations regarding black
box warnings and drug withdrawals from the market (where
relevant). Information on AE characteristics, timecourse, doseresponse, and recovery may enhance clinical practice by
improving information used in physician decision making.
Findings specific to lipid-lowering drugs are timely in light of
recent reconsideration of over-the-counter (OTC) status for
statins; and continued consideration of statins for an
expanded set of indications.
Primary Funding Source: RWJF
• A Life Stage Perspective on Women’s Primary Care:
Patterns of Provider Use and Satisfaction with Care
Jillian Henderson, M.P.H., Ph.D.
Presented by: Jillian Henderson, M.P.H., Ph.D., Postdoctoral
Fellow, Center for Reproductive Health Research & Policy,
University of California, San Francisco, 3333 California Street,
Suite 335, San Francisco, CA 94118; Tel: 415.502.8544; Fax:
415.502.8479; E-mail: hendersonj@obgyn.ucsf.edu
Research Objective: Women’s access to primary health care
that meets a range of needs – both reproductive and nonreproductive and across the life span - is an explicit goal of
quality improvement efforts. Women’s primary health care
has been described as fragmented because many women see
both a generalist and a reproductive health specialist for
primary health care. There is evidence that women seeing
both types of providers receive more recommended clinical
preventive services (screening and counseling), but less is
known about the implications of this use pattern for
satisfaction. The purpose of this study was to investigate
women’s pattern’s of provider use at different stages of life
and how patterns of use influence satisfaction with primary
care.
Study Design: The study is based on secondary analysis of
data from the 2001 Measurement of Women’s Satisfaction
with Primary Care project. Adult women making routine
health care visits to clinical sites affiliated with three academic
health centers were invited to participate in the study.
Participants completed questionnaires prior to and
immediately following a primary health care visit. Bivariate
and multivariate analyses were conducted to examine
relationships between patterns of provider use and
satisfaction with care among women at different stages of life.
A new patient satisfaction tool validated for women’s primary
care provides the dependent variables.
Population Studied: The sample consists of 1,202 adult
woman ages 18 to 87. The following four life stages were
defined based on theoretical, empirical, and data
considerations: early reproductive years (18 to 34), later
reproductive years (35 to 44), peri-menopause (45 to 55), and
post-menopause (56 to 87).
Principal Findings: Over one-third of women in all stages of
life reported seeing both a generalist provider and a
reproductive health specialist in the past year, and women
who saw both were less satisfied with the coordination and
comprehensiveness of their primary care. For women in the
early reproductive life stage, satisfaction with care was highest
when only a reproductive specialist was seen; for women in
the later reproductive years, seeing a single provider for
primary care was associated with greater satisfaction.
Conclusions: The organization of women’s primary health
care fosters diverse provider use patterns that have different
implications for satisfaction depending women’s life stages.
The current study points to disadvantages to this pattern of
use from the perspective of women in all life stages, which
results in lower patient satisfaction.
Implications for Policy, Delivery or Practice: Improvements
in physician education and in the organization and delivery of
women’s primary care are needed to better serve the interests
and needs of women at all stages of life. To avoid lower
satisfaction ratings associated with seeing two primary care
providers, physicians or teams that are able to address
women’s reproductive and non-reproductive health issues in a
coordinated fashion should be encouraged in the policy and
health care management arenas.
Primary Funding Source: NIA
• Cardiovascular Risk Factor Control and Treatment
Patterns in Primary Care
Katharine Hendrix, Ph.D., Katharine Hendrix, Ph.D., Daniel
Lackland, Dr.P.H., Brent Egan, M.D.
Presented by: Katharine Hendrix, Ph.D., Executive
Administrator, Hypertension Initiative of South Carolina,
General Internal Medicine, Medical University of South
Carolina, 96 Jonathan Lucas Street, 826 CSB, Charleston, SC
29425; Tel: 843.792.6340; Fax: 843.792.0816; E-mail:
hendrikh@musc.edu
Research Objective: The objectives of this study were to
assess CV risk factor treatment, control and disparities among
patients with hypertension.
Study Design: A descriptive analysis from a retrospective
medical record audit of data from a large medical record
database.
Population Studied: 35,940 hypertensive patients (>70
primary care sites with >250 providers) in the Southeastern
United States.
Principal Findings: In 50% of 35,940 patients with
hypertension, the last blood pressure reading was less than
140/90 mm Hg. In 62% of 18,627 patients who also had
dyslipidemia, low-density lipoprotein cholesterol (LDL-C) was
<130 mg/dL. In 49% of 6,616 patients with hypertension and
diabetes, glycosylated hemoglobin levels were less than 7%.
Multiple risk factor control was rare, especially among women
and African Americans.
Conclusions: It appears that programs to improve CV risk
factor control using audit and feedback in primary care are
feasible and instructive.
Implications for Policy, Delivery or Practice: Non-invasive
auditing of primary care practice medical records is feasible
and provides a useful source from which feedback can be
generated to encourage evidence-based medical practices and
compliance with national guidelines for treatment of specific
disease states among participating health care providers.
Primary Funding Source: Duke Endowment
• Evaluating Potential Mode Effects in the Group-Level
Consumer Assessment of Health Plans Survey (GCAHPS®)
Kimberly Hepner, Ph.D., Julie Brown, B.A., Ron Hays, Ph.D.
Presented by: Kimberly Hepner, Ph.D., Staff Researcher,
Health, RAND Corporation, 1700 Main Street, Santa Monica,
CA 90407; Tel: 310.393.0411 Ext. 6831; E-mail:
hepner@rand.org
Research Objective: The G-CAHPS® survey focuses on
patient experiences in receiving care from their medical group
practice and has not been previously evaluated for
comparability across methods of administration. To evaluate
the presence of mode effects, we compared mail and
telephone responses to the G-CAHPS® survey in a sample of
patients from physician groups in California.
Study Design: Patients were assigned to either mail or
telephone administration of the G-CAHPS® survey. Patients
assigned to mail administration were contacted by phone if
they did not complete the instrument by mail. The
randomized group comparison included patients who
responded using the mode to which they were randomly
assigned. Because patients who were randomized to the mail
mode could eventually respond by phone, we also conducted
a respondent group comparison that focused on the mode to
which the patient actually responded. Analyses included
response rates, rates of missing data, internal consistency of
multi-item composites, and mean scores.
Population Studied: A random sample of 400 patients were
selected from each of 4 physician groups: two multi-specialty
group practices, one safety net provider, and one independent
practice association (IPA) that included multiple physician
practices.
Principal Findings: A total of 777 completes were obtained
through the mode to which they were randomized (47%
response rate; 343 to mail, 434 to phone). Including those
study participants who were randomized to mail mode first
but completed a telephone interview increased the number of
phone completes to 537 and overall sample size to 880 (54%
response rate). Both the randomized group comparison and
the respondent group comparison yielded similar proportions
of inappropriately missing data. After adjusting for multiple
comparisons, there were no significant differences in internal
consistency by mode for any of the six composites (wait times;
access to needed care; communication between doctors and
patients; coordination between primary care physicians and
specialists; courtesy and respect shown by the office staff, and
advice on preventive health) for either the randomized
comparison or the respondent comparison. Item and
composite means were compared by administration mode
using a series of t-tests. After adjustment for age, language of
survey (English versus Spanish), Hispanic ethnicity, selfreported overall quality of life, and rating of health care in the
United States there were no significant associations of mode
with reports or ratings of care.
Conclusions: Rates of missing data and estimates of internal
consistency reliability were indistinguishable by mode for both
methods of comparison. All sixteen report items, all six
composite items, and two of three global ratings did not differ
by mode of administration. Results suggest that the GCAHPS® survey provides comparable results by telephone
and mail.
Implications for Policy, Delivery or Practice: The GCAHPS® survey provides comparable results whether
administered by mail or telephone.
Primary Funding Source: AHRQ
• Patient Safety Risks in Clinician Communication During
Hospital-Based Pediatric Care
Jane Holl, M.D., M.P.H., Donna Woods, Ph.D., Lori Fewster,
RN, M.S., Edward Ogata, M.D., MM, Gregory Makoul, Ph.D.,
Kevin Weiss, M.D., M.P.H.
Presented by: Jane Holl, M.D., M.P.H., Faculty Associate and
Assistant Professor of Pediatrics and Preventive Medicine,
Institute for Health Services Research and Policy Studies,
Feinberg School of Medicine, Northwestern University, 339 E.
Chicago Avenue, Room 713, Chicago, IL 60611-3071; Tel:
312.503.0392; Fax: 312.503.2936; E-mail:
j-holl@northwestern.edu
Research Objective: To describe effective and problematic
clinician communication (communication between clinicians
providing care for a patient) in hospital-based pediatric care
and to identify, for problematic communication, the
determinant characteristics related to patient safety risk.
Study Design: Clinicians were invited by an Ad Hoc Patient
Safety Committee of a children’s hospital to participate in a
focus group. Focus groups, within each discipline (neurology,
neurosurgery, surgery, intensive care unit (ICU), and
emergency medicine), were convened by profession and
professional level (nurse managers, staff nurses, attending
physicians, fellow/ resident physicians and advanced practice
nurses, nurse administrative coordinators). A standardized
protocol, to assess the main means of communication (i.e.,
in-person, telephone, medical chart), to elicit discussion about
effective and problematic communications, and to identify the
characteristics of the communications, in particular those
related to patient safety risk, was used. Focus group
participants provided verbal consent and remained
anonymous. The 90-minute focus groups were audio-taped
and then, transcribed. Three investigators independently
analyzed the data using a transcript-based approach to
identify emerging themes and then, triangulate the findings to
reach consensus. Basic characteristics (where, when, how,
between whom, and what) were identified for all reported
communications. Determinant characteristics for either
effective or problematic communication with patient safety
risk were sought.
Population Studied: Clinicians in a children’s hospital.
Principal Findings: Twenty focus groups were convened and
included 65 clinicians with 2-7 participants per
group. Daytime, in-person, “team” clinical communications
(e.g. ward rounds, conferences) were reported as effective;
those that included full participation of the entire team were
described as most effective. All participants described
problematic communication. Problematic communications
with identified patient safety risks occurred: (1) during
transitions (patient transfers between services or units,
changes in attending/fellow/resident “coverage”, nursing shift
changes); (2) in the scheduling of and getting studies and
procedures performed; (3) for patients with complex
medical/surgical conditions being cared for by multiple
services; (4) for ICU patients managed by clinicians not
primarily located in the ICU; (5) with non sub-specialized
pediatric clinicians (nurses and physicians); and (6) with nonpediatric trained clincians, particularly surgical clinicians,
about medical management.
Conclusions: This research provides some initial information
about the characteristics of effective and problematic clinician
communications in pediatric hospital-based medical care. Six
distinct contexts were related to problematic clinician
communications with identified patient safety risks.
Implications for Policy, Delivery or Practice: Determinant
characteristics of clinical communication that contribute to
effective (i.e. daytime, team rounds) or problematic
communication (i.e. transitions, non-pediatric trained
clinicians) have been identified and should be useful in
developing interventions to improve clinician communication.
Further understanding of the contributions of additional
factors such as knowledge, training, experience, resources,
and organizational and spatial configuration is needed to
reduce problematic clinical communication.
Primary Funding Source: Children’s Memorial Institute for
Evaluation and Research, Children’s Memorial Hospital,
Chicago, IL
• The Call for Prescribing Safety in Ambulatory Care:
Physician Responses and IT Solutions
John Hsu, M.D., M.B.A., MSCE, Thomas Rundall, Ph.D.,
Connie Uratsu, BA, Stephen Soumerai, M.S.P.H., ScD,
Jennifer Elston Lafata, Ph.D., Jan Simpkins, M.A.
Presented by: John Hsu, M.D., M.B.A., MSCE, Division of
Research, Kaiser Permanente, 2000 Broadway, Oakland, CA
94612; Tel: 510.891.3601; Fax: 510.891.3606; E-mail:
jth@dor.kaiser.org
Research Objective: The majority of drug prescriptions occur
in ambulatory care. Little is known about physicians’ views
regarding prescribing safety in this setting. We investigated
physician perceptions of existing problems and potential
methods for improving prescribing safety.
Study Design: We conducted semi-structured interviews
among a purposeful sample of practicing physicians from
three integrated health delivery systems. We asked about
perceptions of five pre-specified ambulatory prescribing safety
issues, other clinically relevant issues, current safety practices,
and recommended approaches for improving prescribing
safety. The five pre-specified safety issues focused on three
general safety areas: prescribing in the elderly, lab monitoring,
and co-prescribing contraindicated drugs. Using a content
analysis approach, three researchers independently coded
responses; one investigator collapsed the coded data into
themes.
Population Studied: All 17 subjects were internists (59%) or
family practitioners (41%) in one of three large integrated
delivery systems located in Michigan, New Mexico, and
California. These subjects tended to be male (59%), hold
positions of leadership within the health system (65%),
participate on committees within the clinic or health system
(53%), and provide outpatient care only (71%). On average,
the physicians interviewed had practiced for 14 years, spent 12
years with their current health system, and devoted
approximately 84% of their workweek to direct patient care.
Principal Findings: Physicians cited a number of safety
concerns including potential adverse events associated with
drug-drug interactions, allergies, and laboratory monitoring.
Physicians also described a variety of potential solutions to
improve safety, including physician education (28 mentions),
patient education (10 mentions), and creating more accessible
prescribing guidelines (10 mentions). Information technology
(IT)-related solutions received the most mentions (40) across
the five pre-specified safety issues; 26 of these mentions were
related to lab monitoring. Many physicians perceived
deficiencies with their health systems’ current technology. For
example, one physician stated, “It is unconscionable that we
don’t have a medication list in [the electronic medical record
(EMR)].” Another noted, “We don’t have a tickler/reminder
system here yet…it’s being developed but has not been
delivered yet.” As such, the IT solutions suggested ranged
from more passive informational systems, such as a more
comprehensive EMR with complete medication and
conditions history, to more interactive alert systems that
automatically flag and notify physicians or other clinicians of
potential errors or safety issues.
Conclusions: Physicians perceive significant problems with
ambulatory prescribing safety such as limited use of
recommended laboratory monitoring for prescription drugs.
Physicians offered a number of potential solutions, especially
centered on improvements in information technology, and
also noted that many current systems are not adequate for
ensuring safety.
Implications for Policy, Delivery or Practice: Physicians
appear receptive to improving current ambulatory prescribing
practices and enthusiastic about technology-related
prescribing aids. Physicians also report that previous solutions
may not have been appropriately designed. Further research is
necessary to investigate how well individual solutions address
the underlying causes of prescribing problems, and whether
they improve prescribing safety.
Primary Funding Source: AHRQ
• Assessments of Hospital Performance Based on Death
Rate Comparisons Change when Risk Adjustment is
Limited to Diagnoses Present on Admission
John Hughes, M.D., Richard Averill, M.S., Elizabeth
McCullough, M.S., Jean Xiang, M.S.
Presented by: John Hughes, M.D., Medicine, Yale University
School of Medicine, 68 West Rock Avenue, New Haven, CT
06515; Tel: 203.387.9798; E-mail: jshughes@mmm.com
Research Objective: In order to evaluate a hospitalized
patient’s baseline condition more accurately, the state of
California requires for each diagnosis on the computerized
discharge abstract an indicator as to whether the diagnosis
was present on admission or arose after admission. We
hypothesized that evaluations of hospital performance based
on differences between observed and expected death rates
would differ if risk-adjustment was based only on those
diagnoses present on admission rather than all diagnoses
listed in the discharge abstract.
Study Design: We calculated expected death rates for
California hospitals using the All-Patient Refined-Diagnosis
Related Groups (APR-DRG) patient classification system,
which assigns risk categories based on reason for admission
and risk of mortality. We calculated expected death rates using
data from all eligible California Hospitals by means of indirect
standardization. We first calculated expected death rates
based on all diagnoses listed on the discharge abstract. We
then calculated a second set of expected death rates based
only on diagnoses that were present on admission. For both
methods, we ranked all eligible hospitals by the magnitude of
the percentage difference between observed and expected
death rates, calculated as (actual death rate – expected death
rate)/ expected deaths rate. We report the differences in
hospital rankings that resulted from the two methods.
Population Studied: We analyzed discharge abstract data for
5.2 million admissions to all California hospitals for 1999 and
2000. We eliminated 88 hospitals that had not recorded the
present on admission indicator accurately or consistently, and
eliminated another 83 hospitals with fewer than 1,000
admissions or fewer than 30 deaths, leaving a total of 257
hospitals and 3.90 million admissions.
Principal Findings: Overall actual hospital death rates ranged
from 0.53% to 6.96%. Deviation from expected death rates
based on APR-DRGs and calculated using all discharge
diagnoses ranged from 66% lower than expectd to 44%
higher. Using only discharge diagnoses present on admission,
observed death rates ranged from 64% lower to 68% higher
than expected death rates.
For many hospitals there was a substantial disparity in the
rank order of observed and expected death rate differences
assigned by the two methods. 57 hospitals had at least a 10%
difference in rank order assigned by the two methods. Of 26
hospitals ranked in the bottom decile using all diagnoses, 9
moved up at least 1 decile when only diagnoses present on
admission were used to generate the expected death rates; 5
of 26 dropped out the top decile when only diagnoses present
on admission were used.
Conclusions: Hospital ranking based on differences in
observed versus expected death rates can vary depending on
whether risk adjustment uses all discharge diagnoses or is
limited to those diagnoses present on admission.
Implications for Policy, Delivery or Practice: The ability to
identify diagnoses present on admission allows for more
accurate rating of patient baseline risk when comparing
hospital performance. It should also prove useful when
comparing rates of in-hospital complications.
Primary Funding Source: 3M Health Information Systems
• Assessing ACSC Hospitalizations Among Fee-for-Service
and Managed Care Medicaid Beneficiaries: a Multi-State
Analysis
Ronda Hughes, Ph.D., M.H.S., R.N., Renee Menetnech,
MHS, R.N., Helen Burstin, M.D., M.P.H.
Presented by: Ronda Hughes, Ph.D., M.H.S., R.N., Health
Scientist Administrator, Center for Primary Care, Prevention,
and Clinical Partnerships, Agency for Healthcare Research and
Quality, 540 Gaither Road, Rockville, MD 20852; Tel:
301.427.1578; Fax: 301.427.1595; E-mail: rhughes@ahrq.gov
Research Objective: This 13 state study evaluates the impact
of Medicaid managed care on resource utilization among
adult beneficiaries by type of Medicaid eligibility. It is
hypothesized that as patients move into managed care
arrangements, the clinical detail of the administrative data
diminishes, therefore limiting the ability to assess the quality
of care. This study uses State Medicaid Research Files
(SMRF) maintained by the Centers for Medicare and Medicaid
Services for 1996 through 1998, to assess outpatient and
inpatient utilization among all adult Medicaid beneficiaries,
age 18 and over. This includes patients that qualify for
Medicaid services under the former Aid for Families with
Dependent Children (now Section 1931 eligibility) income
provisions, the blind and disabled part including
Supplemental Security Income (SSI), and the aged provisions
of the Medicaid program.
Study Design: SMRF data was used to assess the diagnostic
information associated with specific health care services,
specifically rates of hospitalizations for ambulatory care
sensitive conditions (ACSC). The data were used to define
ACSC hospitalizations, marker conditions, and referral
sensitive conditions (based on the primary International
Classification of Diseases-Clinical Modification, Ninth
Revision codes (ICD-9-CM) for adult beneficiaries. These are
based on ICD-9-CM codes developed by Billings et al.
Beneficiaries were further stratified by: 1.) length of enrollment
in Medicaid (e.g., individuals enrolled in Medicaid 0 – 5
months, 6 to 12 months, and at least 12 continuous months);
2.) enrollment in either managed care or fee-for-service (FFS)
systems; and 3.) type of eligibility (e.g., Section 1931, blind and
disabled non-elderly, and elderly).
Population Studied: Adult Medicaid beneficiaries in 13 states.
Principal Findings: Results show that rates of ACSC
hospitalizations by type of Medicaid eligibility, and by state
range from 2.3 to 412.9 per 1,000 population. In comparing
states, on average there were no significant differences in
rates of ACSC hospitalizations between Medicaid enrollees in
managed care and enrollees in FFS. In some states with high
managed care penetration, certain managed care enrollees
had up to 48 percent fewer ACSC hospitalizations. Rates of
hospitalization for certain conditions, such as diabetes and
congestive heart failure, increased from 1996 to 1998.
Nonwhite, female, blind and disabled non-elderly enrollees
were more likely to have ACSC and referral-sensitive
admissions. Differences in rates of ACSC hospitalizations
were not significant among SSI enrollees, regardless if
patients were in FFS or managed care arrangements. Older
enrollees age 65 and over, were more likely to be hospitalized
for non-ACSC than were younger enrollees.
Conclusions: Our results reveal the presence of wide
disparities in rates of ACSC hospitalizations, particularly by
type of Medicaid enrollment and by state. Comparing rates of
ACSC hospitalizations may be useful in monitoring access to
quality primary care for among patients in managed care and
among states. Since Medicaid claims data is primarily used
for payment, there are limited used of the data for research
purposes.
Implications for Policy, Delivery or Practice: The
implications of these findings for health care policy and future
research will be discussed, including future research directions
for Medicaid claims files.
• Quality of Life of Newly Diagnosed Prostate Cancer
Patients
Ravishankar Jayadevappa, Ph.D., Sumedha Chhatre, Ph.D.,
Katarina Johnson, BA, Bernard Bloom, Ph.D., Bruce
Malkowicz, M.D.
Presented by: Ravishankar Jayadevappa, Ph.D., Research
Assistant Professor, Department of Medicine, University of
Pennsylvania, 224, Ralston-Penn Center, 3615 Chestnut Street,
Philadelphia, PA 19103-8684; Tel: 215.898.3798; Fax:
215.573.8684; E-mail: jravi@mail.med.upenn.edu
Research Objective: Quality of life has become an integral
part of cancer outcome research. Multiple factors
(demographic, clinical, social and economic) influence the
Health Related Quality of Life (HRQoL) and must be assessed
for effective management and treatment of diverse prostate
cancer (PC) patients. Little information is available regarding
effects of differential treatment patterns for ethnic or age
groups on quality of life of newly diagnosed PC patients. The
objective of this study is to analyze the variations in HRQoL of
newly diagnosed PC patients by ethnicity and age over a 3month follow-up.
Study Design: For this prospective study, we recruited 316
newly diagnosed PC patients from the urology clinics of an
urban academic hospital and Veterans Administration
hospital. Participants completed SF-36 and UCLA-PCI surveys
prior to their treatment, and at 3-month follow-up.
Demographics and HRQoL were compared across ethnicity
using t-test and chi-sq. Log linear regression model was used
to assess factors associated with general and PC-specific
HRQoL. Independent variables were age, ethnicity, treatment
facility, income, marital and surgical status.
Principal Findings: Caucasians (C) had significantly higher
income, education and were more likely employed. For
Caucasians there was no significant variation in education by
age (some college or greater: <65 C = 75.2%, = >65 C =
73.81%), however African Americans (AA) showed significant
variation (among <65 AA, 51.11% had some college or greater;
in the = >65 AA, 80% were high school graduates).
Caucasians and AA showed no important variation in marital
status by age, however more AA lived alone than Caucasians.
Type of treatment varied significantly across ethnicity; more
Caucasians received surgery (71.25% C vs. 44.19% AA, p =
.0009) whereas more AA received radiation treatment (13.13%
C vs. 25.58% AA, p = .047). Baseline mean scores of general
HRQoL demonstrated that AA were substantially less healthy
by all physical, psychological and social measures. The PCspecific HRQoL did not differ by ethnicity. At 3-months,
general HRQoL scores remained significantly higher for
Caucasians except for vitality, mental health, and social
function. The mean scores for both groups in general HRQoL
and PC QoL declined from baseline levels, though the groups’
relative divergence narrowed. PC-specific QoL demonstrated
important differences by ethnicity. Caucasians reported
significantly greater bowel function (87.86 C vs. 81.47 AA, p =
0.02) and less bowel bother (88.75 C vs. 78.41 AA, p = 0.006),
while AA reported significantly greater sexual function (20.74
C vs. 29.06 AA, p = 0.045). Regression analysis for baseline
data indicated that income and presence of other illnesses
were significantly positively associated with general health and
physical function. Regression analysis for the 3-month followup data indicated that patients receiving surgery
demonstrated significant negative association with urinary
function.
Conclusions: Variations exist at baseline characteristics and
HRQoL of newly diagnosed PC patients by ethnicity. At 3month, the variations in HRQoL by ethnicity narrowed as
HRQol for both the ethnic groups declined.
Implications for Policy, Delivery or Practice: Assessment of
HRQoL and comorbidities is crucial for effective management
of PC.
Primary Funding Source: Department of Defense Prostate
Cancer Research Program
• The Effect of the Fit between Organizational Culture and
Structure on Medication Errors in Medical Group Practices
Amer Kaissi, Ph.D., John Kralewski, Ph.D., Bryan Dowd, Ph.D.,
Alan Heaton, Pharm. D.
Presented by: Amer Kaissi, Ph.D., Assistant Professor, Health
Care Administration, Trinity University, One Trinity Place, #
58, San Antonio, TX 78212; Tel: 210.999.8132; Fax:
210.999.8108; E-mail: amer.kaissi@trinity.edu
Research Objective: It is widely acknowledged that many
prescription drug errors occur in the ambulatory care setting
and that they have serious quality of care implications.
Previous research examining this issue has focused on
hospitals and on individual-level factors. This study adopts an
organizational perspective to assess the effects of
organizational culture, organizational structure and their fit
(i.e., their congruence) on medication errors in medical group
practices.
Study Design: The cultural dimensions included in this
research are “autonomy”, “collegiality” and “patient
emphasis”. The structural variables included measure the
degree of standardization/formalization (clinical practice
guidelines use and benchmarking methodologies) and
information processing capacity (having an electronic medical
record and providing physicians with computerized drug
information at care site) of the organization. Group practice
size, geographic location, physician workload and patient
complexity were controlled for. The dependent variable was
calculated as a rate of medication errors per prescriptions
written for Blue Cross Blue Shield of Minnesota patients by
physicians in the clinic. To test for interactions between pairs
of culture-structure variables, moderated regression analyses
were conducted.
Population Studied: Variables that measure the
organizational culture and structure were taken from two
surveys of medical group practices that contracted with
BCBSof Minnesota in 2001. Medication errors data were
obtained from BCBS of Minnesota using a computerized drug
utilization review system. Seventy-eight medical group
practices were included in the analyses.
Principal Findings: The overall error rate was 29%; nearly one
out of every three prescriptions results in a medication error of
some kind. However this rate drops to 18% when the more
difficult to evaluate overdose errors are excluded. Results from
the moderated regression revealed a general pattern of fit
between each of the “patient emphasis” and “collegiality”
cultural traits and structures related to “standardization/
formalization.” More specifically, the use of benchmarking
methodologies and clinical practice guidelines was associated
with decreased error rates in group practices that encourage
“patient emphasis” (F=6.63 and F=7.99 respectively) and
“collegiality” (F=4.55 and F=6.91 respectively). However, the
relationship between structures related to information
processing capacity and the cultural dimensions was not
statistically significant.
Conclusions: The findings suggest that the interaction
between specific cultural traits and structural dimensions can
help understand some of the relationships between
organizational culture, structure and medication errors.
Organizational structures do not exist in a vacuum, but rather
their effect on patient safety outcomes is “moderated” by the
existing organizational culture.
Implications for Policy, Delivery or Practice: The
implications are that medical group practice administrators
and medical directors have alternate ways to prevent or reduce
medication errors and that they should be attentive to the
cultures of their practices when considering those options.
Primary Funding Source: AHRQ
• Testing an Innovative Method to Collect Adverse Events
Data: The Shift Coupon
Victoria Kellogg, R.N., CRNP, M.B.A., Ph.D., Donna Havens,
Ph.D., R.N.
Presented by: Victoria Kellogg, R.N., CRNP, M.B.A., Ph.D., 131
Birchtree Court, State College, PA 16801; Tel: 814.861.2665;
Fax: 814.861.4474; E-mail: vak107@adelphia.net
Research Objective: The Shift Coupon (Kellogg, 2003) was
designed to provide a more accurate measure of adverse
events occurring in the hospital setting by eliminating the two
overriding barriers to reporting adverse events: “[1] fear and [2]
lack of belief that it [reporting the adverse event] results in
improvement” (Leape, 1999, p. 1). Thus, the study purpose
was to test the ability of the Shift Coupon to collect adverse
events data in the hospital setting. The specific aims of the
study were: 1.) To determine the extent to which adverse
events occur; 2.) To determine the causes of the adverse
events reported; and 3.) To determine if there was a difference
between the number of adverse events reported on Shift
Coupons versus incident reports.
Study Design: A nonexperimental descriptive, comparative
study design was used. The instrument tested was the Shift
Coupon. The procedure used in this study was: 1.) Registered
nurses were mailed the instrument following Dillman’s (2000)
Tailored Design Method; 2.) Registered nurses completed
Shift Coupons for five shifts; and 3.) Registered nurses
returned completed coupons to the researcher. Data from
Shift Coupons were entered into a SPSS data base and
analyzed using descriptive statistics and chi-square.
Population Studied: The sampling frame was all individuals
possessing a registered nursing license in Pennsylvania. The
inclusion criteria were: individuals of at least 18 years of age,
individuals with a current Pennsylvania registered nursing
license, and individuals with a current Pennsylvania address.
From the individuals who met the inclusion criteria, a random
sample of 1,000 registered nurses was selected.
Principal Findings: A total of 355 registered nurses returned
1937 Shift Coupons (46.7% response rate), with 247 (69.6%)
registered nurses returning 1369 coupons for shifts worked in
a hospital. On the majority of Shift Coupons (70.9%)
registered nurses reported the occurrence of no adverse event
during the shift. On the 397 Shift Coupons where registered
nurses reported the occurrence of an adverse event, the most
commonly reported adverse events were: patient complaints
(21.4%), medication errors (18.9%), family complaints
(17.6%), and patient falls (12.1%). When the data were
analyzed according to hospital unit type, variations appeared
in the types of adverse events reported by registered nurses.
Registered nurses identified lack of staff (36.8%), lack of
communication (30.5%), and work overload (29.5%) as the
most frequent causes of all the adverse events reported.
When the date were analyzed by type of adverse event, these
adverse event causes remained dominate. Finally, there were
significantly more adverse events reported on Shift Coupons
than were reported on incident reports (p<.001).
Conclusions: The Shift Coupon is a viable method to collect
adverse events data.
Implications for Policy, Delivery or Practice: The Shift
Coupon provides a method for collecting adverse events data
in the hospital setting that attenuates barriers to reporting
adverse events while placing front-line healthcare
professionals in a position to identify direct and indirect
adverse event cause(s). Shift Coupon data can be used to
isolate areas needing quality improvement and to develop
appropriate quality improvement initiative(s) to decrease the
occurrence of adverse events.
Primary Funding Source: Sigma Theta Tau International,
Beta Sigma Chapter
• The Relationship between Management Approach and
Medical Errors: A Qualitative Study
Naresh Khatri, Ph.D., Justin Kauk, Medical Student, Timothy
Patrick, Ph.D.
Presented by: Naresh Khatri, Ph.D., Assistant Professor,
Health Management and Informatics, University of Missouri,
324 Clark Hall, Columbia, MO 65211; Tel: 573.884.2510; Fax:
573.882.6158; E-mail: KhatriN@health.missouri.edu
Research Objective: We contrast two alternative
management views, control-based and commitment-based,
and examine the hypothesis that incidence of medical errors is
significantly higher and quality of service significantly lower in
a control-based management than in a commitment-based
management.
Study Design: We employed a qualitative methodology
(multiple-case design) and interviewed 24 individuals of four
health care organizations in Missouri with the help of a semistructured interview questionnaire. The individuals included 2
Chief Executive Officers, 7 Directors of Medical Units, 5
Doctors, 4 Chief Nursing Officers/Nurse Supervisors, 4
Registered Nurses, and 2 Technicians. Each interview lasted
about an hour. Interviews were tape-recorded and transcribed
verbatim for content analysis.
Population Studied: Four health care organizations in
Missouri
Principal Findings: We found a great disparity in types of
work environments, even within the same organization.
Units/organizations employing a control-based approach were
characterized by hierarchy (power differences according to
rank), silos (poor communication or coordination between
departments), and status differences or ‘caste structure’ based
on professional identities. The outcome was a culture of
blame and suppression of reporting of medical errors and lack
of learning from mistakes. Further, close monitoring and
control of employee behaviors via elaborate machinery or
‘system’ demoralized and frustrated employees, leading to
poor morale and effort. Poor effort, in turn, resulted in less
than satisfactory quality of patient care and safety.
Units/organizations using a commitment-based approach, on
the other hand, involved employees in decision-making and
relied on managers/leaders who possessed transformational
qualities. They emphasized creating an open and trusting
culture that encouraged reporting and detection of errors,
thus allowing them to learn from mistakes. The employee
morale in these units/organizations was high resulting in
better quality of care and patient safety.
Two of the four organizations in the sample were trying to
move away from culture of blame to an open culture in which
employees were not penalized for reporting of errors.
There were formal and elaborate mechanisms for reporting of
errors for nurses, but reporting of errors for physicians was
still a closed system.
Conclusions: The broad management approach (controlbased or commitment-based) affects incidence of medical
errors and quality of care in two ways: (1) by reporting and
detection of errors and (2) by affecting the employee morale
and effort. The commitment-based management approach
increases reporting and detection of errors by creating an
open and trusting culture. It also enhances morale/effort of
employees. The increased employee morale/effort impacts on
quality of patient care and incidence of medical errors.
Implications for Policy, Delivery or Practice: Efforts to
reduce medical errors and enhance quality of patient care will
not be successful unless the basic culture and systems of
management in the health care industry are transformed from
control-based to commitment-based. The needed
transformation can be achieved in four ways: (1) breaking
down of hierarchy, silos, and ‘caste structure’, (2) fostering
involvement and communication, (3) instituting just and fair
management practices, and (4) placing transformational
leaders in key positions in health care organizations.
• The Influence of the Structure and Culture of Medical
Group Practices on Prescription Drug Errors
John Kralewski, Ph.D., Bryan Dowd, Ph.D., Alan Heaton,
Pharm.D., R.Ph, Amer Kaissi, Ph.D.
Presented by: John Kralewski, Ph.D., William Wallace
Professor, Health Services Research and Policy, University of
Minnesota, 420 Delaware Street SE, MMC 729, Minneapolis,
MN 55455; Tel: 612.624.2912; Fax: 612.624.2196; E-mail:
krale001@umn.edu
Research Objective: This project was designed to identify the
magnitude of prescription drug errors in medical group
practices and to explore the influence of the practice structure
and culture on those error rates. Seventy-eight practices
serving a Blue Cross managed care plan during 2001 are
included in the study.
Study Design: Using Blue Cross claims data, prescription
drug error rates were calculated at the enrollee level and then
were aggregated to the group practice that each enrollee
selected to provide and manage their care. Practice structure
and culture data were obtained from surveys of the practices.
Data were analyzed using multivariate regression.
Population Studied: Seventy-eight medical group practices
serving 114,746 enrollees in a Blue Cross managed care
product. During the study period (2001) there were 250,024
prescriptions filled for these patients.
Principal Findings: Both the culture and the structure of
these group practices appear to influence prescription drug
error rates. Seeing more patients per clinic hour, more
prescriptions per patient and being cared for in a rural clinic
are all strongly associated with more errors. Conversely,
having a case manager program is strongly related to fewer
errors in all of our analyses. The culture of the practices
clearly influences error rates but the findings are mixed.
Practices with cohesive cultures have lower error rates but
contrary to our hypothesis, cultures that value physician
autonomy and individuality also have lower error rates than
those with a more organizational orientation. Our study
supports the contention that there are a substantial number of
prescription drug errors in the ambulatory care sector. Even
by the strictest definition, there were about 13 errors per 100
prescriptions for Blue Plus patients in these group practices
during 2001.
Conclusions: Our study supports the contention that there
are a substantial number of prescription drug errors in the
ambulatory care sector. Even by the strictest definition, there
were about 13 errors per 100 prescriptions for Blue Plus
patients in these group practices during 2001. Our data
provides insights into the nature of these errors and direction
for future research. Our study also provides information that
will enable medical group practices to deal more effectively
with drug errors.
Implications for Policy, Delivery or Practice: This study
provides information which can be used by medical group
practices to reduce prescription drug errors and improve the
quality of their patient care.
Primary Funding Source: AHRQ
• National Measurement of Health Care Quality: The
Imperative of Public Accountability and Hospitals’ Pursuit
of Optimal Clinical Practice
Mel Krasner, Ph.D., Marianne Brassil, R.N.
Presented by: Mel Krasner, Ph.D., Senior Director, Clinical
Evaluation and Outcomes Research, New York University
Medical Center, 550 1st Avenue, GBH C-124, New York, NY
10016; Tel: 212.263.8199; Fax: 212.263.0096; E-mail:
mel.krasner@med.nyu.edu
Research Objective: To assess the impact of a national health
quality measurement program on a participating hospital, and
to identify weaknesses and potential refinements in program
design and methodology.
Study Design: This paper is a constructive critique of the
national program and encompasses two components: (a) a
critical review of 1400 hospitalizations to determine the extent
to which program definitions and specifications are likely to
yield consistent and meaningful measurement of quality; and
(b) a case-study report and commentary describing the
impact and challenges of program participation at an urban
academic medical center.
Population Studied: Hospital inpatients discharged from an
urban academic medical center in New York City between July
1, 2002 and September 30, 2003 with a principal diagnosis of
heart failure, acute myocardial infarction, or pneumonia.
Principal Findings: (a) The program diverted substantial
resources from the institution’s internal quality improvement
agenda; (b) The complexity of data collection specifications
intensified difficulties in influencing clinical practice and
documentation; (c) Program specifications did not adequately
distinguish patient categories that required very different
clinical management.
Conclusions: Our experience and analysis underscored the
following recommendations for mutli-hospital quality
measurement efforts:
1.Organize comprehensive pilot testing over adequate time
periods and sites to identify and repair flaws that only emerge
in clinical practice, e.g., definitional ambiguities, exclusion
criteria, etc.
2.Preserve hospitals’ ability to pursue institutionally
designated quality improvement priorities. Make a hospital’s
internal process for identifying and addressing quality
improvement opportunities a significant component of
comparative quality evaluation and scoring.
3.Develop data collection rules and definitions that closely
conform to the day-to-day reality of good clinical practice and
expert professional thinking.
4.Carefully consider the potential adverse impact of
prematurely publicizing hospital-specific results on the
accuracy and comparability of the data, particularly when
abstraction and reporting criteria are complex and subject to
interpretation.
5.Consider how program specifications affect the difficulty and
reliability of verification and audit, which are obviously
important to ensure integrity and comparability.
6.Allow ample time for quality improvement efforts to become
evident.
Implications for Policy, Delivery or Practice: Obviously,
measurement and dissemination of quality indicators can be a
powerful motivator. But our experience suggests a significant
potential exists for diverting resources from the nuts and bolts
of quality improvement.
The bottom line is that a net benefit is not automatically
assured, but rather depends on judicious design and strategic
assessment of the scope, effectiveness, and external
consequences of the program.
• Variation in ICU Mortality Model Calibration among
Clinical and Demographic Subgroups of ICU Populations:
Can Risk Adjustment Really Correct for Case Mix
Michael Kuzniewicz, M.D., M.P.H., Rondall Lane, M.D.,
M.P.H., Mitzi Dean, M.S., MHA, Deborah Rennie, BA, R.
Adams Dudley, M.D., M.B.A.
Presented by: Michael Kuzniewicz, M.D., M.P.H., Fellow,
Institute for Health Policy Studies, University of California, San
Francisco, 3333 California Street, Suite 265, San Francisco, CA
94118; Tel: 415.476.1061; Fax: 415.476.0705; E-mail:
mkuz3274@itsa.ucsf.edu
Research Objective: We have previously shown that the most
commonly used ICU mortality models—the Simplified Acute
Physiology Score II (SAPS-II), Mortality Probability Model II
(MPM-II), and Acute Physiology and Chronic Health
Evaluation II (APACHE-II), all of which were developed in the
1980s—have calibration problems manifest by overprediction
of mortality rates. To determine how to improve them, we
now compare the calibration of these models in subsets of
patients grouped by location prior to ICU admission, type of
admission, age, diagnosis, and organ system affected.
Study Design: Retrospective analysis of data from the
California Intensive Care Outcomes (CALICO) project, in
which hospitals collected 163 variables on consecutive ICU
patients. APACHE-II, SAPS-II, and MPM-II discrimination
(area under the receiver operating characteristic curve, AUC)
and calibration (using Hosmer-Lemeshow’s C test) were
measured in each subgroup.
Population Studied: 3981 adults admitted to ICUs in the 23
CALICO hospitals in 2002.
Principal Findings: All models performed well on patients
admitted from other hospitals and the inpatient floor, but
were poorly calibrated for patients coming from the operating
room and emergency department. Hosmer-Lemeshow
statistics were worse for medical patients than for surgical
patients, and APACHE-II and SAPS-II showed especially poor
calibration in patients over 45. When looking at patients
grouped by the major organ system affected, all models
performed well in surgical patients with the cardiac,
respiratory, neurologic, or gastrointestinal disorders. The
models performed poorly in medical patients with the cardiac
and respiratory system affected, especially those with acute
coronary syndromes, COPD, and pneumonia. The models
discriminate poorly among patients with septic shock and
CHF.
Conclusions: All three models show substantial variation in
their performance according to location prior to admission,
type of admission, and diagnosis.
Implications for Policy, Delivery or Practice: Since the older
individuals are more likely to have chronic conditions, the
poor performance at increasing age may represent
overweighting of the chronic health variables in the models.
The poor performance among patients coming from the
emergency department and operating room may reflect that
less effort is made to correct these patients' vital signs before
ICU transfer than for patients coming from other locations
within the hospital or from other hospitals. Finally, the models
exhibited their greatest tendency to overpredict mortality
among medical cardiac and respiratory patients, perhaps
because ICUs are more able to save such patients than they
were when the models were first developed. ICU mortality
model calibration needs to be substantially improved before
ICU performance assessments can be accurate. We have
identified specific patient subgroups on which to focus model
improvement efforts.
Primary Funding Source: California Office of Statewide
Health Planning and Development
• Assessing Emergency Department Length of Stay Prior to
ICU Admission as a Key Variable in Case-Mix When
Utilizing Risk Adjustment Models for Quality Comparison
Rondall Lane, M.D., M.P.H., Michael Kuzniewicz, M.D.,
M.P.H., Mitzi Dean, M.S., MHA, Deborah Rennie, BA, R.
Adams Dudley, M.D., M.B.A.
Presented by: Rondall Lane, M.D., M.P.H., Fellow, Institute
for Health Policy Studies, University of California San
Francisco, 3333 California Street, Suite 265, San Francisco, CA
94118; Tel: 415.514.2176; Fax: 415.476.0705; E-mail:
adudley@itsa.ucsf.edu
Research Objective: The National Quality Forum and the
Leapfrog Group have identified the intensive care unit (ICU)
as a priority area in which to focus patient safety efforts. The
Joint Commission on the Accreditation of Healthcare
Organizations (JCAHO) has proposed making risk-adjusted
ICU mortality rate a core measure for hospital performance
assessment. We have shown that ICU mortality models,
including Acute Physiology and Chronic Health Evaluation II
(APACHE-II), Simplified Acute Physiology Score II (SAPS-II),
and Mortality Probability Model II (MPM-II), are more poorly
calibrated for ICU patients admitted through the emergency
department (ED) than for patients admitted from other
locations, manifest by overprediction of death among ED
patients. We hypothesized that this miscalibration may reflect
that ICU mortality models base severity evaluations on
physiological measurements taken at or shortly after ICU
admission but do not use measurements taken more than
several hours before admission. Therefore, patients under
medical management for some time before ICU transfer (who
usually have had their vital signs stabilized) may appear less ill
than patients who pass quickly through the ED without
stabilization but who have the same underlying process and
probability of mortality. If this is the case, one would expect
patients with relatively short ED stays to have lower
standardized mortality rate (SMR) than patients who either
had prolonged ED stays (and were presumably stabilized) or
who came from other locations. The implication for
performance assessment is that ICU performance may be
influenced by length of stay (LOS) in the ED.
Study Design: Retrospective analysis of data from the
California Intensive Care Outcomes (CALICO) project.
Twenty-three hospitals collected 163 variables on 3955
consecutive adult ICU patients at hospital discharge. SMRs
were determined for APACHE-II, SAPS-II and MPM-II for:
Group ED-Short = ED LOS <12 hours, Group ED-Long = ED
LOS > 12 hours, and Group Non-ED = admitted from a nonED location. Chi squares were calculated to determine
significance between SMRs.
Population Studied: 1958 patients were admitted from an ED
(1783 with ED LOS < 12 hours, 175 with LOS > 12 hours), and
1997 came from other sources.
Principal Findings: The SMRs for APACHE-II were 0.61, 0.75,
and 0.75 for Groups ED-Short, ED-Long, and Non-ED,
respectively. For SAPS-II the SMRs were 0.59, 0.81, and 0.80
for Groups ED-Short, ED-Long, and Non-ED, respectively; and
for MPM-II, the SMRs were 0.68, 0.81, and 0.96. For all three
models, the ED-Long SMRs were not statistically different
from the Non-ED SMRs. For all three models, the ED-Short
group had significantly lower SMRs than the combined EDLong and Non-ED group (p<.05 for all three models).
Conclusions: For all three mortality models, poor calibration
and overestimation of mortality is worst among ED patients
with short ED LOS.
Implications for Policy, Delivery or Practice: Using current
ICU mortality models, ED LOS, which varies among hospitals
because of staffing and ED management policies that have
little to do with the ICU, may influence apparent ICU mortality
performance.
Primary Funding Source: California Office of Statewide
Planning and Development
• Do Patient Intentions Predict Vaccination Behavior over
Time?
Sherri LaVela, M.B.A., M.P.H., Marcia Legro, Ph.D., Frances
Weaver, Ph.D., Barry Goldstein, , M.D., Ph.D., Bridget Smith,
M.P.A., Carolyn Wallace, Ph.D.
Presented by: Sherri LaVela, M.B.A., M.P.H., Social Science
Analyst/Project Manager, Midwest Center for Health Services
and Policy Research, Department of Veterans Affairs, 5th
Avenue and Roosevelt Road, Hines, IL 60141; Tel:
708.202.5895; Fax: 708.202.2499; E-mail:
lavela@research.hines.med.va.gov
Research Objective: To determine if the proportion of
veterans with spinal cord injuries and disorders (SCI&D) who
received an influenza vaccine and/or a pneumococcal
polysaccharide vaccine (PPV) increased over time. Also, to
examine whether intention to receive influenza vaccine was
related to subsequent behavior.
Study Design: Questionnaires to assess self-reported rates of
annual influenza vaccination and PPV (ever received),
attitudes, intentions and behaviors were conducted following
each of two vaccination seasons. Data were analyzed with
descriptive statistics for a national cohort of veterans who
completed questionnaires for both years (n=1200).
Population Studied: Veterans with spinal cord injuries and
disorders
Principal Findings: Respondents’ mean age was 58, 98%
were male, 76% were white. Influenza vaccination rates
increased significantly in the second year compared to the first
year (68% vs. 65%; p < 0.0001). Rates for ever having
received a PPV were 67% in year 1 and 75% in year 2; p<
0.0001. Of respondents who indicated in year 1 intention to
get an influenza shot during the following season, 86% did.
Of those planning not to receive an influenza vaccine, 84%
did not. Of respondents who were unsure of their intentions,
40% received the influenza vaccine during the following year.
Conclusions: Vaccination rates increased over time for
influenza and PPV. Intention to receive influenza vaccine is
related to actual behavior in veterans with SCI&D.
Implications for Policy, Delivery or Practice: Respiratory
complications, the leading cause of death in the SCI&D
population, can be reduced using inexpensive vaccines. A
focus on individuals who are unsure and those who have no
intention of being vaccinated using methods aimed at altering
intentions may be warranted.
Primary Funding Source: VA
• Improving PPO Physician Adherence to Evidence-based
Care: A Four-Year Longitudinal Evaluation of an Incentive
Program
Richard Chung, M.D., Antonio Legorreta, M.D., M.P.H., John
Berthiaume, M.D., Helen Chernicoff, M.D.
Presented by: Antonio Legorreta, M.D., M.P.H., President
and CEO, Health Benchmarks, 21650 Oxnard Street, Suite 550,
Woodland Hills, CA 91367; Tel: 800.465.6575; Fax:
818.715.9934; E-mail: alegorreta@healthbenchmarks.com
Research Objective: The objective of this study is to evaluate
the effect of a quality-based incentive program on improving
the quality of care rendered to patients enrolled in health plans
for a large not-for-profit insurer in Hawaii.
Study Design: Administrative claims data were used from a
four-year period including one year prior to program
implementation and three years thereafter. Eleven of the
twelve clinical quality indicators first used in the program were
chosen for evaluation, first, because they had longitudinal data
available for all program years; second, because they applied
to the widest denominator populations, and third because
they were of clinical interest. Logistic regression analysis was
performed on the data to estimate the effect of the program
by examining outcomes prior to- and post- program
implementation. The regression model controlled for patient
age and comorbidities. Overall performance rates for the
indicators were calculated to identify the trends in quality of
patient care received over the four-year period for each of the
indicators.
Population Studied: The study assessed physicians
participating in the Hawaii Medical Service Association
(HMSA)'s PPO and all HMSA PPO members.
Principal Findings: The results of the regression model reveal
positive association between patients having visited a PQSR
program-participating practitioner after program
implementation and receiving recommended care for eight of
the eleven indicators. Six of these eight indicators were
statistically significant (p<.05). For the indicators with
statistically significant results, a patient was 13 - 52 percent
more likely to receive appropriate care from a participating
practitioner than from a non-participating practitioner,
depending on the indicator. For the indicators with statistically
insignificant results, insufficient sample size of the
comparison groups was in part responsible for the statistically
insignificant results. The annual population-based rates of
recommended care delivery for six of the eleven indicators
were observed to increase over time since the program was
implemented.
Conclusions: This study provides evidence suggesting that
the PQSR program is associated with improvements in the
quality of healthcare delivered to HMSA members during the
observation period 1998-2001. The PQSR program has
demonstrated improvements in key measurements of clinical
quality, and has also demonstrated sustainability of this
improvement over time for many of these measurements.
Implications for Policy, Delivery or Practice: The findings
from this assessment of a PPO quality-based incentive
program indicate that pay-for-performance may be an effective
approach for improving the quality of care.
Primary Funding Source: Hawaii Medical Service Association
• Costs of IRB Compliance Processes in a Multi-Site
Evidence-Based Quality Improvement Implementation
Study
Chuan-Fen Liu, M.P.H., Ph.D., Laura Bonner, Ph.D., Edmund
Chaney, Ph.D., Barbara Simon, M.A., Mona Ritchie, L.C.S.W.
Presented by: Chuan-Fen Liu, M.P.H., Ph.D., HSR&D
Investigator, Health Services Research and Development, VA
Puget sound Health Care System, HSRD (152), 1660 S.
Columbian Way, Seattle, WA 98108; Tel: 206.764.2587; Fax:
206.764.2935; E-mail: chuan-Fen.liu@med.va.gov
Research Objective: Implementation studies of evidenced
based practices in routine clinical settings face a unique
challenge in human subject protection. The implementation
of evidence-based practices could be considered of “minimum
risk” to patients and in fact may actually reduce the risk of
ineffective treatment. Yet, the evaluation of these
implementation efforts is frequently associated with the
assessment of patient specific outcomes and therefore
requires gathering protected health information from
individual patients or administrative databases. Anecdotal
observations suggest that the IRB compliance process
requires allocation of a substantial amount of project
resources. The study documented the costs to the research
program of IRB compliance process and safety monitoring
procedures based on a national multi-site study that
implemented evidence-based depression treatment practices
in primary care in 2002 - 2003.
Study Design: We used two main data sources – project logs
and records and an IRB activity tracking system. First, we
performed data reduction and analysis on all project logs and
records to identify and document IRB related activities,
including initial applications, reviews, modifications, renewals,
project meetings and conference calls, communications with
IRB personnel, safety monitoring and adverse event reporting.
Second, we conducted an analysis of the IRB activity-tracking
system to estimate project staff time on IRB related activities.
Finally, we quantified these IRB efforts by activity type and
associated personnel costs.
Population Studied: IRB compliance processes from 3
administrative sites and 10 clinical sites
Principal Findings: The IRB approval process varies greatly
across all the participating sites. The length of time from the
date of initial IRB application submission to the date of
approval ranged from 15 days to 227 days across all the IRB
sites, with an average of 105 days. For the initial IRB
submission, the number of IRB–required forms across sites
ranged from 1 to 7; the number of supporting documents
ranged from 12 to 32; the number of required questions on
initial IRB protocol forms ranged from 9 to 30; and the
number of stipulations requiring modification of the initial IRB
submission ranged from 2 to 17. Project personnel costs
related to IRB activities and patient safety monitoring were
significant.
Conclusions: The IRB compliance process requires a
substantial amount of project resources in multi-site
dissemination study. Given the great variation in IRB
compliance procedures and requirements across sites, there
is need for more research to relate compliance cost to benefit.
Implications for Policy, Delivery or Practice: This study
raises an important concern over the efficiency of using local
IRB review with duplication of oversight efforts for multi-site
low-risk evidenced-based implementation studies.
Primary Funding Source: VA
• Learning From Errors in Ambulatory Pediatrics
Julie Mohr, MSPH, Ph.D., Carole Lannon, M.D., M.P.H.,
Kathleen Thoma, M.A., Eric Slora, Ph.D., Richard Wasserman,
M.D., Donna Woods, Ph.D.
Presented by: Julie Mohr, MSPH, Ph.D., Assistant Professor,
Department of Medicine, University of Chicago, 5841 S.
Maryland Avenue, MC2007, Chicago, IL 60637; E-mail:
jmohr@medicine.bsd.uchicago.edu
Research Objective: Approximately 70% of pediatric care
occurs in ambulatory settings, yet there has been little
research on errors and harm in these settings. Given the
importance of understanding harm in ambulatory pediatrics,
this study was funded by AHRQ as part of the University of
North Carolina (UNC) Center for Education and Research on
Therapeutics (CERTs). UNC CERTs partnered with the
American Academy of Pediatrics (AAP) Pediatric Research in
Office Settings (PROS) Network to conduct the study.
Learning from Errors in Ambulatory Pediatrics (LEAP) was
designed to (1) develop a secure, web-based tool for reporting
errors; (2) identify types and range of errors; and (3) identify
errors that are generalizable across multiple practices.
Study Design: Data collection was piloted in 5 pediatric
practices in March 2003 using the secure, web-based tool.
After revision to the tool, 14 sites collected data from June -
September 2003. Three members of the research team (1
pediatrician and 2 patient safety researchers) independently
coded the qualitative error reports using the constant
comparative method. Reports were coded by medical
domain, problem types, and child specific factors. Coding
discrepancies were reconciled by consensus.
Population Studied: Study participants included communitybased Pediatricians who are part of the Pediatric Research in
Office Settings (PROS) Network of the American Academy of
Pediatrics.
Principal Findings: Study participants reported 136 errors.
Data collection via the web-based tool was very successful;
participating practitioners reported a high degree of
satisfaction and a minimal number of problems with using the
tool. Errors were reported in several domains: prevention,
diagnosis, treatment, patient identification, communication,
falls, equipment, and administration. For example, one
reported treatment error was “Prescription changed from
liquid to capsule form of anticonvulsant. Mom misunderstood
directions and gave both meds for one week. Child developed
blurred vision, stuttering, and ataxia.” An example of an
administrative error was “Test results were not received by the
parent in the mail. When the parent called to ask, it was found
that they had listed two addresses, and I had sent it to the
wrong address.” One reported medication error was “Patient
was prescribed an antibiotic that patient was known allergic.
Past history of rash. Antibiotic was filled but problem was
noted prior to giving medication to patient.”
Conclusions: Physicians reported errors, yet various members
of the care team (parents, nurses, pharmacists, etc.)
discovered the errors. This suggests that everyone has a role
preventing errors from reaching the child. Information learned
from this study will be instrumental to subsequent design of
interventions to reduce errors and improve pediatric patient
safety.
Implications for Policy, Delivery or Practice: Information
learned from this study will be instrumental to the subsequent
design of interventions to reduce errors and improve patient
safety for children. Further research will clarify categories of
harm in ambulatory settings and explore venues for
presenting errors and collaboratively designing and testing
solutions. The success of the web-based, data collection tool
points the way for future on-line data collection efforts for the
PROS Network.
Primary Funding Source: AHRQ
• Standardized Patients as Witnesses to Context & Quality
of Care
Debora Paterniti, Ph.D., Carol Franz, Ph.D., Richard Kravitz,
M.D., M.S.P.H., Mitchell Feldman, M.D., MPhil, Ronald
Epstein, M.D., Ph.D.
Presented by: Debora Paterniti, Ph.D., Assistant Adjunct
Professor, Internal Medicine, University of California, Davis,
2103 Stockton Boulevard, Suite 2224, Sacramento, CA 95817;
Tel: 916.734.2367; Fax: 916.734.2349; E-mail:
dapaterniti@ucdavis.edu
Research Objective: The purpose of this study was to assess
whether standardized patients (trained actors)can be trained
to “observe” with an ethnographic eye the context and quality
of physician visits.
Study Design: We undertook an ethnographic study of
physician office visits using standardized patients (SPs) to
provide an understanding of “patient” views of the context
and quality of physician visits. SPs trained in specific roles
were instructed on making ethnographic observations of all
aspects of their visit. SPs were provided a literature-generated
list of visit aspects they might consider as important to
observation; however, SPs were instructed to reflect on all
aspects of their visit and to describe those aspects that were
most salient to them as “patients.” SPs dictated field notes
immediately following each visit. Transcripts of the notes were
reviewed for persistent and recurring themes related to SP
perceptions of the context and quality of their visit and its
effect on them as “patient.”
Population Studied: Twelve SPs scheduled and attended new
patient visits with 52 different physicians at both fee-for-service
and HMO practice settings.
Principal Findings: Analysis of 58 field notes from office visits
reveals the impact of the dual role of “patient” and actor on
SP perspectives of the visit. SP perceptions of their role as
actor impacted their “patient” perspectives on satisfaction
with the visit and trust in physician recommendations for
treatment (e.g., having the physician probe into SP
occupational status or familiar relationships in town, delving
too much into the "patient's" personal life). SPs were careful
to note the behaviors they tried to avoid as actors that they
believed might jeopardize their role as an authentic patient in
their relationship with the physician (e.g., accepting a "dirty"
thermometer, being younger than the average age of waiting
room patients, reporting back pain and bending improperly in
the waiting room). SPs also reported tolerating or accepting
behaviors in the role of actor that they believed they would not
accept as an authentic patient.
Conclusions: Our findings reveal the potential impact of the
dual role on the use of SPs as “expert witnesses” of the
medical encounter. Potential conflict between the roles of
patient actor and authentic patient may be a barrier to using
SPs as ethnographic observers. Understanding such conflict,
however, may be useful in designing more directive training
for SPs who may have an opportunity to be an "expert
witness."
Implications for Policy, Delivery or Practice: Standardized
patients are increasingly used to assess quality of care in
medical settings, as part of medical education, and in
research. SPs have an advantage over patients in witnessing
the delivery of care for the same condition with a variety of
physicians and in various contexts. Yet, SPs must deal with
the dual role of both “patient” and actor. The dual role of SP
should be considered in assessment of SP reports of quality of
care and in the training of standardized patients.
Primary Funding Source: NIMH
• Creating and Testing Measures of Systemness
L. Gregory Pawlson, M.D., M.P.H., Sarah Scholle, Dr.P.H., Lief
Solberg, M.D., Sarah Shih, M.P.H.
Presented by: L. Gregory Pawlson, M.D., M.P.H., Executive
Vice President, NCQA, 2000 L Street, N.W., Washington, DC
20037; Tel: 202.955.5170; Fax: 202.955.3559; E-mail:
pawlson@ncqa.org
Research Objective: In the IOM report, “Crossing the Quality
Chasm,” chronic disease care is identified as the highest
priority and systems within practice organizations as the most
important element needing change. Ed Wagner and others
have created a theoretical model (the Planned Care ModelPCM), based on available empiric evidence and expert
opinion, that has been successfully used to guide
interventions aimed at improving health system performance
related to chronic illness and preventive services. This model
includes four domains that reside within the individual
practice office setting: clinical information systems, delivery
system design, self-management support, and decision
support systems. In order to extend use of the Model beyond
experimental settings, it must be translated into specific
metrics that can be used by practices, or those assessing
practices. Such an instrument could be used in the evaluation
of quality and/or rewarding performance by accrediting
bodies, purchasers and health plan and potentially by
malpractice insurers as a basis of differential malpractice
premiums.
Study Design: Survey development: A research team led by
Greg Pawlson at NCQA and including Ed Wagner, Shelly
Greenfield and Sherry Kaplan (UCI) Barbara Fleming (CMS)
and Lief Solberg (HealthPartners-ICSI), has developed a set of
systems measures based on the PCM. This set of measures
has been termed the Practice Systems Assessment Survey
(PSAS) based on the PCM. A alternative formulation of the
instrument, called the Physician Office Link (POL), using a
web based collection system, was developed and
implemented in the GE led, Bridges to Excellence Project. The
PSAS and POL are designed to assess the presence, content,
and extent of use of systems at the office practice level of the
health care system and to be useful in small, single physician
offices as well as in larger medical groups.
Testing of reliability: See below for sample. The PSAS (with
added questions at the end regarding perceived validity and
any problems with understanding) was mailed with a cover
letter from the medical group leader requesting their
cooperation in this study. Follow-up included a post card
reminder at two weeks, a second survey and cover letter at 5
weeks, and telephone contact for survey completion over the
phone to non-respondents at 8 weeks. We have previously
been able to obtain a 60-85% response rate among medical
personnel with this approach. A sub-sample of respondents
were asked to complete an identical second survey two
months after the first one, to test for intra-rater reliability.
Comparing the answers from similar respondent types from
the same site will assess inter-rater reliability.
Population Studied: The project is being done using primary
care practices within medical groups and their component
sites that are members of ICSI (the Institute for Clinical
Systems Improvement) in Minnesota. This collaborative for
quality improvement is sponsored by all of the health plans in
the state and currently has 40 group members ranging in size
from the Mayo Clinic and Mayo Health System to a single site
rural medical group practice of eight family physicians. The
groups vary widely in their structure and governance.
Collectively ICSI groups include nearly 65% of the physicians
in the state and an even higher proportion of those in primary
care. While some representatives of these medical groups
may have some understanding of the PCM, the great majority
of clinical personnel will not be familiar with it. .
Principal Findings: The testing of the PSAS involved review
by an expert advisory panel; “alpha” testing in six practices
edited for face validity, understandability, and feasibility. The
instrument is currently being evaluated in twelve practices
with more than 200 physicians. The practices range from
single sites with two physicians to a 100-physician group with
12 sites. The specific analytical aims of the study are to assess
the PSAS on: inter-rater agreement/reliability within practice
level (e.g. practice site vs. medical group) and site, inter-rater
agreement and reliability within job type, intra-rater reliability,
inter-rater reliability for the audit and validity with respect to
audit results by job type, practice level, and practice size.
These analyses are being conducted for each of the PSAS
individual items and domains. Results will also be aggregated
when possible across the various subgroups described (job
type, practice level, site size) to produce overall estimates of
reliability and validity of the PSAS. In addition, preliminary
results from the implementation of the POL in practice sites in
the GE BTE project will be presented.
Conclusions: We have successfully developed and tested two
instruments to reliably collect self reported or observationally
(audit) reported structural and process measures related to
the presence and functioning of systems in the office practice
setting that are linked via literature review to enhanced clinical
processes and outcomes. Further testing and correlation for
external validity (beyond the empiric data used to create the
tool) is underway in separate studies.
Implications for Policy, Delivery or Practice: The presence
of a set of reliable and valid measures of “systemness” should
be useful in research for assessing the level and functioning of
key clinical systems of care present before and after
interventions. In addition, given the barriers and complexity of
measuring clinical process and outcomes at the physician
office level, the PSAS could be used to evaluate and reward
performance at the office level by purchasers and plans. A
version of the PSAS, the POL, has already been adapted for,
and is being used in the GE led pay for performance project
entitled “Bridges to Excellence”.
• Improving Data Quality: A Parent-Completed Computer
Interview to Capture Medication History
Stephen Porter, M.D., M.P.H., Zhaohui Cai, M.D., Ph.D., Isaac
Kohane, M.D., Ph.D., Donald Goldmann, M.D.
Presented by: Stephen Porter, M.D., M.P.H., Assistant
Professor, Department of Medicine, Children's Hospital
Boston, 300 Longwood Avenue, Boston, MA 02115; Tel:
617.355.6624; Fax: 617.730.0335; E-mail:
stephen.porter@childrens.harvard.edu
Research Objective: The quality of information available to
clinicians impacts the successful implementation of guidelinesupported care. Physicians in the emergency department
(ED) face multiple barriers in gathering and using historical
data to support quality and reduce errors. We developed and
tested a bilingual, multimedia, touch-screen interface called
'the asthma kiosk' to examine parents' provision of
medication data. Our specific aims were: 1) To estimate the
validity of parents' electronically-entered medication history for
asthma, and 2) To compare the parents' kiosk entries to the
medication data documented by the triage nurse and ED
physician.
Study Design: We recruited a prospective cohort of parents to
use the kiosk and independently enter their children's detailed
medication history regarding name, route of delivery, form,
dose and frequency. Clinical providers were blinded to
parents' kiosk data. The gold standard (GS) for comparison
was a structured telephone interview conducted with parents
3-5 days after the ED visit during which parents gathered and
reviewed all asthma medications in the home. Report of a
specific medication was considered valid if it was both
accurate and complete according to the GS.
Population Studied: Parents of asthmatic children ages 1 year
to 12 years presenting to the ED of an urban Children's
hospital were eligible for enrollment.
Principal Findings: Sixty-six of 114 eligible parents (57.9%)
participated. Forty-nine of 66 parents (74.2%) completed the
GS interview. Data from 40/49 parent kiosk entries, from
40/49 nurse records, and from 47/49 physician records were
compared to the GS. The GS interview generated 99 instances
of medications across the 40 parent kiosk entries, 95
instances of medications across 40 nurse records, and 116
instances across 47 physician records. Parents documented
medication name with a significantly higher rate of validity
[92/99 (92.9%, 95% LCI 86.0%)] compared to both nurses
[54/95 (56.8%, 95% UCI 67.0%)] and physicians [85/116
(73.2%, 95% UCI 81.1%)]. For report of route, form and dose
of medications, the parents' kiosk entries demonstrated
significantly higher validity than documentation by nurses or
physicians. Twelve of 40 parents [30%, (95% LCI 16.6%)]
documented a valid detailed account of all asthma
medications compared to 0/40 nurse records [0%, (95% UCI
8.8%] and 0/47 physician records [0%, (95% UCI 7.6%)].
Conclusions: Parents' electronic report of asthma-specific
medications improves the validity of medication history as
currently documented by ED nurses and physicians.
Implications for Policy, Delivery or Practice: These findings
support a role for patients in quality improvement processes
and initiatives. Patients can be a reliable source of data to
inform computerized clinical guidelines as well as to promote
medication safety.
Primary Funding Source: AHRQ, Charles H. Hood
Foundation
• Medication Errors in the Pediatric Intensive Care Unit:
Risk Factors and Outcome
Joel Portnoy, M.D., Troy Dominguez, M.D., MSCE, Richard
Lin, M.D., M.S.CE, Timothy Yeh, M.D., Henry Glick, Ph.D.,
Jeffrey Silber, M.D., Ph.D.
Presented by: Joel Portnoy, M.D., Assistant Professor, Center
for Outcomes Research, Department of Anesthesiology and
Critical Care Medicine, Children's Hospital of Philadelphia,
3535 Market Street, Suite 1029, Philadelphia, PA 19104; Tel:
215.590.5758; Fax: 215.590.2378; E-mail:
portnoy@email.chop.edu
Research Objective: Medication errors are prevalent in the
Pediatric Intensive Care Unit (PICU) due to patients' severity
of illness, a wide range of diagnoses, comorbid diseases, and
therapies, and weight-based dosing. This study will describe
medication errors in the PICU, assess patient factors
associated with those errors, and analyze the effect of errors
on outcome by combining data from clinical, administrative,
and continuous quality improvement (CQI) databases.
Study Design: This IRB approved cohort study includes
patients admitted to the PICU of two different children's
hospitals. Patients were identified as having been exposed to a
medication error using the incident reporting process of the
hospitals’ CQI systems. Demographic information, diagnoses,
and procedures were obtained from the hospitals’
administrative databases; clinical information was recorded in
departmental databases; and medication error event data were
collected in the hospitals’ CQI databases. CQI department
staff identified the medication delivery process where the error
occurred and classified the error type. Physicians trained in
pediatric critical care medicine assigned severity of the error
blinded to the actual patient outcome using the National
Coordinating Council for Medication Error Reporting and
Prevention algorithm. We used logistic regression with data
from all three sources to study the effect of patient factors on
exposure to errors, and to assess the risk of ICU mortality
associated with exposure, while controlling for confounding by
demographics, diagnosis category, and severity of illness.
Population Studied: Patients admitted to two tertiary care
pediatric intensive care units between 1999-2002.
Principal Findings: There were 11,828 admissions to the
PICUs and 581 reported medication errors during the study
period. Errors occurred during prescribing (12.6%),
preparation and dispensing (16.0%), and administration
(68.9%). Errors were classified as wrong dose (46.5%),
medication (28.1%), time (19.8%), route (4.3%), or patient
(1.4%). Twenty-seven percent of the errors had the potential
for temporary harm, and 6.2% had the potential for
permanent harm or death. There were 7116 patients admitted
to the PICUs during the study period with complete data, and
305 medication errors identified in those patients. Risk of a
medication error was significantly associated with increased
age, longer length of stay, and occurrence of a procedure. A
trend for increased risk was associated with severity of illness
and the diagnoses asthma and soft-tissue malignancy.
Exposed patients had a significantly increased risk of death
(adjusted odds ratio 4.2, 95% confidence interval 2.7-6.5,
p<0.001).
Conclusions: Medication errors are associated with higher
odds of death. Longer length of stay and specific patient
demographic and clinical factors are associated with exposure
to error. While not necessarily causative, as errors may be
more likely near death, this study adds to our understanding
of the determinants of medication errors and may aid in
decreasing their occurrence and improving outcomes.
Implications for Policy, Delivery or Practice: Medication
errors are common in the pediatric intensive care unit
environment and increase the risk of an adverse outcome.
Structure and process improvements aimed at decreasing the
incidence of errors in the PICU and improving care after an
error occurs are clearly needed.
Primary Funding Source: AHRQ
• Medi-Cal HMO Promotion of STD Guidelines and
Delivery of STD Care by Contracted Physicians
Nadereh Pourat, Ph.D., Jas Nihalani, M.P.H., Gail Bolan, M.D.
Presented by: Nadereh Pourat, Ph.D., Senior research
Scientist, UCLA Center for Health Policy Research, 10911
Weyburn Avenue, # 300, Los Angeles, CA 90024; Tel:
310.794.2201; Fax: 310.794.2686; E-mail: pourat@ucla.edu
Research Objective: To examine whether the promotion of
best STD care as identified in CDC and other existing
guidelines improves the delivery of STD services by physicians
contracted with Medi-Cal HMOs
Study Design: A cross section of primary care physicians
(PCPs) that contracted with Medi-Cal HMOs in California
were surveyed on their delivery of STD services that
corresponded to the Centers for Disease Control and
Prevention (CDC) and other relevant guidelines. Dependent
variables included the frequency (always, usually or sometime,
to rarely or never) they screened females 15-25 years of age or
women over 25 with a history of STD or multiple sexual
partners; presumptively treated for chlamydia in presence of
gonorrhea; observed patients taking the medication while in
the office; followed up with patients to see if medication was
taken; provided the patients with medications for the partner;
treated the partners regardless of payment or enrollment in
plan; treated minors without parental consent; notified the
health department for partner notification; and counseled
patients to notify the partner.
Almost all (19/20) Medi-Cal HMOs in the corresponding
counties were also surveyed on their promotion of guidelines
and STD medications. Dependent variables included PCP
adherence to guidelines (screening, treatment, reporting, and
counseling). Independent variables were the presence of
guidelines on the same topics from the contracted HMOs as
reported by HMOs. Logistic models identified the probability
of physician adherence given the HMO’s recommendation.
Each model was controlled for PCP characteristics such as
specialty, experience, practice setting, Medi-Cal patient load,
recent STD training.
Population Studied: A cross-section of PCPs contracting with
Medi-Cal HMOs in eight California counties with the largest
numbers of Medi-Cal recipients and highest rates of
Chlamydia were surveyed. A total of 948 PCPs participated in
the telephone survey, with a response rate of 40%
Principal Findings: Preliminary analyses show that PCPs with
only one Medi-Cal HMO affiliation are more likely to screen 15
to 25 year old female patients annually for chlamydia. No
other HMO guidelines seemed to change the likelihood of
PCP practices. Rather, the determinants of consistently
following guidelines by PCPs in California’s Medi-Cal HMO
varied depending on the guidelines.
Conclusions: Although many HMOs reported having STD
screening, treatment, and partner management guidelines,
only HEDIS chlamydia guidelines are effectively
communicated to PCPs.
Implications for Policy, Delivery or Practice: This study
confirms that the presence of HEDIS chlamydia screening
guidelines has impacted screening by PCPs. Effective
communication of additional STD clinical guidelines by
HMOs may improve overall STD prevention and treatment by
PCPs.
Primary Funding Source: California STD Control Branch
• Reducing the Risk of Financial Exploitation of Older
Persons
Donna Rabiner, Ph.D., M.H.A., David Brown, MA
Presented by: Donna Rabiner, Ph.D., M.H.A., Senior
Researcher and Health Policy Analyst, Aging, Long-term Care
and Disablement Group, RTI International, 3040 Cornwallis
Road, PO Box 12194, RTP, NC 27707; Tel: 919.541.1220; Fax:
919.990.8454; E-mail: rabiner@rti.org
Research Objective: The purpose of this federally funded
study is threefold: (1) to develop a conceptual model to guide
an operational definition of financial exploitation of older
persons; (2) to describe effective approaches that: (a) identify
risk factors for financial exploitation; (b) prevent financial
exploitation; and (c) intervene to stop financial exploitation;
and (3) to recommend areas for policy development, data
collection, and research for the Congress, federal agencies,
state legislature, state agencies and other groups.
Study Design: Multiple method approach to data collection
and analysis, including: in-depth interviews with 25 national
experts, consultation with a national panel of technical
experts, site visits, comprehensive literature reviews, and
analysis of agency websites.
Population Studied: Older persons at risk of being financially
exploited and older victims of financial abuse.
Principal Findings: This study provided three formal
deliverables to the federal government: (a) a comprehensive
literature review and conceptual framework, which may be
used theoretically or empirically; (b) a compendium of best
practices and model federal, state and local programs
throughout the country; and (c) a series of formal
recommendations to the field to improve our nation's ability
to prevent financial exploitation of older persons, to identify
and support individuals who have been financially exploited,
and to quickly and effectively intervene to stop financial
exploitation of older persons.
Conclusions: A series of 13 formal recommendations,
presented under the following 3 headers---research
recommendations, service delivery system changes, and legal
protections---will be described as part of this presentation.
These formal recommendations have been submitted to the
U.S. Congress for further study and subsequent consideration.
Implications for Policy, Delivery or Practice: This study
contributes to the field by providing the federal government
with a better understanding of the current problem, policies,
practices and issues in the area of financial exploitation of
older persons. With the information from this study in-hand,
the Department of Health and Human Services is better able
to determine the appropriate federal role in providing an
effective public policy response to the financial exploitation of
older Americans.
Primary Funding Source: Dept. of Health and Human
Services
• Osteoporosis Management in Medcare+Choice Health
Plans
Philip Renner, M.B.A., Russell Mardon, Ph.D.
Presented by: Philip Renner, M.B.A., Director, Measures
Development, , National Committee for Quality Assurance,
2000 L Street, N.W., Washington, DC 20036; Tel:
202.955.5192; Fax: 202.955.3599; E-mail: renner@ncqa.org
Research Objective: To evaluate the effectiveness of
assessment and treatment of osteoporosis in women enrolled
in Medicare+Choice plans, and to test the deployment of a
potential new HEDIS measure for M+C plans.
Study Design: Observational study conducted in five M+C
plans. A clinical expert panel was convened to develop
measurement specifications based on existing guidelines.
Appropriate followup for the purpose of secondary prevention
of osteoporotic fracture was defined as use of bone mineral
density testing or use prescription of agents to treat
osteoporosis within the six months after index fracture.
Analyses were run to calculate followup rates, and to
determine the effects of prior treatment, use of HRT, and
fracture site on rates of followup
Population Studied: Administrative claim, encounter, and
pharmacy data were collected on 4,876 women age 67 years
and over enrolled in five M+C plans who suffered a fracture
between 7/1/00 and 6/30/01. Medical records were examined
for 500 of these patients.
Principal Findings: We found a relatively low rate of
evaluation and treatment following fracture in the study
population. Among patients who had not been receiving
treatment for osteoporosis, between 11% and 14% of patients
received appropriate followup after fracture. Including those
patients being treated with medication at the time of fracture,
the rate of appropriate followup increased to between 30%
and 41%. Of those patients on medication at the time of
fracture, 64% were taking HRT.
Conclusions: As defined in our study fewer than 15% of
patients over 65 who are not on treatment for osteoporosis
and suffer a fracture receive appropriate followup. This
finding is consistent with prior literature, and represents an
opportunity for improvement. Performance is higher when
including patients being treated at the time of fracture, but is
still well below optimal rates and is confounded by the use of
HRT in this population.
Implications for Policy, Delivery or Practice: Despite the
development of effective methods to evaluate osteoporosis
and effective medications for the treatment of osteoporosis,
physicians are not following up in a manner consistent with
guidelines. Performance measurement through a new HEDIS
measure for appropriate followup of fracture in older women
can be used to highlight low followup rates and to improve
assessment and treatment of osteoporosis in this population.
Primary Funding Source: CMS
• Glaucoma Screening in Medcare+Choice Health Plans
Philip Renner, M.B.A., Lok Wong, MHS, Russell Mardon,
PHD, Phillipe Gwet, Ph.D.
Presented by: Philip Renner, M.B.A., Director, Measures
Development, Performance Measure Development, National
Committee for Quality Assurance, 2000 L Street, N.W.,
Washington, DC 20036; Tel: 202.955.5192; Fax: 202.955.3599;
E-mail: renner@ncqa.org
Research Objective: To evaluate the frequency of biennial eye
examinations to screen for glaucoma among people age 65
and older enrolled in Medicare+Choice plans, and to test the
deployment of a potential new HEDIS measure for M+C
plans.
Study Design: Observational study conducted in five M+C
plans. A clinical expert panel was convened to develop
measurement specifications based on existing guidelines.
Using administrative data, glaucoma screening was identified
using CPT coding for eye examinations administered by eye
care professionals including codes for Evaluation and
Management, General Ophthalmological Services, Visual Field
Testing, and Optic Nerve Imaging. Analyses were run to
calculate screening rates, and to determine the validity of
administrative data as compared to the medical record.
Population Studied: Administrative claim, encounter, and
pharmacy data were collected on 69,555 people age 67 years
and over enrolled in five M+C plans between 1/1/2001 and
12/31/2002. Medical records were examined for 756 of these
patients.
Principal Findings: Approximately 75% of the members in the
study population received some form of glaucoma screening,
with plan-specific rates using adminstrative data ranging from
22.8% to 93.3%. Men and women received screening
examinations at similar rates. Screening rates increased from
age 65 to age 79, and then decreased. Members who had at
least one eye related condition, including cataract and macular
degeneration, received screening at a higher rate than those
with no known eye-related conditions. People with diabetes
received screening at a lower rate than the general population.
Concordance between administrative data and medical
records for eye-related conditions was high, ranging from 94%
for glaucoma to 71% for cataract. Concordance between the
administrative data and the medical records for glaucoma
screening examinations was greater than 80% for most plans,
except for one plan with 59% concordance.
Conclusions: Three quarters of the M+C members in our
study were screened for glaucoma as measured using
administrative data. However, there is significant variation
among health plans, indicating room for improvement in
screening rates. The screening rates in the study exceeded
those found in Fee for Service Medicare, which are estimate to
be 46-48%. Two of the study sites were unable to capture a
significant number of screenings in the administrative data
that were present in the medical records, implying a need to
improve data capture in electronic systems. This finding is
consistent with prior literature, and represents an opportunity
for improvement in helath plan data systems.
Implications for Policy, Delivery or Practice: Despite the
promulgation of guidelines recommending glaucoma
screening by specialty societies and public health agencies,
screening rates show wide variation and significant room for
improvement. Performance measurement with a HEDIS
measure for glaucoma screening can be used to demonstrate
variable screening rates and to improve data capture for
population management.
Primary Funding Source: CMS
• Estimating Increased Mortality, Cost, and Length of Stay
Associated with Safety Events: An Application of Patient
Safety Indicators in the Veterans Health Administration
Peter Rivard, M.H.S.A, Shibei Zhao, M.P.H., Susan Loveland,
M.A.T., Cindy Christiansen, Ph.D., Anne Elixhauser, Ph.D.,
Amy Rosen, Ph.D.
Presented by: Peter Rivard, M.H.S.A, Research Associate,
Center for Health Quality, Outcomes, and Economics
Research, Veterans Health Administration, 200 Springs Road,
#152, Bedford, MA 01730-1114; Tel: 781.687.3573; Fax:
781.687.3106; E-mail: rivardp@bu.edu
Research Objective: Practical tools are being developed to
measure potential patient safety events and to evaluate their
likely impact. The Agency for Healthcare Research and Quality
(AHRQ) Patient Safety Indicators (PSIs) use inpatient
administrative data to identify hospitalizations where medical
harm may have occurred. The purposes of this study are to:
(1) use PSIs to estimate the frequency of safety events and
their impact on mortality, cost of care, and length of stay in
the Veterans Health Administration (VHA) system, and (2)
compare these findings with those from AHRQ data on nonVA hospitals.
Study Design: We examine hospital discharge abstracts for
occurrence of PSIs using cross-sectional data from fiscal 2001.
For each of sixteen PSIs, we employ two independent
methods—regression analysis and a case-matching
protocol—to predict the effect of the PSI on cost, length of
stay (LOS), and mortality. Age, sex, race, and comorbidities
are controlled for; the comorbidity index employs a previously
validated 27-item scale.
Population Studied: VHA hospital discharges (n=439,537)
and AHRQ hospital discharges (Healthcare Cost and
Utilization Project Nationwide Inpatient Sample, or NIS;
n=7.45 million) are examined.
Principal Findings: Compared to hospitalizations without
PSIs, the presence of a PSI in a hospitalization is associated
with significant increments in cost, LOS, and mortality for two
of the three PSIs tested thus far. (1) Within the VHA,
hospitalizations with the post-operative pulmonary embolism
or deep-vein thrombosis (PE/DVT) PSI present have 88
percent higher cost per hospitalization, 90 percent greater
LOS, and 3.75 times greater odds of in-hospital death, relative
to those in the risk pool for the PSI but with no PSI present.
The rate of PE/DVT PSIs in the VHA is 10.30 per thousand,
higher than the NIS rate of 9.34 per 1000. (2) The presence of
the decubitus ulcer PSI is associated with 59 percent higher
cost per hospitalization, 55 percent greater LOS, and 3.18
times the odds of dying during hospitalization. The rate of
decubitus ulcer PSIs in the VHA is 15.78 per 1000, lower than
the NIS rate of 21.51 per 1000. (3) The VHA rate of
complications of anesthesia is 0.56 per 1000, lower than the
NIS rate of 0.71 per 1000. The effects of this PSI on cost,
LOS, and mortality are not significant, possibly due to
insufficient power from the small number of cases.
Conclusions: Safety events are likely to account for significant
increases in cost, LOS, and mortality. The impact on cost and
LOS appears substantial for certain safety events. Preliminary
comparison between health systems suggests that VHA rates
of safety events are generally no worse than non-VHA rates
and that relative rates depend on the safety event examined.
Implications for Policy, Delivery or Practice: Data
availability and cost-effectiveness make the PSIs a useful tool
for research and further evaluation. PSIs facilitate
benchmarking within and across systems, particularly within
health care systems that have relatively uniform data. By
demonstrating the value of PSIs in the VHA, the largest health
care system in the U.S., this study helps pave the way for use
of PSIs in other large health care systems for safety and quality
improvements.
Primary Funding Source: VA
• What Do the Publicly Reported Outcomes Available from
Home Health Compare Tell Us about Quality?
Robert Rosati, Ph.D., Huei-Ling Chen, MA
Presented by: Robert Rosati, Ph.D., Director of Outcomes
Analysis and Research, Center for Home Care Policy &
Research, Visiting Nurse Service of New York, 5 Penn Plaza,
11th Floor, New York, NY 10001; Tel: 212.609.5776; Fax:
212.290.3756; E-mail: robert.rosati@vnsny.org
Research Objective: In November 2003, the Centers for
Medicare and Medicaid Services (CMS) publicly released data
on 11 outcome measures based on patients who had received
care from home health agencies throughout the country to
help consumers decide where to get the best services. To
allow for meaningful comparisons across agencies, all of the
outcome measures are case mix adjusted by CMS. However,
there are no recognized clinical benchmarks for these
outcomes and no information available about the extent to
which outcomes could vary based on agency characteristics.
The present study explored the variation that exists in
outcomes based on the location of agencies in the country,
profit status and the variety of services offered by the agencies.
Further, using data from New York State a comparison was
made between traditional agencies and agencies that provide
care to long-term care patients.
Study Design: National data on the following OASIS-based
outcomes were analyzed: improvement in
ambulation/locomotion, improvement in transferring,
improvement in toileting, improvement in pain interfering
with activity, improvement in bathing, improvement in
management of oral medications, improvement in upper body
dressing, stabilization in bathing, acute care hospitalization,
any emergent care provided and improvement in confusion
frequency. Comparisons were made between geographic
regions, urban vs. rural, not-for-profit vs. for-profit agencies
and long term vs. traditional agencies using descriptive,
multivariate and bivariate analyses.
Population Studied: All agencies in the Home Health
Compare database (n=6,947) were included. The file
contained information on patients served from January 2002
to December 2002. The aggregate data for each agency
comprised all adult (age >=18) home healthcare patients
whose care was covered by Medicare or Medicaid and
provided by a Medicare certified home health agency.
Principal Findings: Regional variations indicated better
outcomes for agencies in the western states (p<.01). Urban
agencies had better outcomes on most measures (p<.01).
The types and number of services offered by agencies
significantly impacted patient outcomes (p<.01) in a positive
direction. Traditional agencies had (p<.01) better outcomes
for all indicators compared to the long term home health
agencies. Lastly, overall, not-for-profit agencies performed
better than for-profits (p<.01), except in management of oral
medications and stabilization in bathing.
Conclusions: There was considerable variability in home
health outcomes across the country and by the type of agency
providing care. However, it is difficult to determine whether
these better outcomes truly reflect that some agencies are
providing better care or whether the case mix adjustment does
not fully account for differences in patient characteristics. This
issue is particularly evident in New York State where the longterm agencies are performing much worse than traditional
agencies.
Implications for Policy, Delivery or Practice: The current
study is a first step towards evaluating the efficacy of Home
Health Compare, highlighting the importance of defining
suitable measures of quality of care in home health care,
examining the value of public reporting and making the
appropriate adjustments in case mix when reporting on
variation in patient outcomes. Refinements in these areas
may allow the development of a more robust method by
which we can more accurately compare performance.
• Into the Big Muddy and Out Again: Error Persistence and
Crisis Management in the OR
Jenny Rudolph, Ph.D.
Presented by: Jenny Rudolph, Ph.D., Research Scientist,
Management Decision and Research Center, US Department
of Veterans Affairs, 150 South Huntington Avenue, Boston,
MA 02130-4893; Tel: 617.638.5064; Fax: 617.638.5374; E-mail:
JRudolph@bu.edu
Research Objective: This study seeks to understand and
reduce diagnostic errors by anesthesiologists during medical
crises in the OR. The study focuses on fixation error, the
phenomenon of clinging to a single presumed diagnosis
despite mounting cues that one is on the wrong track.
Transcending fixation error and managing the crisis effectively
requires a transition from a routine operating mode, where the
underlying situation is assumed to be known, to an errorcorrecting mode that questions one’s own view of the
situation. The study seeks to identify systematic patterns in
problem solving that help or hinder clinicians in discovering
and treating the critical clinical problem.
Study Design: First, the study used at quasi-experiment to
test the effectiveness of a fixation-reduction training program
developed by Harvard University medical faculty in
collaboration with the author. The goal of the training
program was to increase self-correcting communication and
diagnostic tactics, which in turn were predicted to reduce
fixation. Decreased fixation was hypothesized to improve
resolution of the clinical problem (a blocked endotracheal
tube). The research site is the Center for Medical Simulation
in Boston which is a fully outfitted OR staffed by clinician
actors; the “patient” is a computer-controlled plastic
mannequin with heart, lung, and voice sounds, and
pharmacological reactions to about 100 medications. Second,
to analyze the types and flow of problem-solving, the project
used “process tracing,” a method from human factors
research. We also analyzed phrase-by-phrase, action-by-action
codes of diagnostic, therapeutic and communication tactics.
Videos of the crisis scenarios and their transcripts (including
dialogue, diagnostic and therapeutic actions, and vital signs)
provided the raw data.
Population Studied: 39 mostly second- and third-year
anesthesiology residents from Boston-area teaching hospitals
participated in this study. 40% were women, 60% were men;
62.5% were of European extraction, 38.5% were of East or
South Asian origin.
Principal Findings: The treatment condition (received the
new training or not) had no statistically significant impact on
the enactment of self-correcting behaviors. There was,
however, a significant inverse relationship between the
enactment of self-correcting behavior and degree of fixation.
There was a strong inverse relationship between fixation and
the outcome variable of clinical problem resolution. The
study identified four distinctive problem-solving modes:
“Effective” (23% of cases) in which the clinician generates and
tests diagnoses systematically; “Diagnostic Vagabonding,”
(44%) in which the clinician jumps from diagnosis to
diagnosis without systematically treating or ruling any one out,
“In-the-Doldrums,” (5%) in which the clinician neither
generates diagnoses nor treats symptoms; and “Fixation”
(28%) in which the clinician focuses on treating one diagnosis
and ignores other possible etiologies.
Conclusions: This study finds that, in a crisis, 75% of
advanced anesthesiology residents did not or could not
produce the canonical problem-solving approach of
systematically generating and testing diagnoses needed to
help the patient. The crisis scenario under study was based
on a standard airway problem that residents had the
knowledge and skill to remedy. This suggests that failures to
handle OR crises effectively may be related not to weaknesses
in technical medical knowledge but rather to limits in
reflection, communication, and “metacognition” skills that
allow clinicians to detect and correct their errors quickly on the
fly.
Implications for Policy, Delivery or Practice: Safety and
reliability in diagnostic OR crises are enhanced by learning to
make one’s thinking “visible:” articulating assumptions and
action plans and seeking corrective input. Yet these skills run
counter to professional norms in medicine that equate
competence with certainty. Performance in OR crises might
be improved if house staff in residency programs legitimized
collaborative, public testing of existing diagnoses and a
willingness to be wrong by modeling these skills and
supporting trainees in developing them.
Primary Funding Source: National Patient Safety Foundation
• Carotid Endarterectomy Utilization and Mortality in Ten
States
Shadi Saleh, Ph.D. M.P.H., Edward Hannan, Ph.D.
Presented by: Shadi Saleh, Ph.D. M.P.H., Assistant Professor,
Health Policy, Management and Behavior, SUNY-Albany, One
University Place, Rensselaer, NY 12144; Tel: 518.402.0299; Fax:
518.402.0414; E-mail: ssaleh@albany.edu
Research Objective: Studies that examined the rates of and
mortality following carotid endarterectomy (CEA) mainly were
confined to a limited geographical location and/or population.
The primary purposes of this study are to examine the
variation of risk-adjusted in-hospital mortality rates following
CEA in ten states, and utilization rates per capita of CEA.
Study Design: An analysis of hospital discharge data from ten
states extracted from the Agency for Health Research and
Quality’s national database, Healthcare Cost and Utilization
Project (HCUP) was conducted.
Population Studied: Patients who had CEA (code 38.12 of the
International Classification of Diseases, Ninth Revision) listed
as the principal procedure were identified and included in the
study.
Principal Findings: The rates of CEA per capita were found to
differ among the ten states examined. No significant
association was detected between geographic location and the
adjusted risk of in-hospital mortality. Gender, age, type of
admission and several comorbidities were found to be
significant risk factors.
Conclusions: Rates of CEA per capita differ among states.
However, geographical location does not affect the likelihood
of risk-adjusted mortality following the procedure.
Implications for Policy, Delivery or Practice: It is important
to examine regional differences in prevalence and mortality
rates of major procedures. Findings from such studies can
serve as benchmarks of access and quality that states
(regions) can measure their performance against.
• Cost-Effectiveness of Case Management in Substance
Abuse Treatment
Shadi Saleh, Ph.D. M.P.H., Thomas Vaughn, Ph.D., Samuel
Levey, Ph.D., Laurence Fuortes, Ph.D., Tanya Uden-Holmen,
Ph.D., James Hall, Ph.D.
Presented by: Shadi Saleh, Ph.D. M.P.H., Assistant Professor,
Health Policy, Management and Behavior, SUNY-Albany, One
University Place, Rensselaer, NY 12144; Tel: 518.402.0299;
E-mail: ssaleh@albany.edu
Research Objective: The purpose of this study, which is part
of a larger clinical trial, was to examine the cost effectiveness
of case management for individuals treated for substance
abuse in a residential setting.
Study Design: Clients who agreed to participate were
randomly assigned to one of four study groups. Two groups
received face-to-face case management and one
telecommunication case management, while the fourth was
the control group.
Population Studied: Clients in the residential treatment
program were recruited to the study if they met any of the
following criteria: 1) had more than one drug or alcohol related
offense, 2) had a breathalyzer test with a blood alcohol
content of 0.2 or higher, or 3) were involved in a drug or
alcohol related accident.
Principal Findings: Using a ratio of cost to days free from
substance abuse, the case management groups were less
cost-effective than the control group at 3 months, 6 months,
and 12 months. The telecommunication case management
was least cost-effective of the three case management
conditions.
Conclusions: Results from the analysis revealed case
management is not cost effective as a supplement to
traditional drug treatment over a 12-month follow-up period.
Implications for Policy, Delivery or Practice: This study
emphasizes the need for use of evidence-based interventions
with substance abuse treatment clients whenever possible.
Clinical wisdom can guide when data are not available, but
studies on effectiveness and cost-effectiveness should help
identify these evidence-based interventions and lead to better
methods to evaluate these models. As public and private
agency budgets respond to the changing goals of funding
agencies, data on effectiveness and cost-effectiveness will
become even more important and possibly required in the
near future.
Primary Funding Source: NIDA
• Measuring Access to Behavioral Health Care: Challenges
and Opportunities
Sarah Sampsel, M.P.H., John Ludden, M.D., Russell Mardon,
Ph.D., Philippe Gwet, Ph.D., Philip Renner, M.B.A.
Presented by: Sarah Sampsel, M.P.H., Senior Health Care
Analyst, Quality Measurement, National Committee for
Quality Assurance, 2000 L Street, N.W., Suite 500,
Washington, DC 20036; Tel: 202.955.1716; Fax: 202.955.3599;
E-mail: sampsel@ncqa.org
Research Objective: To explore the development and
evaluation of a performance measure focused on access to
behavioral health services. Studies suggest a gap between the
prevalence of behavioral health disorders and the use of
behavioral health services, and there are few tools available to
measure access to behavioral health. Failures of adequate
access to behavioral health care may result in increased
morbidity and suffering.
Study Design: Observational study conducted in four health
plans. A technical expert panel was convened to develop
measurement specifications based on consensus and
literature supporting the efficacy of treatment for behavioral
health conditions. Access was defined as the ability of
members to get the services they require from a health care
system. Analyses were run to calculate the length of time
between a plan authorization for care and behavioral health
visit, standard length of time to receive visits based on
diagnosis and provider specialty, and variances between plan
type (MCO vs. MBHO), gender, and diagnosis.
Population Studied: Four health plans participated in the
field test by providing patient-level administrative data to
NCQA under the terms of a formal data-sharing agreement.
The enrollments of these plans ranged from fewer than
10,000 to nearly 3.5 million individuals. The plans included
Managed Behavioral Health Organizations (MBHOs) and
Managed Care Organizations (MCOs), and were located in
several geographic regions of the country.
Principal Findings: We found the prevalence of contacts to
plans to initiate behavioral health services lower than the
population prevalence estimates for behavioral health
conditions. In addition, the percentage of members that
received routine behavioral health services within 14 days was
lower than expected based on information supplied by three of
the four field-test sites which indicated they had been using an
internal metric to measure visit timeliness with resulting rates
close to 95%. For the field-test, the range of performance for
members receiving services within 14 days of a
contact/authorization ranged from 34 – 69% for commercial
plans The ranges were similar for both Medicaid and
Medicare plans.
Conclusions: Access to care is a vitally important but difficult
construct to measure and interpret.
Evidence based guidelines concerning access are rare and
determining the amount of services needed relies heavily on
individual characteristics. It is also difficult to define “good”
access except in comparison to other utilization data. And
measuring “bad” access requires measuring a service or visit
that was needed but did not occur.
Implications for Policy, Delivery or Practice: The visit
timeliness measure is available to systems for use in
assessing and improving access to care. As a tested and
standardized measurement tool, it may be valuable to certain
systems of care, including Medicaid in some states. Because
of system, benefit, and referral variations, the visit timeliness
measure cannot provide a basis for comparability and is
unsuitable for inclusion in the HEDIS framework. As part of
an overall strategy for measuring and improving the ability of
patients to obtain the behavioral health care that they desire,
this tool provides an important scaffold around which to build
other relevant measures that can build assurance of access to
needed behavioral health services.
Primary Funding Source: HRSA
• Developing Performance Measures of Over-use of Health
Care Services
Sarah Sampsel, M.P.H., Russell Mardon, Ph.D., Rich
Mierzejewski, M.B.A., M.S., Philip Renner, M.B.A.
Presented by: Sarah Sampsel, M.P.H., Senior Health Care
Analyst, Quality Measurement, National Committee for
Quality Assurance, 2000 L Street, N.W., Suite 500,
Washington, DC 20036; Tel: 202.955.1716; Fax: 202.955.3599;
E-mail: sampsel@ncqa.org
Research Objective: There is a poor correlation of x-ray
findings with low back problems; additionally, the costs of
imaging studies, especially those that may not be warranted
contribute to the increasing burden of health care costs to
society. Consequently, patients with low back pain and no
indicators of serious pathology should not receive an imaging
study during the first four to six weeks of a new episode. A
performance measure assessing the appropriate use of
imaging studies for health plan members with acute low back
pain was developed.
Study Design: Observational study conducted in three health
plans. A clinical expert panel was convened to identify and
develop measure specifications based on guidelines and
evidence. In addition to measuring the rate of imaging
studies for patients with low back pain, data analysis was
conducted to assess the following: reliability of identifying low
back pain episode start dates and duration, prevalence and
impact of exclusionary diagnoses (i.e. red flags), and the mean
time to imaging for low back pain patients.
Population Studied: Three health plans participated in the
field test by providing patient-level administrative and medical
record data. The enrollments of these plans ranged from
approximately 150,000 to 500,000. The plans included
network models and staff models, and were located in several
geographic regions of the country. Health plan populations
included both commercial and Medicaid members.
Principal Findings: We found inappropriate imaging in
approximately 22 percent of low back pain episodes indicating
the potential for improvement. The measure rates were fairly
consistent across plan, product line, age, and sex. The rate of
imaging for orthopedic surgeons (63.8%) was significantly
higher than those for primary care providers (19.6%).
Conclusions: Low back pain is a highly prevalent condition in
the working population and is a leading cost driver for direct
and indirect (lost productivity, absenteeism, etc.) costs. The
field-testing of the performance measure indicates the face
and content validity of the metric is moderate to high and
there is room for improvement. A threat to validity is due to 510% of patients with low back pain developing persistent back
problems. The condition tends to relapse, so most patients
will experience multiple episodes. This clinical feature poses
challenges in identifying the acute low back pain population,
however, recurrence of low back pain is not an indication for
imaging studies for most patients.
Implications for Policy, Delivery or Practice: Plans are
becoming increasingly interested in measuring
appropriateness of procedures and diagnostic testing and
their relation to outcomes for their members. The
appropriateness of imaging studies is frequently mentioned as
a cost/over-utilization concern for plans. The routine and
standardized collection of this performance measure can
assist plans in benchmarking the utilization of imaging
studies and develop interventions to decrease utilization
where appropriate.
Primary Funding Source: Pharmaceutical - Corporate Grant
• Impact of Risk Communication on Tuberculin Skin
Testing for Users of Infliximab, a Biologic Therapy
Deborah Shatin, Ph.D., Nigel Rawson, Ph.D., M. Miles Braun,
M.D., M.P.H., Jeffrey Curtis, M.D., M.P.H., Larry Moreland,
M.D., Kenneth Saag, M.D., M.Sc.
Presented by: Deborah Shatin, Ph.D., Senior Researcher,
Center for Health Care Policy and Evaluation, UnitedHealth
Group, 12125 Technology Drive, MN002-0260, Minneapolis,
MN 55344; Tel: 952.833.7087; Fax: 952.833.7090; E-mail:
deborah_shatin@uhc.com
Research Objective: Infliximab, a TNF-a antagonist biologic
product used for the treatment of rheumatoid arthritis and
Crohn's disease, is associated with an increased risk of
tuberculosis (TB) based on spontaneous reports to FDA and
other data. Given various FDA, industry, and scientific efforts
to communicate this risk and to recommend tuberculin skin
testing (TST) for new users, the objective of this study was to
evaluate the impact of such efforts to improve patient safety.
National risk communication (RC) efforts included national
Dear Healthcare Provider (DHP) letters, revisions to the
package insert (such as text changes and a boxed warning),
and presented/published scientific reports of TB cases.
Study Design: We conducted a retrospective descriptive
cohort study of 1419 patients, identified from automated
claims data, who were exposed to infliximab between 1/1/00
and 6/30/02. Three successive cohorts over time were
studied to evaluate the impact of the risk communication
efforts.
Population Studied: Infliximab users were identified from 11
geographically diverse health plans within one health care
system. We analyzed medical claims for tuberculin skin
testing for patients with at least 3 months enrollment prior to
their Index Date (first known infliximab exposure). The three
cohorts based on Index Date were defined by successive RC
efforts: Cohort 1 (N=311, 1/1/00-10/31/00; no specific TB risk
communication done); Cohort 2 (N=642, 11/1/00-10/10/01;
risk communication included labeling change and DHP letter);
and Cohort 3 (N=466, 10/11/01 – 6/30/02; risk
communication included NEJM article and boxed warning).
Principal Findings: The rate of any tuberculin skin testing
increased from 15% (Cohort 1) to 23% (Cohort 2) and, finally,
31% (Cohort 3). Testing prior to exposure in the 3 cohorts as
recommended in various communications showed a greater
rate of increase, from 0% to 7% to 28% respectively;
conversely testing post-initiation of infliximab decreased from
15% (Cohort 1) to 3% (Cohort 3). Of the patients who received
the test, none of the tests in Cohort 1 were completed prior to
exposure to infliximab compared to 90% in Cohort 3. It
should be noted that each successive cohort had a shorter
follow-up time period than the previous one.
Conclusions: Although tuberculin skin testing rates after
various risk communications were still less than optimal,
which in part may be due to under-ascertainment of testing
that may have occurred, they suggest that these efforts had a
notable impact since TST rates doubled between 2000 and
2002. Even more importantly, the timing of TST with each
successive cohort increasingly occurred, as recommended,
prior to biologic exposure rather than during treatment.
Implications for Policy, Delivery or Practice: Since a
number of risk communicaton efforts took place concurrently,
we are not able to differentiate which specific one was most
effective or whether a multi-modality effort is necessary for
effective risk communication. However, it is possible that
multiple risk communication efforts may be important in
optimizing patient safety in the use of new therapies.
Primary Funding Source: Engalitcheff Initiative funded by the
Maryland Chapter of the National Arthritis Foundation
• Development of a Taxonomy: Patient and Provider
Perspectives on Health Care Quality
Steven Garfinkel, Ph.D., Karen Shore, Ph.D., Jim Lubalin,
Ph.D., Kristin Carman, Ph.D., Margarita Hurtado, Ph.D., Roger
Levine, Ph.D., Judy Mitchell, M.S.
Presented by: Karen Shore, Ph.D., Senior Research Scientist,
American Institutes for Research, 1791 Arastradero Road, Palo
Alto, CA 94304; Tel: 650.843.8121; Fax: 650.858.0458; E-mail:
kshore@air.org
Research Objective: To increase the validity of a CAHPS®
individual provider survey by using the Critical Incident (CI)
Technique to develop a complete taxonomy of the
components of quality ambulatory health care, based on both
patient and clinician perspectives. Specific objectives of this
study are to: 1) confirm that the domains measured by an
existing draft CAHPS® instrument are salient to patients and
physicians and can be assessed by patients; 2) identify any
salient and measurable domains that heretofore have not
been included in the instrument, but could be measured
through a consumer survey; and 3) determine that the
domains measured by the instrument are salient and
measurable for both men and women, for individuals with
different levels of education, and across a range of racial and
ethnic groups.
Study Design: The CI technique involves in-depth interviews
with clinicians (physicians, nurse practitioners, and physician
assistants) and patients to elicit examples of specific
behaviors that respondents characterize as having involved
either high quality care or low quality care. Provider and staff
behaviors reported to be responsible for the quality of care in
these encounters are analyzed and categorized, producing a
taxonomy of behaviors that constitute quality care. These
categories of behaviors are then used to develop reporting
domains and survey questions about each domain.
Population Studied: We are collecting incidents from 20
providers and 80 patients from each of two health plan
partners located in different geographic areas of the US (for a
total sample of 40 providers and 160 patients). Patient
respondents at each site are divided approximately equally
among four different racial/ethnic groups, to support
comparative analyses, and are also divided approximately
equally between females and males, to support gender
analyses. Although we are collecting incidents from a
relatively limited number of patients and providers, we
typically gather information on 10 or more incidents per
interview, and our sample size is based on the expectation we
would have 2,000 incidents (1,600 from patients and 400
from providers). All incidents are reviewed and used to build
the complete taxonomy.
Principal Findings: While this research is still in progress
(completion date of April 2004), we anticipate finding several
distinct categories of provider and staff behaviors that result in
high quality care.
Conclusions: A complete taxonomy of behaviors constituting
quality ambulatory provider care both validates and expands
the domains included in the draft CAHPS® provider survey.
Implications for Policy, Delivery or Practice: The CI
taxonomy is useful in identifying the domains of real concern
to patients, and the extent to which these domains differ as a
function of race/ethnicity. This information should be
invaluable for developing policies and practices that effectively
address the areas of greatest concern to patients. Based on
the results of this CI research, we believe that revisions and
additional use and testing of a CAHPS® provider survey are
appropriate. The revised instrument should be useful for
comparing the performance of individual practitioners, as well
as for quality improvement and pay-for-performance systems.
Primary Funding Source: AHRQ
• Appropriateness of Healthcare Received by Women with
Abnormal Pap Smears in the Los Angeles County
Healthcare System
Rita Singhal, M.D., Christine Holschneider, M.D., Mingming
Wang, M.P.H., Christine Aque, BA, Michael Broder, M.D.,
MSHS, Lisa Rubenstein, M.D., MSPH
Presented by: Rita Singhal, M.D., Women's Health and
Health Services Research Fellow, VA Greater Los Angeles
HSR&D Center of Excellence, 16111 Plummer Street,
Sepulveda, CA 91343; Tel: 818.895.9555; Fax: 818.895.9453;
E-mail: rsinghal@ucla.edu
Research Objective: To evaluate the appropriateness of
follow-up care received by women with abnormal Pap smears
in the Los Angeles County healthcare system. To determine
rates of women who are lost to follow-up and rates of women
receiving appropriate evaluation and treatment. To determine
regional variations and variations based on facility complexity
and size.
Study Design: This is a historical prospective study of a
cohort of women with abnormal Pap smears at Los Angeles
County healthcare facilities in 2000. These women were
followed through cytology and histology records from private
and hospital laboratories to determine all follow-up studies
that were performed through December 2002. An
appropriateness of care protocol was devised using current
standards of practice to determine what follow-up studies are
required within specific time intervals for each category of Pap
smear abnormality. The protocol was used to categorize each
case as optimal, tolerable or inadequate.
Population Studied: All women 18 years and greater that had
an abnormal Pap smear at a Los Angeles County healthcare
facility between January and December 2000. This included
8,557 women.
Principal Findings: 740 (30.7%) of the 2413 cases with an
abnormal Pap smear of AGUS, HSIL, squamous cell
carcinoma (SCC) and unsatisfactory received no follow-up
care. Follow-up rates were100% for SCC (n=25), 85.4% for
HSIL (n=479), 73.6% for unsatisfactory (n=1210) and 49.8%
for AGUS (n=699). Of the 1673 cases with follow-up, 1475
(88.2%) received optimal care, 112 (6.7 %) received tolerable
care and 86 (5.1%) received inadequate care. Rates of optimal
care were 92.8% for unsatisfactory cases, 88.5% for HSIL,
80% for SCC and 76.4% for AGUS.
Variations by facility complexity and size were: 411 (27.6%) of
cases in hospitals received no follow-up care compared with
198 (38.4%) cases at mid-sized clinics and 131 (31.9%) at
small clinics. Rates of optimal care were 90.7% at hospitals,
84.6% at the small clinics and 82.7% at mid-sized clinics.
Regional differences showed rates of loss to follow-up ranging
from 28.1% to 39.2% among four distinct areas in the county.
Rates of optimal care also varied from 51.1% to 65.7% with
one region consistently scoring higher for each category of
Pap smear abnormality.
Conclusions: In addition to an overall loss to follow-up rate of
30.7% for women with abnormal Pap smears, over 10% of
women received suboptimal care. There were variations in
appropriateness of care based on facility complexity and size,
with hospitals providing higher rates of optimal care
compared with smaller, less complex facilities. Regional
variations were present with facilities in one region
consistently providing more appropriate care.
Implications for Policy, Delivery or Practice: Providing
appropriate diagnostic evaluations for detected cervical
disease remains an important aspect of cervical cancer
prevention and early detection. This study provides insight
into the quality of care being provided to women with
abnormal Pap smears in a healthcare system serving a high
risk population. Information on the organizational factors that
influence the quality of care serves to identify specific
healthcare delivery practices for quality improvement
initiatives.
Primary Funding Source: VA
• Improving the Quality of Heart-Failure Care in Minority
Communities
Jane Sisk, Ph.D., Paul Hebert, Ph.D., Carol Horowitz, M.D.,
M.P.H., Mary Ann McLaughlin, M.D., M.P.H., Jason Wang,
Ph.D.
Presented by: Jane Sisk, Ph.D., Professor, Health Policy,
Mount Sinai School of Medicine, One Gustave L. Levy Place,
New York, NY 10029-6574; Tel: 212.659.9567; Fax:
212.423.2998; E-mail: jane.sisk@mssm.edu
Research Objective: Congestive heart failure, the leading
cause of hospitalization among elderly people,
disproportionately afflicts African-American and other nonwhite populations. Prevalence, hospitalization, and death are
greater among non-whites than whites. Patients often lack the
skills to manage this chronic condition, and do not receive
proven-effective therapies. Health plans and providers are
increasingly using nurse management, but the strategy has
not been rigorously evaluated for ambulatory patients or
adapted to ethnically-diverse, inner-city communities. This
study evaluated the effectiveness and cost-effectiveness of
nurse management for heart failure in Harlem.
Study Design: A randomized controlled effectiveness trial
allocated patients between nurse management and usual care,
and compared functioning and hospitalizations after the 12month intervention and 6-month follow-up. Bilingual nurses
counseled patients on the condition, low-salt diets, and
medication adherence, with regular telephone follow-up;
encouraged clinicians to prescribe ACE-inhibitors and betablockers at proven-effective doses; and facilitated social work
and insurance coverage. Patient surveys and provider billings
provided data on functioning, use, and cost. Longitudinal
analysis of differences in functioning used Generalized
Estimating Equations. Analysis of the cumulative count of
hospitalizations used a negative binomial model. Following
guidelines for cost-effectiveness analysis from a societal
perspective, cost calculations included the nurse-management
program, medical services, and patients’ time and
transportation, and discounted health effects and costs
beyond 1 year at 3%.
Population Studied: From clinics at the four hospitals in
Harlem, New York City, we enrolled 406 adults with
documented systolic dysfunction: 46% African-American or
Black, 33% Hispanic, 37% 65 years or older, and 47% female.
Principal Findings: Nurse-management patients had
significantly better functioning than usual-care patients at 12
months on the generic SF-12 Physical Component Score and
condition-specific Minnesota-Living-with-Heart-Failure
Questionnaire (p<.001). This effect, maintained 6 months
following the intervention, was equivalent to nursemanagement patients’ remaining at New York Heart
Association Class 3 functioning, while usual-care patients
declined to Class 2. Nurse-management patients also had
significantly fewer hospitalizations at 12 months (p=.054).
The nurse-management program averaged $2,177 per patient.
Cost based on patient surveys did not differ significantly
between nurse and usual-care patients, each about $22,400.
The final analysis will also include prescription drugs. The
cost-effectiveness analysis calculates the cost per qualityadjusted life year during the 12-month trial and through
patients’ life expectancies.
Conclusions: In diverse inner-city communities, nurse
management proved to be successful in achieving better
health-related outcomes for primarily-minority heart-failure
patients with systolic dysfunction. Nurse-management and
usual-care patients have not differed significantly in medical
costs, excluding prescription drugs.
Implications for Policy, Delivery or Practice:
Implementation of the nurse-management strategy in clinical
practice could achieve better health outcomes for ethnicallydiverse systolic-dysfunction patients. Even if the strategy is
cost-effective from a societal perspective, however, current
payment incentives may make it difficult to justify a business
case for providers to incorporate the program. Providers
would bear increased nurse costs, and might have decreased
revenue from fewer hospitalizations. Medicare payment for
such a disease-management program for this condition could
align incentives to adopt this strategy to improve the quality of
care.
Primary Funding Source: AHRQ
• Trust in Providers among Individuals with Poor Health
Maureen Smith, M.D., M.P.H., Ph.D., Kathryn Flynn, M.S.,
Jessica Bartell, M.D., M.S.
Presented by: Maureen Smith, M.D., M.P.H., Ph.D., Assistant
Professor, Population Health Sciences, University of
Wisconsin-Madison Medical School, 603 WARF, 610 Walnut
Street, Madison, WI 53726; Tel: 608.262.4802; Fax:
608.263.2820; E-mail: maureensmith@wisc.edu
Research Objective: Trust in a medical provider is an
essential part of effective health care delivery and has been
shown to improve health outcomes largely by affecting
adherence to treatment. For persons in poor health,
adherence to treatment is critical and disruptions in the
stability of the primary health care relationship may be felt
more acutely. However, it is unclear whether the importance
of a continuous usual provider to a trusting relationship
differs depending on patient health. Our objective is to
determine how patient trust relates to the stability of the
primary health care relationship for patients in good and poor
health.
Study Design: Cross-sectional data from the 2000-2001
Community Tracking Study Household Survey, a nationally
representative telephone survey of almost 60,000 civilian,
non-institutionalized individuals. The dependent variable was
trust, defined as, I trust my doctor to put my medical needs
above all other considerations when treating my medical
problems, coded on a five-point scale from strongly agree to
strongly disagree, and dichotomized as less than or equal to
strongly agree. Four categories measured decreasing stability
in the primary health care relationship: a continuous usual
provider and usual place of care over the last 12 months, a
usual provider and place of care with recent change in the last
12 months in one or both, a usual place of care but no usual
provider, and no usual place of care. Poor health was defined
as fair or poor on the 5-point scale of self-rated overall health.
Logistic regressions predicting low trust accounted for
complex survey design and adjusted for age, gender,
race/ethnicity, marital status, urban/rural, education,
household size, income, insurance type, preference for risk,
willingness to trade cost for provider choice, and satisfaction
with provider listening, explanation, and thoroughness.
Population Studied: 28,368 respondents aged 18-91 who
visited a provider in the previous 12 months.
Principal Findings: 27% of respondents reported lower trust
in their provider, and 18% of respondents reported poor
health. For patients in poor health, lacking a usual provider –
with or without a usual place of care – was associated with
lower trust when compared to patients with a continuous
usual provider and place: OR=1.86, CI=1.53-2.27 for patients
without a usual provider but with a usual place; OR=1.60,
CI=1.09-2.34 for patients lacking both a usual provider and
place. The association between lacking a usual provider and
lower trust was significantly greater for patients in poor health
than for those in good health, p-value for interaction =0.03.
Conclusions: Patients in poor health who lack a usual
provider have a greater likelihood of low trust when compared
to patients in good health.
Implications for Policy, Delivery or Practice: These data
suggest the need to distinguish patients in poor health when
examining the importance of a usual provider on trust and
adherence to treatment. Given the critical role of provider
trust in adherence to treatment, lacking a usual provider
should be viewed as a fundamental risk factor for patients in
poor health.
Primary Funding Source: AHRQ
• Inappropriate Increases in Prescribing of Lipid Lowering
Agents
Maureen Smith, M.D., Ph.D., M.P.H., Elizabeth Cox, M.D.,
M.S., Jessica Bartell, M.D., M.S.
Presented by: Maureen Smith, M.D., Ph.D., M.P.H., Assistant
Professor, Population Health Sciences, University of
Wisconsin Medical School, 610 Walnut Street, Madison, WI
53726; Tel: 608.262.4802; Fax: 608.263.2820; E-mail:
maureensmith@wisc.edu
Research Objective: Lack of adherence to guidelines is widely
recognized, including guidelines for hyperlipidemia
management. Much research has focused on increasing
appropriate prescribing of lipid lowering agents, but few
studies have examined inappropriate prescribing. We
examine inappropriate increases in prescribing of lipid
lowering agents as well as the relationship of these
inappropriate increases to physician characteristics, practice
characteristics, and incentives intended to shape physician
behavior.
Study Design: Longitudinal data from a subpopulation of
respondents to Rounds 1 (1996-1997) and 2 (1998-1999) of
the Community Tracking Study, a nationally-representative
survey of 17,740 direct patient care physicians.
Population Studied: Physicians, n=2170, responding
longitudinally to a vignette about the percentage of 50-year-old
men without other cardiac risk factors for whom the physician
would recommend an oral agent for a total cholesterol of 240,
LDL 150 and HDL 50 after six months on a low cholesterol
diet. Guidelines concurrent with both surveys (ATP2) would
not recommend prescribing an oral lipid lowering agent for
this patient. Logistic regression examined predictors of
inappropriate increases in prescribing, adjusted for complex
sample design and subpopulation analysis, with
0=appropriate recommendation for no prescribing in both
rounds and 1=inappropriate increase in prescribing between
the two rounds. 883 physicians who did not meet the
definition of inappropriate or appropriate prescribing were
excluded from logistic analysis. Explanatory variables
obtained from Round 1 and significant at p<0.20 included
physician characteristics—specialty, board certification, and
osteopathic physician; practice characteristics—solo or 2physician practice, number of managed care contracts,
percent practice revenue from Medicaid or Medicare, and
availability of quality ancillary services; as well as incentives—
having profiling affect salary, percent of patients for whom the
physician is gatekeeper, physician ability to make clinical
decisions in patients’ best interests without the possibility of
reducing his/her income, physician perception of the effect of
guidelines or reminders on his/her practice, as well as change
in severity or complexity of patients managed without referral
or the number of patients referred to specialists over the prior
two years.
Principal Findings: Among 2,170 longitudinal responders to
the vignette, mean prescribing increased 11% between the
surveys. In logistic analysis, family physicians, osteopathic
physicians, and those in solo or 2-physician practices were
significantly more likely to have inappropriate increases in
prescribing, as were physicians who perceived that reminders
had a large effect on their practices. Board certified physicians
and physicians who reported frequently obtaining quality
ancillary services such as nutritional counseling were
significantly less likely to have inappropriate increases in
prescribing, as were those who referred more patients to
specialists over the prior two years.
Conclusions: Inappropriate prescribing of lipid lowering
agents increased substantially during the two years between
surveys. Inappropriate increases were associated with
physician and practice characteristics, as well as physician
perceptions that reminders influenced their practices.
Implications for Policy, Delivery or Practice: Evaluation of
guidelines for hyperlipidemia management should consider
over-utilization as well as under-utilization of lipid lowering
agents. Further, evaluation of incentives to shape physician
behavior should consider unintended consequences such as
inappropriate prescribing.
Primary Funding Source: AHRQ
• Risk Management and Medical Liability
Dean Smith, Ph.D.
Presented by: Dean Smith, Ph.D., Professor and Chair, Health
Management & Policy, University of Michigan, 109
Observatory, Ann Arbor, MI 48109-2029; Tel: 734.936.1196;
Fax: 734.764.4338; E-mail: deans@umich.edu
Research Objective: Examine the relationships among
physician office systems, including resources devoted to risk
management programs and the medical liability experience of
physicians.
Study Design: A questionnaire focusing risk management
was developed and administered to a random sample of
physicians in Michigan, stratified by medical liability
experience. Mailing labels of physicians were obtained from a
medical liability insurance company. A post-card follow-up
was sent. We used counts of specific elements employed as
measures of office systems and use of specific tools and time
spent as measures of risk management activity. Numbers of
liability claims made against physicians were counted over the
past two years.
Population Studied: Physicians in Michigan with medical
liability insurance.
Principal Findings: A total of 882 valid questionnaires were
mailed and 393 fully completed for a 45% response rate.
Characteristics of responding physicians are similar to the
Michigan physician population. Michigan physicians report
having increased numbers of office staff, having purchased
new computer systems and having changed referral patterns
prescribing patterns over the past five years. Physicians report
spending a substantial percentage of time on risk
management activities (18%), though there is clear doublecounting with managed care activities in the case of activities
such as quality assurance and utilization review. Multinomial
logit analyses suggest that physicians who invest more in
office systems spend more time on risk management have
lower liability risks. Among the variables with statistically
significant relations with more claims (at the 0.05 level)
include: higher managed care case-loads, higher levels of
documentation, male physicians, international medical
graduates and physicians with a specialty other than family
medicine/general practice. Interaction terms suggest that
spending more time on risk management partially offsets the
increased risk associated with managed care and diminishes
the benefit of office systems.
Conclusions: Investing more in office systems and spending
more time on risk management are directly associated with a
lower likelihood of having claims. Each 10% increase in time
spent on risk management is associated with a 2% decrease
in the likelihood of having claims made against them – a
substantial decrease in relative risk.
Implications for Policy, Delivery or Practice: Proposed
solutions to the current medical liability crisis have focused on
political remedies. These results suggest that actions taken
directly by physicians may enable reductions in claims and
offer some relief from the liability crisis.
Primary Funding Source: Blue Cross Blue Shield of
Michigan Foundation
• Identifying Outcome Measures to Assess the Quality of
Cancer Care
Claire Snyder, M.H.S., Joseph Lipscomb, Ph.D., Carolyn Cook
Gotay, Ph.D.
Presented by: Claire Snyder, M.H.S., Contractor, National
Cancer Institute, 1900 Thames Street #206, Baltimore, MD
21231; Tel: 410.732.7113; Fax: 301.435.3710; E-mail:
claire.snyder@alumni.duke.edu
Research Objective: The National Cancer Institute (NCI) has
made enhancing the quality of cancer care a research priority,
and the NCI seeks to identify patient-reported outcome (PRO)
measures that can be used in cancer-care decision-making
and quality of care evaluations.
Study Design: In 2001, the NCI created the Cancer Outcomes
Measurement Working Group (COMWG) and invited 35
leading experts to evaluate the state of the science of
outcomes assessment in cancer and to identify areas requiring
further development. COMWG members reviewed the
literature and reported their individual findings and
recommendations.
Population Studied: The COMWG evaluated three outcomes
(health-related quality of life/patient
satisfaction/economic burden) in four cancers
(breast/prostate/colorectal/lung) across the continuum of
care (prevention-screening/treatment/survivorship/end of
life). The COMWG also considered a range of applications for
outcome measures, including research (clinical
trials/observational studies), physician-patient interactions,
population monitoring, and policy making.
Principal Findings: Currently available instruments used to
assess PROs are valid, reliable, and responsive in many
circumstances, particularly for research applications. These
instruments may be generic, general cancer, or cancer sitespecific.
No instrument stands apart from the alternatives as a gold
standard that could be defined as a core measure. Rather, the
field is still developing, and it is premature to select particular
instruments and define them as core
measures. Selecting the best instrument for a particular study
depends on the study's goals and intended audience. Because
most research to
date has focused on picking the "winner" among treatment
alternatives, researchers have used instruments that target
specific areas expected to
differ between groups. Such specific measurement strategies
make it difficult to compare findings across studies, to draw
general conclusions on
the quality of cancer care, and to make broader policy
decisions. Thus, findings from these research applications
shed little light on how outcomes data are used in other areas
of decision-making such as selecting a particular patient's
treatment, making regulatory decisions (e.g., by the FDA),
developing coverage and reimbursement policies (e.g., by
CMS), or conducting meta-analyses to evaluate the quality of
cancer care.
Conclusions: A variety of instruments meet minimum
scientific standards for assessing PROs. Due to the diversity of
instruments used, researchers who seek to evaluate the overall
quality of cancer care have difficulty when attempting to
interpret findings across studies. Because most applications
of outcomes assessment have been in the research setting,
there is little information on how outcomes data are used or
could be used in day-to-day decision-making or in policy
development.
Implications for Policy, Delivery or Practice: Until a core set
of outcome measures is identified or modern methods (e.g.,
item response theory) are applied that enhance comparability
across studies, the value of outcomes assessment in decisionand policy-making will remain uncertain. To increase the
usefulness of outcomes data across applications, we need
measures that provide information relevant to a range of
decision-makers and the ability to compare findings across
studies. The public and private sectors should cooperate in
this endeavor to increase the value of cancer outcomes
assessment for improving the quality of cancer care.
Primary Funding Source: NCI
• Risk Adjustment with Diagnoses Present at Admission
Reduces the Significance of the Relationship between
Hospital Lung Cancer Surgery Volume and Patient
Mortality
George Stukenborg, Ph.D., Kerry Kilbridge, M.D., Douglas
Wagner, Ph.D., Frank Harrell, Jr., Ph.D., M. Norman Oliver,
M.D., Alfred Connors, Jr., M.D.
Presented by: George Stukenborg, Ph.D., assistant professor,
Health Evaluation Sciences, University of Virginia School of
Medicine, P.O. Box 800821, Charlottesville, VA 22908; Tel:
434.924.8649; Fax: 434.243.5787; E-mail:
gstukenborg@virginia.edu
Research Objective: Prior studies have demonstrated
statistically significant inverse relationships between hospital
lung cancer surgery volume and mortality risk. Reviews of
these studies however emphasize that improved mortality risk
adjustment methods are needed. This study reexamines the
relationship between lung cancer surgery volume and
mortality among California hospital patients using present at
admission diagnoses to adjust for comorbid disease and for
other characteristics. These results are compared to results
from mortality risk adjustment methods similar to those used
by prior studies applied to the same study population.
Study Design: ICD-9-CM diagnoses reported as present at
admission were separated into categories of comorbid disease
and conditions closely related to lung cancer by a physician
panel. Panelists used software to review codes and record
scores. Panelists discussed their scores and clinical reasoning,
and disagreements were resolved using the Delphi method.
Multivariable logistic regression was used to adjust the
relationship between surgical volume and mortality for
present at admission diagnoses, race, Hispanic ethnicity,
gender, emergency admission, transfer status, and age. The
functional relationship between surgical volume and mortality
was specified using piecewise polynomial functions to account
for nonlinearity. Discrimination was quantified using the C
statistic and validated using bootstrap analysis to address the
potential for model overfitting. The statistical significance of
surgical volume was measured using the log-likelihood ratio
chi-square statistic with appropriate degrees of freedom. Loglikelihoods were also used to quantify the contribution to the
model’s predictive performance of information from
diagnoses present at admission. Results from the model
using present at admission diagnoses were compared to
results from a similar model with comorbid disease measured
using the Deyo et al. adaptation of the Charlson index and
with another model using the method of Elixhauser et al.
Population Studied: 14,456 hospitalizations in California
from 1996 through 1999 for lung cancer surgery.
Principal Findings: The relationship between lung cancer
surgery volume and mortality was less statistically significant
(p = 0.08) after risk adjustment using present at admission
diagnoses than after adjustment with both alternative
methods of measuring comorbid disease (p < 0.01). The
mortality risk adjustment model using present at admission
diagnoses yielded better validated discrimination than the
comparison models.
Conclusions: The statistical relationship between hospital
lung cancer surgery volume and patient mortality risk is
sensitive to improvements in mortality risk adjustment.
Present at admission diagnoses yield both improved mortality
risk adjustments and results that differ meaningfully from risk
adjustments using established methods.
Implications for Policy, Delivery or Practice: Evidence from
prior studies of the relationship between surgical volume and
mortality is being used to support recommendations for the
regionalization of cancer surgery and for selected volume
thresholds as criteria for patient referral. Reviewers disagree
about the quality of these studies and whether the evidence is
adequate to support this purpose. We found that the
relationship between lung cancer surgery volume and patient
mortality was not statistically significant (p > 0.05) after more
complete adjustments for the confounding effects of
comorbid disease using diagnoses present at admission.
Hospital discharge data that identifies which diagnoses were
present at admission can be used to meaningfully improve
mortality risk adjustment.
Primary Funding Source: AHRQ
• Defining and Learning from Safety-Related Events:
Developing a Self-Assessment Tool for Hospitals
Michal Tamuz, Ph.D., Eric Thomas, M.D., M.P.H., Cynthia
Russell, Ph.D., Richard Faris, Ph.D.
Presented by: Michal Tamuz, Ph.D., Associate Professor,
Center for Health Services Research, University of Tennessee
Health Science Center, 66 North Pauline, Suite 463, Memphis,
TN 38163; Tel: 901.448.3716; Fax: 901.448.8009; E-mail:
mtamuz@utmem.edu
Research Objective: 1. To examine how the definition and
classification of safety-related events influence processes that
contribute to or comprise organizational learning.
2. To develop a self-assessment tool for hospitals to gauge
and expand their capacity to learn from safety-related events.
Study Design: This study involved two phases. In phase 1, we
conducted a qualitative study of how a hospital defines and
learns from safety-related events. Using semi-structured
interviews, we asked participants to describe programs
designed to monitor medication safety. Interviews, ranging
from 45 minutes to 8 hours, were audio-recorded, transcribed,
and analyzed using qualitative methods. Fieldnotes,
document review, and observations of routine activities
supplemented the interviews. We inferred organizational
processes from analyses of critical examples and patterns in
the data.
In phase 2, a multidisciplinary investigator team (e.g.,nurse,
pharmacist, physician) identified findings critical to the
development of an organizational learning self-assessment
tool. Using findings generated from and illustrated by
interview examples, we developed vignettes and questions
designed to enable participants to assess how their hospital
safety-information systems operate.
Population Studied: 86 healthcare professionals in an urban,
tertiary-care teaching hospital. (36 pharmacy staff; 36 nurses,
physicians, and other patient-care unit staff; 14 hospital
administrators).
Principal Findings: Finding 1:
Event classification influenced information collection by
activating organizational standard operating procedures.
Example:
When pharmacists called physicians to clarify prescriptions, it
was defined as “interventions” that pharmacists were
rewarded for reporting, increasing available information.
When nurses made the same calls, they did not define nor
gather data about them.
Vignette-Objectives:
Demonstrate how similar events can be defined differently
and trigger alternative routines for data-gathering and
analysis.
Identify “defined-away” data sources.
Finding 2:
Event classification influenced information collection through
reward distribution. Example:
Pharmacy rewarded intervention reporting.
Vignette-Objectives:
Identify distribution of rewards/punishments that create
(dis)incentives for
event-reporting.
Finding 3:
Event classification influenced analysis through accountabilitybased databases that resulted in truncated data analysis.
Example:
Prescribing errors were divided between separate
accountability-based databases, depending on their
classification as “interventions” (pharmacy database) or
“incidents” (hospital-wide database).
Vignette Objectives:
Clarify trade-offs of using accountability (or learning-based)
criteria for designing databases.
Finding 4:
Event classification influenced analysis through
communication channel construction.
Example:
Because prescribing errors were defined as pharmacist
interventions rather than physician errors, communication
channels were constructed only to give feedback to
pharmacists.
Vignette-Objectives:
Identify vertical/horizontal channels for communicating
feedback.
Finding 5:
Event classification influenced decision-making and
implementation through locus of decision-making authority.
Example:
Classifying events as interventions resulted in decentralized
decision-making and reduced implementation authority.
Vignette-Objectives:
Highlight trade-offs between hospital and departmental-level
decision-making.
Differentiate decision-making expertise and authority.
Conclusions: We developed vignettes to illustrate how the
definition and classification of safety-related events shapes
information collection and organizational learning processes.
Incorporating vignettes in a self-assessment tool, we seek to
enable hospitals to reduce barriers to learning.
Implications for Policy, Delivery or Practice: In phase 3, we
will test and validate the self-assessment tool. Hospital staff
will read and reflect on the vignettes. After answering
questions about hypothetical situations, participants will
describe how these organizational processes operate in their
hospital. They will be trained to identify conditions that foster
(hinder) learning and clarify trade-offs inherent in system
design choices.
Primary Funding Source: Aetna Foundation Quality Care
Research Fund
• Emergency Department Performance: Measuring and
Improving Quality
Deborah Tregunno, R.N., M.H.S.A, Ph.D., G. Ross Baker,
Ph.D.
Presented by: Deborah Tregunno, R.N., M.H.S.A, Ph.D.,
Postdoctoral Fellow, Faculty of Nursing, University of Toronto,
136 Airdrie Road, Toronto, Ontario, Canada ; E-mail:
deborah.tregunno@utoronto.ca
Research Objective: The importance of understanding
context and strategy in quality measurement and
improvement is powerfully illustrated in assessments of
Emergency Department (ED) performance. This study
identifies contextual factors that influence the collection and
use of ED performance data, and proposes a framework that
integrates performance interests and quality improvement
profiles. Important implications for the development and use
of comparative performance reports are identified.
Study Design: Two cross sectional surveys are used in this
study. Sample one involved a field study to obtain ED
stakeholder (physicians, nurses and managers) perspectives
of important dimensions and indicators of ED performance.
In sample two, a survey was used to obtain information on the
collection and use of performance data to improve the quality
of ED patient care. Qualitative methods are used to validate
the study findings.
Population Studied: To develop a profile of ED performance
interests, a systematic sample of 600 internal stakeholders
(physicians, nurses and managers) was identified from a
stratified random sample of 50 acute care hospitals (small,
community and teaching) in the Canadian province of
Ontario. An 57% response rate was achieved in sample one.
Sample two included the population of 121 acute care
hospitals in Ontario that participated in a comparative report
of ED performance (Hospital Report: Emergency Department
Care ’03). Interview data was collected from a cross section of
Ontario EDs. An 87% response rate was achieved in sample
two. Interviews were conducted with key informants from a
cross section of Ontario hospitals.
Principal Findings: The results indicate that a range of ED
performance profiles, and that stakeholder performance
interests are robust across hospital types. For example, three
aspects of ED performance – timeliness of care, linkages with
community providers, and medical and nursing staff turnover
– are less important for small hospital stakeholders. These
findings are consistent with quality measurement and
improvement practices reported by small EDs, which indicate
lower levels of coordination and involvement with the
community and lower levels of clinical data collection and
dissemination. Examination of the performance and quality
improvement
profiles suggests the influence of a number of contextual and
strategic factors, including the healthcare needs of the local
community, community partnerships, investment in
information technology, and promotion of the use of
performance data by organizational leaders.
Conclusions: This study provides a useful framework for
evaluating and improving the quality of ED care. The findings
indicate a range of ED performance profiles reflective of a
range of attention to different aspects of patient care and
responses to quality issues.
Implications for Policy, Delivery or Practice: Organizations
and those who fund and design performance reports need to
recognize that stakeholder interests and organizational
context play an important role in determining the salience and
use of performance data for quality improvement.
Primary Funding Source: Ontario Ministry of Health and
Long Term Care
• Evaluating Adherence to CDC Treatment Guidelines in
Two Large Health Plans: An Example of Chlamydia
Waimar Tun Ph.D., M.H.S., Cathleen Walsh, Dr.P.H., Michael
Stiffman, M.D., M.S.P.H., David Magid, M.D., M.P.H., Lynda
Anderson, Ph.D., Terese DeFor, M.S., Ella Lyons, M.S.,
Kathleen Irwin, M.D., M.P.H.
Presented by: Waimar Tun, M.H.S., Ph.D., Epidemiologist,
Division of STD Prevention/Health Services Research &
Evaluation, Centers for Disease Control and Prevention, 1600
Clifton Road, MS-E80, Atlanta, GA 30333; Tel: 404.639.8297;
Fax: 404.639.8607; E-mail: wct4@cdc.gov
Research Objective: Clinical practice guidelines have been
developed and disseminated to improve the quality of care
and guide decision making. Since 1982, the Centers for
Disease Control (CDC) has issued guidelines for the
treatment of sexually transmitted diseases that included
Chlamydia, to encourage use of safe, effective, acceptable, and
cost-effective antibiotics. An estimated 3 million Chlamydial
infections occur annually, and if not detected and treated
promptly, serious and costly sequelae can result, including
pelvic inflammatory disease and infertility. Because clinicians
in private practice diagnose over half of reported cases of
Chlamydia, and yet they are rarely studied, we examined
clinicians’ adherence to CDC’s Chlamydia treatment
guidelines among those practicing in two private health plans.
Study Design: In 2000, 743 (82%) of 907 clinicians in two
large health plans completed a mailed survey on Chlamydia
treatment practices, and use of CDC’s most recent STD
Treatment Guidelines. Guideline adherence was defined as
reported use of any CDC-recommended antibiotics for
patients with laboratory-confirmed Chlamydial infection.
Clinical scenarios were used to assess prescribing practices.
Weighted percentages and adjusted odds ratios were
calculated to identify factors associated with reported
adherence because we used a stratified probability sample.
Population Studied: The sample included physicians and
midlevel providers in specialties that are most likely to see
patients with STDs. Analysis was restricted to 578 clinicians
who treated at least one Chlamydia-infected patient in the past
year.
Principal Findings: The majority of respondents were men
(66%), physicians (94%), and practiced
internal/family/general medicine (74%). Although 90% were
found to be adherent with STD treatment guidelines, only
62% reported they would prescribe single-dose azithromycin
to injection drug users (IDUs), which is known to optimize
adherence. Although CDC did not recommend azithromycin
as a first-line antibiotic for pregnant women, 48% reported
they would prescribe it. Compared to practicing
internal/family/general medicine, pediatricians (Odds ratio
[OR]=5.7; 95% confidence interval [CI]=2.0-16.5) and
emergency medicine specialists (OR=3.7; CI=1.1-13.5) were
more likely to report adherence to recommendations from the
STD treatment guidelines. Clinicians who practiced =20 years
(OR=3.9; CI=1.3-11.4) versus >20 years, and clinicians who did
not obtain STD treatment information from colleagues
(OR=3.6; CI=1.8-12.2) versus those who did were more likely to
report adherence. Reported adherence was not independently
associated with being a physician, gender, or number of cases
treated annually. Clinicians reported using several sources for
STD treatment information, which were not associated with
reported adherence.
Conclusions: Most clinicians reported referring and adhering
to CDC STD treatment guidelines. However, interventions
may be needed to prevent use of contraindicated drugs for
pregnant women and to increase use of single-dose
Azithromycin for IDUs, and other populations who may have
poor compliance with multi-day regimens.
Implications for Policy, Delivery or Practice: Given the high
levels of adherence to CDC STD recommended treatments at
these health plans, these health plans should consider other
quality improvement interventions to control Chlamydia, such
as increasing detection of asymptomatic infection through
routine screening, and prevention through risk reduction
counseling, and management of sex partners.
Primary Funding Source: CDC
• Does Presentation of Comparative Health Plan
Information or Medium Affect the Comprehension,
Perceptions or Decision-Making of Older Consumers?
Jennifer Uhrig, Ph.D., M.H.A., Lauren Harris-Kojetin, Ph.D.,
Carla Bann, Ph.D., Tzy-Mey Kuo, Ph.D.
Presented by: Jennifer Uhrig, Ph.D., M.H.A., Health Services
Researcher, Health Communication, RTI International, 3040
Cornwallis Road, P.O. Box 12194, Research Triangle Park, NC
27709-2194; Tel: 919.316.3311; Fax: 919.990.8454; E-mail:
uhrig@rti.org
Research Objective: To determine whether the strategy or
medium for presenting comparative health plan information
has differential effects on comprehension, perception of, and
use of quality information in health plan choices.
Study Design: We conducted two separate randomized
experimental studies with employees/early retirees ages 58-64
in Portland, Oregon and Research Triangle Park, North
Carolina. The first experiment manipulated the presentation of
printed health plan choice materials. Participants (n=150) were
randomly assigned to one of three conditions—integrated,
alternate or control. The integrated group intervention
presented cost, benefit, and quality information together
organized by substantive themes (e.g., preventive care). The
alternate group intervention contained the same cost, benefit,
and quality information but each was presented in a separate
section (formatted like Medicare’s print on demand
comparative health plan information). The control group
intervention had both the content and format of Medicare’s
print on demand comparative health plan information. The
second experiment (n=150) manipulated the medium. Half of
the participants were randomly assigned to receive
comparative health plan information in a printed booklet while
the other half received that information via an interactive
website. In both experiments, participants were told to
imagine that they were about to turn 65, and their task was to
choose a health plan for themselves. After the participants
reviewed the information, they completed a questionnaire.
Population Studied: Individuals ages 58-64.
Principal Findings: Both experiments examined the effects of
the interventions on comprehension (awareness and
understanding of Medicare and health plan choices, and
interpretation of comparative cost and quality information),
perceptions of the materials (utility of, trust in, and
satisfaction with the materials), and decision-making (the role
that quality information played in the decision process and
confidence in plan choice). For the presentation experiment,
we hypothesize that participants in the integrated group will
show greater awareness, understanding, more positive
perceptions, and attribute greater importance to quality of
care in decision making than participants in the alternate or
control groups. For the medium experiment, we hypothesize
that the computer group will find the materials more useful,
easier to use, be more satisfied with the materials, and more
confident in their plan choice than the print group. We have
completed data collection, and the analysis files are currently
being constructed. We will compute descriptive statistics for
each condition, and then use Chi-square and t-tests to explore
relationships between different groups. Finally, we will
estimate multivariate models to test our hypotheses. These
findings will be completed in time for this presentation,
should this abstract be accepted.
Conclusions: Findings from these controlled experimental
studies will provide empirical evidence as to whether
presentation or medium have differential effects on
comprehension, perceptions and decision-making.
Implications for Policy, Delivery or Practice: Findings can
be applied to the development of future health plan choice
materials for older consumers.
Primary Funding Source: AHRQ,
• Does the Discharge Instruction Component of the
JCAHO Heart Failure Core Measure Have an Effect on the
Readmission of Heart Failure Patients?
Monica VanSuch, M.B.A., James Naessens, M.P.H., Robert
Stroebel, M.D., Amy Wagie, BS
Presented by: Monica VanSuch, M.B.A., Project Coordinator,
Health Care Policy and Research, Mayo Clinic, 200 First
Street, S.W., PB 3-25, Rochester, MN 55905; Tel: 507.284.1166;
Fax: 507.284.1731; E-mail: vansuch.monica@mayo.edu
Research Objective: To identify if compliance with all or any
of the six discharge instruction categories as required by the
Joint Commission on Accreditation of Health Care
Organizations(JCAHO) heart failure core measure decrease
readmissions for heart failure patients.
Study Design: Using data from July 2002 through September
2003, a retrospective study was conducted of 1162 randomly
chosen patients discharged with a principal diagnosis of
congestive heart at a single hospital. 811 patients met the
criteria where receiving discharge instructions were
recommended. Data was analyzed to determine the extent of
compliance with all or any of the six discharge instruction
categories: activity level, diet, discharge medications, follow-up
appointment, weight monitoring and what to do if symptoms
worsen. Associations between documentation of discharge
instructions and time to hospital readmission were assessed
with Kaplan-Meier survival analysis, log-rank tests and Cox
regression models. Analyses were performed for both any
hospital readmission and restricted to only readmissions for
heart failure. Potential covariates in multivariable models
included patient demographics, severity of illness, discharge
disposition and comorbidities.
Population Studied: Patients 18 years of age and older with a
principle diagnosis of heart failure, hypertensive heart disease
with heart failure, or hypertensive heart and renal disease with
heart failure discharged to home, home care, or home care
with IV therapy. Patients in all stages of heart failure (New
York Heart Association Functional Classification I-IV) were
included.
Principal Findings: 68% of patients had documentation of
receiving all 6 types of instruction. 6% of patients had no
documentation of any of the types of instruction. Of the
instruction types, patients received medication instructions
the most (91%), while they received weight monitoring
instructions the least (70%). Overall, 8% of patients were
readmitted for heart failure within 14 days, 16% within 30 days,
25% within 60 days and 29% within 90 days. Similarly,
readmissions for any cause were 9%, 17%, 27%, and 33% at
14, 30, 60 and 90 days after discharge. In analysis unadjusted
for covariates, patients with all instructions had significantly
longer time to readmission for both all causes(p=0.003) and
for readmission for HF (p=0.033) than those who were
missing at least one type of instruction. Missing instructions
on activity seemed to have the biggest impact on time to
readmission for HF and on time to any readmission. When
covariates were included in a Cox regression model, discharge
instructions were still significantly associated with time to any
readmission (p=0.01) and had some association with time to
HF readmission (p=0.09).
Conclusions: Documentation of compliance with the
discharge instruction component of the JCAHO heart failure
core measure was associated with increased time to
readmission. Significant association was seen between
discharge instructions for heart failure patients and time free
of readmission, particularly with instructions on activities.
Implications for Policy, Delivery or Practice: Discharge
instructions appear to contribute to keeping heart failure
patients out of the hospital. The JCAHO core measure
assessing these instructions may be a valid indicator of quality
of care.
• The Effect of the Balanced Budget Act of 1997 on Process
of Care for Patients with Acute Myocardial Infarction
Kevin Volpp, M.D., Ph.D., Eric Peterson, M.D., M.P.H.,
Jingsan Zhu, M.A., Lori Parsons, A.B., J. Sanford Schwartz,
M.D.
Presented by: Kevin Volpp, M.D., Ph.D., Assistant Professor
of Medicine and Health Care Systems, Philadelphia VAMC,
University of Pennsylvania School of Medicine, the Wharton
School, CHERP, Philadelphia VAMC, University and Woodland
Avenues, Philadelphia, PA 19104; Tel: 215.573.0270; Fax:
215.573.8778; E-mail: volpp70@mail.med.upenn.edu
Research Objective: The Balanced Budget Act (BBA) of 1997
was designed to cut Medicare payments by $116.4 billion from
1998 to 2002. It is well known that better outcomes are
associated with more rapid treatment of patients with acute
myocardial infarction (AMI). We tested using a clinical
database of AMI patients whether BBA-induced hospital
financial stress was associated with delays in care and whether
more vulnerable groups such as the uninsured or blacks were
disproportionately affected.
Study Design: Observational Study. Hospitals were grouped
into quartiles based on the projected impact of BBA on net
patient revenues using an American Hospital Association
simulator. These ranks were interacted with year dummy
variables to examine using multiple regression how different
process measures of AMI care changed over time in
accordance with hospital financial stress from the pre-BBA
period to the years post-BBA adjusting for clinical
comorbidities and hospital-level clustering.
Population Studied: 260,735 patients were identified from
280 hospitals with interventional capabilities that enrolled
>=20 patients in each fiscal year from 1996-2001 from the
National Registry of Myocardial Infarction (NRMI).
Principal Findings: Hospitals in the quartile group least
affected by BBA (quartile 1) were expected to lose an average
of 1.9% in net patient revenues from 1998 to 2000 compared
to 3.9% in the fourth quartile group. Relative to patients in
quartile 1, insured patients in quartile 4 experienced no
significant increase in the time from hospital arrival to
ordering of thrombolytic therapy from pre-BBA to 1999, while
time to treatment increased significantly for uninsured
patients (0.29 hours, p-value 0.003). Blacks (increase of .10
hours, p-value 0.06) had a similar degree of worsening as
whites (increase of .11 hours, p-value 0.05). There was no
significant relative increase in quartile 4 from pre-BBA to 2000
or 2001. Significant increases from pre-BBA in time from
hospital arrival to initiation of thrombolytic therapy in quartile
4 relative to quartile 1 were observed among the insured in
2000 (.09 hours, p=.05); uninsured in 1999 (0.18 hours,
p=.09), whites in 1998-2000 (increase of between 0.09-.10
hours, p between .01-.05) and blacks in 1999-2001 (increase of
.07-.08 hours, p-value between .04 and .10). While there was a
non-significant improvement in time from hospital arrival to
cardiac catheterization in quartile 4 vs quartile 1 from pre-BBA
to 2001 among the insured (-0.5 hours, p=.06), among the
uninsured there was a non-significant relative increase in 2001
(0.34 hours, p=.69).
Conclusions: AMI patients at hospitals most affected by the
BBA suffered some adverse effects, but this was not the case
for all measures. While blacks did not fare worse than whites,
the uninsured generally fared worse than the insured.
Implications for Policy, Delivery or Practice: Cost-saving
reforms may adversely affect quality of care and contribute to
health disparities.
Primary Funding Source: Doris Duke Foundation
• A Reliability Assessment of Performance Measure Data
Ann Watt, M.B.A., RHIA, CQM, Scott Williams, PsyD, Richard
Koss, M.A., Jerod Loeb, Ph.D., David Morton, M.S.
Presented by: Ann Watt, M.B.A., RHIA, CQM, Associate
Project Director, Health Policy Research, Joint Commission on
Accreditation of Healthcare Organizations, One Renaissance
Boulevard, Oakbrook Terrace, IL 60181; Tel: 630.792.5944; Fax:
(630) 792-4944; E-mail: awatt @jcaho.org
Research Objective: The objective of this study, conducted by
the Joint Commission on Accreditation of Healthcare
Organizations (JCAHO), is to assess the reliability of core
performance measure data and to evaluate an approach for
on-going reliability monitoring.
Study Design: JCAHO staff members conducted on-site visits
to a stratified, random sample of 30 hospitals at which they
reabstracted approximately 30 medical records that had been
abstracted by hospital personnel. The original hospital
abstraction, and the JCAHO reabstraction were then
compared data element by data element. In addition, a
random, stratified sample of 85 hospitals was selected to
perform self-reabstraction of approximately 20 previously
abstracted medical records. Structured interviews were
conducted with hospital staff to determine systemic causes for
data discrepancies and to determine the mechanics of each
hospital’s core performance measure data abstraction
process. The degree of agreement between the original
abstraction and the reabstraction for each data element, and
for the overall performance measure rates, was computed.
Population Studied: Samples consisted of JCAHO-accredited
short term acute care US hospitals collecting core
performance measure data. Stratification criteria included
geographic location, urban or rural status, hospital size, core
measure set collected, and profit or not-for-profit status.
Principal Findings: Overall data element agreement rates
varied among measure sets and, in general, Joint Commission
independent reabstractors identified more data element
discrepancies than did the self-reabstractors. The acute
myocardial infarction (AMI) measure set had an overall data
element agreement rate of 0.91 for JCAHO abstractors. That
is, 91% of the time JCAHO abstractors found the same data
element value as the original hospital abstractors, averaged
across all data elements collected. For hospital selfreabstractors the overall agreement rate for the AMI set was
0.96. Similar findings were observed for the other measure
sets (heart failure = 0.93 for JCAHO and 0.96 for selfreabstractors, pneumonia = 0.91 for JCAHO and 0.95 for selfreabstractors, and pregnancy = 0.95 for JCAHO and 0.98 for
self-reabstractors).
Conclusions: Both the JCAHO and the hospital data
reabstraction approaches identified opportunities for
improvement in core measure data collection efforts, although
reabstraction by JCAHO staff identified more discrepancies
than did self-reabstraction by hospital staff. The majority of
the hospitals included in the study did not routinely perform
formal quality assessment of their core performance measure
abstracted data.
Implications for Policy, Delivery or Practice: Because of the
various uses stakeholders will be making of core performance
measure data, it is essential that data reliability be attained
and sustained at an optimal level. In order to accomplish this,
data abstraction accuracy and completeness must be routinely
and continuously monitored by health care organizations.
This study has demonstrated that self-reabstraction appears to
be an effective and easily implementable method for
accomplishing this, despite being somewhat less rigorous
than approaches that rely on independent reabstraction of
medical record data.
Primary Funding Source: AHRQ
• The Bridges to Excellence Program Demonstrates
Success of Physician Incentives
Audrey Weiss, Ph.D., Jonathan Conklin
Presented by: Audrey Weiss, Ph.D., Senior Research
Manager, Medstat, 5425 Hollister Avenue, Santa Barbara, CA
93111; Tel: 805.681.5828; Fax: 805.681.5888; E-mail:
audrey.weiss@medstat.com
Research Objective: Assess the impact of physician
performance incentives on the quality of diabetes care, cardiac
care, and office systems and practices.
Study Design: This study assesses the impact of Bridges to
Excellence (BTE), a "pay for performance" program targeting
physicians and their offices, sponsored by employers and
health plans in four large urban markets. The BTE program
evaluation is monitoring physician participation and
performance, patient experience of care, and overall changes
in costs of care.
Population Studied: All covered lives (including employees,
dependents, and retirees) of 10 large employers in four major
urban markets (Boston, Cincinnati, Louisville, Albany).
Principal Findings: Physician participation in a national
quality-of-care recognition program has increased dramatically
in the first 8 months of BTE program rollout in initial markets.
Conclusions: Financial incentives can influence physician
participation in quality improvement and performance
recognition programs.
Implications for Policy, Delivery or Practice: Employersponsored "Pay for Performance" programs may improve
market-wide quality of care and may reduce overall health care
costs.
Primary Funding Source: Large Employers
• State Patient Safety Reporting Systems: Views from
Hospital Leadership
Joel Weissman, Ph.D., Catherine Annas, JD, Brian Clarridge,
Ph.D., Arnold Epstein, M.D., Sandra Feibelmann, M.P.H., Eric
Schneider, M.D.
Presented by: Joel Weissman, Ph.D., Associate Professor,
Institute for Health Policy, MGH/Harvard, 50 Staniford Street,
9th Floor, Boston, MA 02114; Tel: 617.724.4731; Fax:
617.724.4738; E-mail: jweissman@partners.org
Research Objective: To examine the views of hospital leaders
toward the effect of mandatory state reporting systems on
patient safety in their hospital and in the state, and to
determine whether type of state reporting system is associated
with hospital patient safety practices.
Study Design: Telephone interviews administered by UMASS
Center for Survey Research.
Population Studied: CEOs and COOs of 205 acute, nonfederal hospitals in six states covering a spectrum of reporting
systems and U.S. region (response rate = 63%). States
systems at the time of the survey (2002-3) included:
Massachusetts and Colorado (Mandatory state systems with
public disclosure); Florida and Pennsylvania (Mandatory state
system with non-disclosure); and, Georgia and Texas (Nonmandatory).
Principal Findings: Subjects were asked about the perceived
effect of a mandatory, non-confidential system on patient
safety practices. About two-thirds thought that it discourages
the likelihood of reports being made internally within
hospitals, and three-quarters felt it encourages lawsuits.
Regarding actual patient safety, 29% felt that such a system
has a positive effect, 30% that it has a negative effect, and
40% that it has no effect. There were no significant
differences among the states.
When asked about the type of mandatory reporting system
that would most reduce errors, more than four out of five
respondents, overall, preferred one that kept confidential both
the hospital and the name of the practitioner involved.
However, respondents from mandatory, non-confidential
states were far more likely to accept systems that made public
only the name of the hospital (22% versus 6% for all other
states, p=.004).
A major policy concern of state decision-makers is whether to
tell the patient or family member about an incident that is
reported to the state. About 37% of hospital leaders thought
such disclosure should never happen, 23% that it should
always happen, and 40% that it should happen sometimes.
Circumstances in which the patient or family should be told
include imminent release to the press (98% of those
responding “sometimes”), patient harm (28%), or request by
the patient or family (72%).
Finally, we asked respondents about policies and practices in
their hospitals, including the frequency of patient safety on the
formal agenda of their boards, the priority of protecting
individual error reporters, and written policies recommending
disclosure of injury and errors to patients. Although
responses varied among respondents, there were no
significant differences by state.
Conclusions: Hospital leaders generally had negative views
toward mandatory reporting systems, especially those with
public disclosure of providers. They believed that the
presence of mandatory state reporting systems has had little
effect on patient safety practices in hospitals.
Implications for Policy, Delivery or Practice: Any state
reporting system must rely on effective implementation at the
provider level. The attitudes of hospital leaders are likely to be
important to the successful implementation of these efforts..
Our results suggest that states wishing to adopt hospital error
reporting systems may need to demonstrate their value to
institutional leaders.
Primary Funding Source: AHRQ
• Patient Safety Leadership Assessment Tool: Web-Based
Tool Improves Patient Safety
John Wendling, B.S., Donald Casey, M.D., M.P.H., M.B.A.,
FACP
Presented by: John Wendling, B.S., Corporate Director,
Clinical Resource Management, Quality & Safety, Catholic
Healthcare Partners, 615 Elsinore Place, Cincinnati, OH 45202;
Tel: 513.639.2841; Fax: 513.639.2773; E-mail:
jwwendling@health-partners.org
Research Objective: This presentation will review the critical
steps of planning, developing, promoting and disseminating a
web-based tool to support policies and programs to improve
patient safety across the continuum (acute and post-acute
care hospitals, nursing homes, home health, outpatient clinics
and hospice). The web-based tool, which is the largest
organizational assessment tool in the United States, provides
leadership with the ability to monitor and guide patient safety
initiatives and focus for the organization.
In January 2002, the board of trustees and executive
leadership of a 31-hospital health system addressed the need
to improve the safety of the patient population. This strategy
would strengthen system-wide initiatives already in place. The
patient safety leadership assessment web-based tool was the
vehicle used to provide leadership with the information
necessary to drive patient safety initiatives and focus for the
organization.
Study Design: Previous patient safety initiatives were
performed in a vacuum (“silo effect”) and reported to various
levels within the organization. These efforts often failed to
engage leadership with the knowledge necessary to support
and drive these initiatives. In addition, there was nothing in
place to assess the organization’s “strengths” and
“weaknesses” with various patient safety issues.
There are several measures used to monitor, track and assess
the effectiveness of the web-based patient safety leadership
assessment tool. These measures are grouped into two
categories (engagement and utilization). Engagement
measures includes the following: 1) Volume and type of
individuals (clinical/non-clinical); 2) Types of organizations
(hospital, nursing home, hospice, etc.); 3) Positions using the
tool (President/CEO, physicians, nurse, etc.); 4) Time
reduction study with paper-less process. Utilization measures
include improvement scores with the thirty-eight (38)
leadership assessment measures. These measures include
the following: 1) Education of staff, patients, families,
consumers and media; 2) Practicing a culture of safety (nonpunitive environment, executive engagement, patient/ family
notification); 3) Improved patient outcome (medication
errors, etc.).
Population Studied: Interviews were conducted with key
constituents of the health system (management, physicians,
nurses, pharmacists, social workers, etc.). A patient safety
survey was provided to executive and senior leadership. The
information was reviewed. Patient safety measures were
reviewed. Observational studies were conducted on nursing
units as well as specialty areas (radiology, surgery, etc.) to
identify the level of health professional understanding and
engagement with patient safety issues.
Principal Findings: The analysis provided the following
information: 1) Key participants were unaware of patient
safety issues; 2) Executive and senior leadership were not
engaged with patient safety initiatives; 3) Participants were not
part of the performance improvement process; 4) Healthcare
professionals feared being reprimanded for reporting medical
errors; 5) Fear of communicating errors with patients and
family members; 6) Opportunity to improve patient safety
measures.
Conclusions: Since the implementation of the web-based tool
the health system has experienced: 1) Improved education
and communication of patient safety policies and procedures;
2) Increased awareness by executive/senior management on
patient safety issues and initiatives; 3) Increased system-wide
sharing of best practices; 4) Improved scores on the 38
assessment measures; 5) Improved patient/family satisfaction
scores; 5) Reduced medication errors; 6) Decreased patient
falls.
The web-based tool undergoes quarterly assessments. As new
interventions are developed and behavioral expectations
change, the web-based tool can easily be edited and updated.
The web-based tool has surpassed 400 members spanning 45
organizations (including all 31 hospitals). Additional webbased tools are utilized to support other quality and patient
safety initiatives (33 web-based tools).
Implications for Policy, Delivery or Practice: There was a
significant commitment by leadership to change the way work
was being performed so that patient safety came first. With
the knowledge that all healthcare organizations incur errors,
the way our health system handle these errors will either build
or erode trust with our patients, family members and staff.
This required a cultural transformation that affects both
clinicians and non-clinicians. Systems were redesigned to
prevent errors, greater accountability was established,
interdisciplinary practice was emphasized, and we partnered
with patients and families to ensure they have a say in how
care is delivered.
• Risk Assessment, Patient Safety and Transitions of Care
Jonathan Wilwerding, Ph.D., M.P.A., Alan White, Ph.D., Laura
Graham, B.A., George Apostolakis, Ph.D., Paul Barach, M.D.,
M.P.H., Brian Fillipo, M.D.
Presented by: Alan White, Ph.D., Senior Associate, Health
Services Research and Evaluation, Abt Associates Inc., 55
Wheeler Street, Cambridge, MA 02138; Tel: 617.349.2489; Email: alan_white@abtassoc.com
Research Objective: The nuclear power and aviation
industries have used probabilistic risk assessment (PRA) to
identify and mitigate the risk of accidents. Fault trees, for
example, are a useful method to evaluate the failure modes of
nuclear power plants and to analyze the risk of adverse events.
The success of these methods makes it natural to ask whether
they could be deployed to analyze and mitigate risks to patient
safety in hospitals. Transitions of care are moments in which
one hospital team or unit relinquishes responsibility for a
patient and another team or unit assumes it. Transitions
introduce the need for communication and coordination not
just between members of a single healthcare team, but across
teams or units. For this reason, it is natural to expect them to
be occasions on which the risk of medical error is particularly
high.
The proposed paper examines the applicability of fault tree
analysis to predict medical errors associated with transitions
of care. This application raises a series of methodological
issues. First, nuclear power plants are more or less
deterministic systems, in which component failures more or
less deterministically cause adverse events. Component
failures in hospitals frequently have only stochastic relation to
adverse outcomes. Second, component failures are rare in
nuclear power plants. Component failures in hospitals are
common. Partly for this reason, hospitals display very high
degrees of system redundancy. Finally, errors associated with
transitions will be caused, in some measure, by failures of
units to convey information and coordinate care. Modeling
the risk of error associated with transitions of care thus
requires us to treat failures to generate, interpret and convey
data as failure modes—along side failures to treat patients
consistent with accepted practice. Our paper seeks to
describe and, where possible, address these issues
confronting the application of PRA to patient safety and to
evaluate the potential value of PRA in healthcare settings.
Study Design: Develop fault tree model of transition from ER
to ICU for trauma patients. Estimate model using data from
incident reporting system and expert interviews.
Population Studied: Trauma patients at Geisinger Medical
Center.
Principal Findings: In principle, it is possible to devise fault
tree models of transitions from the ER to the ICU. But the
system's capacity for nearly continuous and its highly
stochastic nature make parameter estimation very difficult.
Conclusions: Absent models simpler than the ones that we
study, the application of PRA using fault tree analysis in will be
very restricted.
Implications for Policy, Delivery or Practice: The next stage
of research is to ascertain whether models can be made
simple enough, and data collection feasible. It will otherwise
be difficult for healthcare institutions to use the method to
analyze transitions of care.
Primary Funding Source: AHRQ
• Use of Electronic Medical Records to Reduce Falls
Among the Rural Elderly
Alan White, Ph.D., Valerie Weber, M.D., Donna Hurd, R.N.
Presented by: Alan White, Ph.D., Health Economist, Health
Services Research and Evaluation, Abt Associates, 55 Wheeler
Street, Cambridge, MA 02138; Tel: 617.349.2489; Fax:
617.349.2675; E-mail: alan_white@abtassoc.com
Research Objective: Falls are a common problem threatening
the independence of older individuals. One of every three
adults over the age of 65 in the United States suffers one or
more falls each year. Falls are the leading cause of injuryrelated deaths among the elderly population.
We evaluated the impact of a electronic medical record (EMR)
based intervention intended to improve outcomes for a rural
elderly population at risk for falls. We examined the impact on
the incidence of falls, utilization and costs, and medications.
Study Design: The intervention used the EMR to identify
elderly patients at risk of falls due to the number or types of
medications that they are taking. Their primary care physician
was notified of the results of a review of their medications and
provided with information on best practices for reducing falls.
In one clinic, physicians were also advised to refer the patient
to a multidisciplinary geriatric clinic.
Patients were assigned to intervention or comparison group
by clinic. There were two intervention clinics—one at which
only the medication review took place and a second at which
patients also received referral to the geriatric clinic. At the two
comparison clinics, patients received their usual patterns of
care.
Utilization and cost data were obtained from the EMR. Study
participants were also contacted quarterly to collect selfreported fall information. Patients’ primary care physicians
were surveyed to learn whether they had read the messages
they received and whether their care practices were influenced.
Population Studied: The study population included patients
at Geisinger Health Systems, which serves patients in rural
Pennsylvania. All of the patients in the study were aged 70 or
over and were identified at risk for falls based on their
medications and other risk factors identified in the literature.
Principal Findings: All respondents to the physician survey
read some or all of the, and more than 1/3 indicated that their
medical management had changed as a result, most often a
review of their patients for high-risk medications.
After six months, it appeared that the intervention was having
an impact on fall rates. Based on a combination of fall-related
medical encounters and self-reported falls, 23 percent of
comparison group members had one or more falls, compared
to 17 per cent of those in the intervention group. With regard
to medications, we found little impact on overall medication
use, but the intervention group had a reduction in the number
of psychoactive medications.
Given the large variance in costs across patients, it was not
possible to draw conclusions about the impact of the
intervention on costs.
Conclusions: Review of EMR can lead physicians to change
their medical management, reducing falls even for patients
who have yet to experience a fall. This is important because
most existing studies use a previous history of falls, not
medications, as the primary risk factor.
Implications for Policy, Delivery or Practice: Consequences
of falls may be more severe for the rural elderly, who are often
isolated and have difficulty accessing health services. This
type of EMR-based program is a relatively low-cost way to
identify patients at risk for falls and target them for medication
review.
Primary Funding Source: AHRQ
• Developing Relevant Quality Indicators for Rural
Hospitals?
Ira Moscovice, Ph.D., Doug Wholey, Ph.D., Jill Klingner, M.S.,
Astrid Knott, Ph.D.
Presented by: Doug Wholey, Ph.D., Professor, Health
Services Research & Policy, University of Minnesota, Box 729,
420 Delaware Street, S.E., Minneapolis, MN 55455-0392; Tel:
612.626.4682; E-mail: whole001@tc.umn.edu
Research Objective: Numerous organizations – including
JCAHO, Leapfrog, and NQF – have proposed new strategies
for the implementation of hospital quality measures and
systems. The multitude of efforts can lead to confusion about
how to measure quality. In addition, most of the efforts
consider all hospitals to be equal. However, rural hospitals
differ from urban hospitals in their smaller size, lesser
complexity, greater reliance on generalists, closer link to their
communities where they often are the only hospital, and their
referral link between rural patients and urban care facilities.
The goal of this study is to identify quality measures that are
relevant to the rural hospital context.
Study Design: First, we develop a conceptual model for
measuring rural hospital quality, with a focus on the special
issues (e.g., volume, condition prevalence, care delivery across
multiple sites) posed by the rural hospital context for quality
measurement. Second, with the assistance of a panel of rural
hospital and hospital quality measurement experts, hospital
quality measures from national and rural organizations are
reviewed for their fit to rural hospitals. Based on this analysis,
an initial core set of rural hospital quality measures is
recommended. Third, based on the special measurement
needs posed by the rural hospital context, we suggest avenues
for quality measure development. Finally, we discuss our
recommendations with a focus on issues related to
implementing and extending a rural hospital quality
measurement system.
Population Studied: Potential rural hospital quality measures
were identified by examining hospital quality measures
currently in use by accrediting organizations and hospital
associations. We focused on measurement sets from major
national organizations or measurement sets that are
predominantly used by rural hospitals.
Principal Findings: 346 measures were identified across 8
existing measurement sets. After eliminating duplicates and
combining similar measures, 68 measures in 13 categories
(diagnosis/condition specific, medication management,
infection and infection control, surgical complications,
surgical scheduling, emergency room, mortality, admission
rates, procedure rates, volume, length of stay, employee
health, financial, and other) were reviewed by the expert panel
with regard to prevalence, ease of data collection, and internal
and external usefulness in the rural context. Based on
rankings by the expert panel, 20 measures were identified as
relevant for rural hospitals.
Conclusions: Efforts for rural hospital quality measurement
face a number of issues specific to the environments of rural
hospitals. The 21 measures identified as relevant to rural
hospitals include 10 JCAHO core measures and measures
related to infection prophylaxis and medication errors, adverse
events and teaching, among others.
Implications for Policy, Delivery or Practice: Because of the
difficulty in implementing measurement efforts in
environments with limited prior experience and limited
technological resources, the implementation of a smaller core
measure set and the subsequent development of
measurement capacity seems advisable. A number of areas
not currently covered by hospital quality measures but
important to rural hospitals were also identified, including
triage and transfer decisions and care in the ER and
coordination of care with referral centers.
Primary Funding Source: HRSA
• Specificity and Sensitivity of Claims-Based Algorithms for
Identifying Members of Medicare+Choice Health Plans
that have Chronic Medical Conditions
Thomas Rector, Ph.D., Steven Wickstrom, M.S., Jeannette
Rogowski, Ph.D., N. Thomas Greenlee, M.S., Vicki Freedman,
Ph.D., John Adams, Ph.D., Jose Escarce, M.D., Ph.D.
Presented by: Steven Wickstrom, M.S., Director of Statistical
Modeling and Analysis, Applied Healthcare Research, Ingenix,
12125 Technology Drive, Eden Prairie, MN 55344-7302; Tel:
952.833.7089; Fax: 952.833.7090; E-mail:
steven.wickstrom@ingenix.com
Research Objective: To examine the effects of varying
diagnostic and pharmaceutical criteria on the performance of
claims-based algorithms for identifying beneficiaries with
hypertension, heart failure, chronic lung disease, arthritis,
glaucoma and diabetes.
Study Design: This study is a retrospective analysis of
algorithms for specificity and sensitivity for chronic conditions.
Physician, facility and pharmacy claims data were extracted
from electronic records for a sample of 3,633 continuously
enrolled beneficiaries who responded to an independent
survey that included questions about chronic diseases.
Population Studied: Secondary 1999-2000 data from two
Medicare+Choice health plans representing Midwestern and
Northeastern metropolitan areas.
Principal Findings: Compared to an algorithm that required a
single medical claim in a one year period that listed the
diagnosis, either requiring that the diagnosis be listed on two
separate claims or that the diagnosis to be listed on one claim
for a face-to-face encounter with a health care provider
significantly increased specificity for the conditions studied by
0.03 to 0.11. Specificity of algorithms was significantly
improved by 0.03 to 0.17 when both a medical claim with a
diagnosis and a pharmacy claim for a medication commonly
used to treat the condition were required. Sensitivity improved
significantly by 0.01 to 0.20 when the algorithm relied on a
medical claim with a diagnosis or a pharmacy claim, while
using two years of claims data significantly increased
sensitivity by 0.05 to 0.17. Algorithms that had specificity over
0.95 were found for all six conditions. Sensitivity above 0.90
was not achieved in all conditions.
Conclusions: Varying claims criteria had consistent effects on
the performance of case-finding algorithms for six chronic
conditions. Highly specific, and sometimes sensitive,
algorithms for identifying members of health plans with
several chronic conditions can be developed using claims
data.
Implications for Policy, Delivery or Practice: The results of
this study provide valuable information that can guide health
plans in using claims data to identify enrollees with chronic
conditions. Using these results can provide the opportunity
for health plans to enhance quality improvement and care
management efforts.
Primary Funding Source: AHRQ
• Developing Survey Items for a Health Plan Performance
Measure Assessing the Management of Urinary
Incontinence (UI) in M+C plans
Lok Wong, M.H.S., Philip Renner, M.B.A., Claudia Squire, M.S.
Presented by: Lok Wong, M.H.S., Senior Health Care Analyst,
Quality Measurement, National Committee for Quality
Assurance, 2000 L Street, N.W., Suite 500, Wahsington, DC
20036; Tel: 202.955.1784; Fax: 202.955.3599; E-mail:
wong@ncqa.org
Research Objective: To develop valid survey questions for
Medicare beneficiaries enrolled in Medicare+ Choice (M+C)
plans that will evaluate the quality of care received by seniors
with urinary incontinence to assess health plan performance.
Study Design: Cognitive testing of survey questions with
Medicare beneficiaries and validation of self-reported urinary
incontinence (UI). A clinical expert panel was convened to
develop measurement specifications and survey questions
based on existing guidelines. Via cognitive interviews with 87
seniors, respondents’ question comprehension, recall and
decision processes were tested with five new questions to
identify: UI status, whether a physician discussed and treated
the patient’s urine leakage. Validation of the new survey items
was also conducted against a validated instrument (Sandvik’s
Severity Index). Cross-tabulations of responses against
respondent characteristics were calculated and correlation
analyses conducted.
Population Studied: In 2002, a total of 87 Medicare
beneficiaries 65-85 years who had not participated in a health
study in the last 6 months were recruited into the study from
North Carolina, representing a diverse selection of education,
race, age, sex, type of residence and UI profile.
Principal Findings: Respondents understood examples of UI
treatment but were less familiar with behavioral therapies
such as exercises and bladder training. Less than half of all
respondents did considered “adult diapers” to be treatment
for UI. Most respondents had heard of the term “urinary
incontinence” defined as the “leakage of urine”. On whether
UI was “a problem” for the respondent was correlated with UI
frequency and severity. A three-point “problem” scale (big
problem, small problem, not a problem) captured more
respondents who displayed some UI symptoms than a (yes,
no) binomial scale. Analyses of patient self-reported UI
indicated a higher correlation with the frequency and severity
of UI using the “problem” scale (r=0.62) than the binomial
scale (r=0.28) for respondents with stress incontinence as well
as urge incontinence (r=0.75 compared to r=0.18). Education
did not appear to be a significant covariate. Correlations
between similar questions ascertaining patient perception of
the degree to which UI is a “problem” or a “bother” or affects
their daily activities were also high (r=0.87-0.89).
Conclusions: The study validated survey items to identify the
denominator population for a new HEDIS performance
measure that will assess how well health plans manage
urinary incontinence in M+C enrollees 65 years and older. The
clinical logic of the HEDIS measure and survey items assessing whether patients with self-reported UI problem
discussed UI with a physician and received treatment - is
supported by the study results.
Implications for Policy, Delivery or Practice: Rates of underdiagnosis and under-treatment for this debilitating condition
are expected to improve with the implementation of the new
HEDIS measure. Health plans can develop quality
improvement interventions to educate patients and providers
on UI. Appropriate and timely identification and management
of urinary incontinence will improve the quality of life and
functioning for seniors.
Primary Funding Source: CMS
• Incidences, Outcomes and Factors Associated with
Latrogenic Pneumothorax in Hospitalized Patients
Chunliu Zhan, M.D., Ph.D., Maureen Smith, M.D., Ph.D.,
Daniel Stryer, M.D.
Presented by: Chunliu Zhan, M.D., Ph.D., Senior Service
Fellow, Center for Quality Improvement and Patient Safety,
Agency for Healthcare Research and Quality, 540 Gaither
Road, Rockville, MD 20850; Tel: 301.427.1225; Fax:
301.427.1341; E-mail: czhan@ahrq.gov
Research Objective: To assess incidence rates, adverse
outcomes and factors associated with iatrogenic
pneumothorax (IP) in hospitalized patients.
Study Design: We identified IP cases by screening ICD-9-CM
code (512.1) in 7.45 million hospital discharge abstracts in the
AHRQ Healthcare Cost and Utilization Project, Nationwide
Inpatient Sample database. Overall IP incidence rates,
principal diagnoses and procedures associated with high IP
incidences were identified. Multivariable matching and
regression were used to examine the excess length of stay,
charges, and deaths attributable to and factors associated with
IP.
Population Studied: Hospitalized patients from 994 shortterm acute care hospitals across 28 states in 2000.
Principal Findings: Patients with a principal diagnosis of
pleurisy or with cancer of the kidney or renal pelvis had the
highest incidence rates of IP, at 2.24% and 1.14% respectively.
The procedures with highest rates were thoracentesis and
non-operating room (OR) therapeutic procedures on
respiratory systems, at 2.68% and 2.77%, respectively. About
8% of patients who were admitted for secondary malignancies
and underwent non-OR therapeutic procedures on respiratory
systems developed IP. Depending on age and comorbidities,
iatrogenic pneumothorax was associated with 4 to 7 excess
days in length of stay, between $17,000 and $45,000 in excess
charges, and 1% to 14% in excess mortality. IP rates were
highest among white female patients with multiple comorbid
diseases.
Conclusions: Iatrogenic pneumothorax is a significant threat
to hospitalized patients. The risks and adverse outcomes of IP
are variable depending on patient characteristics, principal
diagnoses, and procedures.
Implications for Policy, Delivery or Practice: Focusing on
procedures and patients with high risks for iatrogenic
pneumothorax could substantially improve patient safety.
Primary Funding Source: AHRQ
Invited Papers
Improving Quality of Care in the VA: Wins, Losses, Errors
& Ties
Chair: Lisa Rubenstein, M.D., M.S.P.H.
Monday, June 7 • 10:30 a.m.-12:00 p.m.
•
Panelists: Steven Asch, VA Greater Los Angeles
Healthcare System; Robert Brook, RAND; Bradley
Doebbeling, Indiana University-Purdue University
Indianapolis; Laura Petersen, Houston VA Medical
Center; Elizabeth Yano, VA Greater Los Angeles
Healthcare System (no abstracts provided)
Invited Papers
Public Reporting of Data on Quality: Why & How?
Chair: Judith Hibbard, Dr.P.H.
Monday, June 7 • 2:00 p.m.-3:30 p.m.
•
Panelists: Kristin Carman, American Institutes for
Research; Gregory Pawlson, National Committee for
Quality Assurance; Meredith Rosenthal, Harvard
University; Dana Gelb Safran, Tufts-New England Medical
Center (no abstracts provided)