Presentation outline
CNAC
N
A Comparison of Medical Error Reports
Submitted to a Voluntary Patient Safety
Reporting System by Different Types of
Reporters: A report from the ASIPS Collaborative
Daniel M. Harris, PhD, The CNA Corporation,
Wilson Pace, MD & Doug Fernald, MA, Univ of CO DFM,
Linda Marr, MS, The CNA Corporation
¾Background
¾Research objective
¾Study design
¾Findings
¾Conclusions & implications
AcademyHealth Annual Research Meeting
June 6, 2004
2
1
Background
Research objective
¾ Applied Strategies for Improving Patient Safety
(ASIPS) is a 3-yr AHRQ-funded demonstration
project based at the Univ of CO Dept of Fam Med
¾ Developed patient safety reporting system (PSRS)
¾ Medical error reporting systems are advocated as a
strategy for improving patient safety
™
™
™
Collect voluntary narrative reports of medical errors from
providers, other clinical staff, and non-clinical staff at
primary care practices in two CO PBRN
Anonymous or confidential-to-anonymous reports
Instructed to report “any event you don’t wish to have
happen again that might represent a threat to patient safety”
¾ Overall purpose of project is to test ability of the
PSRS to collect incident reports and to use them to
design interventions to improve patient safety
3
Study design
™
™
Reports are consensually coded into multi-axial taxonomy of
400+ attributes by teams of coders
Attributes characterize events by type of participants & their
contribution, complexity, setting, clinical intent, activities, underlying causes, patient outcomes, interventions, and “discoverer”
Attributes are coded “1” if present in report; “0” if not present
¾ Reporters self-classified as provider, other clinical
staff, or non-clinical staff
™
Learning from errors requires receiving reports on representative range of errors
¾ Limited evidence exists re: nature of events reported to such systems by different reporter types,
especially in ambulatory primary care settings
™
Do different types tend to report different types of events?
¾ Our objective is to (1) analyze reports submitted to
ASIPS PSRS to ascertain “who reports what,” (2)
identify similarities and differences in reports, and
(3) characterize differentiating event attributes by
reporter type
4
Findings
¾ Error incident reports are received from 34 primary care
practices throughout Colorado
™
™
¾ Analysis of report
Reports by reporter type
Nature of report
™ Event “discoverer”
™
™
¾ Analysis of event reported
Participants
Patient harm
™ Communication errors
™ Other event characteristics
™ Discriminant analysis
™
™
522 (85.8%) of 608 reports received thru Aug 2003 identified a
reporter type
¾ Report content compared by reporter type using cross
tabs, ANOVA, and discriminant analysis
5
6
1
Reports by reporter type
¾ Most reports (68.6%)
from providers
™
Unchanged from yr1 to yr2
¾ About 25% from other
clinical staff
™
Increased from yr1 to yr2
¾ Fewest reports (6.9%)
from non-clinical staff
™
Deceased from yr1 to yr2
despite project effort to
increase participation
Nature of report
Percent of Reports within
Year, by Reporter Type
¾ No difference by
reporter type:
80
¾ Differences by type:
™ Patient
40
20
0
Prov
Oth Clin Non-Clin
Total
Yr 1
age
• Mean
lowest for nonclin; highest for provider
Patient gender
™ Perceived preventability
™ Perceived pt knows
™ Mean taxonomy,
event activity, and
“don’t know” codes
™
60
™ At
least 1 “insufficient
information” code
• Non-clin
™ Noted
most likely
in Med Rec’d
• Non-clin
least likely
™ Perceived
• Oth
Yr 2
pt harm
clin least likely
7
8
Event “discoverer”
Event participants
¾ Providers & other clinical
staff likely to report events
discovered by caregivers
like themselves
¾ Non-clinical staff likely to
report events discovered
by office staff like
themselves
¾ Providers somewhat less
likely to report events
discovered by patients or
patient’s families
¾ Each reporter type is
more likely to report an
event with a participant of
the same type
Percent of reports discovered by
various parties, w/in reporter
type
100
80
™
60
All relationships statistically
significant by χ2
¾ No significant association
by reporter type for other
types of participants:
40
20
™
™
0
Caregvr
Prov
Ofc Staff
Oth Clin
Pt/Fam
Percent of reports with
selected type of participants,
within reporter type
60
50
40
30
20
10
0
Patient/family or 3rd party
Number of participants or
number of participant types
(by ANOVA)
Prov
Prov
Non-Clin
Oth Clin
Oth Clin
NonClin
Non-Clin
9
Patient harm
Communication
¾ Overall, 26.4% of reported events coded w/some
form of patient harm
™
Additional 7.7% of reports,
coded w/patient may have
been harmed, but too early
to tell
¾ Significant differences in
type of harm (by χ2):
Provider reports most
likely coded w/clinical harm
™ Non-clinical staff reports
most likely coded with nonclinical harm
™ Other clinical staff
reports least likely coded
with any harm
™
10
¾ Over half (57%) of reports
involve a communication
error between parties
Percent of reports coded with
patient harm, w/in reporter
type
35
30
25
20
15
10
5
0
™
Clin Harm Non-Clin Any Harm
Harm
Prov
Oth Clin
Non-Clin
11
Each reporter type likely to
report communication
involving their own type
¾ Oth clin staff less likely
to report within office
communication errors
¾ Overall, non-clin staff
less likely to report
communication errors
¾ Providers more likely to
report communication
involving patients
50
Percent of reports with
communication errors
involving selected parties,
w/in reporter type
40
30
20
10
0
Att Prov
Party
Prov
Non-Phy
Party
Oth Clin
Ofc Staff
Party
Non-Clin
12
2
Other event characteristics
Discriminant Analysis (1)
¾ No difference by
reporter type:
¾ Identify event attributes that differentiate be
tween reports submitted by each reporter type
™
™
™
™
™
™
™
™
Missing information
General proc issue
Medication error
Supervision error
Intervention following
error detection
Judgement error
Clinical knowledge or
skill error
Resource function or
availability error
¾ Differences by type:
™ Documentation
• Oth
clin most likely
™ Specific
in Dx and Tx – prov
most likely
• Delay in testing – oth clin
staff most likely
• Dx testing error – non-clin
least likely
™ Pt
mgmt error
• Non-clin
• Prov
• Prov
•
•
Group centroids are significantly different
Provider reports are most different from other 2 types
Canonical correlations (~ANOVA Eta) for the
functions are .455 & .346; jointly account for 30% of
variance in group scores
™ Discriminant scores correctly classify 63% of reports
™ Classify unknown reports in similar proportion
14
™
most likely
™ Distraction/inattention
least likely
™ System
Stepwise method: attributes enter that maximize
distance between 2 closest groups (16 of 32 entered)
™ Resulting 2 discriminant functions differentiate
between the 3 reporter types
™
proc issues
• Delay
issue
least likely
13
Discriminant Analysis (2)
Conclusions
¾ The analysis identified attributes that best discriminate
between events reported by each reporter group
¾ Provider reports most differentiated by:
¾ Different types of reporters tend to submit
reports of different kinds of medical errors to a
voluntary PSRS
™
provider participant, communication involving a provider,
disclosure to a patient, delay in diagnosis, diagnostic testing
error, and problem with resource (availability or function)
¾ Other clinical staff reports most differentiated by:
™
Non-physician provider participant, third party participant,
communication involving a provider, communication within the
office, delay in performing a test, diagnostic testing error, and
misuse of a system
¾ Non-clinical staff reports most differentiated by:
™
Non-clinical staff participant, non-clinical harm, error in patient
management, distraction/inattention error, misuse of a system, and malfunction of a system
Each type of reporter tends to report errors
involving the kinds of activities and participants
they can be most expected to observe and believe
should be reported
™ For example:
™
•
•
Providers tend to report clinical harm events while nonclinical staff tend to report non-clinical harm events
Providers tend to report events involving clinical procedures while non-clinical staff tend to report patient
management and system issue events
15
Implications
The CNA Corporation
¾ To be effective in identifying a full and repre
sentative range of errors and threats to patient
safety, a voluntary PSRS needs to assure that
it receives incident reports from all types of
staff (and patients?) who are in a position to
observe errors
™
16
Questions?
Provider exclusive or dominated reporting systems
will restrict this range
¾ Our experience demonstrates the difficulty of
obtaining full participation of non
- providers
™
More effort necessary to achieve their participation
17
18
3