CNAC
N
A Comparison of Medical Error Reports
Submitted to a Voluntary Patient Safety
Reporting System by Different Types of
Reporters: A report from the ASIPS Collaborative
Daniel M. Harris, PhD, The CNA Corporation,
Wilson Pace, MD & Doug Fernald, MA, Univ of CO DFM,
Linda Marr, MS, The CNA Corporation
AcademyHealth Annual Research Meeting
June 6, 2004
1
Presentation outline
Background
Research objective
Study design
Findings
Conclusions & implications
2
Background
 Applied Strategies for Improving Patient Safety
(ASIPS) is a 3-yr AHRQ-funded demonstration
project based at the Univ of CO Dept of Fam Med
 Developed patient safety reporting system (PSRS)



Collect voluntary narrative reports of medical errors from
providers, other clinical staff, and non-clinical staff at
primary care practices in two CO PBRN
Anonymous or confidential-to-anonymous reports
Instructed to report “any event you don’t wish to have
happen again that might represent a threat to patient safety”
 Overall purpose of project is to test ability of the
PSRS to collect incident reports and to use them to
design interventions to improve patient safety
3
Research objective
 Medical error reporting systems are advocated as a
strategy for improving patient safety

Learning from errors requires receiving reports on representative range of errors
 Limited evidence exists re: nature of events reported to such systems by different reporter types,
especially in ambulatory primary care settings

Do different types tend to report different types of events?
 Our objective is to (1) analyze reports submitted to
ASIPS PSRS to ascertain “who reports what,” (2)
identify similarities and differences in reports, and
(3) characterize differentiating event attributes by
reporter type
4
Study design
 Error incident reports are received from 34 primary care
practices throughout Colorado



Reports are consensually coded into multi-axial taxonomy of
400+ attributes by teams of coders
Attributes characterize events by type of participants & their
contribution, complexity, setting, clinical intent, activities, underlying causes, patient outcomes, interventions, and “discoverer”
Attributes are coded “1” if present in report; “0” if not present
 Reporters self-classified as provider, other clinical
staff, or non-clinical staff

522 (85.8%) of 608 reports received thru Aug 2003 identified a
reporter type
 Report content compared by reporter type using cross
tabs, ANOVA, and discriminant analysis
5
Findings
 Analysis of report
Reports by reporter type
 Nature of report
 Event “discoverer”

 Analysis of event reported
Participants
 Patient harm
 Communication errors
 Other event characteristics
 Discriminant analysis

6
Reports by reporter type
 Most reports (68.6%)
from providers

Unchanged from yr1 to yr2
 About 25% from other
clinical staff

Increased from yr1 to yr2
 Fewest reports (6.9%)
from non-clinical staff

Deceased from yr1 to yr2
despite project effort to
increase participation
Percent of Reports within
Year, by Reporter Type
80
60
40
20
0
Prov
Oth Clin Non-Clin
Total
Yr 1
Yr 2
7
Nature of report
 No difference by
reporter type:
Patient gender
 Perceived preventability
 Perceived pt knows
 Mean taxonomy,
event activity, and
“don’t know” codes

 Differences by type:
 Patient
age
• Mean
lowest for nonclin; highest for provider
 At
least 1 “insufficient
information” code
• Non-clin
 Noted
most likely
in Med Rec’d
• Non-clin
least likely
 Perceived
• Oth
pt harm
clin least likely
8
Event “discoverer”
 Providers & other clinical
staff likely to report events
discovered by caregivers
like themselves
 Non-clinical staff likely to
report events discovered
by office staff like
themselves
 Providers somewhat less
likely to report events
discovered by patients or
patient’s families
Percent of reports discovered by
various parties, w/in reporter
type
100
80
60
40
20
0
Caregvr
Prov
Ofc Staff
Oth Clin
Pt/Fam
Non-Clin
9
Event participants
 Each reporter type is
more likely to report an
event with a participant of
the same type

All relationships statistically
significant by 2
 No significant association
by reporter type for other
types of participants:


Patient/family or 3rd party
Number of participants or
number of participant types
(by ANOVA)
Percent of reports with
selected type of participants,
within reporter type
60
50
40
30
20
10
0
Prov
Prov
Oth Clin
Oth Clin
NonClin
Non-Clin
10
Patient harm
 Overall, 26.4% of reported events coded w/some
form of patient harm

Additional 7.7% of reports,
coded w/patient may have
been harmed, but too early
to tell
 Significant differences in
type of harm (by 2):
Provider reports most
likely coded w/clinical harm
 Non-clinical staff reports
most likely coded with nonclinical harm
 Other clinical staff
reports least likely coded
with any harm

Percent of reports coded with
patient harm, w/in reporter
type
35
30
25
20
15
10
5
0
Clin Harm Non-Clin Any Harm
Harm
Prov
Oth Clin
Non-Clin
11
Communication
 Over half (57%) of reports
involve a communication
error between parties

Each reporter type likely to
report communication
involving their own type
 Oth clin staff less likely
to report within office
communication errors
 Overall, non-clin staff
less likely to report
communication errors
 Providers more likely to
report communication
involving patients
50
Percent of reports with
communication errors
involving selected parties,
w/in reporter type
40
30
20
10
0
Att Prov
Party
Prov
Non-Phy
Party
Oth Clin
Ofc Staff
Party
Non-Clin
12
Other event characteristics
 No difference by
reporter type:








Missing information
General proc issue
Medication error
Supervision error
Intervention following
error detection
Judgement error
Clinical knowledge or
skill error
Resource function or
availability error
 Differences by type:
 Documentation
• Oth
clin most likely
 Specific
proc issues
in Dx and Tx – prov
most likely
• Delay in testing – oth clin
staff most likely
• Dx testing error – non-clin
least likely
• Delay
 Pt
mgmt error
• Non-clin
most likely
 Distraction/inattention
• Prov
least likely
 System
• Prov
issue
least likely
13
Discriminant Analysis (1)
 Identify event attributes that differentiate between reports submitted by each reporter type
Stepwise method: attributes enter that maximize
distance between 2 closest groups (16 of 32 entered)
 Resulting 2 discriminant functions differentiate
between the 3 reporter types

•
•
Group centroids are significantly different
Provider reports are most different from other 2 types
Canonical correlations (~ANOVA Eta) for the
functions are .455 & .346; jointly account for 30% of
variance in group scores
 Discriminant scores correctly classify 63% of reports
 Classify unknown reports in similar proportion
14

Discriminant Analysis (2)
 The analysis identified attributes that best discriminate
between events reported by each reporter group
 Provider reports most differentiated by:

provider participant, communication involving a provider,
disclosure to a patient, delay in diagnosis, diagnostic testing
error, and problem with resource (availability or function)
 Other clinical staff reports most differentiated by:

Non-physician provider participant, third party participant,
communication involving a provider, communication within the
office, delay in performing a test, diagnostic testing error, and
misuse of a system
 Non-clinical staff reports most differentiated by:

Non-clinical staff participant, non-clinical harm, error in patient
management, distraction/inattention error, misuse of a system, and malfunction of a system
15
Conclusions
 Different types of reporters tend to submit
reports of different kinds of medical errors to a
voluntary PSRS
Each type of reporter tends to report errors
involving the kinds of activities and participants
they can be most expected to observe and believe
should be reported
 For example:

•
•
Providers tend to report clinical harm events while nonclinical staff tend to report non-clinical harm events
Providers tend to report events involving clinical procedures while non-clinical staff tend to report patient
management and system issue events
16
Implications
 To be effective in identifying a full and representative range of errors and threats to patient
safety, a voluntary PSRS needs to assure that
it receives incident reports from all types of
staff (and patients?) who are in a position to
observe errors

Provider exclusive or dominated reporting systems
will restrict this range
 Our experience demonstrates the difficulty of
obtaining full participation of non-providers

More effort necessary to achieve their participation
17
The CNA Corporation
Questions?
18