Assuring Data Quality
Jennie Wilson
Programme Leader – SSI Surveillance
Dept. of Healthcare-Associated Infection & Antimicrobial Resistance,
Health Protection Agency
8.0%
2005 Hip prosthesis: inter-country rate (cumulative incidence)
SSI cumulative incidence (%)
7.0%
6.0%
5.0%
4.0%
3.0%
2.0%
1.0%
0.0%
AT
BE
DE
ES
FI
FR
HU
LT
NL
NO
PL
UE
UN
US
UW
1166
544
30478
379
6103
4844
1203
474
6081
1009
1325
39684
3941
8764
2250
N=6
N=6
N=100
N=5
N=12
N=200
N=7
N=6
N=34
N=20
N=17
N=172
NA
NA
NA
N in-hospital SSI/1000 post-op. pt.-days
7
2005 Hip prosthesis: inter-country rate (incidence density)
6
5
4
3
2
1
0
AT
BE
DE
ES
FI
FR
HU
LT
NL
NO
UE
UN
US
UW
1164
544
5987
379
3896
4775
1203
473
6011
1007
39678
3933
8462
2098
N=6
N=6
N=54
N=5
N=6
N=200
N=7
N=6
N=33
N=20
N=172
NA
NA
NA
External benchmarks
External benchmarks are a powerful driver
for effecting change, but require
standardised data collection methods
standardised analysis
high data quality
central co-ordination
Gaynes 1997
Why is data quality so
important locally?
Do you know whether action is required?
• real problems?
• poorly collected data?
SSI Surveillance
Basic methodology
Targeted at categories of clinically similar
operative procedures
Data collection form completed for each relevant
operation (denominator)
Systematic (active) surveillance after each
operation to detect infections (numerator)
Methods of identifying patients
with SSI (numerator)
Active
Designated, trained personnel, use a variety of data sources to
determine whether an HAI has occurred
Sensitivity = 85-100%
Passive
HAI identified and reported by people other than designated,
trained personnel. Requires fewer people but unreliable,
definition not applied consistently
(Perl, 1998)
Sensitivity: 14-34%
Surveillance methods:
Sensitivity of case finding
Lab-based phone
Sensitivity 36%
1.2hrs / 100 beds / week
Temperature / treatment chart
Sensitivity 65%
6.5 hours / 100 beds / week
Lab-based, ward liaison
Sensitivity 76%
6.4 hours / 100 beds / week
Glenister et al 1992
Systematic surveillance for SSI
Lab-based ward liaison
1. Visit ward/patient 3 times per week
–
discuss patients with ward staff
–
check medical / nursing records
–
check temperature / treatment charts
2. Review microbiology reports daily
–
identify positive surgical site reports
Definitions of surgical site
infection (CDC)
Superficial incisional
•
•
involves only skin or subcutaneous
tissue
occurs within 30 days of surgery
Deep incisional
•
•
involves fascial or muscle layers
occurs within 30 days, implants within 1
year
Organ/space
•
•
•
part of anatomy opened / manipulated
infection appears related to surgery
occurs within 30 days, implants within 1
year
Superficial Incisional Infection
Must meet one of the following criteria:
1. Purulent drainage from superficial incision
2. Culture of organisms from fluid/tissue
3. At least 1 symptom of inflammation (pain,
tenderness, localised swelling, redness, heat) and
incision deliberately opened to manage infection
4. Clinicians diagnosis of superficial SSI
Deep Incisional Infection
Must meet one of the following criteria:
1. Purulent drainage from deep incision
2. Deep incision dehisces / deliberately opened and
patient has 1 symptom : fever, localised
pain/tenderness
3. Abscess / other evidence of infection in deep
incision: re-operation / histopathology / radiology
4. Clinicians diagnosis of deep infection
Identifying SSI
Review patients systematically whilst they are in hospital
Do not rely on reviewing case-notes after discharge to find
SSIs
If a patient is prescribed antibiotics do not assume these are
for SSI – check with clinician
Check significance of positive microbiology cultures
Make sure any SSI identified post-discharge also meets the
definition
Is this an SSI…….?
Nursing record states:
‘Wound oozing ++ from small lower section.
Pressure dressing applied’
Oozing what:
•Clear (serous) fluid, blood, pus?
What was the condition of the suture line?
•Red, swollen, dehisced
Was a wound swab taken, if so why?
Criteria for SSI checklist
Weblink data entry (SSISS)
Validation studies
Mannien et al 2007: PREZIES, Netherlands
• Reviewed 859 medical charts; 149 SSI
• Validation team = ‘gold standard’
• PPV = 0.97; NPV = 0.99
McCoubrey et al 2005: SSI surveillance, Scotland
• 91 SSI reported validated by case note review
• 10/27 hospitals criteria not recorded
• PPV 94.6% (95%CL 87.9 – 98.2); NPV 99.4 (95% CL 98.3 – 99.9)
(assuming not recorded data valid)
NNIS SSI ‘Risk Index’
Each operation is scored, and results stratified,
using 3 major risk factors associated with SSI*:
• ASA pre-operative assessment score
• Wound class
• Duration of surgery (T time)
Score between 0 and 3
*Culver et al (1991)
Risk Index factors
ASA classification of physical
illness
Wound classification
1: normal healthy patient
Clean: no signs of infection, no
body ‘tracts’
2: mild systemic disease
Clean-contaminated: body tract
entered
3: severe systemic disease
4: incapacitating systemic disease
5: moribund, little chance of
survival
Contaminated: spillage form GIT,
inflammation, open trauma
Dirty: pus, perforated viscera,
delayed open trauma, faecal
contamination
Changed by pre-op and intra-op
events
T time
association between p value and cut point for duration
of operation (abdominal hysterectomy)
Leong et al 2006
Large bowel surgery
Trend in rate
of SSI by Risk
index group
25
% infected
20
15
10
5
0
0
35
% infected
30
25
20
15
10
5
0
1
2
3
Risk Index Group
2
3
Risk Index Group
Vascular surgery
0
1
u/k
All
u/k
All
Effect of indirect standardisation on
crude rates of SSI (vascular surgery)
20
15
Rate of
SWI (%)
10
5
0
crude
adjusted
Bi
le
Ab
do
m
ia
liv
lh
er
ys
o
r
te
C
pa
or
re
on
nc
ct
om
re
ar
at
y
y
Ar
ic
te
su
ry
rg
By
er
y
pa
ss
G
G
as
ra
To
tri
ft
c
ta
su
lh
rg
ip
er
re
y
H
p
ip
la
c
he
em
m
en
ia
t
rth
Kn
ro
pl
ee
as
re
ty
La
pl
ac
rg
em
e
bo
en
w
t
el
s
L
O
ur
im
pe
ge
b
n
ry
a
re
m
du
pu
ct
ta
io
tio
n
n
Sm
of
f
al
ra
lb
ct
ur
ow
e
el
s
Va
ur
ge
sc
ul
ry
ar
su
rg
er
y
du
ct
,
Distribution of the incidence of surgical site infection
by category of surgical procedures
25
Percentiles
Source: SSI Surveillance Service, CDSC
90th
75th
20
50th
25th
10th
15
10
5
0
October 1997 to December 2003
Crude rates of SWI for
vascular surgery (95%
45
CL)
by hospital
40
Rate of SWI (%)
35
Hospital
30
25
20
15
90th percentile
10
50th percentile
5
0
0
5
10
15
20
25
30
35
40
Order
Data to December 2001
25
Cumulative incidence
20
95% CI
99% CI
5
10
15
Hospital
0
Cumulative incidence
Funnel plots used to account
for variation in sample size
0
1000
2000
3000
number of operations
4000
Total hip prosthesis, January 2000 – March 2005
Cumulative incidence
Total hip
prosthesis
1.24
Hip
hemiarthroplasty
4.05
Knee prosthesis
0.65
Open reduction
long bone fracture
2.01
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
% infected
Incidence density
Total hip prothesis
1.36
Hip hemiarthroplasty
2.29
Knee prosthesis
0.76
Open reduction of long bone fracture
1.67
0.0
0.5
1.0
1.5
2.0
Incidence density per 1000 days
2.5
4.0
4.5
Funnel plots to adjust for variation in
sample size and length of post-op stay
15
Incidence density/ 1000 post-op in-patient days
95% CI
99% CI
0
5
10
Hospital
0
10000
20000
30000
in-patient post-operative days
40000
Length of stay in
elective surgery is reducing
Median length of stay
Total hips
Total knee
Hip hemi
16
14
12
10
8
6
4
2
0
1998 1999 2000 2001 2002 2003 2004
Year
Post discharge
Pre-discharge
40
30
20
Number of SSIs
Proportion of SSI detected
pre & post discharge Barrett et al 2000
60
50
10
0
y
om ns
ct io
er te s
th s Le
O ecy of
l
ho on
C sici e
ci tiv r
Ex ora pai
p l Re
m
Ex a y
yo
ni
/M
er om
O
H rot
n S
pa ea /B
La sar my
o
ae ct
C ere ein
t
V
ys e
H os
y
ric
m el
Va ial cto ow
r
te ice B
Ar d g e
r
n
pe La ct
e
nc
Ap l & ra
al I T der ara
Sm er G lad cle
y
pp B ar
U al & xill
a
en
R st & ck
e
ea N
Br d & ic
d
ea e
H opa
rth
O
Post-discharge surveillance
study Barrett et al 2000
Post-discharge surveillance method
Resources +++ - data collection, informing/contacting patients
General practitioners/district nurses – poor response rate to
questionnaire
Patients – better response; +/- reliability
Sensitivity of case-finding
active vs. passive surveillance
reliability
Response rate to PDS
patient questionnaires
Response rate affected by ethnic group and age
22%
27%
No Response
Response No Reminder
Response With Reminder
n = 6159
51%
Barrett et al 2000