How accurate are Scottish
Morbidity Record (SMR01) data
for elective AAA procedures?
Sarah Couper
John Connor
Donna Nicholson
Jennifer Armstrong
SpR in Public Health Medicine
Principal Information Analyst, ISD
Senior Information Analyst, ISD
Senior Medical Officer, Scottish Government
Background
• Scottish screening programme for AAA due for
phased roll-out 2011/2012
• Some evidence exists which shows that the
outcome of AAA surgery is dependent on the
number of interventions per unit
• NHS QIS has produced clinical standards which
refer to units undertaking a minimum of 20
elective interventions each year. (Level B)
• We wanted to answer the question of how many
interventions were happening in Scottish units
SMR submission process
SMR PATIENT DATA FLOWS
Sources
Secondary Care
ISD
SMR DATA RECORDING
RULES
VALIDATION RULES
REFERENCE FILES
HOSPITAL PAS/PMS
M
A
I
N
Elective referrals
Emergency
H
O
S
P
I
T
A
L
A/E
Patient Types
OUTPATIENTS
DAY CASES
INPATIENTS
New SMR?
SPECIALTY
SIGNIFICANT
FACILITY
CONSULTANT/HCP
D
I
S
C
H
A
R
G
E
/
T
R
A
N
S
F
E
R
&
C
L
I
N
I
C
A
L
C
O
D
I
N
G
D
A
T
A
S
M
R
D
A
T
A
V
A
L
I
D
A
T
I
O
N
Submission
of SMRs
V
E
R
I
F
I
C
A
T
I
O
N
Data Processing &
Monitoring
Data Users
COMPLETENESS
OF DATA
FLAT FILES
RECORD LINKAGE
TIMELINESS OF
DATA
DATA QUALITY
DERIVED DATA
ITEMS
CORRECTING
ERRORS
DATA MARTS
INFORMATION
REQUESTS
PQs
PUBLICATIONS
What we asked for
– date of discharge between 1st April
2007 to 31st March 2010
– elective cases only, emergencies and
transfers excluded
– case attributed to main consultant
– specific ICD10/OPCS4 code
combinations (supplied by Julie
Brittenden)
Diagnosis and procedure codes
Diagnosis Code
I71.4 Abdominal aortic aneurysm, without mention of rupture
Procedure Codes
Open Repair
L19.4 Replacement of aneurysmal segment of infrarenal abdominal aorta by
anastomosis of aorta to aorta NEC
L19.5 Replacement of aneurysmal segment of abdominal aorta by anastomosis of aorta to
aorta NEC
L19.6 Replacement of aneurysmal bifurcation of aorta by anastomosis of aorta to iliac artery NEC
L19.8 Other specified other replacement of aneurysmal segment of aorta.
L19.9 Unspecified other replacement of aneurysmal segment of aorta.
L21 Other bypass of segment of aorta
EVAR
L27.1 Endovascular insertion of stent graft for infrarenal abdominal aortic aneurysm.
L27.5 Endovascular insertion of stent graft for aortic aneurysm of bifurcation.
L28.1 Endovascular stenting for infrarenal abdominal aortic aneurysm.
L28.8 Other specified transluminal operations on aneurysmal segment of aorta.
L28.9 Unspecified transluminal operations on aneurysmal segment of aorta.
Outline of data collection
process
ISD generates case numbers based on
agreed criteria
Data reviewed by consultant
for accuracy
Consultant provides updated case numbers
Consultant provides updated case numbers
and requests patient details for verification
ISD provides patient level case listings
Consultant reviews data and verifies
against local records
Consultant sends additional case list with patient
identifiers back to ISD
ISD cross checks consultant responses
against SMR01 and original criteria
Why did we ask for patient identifiers?
• Investigate the discrepancies in the data and
correct the system for future use
• Avoid double counting of cases
• Clarify cases fit the case definition
44
Consultants
contacted
15
No reply
29 (66%)
Replied
9 (20%)
Data reported as inaccurate
by consultant but no
amended data provided
2 no reason
3 too much time
4 in progress
12 (27%)
Provided data
No patient identifier
8 (18%)
Provided patient
identifiers
Method
Number of interventions from SMR01 data
+
Number of additional cases reported by
those consultants who responded
=
Total number of interventions
Number of elective
AAA interventions
Number of elective AAA interventions per
Health Board per year
70
60
50
40
30
20
10
0
1
2
3
4
5
6
Health Boards
7
8
9
10
Level of discrepancy
* only refers to those consultants who responded
to exercise
Discrepancy reported by responding
consultants between SMR01 data
and their records
-43%
0%
-20%
-23%
-20%
-62%
-10%
-62%
-17%
-32%
Analysis of the discrepancies for
one Health Board
Different codes
Emergency/transfer
Out with date range
Agreement with
SMR01 data
SMR01
Consultants' records
87%
Agreement
Conclusions
• Selecting search criteria and ensuring these are adhered to is a
complex exercise
• Cross-checking data with consultants was challenging due to
workloads and data confidentiality/disclosure issues
• Data collection process could perhaps have been streamlined by
asking consultants to provide patient level case details direct to ISD
for verification
• Clinicians sometimes query the accuracy of nationally collected data
• Provisional analysis of the discrepancies show that many seem to
be due to data extraction rather than inherent inaccuracy within the
SMR01 records
• Further work cross-checking clinicians’ records to discuss the
discrepancies with them would have been useful but out with the
scope of this exercise