Computer Assisted Evaluation of Clinical Data Quality
Nordic Casemix Conference
4.6.2010
Olafr Steinum, Sequelae AB
Seppo Ranta, Datawell Oy
2
Introduction

Reported health care information is widely used
by authorities
•
•
•
•


For health care planning
For quality analyses
For reimbursement
For science and research
It is of paramount importance that the reported
health data are correct and valid
Quality assurance is needed
•
•
Coding audits
Automatized controls
3
Datawell DRG QA for data quality assurance
 DRG
QA is a Datawell product
 Uses
three different logics for data quality evaluation
1.
2.
3.
 The
•
•
•
Rules and reference databases (e.g. diagnosis codes) used for NordDRG
grouping
Evaluation of the order of diagnoses (which one is the primary diagnosis, which
are secondary diagnoses) inferrred from large statistical database
(Normalization)
Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB
software can be used as
part of encoder system for immediate feedback of coding quality
standalone system for evaluation of organization data quality
benchmarking system comparing several peer organizations
4
Datawell DRG QA
 DRG
QA is a Datawell product ****
 Uses
Sequelae
ABevaluation
three different logics
for data quality
1.
2.
3.
Rules and reference databases (e.g. diagnosis codes) used for NordDRG
grouping
Is a joint colloboration between
Evaluation of the order of diagnoses (which one is the primary diagnosis, which
are secondary diagnoses) inferrred from large statistical database
(Normalization)
EmendorRulebase
Consulting
AB, (Staffan
Clinical Validation
(CVRB)
created Bryngelsson)
and maintained by Sequelae AB
•
software can be used as and
Olafr
part of encoder system
for Steinum
immediate(diaQualos
feedbackAB)
of coding quality
Gunnar
Henriksson
(DRG Henriksson
standalone system
for evaluation
of organization
dataAB)
quality
•
benchmarking system comparing several peer organizations
 The
•
5
Datawell DRG QA
 DRG
QA is a Datawell product
 Uses
three different logics for data quality evaluation
1.
2.
3.
 The
•
•
•
Rules and reference databases (e.g. diagnosis codes) used for NordDRG
grouping
Evaluation of the order of diagnoses (which one is the primary diagnosis, which
are secondary diagnoses) inferrred from large statistical database
(Normalization)
Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB
software can be used as
part of encoder system for immediate feedback of coding quality
standalone system for evaluation of organization data quality
benchmarking system comparing several peer organizations
6
DRG QA – An Example of Indicator Calculation Logic
PatId
Input data set
Dg-a
Dg-d
Pr
13213
LOS
Age
Dischg
Sex
3
134
HOME
M
43242
H10.1
J80
WX101
1
54
HOME
N
43242
E756
GD1BD
6
3
HOSP
N
64243
F0289
V02.0
2
41
HOME
N
34212
O75.7
4
34
HOME
M
MAF00
Validations
Missing principal diagnosis
PatId
Age not within acceptable limits
Local procedure
code
Dg-a
Dg-d
Pr
13213
Erroneous
ICD-10 code
LOS
Age
Dischg
Sex
3
134
HOME
M
43242
H10.1
J80
WX101
1
54
HOME
N
43242
E756
GD1BD
6
3
HOSP
N
64243
F0289
V02.0
2
41
HOME
N
34212
O75.7
4
34
HOME
M
External cause code as
principal diagnosis
MAF00
Mismatch of diagnosis and gender
7
DRG QA Pilot Benchmark Database
 Seven
•
•
DRG QA Database contains patient cases from the Ecomed KPP databases from
2008
Data source: Ecomed KPP used in the 7 hospitals
 Three
•
•
Hospital Districts in Finland
County Councils in Sweden
DRG QA Database contains all patient visits and stays from 2008
Data source: Patient Administrative Systems in corresponding county councils
 Number
•
•
of patient cases
Finland n = 4.928.113
Sweden n = 4.332.206
8
DRG QA Database Formation Process
Hospital Districts’
Ecomed KPP databases
(FI), or
similar data retrieval from
Patient Administrative
Systems (SE)
District A
District B
District C
County
Council A
County
Council B
County
Council C
etc.
Datawell
DRG QA
ETL
Datawell
DRG QA
Indicator
Calculation
Ecomed
DRG QA
Database
Ecomed
Analyzer
Reporting
Analysis of
Data Quality
• Data format
transformations:
hospital code 
common code mappings
• Calculation of DRG
grouping indicators
• DRG normalization
• Calculation of CVRB
matching
• Includes refence
population data
(1-year intervals) for
standardization
9
Results from the DRG QA Pilot Benchmark Database were presented
in the meeting.
10
The classification of diagnosis (ICD-10)
A
•
•
complex system for collecting data for statistics
Many axes
Many rules
• Explicit
rules
• Rules expressed in the Tabular volume in connection to code categories
• Rules assumed, but not explicitely expressed
 Clinical
•
validation rule base - CVRB
A collection of identified rules
Rate of Z51.1 Chemotherapy session as Principal or Secondary diagnosis.
Swedish county councils 2008
11
100%
90%
80%
70%
60%
50%
Huvuddiagnos
Principal
dx
Secondary
Bidiagnos dx
40%
30%
20%
10%
kh
ol
de
rm m
an
lan
d
Up
ps
ala
Vä
rm
lan
Vä
d
ste
r
b
Vä
ot
te
ste
n
rn
or
rla
Vä
stm nd
an
Vä
la
str
a G nd
öt
al
an
d
Ö
Ös reb
ro
te
rg
öt
la
nd
ÖÖ
Ri
ke
t
Sö
Sk
ån
e
St
oc
g
Ka
lm
ar
Kr
on
ob
er
No
g
rrb
ot
te
n
pin
d
kö
tla
n
Jö
n
nd
Jä
m
Ha
lla
bo
rg
nd
Gä
v le
Go
tla
rn
a
Da
la
Bl
e
kin
ge
0%
Data from Swedish National Patient Registry
Some examples of CVRB Rules 2009
Code not to be used
Not to be used for children < 15 years
Ought not to be used for children < 15 yrs
Not to be used in inpatient care
Ought not to be used in inpatient care
Rare code inpatient care
Must be combined with code2
Ought not to be combined with code2
Ought not to be used as principal dx
Not to be used as secondary dx
Ought not to be used as secondary dx
etc.
© Sequelae AB
12
CVRB violation rule
Violation Rule
CVRB33
CVRB32
CVRB31
CVRB30
CVRB29
CVRB28
CVRB27
CVRB26
CVRB25
CVRB24
CVRB23
CVRB22
CVRB21
CVRB20
CVRB19
CVRB18
CVRB17
CVRB16
CVRB15
CVRB14
CVRB13
CVRB12
CVRB11
CVRB10
CVRB 9
CVRB 8
CVRB 7
CVRB 6
CVRB 5
CVRB 4
CVRB 3
CVRB 2
CVRB 1
Cases with violation
13
Distribution of CVRB violation in test database (10 provinces)
14 000
12 000
10 000
8 000
6 000
4 000
2 000
0
Finland
Sw eden
14
CVRB Violation rules
0
5 000
10 0 0 0
15 0 0 0
CVRB 1
CVRB 2
CVRB 5
CVRB 7
CVRB 8
CVRB 9
CVRB10
Ought not to be used as principal dx
CVRB11
Not to be used as secondary dx
CVRB13
CVRB16
CVRB18
CVRB19
CVRB21
CVRB22
CVRB24
CVRB26
CVRB27
CVRB31
CVRB32
15
Information Process and the Identified Sources of Quality Failure
Usability and maintenance of
national code systems
(ICD, NCSP, DRG etc.)
Code systems
Entry of data
Processing of data
Utilization of data
and information
Human-Computer
interface
Processing rules and
logics of the information
systems
Feeding of structured
information into the PAS
Current transversal study
of the information process
16
Information Process and Benefits of Datawell DRG QA
Code systems
Entry of data
Processing of data
Utilization of data
and information
Immediate feedback of
coding results to coding
personnel
Information on organization
data quality for focusing
education and other corrective
actions.
Benchmarking data quality with
peer organizations.
Reports of data quality
incorporated with other
reporting
17
Data which nobody is using
has a quality that nobody
wants
Thank you!