Using routine data to measure
and promote safety and quality
in hospitals
Stephen Duckett
Adjunct Professor, ACERH/UQ
(Joint work with Dr Terri Jackson
Associate Professor, ACERH/UQ)
THE UNIVERSITY OF
WESTERN AUSTRALIA
Overview
 What do we know about patient safety and
patient harms in Australia?
 What do we count as patient harm?
 How can we measure and monitor it?
 How much does it cost?
 How can use of routine data help us understand
the economics of improving patient safety?
 Payment policy and patient safety
What do we know about
Australia?
 Landmark 1995 QAHC study:
•
•
•
•
Careful (expensive) methods
Incidence: 16.6 % of multi-day inpatient stays
Revised down to 10% (comparative US study)
Costs to system: $900 mil pa
 The newspapers:
• Bundaberg
• The NSW ‘Cam affair’ (Cambelltown & Camden)
• WA’s Royal Commission
 Australian Commission on Safety & Quality in
Health Care (Building on Aust Council for S&QHC (1999-2005)
What and how you ‘count’ depends on
why you want to count it
WHY count?
We need to monitor rates routinely:
using hospital discharge data
 ICD-10-AM has specific codes:
• T 80.0-88.9 ‘Complications of surgical & medical
care’
• ‘End of chapter’ codes
• Y 40-84.9 ‘External cause--complication of surgical
or medical care’
 Australia has world-class quality hospital data
 Victorian (and now Oz) ‘condition-onset flag’
(C-prefix) denoting:
• Condition required treatment or extended LOS
• Condition was not present on admission
Improving patient safety requires
hospital-acquired incident cases
 Problem of distinguishing ‘comorbidities’ from
‘adverse events’
 Incident vs prevalent cases
• Not GP, nursing home, admission from another hospital
• Overall rate in Victoria (2000/01): 8.25% (15.9% for multi-day
stays)
• Only two-thirds (5.61%) were ‘Incident’ cases
 The ‘C-prefix’ adds valuable information
• 41% of all hospital-acquired diagnoses were missed by ICD
alone, eg, UTI, atrial fibrillation, pneumonia
• Result: new national ‘Condition Onset’ flag on all Dx
Jackson TJ, Duckett SJ, Shepheard J, & Baxter KG. ‘Measurement of adverse events
using ‘incidence flagged’ diagnosis codes’ Journal of Health Services Research and
Policy, 11 (1):21-25; 2006.
Strengths of these data
 Strengths
•
•
•
•
•
Timely & cheap data collection
Standardised definitions and coding rules
Includes both dramatic and mundane
Independent reporting (not bedside)
Current payment incentives for thorough
coding
Weaknesses of these data
• May miss same-day patient harms
 Less coding investment
 Symptoms appearing post-discharge
• Prefix not currently audited
• Coders may miss clues clinicians could
spot
• No judgement about ‘preventability’
 Notorious inter-rater reliability problems in this judgement
 May be a ‘strength’ as today’s routine complication becomes
tomorrow’s ‘preventable’ adverse event…
Risk factors and outcomes for
hospital-acquired diagnoses (HA Dx)
 Age is an important predictor of HA Dx
• 29% of patients over 85 vs 9% in the 5-9 age group
 Risk of HA Dx varies considerably by medical specialty
• Gastroenterology 86.0% vs ENT surgery 10.5%
 ALOS strongly associated with HA Dx
• 11.2 days with HA Dx vs 6.9 days without
 ALOS both a risk factor and an outcome
 In-hospital mortality associated (not caused)
• 4.09% with HA Dx vs 1.75% without
Duckett SJ, Jackson TJ, Hong Son Nghiem (2008). Risk factors and outcomes for incident complications
in multi-day admissions to Victorian hospitals (manuscript).
Voluntary Sentinel Event reporting
compared with routinely-coded hospitalacquired diagnoses*
 Study Objective: To compare two sources of
data on hospital sentinel events to evaluate
strengths and weaknesses of both
 Problems with voluntary reporting:
•
•
•
•
Stigma and blame attached to involvement in adverse outcomes
Resulting reluctance to report
Safety-aware hospitals appear to have ‘worse’ outcomes
Focus on single events rather than rates reinforces individual
rather than system causation
Jackson T, Moje C, Shepheard J, and McMillan A. Poster presented to the 2007 Australian Conference on
Safety and Quality in Healthcare, August, 2007.
Using routine data to identify sentinel
events
This
Results
SE Reporting
Study
Victoria
National
05/06
05/06
04/05
SE1 Wrong patient or body part
13
25
53
SE2 Suicide in an inpatient unit
3
7
25
53
6
27
SE4 Intravascular gas embolism
6
0
1
SE5 Transfusion reaction
0
0
1
20
2
7
2
16
SE3 Retained instruments /material
SE6 Medication error
SE7(1) Maternal death (O95, O96, O97)
0
SE7(2) Any maternal death
0
Public Reporting
 Risk adjustment important for public
comparisons
• Reputation risk
• Risk of patients being denied treatment
• Fairness to providers
 Public officials expected to intervene when
quality declines
 What do you report?
• What providers have ‘done’
or
• What they are now doing to improve?
Any measurement approach must
balance the risks
‘GOLD STANDARD’
True Positive
True Negative
Identified
Positive
Identified
Negative

False Negative:
harm to future
patients
False Positive:
harm to reputations

Problems with public reporting using
routine clinical data
 Timeliness – data provided on an annual
basis, about 15 month delay in publication
 Mistrust of ‘administrative’ data
•
•
Not ‘risk adjusted’
Insufficient clinical detail
 Aggregated Data – unable to detect runs
 Nothing about cause of differences in rates
Queensland Health’s Variable Life
Adjusted Display (VLAD)
Plot of the cumulative difference between
expected and actual outcomes over a period
of time
AMI VLAD - ( July 2003 - March 2006 )
VLAD
Estimated Statistical Lives Gained
7
6
5
4
3
2
1
0
0
20
40
60
80
100
120
140
160
180
Case number
Coory M, Duckett SJ, Sketcher-Baker K. Using control charts to monitor quality of hospital care with
administrative data. International Journal for Quality in Health Care 20(1): 31-39.
.
Simplified Variable Life Adjusted
Display (VLAD)
Estimated Statistical Lives
Gained
Patient survives - VLAD increases by the probability
of the patient dying
2
1.5
1
0.5
0
-0.5 0
1
-1
-1.5
-2
Case number
2
Simplified Variable Life Adjusted
Display (VLAD) / 2
Estimated Statistical Lives Gained
Patient dies: VLAD decreases by the
probability of the patient surviving
2
1.5
1
0.5
0
-0.5
0
1
-1
-1.5
-2
Case number
2
Simplified Variable Life Adjusted
Display (VLAD) / 3
Estimated Statistical Lives Gained
2nd patient survives: VLAD increases by the
probability of the 2nd patient dying
2
1.5
1
0.5
0
0
1
2
-0.5
-1
-1.5
-2
Case number
3
Simplified Variable Life Adjusted
Display (VLAD) / 4
Estimated Statistical Lives Gained
2nd patient dies: VLAD decreases by the
probability of the 2nd patient surviving
2
1.5
1
0.5
0
0
1
2
-0.5
-1
-1.5
-2
Case number
3
Variable Life Adjusted Display
(VLAD)
Plot of the cumulative difference between
expected and actual outcomes
over a period of time
AMI VLAD - ( July 2003 - March 2006 )
VLAD
Estimated Statistical Lives Gained
7
6
5
4
3
2
1
0
0
20
40
60
80
100
Case number
120
140
160
180
Lessons:
 Sometimes statisticians’ methods are somewhat
dubious
 Be wary of replicating them, even if you think
you understand them
Specific Clinical Indicators
 Developed in consultation with clinical expert
groups
 Indicators reviewed annually and refined on basis
of feedback
 Data from the Queensland Hospitals Admitted
Patients Data Collection and the Perinatal Data
Collection
 Sex, age and comorbidities used to risk-adjust for
illness severity and co-existing conditions
Clinical Indicators
being developed
Mortality

Acute Myocardial Infarction




Heart Failure
Stroke
Pneumonia
Fractured Neck of Femur
Readmission







Complications of Surgery








Laparoscopic Cholecystectomy
Vaginal Hysterectomy
Abdominal Hysterectomy
Fractured Neck of Femur
Colorectal Carcinoma
Knee Replacement
Hip Replacement
Prostatectomy
Acute Myocardial Infarction
Heart Failure
Knee Replacement
Hip Replacement
Depression
Schizophrenia
Paediatric Tonsillectomy and Adenoidectomy
O&G



Future





Caesarean Section Rate
Std Primiparae Induction of Labour Rate
Std Primiparae Perineal Tears (3rd or 4th
Degree)
Hysterectomy – on women < 35 yo
Falls
Pressure ulcers
Overall Surgical Complications
AHRQ Patient Safety Indicators
Web-based documentation of VLADs
How bad is a ‘bad run’?
Variable Life Adjusted Display
with control limits
Statistical Lives Gained
5
0
0
50
100
-5
-10
-15
VLAD
Lower Control limit
-20
Upper Control limit
Case number
150
200
Flagging criteria
• 30% higher than expected mortality– automated
message emailed to the district manager and clinical
lead, encouraging internal investigation and report to
Area Clinical Governance Unit (50% for non-mortality
flags)
• 50% higher than expected – flagged to Area Clinical
Governance Units to ensure they are involved in further
investigation (75% non-mortality)
• 75% higher than expected - identified to State Patient
Safety and Quality board and in public reporting as
being statistically significantly different from the
average (100% non-mortality)
Duckett SJ, Coory M, Sketcher-Baker K. Identifying variations in quality of care in
Queensland hospitals. Medical Journal of Australia 187 (10): 571-575.
VLAD charts allow tracking to
specific case files
Detailed data underpinning the VLAD
system
QH Pyramid Model of Investigation
Professional
Process of Care
Structure of Resource
Patient Case Mix
Data
Figure 1 : Pyramid Model of Investigation
Key strength of routine data:
links to patient costs
 Victorian and Queensland CWS data
• n≈ 1.6 mil records pa
• n≈ 100 large public hospitals
• Validated for use in hospital funding
 What can cost data tell us?
• Economic burden of adverse events
• Economic priorities for prevention efforts
• Business case for:
 prevention efforts
 medical research
• Cost-effectiveness analyses for:
 prevention programs
 new patient safety devices and procedures
Incident cases represent a large economic
burden to the health care system
 Patients with at least one C-prefixed adverse
event:
• Stay nearly 10 days longer than other patients
• Cost $ 6826 more per episode (controlling for DRG, age and comorbidity)
 Extrapolated to entire hospital system:
•
•
•
•
At least $511.5 mil additional cost in Victoria (2003/04)
Adds 18.6% to hospital expenditures
Around $2 bil pa nationally
Even if only 40% preventable: $200 mil pa could be saved in
Vic; $800 mil nationally
Ehsani JE, Jackson TJ and Duckett SJ. ‘The incidence and cost of adverse events in
Victorian hospitals, 2003-04’ Medical Journal of Australia, 184;11; 5 June 2006
Re-admissions* add to this cost
burden
 Current work on Victorian separations with a
PDx in the T80-88.9 range of ICD-10-AM:
• 16,734 admissions with a PDx of a
‘complication of surgical or medical care’
• $70.6 mil pa additional public expenditure on
these cases in Victoria**
*Includes
admissions for HA Dx from primary care and nursing homes
**McNair P, Jackson TJ, Borovnicar D. ‘Costs of Victorian admissions for treatment of adverse-event
principal diagnoses, 2005/06’ submitted for publication, August 2008.
Queensland Health P4P approach
 Incentives at margin ($8M out of $7B budget,
0.1% of total, larger % of specialty)
 Range of specialties, data collection
approaches, foci
 Specialties:
•
•
•
•
•
Mental health
Stroke
Emergency department care
COPD
Medication
Duckett, S.J et al (2008) ‘P4P in Australia: Queensland’s New Clinical
Practice Improvement Payment’ Journal of Health Services
Research and Policy 13:174-177
Mental health indicators
Indicator
Patients with the DRG Schizophrenia seen
by a community mental health professional
within 7 days following discharge from the
same district mental health service
provider.
Recording of antipsychotic injection (depot)
medication on iPharmacy for DRG
Schizophrenia
Payment
amount etc
$100
Volume: 4518
Electronic data
collection in place
$100
Volume: 4518
Electronic data
collection in place
Acute stroke indicators
Indicator
Payment
amount etc
Acute Stroke patients with acute ischaemic
stroke receiving antiplatelet therapy within
$50
48 hours if clinically appropriate
Acute Stroke patients receiving dysphagia
screen (minimum requirement) within 24
hours
Volume: 6777
Manual data
collection system in
place
$125
Volume: 8699
Manual data
collection system in
place
Emergency indicator
Indicator
Payment amount
etc
All patients aged 65 years (or 50 years if ATSI) and
$100
over who are admitted to and discharged from an
Volume: 28962
emergency department to home/nursing home have
Electronic data
collection in place
evidence of communication back to the GP/LMO
Medication indicator
Indicator
Electronic Discharge Medication Record completed
where patient is over 65 years and has a complex
medication regimes
Payment amount
etc
$100
Volume: 70000
Electronic data
collection in place
COPD indicator
Indicator
Payment amount
etc
Pulmonary rehabilitation program meets an
acceptable standard as defined by the Statewide
Chronic Obstructive Pulmonary Disease (COPD)
Clinical Network.
$10 000
Volume: 20 (review of
unit)
12 month audit of all
sites (one off data
collection)
Queensland Health approach



Moral suasion will be used to
encourage most of (80%) CPIP payment
to flow to clinical unit where indicator
activity is being undertaken.
CPIP only to be used for non recurrent
expenditure items
CPIP to be used to enhance not expand
service (eg increase number of
operations performed)
CPIP - summary
 Started in January 2008
 Experiments with very different
approaches initially
 Mostly Pay for process
adherence/reporting rather than
outcomes
 Can be rolled out to other indicators,
add/drop indicators
 Stephen_duckett@health.qld.gov.au