Cameron Willis
Measuring the Process-Outcome link
with Composite Indices in Trauma
services
Trauma, Trauma Systems & Quality Indicators
•
Leading cause of mortality in those aged 1 to 44 years of age
•
Optimal outcomes from trauma are largely dependent on the
timely assessment and treatment of injuries
•
Trauma systems allow rapid delivery of patients to definitive care
•
Trauma systems underwent major development in US
– Illinois
– Orange County
– San Diego
•
ACSCOT Quality Indicators
– Validity, reliability, links with outcome?
Process measures and Patient Level
Outcomes
•
•
•
•
•
Many indicators not associated with mortality, length of stay or use of
ICU
Many indicators inversely associated with outcome
Poorly suited to patient level measurement?
More useful as hospital or system level tools?
Role for composite measures?
Steps to Composite Construction
(Adapted from Nardo et al., 2005)
Development
Evaluation
Use
Robustness and
Sensitivity
Visualisation
Links with other
variables
Back to the Real Data
Purpose:
Theoretical Framework
Managing Data:
Imputation
Combining QIs:
Data selection
Normalisation
Multivariate Analysis
Composite Generation:
Weighting and
Aggregation
Methods: Data Sources
•
Victorian State Trauma Registry (VSTORM)
– First state-wide trauma registry in
Australia
> 2001 to Present
•
Trauma Audit Research Network (UK)
– Trauma cases in England and Wales
> 1989 to Present
> Rated as 90-97% representative of
target population (National Centre for
Health Outcomes Development)
Composite Approaches (see Shwartz et al. 2008)
•
DBW Hospital (proposed by
CMS)
 n

 d qh 

 qh  ×  d qh  =

∑q  n   n  ∑q  n  = ∑q d qh
 ∑q qh 
 ∑q qh   qh 
Patients Meeting QI
•
Patients eligible for QI
DBW All Hospitals
(proposed by AHRQ)
(∑ n
h
•
qh
)
 d qh 
n
×
∑qh qh  n 
 qh 
Factor Analysis Composite
(proposed by OECD)
– Factor Analysis with
Principal Components
∑
q
nqh
Analysis
• Poisson regression:
– Count outcome: in-hospital mortality
– Covariates: Composite methods (3 models)
– Fixed co-efficient exposure variable: expected
mortality count (TRISS)
– Hospital and Hospital/Year analysis
– Clustered by hospital & structured for serial
correlation over time
– Included: blunt trauma, ISS>15, age>15 &
Hospitals>200 cases
Data
Parameter
N
Patients
9218
Hospitals
14
Years of Data
6
Hospital/Year Observations
84
Hospitals missing data
3
Total Hospital/Year Observations
78
Individual Indicators
Indicator
n
%
Eligible
Median
Hours
Team activated for major trauma patients
6492
85.13
-
Fixation of femoral diaphyseal fracture in adult trauma patients
394
79.44
-
Head CT received within 2 Hours
4313
66.32
1.47 (1.54)
GCS < 13 and head CT received within 2 Hours
1760
67.95
1.2 (1.21)
Sub/ Epidural Haematoma receiving craniotomy within 4 Hours
547
58.50
3.12 (9.49)
Cranial Surgery < 24 Hours
1535
87.86
3.84 (6.54)
Abdominal surgery < 24 Hours
770
87.80
3.36 (5.86)
Interval < 8 hrs between arrival and treatment of blunt, compound tibial
fracture
217
71.85
4.71 (6.25)
Laparotomy performed less than 2 hours after arrival at ED
259
37.27
2.88 (4.55)
Variation in Hospital Rank
Hospital
Rank
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
DBWHosp
DBWAll
Factor Comp
A
G
C
K
I
J
L
H
Hospital
N
E
F
M
D
B
Composites and in-hospital mortality
Composite
Index
IRR
P
95% CI
IQR
% Mortality
Decrease over IQR
DBW Hosp
0.9947
0.007
0.9909, 0.9985
22.69
11.99 (3.38, 20.73)
DBW All
0.9941
0.007
0.9899, 0.9984
23.09
13.58 (3.76, 23.36)
Factor
Composite
0.9943
0.020
0.9895, 0.9991
28.28
16.13 (2.57, 28.62)
Discussion
•
Composite popularity:
–
–
–
–
–
–
–
–
•
Jencks et al. 2000
Olson et al. 2001
Landon et al. 2006
Peterson et al. 2006
Bradley et al. 2006
Jha et al. 2007
Jegger et al. 2007
O’Brien et al. 2007
More predictable relationship between composite
measures and outcomes
–
–
Individual QIs and patient level outcomes: ‘Cause and effect’?
Composite measures: ‘Markers of overall quality’?
Considerations
•
Reporting composite performance
– Ranking Instability based on composite method
> e.g. in central band
– Reporting by performance bands?
> Sufficient variation to warrant shift in band
•
Applications
–
–
–
•
Develop measures fit for purpose:
–
•
Government monitoring?
System surveillance?
National/ International Comparisons?
composite approaches may not be appropriate for all stakeholders
Is one composite measure the most appropriate solution?
Limitations
•
•
Focus on Mortality: accurately recoded outcome on VSTR and TARN
Relationship between composites and morbidity:
– Functional Status
– Return to Work etc
•
Risk Adjustment using TRISS
– Commonly cited deficiencies:
> Age
> Physiological parameters
> Missing Data
> Predictive Power
•
Small sample size
– Impact on generalisability
– Factor Analysis: Factor Splintering
Conclusions
Composite indices may offer useful tools
for measuring aspects of hospital level
performance in trauma care
Acknowledgements
•
Thanks to:
Professor Johannes Stoelwinder
Dr. Fiona Lecky
Mrs. Maralyn Woodford
Mr. Tom Jenks
Dr. Omar Bouamra
Professor Peter Cameron
Associate Professor Damien Jolley
Professor Andrew Forbes
Professor Michael Shwartz
Dr. Nick Andrianopoulos
•
VSTR and TARN
•
Department of Epidemiology and Preventive Medicine, Monash University
•
National Health and Medical Research Council of Australia (NHMRC)