Jack Chen MBBS PhD Annual Health Service Research
Meeting, 26-28 June 2005 Boston

Background
•
Hospitals are unsafe places
•
Most patients who suffer adverse outcomes have documented deterioration
•
Medical Emergency Team system educates and empowers staff to call a skilled team in response to specific criteria or if “worried”
•
Team is called by group pager and responds immediately

• Criteria identifying seriously ill early
• Rapid response to those patients
(similar to a cardiac arrest team)
• Resuscitation and triage

MET Calling Criteria

Medical
Early
Response
Intervention AND
Therapy

•
CAT - Cardiac arrest team
•
NFR - Not for resuscitation (DNR, DNAR)
•
Events -
– Deaths without NFR
– Cardiac arrests without NFR
– Unplanned ICU admissions
– MET and CAT calls independent of above

• The primary aim of this study was to test the hypothesis that the implementation of the hospital-wide MET system will reduce the aggregate incidence of:
– Unplanned ICU admissions (mainly general wards)
– Cardiac Arrests (-NFR)
– Unexpected deaths (-NFR)

STUDY SAMPLE & SAMPLE
SIZE:
(at design stage)
• 23 hospitals with at least 20,000 estimated admissions per year
• This will provide us with a 90% chance to detect a 30% reduction in the incidence at the significant level of 5%
Kerry & Bland (1998)

• More complex to design
• More participants to obtain equivalent statistical power
• Key determinants are number of individual units; the intracluster correlation; and cluster size
• More complex analysis than ordinary randomised trial
• Randomised at one time, rather than one at a time

CONSORT STATEMENT: extension to cluster randomised trials
BMJ 2004;328:702

Assessed for eligibility (46 hospitals)
Excluded: 9 already had a MET system, 14 declined stating resource limitations
Two months baseline period (23 hospitals)
Randomized (23 hospitals)
Allocated to MET : (12 hospitals) median admission number at the baseline = 6494, range: 958 - 11026
Allocated to control : (11 hospitals) median admission number over the baseline =5856; range:
1937 –7845.
Four months implementation of MET with continued data collection
Six months study period with MET system operational
Four months period with continued data collection
Six months study period
Lost to follow up : none
Analyzed: 12 hospitals, median admission number over the study period = 18512; range: 2667 - 33115
Lost to follow up : none
Analyzed: 11 hospitals, median admission number over the study period = 17555; range: 5891 - 22338

• Stratified – blocked randomisation
(4) based on teaching hospital status
• Independent statistician

• 18178 EVENT forms
• 2418 corrections (13.3%)
• Final EVENTS - 13142 after third round data consistency and logic checking
• In-patients – 750,000

• Log books
• Scannable technology
• Photocopy forms kept by hospital
• Filing of forms and storage in Simpson
Centre
• Web-based tracking data
• 4 databases
• Separate neutral data repository

• 10 step standardised data entry and correction procedure
• Weekly data entry meeting between statistician, data manager,
IT manager and research assistants

Statistical methods used at cluster level and individual/multilevel (unadjusted and adjusted analyses)
Types of analyses
Cluster (Hospital) Level Individual / Multilevel
Unadjusted analysis
Weighted t-test
(weighted by hospital admission number)
Rao-Scott Chi-square
Adjusted t-test;
Adjusted analysis: Analytically weighted regression
(weighted by the admission number of the hospital) adjusting for teaching status, number of bed and baseline outcome
Multi-level logistic regression
(adjusting for teaching status, number of bed, age and gender of the patients)

• Weighting : by the number of admissions during the study period
• Cluster Adjustment for : teaching hospital status, bed size and baseline outcome variables, with hospitals weighted by the number of admissions during the study period
• Multilevel model adjustment for : teaching hospital status, bed size, age and gender of the patients

Non-MET
Hospitals
Number
Teaching
Non-teaching
Median bed size
11
8
3
315
(119-630)
MET
12
9
3
364
(88-641)

Outcomes (incidence rate/
1000 admissions )
Non-MET MET
Primary Outcome 6.775
Cardiac arrests (- NFR) 2.606
Unplanned ICU admissions 4.132
Unexpected deaths (- NFR) 1.605
No significant differences
6.291
1.597
4.267
1.648

RESULTS - DIFFERENCE BETWEEN
MET & NON-MET HOSPITALS
Incidence Rate/1000 admissions
OUTCOMES MET P
Primary outcome
NON-
MET
5.860
5.306
% AGE
CHANGE
10% 0.804
Cardiac arrest – NFR
Unplanned ICU admission
Unexpected deaths ( –
NFR)
1.640
4.683
1.175
1.31
4.185
1.063
25.1%
12%
10%
0.306
0.899
0.564

OUTCOME RATES/1000 ADMISSIONS OVER
BASELINE, IMPLEMENTATION AND STUDY PERIODS
Aggregate outcome
Control
Hospitals
Baseline Implementation Study
1 2.07 2.65 3.05
2
3
4
5
6
3.77
5.10
5.47
5.72
5.98
6.03
4.47
3.55
4.33
2.82
6.32
4.21
2.64
3.53
2.73
7
8
9
10
11
MET
Hospitals
12
13
6.83
7.86
9.04
9.42
13.29
0.58
6.28
5.92
10.57
7.82
18.10
0.89
5.07
4.72
8.83
7.63
13.92
1.31
14
15
16
17
18
19
20
21
22
23
1.60
1.85
2.95
3.87
4.26
6.39
6.39
7.29
7.44
13.04
19.83
3.36
5.80
3.40
3.60
4.19
7.27
4.48
4.98
6.18
6.89
14.43
4.61
3.42
3.22
2.86
4.66
7.08
4.44
5.90
7.07
5.59
12.75
* Excludes patients with prior NFR orders
Baseline
1.03
1.74
2.29
2.02
3.25
3.76
3.32
1.97
2.85
2.49
3.95
0.29
0.37
0.46
1.03
2.35
0.82
3.05
4.35
1.56
2.45
1.40
1.04
Cardiac arrests
Implementation
1.45
2.24
1.94
1.31
3.16
1.32
3.29
1.27
1.76
3.41
7.36
0.59
1.16
0.67
2.03
1.56
0.75
2.75
2.60
1.70
1.89
1.46
2.96
Study
2.03
1.93
1.75
1.15
1.47
1.54
1.69
1.11
0.85
1.88
2.65
1.11
0.78
0.45
1.04
1.24
1.49
2.34
1.62
2.05
1.78
1.07
0.75
Unplanned ICU admissions
Baseline
0.52
1.45
3.31
2.59
2.32
2.22
2.54
6.46
6.66
7.34
10.06
0.00
1.23
1.39
2.05
1.88
2.46
3.34
2.04
4.17
5.35
11.64
15.66
Implementation
0.24
3.71
2.74
1.85
1.58
1.50
3.29
4.93
8.30
4.97
11.93
0.00
2.39
4.46
1.57
2.16
3.60
4.90
2.02
3.01
4.48
5.43
11.10
Unexpected deaths
Baseline
0.52
1.45
1.15
2.74
1.70
2.22
3.51
1.40
0.95
1.66
0.36
0.58
0.37
0.46
0.64
0.82
1.48
2.03
2.18
2.86
1.27
1.86
5.22
Study
0.68
4.49
2.69
1.50
2.39
1.25
3.25
3.61
7.81
5.99
12.07
0.10
4.16
2.38
2.22
1.99
2.87
4.84
2.67
3.16
5.74
4.66
10.87
Implementation
2.41
1.78
1.14
1.31
0.75
1.15
2.39
0.71
1.26
1.00
1.79
0.74
0.52
0.67
1.24
0.66
0.92
1.30
1.44
1.05
1.18
1.04
4.07
Study
1.53
1.31
0.94
0.95
0.60
0.74
1.50
1.02
1.19
1.45
1.72
1.01
0.45
0.89
0.68
0.66
1.49
1.27
1.38
1.37
1.27
0.40
1.88

CALLING RATE/HOSPITAL/1,000
ADMISSIONS
CONTROL HOSPITALSMET HOSPITALS p
3.1 (1.5-5.8) 8.7 (3.5-16.5) <0.001

CALLS NOT ASSOCIATED WITH
AN EVENT/1,000 ADMISSIONS
CONTROL
HOSPITALS
1.2 (0-3.3)
194/528 (36.7%)
MET
HOSPITALS p
6.3 (2.5-11.2) <0.001
1329/1886 (70.5%) <0.001

CONTROL
HOSPITALS
MET
HOSPITALS p
236/246 (96%) 244/250 (97.6%) 0.359 Cardiac arrests
Unplanned
ICU admissions
Unexpected deaths
54/568 (9.5%) 209/611 (34.2%) 0.001
5/59 (17.2%) 4/48 (8.3%) 0.420

EVENTS WHICH HAD MET CRITERIA
BEFOREHAND (<15 min)
CONTROL
HOSPITALS
MET
HOSPITALS p
Cardiac arrests
Unexpected deaths
130/246 (53%) 115/250 (46%) 0.664
Unplanned ICU 121/568 (21%) 219/611 (36%) 0.090 admissions
10/29 (35%) 12/48 (25%) 0.473

EVENTS WHICH HAD MET CRITERIA
BEFOREHAND (>15 min)
CONTROL
HOSPITALS
MET
HOSPITALS p
Cardiac arrests
Unexpected deaths
109/246 (44%) 76/250 (30%) 0.031
Unplanned ICU 314/568 (55%) 313/611 (51%) 0.596 admissions
16/29 (55%) 24/58 (50%) 0.660

CALLS WHEN MET CRITERIA WERE
PRESENT (<15 min before event)
CONTROL
HOSPITALS
MET
HOSPITALS p
Cardiac arrests
Unexpected deaths
124/130 (95%) 112/115 (97%) 0.545
Unplanned ICU 28/121 (23%) 112/219 (51%) 0.049 admissions
4/16 (25%) 2/12 (17%) 0.298

CALLS WHEN MET CRITERIA WERE
PRESENT (>15 min before event)
CONTROL
HOSPITALS
MET
HOSPITALS p
Cardiac arrests
Unexpected deaths
104/109 (95%) 72/76 (95%) 0.874
Unplanned ICU 27/314 (9%) 95/313 (30%) 0.009 admissions
4/16 (25%) 2/24 (8%) 0.231

Prior NFR/1000 admissions
Non-MET MET
9.404
9.434
Prior NFR/Deaths 1.01
1.05
NFR made at time of event/
1000 admissions
NFR made at time of event/
1000 events
0.274
17.189
0.799
38.424

NFR ORDERS IN CALLS NOT
ASSOCIATED WITH AN EVENT
CONTROL
HOSPITALS
6/197 (3%)
MET
HOSPITALS
106/1332 (8%) p
0.048

DIFFERENCES BETWEEN BASELINE
AND STUDY PERIOD/1,000
ADMISSIONS (%)
Primary outcome
Cardiac arrests
Unplanned ICU admission
Unexpected deaths
-0.85 (13%)
-0.68 (33%)
-0.23 (5%)
-0.48 (30%) p
0.089
0.003
0.577
0.010

IN SUMMARY
• Randomisation was successful and appeared balanced
• Call rate was much higher in MET hospitals mostly due to calls not associated with events
• More of these event-free calls led to NFR orders in MET hospitals, but overall NFR rate was unaffected

IN SUMMARY
• There was no STATISTICALLY SIGNIFICANT decrease in the incidence of the primary outcome in MET hospitals
• There was no STATISTICALLY SIGNIFICANT decrease in the incidence of the secondary outcomes in MET hospitals
• WHEN ALL HOSPITALS CONSIDERED
TOGETHER , The incidence of cardiac arrests and unexpected deaths decreased from baseline to study period

IN SUMMARY

IN SUMMARY
There was an increase in calls before ICU admission in MET hospitals but not before cardiac arrests or unexpected deaths

IN SUMMARY

IN SUMMARY

IN SUMMARY

IN SUMMARY
• Estimated primary outcome incidence
3% - actual rate 0.57%
• Between hospital variability high
• Intra-class correlation co-efficient high

•The MET may be ineffective;
•The implementation is less optimal;
•The participating hospitals are unrepresentative;
•We studied wrong outcome;
•The documentation of the vital signs is poor;
•The calling rate is low given the existing calling criteria;
•The contamination;
•The low statistical power

• First large hospital system change trial ever conducted according to rigorous principles of design and statistical analysis
• It encompassed close to 750,000 admissions
• Although we did not demonstrate a significant difference in the primary outcome, the study produced a large body of useful data on patient care, documentation and outcomes, which will hopefully illuminate future studies

CONDUCTED BY:
Simpson Centre for Health Services
Research
ANZICS Clinical Trials Group
FUNDED BY:
NHMRC
Australian COUNCIL FOR Quality and Safety in Health Care (AQSHC)

MANAGEMENT COMMITTEE
Prof. Ken Hillman (Chair)
Prof. Rinaldo Bellomo
Mr. Daniel Brown
Dr. Jack Chen
Dr. Michelle Cretikos
Dr. Gordon Doig
Dr. Simon Finfer
Dr. Arthas Flabouris

PARTICIPATING HOSPITALS,
INVESTIGATORS & RESEARCH NURSES
•
Bendigo
–
John Edington, Kath Payne
• Box Hill – David Ernest, Angela Hamilton
• Broken Hill – Coral Bennet, Linda Peel,
Mathew Oliver, Russell Schedlich,
Sittampalam Ragavan, Linda Lynott
•
Calvery
–
Marielle Ruigrok, Margaret
Willshire,
• Canberra – Imogen Mitchell, John
Gowardman, David Elliot, Gillian Turner,
Carolyn Pain
•
Flinders
– Gerard O’Callaghan, Tamara Hunt
• Geelong – David Green, Jill Mann, Gary
Prisco
• Gosford – Sean Kelly, John Albury
•
John Hunter
– Ken Havill, Jane O’Brien
• Mackay – Kathryn Crane, Judy Struik
• Monash – Ramesh Nagappan, Laura Lister
•
Prince of Wales
–
Yahya Shahabi, Harriet
Adamsion
• Queen Elizabeth – Sandy Peake, Jonathan
Foote
•
Redcliffe
–
Neil Widdicombe, Matthys Campher,
Sharon Ragou, Raymond Johnson
• Redland – David Miller, Susan Carney
• Repatriation General – Gerard O’Callaghan,
Vicki Robb
•
Royal Adelaide
–
Marianne Chapman, Peter
Sharley, Deb Herewane, Sandy Jansen
• Royal North Shore Simon Finfer, Simeon Dale
• St. Vincent’s – John Santamaria, Jenny Holmes
•
Townsville
–
Michael Corkeron, Michelle
Barrett, Sue Walters
• Wangaratta – Chris Giles, Deb Hobijn
• Wollongong Sunny Rachakonda, Kathy
Rhodes
•
Wyong
–
Sean Kelly, John Albury