IS AUTOMATIC DATA
COLLECTION FOR QUALITY
INDICATORS POSSIBLE?
17.3.2011
Matti Reinikainen
North Karelia Central Hospital
Joensuu, Finland
Pohjois-Karjalan sairaanhoito- ja sosiaalipalvelujen kuntayhtymä
www.pkssk.fi
CONFLICTS OF INTEREST STATEMENT:
MATTI REINIKAINEN, MD
- Position: Chief Physician, Dept of Intensive Care, North
Karelia Central Hospital, Joensuu, Finland
- Position of responsibility: 1st Secretary, Finnish Society of
Intensive Care
- Connection with the Finnish Intensive Care Consortium:
1. Using reports and analyses as department leader
2. Using database for research purposes, PhD thesis
under preparation
- Economic interests in this subject: none
Pohjois-Karjalan sairaanhoito- ja sosiaalipalvelujen kuntayhtymä
www.pkssk.fi
TERMS USED IN THIS PRESENTATION
• Clinical information system (CIS)
– a computer system that replaces bedside paper
documentation
– the system automatically collects data from patient
monitors and ventilators and the lab system and shows
the data both numerically and as graphic trends
– bedside screen(s)
• Data collection software
– a computer system that is linked to the CIS and
automatically transfers data to the centralised database of
the Finnish Intensive Care Consortium
– data accuracy is checked before submission
QUESTIONS TO BE ANSWERED
• If automatic data collection were possible,
would there be associated benefits?
• Is it possible?
• Are there drawbacks?
When you are taking care of a patient who is bleeding
…and who is haemodynamically unstable
Do you have time for
careful documentation of blood
pressures etc.?
Or would it be helpful if the data were
collected automatically?
Automatic data capture into a clinical
information system decreases the time
spent by nurses on documentation and
increases the time spent on patient care
• Wong DH et al. Changes in intensive care unit nurse task
activity after installation of a third-generation intensive
care unit information system. Crit Care Med 2003; 31:
2488-94.
– Before and after installation of clinical information
system
– The percentage of time ICU nurses spent on
documentation decreased by > 30% and the time spent
on patient care increased
Automatic data capture into a clinical
information system decreases the time
spent by nurses on documentation and
increases the time spent on patient care
• Bosman RJ et al. Intensive care information system reduces
documentation time of the nurses after cardiothoracic
surgery. Intensive Care Med 2003; 29: 83-90.
– Randomised controlled trial! – documentation on paper
vs. into an information system
– A 30% reduction in documentation time (p < 0.001) was
achieved, corresponding to 29 min per 8 h nursing shift
– This time was completely re-allocated to patient care
… or does it?
• Saarinen K, Aho M. Does the implementation of a clinical
information system decrease the time intensive care nurses
spend on documentation of care? Acta Anaesthesiol Scand
2005; 49: 62-5.
– After the implementation of a CIS, there was a small
(statistically non-significant) increase in the time spent
on documentation
– However, simultaneously there was a significant
increase in the time spent on patient care
– ”…any plans to reduce the ICU staff with the aid of
computers were not justified.”
Finland
Finland
North Karelia Central Hospital
Pirjo Kontio
Population in the district:
173 000 (+ 200-300 bears)
FINLAND
• Population 5,3 million
• Area 338 000 km2
- The Finnish Intensive Care Study, 1986-87
- 25 ICUs
- Niskanen M, Kari A, Halonen P. Five-year survival after
intensive care – comparison of 12 180 patients with the
general population. Crit Care Med 1996: 24: 1962-1967.
- The Severity Study
- Le Gall J-R et al. A new Simplified Acute Physiology
Score (SAPS II) Based on a European / North American
Multicenter Study. JAMA 1993: 270: 2957
- 13 152 patients (720 from 7 Finnish hospitals)
- Aarno Kari as country coordinator
THE FINNISH INTENSIVE
CARE CONSORTIUM
- A quality assurance project
started in 1994
- Strong growth since 1998
- university hospitals joined
in 2000-2002
1994
THE FINNISH INTENSIVE
CARE CONSORTIUM
1994
2007
• 20 hospital districts on Finnish mainland
• The main hospital in each district is called the
Central hospital
• 15 non-university hospitals, all adult ICUs
participate in the Consortium
• 5 university hospitals
– All ICUs from 3 of these participate
– In 2 university hospitals: in addition to participating
units, some specialised units not participating
• Apart from 1 ICU, all units use clinical
information systems and automatic data transfer
into a centralised database
Data collected by clinical information
systems, including laboratory test results
• … are automatically transferred by a data
collection software to the centralised database of
the Finnish Intensive Care Consortium
• The database is handled by Tieto Healthcare &
Welfare (previously by Intensium)
• The results in key performance indicators are
calculated and reported, many of them
automatically
QUALITY INDICATORS
•
•
•
•
DATA COMPLETENESS
ADEQUACY OF PATIENT SELECTION
OUTCOMES
RESOURCE CONSUMPTION
EXAMPLES OF REPORTS THAT ARE
UPDATED (SEMI)AUTOMATICALLY
In the following slides:
• Blue squares = North Karelia Central Hospital
• Red circles = the rest of the Finnish Intensive Care
Consortium (i.e. ”the average ICU”)
• Each square / circle is based on data from the
previous 6 months
DATA COMPLETENESS
The data completeness index: the second best performing unit (next to the one
with the least missing data) gets the index figure 100. The second worst
performer (next to the one with most missing data) gets the index figure 50.
Other units get index figures based on how close their performance is to these
two.
ADEQUACY OF PATIENT SELECTION
• Basic idea: the indication for intensive care is a
temporary danger to life and a possibility to
prevent death by intensive care
• ICU admissions may be ”inadequate” when
– there is no danger to life and no care of high intensity is
needed – could these patients be managed elsewhere?
– patients are moribund – care is futile
High risk and intensive care
The percentage of patients with a high risk of death (> 0,3) and a high intensity of
care (TISS score > 30/d)
Low risk – unnecessary ICU admission?
The percentage of patients who had a low risk of death (< 0,05), received care of
low intensity (maximal TISS score < 15/d) and were discharge alive
OUTCOMES
• Hospital mortality rate (crude & standardised
mortality ratios)
• Mortality after long ICU stays
• Post-icu mortality
• (Also measured, though with a considerable
amount of manual work: 6-month mortality &
health-related quality of life at 6 months)
VLAD curve, the cumulative amount of ”extra
lives saved”; curves for 3 ICUs
SMR
Standardised mortality ratios (O/E-ratio, the number of observed deaths divided by
the number of expected deaths, the expected number here being based on the
SAPS II model)
Hospital mortality after prolonged ICU care
Hospital mortality of patients who were treated in the ICU for > 6 days
Prolonged care – poor outcome
The percentage of patients who were treated in the ICU for > 6 days and who died
in the ICU
Post-ICU hospital mortality
A problem
here?
The percentage of patients who died in hospital after discharge from ICU
Readmissions within 48 hrs after discharge
Shortage
of beds?
The percentage of patients who were readmitted to the ICU within 48 hrs after ICU
discharge
RESOURCE CONSUMPTION
• In relation to care days produced
• (Also measured: ”Cost of lives saved” – the
amount of resources consumed per hospital
survivor. However, comparisons are difficult
because all ICU costs are not easily obtained in
Finnish hospitals.)
Care days / nurse / day
The number of patient days (24-h-periods) per each nurse / shift
IN PRACTICE, WE CAN SPEAK
ABOUT SEMI-AUTOMATED DATA
COLLECTION
• Many data are entered manually into the clinical
information system
– Admission data
– ICU and hospital discharge data (incl. outcome)
– TISS items are documented partly automatically, partly
manually
– Some physiological data need to be entered manually
Even automatically
collected data are
checked and validated,
… in North Karelia by
this team
Admission data
are checked for
accuracy and for
missing data
TISS items are
checked
• Lab test results are mostly transferred without
problems
Some derived parameters can be problematic
(mmHg)
The PaO2/FIO2-ratio is
correct only if both
values are documented
correctly
FIO2 is documented automatically
but monitoring may not be on e.g.
when inhalational drugs are given
• Values of some physiological parameters have
to be checked for possible technical artifacts
– e.g. blood pressure (trends of systolic BP in
the next examples)
Patient 1
Unfiltered raw data
Patient 1
Median filtering (10 min) eliminates most technical artifacts
Patient 2
Unfiltered raw data
Patient 2
Median filtering eliminates most technical artifacts –
but not all of them
Patient 2
Manual correction needed
HAS DATA COMPLETENESS
CHANGED IN ASSOCIATION
WITH THE AUTOMATION OF
DATA COLLECTION?
Missing values of SAPS II parameters in the database of the
Finnish Intensive Care Consortium:
Median
(quartiles)
Mean ± SD
1998
2003
2008
p
1 (1-2)
1 (0-2)
0 (0-0)
< 0.001
2.0 ± 2.3
1.2 ± 1.6
0.25 ± 0.85
< 0.001
Association of data completeness with
installation of data collection software
- The proportion of 100%
complete datasets was
determined (incl.
admission and discharge
data and data on
diagnoses)
100,00
Ratio of 100% complete records
Mussalo P, Tenhunen J.
Abstract presented at the
ISICEM Congress, 2007.
80,00
60,00
40,00
20,00
0,00
-9
- This proportion
increased after installation
of data collection software
-8
-7
-6
-5
-4
-3
-2
-1
0
1
2
3
4
Years before and after IVT installation. Installation Year =0
Installation of data collection software
How often is it possible to measure
the outcome?
Mussalo, P. 1,2, Reinikainen, M.3, Karlsson, S.4, Ruokonen, E.5
For the Finnish Intensive Care Quality Consortium
1) Intensium Ltd, Kuopio, for the Finnish Intensive Care Quality Consortium,
2) Department of Computer Science, University Of Kuopio, Kuopio,
3) Department of intensive Care, North Karelia Central Hospital, Joensuu, Finland,
4) Department of Intensive Care, Tampere University Hospital, Tampere, Finland,
5) Department of Intensive Care, Kuopio University Hospital, Kuopio, Finland
ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen
The mean time from discharge to registration of
hospital discharge data is 1 month
STATUS
ALL
SURVIVOR
DEAD
N
17146
14297
2849
Delay to HDD registration
From ICU discharge
From hospital discharge
Mean CI95%
Mean CI95%
38,5 (37,93-39,16)
31,2 (30,63-31,78)
40,8 (40,21-41,46)
32,6 (31,92-33,21)
27,1 (25,20-28,96)
24,3 (23,14-25,56)
ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen
Outcome reporting is quick – situation 1st September, 2008
Selected group of hospitals
50
40
30
Cumulative E-O
20
10
0
-10
-20
-30
ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen
Does this hospital have a summertime problem?
ARE THERE DRAWBACKS?
The sensitivity of sophisticated computerized methods in
detecting artifacts in blood pressure trends is inferior to that
of experienced human observers
IS THERE A POTENTIAL FOR
ERRORS GOING UNNOTICED?
Care days / nurse / day
The number of patient days (24-h-periods) per each nurse / shift
IS THERE A POTENTIAL FOR
ERRORS GOING UNNOTICED?
Care days / nurse / day
The number of patient days (24-h-periods) per each nurse / shift
This peak is
caused by an
error in the
data:
erroneous data
submission
after the fusion
of two units
• Morrison C et al. Electronic patient record use during
ward rounds: a qualitative study of interaction
between medical staff. Crit Care 2008; 12: R148
– The installation of an electronic patient record
system had a negative impact on multidisciplinary
communication during ward rounds
• ”Pen & paper” is a very flexible system
• A highly sophisticated computer system
may not always be that flexible
CONCLUSIONS
• Fully automatic data collection for quality indicators
is not possible
– Parts of data have to be entered manually
– (At least parts of) data collected automatically have to be
checked and validated in order to assure good data quality
• Semi-automatic data collection is possible, it saves
time used for documentation and improves data
completeness