Evaluation of an Automatic Surveillance System of
Nosocomial Urinary Tract Infections
Shwu-Fen Chiou, RN, MSN; Yin-Yin Chen, RN, PhD; Fu-Der Wang, MD; Jen-Hsiang
Chuang, MD, MS, PhD
Introduction
Since nosocomial infections in clinical settings have been a common problem in
hospitals, an effective surveillance and control of the nosocomial infections is considered to
be a critical indicator of evaluating the caring quality of the hospital. (Burke, 2003） In the
Unite State, between the 5and 10 percent of patients admitted to acute care hospitals acquire
one or more infections. These adverse events affect approximately 2million patients each year,
result in some 90,000 deaths, and add an estimated $ 4.5 to 5.7 billion per year to the cost of
patients care. Thus, it has become an imminent challenge to hospitals how to rapidly and
effectively prevent and monitor the nosocomial infection in the clinical setting.
Currently, the collection of the surveillance data of the nosocomial infection in the
clinical setting is still conducted by manual chart review, which forms a burden to infection
control practitioners. Meanwhile, because the monitoring data can not be incorporated into the
information system of the hospital, the sudden infection in the clinical setting can not be
found immediately. This in turn reduces the effectiveness of prevention, delaying the timing
for dealing with these patients.
In recent years, with the rapid development of information technology, some studies have
already used information technology to deal with the clinical data relating to the nosocomial
infection, setting up the automatic surveillance system to detect the nosocomial infection. In
this way, they can detect the nosocomial infection patients in the clinical setting as early as
possible.
1
Therefore, this study aims to integrate the electronic medical records relating to the
nosocomial infection in order to establish an automatic surveillance system for the detection
of patients with nosocomial urinary tract infections, and to evaluate the performance of the
surveillance system.
Methods
A retrospective study was conducted in a 2,700-beds medical center in the Taipei area.
The research data for nosocomial urinary tract infections from the hospital information system,
which consisted of the demographic data of the patients, physicians’ prescriptions, the
laboratory data(albumin, white blood cells, hemoglobin, platelets, urine routine, and urine
microbiologic report), and medical records during hospitalization.
From September, 2004 to December, 2004, 1,609 patients with positive urine culture
were collected as training dataset. The training dataset was using to develop the rules for
nosocomial urinary tract infections according to the definitions of the Centers for Disease
Control and Prevention. In addition, another 624 patients with positive urine culture were
collected as testing dataset serving to validate the accuracy of these rules from January 2005
to February 2005.
In this study, Detection results from the automatic surveillance system were compared
with the infection control practitioners (ICP) used by traditional chart review method for
nosocomial urinary tract infections. And then, sensitivity, specificity, positive predictive value,
and negative predictive value were calculated to evaluate the performance of the automatic
surveillance system. Because ICP investigate the medical records was rather time-consuming,
only part of the samples of the training dataset and testing dataset were randomly picked up.
Finally, 480 samples were chosen out of the training dataset, while 240, out of testing dataset.
2
Results
Concerning all of the related data of detecting the nosocomial urinary tract infections, the
existing electronic medical record data were processed by Microsoft Access 2003 to establish
database. And the rule-base method was used to develop the judging rules of nosocomial
urinary tract infections. Meanwhile, SQL language was employed to write out the rules of the
nosocomial urinary tract infections. Until now, five rules of judging the infection were carved
out. And Asp.Net was used as a tool to design the Web with the interface of the surveillance
system shown as follows.
Of the 240 patients detected by the surveillance system, 66 ones belonged to the
nosocomial urinary tract infections, while 174 ones did not belong to the nosocomial urinary
tract infections. The sensitivity of the automatic surveillance system was 99 %, while
specificity was 47 %. The positive predictive value (PPV) was 41 %, and the negative
predictive value (NPV) was 99 %.
Discussion
According to the results of the research, the sensitivity of the automatic surveillance
system reached 99%, which meant that this system as a whole could fully detect the patients
with nosocomial urinary tract infections. However, the specificity was only 47%, which
meant that this system could detect 47% of the cases without nosocomial urinary tract
3
infections. Although this value was seemingly low, this system could save 47% of time the
infection control practitioners averagely spent surveying the patients’ medical records and
making judgments on when the patients were infected. This could reduce the regular
workload of the infection control practitioners.
Compared to the study of et al. Bouam（2003）, the specificity of this project was much
lower that that of Bouam since the sensitivity and specificity of his automatic surveillance
system were 91% and 91%, respectively. This was perhaps because his automatic surveillance
information system included complete surveillance data of the nosocomial infections.
However, the system developed by this study included only some basic data as well as related
laboratory reports, lacking some important surveillance data such as body temperatures,
patients’ clinical symptoms as the existing incomplete electronic medical records did affect
the effectiveness of this system. Judging from these, it could be seen that the completeness of
data played a critical part in determining the effectiveness of the automatic surveillance
system.
Conclusion
This study has integrated the current electronic medical records to set up an automatic
surveillance system of detecting the nosocomial urinary tract infections, validating the
accuracy of its function. It was found that due to the accuracy of the nosocomial urinary tract
infections, this system not only could reduce the workload of the infection control
practitioners but reach the goal of immediate and effective detection. This could also push the
medical and nursing staff to put this system into practice in order to boost the medical and
caring quality.
4
Limitation
Because of the limitation of time and human resources, this study could only survey the
medical records of part of the recruited patients to evaluate the effectiveness of this automatic
surveillance system. This may affect the results of this study. In the near future more and
more patients can be recruited into this automatic surveillance system for detection, which
will persistently correct the program and enhance the accuracy of this system.
Acknowledgements
We appreciated the grant offered by “Veterans General Hospital University System of
Taiwan Joint Research Program” as well as the sincere help offered by the information
management department of the hospital. Thanks a lot for they provided information and
support. In addition, we ought to pay our tribute to all the infection control practitioners
working for the infection control department for their suggestions about nosocomial urinary
tract infections.
Reference
1. Bates, D. W., & Gawande, A. A. (2003). Improving safety with information technology.
N.Engl.J.Med., %19;348(25), 2526-2534.
2. Bates, D. W., Evans, R. S., Murff, H., Stetson, P. D., Pizziferri, L., & Hripcsak, G. (2003).
Detecting adverse events using information technology. J.Am.Med.Inform.Assoc.,
10(2), 115-128.
3. Bouam, S., Girou, E., Brun-Buisson, C., Karadimas, H., & Lepage, E. (2003). An
intranet-based automated system for the surveillance of nosocomial infections:
prospective validation compared with physicians' self-reports. Infect.Control
Hosp.Epidemiol., 24(1), 51-55.
4. Burke, J. P. (2003). Infection control - a problem for patient safety. N.Engl.J.Med., 348(7),
651-656.
5. Evans, R. S., Larsen, R. A., Burke, J. P., Gardner, R. M., Meier, F. A., Jacobson, J. A., et al.
(1986). Computer surveillance of hospital-acquired infections and antibiotic use. The
Journal of the American Medical Association, 256(8), 1007-1011.
6. Fiszman, M., Chapman, W. W., Aronsky, D., Evans, R. S., & Haug, P. J. (2000).
Automatic detection of acute bacterial pneumonia from chest X-ray reports.
5
J.Am.Med.Inform.Assoc., 7(6), 593-604.
7. Haas, J. P., Mendonca, E. A., Ross, B., Friedman, C., & Larson, E. (2005). Use of
computerized surveillance to detect nosocomial pneumonia in neonatal intensive care
unit patients. American Journal of Infection Control.33(8):439-43, 33(8), 439-443.
8. Jarvis, W. R. (1996). Selected aspects of the socioeconomic impact of nosocomial
infections: morbidity, mortality, cost, and prevention. Infect Control Hosp Epidemiol,
17(8), 552-557.
9. Jarvis, W. R. (2001). Infection control and changing health-care delivery systems. Emerg
Infect Dis, 7(2), 170-173.
10. Lamma, E., Mello, P., Nanetti, A., Riguzzi, F., Storari, S., & Valastro, G. (2006).
Artificial intelligence techniques for monitoring dangerous infections. IEEE
Trans.Inf.Technol.Biomed., 10(1), 143-155.
11. Pittet, D., Safran, E., Harbarth, S., Borst, F., Copin, P., Rohner, P., et al. (1996). Automatic
alerts for methicillin-resistant Staphylococcus aureus surveillance and control: role of
a hospital information system. Infect Control Hosp Epidemiol, 17(8), 496-502.
12. Pokorny, L., Rovira, A., Martin-Baranera, M., Gimeno, C., onso-Tarres, C., & Vilarasau,
J. (2006). Automatic detection of patients with nosocomial infection by a
computer-based surveillance system: a validation study in a general hospital.
Infect.Control Hosp.Epidemiol., 27(5), 500-503.
13. Weinstein, R. A. (1998). Nosocomial infection update. Emerg Infect Dis, 4(3), 416-420.
14. Wilson, J. W., Oyen, L. J., Ou, N. N., McMahon, M. M., Thompson, R. L., Manahan, J.
M., et al. (2005). Hospital rules-based system: the next generation of medical
informatics for patient safety. Am.J.Health Syst.Pharm., 62(5), 499-505.
15. Wisniewski, M. F., Kieszkowski, P., Zagorski, B. M., Trick, W. E., Sommers, M., &
Weinstein, R. A. (2003). Development of a clinical data warehouse for hospital
infection control. J.Am.Med.Inform.Assoc., 10(5), 454-462.
16. Wright, M. O., Perencevich, E. N., Novak, C., Hebden, J. N., Standiford, H. C., & Harris,
A. D. (2004). Preliminary assessment of an automated surveillance system for
infection control. Infect.Control Hosp.Epidemiol., 25(4), 325-332.
6