JAC Advance Access originally published online on April 21, 2007
Journal of Antimicrobial Chemotherapy 2007 59(6):1204-1207; doi:10.1093/jac/dkm107
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Prediction of specific pathogens in patients with sepsis: evaluation of TREAT, a computerized decision support system
1 Department of Medicine E, Rabin Medical Center, Beilinson Campus, Petah-Tiqva 49100, Israel 2 Sackler Faculty of Medicine, Tel-Aviv University, Ramat-Aviv, Israel 3 Center for Model-based Medical Decision Support, Aalborg University, Aalborg, Denmark 4 Department of Infectious Diseases, Gemelli Hospital in Rome, Universitá Cattolica del Sacro Cuore School of Medicine, Rome, Italy 5 Department of Clinical Microbiology and Hospital Hygiene, Freiburg University Hospital, Freiburg University, Freiburg, Germany
* Corresponding author. Tel: +972-3-9376501; Fax: +972-3-9376512; E-mail: leibovic{at}post.tau.ac.il
Received 29 August 2006; returned 23 February 2007; revised 8 March 2007; accepted 21 March 2007
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
Background: Prediction of bacterial infections and their pathogens allows for early, directed investigation and treatment. We assessed the ability of TREAT, a computerized decision support system, to predict specific pathogens.
Methods: TREAT uses data available within the first few hours of infection presentation in a causal probabilistic network to predict sites of infection and specific pathogens. We included 3529 patients (920 with microbiologically documented infections) participating in the observational and interventional trials of the TREAT system in Israel, Germany and Italy. Discriminatory performance of TREAT to predict individual pathogens was expressed by the AUC with 95% confidence intervals. Calibration was assessed using the Hosmer–Lemeshow goodness-of-fit statistic.
Results: The AUCs for Gram-negative bacteria, including Pseudomonas aeruginosa, Acinetobacter baumannii, Klebsiella spp. and Escherichia coli, ranged between 0.70 and 0.80 (all significant). Adequate calibration was demonstrated for any Gram-negative infection and individual bacteria, except for E. coli. Discrimination and calibration were acceptable for Enterococcus spp. (AUC 0.71, 0.65–0.78), but not for Staphylococcus aureus (AUC 0.63, 0.55–0.71). The few infections caused by Candida spp. and Clostridium difficile were well predicted (AUCs 0.74, 0.54–0.95; and 0.94, 0.88–1.00, respectively). The coverage with TREAT's recommendation exceeded that observed with physicians' treatment for all pathogens, except Candida spp.
Conclusions: TREAT predicted individual pathogens causing infection well. Prediction of S. aureus was inferior to that observed with other pathogens. TREAT can be used to triage patients by the risk for specific pathogens. The system's predictions enable it to prescribe appropriate antibiotic treatment prior to pathogen identification.
Keywords: sepsis , pathogen , prediction , appropriate antibiotic treatment , computerized decision support
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
Optimal empirical antibiotic treatment of patients with sepsis should balance between broadness of antibiotic spectrum and costs: monetary costs, possible adverse effects and future resistance. We have shown in a randomized controlled trial that the use of a computerized decision support system for antibiotic treatment (TREAT) in medical departments, improved the percentage of appropriate empirical treatment while reducing overall costs.1 The system calculates probabilities for individual pathogens and their probability for antibiotic resistance. TREAT then balances between coverage and costs and advises whether and which antibiotic/s should be prescribed. TREAT is calibrated to local data.
In the present analysis, we assess the accuracy of TREAT's predictions as to the pathogen of infection. Accurate and early prediction of individual pathogens will not only improve empirical antibiotic treatment but can also define patient groups for treatment of specific pathogens (e.g. for a trial on an antipseudomonal drug), novel treatments for sepsis (e.g. activated protein C) or rapid molecular methods of pathogen identification.2 We present results pertaining to seven pathogens that are often linked to inappropriate empirical antibiotic treatment3 and mortality: Pseudomonas aeruginosa, Acinetobacter baumannii, Klebsiella spp., Enterococcus spp., Candida spp., Staphylococcus aureus and Clostridium difficile. We also show the system's predictions for Escherichia coli, to assess the system's performance with regard to common pathogens.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
The TREAT system and trial have been previously described.1,4–6 The logical platform of the TREAT system is a causal probabilistic network. In this model, variables are drawn and linked following the pathogenesis of infection and each variable is associated with a probability value or a conditional probability table. TREAT assigns probabilities to different sites of infection by using data on signs, symptoms, results of laboratory tests, imaging procedures and available microbiological data input to the system. Distribution of pathogens is incorporated to each site of infection according to local pathogen distribution in community- and hospital-acquired infections, accounting for the type of department (e.g. medical, surgical, intensive care unit etc.) and the length of stay in hospital prior to infection. Pre-defined risk factors increase the probability for specific pathogens (e.g. prior antibiotic treatment for Candida spp., intravenous drug abuse for S. aureus etc.). The system calculates the overall probability for infection caused by a specific pathogen. We assessed the system's accuracy with regard to pathogen prediction using this probability.
The system was tested in a cohort non-interventional study and in a cluster randomized interventional trial.1 Both studies were conducted in: Israel (in six wards of internal medicine, 240 beds) at the Rabin Medical Center, Beilinson Campus; Germany (two gastroenterology, two nephrology and two intensive care wards, 94 beds) at the University Hospital of Freiburg; and Italy (three infectious disease wards, 90 beds) at the Universitá Cattolica del Sacro Cuore School of Medicine, Gemelli Hospital in Rome. All hospitals are university-affiliated primary and tertiary care centres. For the purpose of this analysis, we used data derived from the two studies. Both studies were approved by the research ethics committees at the three institutions.
We included patients for whom antibiotics were prescribed, or should have been considered (for full inclusion and exclusion criteria please see reference 1). The data used by the system were available to physicians within the first few hours of infection presentation (admission in community-acquired and suspicion of infection in hospital-acquired infections). We collected all microbiological data available within 30 days of infection presentation. We used pre-defined rules to assign clinical significance to all isolated pathogens.1 A patient was assigned ‘yes’ for a specific pathogen when the pathogen was isolated from any sample and growth was considered clinically significant. Discriminatory performance of the system was assessed using a receiver-operating curve (ROC) analysis. The ROC plots true-positive versus false-positive predictions for a specific pathogen. We present 95% confidence intervals (CIs) for the area under the curve and assess whether the 95% CIs include 0.5. Calibration was assessed using the Hosmer–Lemeshow goodness-of-fit statistic.7 The Hosmer–Lemeshow test compares predicted versus observed probabilities for specific pathogens within strata. Lower 
2 values and higher P values (>0.05) for the Hosmer–Lemeshow test indicate acceptable model calibration with failure to reject the null hypothesis (no difference between observed and predicted values). Analyses were conducted using SPSS 13.0.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
During the study period, 3529 patients were included in the two studies, of which 920 had microbiologically documented infections. A description of patients, pathogens and sources of infection is given by Paul et al.1
The areas under the ROCs were significantly above 0.5 for all pathogens (Figure 1). Discrimination was better for Gram-negative bacteria, with areas under the ROC curve > 0.70 for all Gram-negative bacteria, including less common pathogens such as A. baumannii (Table 1). Calibration was adequate for all Gram-negative bacteria except for E. coli, where we found lower than observed predictions in the lower risk strata and higher than observed predictions in the high-risk strata. TREAT's antibiotic advice provided significantly higher coverage than physician's treatment for all Gram-negative bacteria except A. baumannii, where despite adequate predictive capability the system did not recommend covering treatment (Table 1). Among Gram-positive bacteria discrimination and calibration were adequate for infections caused by Enterococcus spp., but not for those caused by S. aureus. Nonetheless, recommended treatment was better than physicians' treatment for both pathogens. Overall, calibration for Gram-negative bacteria was superior to the system's calibration for Gram-positive bacteria [Figure S1, available as Supplementary data at JAC Online (http://jac.oxfordjournals.org/)].
|
|
Few cases of Candida spp. and C. difficile infections occurred. Adequate discrimination and calibration were found for both pathogens. Both TREAT's and physicians' empirical coverage for Candida infections was poor.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
TREAT predicted well specific pathogens causing infections among inpatients (area under the ROC > 0.7, significantly different from 0.5), with the exception of S. aureus. This is the first computerized decision support system shown to predict specific pathogens among inpatients. The clinical utility of TREAT's pathogen predictions has been proven in a randomized controlled trial, where the use of TREAT resulted in improved appropriate empirical antibiotic treatment.1
Prediction of Gram-negative infections carries important clinical consequences. Severe infections may be rapidly fatal if untreated.3 Increasingly, multidrug-resistant Gram-negative bacteria are encountered in the hospital. Early identification of these infections may permit adequate empirical antibiotic treatment and targeted novel treatments for sepsis.8
S. aureus was included in the TREAT model for all sites of infections. Specific risk factors included haemodialysis, presence of intravascular catheter, intravenous drug abuse and previous S. aureus bacteraemia. However, among recruited patients, the positive likelihood ratio for these risk factors was higher than those used in the TREAT model. In addition, we did not model chronic infections such as osteomyelitis, decubitus ulcers and abscesses, since the system was designed to target infections in which appropriate empirical antibiotic treatment improves survival. Bates et al.9 derived and validated several prediction rules for bacteraemic infections. In the rule for S. aureus bacteraemia, local symptoms received the highest odds ratio in addition to haemodialysis and ventilator use. Addition of chronic infections and adjusting the effect of previously modelled risk factors will improve TREAT's ability to diagnose S. aureus infections.
Excellent prediction of specific pathogens within the first hours of suspecting an infection should not be expected. The inability to predict specific pathogens is inherent to the common pathway leading to sepsis and the similarity of local manifestations with different pathogens. Thus, the AUCs shown do not conform to common standards of excellent discrimination using ROC analysis. However, a causal probabilistic model offers several advantages to classical multivariable models used for prediction. Multivariable models are derived from one database, can only use variables included in the final model, do not deal well with co-linearity, do not differentiate between cause and effect and do not deal well with missing data. The advantages of a causal probabilistic model include the possibility to use many variables that are adjusted for co-linearity within the model, while missing data are handled naturally since their probability values are present in the model. The system can predict in a single run a range of pathogens, including all those modelled within the causal probabilistic network.
Future research will be directed at calibrating the system using data from the present cohort. Rapid identification of bacteria using molecular techniques may soon enter clinical practice.2,10 The use of rapid diagnostic techniques in combination with systems such as TREAT, providing broad pathogen and antibiotic resistance predictions based on clinical data, should be pursued.
|
Transparency declarations |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
If TREAT were to be developed into a commercial system, all participating organizations might profit from it.
|
Supplementary data |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
Figure S1 is available as Supplementary data at JAC Online (http://jac.oxfordjournals.org/).
|
Acknowledgements |
|---|
The TREAT project was funded by the EU 5th framework, Information Society Technologies, contract no.: IST-9999-11 459. The funding source had no role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionTransparency declarationsSupplementary dataReferences |
|---|
1 Paul M, Andreassen S, Tacconelli E, et al. Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial. J Antimicrob Chemother (2006) 58:1238–45.
2
Cleven BE, Palka-Santini M, Gielen J, et al. Identification and characterization of bacterial pathogens causing bloodstream infections by DNA microarray. J Clin Microbiol (2006) 44:2389–97.
3 Leibovici L, Konisberger H, Pitlik SD, et al. Patients at risk for inappropriate antibiotic treatment of bacteraemia. J Intern Med (1992) 231:371–4.[Web of Science][Medline]
4 Andreassen S, Leibovici L, Paul M, et al. A probabilistic network for fusion of data and knowledge in clinical microbiology. In: Probabilistic Modeling in Bioinformatics and Medical Informatics—Husmeier D, Dybowski R, Roberts S, eds. (2004) London: Springer Verlag. 451–72.
5 Leibovici L, Fishman M, Schonheyder HC, et al. A causal probabilistic network for optimal treatment of bacterial infections. IEEE Trans Knowl Data Eng (2000) 12:517–28.[CrossRef]
6 Paul M, Andreassen S, Nielsen AD, et al. Prediction of bacteremia using TREAT, a computerized decision-support system. Clin Infect Dis (2006) 42:1274–82.[CrossRef][Web of Science][Medline]
7 Hosmer DW, Lemeshow S. A goodness-of-fit test for the multiple regression model. Commun Stat (1980) A10:1043–69.
8
Bernard GR, Vincent JL, Laterre PF, et al. Efficacy and safety of recombinant human activated protein C for severe sepsis. N Engl J Med (2001) 344:699–709.
9 Bates DW, Sands K, Miller E, et al. Predicting bacteremia in patients with sepsis syndrome. Academic Medical Center Consortium Sepsis Project Working Group. J Infect Dis (1997) 176:1538–51.[Web of Science][Medline]
10 Raoult D, Fournier PE, Drancourt M. What does the future hold for clinical microbiology? Nat Rev Microbiol (2004) 2:151–9.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  K. L. Lawrence and M. H. Kollef Antimicrobial Stewardship in the Intensive Care Unit: Advances and Obstacles Am. J. Respir. Crit. Care Med., March 15, 2009; 179(6): 434 - 438. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||