We investigated the feasibility of using ML models primarily based on biochemical variables for the prediction of BSI. The results suggest poor-to-moderate performance for the model to identify BSI cases. The model may still be useful to screen as a rule-out tool (vs. being a rule-in tool) with its particularly high NPV. Despite the overall low PPV, the model was found to have elevated precision (0.66) for detection of the most common infectious pathogens (Fig. 1c) than when we take all pathogens into consideration on the test set. As outlined in Fig. 2, platelets contributed the most to the predictions, and its impact was in proximity of leukocytes, neutrophils-to-lymphocytes ratio, monocytes, and CRP as other top contributing features. While the model’s predictive performance may not warrant its deployment as a standalone diagnostic tool for identifying BSI cases, it does underscore the significance of key biochemical parameters in BSI prediction.
Our results underline the diagnostic values of other variable previously found to associate with BSI or other pathological states7,15,16,17. Increased neutrophils-to-lymphocytes ratio drove the model predictions toward increased probability of BSI and vice versa, in line with studies that associate increased neutrophils-to-lymphocytes ratio with pathological states15. Elevated CRP reflected increased predicted probability of BSI in the LightGBM model (Fig. 2), aligned with previous research16,17. Our observations of leukocytes and CRP underlines their relative importance in prediction of BSI in concordance with the evidence presented in earlier research7. Similar observations have been reported for platelets18, albumin19, creatinine20, lactate dehydrogenase21, calcium ionized22 as risk factors associated with BSI.
Hospital-acquired infections are common in non-infectious patients, with up to 50% of intensive care admissions involving infection. Obtaining microbiological samples to confirm infection is difficult due to broad-spectrum antibiotic use in critically ill patients23,24. Previous research assessed CRP, PCT, and leukocytes as predictors of BSI in these patients. Low levels of PCT and CRP effectively excluded BSI, with PCT performing better7, in a study determining the diagnostic accuracy of the biomarkers for the diagnosis of BSI in critically ill patients admitted to the intensive care unit in the Capital Region, Denmark7. PCT had a higher predictive value on all days, compared with CRP and leukocyte with an AUC of 0.76 (95% CI 0.72–0.80) on day 1. Combining the three biomarkers yielded similar results as PCT alone using this biomarker at a threshold of 0.4 ng/mL7, and both a CRP cut-off (Cut-off = 155.62 mg/L) and a fixed low threshold (CRP = 20 mg/L) was used. Combining biomarkers did not improve predictive performance in our previous study7.
We found S. aureus, E. coli, and E. faecium to be the most common pathogens in our dataset, congruent with their commonality worldwide. E. coli is one of the most commonly detected pathogen in BSI worldwide, followed by S. aureus, S. epidermidis, and K. pneumoniae25. According to the European Antimicrobial Resistance Surveillance Network, E. coli was the most reported bacterial species (41.3%), followed by S. aureus (21.9%), K. pneumoniae (11.9%), E. faecalis (8.4%), P. aeruginosa (6.2%), E. faecium (5.5%), S. pneumoniae (2.6%), and Acinetobacter species (2.3%) for the period 2016 to 202026. Antimicrobial resistance becomes a critical concern when treating infections caused by S. aureus, E. coli, or E. faecium, as these pathogens often develop resistance to commonly used antibiotics, necessitating careful selection of treatment options27. The model performed reasonably well in identification of negative cases (NPV = 0.96 and specificity = 0.74 on the test set) with the presence of a diverse range of pathogens, that may imply that the model could find some complex patterns in biochemical variables. This capability when dealing with BSI could promote responsible antibiotic administration. It could also promote taking fewer blood cultures adding to reduced hospital costs.
The study used a large dataset spanning over a decade (2010–2020) with 144,398 instances from 54,188 patients. Despite this substantial data, the best performing model (LightGBM) achieved a PPV of only 0.12 on the test set, which is indeed low. The study acknowledges the difficulty in predicting BSI, even with extensive data. This is evident from the model’s performance metrics, including an AUC of 0.69. A better PPV might be achieved if we either differentiate between different hospital departments, as the prevalence and types of BSI can vary across different hospital units, or we perform risk stratification of patients, considering factors such as underlying conditions, invasive procedures, or prior antibiotic use. The low predictive performance is likely due to the complexity of BSI prediction, which involves numerous factors beyond biochemical data, such as patient history, clinical symptoms, and environmental factors. While PPV is higher for certain clinically relevant pathogens, standard techniques like SMOTE did not improve performance in our preliminary experiments. Consequently, we employed class-weighting and used MCC as a balanced metric to account for class imbalance. Furthermore, missing data is a significant limitation and potential bottleneck for machine learning in this context, as it can lead to biased or incomplete representations of patient states, potentially affecting model performance and generalizability. We used the KNNImputer from Scikit-Learn for continuous variables, but we did not extensively test alternative imputation strategies. This could be a potential avenue for future research.
The findings may imply new strategies to take to potentially improve BSI prediction for future studies. Machine learning models, such as this one, could assist in ruling out BSI cases, serving as a complementary decision-support tool. Yet, there is a need for additional validation. First is to investigate impacts of enriching data in hospitalized individuals by including antibiotic usage, vital parameters, and diagnostic codes. Second is to explore pathogens with analogous responses in biochemical variables for refining model sensitivity and reducing potential impacts of data heterogeneity through pathogen-specific models. Third, the patient group consists of immunocompromised patients (transplant and cancer patients), intensive care patients and finally patients admitted for other reasons. The study identified platelets, leukocytes, and neutrophil-to-lymphocyte ratios as the top three predictive features for bloodstream infections in the patient population. Yet, as our hospital handles hematological patients, these indicators may be less reliable in immunocompromised patients. Hematological malignancies often directly affect blood cell production and composition, which can skew these key predictive markers independently of any bloodstream infection. This inherent alteration in blood parameters could potentially reduce the accuracy and reliability of using these indicators to predict or diagnose bloodstream infections in such patients and should be a point for future research. The different patient groups are known to have different patterns of biochemistry values, and this may potentially confuse the ML algorithms. Hence, providing better labels for the different patient groups could prove a valuable strategy to address the data heterogeneity. We validated the model using an independent test set, and while incorporating external datasets for validation would significantly strengthen the study, it was unfortunately not feasible during the study period. It would be beneficial to include external datasets. While our study focused on predicting bloodstream infections using biochemical markers, Sakagianni et al.‘s work highlights the potential of combining machine learning with epidemiological data to address specific antibiotic resistance issues like carbapenem resistance. This synergy could be explored in future studies to enhance our predictive models for BSI28.
The current dataset did not include vital parameters, such as blood pressure and body temperature as well as symptoms. PCT is a recognized biomarker with high predictive value for bloodstream infections. However, PCT was not routinely available at our institution during the study period and is not a common variable in our dataset, this is why PCT could not be evaluated as a biomarker in this study. Again, use of an external dataset could be of value. In addition, there are other risk factors, such as catheter-related treatments and other invasive procedures29 that by including them the sensitivity of BSI prediction models could potentially be improved. Our hospital is the major referral hospital in Denmark with few patients admitted directly from primary care. An example is the low rate of S. pneumoniae BSI´s, which is explained by a low chance of seeing a patient with community acquired pneumonia in this hospital. In prospective academic investigations, it is conceivable that within specific cohorts, such as transplant recipients, an array of informative data sources, such as the microbiome13,30, could be utilized to facilitate a more exacting diagnostic approach for BSI. However, our dataset included immunocompromised patients such as cancer, HIV, organ transplantation, or certain medications with susceptibility to BSI and sepsis.
A weakness of our study approach is the use of controls as patients with negative blood cultures. These patients may indeed have had infections which did not progress to bacteremia/candidemia but still resulted in biochemical response similar to BSI patients; this would reduce the predictive power of the most infection-related biochemical tests. The strength of the study is the size of the patient cohort including the size of the training cohort, a factor which was not used in previous studies.
It is important to note that false-negative blood cultures may occur, particularly due to inadequate sampling, prior antibiotic therapy, or low bacterial load. For example, a recent study reported that sampling blood volumes of 20, 40, and 60 mL was associated with sensitivities of 65.0–75.7%, 80.4–89.2%, and 95.7–97.7%, respectively. Thus, we cannot rule out, that false negative blood cultures are among the negative cases, further skewing the results31. Incorporating other diagnostic tests, such as PCR-based assays or newer molecular diagnostic methods may identify more positive blood cultures. Ironically, new, fast molecular diagnostic methods for identifying bacterial infection may increase the risk of identifying dead bacterial or colonizing bacterial present in very low numbers – making reliable biomarkers and algorithm even more important in the need to recognize true infection from colonization. Therefore, despite challenges, ML models have the potential to enhance the speed and accuracy of diagnosis, which can lead to informed decision of treatment for patients with suspected bloodstream infections.
Our approach to predicting bloodstream infections using machine learning aligns with broader trends in applying ML to infectious disease management. Our focus is on BSI prediction using biochemical markers, and similar methodologies have been applied to predict antimicrobial resistance patterns. As highlighted in a recent literature review by Sakagianni A et al., machine learning techniques have shown promise in predicting antimicrobial resistance across various pathogens and antibiotics. Although our study does not directly address antimicrobial resistance, the methodological framework we employ shares commonalities with these approaches, particularly in the use of ensemble methods and the importance of feature selection in developing predictive models32.
In conclusion, our findings on this relatively large dataset supports the use of biochemical variables to have early estimates of the risk of BSI occurrences. The obvious yet low positive predictive power—as also found in more specific settings by others – suggests other more sensitive biochemical variables could be employed and/or developed. Results indicate clinical use to primarily focus on identifying patients at low risks of developing BSI in advance to receive confirmed blood culture results from microbiology laboratories. The application of ML has offered a systematic approach to identify potential biomarkers for bloodstream infections from diverse data characterized by varying pathogens and biochemical features. This methodology can be further enhanced in future investigations by integrating additional pertinent variables and by stratifying BSI cases based on pathogen groups that may elicit similar immune responses, thereby enabling more precise detection through blood tests. BSI detection is, perhaps, one of the better suited uses for AI in routine care offering near real-time readouts across most clinical settings and the ability to rapidly retrain models for continuous learning and automatic monitoring.
