Development and Validation of a Machine Learning Model to Estimate Bacterial Sepsis Among Immunocompromised Recipients of Stem Cell Transplant

Importance Sepsis disproportionately affects recipients of allogeneic hematopoietic cell transplant (allo-HCT), and timely detection is crucial. However, the atypical presentation of sepsis within this population makes detection challenging, and existing clinical sepsis tools have limited prognostic value among this high-risk population. Objective To develop a full risk factor (demographic, transplant, clinical, and laboratory factors) and clinical factor-specific automated bacterial sepsis decision support tool for recipients of allo-HCT with potential bloodstream infections (PBIs). Design, Setting, and Participants This prognostic study used data from adult recipients of allo-HCT transplanted at the Fred Hutchinson Cancer Research Center, Seattle, Washington, between June 2010 and June 2019 randomly divided into 70% modeling and 30% validation data sets. Tools were developed using the area under the curve (AUC) optimized SuperLearner, and their performance was compared with existing clinical sepsis tools: National Early Warning Score (NEWS), quick Sequential Organ Failure Assessment (qSOFA), and Systemic Inflammatory Response Syndrome (SIRS), using the validation data set. Data were analyzed between January and October of 2020. Main Outcomes and Measures The primary outcome was high-sepsis risk bacteremia (culture confirmed gram-negative species, Staphylococcus aureus, or Streptococcus spp bacteremia), and the secondary outcomes were 10- and 28-day mortality. Tool discrimination and calibration were examined using accuracy metrics and expected vs observed probabilities. Results Between June 2010 and June 2019, 1943 recipients of allo-HCT received their first transplant, and 1594 recipients (median [interquartile range] age at transplant, 54 [43-63] years; 911 [57.2%] men; 1242 individuals [77.9%] identifying as White) experienced at least 1 PBI. Of 8131 observed PBIs, 238 (2.9%) were high-sepsis risk bacteremia. Compared with high-sepsis risk bacteremia, the full decision support tool had the highest AUC (0.85; 95% CI, 0.81-0.89), followed by the clinical factor-specific tool (0.72; 95% CI, 0.66-0.78). SIRS had the highest AUC of existing tools (0.64; 95% CI, 0.57-0.71). The full decision support tool had the highest AUCs for PBIs identified in inpatient (0.82; 95% CI, 0.76-0.89) and outpatient (0.82; 95% CI, 0.75-0.89) settings and for 10-day (0.85; 95% CI, 0.79-0.91) and 28-day (0.80; 95% CI, 0.75-0.84) mortality. Conclusions and Relevance These findings suggest that compared with existing tools and the clinical factor-specific tool, the full decision support tool had superior prognostic accuracy for the primary (high-sepsis risk bacteremia) and secondary (short-term mortality) outcomes in inpatient and outpatient settings. If used at the time of culture collection, the full decision support tool may inform more timely sepsis detection among recipients of allo-HCT. This prognostic study uses machine learning and electronic medical record data to develop and test an automated bacterial sepsis decision support tool for immunocompromised recipients of allogeneic hematopoietic cell transplant. Question Can machine learning be used with electronic medical record data to improve bacterial sepsis prediction among recipients of allogeneic hematopoietic cell transplant (allo-HCT)? Findings In this prognostic study including 1943 recipients of allo-HCT, the population-specific full predictor bacterial sepsis decision support tool (SHBSL) had superior prognostic performance regardless of outcome or patient location compared with the clinical factor-specific SHBSL and existing tools. Additionally, SHBSL had higher positive predictive values relative to sensitivities than existing tools. Meaning These findings suggest that, if used at the time of blood culture collection, the SHBSL could provide relevant information regarding bacterial sepsis risk and antibiotic needs of recipients of allo-HCT.