Effect of Machine Learning on Anaesthesiology Clinician Prediction of Postoperative Complications: The Perioperative ORACLE Randomised Clinical Trial.

Background: Anaesthesiology clinicians can implement risk mitigation strategies if they know which patients are at greatest risk for postoperative complications. Although machine learning models predicting complications exist, their impact on clinician risk assessment is unknown. Methods: This single-centre randomised clinical trial enrolled patients age ≥18 undergoing surgery with anaesthesiology services. Anaesthesiology clinicians providing remote intraoperative telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) also reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury within 7 days. Area under the receiver operating characteristic curve (AUROC) for the clinician predictions was determined. Results: Among 5,071 patient cases reviewed by 89 clinicians, the observed incidence was 2% for postoperative death and 11% for acute kidney injury. Clinician predictions agreed with the models more strongly in the assisted versus unassisted group (weighted kappa 0.75 versus 0.62 for death [difference 0.13, 95%CI 0.10-0.17] and 0.79 versus 0.54 for kidney injury [difference 0.25, 95%CI 0.21-0.29]). Clinicians predicted death with AUROC of 0.793 in the assisted group and 0.780 in the unassisted group (difference 0.013, 95%CI -0.070 to 0.097). Clinicians predicted kidney injury with AUROC of 0.734 in the assisted group and 0.688 in the unassisted group (difference 0.046, 95%CI -0.003 to 0.091). Conclusions: Although there was evidence that the models influenced clinician predictions, clinician performance was not statistically significantly different with and without machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. Trial Registration: ClinicalTrials.gov NCT05042804.

Effect of Telemedicine Support for Intraoperative Anaesthesia Care on Postoperative Outcomes: The TECTONICS Randomised Clinical Trial.

Background: Novel applications of telemedicine can improve care quality and patient outcomes. Telemedicine for intraoperative decision support has not been rigorously studied. Methods: This single centre randomised clinical trial ( clinicaltrials.gov NCT03923699 ) of unselected adult surgical patients was conducted between July 1, 2019 and January 31, 2023. Patients received usual care or decision support from a telemedicine service, the Anesthesiology Control Tower (ACT). The ACT provided real-time recommendations to intraoperative anaesthesia clinicians based on case reviews, machine-learning forecasting, and physiologic alerts. ORs were randomised 1:1. Co-primary outcomes of 30-day all-cause mortality, respiratory failure, acute kidney injury (AKI), and delirium were analysed as intention-to-treat. Results: The trial completed planned enrolment with 71927 surgeries (35956 ACT; 35971 usual care). After multiple testing correction, there was no significant effect of the ACT vs. usual care on 30-day mortality [641/35956 (1.8%) vs 638/35971 (1.8%), risk difference 0.0% (95% CI -0.2% to 0.3%), p=0.96], respiratory failure [1089/34613 (3.1%) vs 1112/34619 (3.2%), risk difference -0.1% (95% CI -0.4% to 0.3%), p=0.96], AKI [2357/33897 (7%) vs 2391/33795 (7.1%), risk difference -0.1% (-0.6% to 0.4%), p=0.96], or delirium [1283/3928 (32.7%) vs 1279/3989 (32.1%), risk difference 0.6% (-2.0% to 3.2%), p=0.96]. There were no significant differences in secondary outcomes or in sensitivity analyses. Conclusions: In this large RCT of a novel application of telemedicine-based remote monitoring and decision support using real-time alerts and case reviews, we found no significant differences in postoperative outcomes. Large-scale intraoperative telemedicine is feasible, and we suggest future avenues where it may be impactful.

An ethnographic study on the impact of a novel telemedicine-based support system in the operating room.

OBJECTIVE: The Anesthesiology Control Tower (ACT) for operating rooms (ORs) remotely assesses the progress of surgeries and provides real-time perioperative risk alerts, communicating risk mitigation recommendations to bedside clinicians. We aim to identify and map ACT-OR nonroutine events (NREs)-risk-inducing or risk-mitigating workflow deviations-and ascertain ACT's impact on clinical workflow and patient safety. MATERIALS AND METHODS: We used ethnographic methods including shadowing ACT and OR clinicians during 83 surgeries, artifact collection, chart reviews for decision alerts sent to the OR, and 10 clinician interviews. We used hybrid thematic analysis informed by a human-factors systems-oriented approach to assess ACT's role and impact on safety, conducting content analysis to assess NREs. RESULTS: Across 83 cases, 469 risk alerts were triggered, and the ACT sent 280 care recommendations to the OR. 135 NREs were observed. Critical factors facilitating ACT's role in supporting patient safety included providing backup support and offering a fresh-eye perspective on OR decisions. Factors impeding ACT included message timing and ACT and OR clinician cognitive lapses. Suggestions for improvement included tailoring ACT message content (structure, timing, presentation) and incorporating predictive analytics for advanced planning. DISCUSSION: ACT served as a safety net with remote surveillance features and as a learning healthcare system with feedback/auditing features. Supporting strategies include adaptive coordination and harnessing clinician/patient support to improve ACT's sustainability. Study insights inform future intraoperative telemedicine design considerations to mitigate safety risks. CONCLUSION: Incorporating similar remote technology enhancement into routine perioperative care could markedly improve safety and quality for millions of surgical patients.

Protocol for the perioperative outcome risk assessment with computer learning enhancement (Periop ORACLE) randomized study.

Background: More than four million people die each year in the month following surgery, and many more experience complications such as acute kidney injury. Some of these outcomes may be prevented through early identification of at-risk patients and through intraoperative risk mitigation. Telemedicine has revolutionized the way at-risk patients are identified in critical care, but intraoperative telemedicine services are not widely used in anesthesiology. Clinicians in telemedicine settings may assist with risk stratification and brainstorm risk mitigation strategies while clinicians in the operating room are busy performing other patient care tasks. Machine learning tools may help clinicians in telemedicine settings leverage the abundant electronic health data available in the perioperative period. The primary hypothesis for this study is that anesthesiology clinicians can predict postoperative complications more accurately with machine learning assistance than without machine learning assistance. Methods: This investigation is a sub-study nested within the TECTONICS randomized clinical trial (NCT03923699). As part of TECTONICS, study team members who are anesthesiology clinicians working in a telemedicine setting are currently reviewing ongoing surgical cases and documenting how likely they feel the patient is to experience 30-day in-hospital death or acute kidney injury. For patients who are included in this sub-study, these case reviews will be randomized to be performed with access to a display showing machine learning predictions for the postoperative complications or without access to the display. The accuracy of the predictions will be compared across these two groups. Conclusion: Successful completion of this study will help define the role of machine learning not only for intraoperative telemedicine, but for other risk assessment tasks before, during, and after surgery. Registration: ORACLE is registered on ClinicalTrials.gov: NCT05042804; registered September 13, 2021.