A Novel Machine Learning Approach to Predict Textbook Outcome in Colectomy.

BACKGROUND: Several calculators exist to predict risk of postoperative complications. However, in low-risk procedures such as colectomy, a tool to determine the probability of achieving the ideal outcome could better aid clinical decision-making, especially for high-risk patients. A textbook outcome is a composite measure that serves as a surrogate for the ideal surgical outcome. OBJECTIVE: To identify the most important factors for predicting textbook outcomes in patients with nonmetastatic colon cancer undergoing colectomy and to create a textbook outcome decision support tool using machine learning algorithms. DESIGN: This was a retrospective analysis study. SETTINGS: Data were collected from the American College of Surgeons National Surgical Quality Improvement Program database. PATIENTS: Adult patients undergoing elective colectomy for nonmetastatic colon cancer (2014-2020) were included. MAIN OUTCOME MEASURES: Textbook outcome was the main outcome, defined as no mortality, no 30-day readmission, no postoperative complications, no 30-day reinterventions, and a hospital length of stay of ≤5 days. Four models (logistic regression, decision tree, random forest, and eXtreme Gradient Boosting) were trained and validated. Ultimately, a web-based calculator was developed as proof of concept for clinical application. RESULTS: A total of 20,498 patients who underwent colectomy for nonmetastatic colon cancer were included. Overall, textbook outcome was achieved in 66% of patients. Textbook outcome was more frequently achieved after robotic colectomy (77%), followed by laparoscopic colectomy (68%) and open colectomy (39%, p < 0.001). eXtreme Gradient Boosting was the best performing model (area under the curve = 0.72). The top 5 preoperative variables to predict textbook outcome were surgical approach, patient age, preoperative hematocrit, preoperative oral antibiotic bowel preparation, and patient sex. LIMITATIONS: This study was limited by its retrospective nature of the analysis. CONCLUSIONS: Using textbook outcome as the preferred outcome may be a useful tool in relatively low-risk procedures such as colectomy, and the proposed web-based calculator may aid surgeons in preoperative evaluation and counseling, especially for high-risk patients. See Video Abstract . UN NUEVO ENFOQUE DE APRENDIZAJE AUTOMTICO PARA PREDECIR EL RESULTADO DE LOS LIBROS DE TEXTO EN COLECTOMA: ANTECEDENTES:Existen varias calculadoras para predecir el riesgo de complicaciones posoperatorias. Sin embargo, en procedimientos de bajo riesgo como la colectomía, una herramienta para determinar la probabilidad de lograr el resultado ideal podría ayudar mejor a la toma de decisiones clínicas, especialmente para pacientes de alto riesgo. Un resultado de libro de texto es una medida compuesta que sirve como sustituto del resultado quirúrgico ideal.OBJETIVO:Identificar los factores más importantes para predecir el resultado de los libros de texto en pacientes con cáncer de colon no metastásico sometidos a colectomía y crear una herramienta de apoyo a la toma de decisiones sobre los resultados de los libros de texto utilizando algoritmos de aprendizaje automático.DISEÑO:Este fue un estudio de análisis retrospectivo.AJUSTES:Los datos se obtuvieron de la base de datos del Programa Nacional de Mejora de la Calidad del Colegio Americano de Cirujanos.PACIENTES:Se incluyeron pacientes adultos sometidos a colectomía electiva por cáncer de colon no metastásico (2014-2020).MEDIDAS PRINCIPALES DE RESULTADO:El resultado de los libros de texto fue el resultado principal, definido como ausencia de mortalidad, reingreso a los 30 días, complicaciones posoperatorias, reintervenciones a los 30 días y una estancia hospitalaria ≤5 días. Se entrenaron y validaron cuatro modelos (regresión logística, árbol de decisión, bosque aleatorio y XGBoost). Finalmente, se desarrolló una calculadora basada en la web como prueba de concepto para su aplicación clínica.RESULTADOS:Se incluyeron un total de 20.498 pacientes sometidos a colectomía por cáncer de colon no metastásico. En general, el resultado de los libros de texto se logró en el 66% de los pacientes. Los resultados de los libros de texto se lograron con mayor frecuencia después de la colectomía robótica (77%), seguida de la colectomía laparoscópica (68%) y la colectomía abierta (39%) (p<0,001). XGBoost fue el modelo con mejor rendimiento (AUC=0,72). Los cinco principales variables preoperatorias para predecir el resultado en los libros de texto fueron el abordaje quirúrgico, la edad del paciente, el hematocrito preoperatorio, la preparación intestinal con antibióticos orales preoperatorios y el sexo femenino.LIMITACIONES:Este estudio estuvo limitado por la naturaleza retrospectiva del análisis.CONCLUSIONES:El uso de los resultados de los libros de texto como resultado preferido puede ser una herramienta útil en procedimientos de riesgo relativamente bajo, como la colectomía, y la calculadora basada en la web propuesta puede ayudar a los cirujanos en la evaluación y el asesoramiento preoperatorios, especialmente para pacientes de alto riesgo. (Traducción-Yesenia Rojas-Khalil ).

Development and Validation of Prediction Models and Risk Calculators for Posthepatectomy Liver Failure and Postoperative Complications Using a Diverse International Cohort of Major Hepatectomies.

OBJECTIVE: The study aim was to develop and validate models to predict clinically significant posthepatectomy liver failure (PHLF) and serious complications [a Comprehensive Complication Index (CCI)>40] using preoperative and intraoperative variables. BACKGROUND: PHLF is a serious complication after major hepatectomy but does not comprehensively capture a patient's postoperative course. Adding the CCI as an additional metric can account for complications unrelated to liver function. METHODS: The cohort included adult patients who underwent major hepatectomies at 12 international centers (2010-2020). After splitting the data into training and validation sets (70:30), models for PHLF and a CCI>40 were fit using logistic regression with a lasso penalty on the training cohort. The models were then evaluated on the validation data set. RESULTS: Among 2192 patients, 185 (8.4%) had clinically significant PHLF and 160 (7.3%) had a CCI>40. The PHLF model had an area under the curve (AUC) of 0.80, calibration slope of 0.95, and calibration-in-the-large of -0.09, while the CCI model had an AUC of 0.76, calibration slope of 0.88, and calibration-in-the-large of 0.02. When the models were provided only preoperative variables to predict PHLF and a CCI>40, this resulted in similar AUCs of 0.78 and 0.71, respectively. Both models were used to build 2 risk calculators with the option to include or exclude intraoperative variables ( PHLF Risk Calculator; CCI>40 Risk Calculator ). CONCLUSIONS: Using an international cohort of major hepatectomy patients, we used preoperative and intraoperative variables to develop and internally validate multivariable models to predict clinically significant PHLF and a CCI>40 with good discrimination and calibration.

Overcoming barriers in the design and implementation of clinical trials for acute kidney injury: a report from the 2020 Kidney Disease Clinical Trialists meeting.

Acute kidney injury (AKI) is a growing epidemic and is independently associated with increased risk of death, chronic kidney disease (CKD) and cardiovascular events. Randomized-controlled trials (RCTs) in this domain are notoriously challenging and many clinical studies in AKI have yielded inconclusive findings. Underlying this conundrum is the inherent heterogeneity of AKI in its etiology, presentation and course. AKI is best understood as a syndrome and identification of AKI subphenotypes is needed to elucidate the disease's myriad etiologies and to tailor effective prevention and treatment strategies. Conventional RCTs are logistically cumbersome and often feature highly selected patient populations that limit external generalizability and thus alternative trial designs should be considered when appropriate. In this narrative review of recent developments in AKI trials based on the Kidney Disease Clinical Trialists (KDCT) 2020 meeting, we discuss barriers to and strategies for improved design and implementation of clinical trials for AKI patients, including predictive and prognostic enrichment techniques, the use of pragmatic trials and adaptive trials.

Selective prediction-set models with coverage rate guarantees.

The current approach to using machine learning (ML) algorithms in healthcare is to either require clinician oversight for every use case or use their predictions without any human oversight. We explore a middle ground that lets ML algorithms abstain from making a prediction to simultaneously improve their reliability and reduce the burden placed on human experts. To this end, we present a general penalized loss minimization framework for training selective prediction-set (SPS) models, which choose to either output a prediction set or abstain. The resulting models abstain when the outcome is difficult to predict accurately, such as on subjects who are too different from the training data, and achieve higher accuracy on those they do give predictions for. We then introduce a model-agnostic, statistical inference procedure for the coverage rate of an SPS model that ensembles individual models trained using K-fold cross-validation. We find that SPS ensembles attain prediction-set coverage rates closer to the nominal level and have narrower confidence intervals for its marginal coverage rate. We apply our method to train neural networks that abstain more for out-of-sample images on the MNIST digit prediction task and achieve higher predictive accuracy for ICU patients compared to existing approaches.

Hand-Assisted Versus Pure Minimally-Invasive Distal Pancreatectomy: Is There a Downside to Lending a Hand?

BACKGROUND: Hand-assisted laparoscopic distal pancreatectomy (HALDP) is suggested to offer similar outcomes to pure laparoscopic distal pancreatectomy (LDP). However, given the longer midline incision, it is unclear whether HALDP increases the risk of postoperative hernia. Our aim was to determine the risk of postoperative incisional hernia development after HALDP. METHODS: We retrospectively collected data from patients undergoing HALDP or LDP at a single center (2012-2020). Primary endpoints were postoperative incisional hernia and operative time. All patients had at minimum six months of follow-up. Outcomes were compared using unadjusted and multivariable regression analyses. RESULTS: Ninety-five patients who underwent laparoscopic distal pancreatectomy were retrospectively identified. Forty-one patients (43%) underwent HALDP. Patients with HALDP were older (median, 67 vs. 61 years, p = 0.02). Sex, race, Body Mass Index (median, 27 vs. 26), receipt of neoadjuvant chemotherapy, gland texture, wound infection rates, postoperative pancreatic fistula, overall complications, and hospital length-of-stay were similar between HALDP and LDP (all p > 0.05). In unadjusted analysis, operative times were shorter for HALDP (164 vs. 276 min, p < 0.001), but after adjustment, did not differ significantly (MR 0.73; 0.49-1.07, p = 0.1). Unadjusted incidence of hernia was higher in HALDP versus LDP (60% vs. 24%, p = 0.004). After adjustment, HALDP was associated with an increased odds of developing hernia (OR 7.52; 95% CI 1.54-36.8, p = 0.014). After propensity score matching, odds of hernia development remained higher for HALDP (OR 4.62; 95% CI 1.28-16.65, p = 0.031) p = 0.03). CONCLUSIONS: Compared with LDP, HALDP was associated with increased likelihood of postoperative hernia with insufficient evidence that HALDP shortens operative times. Our results suggest that HALDP may not be equivalent to LDP.

Approval policies for modifications to machine learning-based software as a medical device: A study of bio-creep.

Successful deployment of machine learning algorithms in healthcare requires careful assessments of their performance and safety. To date, the FDA approves locked algorithms prior to marketing and requires future updates to undergo separate premarket reviews. However, this negates a key feature of machine learning-the ability to learn from a growing dataset and improve over time. This paper frames the design of an approval policy, which we refer to as an automatic algorithmic change protocol (aACP), as an online hypothesis testing problem. As this process has obvious analogy with noninferiority testing of new drugs, we investigate how repeated testing and adoption of modifications might lead to gradual deterioration in prediction accuracy, also known as "biocreep" in the drug development literature. We consider simple policies that one might consider but do not necessarily offer any error-rate guarantees, as well as policies that do provide error-rate control. For the latter, we define two online error-rates appropriate for this context: bad approval count (BAC) and bad approval and benchmark ratios (BABR). We control these rates in the simple setting of a constant population and data source using policies aACP-BAC and aACP-BABR, which combine alpha-investing, group-sequential, and gate-keeping methods. In simulation studies, bio-creep regularly occurred when using policies with no error-rate guarantees, whereas aACP-BAC and aACP-BABR controlled the rate of bio-creep without substantially impacting our ability to approve beneficial modifications.

Rejoinder to Discussions on "Approval policies for modifications to machine learning-based software as a medical device: A study of bio-creep".

We thank the discussants for sharing their unique perspectives on the problem of designing automatic algorithm change protocols (aACPs) for machine learning-based software as a medical device. Both Pennello et al. and Rose highlighted a number of challenges that arise in real-world settings, and we whole-heartedly agree that substantial extensions of our work are needed to understand if and how aACPs can be safely deployed in practice. Our work demonstrated that aACPs that appear to be harmless may allow for biocreep, even when the data distribution is assumed to be representative and stationary over time. While we investigated two solutions that protect against this specific issue, many more statistical and practical challenges remain and we look forward to future research on this topic.

IntraCranial pressure prediction AlgoRithm using machinE learning (I-CARE): Training and Validation Study.

OBJECTIVES: Elevated intracranial pressure (ICP) is a potentially devastating complication of neurologic injury. Developing an ICP prediction algorithm to help the clinician adjust treatments and potentially prevent elevated ICP episodes. DESIGN: Retrospective study. SETTING: Three hundred thirty-five ICUs at 208 hospitals in the United States. SUBJECTS: Adults patients from the electronic ICU (eICU) Collaborative Research Database was used to train an ensemble machine learning model to predict the ICP 30 minutes in the future. Predictive performance was evaluated using a left-out test dataset and externally evaluated on the Medical Information Mart for Intensive Care-III (MIMIC-III) Matched Waveform Database. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Predictors included age, assigned sex, laboratories, medications and infusions, input/output, Glasgow Coma Scale (GCS) components, and time-series vitals (heart rate, ICP, mean arterial pressure, respiratory rate, and temperature). Each patient ICU stay was divided into successive 95-minute timeblocks. For each timeblock, the model was trained on nontime-varying covariates as well as on 12 observations of time-varying covariates at 5-minute intervals and asked to predict the 5-minute median ICP 30 minutes after the last observed ICP value. Data from 931 patients with ICP monitoring in the eICU dataset were extracted (46,207 timeblocks). The root mean squared error was 4.51 mm Hg in the eICU test set and 3.56 mm Hg in the MIMIC-III dataset. The most important variables driving ICP prediction were previous ICP history, patients' temperature, weight, serum creatinine, age, GCS, and hemodynamic parameters. CONCLUSIONS: IntraCranial pressure prediction AlgoRithm using machinE learning, an ensemble machine learning model, trained to predict the ICP of a patient 30 minutes in the future based on baseline characteristics and vitals data from the past hour showed promising predictive performance including in an external validation dataset.

Machine Learning-Based Prediction of Hospitalization During Chemoradiotherapy With Daily Step Counts.

Importance: Toxic effects of concurrent chemoradiotherapy (CRT) can cause treatment interruptions and hospitalizations, reducing treatment efficacy and increasing health care costs. Physical activity monitoring may enable early identification of patients at high risk for hospitalization who may benefit from proactive intervention. Objective: To develop and validate machine learning (ML) approaches based on daily step counts collected by wearable devices on prospective trials to predict hospitalizations during CRT. Design, Setting, and Participants: This study included patients with a variety of cancers enrolled from June 2015 to August 2018 on 3 prospective, single-institution trials of activity monitoring using wearable devices during CRT. Patients were followed up during and 1 month following CRT. Training and validation cohorts were generated temporally, stratifying for cancer diagnosis (70:30). Random forest, neural network, and elastic net-regularized logistic regression (EN) were trained to predict short-term hospitalization risk based on a combination of clinical characteristics and the preceding 2 weeks of activity data. To predict outcomes of activity data, models based only on activity-monitoring features and only on clinical features were trained and evaluated. Data analysis was completed from January 2022 to March 2023. Main Outcomes and Measures: Model performance was evaluated in terms of the receiver operating characteristic area under curve (ROC AUC) in the stratified temporal validation cohort. Results: Step counts from 214 patients (median [range] age, 61 [53-68] years; 113 [52.8%] male) were included. EN based on step counts and clinical features had high predictive ability (ROC AUC, 0.83; 95% CI, 0.66-0.92), outperforming random forest (ROC AUC, 0.76; 95% CI, 0.56-0.87; P = .02) and neural network (ROC AUC, 0.80; 95% CI, 0.71-0.88; P = .36). In an ablation study, the EN model based on only step counts demonstrated greater predictive ability than the EN model with step counts and clinical features (ROC AUC, 0.85; 95% CI, 0.70-0.93; P = .09). Both models outperformed the EN model trained on only clinical features (ROC AUC, 0.53; 95% CI, 0.31-0.66; P < .001). Conclusions and Relevance: This study developed and validated a ML model based on activity-monitoring data collected during prospective clinical trials. Patient-generated health data have the potential to advance predictive ability of ML approaches. The resulting model from this study will be evaluated in an upcoming multi-institutional, cooperative group randomized trial.

Machine learning improves prediction of postoperative outcomes after gastrointestinal surgery: a systematic review and meta-analysis.

BACKGROUND: Machine learning (ML) approaches have become increasingly popular in predicting surgical outcomes. However, it is unknown whether they are superior to traditional statistical methods such as logistic regression (LR). This study aimed to perform a systematic review and meta-analysis to compare the performance of ML vs LR models in predicting postoperative outcomes for patients undergoing gastrointestinal (GI) surgery. METHODS: A systematic search of Embase, MEDLINE, Cochrane, Web of Science, and Google Scholar was performed through December 2022. The primary outcome was the discriminatory performance of ML vs LR models as measured by the area under the receiver operating characteristic curve (AUC). A meta-analysis was then performed using a random effects model. RESULTS: A total of 62 LR models and 143 ML models were included across 38 studies. On average, the best-performing ML models had a significantly higher AUC than the LR models (ΔAUC, 0.07; 95% CI, 0.04-0.09; P < .001). Similarly, on average, the best-performing ML models had a significantly higher logit (AUC) than the LR models (Δlogit [AUC], 0.41; 95% CI, 0.23-0.58; P < .001). Approximately half of studies (44%) were found to have a low risk of bias. Upon a subset analysis of only low-risk studies, the difference in logit (AUC) remained significant (ML vs LR, Δlogit [AUC], 0.40; 95% CI, 0.14-0.66; P = .009). CONCLUSION: We found a significant improvement in discriminatory ability when using ML over LR algorithms in predicting postoperative outcomes for patients undergoing GI surgery. Subsequent efforts should establish standardized protocols for both developing and reporting studies using ML models and explore the practical implementation of these models.

Multi-institutional Development and External Validation of a Machine Learning Model for the Prediction of Distant Metastasis in Patients Treated by Salvage Radiotherapy for Biochemical Failure After Radical Prostatectomy.

BACKGROUND: Up to 40% of patients with prostate cancer may develop biochemical recurrence after surgery, with salvage radiation therapy (SRT) being the only curative option. In 2016, Tendulkar et al. (Contemporary update of a multi-institutional predictive nomogram for salvage radiotherapy after radical prostatectomy. J Clin Oncol 2016;34:3648-54) published a nomogram to predict distant metastasis in a cohort of patients treated with SRT with pre-SRT prostate-specific antigen (PSA) of 0.5 ng/ml after radical prostatectomy. In modern practice, SRT is delivered at lower PSA values. OBJECTIVE: To train and externally validate a machine learning model to predict the risk of distant metastasis at 5 yr in a contemporary cohort of patients receiving SRT. DESIGN, SETTING, AND PARTICIPANTS: We trained a machine learning model on data from 2418 patients treated with SRT at one institution, with a median PSA value of 0.27 ng/ml. External validation was done in 475 patients treated at two different institutions. Patients with cM1, pN1, or pT4 disease were excluded, as were patients with PSA >2 ng/ml or PSA 0, and patients with radiation dose <60 or ≥80 Gy. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Model performance was assessed using calibration and time-dependent area under the receiver operating curve (tAUC). RESULTS AND LIMITATIONS: Our model had better calibration and showed improved discrimination (tAUC = 0.72) compared with the Tendulkar model (tAUC = 0.60, p < 0.001). The main limitations of this study are its retrospective design and lack of validation on patients who received hormone therapy. CONCLUSIONS: The updated model can be used to provide more individualized risk assessments to patients treated with SRT at low PSA values, improving decision-making. PATIENT SUMMARY: Up to 40% of patients with prostate cancer may develop biochemical recurrence after surgery, with salvage radiation therapy as the only potentially curative option. We trained and validated a machine learning model using clinical and surgical data to predict a patient's risk of distant metastasis at 5 yr after treatment. Our model outperformed the reference tool and can improve clinical decision-making by providing more personalized risk assessment.

Development of a Machine Learning Model of Postoperative Acute Kidney Injury Using Non-Invasive Time-Sensitive Intraoperative Predictors.

Acute kidney injury (AKI) is a major postoperative complication that lacks established intraoperative predictors. Our objective was to develop a prediction model using preoperative and high-frequency intraoperative data for postoperative AKI. In this retrospective cohort study, we evaluated 77,428 operative cases at a single academic center between 2016 and 2022. A total of 11,212 cases with serum creatinine (sCr) data were included in the analysis. Then, 8519 cases were randomly assigned to the training set and the remainder to the validation set. Fourteen preoperative and twenty intraoperative variables were evaluated using elastic net followed by hierarchical group least absolute shrinkage and selection operator (LASSO) regression. The training set was 56% male and had a median [IQR] age of 62 (51-72) and a 6% AKI rate. Retained model variables were preoperative sCr values, the number of minutes meeting cutoffs for urine output, heart rate, perfusion index intraoperatively, and the total estimated blood loss. The area under the receiver operator characteristic curve was 0.81 (95% CI, 0.77-0.85). At a score threshold of 0.767, specificity was 77% and sensitivity was 74%. A web application that calculates the model score is available online. Our findings demonstrate the utility of intraoperative time series data for prediction problems, including a new potential use of the perfusion index. Further research is needed to evaluate the model in clinical settings.

Homelessness and Incidence and Causes of Sudden Death: Data From the POST SCD Study.

Importance: Over 580 000 people in the US experience homelessness, with one of the largest concentrations residing in San Francisco, California. Unhoused individuals have a life expectancy of approximately 50 years, yet how sudden death contributes to this early mortality is unknown. Objective: To compare incidence and causes of sudden death by autopsy among housed and unhoused individuals in San Francisco County. Design, Setting, and Participants: This cohort study used data from the Postmortem Systematic Investigation of Sudden Cardiac Death (POST SCD) study, a prospective cohort of consecutive out-of-hospital cardiac arrest deaths countywide among individuals aged 18 to 90 years. Cases meeting World Health Organization criteria for presumed SCD underwent autopsy, toxicologic analysis, and medical record review. For rate calculations, all 525 incident SCDs in the initial cohort were used (February 1, 2011, to March 1, 2014). For analysis of causes, 343 SCDs (incident cases approximately every third day) were added from the extended cohort (March 1, 2014, to December 16, 2018). Data analysis was performed from July 1, 2022, to July 1, 2023. Main Outcomes and Measures: The main outcomes were incidence and causes of presumed SCD by housing status. Causes of sudden death were adjudicated as arrhythmic (potentially rescuable with implantable cardioverter-defibrillator), cardiac nonarrhythmic (eg, tamponade), or noncardiac (eg, overdose). Results: A total of 868 presumed SCDs over 8 years were identified: 151 unhoused individuals (17.4%) and 717 housed individuals (82.6%). Unhoused individuals compared with housed individuals were younger (mean [SD] age, 56.7 [0.8] vs 61.0 [0.5] years, respectively) and more often male (132 [87.4%] vs 499 [69.6%]), with statistically significant racial differences. Paramedic response times were similar (mean [SD] time to arrival, unhoused individuals: 5.6 [0.4] minutes; housed individuals: 5.6 [0.2] minutes; P = .99), while proportion of witnessed sudden deaths was lower among unhoused individuals compared with housed individuals (27 [18.0%] vs 184 [25.7%], respectively, P = .04). Unhoused individuals had higher rates of sudden death (incidence rate ratio [IRR], 16.2; 95% CI, 5.1-51.2; P < .001) and arrhythmic death (IRR, 7.2; 95% CI, 1.3-40.1; P = .02). These associations remained statistically significant after adjustment for differences in age and sex. Noncardiac causes (96 [63.6%] vs 270 [37.7%], P < .001), including occult overdose (48 [31.8%] vs 90 [12.6%], P < .001), gastrointestinal causes (8 [5.3%] vs 15 [2.1%], P = .03), and infection (11 [7.3%] vs 20 [2.8%], P = .01), were more common among sudden deaths in unhoused individuals. A lower proportion of sudden deaths in unhoused individuals were due to arrhythmic causes (48 of 151 [31.8%] vs 420 of 717 [58.6%], P < .001), including acute and chronic coronary disease. Conclusions and Relevance: In this cohort study among individuals who experienced sudden death in San Francisco County, homelessness was associated with greater risk of sudden death from both noncardiac causes and arrhythmic causes potentially preventable with a defibrillator.

Hypertension Requiring Medication Use: a Silent Predictor of Poor Outcomes After Pancreaticoduodenectomy.

BACKGROUND: Although hypertension requiring medication (HTNm) is a well-known cardiovascular comorbidity, its association with postoperative outcomes is understudied. This study aimed to evaluate whether preoperative HTNm is independently associated with specific complications after pancreaticoduodenectomy. STUDY DESIGN: Adults undergoing elective pancreaticoduodenectomy were included from the 2014-2019 NSQIP-targeted pancreatectomy dataset. Multivariable regression models compared outcomes between patients with and without HTNm. Endpoints included significant complications, any complication, unplanned readmissions, length of stay (LOS), clinically relevant postoperative pancreatic fistula (CR-POPF), and cardiovascular and renal complications. A subgroup analysis excluded patients with diabetes, heart failure, chronic obstructive pulmonary disease, estimated glomerular filtration rate from serum creatinine (eGFRCr) < 60 ml/min per 1.73 m2, bleeding disorder, or steroid use. RESULTS: Among 14,806 patients, 52% had HTNm. HTNm was more common among older male patients with obesity, diabetes, congestive heart failure, chronic obstructive pulmonary disease, functional dependency, hard pancreatic glands, and cancer. After adjusting for demographics, preoperative comorbidities, and laboratory values, HTNm was independently associated with higher odds of significant complications (aOR 1.12, p = 0.020), any complication (aOR 1.11, p = 0.030), cardiovascular (aOR 1.78, p = 0.002) and renal (aOR 1.60, p = 0.020) complications, and unplanned readmissions (aOR 1.14, p = 0.040). In a subgroup analysis of patients without major preoperative comorbidity, HTNm remained associated with higher odds of significant complications (aOR 1.14, p = 0.030) and cardiovascular complications (aOR 1.76, p = 0.033). CONCLUSIONS: HTNm is independently associated with cardiovascular and renal complications after pancreaticoduodenectomy and may need to be considered in preoperative risk stratification. Future studies are necessary to explore associations among underlying hypertension, specific antihypertensive medications, and postoperative outcomes to investigate potential risk mitigation strategies.

Development and External Validation of a Machine Learning Model for Prediction of Lymph Node Metastasis in Patients with Prostate Cancer.

BACKGROUND: Pelvic lymph node dissection (PLND) is the gold standard for diagnosis of lymph node involvement (LNI) in patients with prostate cancer. The Roach formula, Memorial Sloan Kettering Cancer Center (MSKCC) calculator, and Briganti 2012 nomogram are elegant and simple traditional tools used to estimate the risk of LNI and select patients for PLND. OBJECTIVE: To determine whether machine learning (ML) can improve patient selection and outperform currently available tools for predicting LNI using similar readily available clinicopathologic variables. DESIGN, SETTING, AND PARTICIPANTS: Retrospective data for patients treated with surgery and PLND between 1990 and 2020 in two academic institutions were used. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: We trained three models (two logistic regression models and one gradient-boosted trees-based model [XGBoost]) on data provided from one institution (n = 20267) with age, prostate-specific antigen (PSA) levels, clinical T stage, percentage positive cores, and Gleason scores as inputs. We externally validated these models using data from another institution (n = 1322) and compared their performance to that of the traditional models using the area under the receiver operating characteristic curve (AUC), calibration, and decision curve analysis (DCA). RESULTS AND LIMITATIONS: LNI was present in 2563 patients (11.9%) overall, and in 119 patients (9%) in the validation data set. XGBoost had the best performance among all the models. On external validation, its AUC outperformed that of the Roach formula by 0.08 (95% confidence interval [CI] 0.042-0.12), the MSKCC nomogram by 0.05 (95% CI 0.016-0.070), and the Briganti nomogram by 0.03 (95% CI 0.0092-0.051; all p < 0.05). It also had better calibration and clinical utility in terms of net benefit on DCA across relevant clinical thresholds. The main limitation of the study is its retrospective design. CONCLUSIONS: Taking all measures of performance together, ML using standard clinicopathologic variables outperforms traditional tools in predicting LNI. PATIENT SUMMARY: Determining the risk of cancer spread to the lymph nodes in patients with prostate cancer allows surgeons to perform lymph node dissection only in patients who need it and avoid the side effects of the procedure in those who do not. In this study, we used machine learning to develop a new calculator to predict the risk of lymph node involvement that outperformed traditional tools currently used by oncologists.

Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees.

OBJECTIVE: After deploying a clinical prediction model, subsequently collected data can be used to fine-tune its predictions and adapt to temporal shifts. Because model updating carries risks of over-updating/fitting, we study online methods with performance guarantees. MATERIALS AND METHODS: We introduce 2 procedures for continual recalibration or revision of an underlying prediction model: Bayesian logistic regression (BLR) and a Markov variant that explicitly models distribution shifts (MarBLR). We perform empirical evaluation via simulations and a real-world study predicting Chronic Obstructive Pulmonary Disease (COPD) risk. We derive "Type I and II" regret bounds, which guarantee the procedures are noninferior to a static model and competitive with an oracle logistic reviser in terms of the average loss. RESULTS: Both procedures consistently outperformed the static model and other online logistic revision methods. In simulations, the average estimated calibration index (aECI) of the original model was 0.828 (95%CI, 0.818-0.938). Online recalibration using BLR and MarBLR improved the aECI towards the ideal value of zero, attaining 0.265 (95%CI, 0.230-0.300) and 0.241 (95%CI, 0.216-0.266), respectively. When performing more extensive logistic model revisions, BLR and MarBLR increased the average area under the receiver-operating characteristic curve (aAUC) from 0.767 (95%CI, 0.765-0.769) to 0.800 (95%CI, 0.798-0.802) and 0.799 (95%CI, 0.797-0.801), respectively, in stationary settings and protected against substantial model decay. In the COPD study, BLR and MarBLR dynamically combined the original model with a continually refitted gradient boosted tree to achieve aAUCs of 0.924 (95%CI, 0.913-0.935) and 0.925 (95%CI, 0.914-0.935), compared to the static model's aAUC of 0.904 (95%CI, 0.892-0.916). DISCUSSION: Despite its simplicity, BLR is highly competitive with MarBLR. MarBLR outperforms BLR when its prior better reflects the data. CONCLUSIONS: BLR and MarBLR can improve the transportability of clinical prediction models and maintain their performance over time.

Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare.

Machine learning (ML) and artificial intelligence (AI) algorithms have the potential to derive insights from clinical data and improve patient outcomes. However, these highly complex systems are sensitive to changes in the environment and liable to performance decay. Even after their successful integration into clinical practice, ML/AI algorithms should be continuously monitored and updated to ensure their long-term safety and effectiveness. To bring AI into maturity in clinical care, we advocate for the creation of hospital units responsible for quality assurance and improvement of these algorithms, which we refer to as "AI-QI" units. We discuss how tools that have long been used in hospital quality assurance and quality improvement can be adapted to monitor static ML algorithms. On the other hand, procedures for continual model updating are still nascent. We highlight key considerations when choosing between existing methods and opportunities for methodological innovation.

Prostate cancer therapy personalization via multi-modal deep learning on randomized phase III clinical trials.

Prostate cancer is the most frequent cancer in men and a leading cause of cancer death. Determining a patient's optimal therapy is a challenge, where oncologists must select a therapy with the highest likelihood of success and the lowest likelihood of toxicity. International standards for prognostication rely on non-specific and semi-quantitative tools, commonly leading to over- and under-treatment. Tissue-based molecular biomarkers have attempted to address this, but most have limited validation in prospective randomized trials and expensive processing costs, posing substantial barriers to widespread adoption. There remains a significant need for accurate and scalable tools to support therapy personalization. Here we demonstrate prostate cancer therapy personalization by predicting long-term, clinically relevant outcomes using a multimodal deep learning architecture and train models using clinical data and digital histopathology from prostate biopsies. We train and validate models using five phase III randomized trials conducted across hundreds of clinical centers. Histopathological data was available for 5654 of 7764 randomized patients (71%) with a median follow-up of 11.4 years. Compared to the most common risk-stratification tool-risk groups developed by the National Cancer Center Network (NCCN)-our models have superior discriminatory performance across all endpoints, ranging from 9.2% to 14.6% relative improvement in a held-out validation set. This artificial intelligence-based tool improves prognostication over standard tools and allows oncologists to computationally predict the likeliest outcomes of specific patients to determine optimal treatment. Outfitted with digital scanners and internet access, any clinic could offer such capabilities, enabling global access to therapy personalization.

ESTIMATION OF CELL LINEAGE TREES BY MAXIMUM-LIKELIHOOD PHYLOGENETICS.

CRISPR technology has enabled cell lineage tracing for complex multicellular organisms through insertion-deletion mutations of synthetic genomic barcodes during organismal development. To reconstruct the cell lineage tree from the mutated barcodes, current approaches apply general-purpose computational tools that are agnostic to the mutation process and are unable to take full advantage of the data's structure. We propose a statistical model for the CRISPR mutation process and develop a procedure to estimate the resulting tree topology, branch lengths, and mutation parameters by iteratively applying penalized maximum likelihood estimation. By assuming the barcode evolves according to a molecular clock, our method infers relative ordering across parallel lineages, whereas existing techniques only infer ordering for nodes along the same lineage. When analyzing transgenic zebrafish data from McKenna, Findlay and Gagnon et al. (2016), we find that our method recapitulates known aspects of zebrafish development and the results are consistent across samples.

Efficient nonparametric statistical inference on population feature importance using Shapley values.

The true population-level importance of a variable in a prediction task provides useful knowledge about the underlying data-generating mechanism and can help in deciding which measurements to collect in subsequent experiments. Valid statistical inference on this importance is a key component in understanding the population of interest. We present a computationally efficient procedure for estimating and obtaining valid statistical inference on the Shapley Population Variable Importance Measure (SPVIM). Although the computational complexity of the true SPVIM scales exponentially with the number of variables, we propose an estimator based on randomly sampling only Θ(n) feature subsets given n observations. We prove that our estimator converges at an asymptotically optimal rate. Moreover, by deriving the asymptotic distribution of our estimator, we construct valid confidence intervals and hypothesis tests. Our procedure has good finite-sample performance in simulations, and for an in-hospital mortality prediction task produces similar variable importance estimates when different machine learning algorithms are applied.