Analysis of Clinical Criteria for Discharge Among Patients Hospitalized for COVID-19: Development and Validation of a Risk Prediction Model.

BACKGROUND: Patients hospitalized with COVID-19 can clinically deteriorate after a period of initial stability, making optimal timing of discharge a clinical and operational challenge. OBJECTIVE: To determine risks for post-discharge readmission and death among patients hospitalized with COVID-19. DESIGN: Multicenter retrospective observational cohort study, 2020-2021, with 30-day follow-up. PARTICIPANTS: Adults admitted for care of COVID-19 respiratory disease between March 2, 2020, and February 11, 2021, to one of 180 US hospitals affiliated with the HCA Healthcare system. MAIN MEASURES: Readmission to or death at an HCA hospital within 30 days of discharge was assessed. The area under the receiver operating characteristic curve (AUC) was calculated using an internal validation set (33% of the HCA cohort), and external validation was performed using similar data from six academic centers associated with a hospital medicine research network (HOMERuN). KEY RESULTS: The final HCA cohort included 62,195 patients (mean age 61.9 years, 51.9% male), of whom 4704 (7.6%) were readmitted or died within 30 days of discharge. Independent risk factors for death or readmission included fever within 72 h of discharge; tachypnea, tachycardia, or lack of improvement in oxygen requirement in the last 24 h; lymphopenia or thrombocytopenia at the time of discharge; being ≤ 7 days since first positive test for SARS-CoV-2; HOSPITAL readmission risk score ≥ 5; and several comorbidities. Inpatient treatment with remdesivir or anticoagulation were associated with lower odds. The model's AUC for the internal validation set was 0.73 (95% CI 0.71-0.74) and 0.66 (95% CI 0.64 to 0.67) for the external validation set. CONCLUSIONS: This large retrospective study identified several factors associated with post-discharge readmission or death in models which performed with good discrimination. Patients 7 or fewer days since test positivity and who demonstrate potentially reversible risk factors may benefit from delaying discharge until those risk factors resolve.

Expert-augmented machine learning.

Machine learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption is limited by the level of trust afforded by given models. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may involve combining the complementary strengths of humans and machines. Here, we present expert-augmented machine learning (EAML), an automated method that guides the extraction of expert knowledge and its integration into machine-learned models. We used a large dataset of intensive-care patient data to derive 126 decision rules that predict hospital mortality. Using an online platform, we asked 15 clinicians to assess the relative risk of the subpopulation defined by each rule compared to the total sample. We compared the clinician-assessed risk to the empirical risk and found that, while clinicians agreed with the data in most cases, there were notable exceptions where they overestimated or underestimated the true risk. Studying the rules with greatest disagreement, we identified problems with the training data, including one miscoded variable and one hidden confounder. Filtering the rules based on the extent of disagreement between clinician-assessed risk and empirical risk, we improved performance on out-of-sample data and were able to train with less data. EAML provides a platform for automated creation of problem-specific priors, which help build robust and dependable machine-learning models in critical applications.

Predicting successful clinical candidates for fiducial-free lung tumor tracking with a deep learning binary classification model.

OBJECTIVES: The CyberKnife system is a robotic radiosurgery platform that allows the delivery of lung SBRT treatments using fiducial-free soft-tissue tracking. However, not all lung cancer patients are eligible for lung tumor tracking. Tumor size, density, and location impact the ability to successfully detect and track a lung lesion in 2D orthogonal X-ray images. The standard workflow to identify successful candidates for lung tumor tracking is called Lung Optimized Treatment (LOT) simulation, and involves multiple steps from CT acquisition to the execution of the simulation plan on CyberKnife. The aim of the study is to develop a deep learning classification model to predict which patients can be successfully treated with lung tumor tracking, thus circumventing the LOT simulation process. METHODS: Target tracking is achieved by matching orthogonal X-ray images with a library of digital radiographs reconstructed from the simulation CT scan (DRRs). We developed a deep learning model to create a binary classification of lung lesions as being trackable or untrackable based on tumor template DRR extracted from the CyberKnife system, and tested five different network architectures. The study included a total of 271 images (230 trackable, 41 untrackable) from 129 patients with one or multiple lung lesions. Eighty percent of the images were used for training, 10% for validation, and the remaining 10% for testing. RESULTS: For all five convolutional neural networks, the binary classification accuracy reached 100% after training, both in the validation and the test set, without any false classifications. CONCLUSIONS: A deep learning model can distinguish features of trackable and untrackable lesions in DRR images, and can predict successful candidates for fiducial-free lung tumor tracking.

Risks and Opportunities to Ensure Equity in the Application of Big Data Research in Public Health.

The big data revolution presents an exciting frontier to expand public health research, broadening the scope of research and increasing the precision of answers. Despite these advances, scientists must be vigilant against also advancing potential harms toward marginalized communities. In this review, we provide examples in which big data applications have (unintentionally) perpetuated discriminatory practices, while also highlighting opportunities for big data applications to advance equity in public health. Here, big data is framed in the context of the five Vs (volume, velocity, veracity, variety, and value), and we propose a sixth V, virtuosity, which incorporates equity and justice frameworks. Analytic approaches to improving equity are presented using social computational big data, fairness in machine learning algorithms, medical claims data, and data augmentation as illustrations. Throughout, we emphasize the biasing influence of data absenteeism and positionality and conclude with recommendations for incorporating an equity lens into big data research.

NSMCE2, a novel super-enhancer-regulated gene, is linked to poor prognosis and therapy resistance in breast cancer.

BACKGROUND: Despite today's advances in the treatment of cancer, breast cancer-related mortality remains high, in part due to the lack of effective targeted therapies against breast tumor types that do not respond to standard treatments. Therefore, identifying additional breast cancer molecular targets is urgently needed. Super-enhancers are large regions of open chromatin involved in the overactivation of oncogenes. Thus, inhibition of super-enhancers has become a focus in clinical trials for its therapeutic potential. Here, we aimed to identify novel super-enhancer dysregulated genes highly associated with breast cancer patients' poor prognosis and negative response to treatment. METHODS: Using existing datasets containing super-enhancer-associated genes identified in breast tumors and public databases comprising genomic and clinical information for breast cancer patients, we investigated whether highly expressed super-enhancer-associated genes correlate to breast cancer patients' poor prognosis and to patients' poor response to therapy. Our computational findings were experimentally confirmed in breast cancer cells by pharmacological SE disruption and gene silencing techniques. RESULTS: We bioinformatically identified two novel super-enhancer-associated genes - NSMCE2 and MAL2 - highly upregulated in breast tumors, for which high RNA levels significantly and specifically correlate with breast cancer patients' poor prognosis. Through in-vitro pharmacological super-enhancer disruption assays, we confirmed that super-enhancers upregulate NSMCE2 and MAL2 transcriptionally, and, through bioinformatics, we found that high levels of NSMCE2 strongly associate with patients' poor response to chemotherapy, especially for patients diagnosed with aggressive triple negative and HER2 positive tumor types. Finally, we showed that decreasing NSMCE2 gene expression increases breast cancer cells' sensitivity to chemotherapy treatment. CONCLUSIONS: Our results indicate that moderating the transcript levels of NSMCE2 could improve patients' response to standard chemotherapy consequently improving disease outcome. Our approach offers a new avenue to identify a signature of tumor specific genes that are not frequently mutated but dysregulated by super-enhancers. As a result, this strategy can lead to the discovery of potential and novel pharmacological targets for improving targeted therapy and the treatment of breast cancer.

Building more accurate decision trees with the additive tree.

The expansion of machine learning to high-stakes application domains such as medicine, finance, and criminal justice, where making informed decisions requires clear understanding of the model, has increased the interest in interpretable machine learning. The widely used Classification and Regression Trees (CART) have played a major role in health sciences, due to their simple and intuitive explanation of predictions. Ensemble methods like gradient boosting can improve the accuracy of decision trees, but at the expense of the interpretability of the generated model. Additive models, such as those produced by gradient boosting, and full interaction models, such as CART, have been investigated largely in isolation. We show that these models exist along a spectrum, revealing previously unseen connections between these approaches. This paper introduces a rigorous formalization for the additive tree, an empirically validated learning technique for creating a single decision tree, and shows that this method can produce models equivalent to CART or gradient boosted stumps at the extremes by varying a single parameter. Although the additive tree is designed primarily to provide both the model interpretability and predictive performance needed for high-stakes applications like medicine, it also can produce decision trees represented by hybrid models between CART and boosted stumps that can outperform either of these approaches.

Exploratory analysis using machine learning to predict for chest wall pain in patients with stage I non-small-cell lung cancer treated with stereotactic body radiation therapy.

BACKGROUND AND PURPOSE: Chest wall toxicity is observed after stereotactic body radiation therapy (SBRT) for peripherally located lung tumors. We utilize machine learning algorithms to identify toxicity predictors to develop dose-volume constraints. MATERIALS AND METHODS: Twenty-five patient, tumor, and dosimetric features were recorded for 197 consecutive patients with Stage I NSCLC treated with SBRT, 11 of whom (5.6%) developed CTCAEv4 grade ≥2 chest wall pain. Decision tree modeling was used to determine chest wall syndrome (CWS) thresholds for individual features. Significant features were determined using independent multivariate methods. These methods incorporate out-of-bag estimation using Random forests (RF) and bootstrapping (100 iterations) using decision trees. RESULTS: Univariate analysis identified rib dose to 1 cc < 4000 cGy (P = 0.01), chest wall dose to 30 cc < 1900 cGy (P = 0.035), rib Dmax < 5100 cGy (P = 0.05) and lung dose to 1000 cc < 70 cGy (P = 0.039) to be statistically significant thresholds for avoiding CWS. Subsequent multivariate analysis confirmed the importance of rib dose to 1 cc, chest wall dose to 30 cc, and rib Dmax. Using learning-curve experiments, the dataset proved to be self-consistent and provides a realistic model for CWS analysis. CONCLUSIONS: Using machine learning algorithms in this first of its kind study, we identify robust features and cutoffs predictive for the rare clinical event of CWS. Additional data in planned subsequent multicenter studies will help increase the accuracy of multivariate analysis.

IMRT QA using machine learning: A multi-institutional validation.

PURPOSE: To validate a machine learning approach to Virtual intensity-modulated radiation therapy (IMRT) quality assurance (QA) for accurately predicting gamma passing rates using different measurement approaches at different institutions. METHODS: A Virtual IMRT QA framework was previously developed using a machine learning algorithm based on 498 IMRT plans, in which QA measurements were performed using diode-array detectors and a 3%local/3 mm with 10% threshold at Institution 1. An independent set of 139 IMRT measurements from a different institution, Institution 2, with QA data based on portal dosimetry using the same gamma index, was used to test the mathematical framework. Only pixels with ≥10% of the maximum calibrated units (CU) or dose were included in the comparison. Plans were characterized by 90 different complexity metrics. A weighted poison regression with Lasso regularization was trained to predict passing rates using the complexity metrics as input. RESULTS: The methodology predicted passing rates within 3% accuracy for all composite plans measured using diode-array detectors at Institution 1, and within 3.5% for 120 of 139 plans using portal dosimetry measurements performed on a per-beam basis at Institution 2. The remaining measurements (19) had large areas of low CU, where portal dosimetry has a larger disagreement with the calculated dose and as such, the failure was expected. These beams need further modeling in the treatment planning system to correct the under-response in low-dose regions. Important features selected by Lasso to predict gamma passing rates were as follows: complete irradiated area outline (CIAO), jaw position, fraction of MLC leafs with gaps smaller than 20 or 5 mm, fraction of area receiving less than 50% of the total CU, fraction of the area receiving dose from penumbra, weighted average irregularity factor, and duty cycle. CONCLUSIONS: We have demonstrated that Virtual IMRT QA can predict passing rates using different measurement techniques and across multiple institutions. Prediction of QA passing rates can have profound implications on the current IMRT process.

Use of TrueBeam developer mode for imaging QA.

The purpose of this study was to automate regular Imaging QA procedures to become more efficient and accurate. Daily and monthly imaging QA for SRS and SBRT protocols were fully automated on a Varian linac. A three-step paradigm where the data are automatically acquired, processed, and analyzed was defined. XML scripts were written and used in developer mode in a TrueBeam linac to automatically acquire data. MATLAB R013B was used to develop an interface that could allow the data to be processed and analyzed. Hardware was developed that allowed the localization of several phantoms simultaneously on the couch. 14 KV CBCTs from the Emma phantom were obtained using a TrueBeam onboard imager as example of data acquisition and analysis. The images were acquired during two months. Artifacts were artificially introduced in the images during the reconstruction process using iTool reconstructor. Support vector machine algorithms to automatically identify each artifact were written using the Machine Learning MATLAB R2011 Toolbox. A daily imaging QA test could be performed by an experienced medical physicist in 14.3 ± 2.4 min. The same test, if automated using our paradigm, could be performed in 4.2 ± 0.7 min. In the same manner, a monthly imaging QA could be performed by a physicist in 70.7 ± 8.0 min and, if fully automated, in 21.8 ± 0.6 min. Additionally, quantitative data analysis could be automatically performed by Machine Learning Algorithms that could remove the subjectivity of data interpretation in the QA process. For instance, support vector machine algorithms could correctly identify beam hardening, rings and scatter artifacts. Traditional metrics, as well as metrics that describe texture, are needed for the classification. Modern linear accelerators are equipped with advanced 2D and 3D imaging capabilities that are used for patient alignment, substantially improving IGRT treatment accuracy. However, this extra complexity exponentially increases the number of QA tests needed. Using the new paradigm described above, not only the bare minimum ­ but also best practice ­ QA programs could be implemented with the same manpower.

Diagnostic Errors in Hospitalized Adults Who Died or Were Transferred to Intensive Care.

Importance: Diagnostic errors contribute to patient harm, though few data exist to describe their prevalence or underlying causes among medical inpatients. Objective: To determine the prevalence, underlying cause, and harms of diagnostic errors among hospitalized adults transferred to an intensive care unit (ICU) or who died. Design, Setting, and Participants: Retrospective cohort study conducted at 29 academic medical centers in the US in a random sample of adults hospitalized with general medical conditions and who were transferred to an ICU, died, or both from January 1 to December 31, 2019. Each record was reviewed by 2 trained clinicians to determine whether a diagnostic error occurred (ie, missed or delayed diagnosis), identify diagnostic process faults, and classify harms. Multivariable models estimated association between process faults and diagnostic error. Opportunity for diagnostic error reduction associated with each fault was estimated using the adjusted proportion attributable fraction (aPAF). Data analysis was performed from April through September 2023. Main Outcomes and Measures: Whether or not a diagnostic error took place, the frequency of underlying causes of errors, and harms associated with those errors. Results: Of 2428 patient records at 29 hospitals that underwent review (mean [SD] patient age, 63.9 [17.0] years; 1107 [45.6%] female and 1321 male individuals [54.4%]), 550 patients (23.0%; 95% CI, 20.9%-25.3%) had experienced a diagnostic error. Errors were judged to have contributed to temporary harm, permanent harm, or death in 436 patients (17.8%; 95% CI, 15.9%-19.8%); among the 1863 patients who died, diagnostic error was judged to have contributed to death in 121 (6.6%; 95% CI, 5.3%-8.2%). In multivariable models examining process faults associated with any diagnostic error, patient assessment problems (aPAF, 21.4%; 95% CI, 16.4%-26.4%) and problems with test ordering and interpretation (aPAF, 19.9%; 95% CI, 14.7%-25.1%) had the highest opportunity to reduce diagnostic errors; similar ranking was seen in multivariable models examining harmful diagnostic errors. Conclusions and Relevance: In this cohort study, diagnostic errors in hospitalized adults who died or were transferred to the ICU were common and associated with patient harm. Problems with choosing and interpreting tests and the processes involved with clinician assessment are high-priority areas for improvement efforts.

Multi-institutional Development and External Validation of a Machine Learning Model for the Prediction of Distant Metastasis in Patients Treated by Salvage Radiotherapy for Biochemical Failure After Radical Prostatectomy.

BACKGROUND: Up to 40% of patients with prostate cancer may develop biochemical recurrence after surgery, with salvage radiation therapy (SRT) being the only curative option. In 2016, Tendulkar et al. (Contemporary update of a multi-institutional predictive nomogram for salvage radiotherapy after radical prostatectomy. J Clin Oncol 2016;34:3648-54) published a nomogram to predict distant metastasis in a cohort of patients treated with SRT with pre-SRT prostate-specific antigen (PSA) of 0.5 ng/ml after radical prostatectomy. In modern practice, SRT is delivered at lower PSA values. OBJECTIVE: To train and externally validate a machine learning model to predict the risk of distant metastasis at 5 yr in a contemporary cohort of patients receiving SRT. DESIGN, SETTING, AND PARTICIPANTS: We trained a machine learning model on data from 2418 patients treated with SRT at one institution, with a median PSA value of 0.27 ng/ml. External validation was done in 475 patients treated at two different institutions. Patients with cM1, pN1, or pT4 disease were excluded, as were patients with PSA >2 ng/ml or PSA 0, and patients with radiation dose <60 or ≥80 Gy. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Model performance was assessed using calibration and time-dependent area under the receiver operating curve (tAUC). RESULTS AND LIMITATIONS: Our model had better calibration and showed improved discrimination (tAUC = 0.72) compared with the Tendulkar model (tAUC = 0.60, p < 0.001). The main limitations of this study are its retrospective design and lack of validation on patients who received hormone therapy. CONCLUSIONS: The updated model can be used to provide more individualized risk assessments to patients treated with SRT at low PSA values, improving decision-making. PATIENT SUMMARY: Up to 40% of patients with prostate cancer may develop biochemical recurrence after surgery, with salvage radiation therapy as the only potentially curative option. We trained and validated a machine learning model using clinical and surgical data to predict a patient's risk of distant metastasis at 5 yr after treatment. Our model outperformed the reference tool and can improve clinical decision-making by providing more personalized risk assessment.

Predicting the Effect of Proton Beam Therapy Technology on Pulmonary Toxicities for Patients With Locally Advanced Lung Cancer Enrolled in the Proton Collaborative Group Prospective Clinical Trial.

PURPOSE: This study aimed to predict the probability of grade ≥2 pneumonitis or dyspnea within 12 months of receiving conventionally fractionated or mildly hypofractionated proton beam therapy for locally advanced lung cancer using machine learning. METHODS AND MATERIALS: Demographic and treatment characteristics were analyzed for 965 consecutive patients treated for lung cancer with conventionally fractionated or mildly hypofractionated (2.2-3 Gy/fraction) proton beam therapy across 12 institutions. Three machine learning models (gradient boosting, additive tree, and logistic regression with lasso regularization) were implemented to predict Common Terminology Criteria for Adverse Events version 4 grade ≥2 pulmonary toxicities using double 10-fold cross-validation for parameter hyper-tuning without leak of information. Balanced accuracy and area under the curve were calculated, and 95% confidence intervals were obtained using bootstrap sampling. RESULTS: The median age of the patients was 70 years (range, 20-97), and they had predominantly stage IIIA or IIIB disease. They received a median dose of 60 Gy in 2 Gy/fraction, and 46.4% received concurrent chemotherapy. In total, 250 (25.9%) had grade ≥2 pulmonary toxicity. The probability of pulmonary toxicity was 0.08 for patients treated with pencil beam scanning and 0.34 for those treated with other techniques (P = 8.97e-13). Use of abdominal compression and breath hold were highly significant predictors of less toxicity (P = 2.88e-08). Higher total radiation delivered dose (P = .0182) and higher average dose to the ipsilateral lung (P = .0035) increased the likelihood of pulmonary toxicities. The gradient boosting model performed the best of the models tested, and when demographic and dosimetric features were combined, the area under the curve and balanced accuracy were 0.75 ± 0.02 and 0.67 ± 0.02, respectively. After analyzing performance versus the number of data points used for training, we observed that accuracy was limited by the number of observations. CONCLUSIONS: In the largest analysis of prospectively enrolled patients with lung cancer assessing pulmonary toxicities from proton therapy to date, advanced machine learning methods revealed that pencil beam scanning, abdominal compression, and lower normal lung doses can lead to significantly lower probability of developing grade ≥2 pneumonitis or dyspnea.

A unified path seeking algorithm for IMRT and IMPT beam orientation optimization.

Objective. Fully automated beam orientation optimization (BOO) for intensity-modulated radiotherapy and intensity modulated proton therapy (IMPT) is gaining interest, since achieving optimal plan quality for an unknown number of fixed beam arrangements is tedious. Fast group sparsity-based optimization methods have been proposed to find the optimal orientation, but manual tuning is required to eliminate the exact number of beams from a large candidate set. Here, we introduce a fast, automated gradient descent-based path-seeking algorithm (PathGD), which performs fluence map optimization for sequentially added beams, to visualize the dosimetric benefit of one added field at a time.Approach. Several configurations of 2-4 proton and 5-15 photon beams were selected for three head-and-neck patients using PathGD, which was compared to group sparsity-regularized BOO solved with the fast iterative shrinkage-thresholding algorithm (GS-FISTA), and manually selected IMPT beams or one coplanar photon VMAT arc (MAN). Once beams were chosen, all plans were compared on computational efficiency, dosimetry, and for proton plans, robustness.Main results. With each added proton beam, Clinical Target Volume (CTV) and organs at risk (OAR) dosimetric cost improved on average across plans by [1.1%, 13.6%], and for photons, [0.6%, 2.0%]. Comparing algorithms, beam selection for PathGD was faster than GS-FISTA on average by 35%, and PathGD matched the CTV coverage of GS-FISTA plans while reducing OAR mean and maximum dose in all structures by an average of 13.6%. PathGD was able to improve CTV [Dmax, D95%] by [2.6%, 5.2%] and reduced worst-case [max, mean] dose in OARs by [11.1%, 13.1%].Significance. The benefit of a path-seeking algorithm is the beam-by-beam analysis of dosimetric cost. PathGD was shown to be most efficient and dosimetrically desirable amongst group sparsity and manual BOO methods, and highlights the sensitivity of beam addition for IMPT in particular.

Development and multi-institutional validation of a convolutional neural network to detect vertebral body mis-alignments in 2D x-ray setup images.

BACKGROUND: Misalignment to the incorrect vertebral body remains a rare but serious patient safety risk in image-guided radiotherapy (IGRT). PURPOSE: Our group has proposed that an automated image-review algorithm be inserted into the IGRT process as an interlock to detect off-by-one vertebral body errors. This study presents the development and multi-institutional validation of a convolutional neural network (CNN)-based approach for such an algorithm using patient image data from a planar stereoscopic x-ray IGRT system. METHODS: X-rays and digitally reconstructed radiographs (DRRs) were collected from 429 spine radiotherapy patients (1592 treatment fractions) treated at six institutions using a stereoscopic x-ray image guidance system. Clinically-applied, physician approved, alignments were used for true-negative, "no-error" cases. "Off-by-one vertebral body" errors were simulated by translating DRRs along the spinal column using a semi-automated method. A leave-one-institution-out approach was used to estimate model accuracy on data from unseen institutions as follows: All of the images from five of the institutions were used to train a CNN model from scratch using a fixed network architecture and hyper-parameters. The size of this training set ranged from 5700 to 9372 images, depending on exactly which five institutions were contributing data. The training set was randomized and split using a 75/25 split into the final training/ validation sets. X-ray/ DRR image pairs and the associated binary labels of "no-error" or "shift" were used as the model input. Model accuracy was evaluated using images from the sixth institution, which were left out of the training phase entirely. This test set ranged from 180 to 3852 images, again depending on which institution had been left out of the training phase. The trained model was used to classify the images from the test set as either "no-error" or "shifted", and the model predictions were compared to the ground truth labels to assess the model accuracy. This process was repeated until each institution's images had been used as the testing dataset. RESULTS: When the six models were used to classify unseen image pairs from the institution left out during training, the resulting receiver operating characteristic area under the curve values ranged from 0.976 to 0.998. With the specificity fixed at 99%, the corresponding sensitivities ranged from 61.9% to 99.2% (mean: 77.6%). With the specificity fixed at 95%, sensitivities ranged from 85.5% to 99.8% (mean: 92.9%). CONCLUSION: This study demonstrated the CNN-based vertebral body misalignment model is robust when applied to previously unseen test data from an outside institution, indicating that this proposed additional safeguard against misalignment is feasible.

Development and External Validation of a Machine Learning Model for Prediction of Lymph Node Metastasis in Patients with Prostate Cancer.

BACKGROUND: Pelvic lymph node dissection (PLND) is the gold standard for diagnosis of lymph node involvement (LNI) in patients with prostate cancer. The Roach formula, Memorial Sloan Kettering Cancer Center (MSKCC) calculator, and Briganti 2012 nomogram are elegant and simple traditional tools used to estimate the risk of LNI and select patients for PLND. OBJECTIVE: To determine whether machine learning (ML) can improve patient selection and outperform currently available tools for predicting LNI using similar readily available clinicopathologic variables. DESIGN, SETTING, AND PARTICIPANTS: Retrospective data for patients treated with surgery and PLND between 1990 and 2020 in two academic institutions were used. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: We trained three models (two logistic regression models and one gradient-boosted trees-based model [XGBoost]) on data provided from one institution (n = 20267) with age, prostate-specific antigen (PSA) levels, clinical T stage, percentage positive cores, and Gleason scores as inputs. We externally validated these models using data from another institution (n = 1322) and compared their performance to that of the traditional models using the area under the receiver operating characteristic curve (AUC), calibration, and decision curve analysis (DCA). RESULTS AND LIMITATIONS: LNI was present in 2563 patients (11.9%) overall, and in 119 patients (9%) in the validation data set. XGBoost had the best performance among all the models. On external validation, its AUC outperformed that of the Roach formula by 0.08 (95% confidence interval [CI] 0.042-0.12), the MSKCC nomogram by 0.05 (95% CI 0.016-0.070), and the Briganti nomogram by 0.03 (95% CI 0.0092-0.051; all p < 0.05). It also had better calibration and clinical utility in terms of net benefit on DCA across relevant clinical thresholds. The main limitation of the study is its retrospective design. CONCLUSIONS: Taking all measures of performance together, ML using standard clinicopathologic variables outperforms traditional tools in predicting LNI. PATIENT SUMMARY: Determining the risk of cancer spread to the lymph nodes in patients with prostate cancer allows surgeons to perform lymph node dissection only in patients who need it and avoid the side effects of the procedure in those who do not. In this study, we used machine learning to develop a new calculator to predict the risk of lymph node involvement that outperformed traditional tools currently used by oncologists.

The Conditional Super Learner.

Using cross validation to select the best model from a library is standard practice in machine learning. Similarly, meta learning is a widely used technique where models previously developed are combined (mainly linearly) with the expectation of improving performance with respect to individual models. In this article we consider the Conditional Super Learner (CSL), an algorithm that selects the best model candidate from a library of models conditional on the covariates. The CSL expands the idea of using cross validation to select the best model and merges it with meta learning. We propose an optimization algorithm that finds a local minimum to the problem posed and proves that it converges at a rate faster than Op(n-1/4). We offer empirical evidence that: (1) CSL is an excellent candidate to substitute stacking and (2) CLS is suitable for the analysis of Hierarchical problems. Additionally, implications for global interpretability are emphasized.

Representational Gradient Boosting: Backpropagation in the Space of Functions.

The estimation of nested functions (i.e., functions of functions) is one of the central reasons for the success and popularity of machine learning. Today, artificial neural networks are the predominant class of algorithms in this area, known as representational learning. Here, we introduce Representational Gradient Boosting (RGB), a nonparametric algorithm that estimates functions with multi-layer architectures obtained using backpropagation in the space of functions. RGB does not need to assume a functional form in the nodes or output (e.g., linear models or rectified linear units), but rather estimates these transformations. RGB can be seen as an optimized stacking procedure where a meta algorithm learns how to combine different classes of functions (e.g., Neural Networks (NN) and Gradient Boosting (GB)), while building and optimizing them jointly in an attempt to compensate each other's weaknesses. This highlights a stark difference with current approaches to meta-learning that combine models only after they have been built independently. We showed that providing optimized stacking is one of the main advantages of RGB over current approaches. Additionally, due to the nested nature of RGB we also showed how it improves over GB in problems that have several high-order interactions. Finally, we investigate both theoretically and in practice the problem of recovering nested functions and the value of prior knowledge.

Prospective Clinical Validation of Virtual Patient-Specific Quality Assurance of Volumetric Modulated Arc Therapy Radiation Therapy Plans.

PURPOSE: Performing measurement-based patient-specific quality assurance (PSQA) is recognized as a resource-intensive and time inefficient task in the radiation therapy treatment workflow. Paired with technological refinements in modern radiation therapy, research toward measurement-free PSQA has seen increased interest during the past 5 years. However, these efforts have not been clinically implemented or prospectively validated in the United States. We propose a virtual QA (VQA) system and workflow to assess the safety and workload reduction of measurement-free PSQA. METHODS: An XGBoost machine learning model was designed to predict PSQA outcomes of volumetric modulated arc therapy plans, represented as percent differences between the measured ion chamber point dose in a phantom and the corresponding planned dose. The final model was deployed within a web application to predict PSQA outcomes of clinical plans within an existing clinical workflow. The application also displays relevant feature importance and plan-specific distribution analyses relative to database plans for documentation and to aid physicist interpretation and evaluation. VQA predictions were prospectively validated over 3 months of measurements at our clinic to assess safety and efficiency gains. RESULTS: Over 3 months, VQA predictions for 445 volumetric modulated arc therapy plans were prospectively validated at our institution. VQA predictions for these plans had a mean absolute error of 1.08% ± 0.77%, with a maximum absolute error of 2.98%. Using a 1% prediction threshold (ie, plans predicted to have an absolute error <1% would not require a measurement) would yield a 69.2% reduction in QA workload, saving 32.5 hours per month on average, with 81.5% sensitivity, 72.4% specificity, and an area under the curve of 0.81 at a 3% clinical threshold and 100% sensitivity, 70% specificity, and an area under the curve of 0.93 at a 4% clinical threshold. CONCLUSIONS: This is the first prospective clinical implementation and validation of VQA in the United States, which we observed to be efficient. Using a conservative threshold, VQA can substantially reduce the number of required measurements for PSQA, leading to more effective allocation of clinical resources.

Artificial Intelligence-Guided Prediction of Dental Doses Before Planning of Radiation Therapy for Oropharyngeal Cancer: Technical Development and Initial Feasibility of Implementation.

Purpose: The aim was to develop a novel artificial intelligence (AI)-guided clinical decision support system, to predict radiation doses to subsites of the mandible using diagnostic computed tomography scans acquired before any planning of head and neck radiation therapy (RT). Methods and Materials: A dose classifier was trained using RT plans from 86 patients with oropharyngeal cancer; the test set consisted of an additional 20 plans. The classifier was trained to predict whether mandible subsites would receive a mean dose >50 Gy. The AI predictions were prospectively evaluated and compared with those of a specialist head and neck radiation oncologist for 9 patients. Positive predictive value (PPV), negative predictive value (NPV), Pearson correlation coefficient, and Lin concordance correlation coefficient were calculated to compare the AI predictions to those of the physician. Results: In the test data set, the AI predictions had a PPV of 0.95 and NPV of 0.88. For 9 patients evaluated prospectively, there was a strong correlation between the predictions of the AI algorithm and physician (P = .72, P < .001). Comparing the AI algorithm versus the physician, the PPVs were 0.82 versus 0.25, and the NPVs were 0.94 versus 1.0, respectively. Concordance between physician estimates and final planned doses was 0.62; this was 0.71 between AI-based estimates and final planned doses. Conclusion: AI-guided decision support increased precision and accuracy of pre-RT dental dose estimates.