Development of a natural language processing algorithm for the detection of spinal metastasis based on magnetic resonance imaging reports.

Background: Metastasis to the spinal column is a common complication of malignancy, potentially causing pain and neurologic injury. An automated system to identify and refer patients with spinal metastases can help overcome barriers to timely treatment. We describe the training, optimization and validation of a natural language processing algorithm to identify the presence of vertebral metastasis and metastatic epidural cord compression (MECC) from radiology reports of spinal MRIs. Methods: Reports from patients with spine MRI studies performed between January 1, 2008 and April 14, 2019 were reviewed by a team of radiologists to assess for the presence of cancer and generate a labeled dataset for model training. Using regular expression, impression sections were extracted from the reports and converted to all lower-case letters with all nonalphabetic characters removed. The reports were then tokenized and vectorized using the doc2vec algorithm. These were then used to train a neural network to predict the likelihood of spinal tumor or MECC. For each report, the model provided a number from 0 to 1 corresponding to its impression. We then obtained 111 MRI reports from outside the test set, 92 manually labeled negative and 19 with MECC to test the model's performance. Results: About 37,579 radiology reports were reviewed. About 36,676 were labeled negative, and 903 with MECC. We chose a cutoff of 0.02 as a positive result to optimize for a low false negative rate. At this threshold we found a 100% sensitivity rate with a low false positive rate of 2.2%. Conclusions: The NLP model described predicts the presence of spinal tumor and MECC in spine MRI reports with high accuracy. We plan to implement the algorithm into our EMR to allow for faster referral of these patients to appropriate specialists, allowing for reduced morbidity and increased survival.

Development of an automated, general-purpose prediction tool for postoperative respiratory failure using machine learning: A retrospective cohort study.

STUDY OBJECTIVE: Postoperative respiratory failure is a major surgical complication and key quality metric. Existing prediction tools underperform, are limited to specific populations, and necessitate manual calculation. This limits their implementation. We aimed to create an improved, machine learning powered prediction tool with ideal characteristics for automated calculation. DESIGN, SETTING, AND PATIENTS: We retrospectively reviewed 101,455 anesthetic procedures from 1/2018 to 6/2021. The primary outcome was the Standardized Endpoints in Perioperative Medicine consensus definition for postoperative respiratory failure. Secondary outcomes were respiratory quality metrics from the National Surgery Quality Improvement Sample, Society of Thoracic Surgeons, and CMS. We abstracted from the electronic health record 26 procedural and physiologic variables previously identified as respiratory failure risk factors. We randomly split the cohort and used the Random Forest method to predict the composite outcome in the training cohort. We coined this the RESPIRE model and measured its accuracy in the validation cohort using area under the receiver operating curve (AUROC) analysis, among other measures, and compared this with ARISCAT and SPORC-1, two leading prediction tools. We compared performance in a validation cohort using score cut-offs determined in a separate test cohort. MAIN RESULTS: The RESPIRE model exhibited superior accuracy with an AUROC of 0.93 (95% CI, 0.92-0.95) compared to 0.82 for both ARISCAT and SPORC-1 (P-for-difference < 0.0001 for both). At comparable 80-90% sensitivities, RESPIRE had higher positive predictive value (11%, 95% CI: 10-12%) and lower false positive rate (12%, 95% CI: 12-13%) compared to 4% and 37% for both ARISCAT and SPORC-1. The RESPIRE model also better predicted the established quality metrics for postoperative respiratory failure. CONCLUSIONS: We developed a general-purpose, machine learning powered prediction tool with superior performance for research and quality-based definitions of postoperative respiratory failure.

Development and validation of a machine learning ASA-score to identify candidates for comprehensive preoperative screening and risk stratification.

OBJECTIVE: The ASA physical status (ASA-PS) is determined by an anesthesia provider or surgeon to communicate co-morbidities relevant to perioperative risk. Assigning an ASA-PS is a clinical decision and there is substantial provider-dependent variability. We developed and externally validated a machine learning-derived algorithm to determine ASA-PS (ML-PS) based on data available in the medical record. DESIGN: Retrospective multicenter hospital registry study. SETTING: University-affiliated hospital networks. PATIENTS: Patients who received anesthesia at Beth Israel Deaconess Medical Center (Boston, MA, training [n = 361,602] and internal validation cohorts [n = 90,400]) and Montefiore Medical Center (Bronx, NY, external validation cohort [n = 254,412]). MEASUREMENTS: The ML-PS was created using a supervised random forest model with 35 preoperatively available variables. Its predictive ability for 30-day mortality, postoperative ICU admission, and adverse discharge were determined by logistic regression. MAIN RESULTS: The anesthesiologist ASA-PS and ML-PS were in agreement in 57.2% of the cases (moderate inter-rater agreement). Compared with anesthesiologist rating, ML-PS assigned more patients into extreme ASA-PS (I and IV), (p < 0.01), and less patients in ASA II and III (p < 0.01). ML-PS and anesthesiologist ASA-PS had excellent predictive values for 30-day mortality, and good predictive values for postoperative ICU admission and adverse discharge. Among the 3594 patients who died within 30 days after surgery, net reclassification improvement analysis revealed that using the ML-PS, 1281 (35.6%) patients were reclassified into the higher clinical risk category compared with anesthesiologist rating. However, in a subgroup of multiple co-morbidity patients, anesthesiologist ASA-PS had a better predictive accuracy than ML-PS. CONCLUSIONS: We created and validated a machine learning physical status based on preoperatively available data. The ability to identify patients at high risk early in the preoperative process independent of the provider's decision is a part of the process we use to standardize the stratified preoperative evaluation of patients scheduled for ambulatory surgery.

Improving Adjuvant Liver-Directed Treatment Recommendations for Unresectable Hepatocellular Carcinoma: An Artificial Intelligence-Based Decision-Making Tool.

PURPOSE: Liver-directed therapy after transarterial chemoembolization (TACE) can lead to improvement in survival for selected patients with unresectable hepatocellular carcinoma (HCC). However, there is uncertainty in the appropriate application and modality of therapy in current clinical practice guidelines. The aim of this study was to develop a proof-of-concept, machine learning (ML) model for treatment recommendation in patients previously treated with TACE and select patients who might benefit from additional treatment with combination stereotactic body radiotherapy (SBRT) or radiofrequency ablation (RFA). METHODS: This retrospective observational study was based on data from an urban, academic hospital system selecting for patients diagnosed with stage I-III HCC from January 1, 2008, to December 31, 2018, treated with TACE, followed by adjuvant RFA, SBRT, or no additional liver-directed modality. A feedforward, ML ensemble model provided a treatment recommendation on the basis of pairwise assessments evaluating each potential treatment option and estimated benefit in survival. RESULTS: Two hundred thirty-seven patients met inclusion criteria, of whom 54 (23%) and 49 (21%) received combination of TACE and SBRT or TACE and RFA, respectively. The ML model suggested a different consolidative modality in 32.7% of cases among patients who had previously received combination treatment. Patients treated in concordance with model recommendations had significant improvement in progression-free survival (hazard ratio 0.5; P = .007). The most important features for model prediction were cause of cirrhosis, stage of disease, and albumin-bilirubin grade (a measure of liver function). CONCLUSION: In this proof-of-concept study, an ensemble ML model was able to provide treatment recommendations for HCC who had undergone prior TACE. Additional treatment in line with model recommendations was associated with significant improvement in progression-free survival, suggesting a potential benefit for ML-guided medical decision making.