Developing an Automated Registry (Autoregistry) of Spine Surgery Using Natural Language Processing and Health System Scale Databases.

BACKGROUND AND OBJECTIVES: Clinical registries are critical for modern surgery and underpin outcomes research, device monitoring, and trial development. However, existing approaches to registry construction are labor-intensive, costly, and prone to manual error. Natural language processing techniques combined with electronic health record (EHR) data sets can theoretically automate the construction and maintenance of registries. Our aim was to automate the generation of a spine surgery registry at an academic medical center using regular expression (regex) classifiers developed by neurosurgeons to combine domain expertise with interpretable algorithms. METHODS: We used a Hadoop data lake consisting of all the information generated by an academic medical center. Using this database and structured query language queries, we retrieved every operative note written in the department of neurosurgery since our transition to EHR. Notes were parsed using regex classifiers and compared with a random subset of 100 manually reviewed notes. RESULTS: A total of 31 502 operative cases were downloaded and processed using regex classifiers. The codebase required 5 days of development, 3 weeks of validation, and less than 1 hour for the software to generate the autoregistry. Regex classifiers had an average accuracy of 98.86% at identifying both spinal procedures and the relevant vertebral levels, and it correctly identified the entire list of defined surgical procedures in 89% of patients. We were able to identify patients who required additional operations within 30 days to monitor outcomes and quality metrics. CONCLUSION: This study demonstrates the feasibility of automatically generating a spine registry using the EHR and an interpretable, customizable natural language processing algorithm which may reduce pitfalls associated with manual registry development and facilitate rapid clinical research.

Health system-scale language models are all-purpose prediction engines.

Physicians make critical time-constrained decisions every day. Clinical predictive models can help physicians and administrators make decisions by forecasting clinical and operational events. Existing structured data-based clinical predictive models have limited use in everyday practice owing to complexity in data processing, as well as model development and deployment1-3. Here we show that unstructured clinical notes from the electronic health record can enable the training of clinical language models, which can be used as all-purpose clinical predictive engines with low-resistance development and deployment. Our approach leverages recent advances in natural language processing4,5 to train a large language model for medical language (NYUTron) and subsequently fine-tune it across a wide range of clinical and operational predictive tasks. We evaluated our approach within our health system for five such tasks: 30-day all-cause readmission prediction, in-hospital mortality prediction, comorbidity index prediction, length of stay prediction, and insurance denial prediction. We show that NYUTron has an area under the curve (AUC) of 78.7-94.9%, with an improvement of 5.36-14.7% in the AUC compared with traditional models. We additionally demonstrate the benefits of pretraining with clinical text, the potential for increasing generalizability to different sites through fine-tuning and the full deployment of our system in a prospective, single-arm trial. These results show the potential for using clinical language models in medicine to read alongside physicians and provide guidance at the point of care.

Clinical Validation of Stimulated Raman Histology for Rapid Intraoperative Diagnosis of Central Nervous System Tumors.

Stimulated Raman histology (SRH) is an ex vivo optical imaging method that enables microscopic examination of fresh tissue intraoperatively. The conventional intraoperative method uses frozen section analysis, which is labor and time intensive, introduces artifacts that limit diagnostic accuracy, and consumes tissue. SRH imaging allows rapid microscopic imaging of fresh tissue, avoids tissue loss, and enables remote telepathology review. This improves access to expert neuropathology consultation in both low- and high-resource practices. We clinically validated SRH by performing a blinded, retrospective two-arm telepathology study to clinically validate SRH for telepathology at our institution. Using surgical specimens from 47 subjects, we generated a data set composed of 47 SRH images and 47 matched whole slide images (WSIs) of formalin-fixed, paraffin-embedded tissue stained with hematoxylin and eosin, with associated intraoperative clinicoradiologic information and structured diagnostic questions. We compared diagnostic concordance between WSI and SRH-rendered diagnoses. Also, we compared the 1-year median turnaround time (TAT) of intraoperative conventional neuropathology frozen sections with prospectively rendered SRH-telepathology TAT. All SRH images were of sufficient quality for diagnostic review. A review of SRH images showed high accuracy in distinguishing glial from nonglial tumors (96.5% SRH vs 98% WSIs) and predicting final diagnosis (85.9% SRH vs 93.1% WSIs). SRH-based diagnosis and WSI-permanent section diagnosis had high concordance (κ = 0.76). The median TAT for prospectively SRH-rendered diagnosis was 3.7 minutes, approximately 10-fold shorter than the median frozen section TAT (31 minutes). The SRH-imaging procedure did not affect ancillary studies. SRH generates diagnostic virtual histologic images with accuracy comparable to conventional hematoxylin and eosin-based methods in a rapid manner. Our study represents the largest and most rigorous clinical validation of SRH to date. It supports the feasibility of implementing SRH as a rapid method for intraoperative diagnosis complementary to conventional pathology laboratory methods.

Identification of Spatial Proteomic Signatures of Colon Tumor Metastasis: A Digital Spatial Profiling Approach.

Over 150,000 Americans are diagnosed with colorectal cancer (CRC) every year, and annually >50,000 individuals are estimated to die of CRC, necessitating improvements in screening, prognostication, disease management, and therapeutic options. CRC tumors are removed en bloc with surrounding vasculature and lymphatics. Examination of regional lymph nodes at the time of surgical resection is essential for prognostication. Developing alternative approaches to indirectly assess recurrence risk would have utility in cases where lymph node yield is incomplete or inadequate. Spatially dependent, immune cell-specific (eg, tumor-infiltrating lymphocytes), proteomic, and transcriptomic expression patterns inside and around the tumor-the tumor immune microenvironment-can predict nodal/distant metastasis and probe the coordinated immune response from the primary tumor site. The comprehensive characterization of tumor-infiltrating lymphocytes and other immune infiltrates is possible using highly multiplexed spatial omics technologies, such as the GeoMX Digital Spatial Profiler. In this study, machine learning and differential co-expression analyses helped identify biomarkers from Digital Spatial Profiler-assayed protein expression patterns inside, at the invasive margin, and away from the tumor, associated with extracellular matrix remodeling (eg, granzyme B and fibronectin), immune suppression (eg, forkhead box P3), exhaustion and cytotoxicity (eg, CD8), Programmed death ligand 1-expressing dendritic cells, and neutrophil proliferation, among other concomitant alterations. Further investigation of these biomarkers may reveal independent risk factors of CRC metastasis that can be formulated into low-cost, widely available assays.

Artificial-intelligence-based molecular classification of diffuse gliomas using rapid, label-free optical imaging.

Molecular classification has transformed the management of brain tumors by enabling more accurate prognostication and personalized treatment. However, timely molecular diagnostic testing for patients with brain tumors is limited, complicating surgical and adjuvant treatment and obstructing clinical trial enrollment. In this study, we developed DeepGlioma, a rapid (<90 seconds), artificial-intelligence-based diagnostic screening system to streamline the molecular diagnosis of diffuse gliomas. DeepGlioma is trained using a multimodal dataset that includes stimulated Raman histology (SRH); a rapid, label-free, non-consumptive, optical imaging method; and large-scale, public genomic data. In a prospective, multicenter, international testing cohort of patients with diffuse glioma (n = 153) who underwent real-time SRH imaging, we demonstrate that DeepGlioma can predict the molecular alterations used by the World Health Organization to define the adult-type diffuse glioma taxonomy (IDH mutation, 1p19q co-deletion and ATRX mutation), achieving a mean molecular classification accuracy of 93.3 ± 1.6%. Our results represent how artificial intelligence and optical histology can be used to provide a rapid and scalable adjunct to wet lab methods for the molecular screening of patients with diffuse glioma.

Methods and Impact for Using Federated Learning to Collaborate on Clinical Research.

BACKGROUND: The development of accurate machine learning algorithms requires sufficient quantities of diverse data. This poses a challenge in health care because of the sensitive and siloed nature of biomedical information. Decentralized algorithms through federated learning (FL) avoid data aggregation by instead distributing algorithms to the data before centrally updating one global model. OBJECTIVE: To establish a multicenter collaboration and assess the feasibility of using FL to train machine learning models for intracranial hemorrhage (ICH) detection without sharing data between sites. METHODS: Five neurosurgery departments across the United States collaborated to establish a federated network and train a convolutional neural network to detect ICH on computed tomography scans. The global FL model was benchmarked against a standard, centrally trained model using a held-out data set and was compared against locally trained models using site data. RESULTS: A federated network of practicing neurosurgeon scientists was successfully initiated to train a model for predicting ICH. The FL model achieved an area under the ROC curve of 0.9487 (95% CI 0.9471-0.9503) when predicting all subtypes of ICH compared with a benchmark (non-FL) area under the ROC curve of 0.9753 (95% CI 0.9742-0.9764), although performance varied by subtype. The FL model consistently achieved top three performance when validated on any site's data, suggesting improved generalizability. A qualitative survey described the experience of participants in the federated network. CONCLUSION: This study demonstrates the feasibility of implementing a federated network for multi-institutional collaboration among clinicians and using FL to conduct machine learning research, thereby opening a new paradigm for neurosurgical collaboration.

Generating novel pituitary datasets from open-source imaging data and deep volumetric segmentation.

PURPOSE: The estimated incidence of pituitary adenomas in the general population is 10-30%, yet radiographic diagnosis remains a challenge. Diagnosis is complicated by the heterogeneity of radiographic features in both normal (e.g. complex anatomy, pregnancy) and pathologic states (e.g. primary endocrinopathy, hypophysitis). Clinical symptoms and laboratory testing are often equivocal, which can result in misdiagnosis or unnecessary specialist referrals. Computer vision models can aid in pituitary adenoma diagnosis; however, a major challenge to model development is the lack of dedicated pituitary imaging datasets. We hypothesized that deep volumetric segmentation models trained to extract the sellar and parasellar region from existing whole-brain MRI scans could be used to generate a novel dataset of pituitary imaging. METHODS: Six open-source whole-brain MRI datasets, created for research purposes, were included for model development. Deep learning-based volumetric segmentation models were trained using 318 manually annotated MRI scans from a single open-source MRI dataset. Out-of-distribution volumetric segmentation performance was then tested on 418 MRIs from five held-out research datasets. RESULTS: On our annotated images, agreement between manual and model volumetric segmentations was high. Dice scores (a measure of overlap) ranged 0.76-0.82 for both in-distribution and out-of-distribution model testing. In total, 6,755 MRIs from six data sources were included in the final generated pituitary dataset. CONCLUSIONS: We present the first and largest dataset of pituitary imaging constructed using existing MRI data and deep volumetric segmentation models trained to identify sellar and parasellar anatomy. The model generalizes well across patient populations and MRI scanner types. We hope our pituitary dataset will be an integral part of future machine learning research on pituitary pathologies.

Predicting oncogene mutations of lung cancer using deep learning and histopathologic features on whole-slide images.

Lung cancer is a leading cause of death in both men and women globally. The recent development of tumor molecular profiling has opened opportunities for targeted therapies for lung adenocarcinoma (LUAD) patients. However, the lack of access to molecular profiling or cost and turnaround time associated with it could hinder oncologists' willingness to order frequent molecular tests, limiting potential benefits from precision medicine. In this study, we developed a weakly supervised deep learning model for predicting somatic mutations of LUAD patients based on formalin-fixed paraffin-embedded (FFPE) whole-slide images (WSIs) using LUAD subtypes-related histological features and recent advances in computer vision. Our study was performed on a total of 747 hematoxylin and eosin (H&E) stained FFPE LUAD WSIs and the genetic mutation data of 232 patients who were treated at Dartmouth-Hitchcock Medical Center (DHMC). We developed our convolutional neural network-based models to analyze whole slides and predict five major genetic mutations, i.e., BRAF, EGFR, KRAS, STK11, and TP53. We additionally used 111 cases from the LUAD dataset of the CPTAC-3 study for external validation. Our model achieved an AUROC of 0.799 (95% CI: 0.686-0.904) and 0.686 (95% CI: 0.620-0.752) for predicting EGFR genetic mutations on the DHMC and CPTAC-3 test sets, respectively. Predicting TP53 genetic mutations also showed promising outcomes. Our results demonstrated that H&E stained FFPE LUAD whole slides could be utilized to predict oncogene mutations, such as EGFR, indicating that somatic mutations could present subtle morphological characteristics in histology slides, where deep learning-based feature extractors can learn such latent information.

Mixed Effects Machine Learning Models for Colon Cancer Metastasis Prediction using Spatially Localized Immuno-Oncology Markers.

Spatially resolved characterization of the transcriptome and proteome promises to provide further clarity on cancer pathogenesis and etiology, which may inform future clinical practice through classifier development for clinical outcomes. However, batch effects may potentially obscure the ability of machine learning methods to derive complex associations within spatial omics data. Profiling thirty-five stage three colon cancer patients using the GeoMX Digital Spatial Profiler, we found that mixed-effects machine learning (MEML) methods† may provide utility for overcoming significant batch effects to communicate key and complex disease associations from spatial information. These results point to further exploration and application of MEML methods within the spatial omics algorithm development life cycle for clinical deployment.

Evaluation of an Artificial Intelligence-Augmented Digital System for Histologic Classification of Colorectal Polyps.

Importance: Colorectal polyps are common, and their histopathologic classification is used in the planning of follow-up surveillance. Substantial variation has been observed in pathologists' classification of colorectal polyps, and improved assessment by pathologists may be associated with reduced subsequent underuse and overuse of colonoscopy. Objective: To compare standard microscopic assessment with an artificial intelligence (AI)-augmented digital system that annotates regions of interest within digitized polyp tissue and predicts polyp type using a deep learning model to assist pathologists in colorectal polyp classification. Design, Setting, and Participants: In this diagnostic study conducted at a tertiary academic medical center and a community hospital in New Hampshire, 100 slides with colorectal polyp samples were read by 15 pathologists using a microscope and an AI-augmented digital system, with a washout period of at least 12 weeks between use of each modality. The study was conducted from February 10 to July 10, 2020. Main Outcomes and Measures: Accuracy and time of evaluation were used to compare pathologists' performance when a microscope was used with their performance when the AI-augmented digital system was used. Outcomes were compared using paired t tests and mixed-effects models. Results: In assessments of 100 slides with colorectal polyp specimens, use of the AI-augmented digital system significantly improved pathologists' classification accuracy compared with microscopic assessment from 73.9% (95% CI, 71.7%-76.2%) to 80.8% (95% CI, 78.8%-82.8%) (P < .001). The overall difference in the evaluation time per slide between the digital system (mean, 21.7 seconds; 95% CI, 20.8-22.7 seconds) and microscopic examination (mean, 13.0 seconds; 95% CI, 12.4-13.5 seconds) was -8.8 seconds (95% CI, -9.8 to -7.7 seconds), but this difference decreased as pathologists became more familiar and experienced with the digital system; the difference between the time of evaluation on the last set of 20 slides for all pathologists when using the microscope and the digital system was 4.8 seconds (95% CI, 3.0-6.5 seconds). Conclusions and Relevance: In this diagnostic study, an AI-augmented digital system significantly improved the accuracy of pathologic interpretation of colorectal polyps compared with microscopic assessment. If applied broadly to clinical practice, this tool may be associated with decreases in subsequent overuse and underuse of colonoscopy and thus with improved patient outcomes and reduced health care costs.