Automating Scoliosis Measurements in Radiographic Studies with Machine Learning: Comparing Artificial Intelligence and Clinical Reports.

Scoliosis is a condition of abnormal lateral spinal curvature affecting an estimated 2 to 3% of the US population, or seven million people. The Cobb angle is the standard measurement of spinal curvature in scoliosis but is known to have high interobserver and intraobserver variability. Thus, the objective of this study was to build and validate a system for automatic quantitative evaluation of the Cobb angle and to compare AI generated and human reports in the clinical setting. After IRB was obtained, we retrospectively collected 2150 frontal view scoliosis radiographs at a tertiary referral center (January 1, 2019, to January 1, 2021, ≥ 16 years old, no hardware). The dataset was partitioned into 1505 train (70%), 215 validation (10%), and 430 test images (20%). All thoracic and lumbar vertebral bodies were segmented with bounding boxes, generating approximately 36,550 object annotations that were used to train a Faster R-CNN Resnet-101 object detection model. A controller algorithm was written to localize vertebral centroid coordinates and derive the Cobb properties (angle and endplate) of dominant and secondary curves. AI-derived Cobb angle measurements were compared to the clinical report measurements, and the Spearman rank-order demonstrated significant correlation (0.89, p < 0.001). Mean difference between AI and clinical report angle measurements was 7.34° (95% CI: 5.90-8.78°), which is similar to published literature (up to 10°). We demonstrate the feasibility of an AI system to automate measurement of level-by-level spinal angulation with performance comparable to radiologists.

Automated segmentation of multiparametric magnetic resonance images for cerebral AVM radiosurgery planning: a deep learning approach.

Stereotactic radiosurgery planning for cerebral arteriovenous malformations (AVM) is complicated by the variability in appearance of an AVM nidus across different imaging modalities. We developed a deep learning approach to automatically segment cerebrovascular-anatomical maps from multiple high-resolution magnetic resonance imaging/angiography (MRI/MRA) sequences in AVM patients, with the goal of facilitating target delineation. Twenty-three AVM patients who were evaluated for radiosurgery and underwent multi-parametric MRI/MRA were included. A hybrid semi-automated and manual approach was used to label MRI/MRAs with arteries, veins, brain parenchyma, cerebral spinal fluid (CSF), and embolized vessels. Next, these labels were used to train a convolutional neural network to perform this task. Imaging from 17 patients (6362 image slices) was used for training, and 6 patients (1224 slices) for validation. Performance was evaluated by Dice Similarity Coefficient (DSC). Classification performance was good for arteries, veins, brain parenchyma, and CSF, with DSCs of 0.86, 0.91, 0.98, and 0.91, respectively in the validation image set. Performance was lower for embolized vessels, with a DSC of 0.75. This demonstrates the proof of principle that accurate, high-resolution cerebrovascular-anatomical maps can be generated from multiparametric MRI/MRA. Clinical validation of their utility in radiosurgery planning is warranted.

Radiologist-supervised Transfer Learning: Improving Radiographic Localization of Pneumonia and Prognostication of Patients With COVID-19.

PURPOSE: To assess the potential of a transfer learning strategy leveraging radiologist supervision to enhance convolutional neural network-based (CNN) localization of pneumonia on radiographs and to further assess the prognostic value of CNN severity quantification on patients evaluated for COVID-19 pneumonia, for whom severity on the presenting radiograph is a known predictor of mortality and intubation. MATERIALS AND METHODS: We obtained an initial CNN previously trained to localize pneumonia along with 25,684 radiographs used for its training. We additionally curated 1466 radiographs from patients who had a computed tomography (CT) performed on the same day. Regional likelihoods of pneumonia were then annotated by cardiothoracic radiologists, referencing these CTs. Combining data, a preexisting CNN was fine-tuned using transfer learning. Whole-image and regional performance of the updated CNN was assessed using receiver-operating characteristic area under the curve and Dice. Finally, the value of CNN measurements was assessed with survival analysis on 203 patients with COVID-19 and compared against modified radiographic assessment of lung edema (mRALE) score. RESULTS: Pneumonia detection area under the curve improved on both internal (0.756 to 0.841) and external (0.864 to 0.876) validation data. Dice overlap also improved, particularly in the lung bases (R: 0.121 to 0.433, L: 0.111 to 0.486). There was strong correlation between radiologist mRALE score and CNN fractional area of involvement (ρ=0.85). Survival analysis showed similar, strong prognostic ability of the CNN and mRALE for mortality, likelihood of intubation, and duration of hospitalization among patients with COVID-19. CONCLUSIONS: Radiologist-supervised transfer learning can enhance the ability of CNNs to localize and quantify the severity of disease. Closed-loop systems incorporating radiologists may be beneficial for continued improvement of artificial intelligence algorithms.

Augmenting Interpretation of Chest Radiographs With Deep Learning Probability Maps.

PURPOSE: Pneumonia is a common clinical diagnosis for which chest radiographs are often an important part of the diagnostic workup. Deep learning has the potential to expedite and improve the clinical interpretation of chest radiographs. While earlier approaches have emphasized the feasibility of "binary classification" to accomplish this task, alternative strategies may be possible. We explore the feasibility of a "semantic segmentation" deep learning approach to highlight the potential foci of pneumonia on frontal chest radiographs. MATERIALS AND METHODS: In this retrospective study, we trained a U-net convolutional neural network (CNN) to predict pixel-wise probability maps for pneumonia using a public data set provided by the Radiological Society of North America (RSNA) comprised of 22,000 radiographs and radiologist-defined bounding boxes. We reserved 3684 radiographs as an independent validation data set and assessed overall performance for localization using Dice overlap and classification performance using the area under the receiver-operator characteristic curve. RESULTS: For classification/detection of pneumonia, area under the receiver-operator characteristic curve on frontal radiographs was 0.854 with a sensitivity of 82.8% and specificity of 72.6%. Using this strategy of neural network training, probability maps localized pneumonia to lung parenchyma for essentially all validation cases. For segmentation of pneumonia for positive cases, predicted probability maps had a mean Dice score (±SD) of 0.603±0.204, and 60.0% of these had a Dice score >0.5. CONCLUSIONS: A "semantic segmentation" deep learning approach can provide a probabilistic map to assist in the diagnosis of pneumonia. In combination with the patient's history, clinical findings and other imaging, this strategy may help expedite and improve diagnosis.

Deployment of artificial intelligence for radiographic diagnosis of COVID-19 pneumonia in the emergency department.

OBJECTIVE: The coronavirus disease 2019 pandemic has inspired new innovations in diagnosing, treating, and dispositioning patients during high census conditions with constrained resources. Our objective is to describe first experiences of physician interaction with a novel artificial intelligence (AI) algorithm designed to enhance physician abilities to identify ground-glass opacities and consolidation on chest radiographs. METHODS: During the first wave of the pandemic, we deployed a previously developed and validated deep-learning AI algorithm for assisted interpretation of chest radiographs for use by physicians at an academic health system in Southern California. The algorithm overlays radiographs with "heat" maps that indicate pneumonia probability alongside standard chest radiographs at the point of care. Physicians were surveyed in real time regarding ease of use and impact on clinical decisionmaking. RESULTS: Of the 5125 total visits and 1960 chest radiographs obtained in the emergency department (ED) during the study period, 1855 were analyzed by the algorithm. Among these, emergency physicians were surveyed for their experiences on 202 radiographs. Overall, 86% either strongly agreed or somewhat agreed that the intervention was easy to use in their workflow. Of the respondents, 20% reported that the algorithm impacted clinical decisionmaking. CONCLUSIONS: To our knowledge, this is the first published literature evaluating the impact of medical imaging AI on clinical decisionmaking in the emergency department setting. Urgent deployment of a previously validated AI algorithm clinically was easy to use and was found to have an impact on clinical decision making during the predicted surge period of a global pandemic.

Cancer-promoting mechanisms of tumor-associated neutrophils.

IMPORTANCE: Neutrophils have classically been considered to mount a defensive response against tumor cells, yet recent evidence suggests tumors modulate neutrophil function to support tumor growth and progression. OBSERVATIONS: Tumor-associated neutrophils (TANs) are phenotypically distinct from circulating neutrophils in terms of their surface protein composition and cyto/chemokine activity and response. Although TANs have been shown to both promote and inhibit tumor advancement, the preponderant activity augments tumor progression. This review discusses these cancer-promoting molecular pathways, relevant diagnostic studies in patients, and subsequent treatment modalities. The tumor promoting mechanisms of TANs include dampening of CD8+ response via Arginase-1; a neutrophil-secreted neutrophil elastase (NE) upregulation of tumor cellular proliferation pathways; degradation of basement membrane and ECM via NE and MMP-9; upregulation of angiogenesis by VEGF, and HGF; and ICAM-1 dependent tumor intravasation, immune protection in circulation, and extravasation into distant, metastatic tissue beds. Clinicians are constrained in treating TANs systemically as it may induce neutropenia, therefore targeting TANs-mediated tumor progression pathways surgically on a loco-regional level is a viable adjuvant treatment modality. CONCLUSION AND RELEVANCE: TANs modulate the tumor microenvironment promoting tumor progression. Mechanistic understanding of TANs role in tumor progression will provide unique therapeutic alternatives.

Spatiotemporal progression of metastatic breast cancer: a Markov chain model highlighting the role of early metastatic sites.

BACKGROUND: Cancer cell migration patterns are critical for understanding metastases and clinical evolution. Breast cancer spreads from one organ system to another via hematogenous and lymphatic routes. Although patterns of spread may superficially seem random and unpredictable, we explored the possibility that this is not the case. AIMS: Develop a Markov based model of breast cancer progression that has predictive capability. METHODS: On the basis of a longitudinal data set of 446 breast cancer patients, we created a Markov chain model of metastasis that describes the probabilities of metastasis occurring at a given anatomic site together with the probability of spread to additional sites. Progression is modeled as a random walk on a directed graph, where nodes represent anatomical sites where tumors can develop. RESULTS: We quantify how survival depends on the location of the first metastatic site for different patient subcategories. In addition, we classify metastatic sites as "sponges" or "spreaders" with implications regarding anatomical pathway prediction and long-term survival. As metastatic tumors to the bone (main spreader) are most prominent, we focus in more detail on differences between groups of patients who form subsequent metastases to the lung as compared with the liver. CONCLUSIONS: We have found that spatiotemporal patterns of metastatic spread in breast cancer are neither random nor unpredictable. Furthermore, the novel concept of classifying organ sites as sponges or spreaders may motivate experiments seeking a biological basis for these phenomena and allow us to quantify the potential consequences of therapeutic targeting of sites in the oligometastatic setting and shed light on organotropic aspects of the disease.

Entropy, complexity, and Markov diagrams for random walk cancer models.

The notion of entropy is used to compare the complexity associated with 12 common cancers based on metastatic tumor distribution autopsy data. We characterize power-law distributions, entropy, and Kullback-Liebler divergence associated with each primary cancer as compared with data for all cancer types aggregated. We then correlate entropy values with other measures of complexity associated with Markov chain dynamical systems models of progression. The Markov transition matrix associated with each cancer is associated with a directed graph model where nodes are anatomical locations where a metastatic tumor could develop, and edge weightings are transition probabilities of progression from site to site. The steady-state distribution corresponds to the autopsy data distribution. Entropy correlates well with the overall complexity of the reduced directed graph structure for each cancer and with a measure of systemic interconnectedness of the graph, called graph conductance. The models suggest that grouping cancers according to their entropy values, with skin, breast, kidney, and lung cancers being prototypical high entropy cancers, stomach, uterine, pancreatic and ovarian being mid-level entropy cancers, and colorectal, cervical, bladder, and prostate cancers being prototypical low entropy cancers, provides a potentially useful framework for viewing metastatic cancer in terms of predictability, complexity, and metastatic potential.