Performance of GPT-4 on the American College of Radiology In-training Examination: Evaluating Accuracy, Model Drift, and Fine-tuning.

RATIONALE AND OBJECTIVES: In our study, we evaluate GPT-4's performance on the American College of Radiology (ACR) 2022 Diagnostic Radiology In-Training Examination (DXIT). We perform multiple experiments across time points to assess for model drift, as well as after fine-tuning to assess for differences in accuracy. MATERIALS AND METHODS: Questions were sequentially input into GPT-4 with a standardized prompt. Each answer was recorded and overall accuracy was calculated, as was logic-adjusted accuracy, and accuracy on image-based questions. This experiment was repeated several months later to assess for model drift, then again after the performance of fine-tuning to assess for changes in GPT's performance. RESULTS: GPT-4 achieved 58.5% overall accuracy, lower than the PGY-3 average (61.9%) but higher than the PGY-2 average (52.8%). Adjusted accuracy was 52.8%. GPT-4 showed significantly higher (p = 0.012) confidence for correct answers (87.1%) compared to incorrect (84.0%). Performance on image-based questions was significantly poorer (p < 0.001) at 45.4% compared to text-only questions (80.0%), with adjusted accuracy for image-based questions of 36.4%. When the questions were repeated, GPT-4 chose a different answer 25.5% of the time and there was no change in accuracy. Fine-tuning did not improve accuracy. CONCLUSION: GPT-4 performed between PGY-2 and PGY-3 levels on the 2022 DXIT, significantly poorer on image-based questions, and with large variability in answer choices across time points. Exploratory experiments in fine-tuning did not improve performance. This study underscores the potential and risks of using minimally-prompted general AI models in interpreting radiologic images as a diagnostic tool. Implementers of general AI radiology systems should exercise caution given the possibility of spurious yet confident responses.

Advancing Precision Medicine: Algebraic Topology and Differential Geometry in Radiology and Computational Pathology.

Precision medicine aims to provide personalized care based on individual patient characteristics, rather than guideline-directed therapies for groups of diseases or patient demographics. Images-both radiology- and pathology-derived-are a major source of information on presence, type, and status of disease. Exploring the mathematical relationship of pixels in medical imaging ("radiomics") and cellular-scale structures in digital pathology slides ("pathomics") offers powerful tools for extracting both qualitative, and increasingly, quantitative data. These analytical approaches, however, may be significantly enhanced by applying additional methods arising from fields of mathematics such as differential geometry and algebraic topology that remain underexplored in this context. Geometry's strength lies in its ability to provide precise local measurements, such as curvature, that can be crucial for identifying abnormalities at multiple spatial levels. These measurements can augment the quantitative features extracted in conventional radiomics, leading to more nuanced diagnostics. By contrast, topology serves as a robust shape descriptor, capturing essential features such as connected components and holes. The field of topological data analysis was initially founded to explore the shape of data, with functional network connectivity in the brain being a prominent example. Increasingly, its tools are now being used to explore organizational patterns of physical structures in medical images and digitized pathology slides. By leveraging tools from both differential geometry and algebraic topology, researchers and clinicians may be able obtain a more comprehensive, multi-layered understanding of medical images and contribute to precision medicine's armamentarium.

Pretreatment Spatially Aware Magnetic Resonance Imaging Radiomics Can Predict Distant Brain Metastases (DBMs) After Stereotactic Radiosurgery/Radiation Therapy (SRS/SRT).

Purpose: Stereotactic radiosurgery/radiation therapy (SRS/SRT) increasingly has been used to treat brain metastases. However, the development of distant brain metastases (DBMs) in the untreated brain remains a serious complication. We sought to develop a spatially aware radiomic signature to model the time-to-DBM development in a cohort of patients leveraging pretreatment magnetic resonance imaging (MRI) and radiation therapy treatment planning data including radiation dose distribution maps. Methods and Materials: We retrospectively analyzed a cohort of 105 patients with brain metastases treated by SRS/SRT with pretreatment multiparametric MRI (T1, T1 postcontrast, T2, fluid-attenuated inversion recovery). Three-dimensional radiomic features were extracted from each MRI sequence within 5 isodose regions of interest (ROIs) identified via radiation dose distribution maps and gross target volume (GTV) contours. Clinical features including patient performance status, number of lesions treated, tumor volume, and tumor stage were collected to serve as a baseline for comparison. Cox proportional hazards (CPH) modeling and Kaplan-Meier analysis were used to model time-to-DBM development. Results: CPH models trained using radiomic features achieved a mean concordance index (c-index) of 0.63 (standard deviation [SD], 0.08) compared with a c-index of 0.49 (SD, 0.09) for CPH models trained using clinical factors. A CPH model trained using both radiomic and clinical features achieved a c-index of 0.69 (SD, 0.08). The identified radiomic signature was able to stratify patients into distinct risk groups with statistically significant differences (P = .00007) in time-to-DBM development as measured by log-rank test. Clinical features were unable to do the same. Radiomic features from the peritumoral 50% to 75% isodose ROI and GTV region were most predictive of DBM development. Conclusions: Our results suggest that radiomic features extracted from pretreatment MRI and multiple isodose ROIs can model time-to-DBM development in patients receiving SRS/SRT for brain metastases, outperforming clinical feature baselines. Notably, we believe we are the first to leverage SRS/SRT dose maps for ROI identification and subsequent radiomic analysis of peritumoral and untargeted brain regions using multiparametric MRI. We observed that the peritumoral environment may be implicated in DBM development for SRS/SRT-treated brain metastases. Our preliminary results might enable the identification of patients with predisposition to DBM development and prompt subsequent changes in disease management.

Automated Detection of Cervical Spinal Stenosis and Cord Compression via Vision Transformer and Rules-Based Classification.

BACKGROUND AND PURPOSE: Cervical spinal cord compression, defined as spinal cord deformity and severe narrowing of the spinal canal in the cervical region, can lead to severe clinical consequences, including intractable pain, sensory disturbance, paralysis, and even death, and may require emergent intervention to prevent negative outcomes. Despite the critical nature of cord compression, no automated tool is available to alert clinical radiologists to the presence of such findings. This study aims to demonstrate the ability of a vision transformer (ViT) model for the accurate detection of cervical cord compression. MATERIALS AND METHODS: A clinically diverse cohort of 142 cervical spine MRIs was identified, 34% of which were normal or had mild stenosis, 31% with moderate stenosis, and 35% with cord compression. Utilizing gradient-echo images, slices were labeled as no cord compression/mild stenosis, moderate stenosis, or severe stenosis/cord compression. Segmentation of the spinal canal was performed and confirmed by neuroradiology faculty. A pretrained ViT model was fine-tuned to predict section-level severity by using a train:validation:test split of 60:20:20. Each examination was assigned an overall severity based on the highest level of section severity, with an examination labeled as positive for cord compression if ≥1 section was predicted in the severe category. Additionally, 2 convolutional neural network (CNN) models (ResNet50, DenseNet121) were tested in the same manner. RESULTS: The ViT model outperformed both CNN models at the section level, achieving section-level accuracy of 82%, compared with 72% and 78% for ResNet and DenseNet121, respectively. ViT patient-level classification achieved accuracy of 93%, sensitivity of 0.90, positive predictive value of 0.90, specificity of 0.95, and negative predictive value of 0.95. Receiver operating characteristic area under the curve was greater for ViT than either CNN. CONCLUSIONS: This classification approach using a ViT model and rules-based classification accurately detects the presence of cervical spinal cord compression at the patient level. In this study, the ViT model outperformed both conventional CNN approaches at the section and patient levels. If implemented into the clinical setting, such a tool may streamline neuroradiology workflow, improving efficiency and consistency.

Attention De-sparsification Matters: Inducing diversity in digital pathology representation learning.

We propose DiRL, a Diversity-inducing Representation Learning technique for histopathology imaging. Self-supervised learning (SSL) techniques, such as contrastive and non-contrastive approaches, have been shown to learn rich and effective representations of digitized tissue samples with limited pathologist supervision. Our analysis of vanilla SSL-pretrained models' attention distribution reveals an insightful observation: sparsity in attention, i.e, models tends to localize most of their attention to some prominent patterns in the image. Although attention sparsity can be beneficial in natural images due to these prominent patterns being the object of interest itself, this can be sub-optimal in digital pathology; this is because, unlike natural images, digital pathology scans are not object-centric, but rather a complex phenotype of various spatially intermixed biological components. Inadequate diversification of attention in these complex images could result in crucial information loss. To address this, we leverage cell segmentation to densely extract multiple histopathology-specific representations, and then propose a prior-guided dense pretext task, designed to match the multiple corresponding representations between the views. Through this, the model learns to attend to various components more closely and evenly, thus inducing adequate diversification in attention for capturing context-rich representations. Through quantitative and qualitative analysis on multiple tasks across cancer types, we demonstrate the efficacy of our method and observe that the attention is more globally distributed.

Convolutional Neural Networks (CNNs) for Pneumonia Classification on Pediatric Chest Radiographs.

BACKGROUND:  Pneumonia is an infectious disease that is especially harmful to those with weak immune systems, such as children under the age of 5. While radiologists' diagnosis of pediatric pneumonia on chest radiographs (CXRs) is often accurate, subtle findings can be missed due to the subjective nature of the diagnosis process. Artificial intelligence (AI) techniques, such as convolutional neural networks (CNNs), can help make the process more objective and precise. However, off-the-shelf CNNs may perform poorly if they are not tuned to their appropriate hyperparameters. Our study aimed to identify the CNNs and their hyperparameter combinations (dropout, batch size, and optimizer) that optimize model performance. METHODOLOGY:  Sixty models based on five CNNs (VGG 16, VGG 19, DenseNet 121, DenseNet 169, and InceptionResNet V2) and 12 hyperparameter combinations were tested. Adam, Root Mean Squared Propagation (RmsProp), and Mini-Batch Stochastic Gradient Descent (SGD) optimizers were used. Two batch sizes, 32 and 64, were utilized. A dropout rate of either 0.5 or 0.7 was used in all dropout layers. We used a deidentified CXR dataset of 4200 pneumonia (Figure 1a) and 1600 normal images (Figure 1b). Seventy percent of the CXRs in the dataset were used for training the model, 20% were used for validating the model, and 10% were used for testing the model. All CNNs were trained first on the ImageNet dataset. They were then trained, with frozen weights, on the CXR-containing dataset.  Results: Among the 60 models, VGG-19 (dropout of 0.5, batch size of 32, and Adam optimizer) was the most accurate. This model achieved an accuracy of 87.9%. A dropout of 0.5 consistently gave higher accuracy, area under the receiver operating characteristics curve (AUROC), and area under the precision-recall curve (AUPRC) compared to a dropout of 0.7. The CNNs InceptionResNet V2, DenseNet 169, VGG 16, and VGG 19 significantly outperformed the DenseNet121 CNN in accuracy and AUROC. The Adam and RmsProp optimizer had improved AUROC and AUPRC compared to the SGD optimizer. The batch size had no statistically significant effect on model performance. CONCLUSION:  We recommend using low dropout rates (0.5) and RmsProp or Adam optimizer for pneumonia-detecting CNNs. Additionally, we discourage using the DenseNet121 CNN when other CNNs are available. Finally, the batch size may be set to any value, dependent on computational resources.

Token Sparsification for Faster Medical Image Segmentation.

Can we use sparse tokens for dense prediction, e.g., segmentation? Although token sparsification has been applied to Vision Transformers (ViT) to accelerate classification, it is still unknown how to perform segmentation from sparse tokens. To this end, we reformulate segmentation as a sparse encoding → token completion → dense decoding (SCD) pipeline. We first empirically show that naïvely applying existing approaches from classification token pruning and masked image modeling (MIM) leads to failure and inefficient training caused by inappropriate sampling algorithms and the low quality of the restored dense features. In this paper, we propose Soft-topK Token Pruning (STP) and Multi-layer Token Assembly (MTA) to address these problems. In sparse encoding, STP predicts token importance scores with a lightweight sub-network and samples the topK tokens. The intractable topK gradients are approximated through a continuous perturbed score distribution. In token completion, MTA restores a full token sequence by assembling both sparse output tokens and pruned multi-layer intermediate ones. The last dense decoding stage is compatible with existing segmentation decoders, e.g., UNETR. Experiments show SCD pipelines equipped with STP and MTA are much faster than baselines without token pruning in both training (up to 120% higher throughput) and inference (up to 60.6% higher throughput) while maintaining segmentation quality. Code is available here: https://github.com/cvlab-stonybrook/TokenSparse-for-MedSeg.

Enhancing Modality-Agnostic Representations via Meta-learning for Brain Tumor Segmentation.

In medical vision, different imaging modalities provide complementary information. However, in practice, not all modalities may be available during inference or even training. Previous approaches, e.g., knowledge distillation or image synthesis, often assume the availability of full modalities for all subjects during training; this is unrealistic and impractical due to the variability in data collection across sites. We propose a novel approach to learn enhanced modality-agnostic representations by employing a meta-learning strategy in training, even when only limited full modality samples are available. Meta-learning enhances partial modality representations to full modality representations by meta-training on partial modality data and meta-testing on limited full modality samples. Additionally, we co-supervise this feature enrichment by introducing an auxiliary adversarial learning branch. More specifically, a missing modality detector is used as a discriminator to mimic the full modality setting. Our segmentation framework significantly outperforms state-of-the-art brain tumor segmentation techniques in missing modality scenarios.

A tumor vasculature-based imaging biomarker for predicting response and survival in patients with lung cancer treated with checkpoint inhibitors.

Tumor vasculature is a key component of the tumor microenvironment that can influence tumor behavior and therapeutic resistance. We present a new imaging biomarker, quantitative vessel tortuosity (QVT), and evaluate its association with response and survival in patients with non-small cell lung cancer (NSCLC) treated with immune checkpoint inhibitor (ICI) therapies. A total of 507 cases were used to evaluate different aspects of the QVT biomarkers. QVT features were extracted from computed tomography imaging of patients before and after ICI therapy to capture the tortuosity, curvature, density, and branching statistics of the nodule vasculature. Our results showed that QVT features were prognostic of OS (HR = 3.14, 0.95% CI = 1.2 to 9.68, P = 0.0006, C-index = 0.61) and could predict ICI response with AUCs of 0.66, 0.61, and 0.67 on three validation sets. Our study shows that QVT imaging biomarker could potentially aid in predicting and monitoring response to ICI in patients with NSCLC.

Deep learning-based approach to the characterization and quantification of histopathology in mouse models of colitis.

Inflammatory bowel disease (IBD) is a chronic immune-mediated disease of the gastrointestinal tract. While therapies exist, response can be limited within the patient population. Researchers have thus studied mouse models of colitis to further understand pathogenesis and identify new treatment targets. Flow cytometry and RNA-sequencing can phenotype immune populations with single-cell resolution but provide no spatial context. Spatial context may be particularly important in colitis mouse models, due to the simultaneous presence of colonic regions that are involved or uninvolved with disease. These regions can be identified on hematoxylin and eosin (H&E)-stained colonic tissue slides based on the presence of abnormal or normal histology. However, detection of such regions requires expert interpretation by pathologists. This can be a tedious process that may be difficult to perform consistently across experiments. To this end, we trained a deep learning model to detect 'Involved' and 'Uninvolved' regions from H&E-stained colonic tissue slides. Our model was trained on specimens from controls and three mouse models of colitis-the dextran sodium sulfate (DSS) chemical induction model, the recently established intestinal epithelium-specific, inducible Klf5ΔIND (Villin-CreERT2;Klf5fl/fl) genetic model, and one that combines both induction methods. Image patches predicted to be 'Involved' and 'Uninvolved' were extracted across mice to cluster and identify histological classes. We quantified the proportion of 'Uninvolved' patches and 'Involved' patch classes in murine swiss-rolled colons. Furthermore, we trained linear determinant analysis classifiers on these patch proportions to predict mouse model and clinical score bins in a prospectively treated cohort of mice. Such a pipeline has the potential to reveal histological links and improve synergy between various colitis mouse model studies to identify new therapeutic targets and pathophysiological mechanisms.

Image analysis reveals molecularly distinct patterns of TILs in NSCLC associated with treatment outcome.

Despite known histological, biological, and clinical differences between lung adenocarcinoma (LUAD) and squamous cell carcinoma (LUSC), relatively little is known about the spatial differences in their corresponding immune contextures. Our study of over 1000 LUAD and LUSC tumors revealed that computationally derived patterns of tumor-infiltrating lymphocytes (TILs) on H&E images were different between LUAD (N = 421) and LUSC (N = 438), with TIL density being prognostic of overall survival in LUAD and spatial arrangement being more prognostically relevant in LUSC. In addition, the LUAD-specific TIL signature was associated with OS in an external validation set of 100 NSCLC treated with more than six different neoadjuvant chemotherapy regimens, and predictive of response to therapy in the clinical trial CA209-057 (n = 303). In LUAD, the prognostic TIL signature was primarily comprised of CD4+ T and CD8+ T cells, whereas in LUSC, the immune patterns were comprised of CD4+ T, CD8+ T, and CD20+ B cells. In both subtypes, prognostic TIL features were associated with transcriptomics-derived immune scores and biological pathways implicated in immune recognition, response, and evasion. Our results suggest the need for histologic subtype-specific TIL-based models for stratifying survival risk and predicting response to therapy. Our findings suggest that predictive models for response to therapy will need to account for the unique morphologic and molecular immune patterns as a function of histologic subtype of NSCLC.

Novel Radiomic Measurements of Tumor-Associated Vasculature Morphology on Clinical Imaging as a Biomarker of Treatment Response in Multiple Cancers.

PURPOSE: The tumor-associated vasculature (TAV) differs from healthy blood vessels by its convolutedness, leakiness, and chaotic architecture, and these attributes facilitate the creation of a treatment-resistant tumor microenvironment. Measurable differences in these attributes might also help stratify patients by likely benefit of systemic therapy (e.g., chemotherapy). In this work, we present a new category of computational image-based biomarkers called quantitative tumor-associated vasculature (QuanTAV) features, and demonstrate their ability to predict response and survival across multiple cancer types, imaging modalities, and treatment regimens involving chemotherapy. EXPERIMENTAL DESIGN: We isolated tumor vasculature and extracted mathematical measurements of twistedness and organization from routine pretreatment radiology (CT or contrast-enhanced MRI) of a total of 558 patients, who received one of four first-line chemotherapy-based therapeutic intervention strategies for breast (n = 371) or non-small cell lung cancer (NSCLC, n = 187). RESULTS: Across four chemotherapy-based treatment strategies, classifiers of QuanTAV measurements significantly (P < 0.05) predicted response in held out testing cohorts alone (AUC = 0.63-0.71) and increased AUC by 0.06-0.12 when added to models of significant clinical variables alone. Similarly, we derived QuanTAV risk scores that were prognostic of recurrence-free survival in treatment cohorts who received surgery following chemotherapy for breast cancer [P = 0.0022; HR = 1.25; 95% confidence interval (CI), 1.08-1.44; concordance index (C-index) = 0.66] and chemoradiation for NSCLC (P = 0.039; HR = 1.28; 95% CI, 1.01-1.62; C-index = 0.66). From vessel-based risk scores, we further derived categorical QuanTAV high/low risk groups that were independently prognostic among all treatment groups, including patients with NSCLC who received chemotherapy only (P = 0.034; HR = 2.29; 95% CI, 1.07-4.94; C-index = 0.62). QuanTAV response and risk scores were independent of clinicopathologic risk factors and matched or exceeded models of clinical variables including posttreatment response. CONCLUSIONS: Across these domains, we observed an association of vascular morphology on CT and MRI-as captured by metrics of vessel curvature, torsion, and organizational heterogeneity-and treatment outcome. Our findings suggest the potential of shape and structure of the TAV in developing prognostic and predictive biomarkers for multiple cancers and different treatment strategies.

Radiomic Deformation and Textural Heterogeneity (R-DepTH) Descriptor to Characterize Tumor Field Effect: Application to Survival Prediction in Glioblastoma.

The concept of tumor field effect implies that cancer is a systemic disease with its impact way beyond the visible tumor confines. For instance, in Glioblastoma (GBM), an aggressive brain tumor, the increase in intracranial pressure due to tumor burden often leads to brain herniation and poor outcomes. Our work is based on the rationale that highly aggressive tumors tend to grow uncontrollably, leading to pronounced biomechanical tissue deformations in the normal parenchyma, which when combined with local morphological differences in the tumor confines on MRI scans, will comprehensively capture tumor field effect. Specifically, we present an integrated MRI-based descriptor, radiomic-Deformation and Textural Heterogeneity (r-DepTH). This descriptor comprises measurements of the subtle perturbations in tissue deformations throughout the surrounding normal parenchyma due to mass effect. This involves non-rigidly aligning the patients' MRI scans to a healthy atlas via diffeomorphic registration. The resulting inverse mapping is used to obtain the deformation field magnitudes in the normal parenchyma. These measurements are then combined with a 3D texture descriptor, Co-occurrence of Local Anisotropic Gradient Orientations (COLLAGE), which captures the morphological heterogeneity and infiltration within the tumor confines, on MRI scans. In this work, we extensively evaluated r-DepTH for survival risk-stratification on a total of 207 GBM cases from 3 different cohorts (Cohort 1 ( n1 = 53 ), Cohort 2 ( n2 = 75 ), and Cohort 3 ( n3 = 79 )), where each of these three cohorts was used as a training set for our model separately, and the other two cohorts were used for testing, independently, for each training experiment. When employing Cohort 1 for training, r-DepTH yielded Concordance indices (C-indices) of 0.7 and 0.65, hazard ratios (HR) and Confidence Intervals (CI) of 10 (6 - 19) and 5 (3 - 8) on Cohorts 2 and 3, respectively. Similarly, training on Cohort 2 yielded C-indices of 0.6 and 0.7, HR and CI of 1 (0.7 - 2) and 3 (2 - 5) on Cohorts 1 and 3, respectively. Finally, training on Cohort 3 yielded C-indices of 0.75 and 0.63, HR and CI of 24 (10 - 57) and 12 (6 - 21) on Cohorts 1 and 2, respectively. Our results show that r-DepTH descriptor may serve as a comprehensive and a robust MRI-based prognostic marker of disease aggressiveness and survival in solid tumors.

RADIomic Spatial TexturAl Descriptor (RADISTAT): Quantifying Spatial Organization of Imaging Heterogeneity Associated With Tumor Response to Treatment.

Localized disease heterogeneity on imaging extracted via radiomics approaches have recently been associated with disease prognosis and treatment response. Traditionally, radiomics analyses leverage texture operators to derive voxel- or region-wise feature values towards quantifying subtle variations in image appearance within a region-of-interest (ROI). With the goal of mining additional voxel-wise texture patterns from radiomic "expression maps", we introduce a new RADIomic Spatial TexturAl descripTor (RADISTAT). This was driven by the hypothesis that quantifying spatial organization of texture patterns within an ROI could allow for better capturing interactions between different tissue classes present in a given region; thus enabling more accurate characterization of disease or response phenotypes. RADISTAT involves: (a) robustly identifying sub-compartments of low, intermediate, and high radiomic expression (i.e. heterogeneity) in a feature map and (b) quantifying spatial organization of sub-compartments via graph interactions. RADISTAT was evaluated in two clinically challenging problems: (1) discriminating nodal/distant metastasis from metastasis-free rectal cancer patients on post-chemoradiation T2w MRI, and (2) distinguishing tumor progression from pseudo-progression in glioblastoma multiforme using post-chemoradiation T1w MRI. Across over 800 experiments, RADISTAT yielded a consistent discriminatory signature for tumor progression (GBM) and disease metastasis (RCa); where its sub-compartments were associated with pathologic tissue types (fibrosis or tumor, determined via fusion of MRI and pathology). In a multi-institutional setting for both clinical problems, RADISTAT resulted in higher classifier performance (11% improvement in AUC, on average) compared to radiomic descriptors. Furthermore, combining RADISTAT with radiomic descriptors resulted in significantly improved performance compared to using radiomic descriptors alone.

Optimizing the Peritumoral Region Size in Radiomics Analysis for Sentinel Lymph Node Status Prediction in Breast Cancer.

RATIONALE AND OBJECTIVES: Peritumoral features have been suggested to be useful in improving the prediction performance of radiomic models. The aim of this study is to systematically investigate the prediction performance improvement for sentinel lymph node (SLN) status in breast cancer from peritumoral features in radiomic analysis by exploring the effect of peritumoral region sizes. MATERIALS AND METHODS: This retrospective study was performed using dynamic contrast-enhanced MRI scans of 162 breast cancer patients. The effect of peritumoral features was evaluated in a radiomics pipeline for predicting SLN metastasis in breast cancer. Peritumoral regions were generated by dilating the tumor regions-of-interest (ROIs) manually annotated by two expert radiologists, with thicknesses of 2 mm, 4 mm, 6 mm, and 8 mm. The prediction models were established in the training set (∼67% of cases) using the radiomics pipeline with and without peritumoral features derived from different peritumoral thicknesses. The prediction performance was tested in an independent validation set (the remaining ∼33%). RESULTS: For this specific application, the accuracy in the validation set when using the two radiologists' ROIs could be both improved from 0.704 to 0.796 by incorporating peritumoral features. The choice of the peritumoral size could affect the level of improvement. CONCLUSION: This study systematically investigates the effect of peritumoral region sizes in radiomic analysis for prediction performance improvement. The choice of the peritumoral size is dependent on the ROI drawing and would affect the final prediction performance of radiomic models, suggesting that peritumoral features should be optimized in future radiomics studies.

Radiologically Defined Tumor-habitat Adjacency as a Prognostic Biomarker in Glioblastoma.

Intratumor heterogeneity in glioblastoma (GBM) has been linked to adverse clinical outcomes including poor survival and sub-optimal response to therapies. Different techniques, such as radiomics, have been used to characterize GBM phenotype. However, the spatial diversity and the interaction between different sub-regions within the tumor (habitats) and its microenvironment has been relatively unexplored. Besides, existing approaches have mainly focused on the radiomic analysis within globally defined regions without considering local heterogeneity. In this paper, we developed a 3D spatial co-localization descriptor based on the adjacency of "habitats" to quantify the diversity of physiologically similar sub-regions on multi-protocol magnetic resonance imaging. We demonstrated the utility of this spatial phenotype descriptor in predicting overall patient survival. Our experimental results on N=236 treatment-naïve MRI scans suggest that the co-localization features in conjunction with traditional clinical measures, such as age and tumor volume, outperform texture based radiomic features. The presented descriptor provides a tool for more complete characterization of intratumor heterogeneity in solid cancers.

Predicting Mechanical Ventilation and Mortality in COVID-19 Using Radiomics and Deep Learning on Chest Radiographs: A Multi-Institutional Study.

In this study, we aimed to predict mechanical ventilation requirement and mortality using computational modeling of chest radiographs (CXRs) for coronavirus disease 2019 (COVID-19) patients. This two-center, retrospective study analyzed 530 deidentified CXRs from 515 COVID-19 patients treated at Stony Brook University Hospital and Newark Beth Israel Medical Center between March and August 2020. Linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and random forest (RF) machine learning classifiers to predict mechanical ventilation requirement and mortality were trained and evaluated using radiomic features extracted from patients' CXRs. Deep learning (DL) approaches were also explored for the clinical outcome prediction task and a novel radiomic embedding framework was introduced. All results are compared against radiologist grading of CXRs (zone-wise expert severity scores). Radiomic classification models had mean area under the receiver operating characteristic curve (mAUCs) of 0.78 ± 0.05 (sensitivity = 0.72 ± 0.07, specificity = 0.72 ± 0.06) and 0.78 ± 0.06 (sensitivity = 0.70 ± 0.09, specificity = 0.73 ± 0.09), compared with expert scores mAUCs of 0.75 ± 0.02 (sensitivity = 0.67 ± 0.08, specificity = 0.69 ± 0.07) and 0.79 ± 0.05 (sensitivity = 0.69 ± 0.08, specificity = 0.76 ± 0.08) for mechanical ventilation requirement and mortality prediction, respectively. Classifiers using both expert severity scores and radiomic features for mechanical ventilation (mAUC = 0.79 ± 0.04, sensitivity = 0.71 ± 0.06, specificity = 0.71 ± 0.08) and mortality (mAUC = 0.83 ± 0.04, sensitivity = 0.79 ± 0.07, specificity = 0.74 ± 0.09) demonstrated improvement over either artificial intelligence or radiologist interpretation alone. Our results also suggest instances in which the inclusion of radiomic features in DL improves model predictions over DL alone. The models proposed in this study and the prognostic information they provide might aid physician decision making and efficient resource allocation during the COVID-19 pandemic.

A Novel Nodule Edge Sharpness Radiomic Biomarker Improves Performance of Lung-RADS for Distinguishing Adenocarcinomas from Granulomas on Non-Contrast CT Scans.

The aim of this study is to evaluate whether NIS radiomics can distinguish lung adenocarcinomas from granulomas on non-contrast CT scans, and also to improve the performance of Lung-RADS by reclassifying benign nodules that were initially assessed as suspicious. The screening or standard diagnostic non-contrast CT scans of 362 patients was divided into training (St, N = 145), validation (Sv, N = 145), and independent validation (Siv, N = 62) sets from different institutions. Nodules were identified and manually segmented on CT images by a radiologist. A series of 264 features relating to the edge sharpness transition from the inside to the outside of the nodule were extracted. The top 10 features were used to train a linear discriminant analysis (LDA) machine learning classifier on St. In conjunction with the LDA classifier, NIS radiomics classified nodules with an AUC of 0.82 ± 0.04, 0.77, and 0.71 respectively on St, Sv, and Siv. We evaluated the ability of the NIS classifier to determine the proportion of the patients in Sv that were identified initially as suspicious by Lung-RADS but were reclassified as benign by applying the NIS scores. The NIS classifier was able to correctly reclassify 46% of those lesions that were actually benign but deemed suspicious by Lung-RADS alone on Sv.

Radiomics and radiogenomics in gliomas: a contemporary update.

The natural history and treatment landscape of primary brain tumours are complicated by the varied tumour behaviour of primary or secondary gliomas (high-grade transformation of low-grade lesions), as well as the dilemmas with identification of radiation necrosis, tumour progression, and pseudoprogression on MRI. Radiomics and radiogenomics promise to offer precise diagnosis, predict prognosis, and assess tumour response to modern chemotherapy/immunotherapy and radiation therapy. This is achieved by a triumvirate of morphological, textural, and functional signatures, derived from a high-throughput extraction of quantitative voxel-level MR image metrics. However, the lack of standardisation of acquisition parameters and inconsistent methodology between working groups have made validations unreliable, hence multi-centre studies involving heterogenous study populations are warranted. We elucidate novel radiomic and radiogenomic workflow concepts and state-of-the-art descriptors in sub-visual MR image processing, with relevant literature on applications of such machine learning techniques in glioma management.

Radiomics-based assessment of ultra-widefield leakage patterns and vessel network architecture in the PERMEATE study: insights into treatment durability.

AIM: To evaluate the potential of radiomics-based ultra-widefield fluorescein angiography (UWFA)-derived imaging biomarkers in retinal vascular disease for predicting therapeutic durability of intravitreal aflibercept injection (IAI). METHODS: The Peripheral and Macular Retinal Vascular Perfusion and Leakage Dynamics in Diabetic Macular Edema and Retinal Venous Occlusions During Intravitreal Aflibercept Injection (IAI) Treatment for Retinal Edema (PERMEATE) study prospectively evaluated quantitative UWFA dynamics in diabetic macular oedema or macular oedema secondary to retinal vascular occlusion. 27 treatment-naïve eyes were treated with 2 mg IAI q4 weeks for the first 6 months, and then administered q8 weeks. Morphological and graph-based attributes were used to model the spatial distribution of leakage areas, while tortuosity measures were used to model the vessel network disorder. Eyes were grouped based on functional tolerance of the first 8-week treatment interval challenge. 'Non-rebounders' (N=15) maintained/improved best-corrected visual acuity (BCVA) following the 8-week challenge. 'Rebounders' (N=12) exhibited worsened BVCA. The image biomarkers were used with a machine learning classifier to preliminarily evaluate their ability to predict BCVA stability. RESULTS: Two new UWFA image-derived biomarkers were identified and extracted. The cross-validated area under the receiver operating characteristic curve (AUC) was 0.77±0.14 using baseline leakage distribution features and 0.73±0.10 for the UWFA baseline tortuosity measures. Additionally, the change in vascular tortuosity between month 4 and baseline yielded an AUC of 0.73±0.08. Three baseline clinical features of letter score, macular volume and central subfield thickness yielded a corresponding AUC of 0.42±0.09. CONCLUSIONS: Two computer-extracted UWFA radiomics-based descriptors were identified as potential biomarkers for predicting treatment durability and tolerance of longer treatment intervals. Conventional treatment parameters were not significantly different between these same groups.