Predicting peritumoral glioblastoma infiltration and subsequent recurrence using deep-learning-based analysis of multi-parametric magnetic resonance imaging.

Purpose: Glioblastoma (GBM) is the most common and aggressive primary adult brain tumor. The standard treatment approach is surgical resection to target the enhancing tumor mass, followed by adjuvant chemoradiotherapy. However, malignant cells often extend beyond the enhancing tumor boundaries and infiltrate the peritumoral edema. Traditional supervised machine learning techniques hold potential in predicting tumor infiltration extent but are hindered by the extensive resources needed to generate expertly delineated regions of interest (ROIs) for training models on tissue most and least likely to be infiltrated. Approach: We developed a method combining expert knowledge and training-based data augmentation to automatically generate numerous training examples, enhancing the accuracy of our model for predicting tumor infiltration through predictive maps. Such maps can be used for targeted supra-total surgical resection and other therapies that might benefit from intensive yet well-targeted treatment of infiltrated tissue. We apply our method to preoperative multi-parametric magnetic resonance imaging (mpMRI) scans from a subset of 229 patients of a multi-institutional consortium (Radiomics Signatures for Precision Diagnostics) and test the model on subsequent scans with pathology-proven recurrence. Results: Leave-one-site-out cross-validation was used to train and evaluate the tumor infiltration prediction model using initial pre-surgical scans, comparing the generated prediction maps with follow-up mpMRI scans confirming recurrence through post-resection tissue analysis. Performance was measured by voxel-wised odds ratios (ORs) across six institutions: University of Pennsylvania (OR: 9.97), Ohio State University (OR: 14.03), Case Western Reserve University (OR: 8.13), New York University (OR: 16.43), Thomas Jefferson University (OR: 8.22), and Rio Hortega (OR: 19.48). Conclusions: The proposed model demonstrates that mpMRI analysis using deep learning can predict infiltration in the peri-tumoral brain region for GBM patients without needing to train a model using expert ROI drawings. Results for each institution demonstrate the model's generalizability and reproducibility.

Assessing the Performance of Models from the 2022 RSNA Cervical Spine Fracture Detection Competition at a Level I Trauma Center.

"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Purpose To evaluate the performance of the top models from the RSNA 2022 Cervical Spine Fracture Detection challenge on a clinical test dataset of both noncontrast and contrast-enhanced CT scans acquired at a level I trauma center. Materials and Methods Seven top-performing models in the RSNA 2022 Cervical Spine Fracture Detection challenge were retrospectively evaluated on a clinical test set of 1,828 CT scans (1,829 series: 130 positive for fracture, 1,699 negative for fracture; 1,308 noncontrast, 521 contrast-enhanced) from 1,779 patients (mean age, 55.8 ± 22.1 years; 1,154 male). Scans were acquired without exclusion criteria over one year (January to December 2022) from the emergency department of a neurosurgical and level I trauma center. Model performance was assessed using area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. False positive and false negative cases were further analyzed by a neuroradiologist. Results Although all 7 models showed decreased performance on the clinical test set compared with the challenge dataset, the models maintained high performances. On noncontrast CT scans, the models achieved a mean AUC of 0.89 (range: 0.81-0.92), sensitivity of 67.0% (range: 30.9%-80.0%), and specificity of 92.9% (range: 82.1%-99.0%). On contrast-enhanced CT scans, the models had a mean AUC of 0.88 (range: 0.76-0.94), sensitivity of 81.9% (range: 42.7%-100.0%), and specificity of 72.1% (range: 16.4%-92.8%). The models identified 10 fractures missed by radiologists. False-positives were more common in contrast-enhanced scans and observed in patients with degenerative changes on noncontrast scans, while false-negatives were often associated with degenerative changes and osteopenia. Conclusion The winning models from the 2022 RSNA AI Challenge demonstrated a high performance for cervical spine fracture detection on a clinical test dataset, warranting further evaluation for their use as clinical support tools. ©RSNA, 2024.

Standardized reporting for Head CT Scans in patients suspected of traumatic brain injury (TBI): An international expert endeavor.

BACKGROUND AND PURPOSE: Traumatic brain injury (TBI) is a major source of health loss and disability worldwide. Accurate and timely diagnosis of TBI is critical for appropriate treatment and management of the condition. Neuroimaging plays a crucial role in the diagnosis and characterization of TBI. Computed tomography (CT) is the first-line diagnostic imaging modality typically utilized in patients with suspected acute mild, moderate and severe TBI. Radiology reports play a crucial role in the diagnostic process, providing critical information about the location and extent of brain injury, as well as factors that could prevent secondary injury. However, the complexity and variability of radiology reports can make it challenging for healthcare providers to extract the necessary information for diagnosis and treatment planning. METHODS/RESULTS/CONCLUSION: In this article, we report the efforts of an international group of TBI imaging experts to develop a clinical radiology report template for CT scans obtained in patients suspected of TBI and consisting of fourteen different subdivisions (CT technique, mechanism of injury or clinical history, presence of scalp injuries, fractures, potential vascular injuries, potential injuries involving the extra-axial spaces, brain parenchymal injuries, potential injuries involving the cerebrospinal fluid spaces and the ventricular system, mass effect, secondary injuries, prior or coexisting pathology).

Clinical, Cultural, Computational, and Regulatory Considerations to Deploy AI in Radiology: Perspectives of RSNA and MICCAI Experts.

The Radiological Society of North of America (RSNA) and the Medical Image Computing and Computer Assisted Intervention (MICCAI) Society have led a series of joint panels and seminars focused on the present impact and future directions of artificial intelligence (AI) in radiology. These conversations have collected viewpoints from multidisciplinary experts in radiology, medical imaging, and machine learning on the current clinical penetration of AI technology in radiology and how it is impacted by trust, reproducibility, explainability, and accountability. The collective points-both practical and philosophical-define the cultural changes for radiologists and AI scientists working together and describe the challenges ahead for AI technologies to meet broad approval. This article presents the perspectives of experts from MICCAI and RSNA on the clinical, cultural, computational, and regulatory considerations-coupled with recommended reading materials-essential to adopt AI technology successfully in radiology and, more generally, in clinical practice. The report emphasizes the importance of collaboration to improve clinical deployment, highlights the need to integrate clinical and medical imaging data, and introduces strategies to ensure smooth and incentivized integration. Keywords: Adults and Pediatrics, Computer Applications-General (Informatics), Diagnosis, Prognosis © RSNA, 2024.

Correlation of Computed Tomography Angiography to Digital Subtraction Angiography in Carotid Stenosis with Real-World Assessment of Overestimation of Carotid Stenosis on Computed Tomography Angiography.

OBJECTIVE: Computed tomography angiography (CTA) is a well-established diagnostic modality for carotid stenosis. However, false-positive CTA results may expose patients to unnecessary procedural complications in cases where surgical intervention is not warranted. We aim to assess the correlation of CTA to digital subtraction angiography (DSA) in carotid stenosis and characterize patients who were referred for intervention based on CTA and did not require it based on DSA. METHODS: We retrospectively reviewed 186 patients who underwent carotid angioplasty and stenting following preprocedural CTA at our institution from April 2017 to December 2022. RESULTS: Twenty-one of 186 patients (11.2%) were found to have <50% carotid stenosis on DSA (discordant group). Severe plaque calcification on CTA was associated with a discordant degree of stenosis on DSA (LR+=7.4). Among 186 patients, agreement between the percentage of stenosis from CTA and DSA was weak-moderate (r2=0.27, P<0.01). Among concordant pairs, we found moderate-strong agreement between CTA and DSA (adj r2=0.37) (P < 0.0001). Of 186 patients, 127 patients had CTA stenosis of ≥70%, and 59 had CTA of 50%-69%. Correlation between CTA and DSA in severe CTA stenosis was weak (r2=0.11, P<0.01). CONCLUSIONS: In patients with stenosis found on CTA, over 88% also had stenosis on DSA, with this positive predictive value in line with previous studies. The percent-stenosis value from CTA and DSA was weakly correlated but does not affect the overall clinical judgement of stenosis. Severe calcification found on CTA may potentially indicate nonstenosis on DSA.

The impact of mechanical thrombectomy on the blood-brain barrier in patients with acute ischemic stroke: A non-contrast MR imaging study using DP-pCASL and NODDI.

BACKGROUND AND PURPOSE: While mechanical thrombectomy (MT) achieves restoration of cerebral blood flow to the area at risk in patients with acute ischemic stroke (AIS), the influx of blood flow may exacerbate the blood-brain barrier (BBB) disruption and extravasation across the BBB, and it therefore remains unclear how reperfusion impacts the blood-brain barrier integrity. In this study, we use diffusion-prepared pseudocontinuous ASL (DP-pCASL) and Neurite Orientation Dispersion and Density Imaging (NODDI) sequence to measure the water exchange rate (kw) in patients who underwent either MT or medical management and determine its impact on the brain tissue microstructure in order to elucidate the impact of MT on BBB complex integrity. MATERIALS AND METHODS: We prospectively enrolled 21 patients with AIS treated at our institution from 10/2021 to 6/2023 who underwent MR imaging at a 3.0-Tesla scanner. Patients underwent DP-pCASl and NODDI imaging in addition to the standard stroke protocol which generated cerebral blood flow (CBF), arterial transit time (ATT), water exchange rate (kw), orientation dispersion index (ODI), intracellular volume fraction (ICVF), and free water fraction (FWF) parametric maps. RESULTS: Of the 21 patients, 11 underwent MT and 10 were treated non-operatively. The average age and NIHSS for the MT cohort and non-MT cohorts were 69.3 ± 16.6 years old and 15.0 (12.0-20.0), and 70.2 ± 10.7 (p = 0.882) and 6.0 (3.8-9.0, p = 0.003) respectively. The average CBF, ATT, and kw in the infarcted territory of the MT cohort were 38.2 (18.4-59.6), 1347.6 (1182.5-1842.3), and 107.8 (79.2-140.1) respectively. The average CBF, ATT, and kw in the stroke ROI were 16.0 (8.8-36.6, p = 0.036), 1090.8 (937.1-1258.9, p = 0.013), 89.7 (68.0-122.7, p = 0.314) respectively. Linear regression analysis showed increasing CBF (p = 0.008) and undergoing mechanical thrombectomy (p = 0.048) were significant predictors of increased kw. CONCLUSION: Using our multimodal non-contrast MRI protocol, we demonstrate that increased CBF and mechanical thrombectomy increased kw, suggesting a better functioning BBB complex. Higher kw suggests less disruption of the BBB complex in the MT cohort.

Evaluating GPT-V4 (GPT-4 with Vision) on Detection of Radiologic Findings on Chest Radiographs.

Background Generating radiologic findings from chest radiographs is pivotal in medical image analysis. The emergence of OpenAI's generative pretrained transformer, GPT-4 with vision (GPT-4V), has opened new perspectives on the potential for automated image-text pair generation. However, the application of GPT-4V to real-world chest radiography is yet to be thoroughly examined. Purpose To investigate the capability of GPT-4V to generate radiologic findings from real-world chest radiographs. Materials and Methods In this retrospective study, 100 chest radiographs with free-text radiology reports were annotated by a cohort of radiologists, two attending physicians and three residents, to establish a reference standard. Of 100 chest radiographs, 50 were randomly selected from the National Institutes of Health (NIH) chest radiographic data set, and 50 were randomly selected from the Medical Imaging and Data Resource Center (MIDRC). The performance of GPT-4V at detecting imaging findings from each chest radiograph was assessed in the zero-shot setting (where it operates without prior examples) and few-shot setting (where it operates with two examples). Its outcomes were compared with the reference standard with regards to clinical conditions and their corresponding codes in the International Statistical Classification of Diseases, Tenth Revision (ICD-10), including the anatomic location (hereafter, laterality). Results In the zero-shot setting, in the task of detecting ICD-10 codes alone, GPT-4V attained an average positive predictive value (PPV) of 12.3%, average true-positive rate (TPR) of 5.8%, and average F1 score of 7.3% on the NIH data set, and an average PPV of 25.0%, average TPR of 16.8%, and average F1 score of 18.2% on the MIDRC data set. When both the ICD-10 codes and their corresponding laterality were considered, GPT-4V produced an average PPV of 7.8%, average TPR of 3.5%, and average F1 score of 4.5% on the NIH data set, and an average PPV of 10.9%, average TPR of 4.9%, and average F1 score of 6.4% on the MIDRC data set. With few-shot learning, GPT-4V showed improved performance on both data sets. When contrasting zero-shot and few-shot learning, there were improved average TPRs and F1 scores in the few-shot setting, but there was not a substantial increase in the average PPV. Conclusion Although GPT-4V has shown promise in understanding natural images, it had limited effectiveness in interpreting real-world chest radiographs. © RSNA, 2024 Supplemental material is available for this article.

Assessing the Performance of Artificial Intelligence Models: Insights from the American Society of Functional Neuroradiology Artificial Intelligence Competition.

BACKGROUND AND PURPOSE: Artificial intelligence models in radiology are frequently developed and validated using data sets from a single institution and are rarely tested on independent, external data sets, raising questions about their generalizability and applicability in clinical practice. The American Society of Functional Neuroradiology (ASFNR) organized a multicenter artificial intelligence competition to evaluate the proficiency of developed models in identifying various pathologies on NCCT, assessing age-based normality and estimating medical urgency. MATERIALS AND METHODS: In total, 1201 anonymized, full-head NCCT clinical scans from 5 institutions were pooled to form the data set. The data set encompassed studies with normal findings as well as those with pathologies, including acute ischemic stroke, intracranial hemorrhage, traumatic brain injury, and mass effect (detection of these, task 1). NCCTs were also assessed to determine if findings were consistent with expected brain changes for the patient's age (task 2: age-based normality assessment) and to identify any abnormalities requiring immediate medical attention (task 3: evaluation of findings for urgent intervention). Five neuroradiologists labeled each NCCT, with consensus interpretations serving as the ground truth. The competition was announced online, inviting academic institutions and companies. Independent central analysis assessed the performance of each model. Accuracy, sensitivity, specificity, positive and negative predictive values, and receiver operating characteristic (ROC) curves were generated for each artificial intelligence model, along with the area under the ROC curve. RESULTS: Four teams processed 1177 studies. The median age of patients was 62 years, with an interquartile range of 33 years. Nineteen teams from various academic institutions registered for the competition. Of these, 4 teams submitted their final results. No commercial entities participated in the competition. For task 1, areas under the ROC curve ranged from 0.49 to 0.59. For task 2, two teams completed the task with area under the ROC curve values of 0.57 and 0.52. For task 3, teams had little-to-no agreement with the ground truth. CONCLUSIONS: To assess the performance of artificial intelligence models in real-world clinical scenarios, we analyzed their performance in the ASFNR Artificial Intelligence Competition. The first ASFNR Competition underscored the gap between expectation and reality; and the models largely fell short in their assessments. As the integration of artificial intelligence tools into clinical workflows increases, neuroradiologists must carefully recognize the capabilities, constraints, and consistency of these technologies. Before institutions adopt these algorithms, thorough validation is essential to ensure acceptable levels of performance in clinical settings.

Lessons Learned in Building Expertly Annotated Multi-Institution Datasets and Hosting the RSNA AI Challenges.

The Radiological Society of North America (RSNA) has held artificial intelligence competitions to tackle real-world medical imaging problems at least annually since 2017. This article examines the challenges and processes involved in organizing these competitions, with a specific emphasis on the creation and curation of high-quality datasets. The collection of diverse and representative medical imaging data involves dealing with issues of patient privacy and data security. Furthermore, ensuring quality and consistency in data, which includes expert labeling and accounting for various patient and imaging characteristics, necessitates substantial planning and resources. Overcoming these obstacles requires meticulous project management and adherence to strict timelines. The article also highlights the potential of crowdsourced annotation to progress medical imaging research. Through the RSNA competitions, an effective global engagement has been realized, resulting in innovative solutions to complex medical imaging problems, thus potentially transforming health care by enhancing diagnostic accuracy and patient outcomes. Keywords: Use of AI in Education, Artificial Intelligence © RSNA, 2024.

Proceedings From the 2022 ACR-RSNA Workshop on Safety, Effectiveness, Reliability, and Transparency in AI.

Despite the surge in artificial intelligence (AI) development for health care applications, particularly for medical imaging applications, there has been limited adoption of such AI tools into clinical practice. During a 1-day workshop in November 2022, co-organized by the ACR and the RSNA, participants outlined experiences and problems with implementing AI in clinical practice, defined the needs of various stakeholders in the AI ecosystem, and elicited potential solutions and strategies related to the safety, effectiveness, reliability, and transparency of AI algorithms. Participants included radiologists from academic and community radiology practices, informatics leaders responsible for AI implementation, regulatory agency employees, and specialty society representatives. The major themes that emerged fell into two categories: (1) AI product development and (2) implementation of AI-based applications in clinical practice. In particular, participants highlighted key aspects of AI product development to include clear clinical task definitions; well-curated data from diverse geographic, economic, and health care settings; standards and mechanisms to monitor model reliability; and transparency regarding model performance, both in controlled and real-world settings. For implementation, participants emphasized the need for strong institutional governance; systematic evaluation, selection, and validation methods conducted by local teams; seamless integration into the clinical workflow; performance monitoring and support by local teams; performance monitoring by external entities; and alignment of incentives through credentialing and reimbursement. Participants predicted that clinical implementation of AI in radiology will continue to be limited until the safety, effectiveness, reliability, and transparency of such tools are more fully addressed.

Performance of the Winning Algorithms of the RSNA 2022 Cervical Spine Fracture Detection Challenge.

Purpose To evaluate and report the performance of the winning algorithms of the Radiological Society of North America Cervical Spine Fracture AI Challenge. Materials and Methods The competition was open to the public on Kaggle from July 28 to October 27, 2022. A sample of 3112 CT scans with and without cervical spine fractures (CSFx) were assembled from multiple sites (12 institutions across six continents) and prepared for the competition. The test set had 1093 scans (private test set: n = 789; mean age, 53.40 years ± 22.86 [SD]; 509 males; public test set: n = 304; mean age, 52.51 years ± 20.73; 189 males) and 847 fractures. The eight top-performing artificial intelligence (AI) algorithms were retrospectively evaluated, and the area under the receiver operating characteristic curve (AUC) value, F1 score, sensitivity, and specificity were calculated. Results A total of 1108 contestants composing 883 teams worldwide participated in the competition. The top eight AI models showed high performance, with a mean AUC value of 0.96 (95% CI: 0.95, 0.96), mean F1 score of 90% (95% CI: 90%, 91%), mean sensitivity of 88% (95% Cl: 86%, 90%), and mean specificity of 94% (95% CI: 93%, 96%). The highest values reported for previous models were an AUC of 0.85, F1 score of 81%, sensitivity of 76%, and specificity of 97%. Conclusion The competition successfully facilitated the development of AI models that could detect and localize CSFx on CT scans with high performance outcomes, which appear to exceed known values of previously reported models. Further study is needed to evaluate the generalizability of these models in a clinical environment. Keywords: Cervical Spine, Fracture Detection, Machine Learning, Artificial Intelligence Algorithms, CT, Head/Neck Supplemental material is available for this article. © RSNA, 2024.

Data Liberation and Crowdsourcing in Medical Research: The Intersection of Collective and Artificial Intelligence.

In spite of an exponential increase in the volume of medical data produced globally, much of these data are inaccessible to those who might best use them to develop improved health care solutions through the application of advanced analytics such as artificial intelligence. Data liberation and crowdsourcing represent two distinct but interrelated approaches to bridging existing data silos and accelerating the pace of innovation internationally. In this article, we examine these concepts in the context of medical artificial intelligence research, summarizing their potential benefits, identifying potential pitfalls, and ultimately making a case for their expanded use going forward. A practical example of a crowdsourced competition using an international medical imaging dataset is provided. Keywords: Artificial Intelligence, Data Liberation, Crowdsourcing © RSNA, 2023.

Harmonization of multi-site diffusion tensor imaging data for cervical and thoracic spinal cord at 1.5 T and 3 T using longitudinal ComBat.

MRI scanner hardware, field strengths, and sequence parameters are major variables in diffusion studies of the spinal cord. Reliability between scanners is not well known, particularly for the thoracic cord. DTI data was collected for the entire cervical and thoracic spinal cord in thirty healthy adult subjects with different MR vendors and field strengths. DTI metrics were extracted and averaged for all slices within each vertebral level. Metrics were examined for variability and then harmonized using longitudinal ComBat (longComBat). Four scanners were used: Siemens 3 T Prisma, Siemens 1.5 T Avanto, Philips 3 T Ingenia, Philips 1.5 T Achieva. Average full cord diffusion values/standard deviation for all subjects and scanners were FA: 0.63, σ = 0.10, MD: 1.11, σ = 0.12 × 10-3 mm2/s, AD: 1.98, σ = 0.55 × 10-3 mm2/s, RD: 0.67, σ = 0.31 × 10-3 mm2/s. FA metrics averaged for all subjects by level were relatively consistent across scanners, but large variability was found in diffusivity measures. Coefficients of variation were lowest in the cervical region, and relatively lower for FA than diffusivity measures. Harmonized metrics showed greatly improved agreement between scanners. Variability in DTI of the spinal cord arises from scanner hardware differences, pulse sequence differences, physiological motion, and subject compliance. The use of longComBat resulted in large improvement in agreement of all DTI metrics between scanners. This study shows the importance of harmonization of diffusion data in the spinal cord and potential for longitudinal and multisite clinical research and clinical trials.

The RSNA Cervical Spine Fracture CT Dataset.

This dataset is composed of cervical spine CT images with annotations related to fractures; it is available at https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/.