Standardized reporting for Head CT Scans in patients suspected of traumatic brain injury (TBI): An international expert endeavor.

BACKGROUND AND PURPOSE: Traumatic brain injury (TBI) is a major source of health loss and disability worldwide. Accurate and timely diagnosis of TBI is critical for appropriate treatment and management of the condition. Neuroimaging plays a crucial role in the diagnosis and characterization of TBI. Computed tomography (CT) is the first-line diagnostic imaging modality typically utilized in patients with suspected acute mild, moderate and severe TBI. Radiology reports play a crucial role in the diagnostic process, providing critical information about the location and extent of brain injury, as well as factors that could prevent secondary injury. However, the complexity and variability of radiology reports can make it challenging for healthcare providers to extract the necessary information for diagnosis and treatment planning. METHODS/RESULTS/CONCLUSION: In this article, we report the efforts of an international group of TBI imaging experts to develop a clinical radiology report template for CT scans obtained in patients suspected of TBI and consisting of fourteen different subdivisions (CT technique, mechanism of injury or clinical history, presence of scalp injuries, fractures, potential vascular injuries, potential injuries involving the extra-axial spaces, brain parenchymal injuries, potential injuries involving the cerebrospinal fluid spaces and the ventricular system, mass effect, secondary injuries, prior or coexisting pathology).

Clinical, Cultural, Computational, and Regulatory Considerations to Deploy AI in Radiology: Perspectives of RSNA and MICCAI Experts.

The Radiological Society of North of America (RSNA) and the Medical Image Computing and Computer Assisted Intervention (MICCAI) Society have led a series of joint panels and seminars focused on the present impact and future directions of artificial intelligence (AI) in radiology. These conversations have collected viewpoints from multidisciplinary experts in radiology, medical imaging, and machine learning on the current clinical penetration of AI technology in radiology and how it is impacted by trust, reproducibility, explainability, and accountability. The collective points-both practical and philosophical-define the cultural changes for radiologists and AI scientists working together and describe the challenges ahead for AI technologies to meet broad approval. This article presents the perspectives of experts from MICCAI and RSNA on the clinical, cultural, computational, and regulatory considerations-coupled with recommended reading materials-essential to adopt AI technology successfully in radiology and, more generally, in clinical practice. The report emphasizes the importance of collaboration to improve clinical deployment, highlights the need to integrate clinical and medical imaging data, and introduces strategies to ensure smooth and incentivized integration. Keywords: Adults and Pediatrics, Computer Applications-General (Informatics), Diagnosis, Prognosis © RSNA, 2024.

Assessing the Performance of Artificial Intelligence Models: Insights from the American Society of Functional Neuroradiology Artificial Intelligence Competition.

BACKGROUND AND PURPOSE: Artificial intelligence models in radiology are frequently developed and validated using data sets from a single institution and are rarely tested on independent, external data sets, raising questions about their generalizability and applicability in clinical practice. The American Society of Functional Neuroradiology (ASFNR) organized a multicenter artificial intelligence competition to evaluate the proficiency of developed models in identifying various pathologies on NCCT, assessing age-based normality and estimating medical urgency. MATERIALS AND METHODS: In total, 1201 anonymized, full-head NCCT clinical scans from 5 institutions were pooled to form the data set. The data set encompassed studies with normal findings as well as those with pathologies, including acute ischemic stroke, intracranial hemorrhage, traumatic brain injury, and mass effect (detection of these, task 1). NCCTs were also assessed to determine if findings were consistent with expected brain changes for the patient's age (task 2: age-based normality assessment) and to identify any abnormalities requiring immediate medical attention (task 3: evaluation of findings for urgent intervention). Five neuroradiologists labeled each NCCT, with consensus interpretations serving as the ground truth. The competition was announced online, inviting academic institutions and companies. Independent central analysis assessed the performance of each model. Accuracy, sensitivity, specificity, positive and negative predictive values, and receiver operating characteristic (ROC) curves were generated for each artificial intelligence model, along with the area under the ROC curve. RESULTS: Four teams processed 1177 studies. The median age of patients was 62 years, with an interquartile range of 33 years. Nineteen teams from various academic institutions registered for the competition. Of these, 4 teams submitted their final results. No commercial entities participated in the competition. For task 1, areas under the ROC curve ranged from 0.49 to 0.59. For task 2, two teams completed the task with area under the ROC curve values of 0.57 and 0.52. For task 3, teams had little-to-no agreement with the ground truth. CONCLUSIONS: To assess the performance of artificial intelligence models in real-world clinical scenarios, we analyzed their performance in the ASFNR Artificial Intelligence Competition. The first ASFNR Competition underscored the gap between expectation and reality; and the models largely fell short in their assessments. As the integration of artificial intelligence tools into clinical workflows increases, neuroradiologists must carefully recognize the capabilities, constraints, and consistency of these technologies. Before institutions adopt these algorithms, thorough validation is essential to ensure acceptable levels of performance in clinical settings.

Lessons Learned in Building Expertly Annotated Multi-Institution Datasets and Hosting the RSNA AI Challenges.

The Radiological Society of North America (RSNA) has held artificial intelligence competitions to tackle real-world medical imaging problems at least annually since 2017. This article examines the challenges and processes involved in organizing these competitions, with a specific emphasis on the creation and curation of high-quality datasets. The collection of diverse and representative medical imaging data involves dealing with issues of patient privacy and data security. Furthermore, ensuring quality and consistency in data, which includes expert labeling and accounting for various patient and imaging characteristics, necessitates substantial planning and resources. Overcoming these obstacles requires meticulous project management and adherence to strict timelines. The article also highlights the potential of crowdsourced annotation to progress medical imaging research. Through the RSNA competitions, an effective global engagement has been realized, resulting in innovative solutions to complex medical imaging problems, thus potentially transforming health care by enhancing diagnostic accuracy and patient outcomes. Keywords: Use of AI in Education, Artificial Intelligence © RSNA, 2024.

Proceedings From the 2022 ACR-RSNA Workshop on Safety, Effectiveness, Reliability, and Transparency in AI.

Despite the surge in artificial intelligence (AI) development for health care applications, particularly for medical imaging applications, there has been limited adoption of such AI tools into clinical practice. During a 1-day workshop in November 2022, co-organized by the ACR and the RSNA, participants outlined experiences and problems with implementing AI in clinical practice, defined the needs of various stakeholders in the AI ecosystem, and elicited potential solutions and strategies related to the safety, effectiveness, reliability, and transparency of AI algorithms. Participants included radiologists from academic and community radiology practices, informatics leaders responsible for AI implementation, regulatory agency employees, and specialty society representatives. The major themes that emerged fell into two categories: (1) AI product development and (2) implementation of AI-based applications in clinical practice. In particular, participants highlighted key aspects of AI product development to include clear clinical task definitions; well-curated data from diverse geographic, economic, and health care settings; standards and mechanisms to monitor model reliability; and transparency regarding model performance, both in controlled and real-world settings. For implementation, participants emphasized the need for strong institutional governance; systematic evaluation, selection, and validation methods conducted by local teams; seamless integration into the clinical workflow; performance monitoring and support by local teams; performance monitoring by external entities; and alignment of incentives through credentialing and reimbursement. Participants predicted that clinical implementation of AI in radiology will continue to be limited until the safety, effectiveness, reliability, and transparency of such tools are more fully addressed.

Data Liberation and Crowdsourcing in Medical Research: The Intersection of Collective and Artificial Intelligence.

In spite of an exponential increase in the volume of medical data produced globally, much of these data are inaccessible to those who might best use them to develop improved health care solutions through the application of advanced analytics such as artificial intelligence. Data liberation and crowdsourcing represent two distinct but interrelated approaches to bridging existing data silos and accelerating the pace of innovation internationally. In this article, we examine these concepts in the context of medical artificial intelligence research, summarizing their potential benefits, identifying potential pitfalls, and ultimately making a case for their expanded use going forward. A practical example of a crowdsourced competition using an international medical imaging dataset is provided. Keywords: Artificial Intelligence, Data Liberation, Crowdsourcing © RSNA, 2023.

Performance of the Winning Algorithms of the RSNA 2022 Cervical Spine Fracture Detection Challenge.

Purpose To evaluate and report the performance of the winning algorithms of the Radiological Society of North America Cervical Spine Fracture AI Challenge. Materials and Methods The competition was open to the public on Kaggle from July 28 to October 27, 2022. A sample of 3112 CT scans with and without cervical spine fractures (CSFx) were assembled from multiple sites (12 institutions across six continents) and prepared for the competition. The test set had 1093 scans (private test set: n = 789; mean age, 53.40 years ± 22.86 [SD]; 509 males; public test set: n = 304; mean age, 52.51 years ± 20.73; 189 males) and 847 fractures. The eight top-performing artificial intelligence (AI) algorithms were retrospectively evaluated, and the area under the receiver operating characteristic curve (AUC) value, F1 score, sensitivity, and specificity were calculated. Results A total of 1108 contestants composing 883 teams worldwide participated in the competition. The top eight AI models showed high performance, with a mean AUC value of 0.96 (95% CI: 0.95, 0.96), mean F1 score of 90% (95% CI: 90%, 91%), mean sensitivity of 88% (95% Cl: 86%, 90%), and mean specificity of 94% (95% CI: 93%, 96%). The highest values reported for previous models were an AUC of 0.85, F1 score of 81%, sensitivity of 76%, and specificity of 97%. Conclusion The competition successfully facilitated the development of AI models that could detect and localize CSFx on CT scans with high performance outcomes, which appear to exceed known values of previously reported models. Further study is needed to evaluate the generalizability of these models in a clinical environment. Keywords: Cervical Spine, Fracture Detection, Machine Learning, Artificial Intelligence Algorithms, CT, Head/Neck Supplemental material is available for this article. © RSNA, 2024.

Harmonization of multi-site diffusion tensor imaging data for cervical and thoracic spinal cord at 1.5 T and 3 T using longitudinal ComBat.

MRI scanner hardware, field strengths, and sequence parameters are major variables in diffusion studies of the spinal cord. Reliability between scanners is not well known, particularly for the thoracic cord. DTI data was collected for the entire cervical and thoracic spinal cord in thirty healthy adult subjects with different MR vendors and field strengths. DTI metrics were extracted and averaged for all slices within each vertebral level. Metrics were examined for variability and then harmonized using longitudinal ComBat (longComBat). Four scanners were used: Siemens 3 T Prisma, Siemens 1.5 T Avanto, Philips 3 T Ingenia, Philips 1.5 T Achieva. Average full cord diffusion values/standard deviation for all subjects and scanners were FA: 0.63, σ = 0.10, MD: 1.11, σ = 0.12 × 10-3 mm2/s, AD: 1.98, σ = 0.55 × 10-3 mm2/s, RD: 0.67, σ = 0.31 × 10-3 mm2/s. FA metrics averaged for all subjects by level were relatively consistent across scanners, but large variability was found in diffusivity measures. Coefficients of variation were lowest in the cervical region, and relatively lower for FA than diffusivity measures. Harmonized metrics showed greatly improved agreement between scanners. Variability in DTI of the spinal cord arises from scanner hardware differences, pulse sequence differences, physiological motion, and subject compliance. The use of longComBat resulted in large improvement in agreement of all DTI metrics between scanners. This study shows the importance of harmonization of diffusion data in the spinal cord and potential for longitudinal and multisite clinical research and clinical trials.

The RSNA Cervical Spine Fracture CT Dataset.

This dataset is composed of cervical spine CT images with annotations related to fractures; it is available at https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/.

Radiomic signatures of meningiomas using the Ki-67 proliferation index as a prognostic marker of clinical outcomes.

OBJECTIVE: The clinical behavior of meningiomas is not entirely captured by its designated WHO grade, therefore other factors must be elucidated that portend increased tumor aggressiveness and associated risk of recurrence. In this study, the authors identify multiparametric MRI radiomic signatures of meningiomas using Ki-67 as a prognostic marker of clinical outcomes independent of WHO grade. METHODS: A retrospective analysis was conducted of all resected meningiomas between 2012 and 2018. Preoperative MR images were used for high-throughput radiomic feature extraction and subsequently used to develop a machine learning algorithm to stratify meningiomas based on Ki-67 indices < 5% and ≥ 5%, independent of WHO grade. Progression-free survival (PFS) was assessed based on machine learning prediction of Ki-67 strata and compared with outcomes based on histopathological Ki-67. RESULTS: Three hundred forty-three meningiomas were included: 291 with WHO grade I, 43 with grade II, and 9 with grade III. The overall rate of recurrence was 19.8% (15.1% in grade I, 44.2% in grade II, and 77.8% in grade III) over a median follow-up of 28.5 months. Grade II and III tumors had higher Ki-67 indices than grade I tumors, albeit tumor and peritumoral edema volumes had considerable variation independent of meningioma WHO grade. Forty-six high-performing radiomic features (1 morphological, 7 intensity-based, and 38 textural) were identified and used to build a support vector machine model to stratify tumors based on a Ki-67 cutoff of 5%, with resultant areas under the curve of 0.83 (95% CI 0.78-0.89) and 0.84 (95% CI 0.75-0.94) achieved for the discovery (n = 257) and validation (n = 86) data sets, respectively. Comparison of histopathological Ki-67 versus machine learning-predicted Ki-67 showed excellent performance (overall accuracy > 80%), with classification of grade I meningiomas exhibiting the greatest accuracy. Prediction of Ki-67 by machine learning classifier revealed shorter PFS for meningiomas with Ki-67 indices ≥ 5% compared with tumors with Ki-67 < 5% (p < 0.0001, log-rank test), which corroborates divergent patient outcomes observed using histopathological Ki-67. CONCLUSIONS: The Ki-67 proliferation index may serve as a surrogate marker of increased meningioma aggressiveness independent of WHO grade. Machine learning using radiomic feature analysis may be used for the preoperative prediction of meningioma Ki-67, which provides enhanced analytical insights to help improve diagnostic classification and guide patient-specific treatment strategies.

Association of partial T2-FLAIR mismatch sign and isocitrate dehydrogenase mutation in WHO grade 4 gliomas: results from the ReSPOND consortium.

PURPOSE: While the T2-FLAIR mismatch sign is highly specific for isocitrate dehydrogenase (IDH)-mutant, 1p/19q-noncodeleted astrocytomas among lower-grade gliomas, its utility in WHO grade 4 gliomas is not well-studied. We derived the partial T2-FLAIR mismatch sign as an imaging biomarker for IDH mutation in WHO grade 4 gliomas. METHODS: Preoperative MRI scans of adult WHO grade 4 glioma patients (n = 2165) from the multi-institutional ReSPOND (Radiomics Signatures for PrecisiON Diagnostics) consortium were analyzed. Diagnostic performance of the partial T2-FLAIR mismatch sign was evaluated. Subset analyses were performed to assess associations of imaging markers with overall survival (OS). RESULTS: One hundred twenty-one (5.6%) of 2165 grade 4 gliomas were IDH-mutant. Partial T2-FLAIR mismatch was present in 40 (1.8%) cases, 32 of which were IDH-mutant, yielding 26.4% sensitivity, 99.6% specificity, 80.0% positive predictive value, and 95.8% negative predictive value. Multivariate logistic regression demonstrated IDH mutation was significantly associated with partial T2-FLAIR mismatch (odds ratio [OR] 5.715, 95% CI [1.896, 17.221], p = 0.002), younger age (OR 0.911 [0.895, 0.927], p < 0.001), tumor centered in frontal lobe (OR 3.842, [2.361, 6.251], p < 0.001), absence of multicentricity (OR 0.173, [0.049, 0.612], p = 0.007), and presence of cystic (OR 6.596, [3.023, 14.391], p < 0.001) or non-enhancing solid components (OR 6.069, [3.371, 10.928], p < 0.001). Multivariate Cox analysis demonstrated cystic components (p = 0.024) and non-enhancing solid components (p = 0.003) were associated with longer OS, while older age (p < 0.001), frontal lobe center (p = 0.008), multifocality (p < 0.001), and multicentricity (p < 0.001) were associated with shorter OS. CONCLUSION: Partial T2-FLAIR mismatch sign is highly specific for IDH mutation in WHO grade 4 gliomas.

Opportunistic detection of type 2 diabetes using deep learning from frontal chest radiographs.

Deep learning (DL) models can harness electronic health records (EHRs) to predict diseases and extract radiologic findings for diagnosis. With ambulatory chest radiographs (CXRs) frequently ordered, we investigated detecting type 2 diabetes (T2D) by combining radiographic and EHR data using a DL model. Our model, developed from 271,065 CXRs and 160,244 patients, was tested on a prospective dataset of 9,943 CXRs. Here we show the model effectively detected T2D with a ROC AUC of 0.84 and a 16% prevalence. The algorithm flagged 1,381 cases (14%) as suspicious for T2D. External validation at a distinct institution yielded a ROC AUC of 0.77, with 5% of patients subsequently diagnosed with T2D. Explainable AI techniques revealed correlations between specific adiposity measures and high predictivity, suggesting CXRs' potential for enhanced T2D screening.