Genetic drivers of heterogeneity in type 2 diabetes pathophysiology.

Type 2 diabetes (T2D) is a heterogeneous disease that develops through diverse pathophysiological processes1,2 and molecular mechanisms that are often specific to cell type3,4. Here, to characterize the genetic contribution to these processes across ancestry groups, we aggregate genome-wide association study data from 2,535,601 individuals (39.7% not of European ancestry), including 428,452 cases of T2D. We identify 1,289 independent association signals at genome-wide significance (P < 5 × 10-8) that map to 611 loci, of which 145 loci are, to our knowledge, previously unreported. We define eight non-overlapping clusters of T2D signals that are characterized by distinct profiles of cardiometabolic trait associations. These clusters are differentially enriched for cell-type-specific regions of open chromatin, including pancreatic islets, adipocytes, endothelial cells and enteroendocrine cells. We build cluster-specific partitioned polygenic scores5 in a further 279,552 individuals of diverse ancestry, including 30,288 cases of T2D, and test their association with T2D-related vascular outcomes. Cluster-specific partitioned polygenic scores are associated with coronary artery disease, peripheral artery disease and end-stage diabetic nephropathy across ancestry groups, highlighting the importance of obesity-related processes in the development of vascular outcomes. Our findings show the value of integrating multi-ancestry genome-wide association study data with single-cell epigenomics to disentangle the aetiological heterogeneity that drives the development and progression of T2D. This might offer a route to optimize global access to genetically informed diabetes care.

Deep learning discrimination of rheumatoid arthritis from osteoarthritis on hand radiography.

PURPOSE: To develop a deep learning model to distinguish rheumatoid arthritis (RA) from osteoarthritis (OA) using hand radiographs and to evaluate the effects of changing pretraining and training parameters on model performance. MATERIALS AND METHODS: A convolutional neural network was retrospectively trained on 9714 hand radiograph exams from 8387 patients obtained from 2017 to 2021 at seven hospitals within an integrated healthcare network. Performance was assessed using an independent test set of 250 exams from 146 patients. Binary discriminatory capacity (no arthritis versus arthritis; RA versus not RA) and three-way classification (no arthritis versus OA versus RA) were evaluated. The effects of additional pretraining using musculoskeletal radiographs, using all views as opposed to only the posteroanterior view, and varying image resolution on model performance were also investigated. Area under the receiver operating characteristic curve (AUC) and Cohen's kappa coefficient were used to evaluate diagnostic performance. RESULTS: For no arthritis versus arthritis, the model achieved an AUC of 0.975 (95% CI: 0.957, 0.989). For RA versus not RA, the model achieved an AUC of 0.955 (95% CI: 0.919, 0.983). For three-way classification, the model achieved a kappa of 0.806 (95% CI: 0.742, 0.866) and accuracy of 87.2% (95% CI: 83.2%, 91.2%) on the test set. Increasing image resolution increased performance up to 1024 × 1024 pixels. Additional pretraining on musculoskeletal radiographs and using all views did not significantly affect performance. CONCLUSION: A deep learning model can be used to distinguish no arthritis, OA, and RA on hand radiographs with high performance.

Artificial intelligence in neuroimaging of brain tumors: reality or still promise?

PURPOSE OF REVIEW: To provide an updated overview of artificial intelligence (AI) applications in neuro-oncologic imaging and discuss current barriers to wider clinical adoption. RECENT FINDINGS: A wide variety of AI applications in neuro-oncologic imaging have been developed and researched, spanning tasks from pretreatment brain tumor classification and segmentation, preoperative planning, radiogenomics, prognostication and survival prediction, posttreatment surveillance, and differentiating between pseudoprogression and true disease progression. While earlier studies were largely based on data from a single institution, more recent studies have demonstrated that the performance of these algorithms are also effective on external data from other institutions. Nevertheless, most of these algorithms have yet to see widespread clinical adoption, given the lack of prospective studies demonstrating their efficacy and the logistical difficulties involved in clinical implementation. SUMMARY: While there has been significant progress in AI and neuro-oncologic imaging, clinical utility remains to be demonstrated. The next wave of progress in this area will be driven by prospective studies measuring outcomes relevant to clinical practice and go beyond retrospective studies which primarily aim to demonstrate high performance.

Evaluating progress in automatic chest X-ray radiology report generation.

Artificial intelligence (AI) models for automatic generation of narrative radiology reports from images have the potential to enhance efficiency and reduce the workload of radiologists. However, evaluating the correctness of these reports requires metrics that can capture clinically pertinent differences. In this study, we investigate the alignment between automated metrics and radiologists' scoring of errors in report generation. We address the limitations of existing metrics by proposing new metrics, RadGraph F1 and RadCliQ, which demonstrate stronger correlation with radiologists' evaluations. In addition, we analyze the failure modes of the metrics to understand their limitations and provide guidance for metric selection and interpretation. This study establishes RadGraph F1 and RadCliQ as meaningful metrics for guiding future research in radiology report generation.

Multi-ancestry genome-wide study in >2.5 million individuals reveals heterogeneity in mechanistic pathways of type 2 diabetes and complications.

Type 2 diabetes (T2D) is a heterogeneous disease that develops through diverse pathophysiological processes. To characterise the genetic contribution to these processes across ancestry groups, we aggregate genome-wide association study (GWAS) data from 2,535,601 individuals (39.7% non-European ancestry), including 428,452 T2D cases. We identify 1,289 independent association signals at genome-wide significance (P<5×10-8) that map to 611 loci, of which 145 loci are previously unreported. We define eight non-overlapping clusters of T2D signals characterised by distinct profiles of cardiometabolic trait associations. These clusters are differentially enriched for cell-type specific regions of open chromatin, including pancreatic islets, adipocytes, endothelial, and enteroendocrine cells. We build cluster-specific partitioned genetic risk scores (GRS) in an additional 137,559 individuals of diverse ancestry, including 10,159 T2D cases, and test their association with T2D-related vascular outcomes. Cluster-specific partitioned GRS are more strongly associated with coronary artery disease and end-stage diabetic nephropathy than an overall T2D GRS across ancestry groups, highlighting the importance of obesity-related processes in the development of vascular outcomes. Our findings demonstrate the value of integrating multi-ancestry GWAS with single-cell epigenomics to disentangle the aetiological heterogeneity driving the development and progression of T2D, which may offer a route to optimise global access to genetically-informed diabetes care.

Efficient labeling of retinal fundus photographs using deep active learning.

Purpose: To compare the performance of four deep active learning (DAL) approaches to optimize label efficiency for training diabetic retinopathy (DR) classification deep learning models. Approach: 88,702 color retinal fundus photographs from 44,351 patients with DR grades from the publicly available EyePACS dataset were used. Four DAL approaches [entropy sampling (ES), Bayesian active learning by disagreement (BALD), core set, and adversarial active learning (ADV)] were compared to conventional naive random sampling. Models were compared at various dataset sizes using Cohen's kappa (CK) and area under the receiver operating characteristic curve on an internal test set of 10,000 images. An independent test set of 3662 fundus photographs was used to assess generalizability. Results: On the internal test set, 3 out of 4 DAL methods resulted in statistically significant performance improvements ( p < 1 × 10 - 4 ) compared to random sampling for multiclass classification, with the largest observed differences in CK ranging from 0.051 for BALD to 0.053 for ES. Improvements in multiclass classification generalized to the independent test set, with the largest differences in CK ranging from 0.126 to 0.135. However, no statistically significant improvements were seen for binary classification. Similar performance was seen across DAL methods, except ADV, which performed similarly to random sampling. Conclusions: Uncertainty-based and feature descriptor-based deep active learning methods outperformed random sampling on both the internal and independent test sets at multiclass classification. However, binary classification performance remained similar across random sampling and active learning methods.

End-to-end artificial intelligence platform for the management of large vessel occlusions: A preliminary study.

OBJECTIVES: In this study, we developed a deep learning pipeline that detects large vessel occlusion (LVO) and predicts functional outcome based on computed tomography angiography (CTA) images to improve the management of the LVO patients. METHODS: A series identifier picked out 8650 LVO-protocoled studies from 2015 to 2019 at Rhode Island Hospital with an identified thin axial series that served as the data pool. Data were annotated into 2 classes: 1021 LVOs and 7629 normal. The Inception-V1 I3D architecture was applied for LVO detection. For outcome prediction, 323 patients undergoing thrombectomy were selected. A 3D convolution neural network (CNN) was used for outcome prediction (30-day mRS) with CTA volumes and embedded pre-treatment variables as inputs. RESULT: For LVO-detection model, CTAs from 8,650 patients (median age 68 years, interquartile range (IQR): 58-81; 3934 females) were analyzed. The cross-validated AUC for LVO vs. not was 0.74 (95% CI: 0.72-0.75). For the mRS classification model, CTAs from 323 patients (median age 75 years, IQR: 63-84; 164 females) were analyzed. The algorithm achieved a test AUC of 0.82 (95% CI: 0.79-0.84), sensitivity of 89%, and specificity 66%. The two models were then integrated with hospital infrastructure where CTA was collected in real-time and processed by the model. If LVO was detected, interventionists were notified and provided with predicted clinical outcome information. CONCLUSION: 3D CNNs based on CTA were effective in selecting LVO and predicting LVO mechanical thrombectomy short-term prognosis. End-to-end AI platform allows users to receive immediate prognosis prediction and facilitates clinical workflow.

A SYSTEMATIC REVIEW OF DEEP LEARNING APPLICATIONS FOR OPTICAL COHERENCE TOMOGRAPHY IN AGE-RELATED MACULAR DEGENERATION.

PURPOSE: To survey the current literature regarding applications of deep learning to optical coherence tomography in age-related macular degeneration (AMD). METHODS: A Preferred Reporting Items for Systematic Reviews and Meta-Analyses systematic review was conducted from January 1, 2000, to May 9, 2021, using PubMed and EMBASE databases. Original research investigations that applied deep learning to optical coherence tomography in patients with AMD or features of AMD (choroidal neovascularization, geographic atrophy, and drusen) were included. Summary statements, data set characteristics, and performance metrics were extracted from included articles for analysis. RESULTS: We identified 95 articles for this review. The majority of articles fell into one of six categories: 1) classification of AMD or AMD biomarkers (n = 40); 2) segmentation of AMD biomarkers (n = 20); 3) segmentation of retinal layers or the choroid in patients with AMD (n = 7); 4) assessing treatment response and disease progression (n = 13); 5) predicting visual function (n = 6); and 6) determining the need for referral to a retina specialist (n = 3). CONCLUSION: Deep learning models generally achieved high performance, at times comparable with that of specialists. However, external validation and experimental parameters enabling reproducibility were often limited. Prospective studies that demonstrate generalizability and clinical utility of these models are needed.

Multi-ancestry genetic study of type 2 diabetes highlights the power of diverse populations for discovery and translation.

We assembled an ancestrally diverse collection of genome-wide association studies (GWAS) of type 2 diabetes (T2D) in 180,834 affected individuals and 1,159,055 controls (48.9% non-European descent) through the Diabetes Meta-Analysis of Trans-Ethnic association studies (DIAMANTE) Consortium. Multi-ancestry GWAS meta-analysis identified 237 loci attaining stringent genome-wide significance (P < 5 × 10-9), which were delineated to 338 distinct association signals. Fine-mapping of these signals was enhanced by the increased sample size and expanded population diversity of the multi-ancestry meta-analysis, which localized 54.4% of T2D associations to a single variant with >50% posterior probability. This improved fine-mapping enabled systematic assessment of candidate causal genes and molecular mechanisms through which T2D associations are mediated, laying the foundations for functional investigations. Multi-ancestry genetic risk scores enhanced transferability of T2D prediction across diverse populations. Our study provides a step toward more effective clinical translation of T2D GWAS to improve global health for all, irrespective of genetic background.

Artificial Intelligence Class Activation Mapping of Bone Age.

Online supplemental material is available for this article.

Federated Deep Learning to More Reliably Detect Body Part for Hanging Protocols, Relevant Priors, and Workflow Optimization.

Preparing radiology examinations for interpretation requires prefetching relevant prior examinations and implementing hanging protocols to optimally display the examination along with comparisons. Body part is a critical piece of information to facilitate both prefetching and hanging protocols, but body part information encoded using the Digital Imaging and Communications in Medicine (DICOM) standard is widely variable, error-prone, not granular enough, or missing altogether. This results in inappropriate examinations being prefetched or relevant examinations left behind; hanging protocol optimization suffers as well. Modern artificial intelligence (AI) techniques, particularly when harnessing federated deep learning techniques, allow for highly accurate automatic detection of body part based on the image data within a radiological examination; this allows for much more reliable implementation of this categorization and workflow. Additionally, new avenues to further optimize examination viewing such as dynamic hanging protocol and image display can be implemented using these techniques.

Artificial intelligence for prediction of COVID-19 progression using CT imaging and clinical data.

OBJECTIVES: Early recognition of coronavirus disease 2019 (COVID-19) severity can guide patient management. However, it is challenging to predict when COVID-19 patients will progress to critical illness. This study aimed to develop an artificial intelligence system to predict future deterioration to critical illness in COVID-19 patients. METHODS: An artificial intelligence (AI) system in a time-to-event analysis framework was developed to integrate chest CT and clinical data for risk prediction of future deterioration to critical illness in patients with COVID-19. RESULTS: A multi-institutional international cohort of 1,051 patients with RT-PCR confirmed COVID-19 and chest CT was included in this study. Of them, 282 patients developed critical illness, which was defined as requiring ICU admission and/or mechanical ventilation and/or reaching death during their hospital stay. The AI system achieved a C-index of 0.80 for predicting individual COVID-19 patients' to critical illness. The AI system successfully stratified the patients into high-risk and low-risk groups with distinct progression risks (p < 0.0001). CONCLUSIONS: Using CT imaging and clinical data, the AI system successfully predicted time to critical illness for individual patients and identified patients with high risk. AI has the potential to accurately triage patients and facilitate personalized treatment. KEY POINT: • AI system can predict time to critical illness for patients with COVID-19 by using CT imaging and clinical data.

DICOM Image ANalysis and Archive (DIANA): an Open-Source System for Clinical AI Applications.

In the era of data-driven medicine, rapid access and accurate interpretation of medical images are becoming increasingly important. The DICOM Image ANalysis and Archive (DIANA) system is an open-source, lightweight, and scalable Python interface that enables users to interact with hospital Picture Archiving and Communications Systems (PACS) to access such data. In this work, DIANA functionality was detailed and evaluated in the context of retrospective PACS data retrieval and two prospective clinical artificial intelligence (AI) pipelines: bone age (BA) estimation and intra-cranial hemorrhage (ICH) detection. DIANA orchestrates activity beginning with post-acquisition study discovery and ending with online notifications of findings. For AI applications, system latency (exam completion to system report time) was quantified and compared to that of clinicians (exam completion to initial report creation time). Mean DIANA latency was 9.04 ± 3.83 and 20.17 ± 10.16 min compared to clinician latency of 51.52 ± 58.9 and 65.62 ± 110.39 min for BA and ICH, respectively, with DIANA latencies being significantly lower (p < 0.001). DIANA's capabilities were also explored and found effective in retrieving and anonymizing protected health information for "big-data" medical imaging research and analysis. Mean per-image retrieval times were 1.12 ± 0.50 and 0.08 ± 0.01 s across x-ray and computed tomography studies, respectively. The data herein demonstrate that DIANA can flexibly integrate into existing hospital infrastructure and improve the process by which researchers/clinicians access imaging repository data. This results in a simplified workflow for large data retrieval and clinical integration of AI models.

Clinical Artificial Intelligence Applications in Radiology: Neuro.

Radiologists have been at the forefront of the digitization process in medicine. Artificial intelligence (AI) is a promising area of innovation, particularly in medical imaging. The number of applications of AI in neuroradiology has also grown. This article illustrates some of these applications. This article reviews machine learning challenges related to neuroradiology. The first approval of reimbursement for an AI algorithm by the Centers for Medicare and Medicaid Services, covering a stroke software for early detection of large vessel occlusion, is also discussed.

Deep Learning for Pulmonary Embolism Detection: Tackling the RSNA 2020 AI Challenge.

In 2020, the Radiological Society of North America and Society of Thoracic Radiology sponsored a machine learning competition to detect and classify pulmonary embolism (PE). This challenge was one of the largest of its kind, with more than 9000 CT pulmonary angiography examinations comprising almost 1.8 million expertly annotated images. More than 700 international teams competed to predict the presence of PE on individual axial images, the overall presence of PE in the CT examination (with chronicity and laterality), and the ratio of right ventricular size to left ventricular size. This article presents a detailed overview of the second-place solution. Source code and models are available at https://github.com/i-pan/kaggle-rsna-pe. Keywords: CT, Neural Networks, Thorax, Pulmonary Arteries, Embolism/Thrombosis, Supervised Learning, Convolutional Neural Networks (CNN), Machine Learning Algorithms © RSNA, 2021.

Deep Learning: An Update for Radiologists.

Deep learning is a class of machine learning methods that has been successful in computer vision. Unlike traditional machine learning methods that require hand-engineered feature extraction from input images, deep learning methods learn the image features by which to classify data. Convolutional neural networks (CNNs), the core of deep learning methods for imaging, are multilayered artificial neural networks with weighted connections between neurons that are iteratively adjusted through repeated exposure to training data. These networks have numerous applications in radiology, particularly in image classification, object detection, semantic segmentation, and instance segmentation. The authors provide an update on a recent primer on deep learning for radiologists, and they review terminology, data requirements, and recent trends in the design of CNNs; illustrate building blocks and architectures adapted to computer vision tasks, including generative architectures; and discuss training and validation, performance metrics, visualization, and future directions. Familiarity with the key concepts described will help radiologists understand advances of deep learning in medical imaging and facilitate clinical adoption of these techniques. Online supplemental material is available for this article. ©RSNA, 2021.

SCU-Net: A deep learning method for segmentation and quantification of breast arterial calcifications on mammograms.

PURPOSE: Measurements of breast arterial calcifications (BAC) can offer a personalized, non-invasive approach to risk-stratify women for cardiovascular diseases such as heart attack and stroke. We aim to detect and segment breast arterial calcifications in mammograms accurately and suggest novel measurements to quantify detected BAC for future clinical applications. METHODS: To separate BAC in mammograms, we propose a lightweight fine vessel segmentation method Simple Context U-Net (SCU-Net). Due to the large image size of mammograms, we adopt a patch-based way to train SCU-Net and obtain the final whole-image-size results by stitching patchwise results together. To further quantify calcifications, we test five quantitative metrics to inspect the progression of BAC for subjects: sum of mask probability metric ( P M ), sum of mask area metric ( A M ), sum of mask intensity metric ( S I M ), sum of mask area with threshold intensity metric T A M X , and sum of mask intensity with threshold X metric T S I M X . Finally, we demonstrate the ability of the metrics to longitudinally measure calcifications in a group of 26 subjects and evaluate our quantification metrics compared with calcified voxels and calcium mass on breast CT for 10 subjects. RESULTS: Our segmentation results are compared with state-of-the-art network architectures based on recall, precision, accuracy, F1 score/Dice score, and Jaccard index evaluation metrics and achieve corresponding values of 0.789, 0.708, 0.997, 0.729, and 0.581 for whole-image-size results. The quantification results all show >95% correlation between quantification measures on predicted masks of SCU-Net as compared to the groundtruth and measurement of calcification on breast CT. For the calcification quantification measurement, our calcification volume (voxels) results yield R2 -correlation values of 0.834, 0.843, 0.832, 0.798, and 0.800 for the P M , A M , S I M , T A M 100 , T S I M 100 metrics, respectively; our calcium mass results yield comparable R2 -correlation values of 0.866, 0.873, 0.840, 0.774, and 0.798 for the same metrics. CONCLUSIONS: Simple Context U-Net is a simple method to accurately segment arterial calcification retrospectively on routine mammograms. Quantification of the calcifications based on this segmentation in the retrospective cohort study has sufficient sensitivity to detect the normal progression over time and should be useful for future research and clinical applications.

Evaluation of a convolutional neural network for ovarian tumor differentiation based on magnetic resonance imaging.

OBJECTIVES: There currently lacks a noninvasive and accurate method to distinguish benign and malignant ovarian lesion prior to treatment. This study developed a deep learning algorithm that distinguishes benign from malignant ovarian lesion by applying a convolutional neural network on routine MR imaging. METHODS: Five hundred forty-five lesions (379 benign and 166 malignant) from 451 patients from a single institution were divided into training, validation, and testing set in a 7:2:1 ratio. Model performance was compared with four junior and three senior radiologists on the test set. RESULTS: Compared with junior radiologists averaged, the final ensemble model combining MR imaging and clinical variables had a higher test accuracy (0.87 vs 0.64, p < 0.001) and specificity (0.92 vs 0.64, p < 0.001) with comparable sensitivity (0.75 vs 0.63, p = 0.407). Against the senior radiologists averaged, the final ensemble model also had a higher test accuracy (0.87 vs 0.74, p = 0.033) and specificity (0.92 vs 0.70, p < 0.001) with comparable sensitivity (0.75 vs 0.83, p = 0.557). Assisted by the model's probabilities, the junior radiologists achieved a higher average test accuracy (0.77 vs 0.64, Δ = 0.13, p < 0.001) and specificity (0.81 vs 0.64, Δ = 0.17, p < 0.001) with unchanged sensitivity (0.69 vs 0.63, Δ = 0.06, p = 0.302). With the AI probabilities, the junior radiologists had higher specificity (0.81 vs 0.70, Δ = 0.11, p = 0.005) but similar accuracy (0.77 vs 0.74, Δ = 0.03, p = 0.409) and sensitivity (0.69 vs 0.83, Δ = -0.146, p = 0.097) when compared with the senior radiologists. CONCLUSIONS: These results demonstrate that artificial intelligence based on deep learning can assist radiologists in assessing the nature of ovarian lesions and improve their performance. KEY POINTS: • Artificial Intelligence based on deep learning can assess the nature of ovarian lesions on routine MRI with higher accuracy and specificity than radiologists. • Assisted by the deep learning model's probabilities, junior radiologists achieved better performance that matched those of senior radiologists.