Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses.

Saliency maps are popularly used to "explain" decisions made by modern machine learning models, including deep convolutional neural networks (DCNNs). While the resulting heatmaps purportedly indicate important image features, their "trustworthiness," i.e., utility and robustness, has not been evaluated for musculoskeletal imaging. The purpose of this study was to systematically evaluate the trustworthiness of saliency maps used in disease diagnosis on upper extremity X-ray images. The underlying DCNNs were trained using the Stanford MURA dataset. We studied four trustworthiness criteria-(1) localization accuracy of abnormalities, (2) repeatability, (3) reproducibility, and (4) sensitivity to underlying DCNN weights-across six different gradient-based saliency methods (Grad-CAM (GCAM), gradient explanation (GRAD), integrated gradients (IG), Smoothgrad (SG), smooth IG (SIG), and XRAI). Ground-truth was defined by the consensus of three fellowship-trained musculoskeletal radiologists who each placed bounding boxes around abnormalities on a holdout saliency test set. Compared to radiologists, all saliency methods showed inferior localization (AUPRCs: 0.438 (SG)-0.590 (XRAI); average radiologist AUPRC: 0.816), repeatability (IoUs: 0.427 (SG)-0.551 (IG); average radiologist IOU: 0.613), and reproducibility (IoUs: 0.250 (SG)-0.502 (XRAI); average radiologist IOU: 0.613) on abnormalities such as fractures, orthopedic hardware insertions, and arthritis. Five methods (GCAM, GRAD, IG, SG, XRAI) passed the sensitivity test. Ultimately, no saliency method met all four trustworthiness criteria; therefore, we recommend caution and rigorous evaluation of saliency maps prior to their clinical use.

Deep learning to detect left ventricular structural abnormalities in chest X-rays.

BACKGROUND AND AIMS: Early identification of cardiac structural abnormalities indicative of heart failure is crucial to improving patient outcomes. Chest X-rays (CXRs) are routinely conducted on a broad population of patients, presenting an opportunity to build scalable screening tools for structural abnormalities indicative of Stage B or worse heart failure with deep learning methods. In this study, a model was developed to identify severe left ventricular hypertrophy (SLVH) and dilated left ventricle (DLV) using CXRs. METHODS: A total of 71 589 unique CXRs from 24 689 different patients completed within 1 year of echocardiograms were identified. Labels for SLVH, DLV, and a composite label indicating the presence of either were extracted from echocardiograms. A deep learning model was developed and evaluated using area under the receiver operating characteristic curve (AUROC). Performance was additionally validated on 8003 CXRs from an external site and compared against visual assessment by 15 board-certified radiologists. RESULTS: The model yielded an AUROC of 0.79 (0.76-0.81) for SLVH, 0.80 (0.77-0.84) for DLV, and 0.80 (0.78-0.83) for the composite label, with similar performance on an external data set. The model outperformed all 15 individual radiologists for predicting the composite label and achieved a sensitivity of 71% vs. 66% against the consensus vote across all radiologists at a fixed specificity of 73%. CONCLUSIONS: Deep learning analysis of CXRs can accurately detect the presence of certain structural abnormalities and may be useful in early identification of patients with LV hypertrophy and dilation. As a resource to promote further innovation, 71 589 CXRs with adjoining echocardiographic labels have been made publicly available.

Deep Learning-Assisted Identification of Femoroacetabular Impingement (FAI) on Routine Pelvic Radiographs.

To use a novel deep learning system to localize the hip joints and detect findings of cam-type femoroacetabular impingement (FAI). A retrospective search of hip/pelvis radiographs obtained in patients to evaluate for FAI yielded 3050 total studies. Each hip was classified separately by the original interpreting radiologist in the following manner: 724 hips had severe cam-type FAI morphology, 962 moderate cam-type FAI morphology, 846 mild cam-type FAI morphology, and 518 hips were normal. The anteroposterior (AP) view from each study was anonymized and extracted. After localization of the hip joints by a novel convolutional neural network (CNN) based on the focal loss principle, a second CNN classified the images of the hip as cam positive, or no FAI. Accuracy was 74% for diagnosing normal vs. abnormal cam-type FAI morphology, with aggregate sensitivity and specificity of 0.821 and 0.669, respectively, at the chosen operating point. The aggregate AUC was 0.736. A deep learning system can be applied to detect FAI-related changes on single view pelvic radiographs. Deep learning is useful for quickly identifying and categorizing pathology on imaging, which may aid the interpreting radiologist.

Deep Learning-Assisted Diffusion Tensor Imaging for Evaluation of the Physis and Metaphysis.

Diffusion tensor imaging of physis and metaphysis can be used as a biomarker to predict height change in the pediatric population. Current application of this technique requires manual segmentation of the physis which is time-consuming and introduces interobserver variability. UNET Transformers (UNETR) can be used for automatic segmentation to optimize workflow. Three hundred and eighty-five DTI scans from 191 subjects with mean age of 12.6 years ± 2.01 years were retrospectively used for training and validation. The mean Dice correlation coefficient was 0.81 for the UNETR model and 0.68 for the UNET. Manual extraction and segmentation took 15 min per volume, whereas both deep learning segmentation techniques took < 1 s per volume and were deterministic, always producing the same result for a given input. Intraclass correlation coefficient (ICC) for ROI-derived femur diffusion metrics was excellent for tract count (0.95), volume (0.95), and FA (0.97), and good for tract length (0.87). The results support the hypothesis that a hybrid UNETR model can be trained to replace the manual segmentation of physeal DTI images, therefore automating the process.

Breast cancer risk prediction combining a convolutional neural network-based mammographic evaluation with clinical factors.

PURPOSE: Deep learning techniques, including convolutional neural networks (CNN), have the potential to improve breast cancer risk prediction compared to traditional risk models. We assessed whether combining a CNN-based mammographic evaluation with clinical factors in the Breast Cancer Surveillance Consortium (BCSC) model improved risk prediction. METHODS: We conducted a retrospective cohort study among 23,467 women, age 35-74, undergoing screening mammography (2014-2018). We extracted electronic health record (EHR) data on risk factors. We identified 121 women who subsequently developed invasive breast cancer at least 1 year after the baseline mammogram. Mammograms were analyzed with a pixel-wise mammographic evaluation using CNN architecture. We used logistic regression models with breast cancer incidence as the outcome and predictors including clinical factors only (BCSC model) or combined with CNN risk score (hybrid model). We compared model prediction performance via area under the receiver operating characteristics curves (AUCs). RESULTS: Mean age was 55.9 years (SD, 9.5) with 9.3% non-Hispanic Black and 36% Hispanic. Our hybrid model did not significantly improve risk prediction compared to the BCSC model (AUC of 0.654 vs 0.624, respectively, p = 0.063). In subgroup analyses, the hybrid model outperformed the BCSC model among non-Hispanic Blacks (AUC 0.845 vs. 0.589; p = 0.026) and Hispanics (AUC 0.650 vs 0.595; p = 0.049). CONCLUSION: We aimed to develop an efficient breast cancer risk assessment method using CNN risk score and clinical factors from the EHR. With future validation in a larger cohort, our CNN model combined with clinical factors may help predict breast cancer risk in a cohort of racially/ethnically diverse women undergoing screening.

Use of a convolutional neural network-based mammographic evaluation to predict breast cancer recurrence among women with hormone receptor-positive operable breast cancer.

PURPOSE: We evaluated whether a novel, fully automated convolutional neural network (CNN)-based mammographic evaluation can predict breast cancer relapse among women with operable hormone receptor (HR)-positive breast cancer. METHODS: We conducted a retrospective cohort study among women with stage I-III, HR-positive unilateral breast cancer diagnosed at Columbia University Medical Center from 2007 to 2017, who received adjuvant endocrine therapy and had at least two mammograms (baseline, annual follow-up) of the contralateral unaffected breast for CNN analysis. We extracted demographics, clinicopathologic characteristics, breast cancer treatments, and relapse status from the electronic health record. Our primary endpoint was change in CNN risk score (range, 0-1). We used two-sample t-tests to assess for difference in mean CNN scores between patients who relapsed vs. remained in remission, and conducted Cox regression analyses to assess for association between change in CNN score and breast cancer-free interval (BCFI), adjusting for known prognostic factors. RESULTS: Among 848 women followed for a median of 59 months, there were 67 (7.9%) breast cancer relapses (36 distant, 25 local, 6 new primaries). There was a significant difference in mean absolute change in CNN risk score from baseline to 1-year follow-up between those who relapsed vs. remained in remission (0.001 vs. - 0.022, p = 0.030). After adjustment for prognostic factors, a 0.01 absolute increase in CNN score at 1-year was significantly associated with BCFI, hazard ratio = 1.05 (95% Confidence Interval 1.01-1.09, p = 0.011). CONCLUSION: Short-term change in the CNN-based breast cancer risk model on adjuvant endocrine therapy predicts breast cancer relapse, and warrants further evaluation in prospective studies.

Deep learning prediction of axillary lymph node status using ultrasound images.

OBJECTIVE: To investigate the ability of our convolutional neural network (CNN) to predict axillary lymph node metastasis using primary breast cancer ultrasound (US) images. METHODS: In this IRB-approved study, 338 US images (two orthogonal images) from 169 patients from 1/2014-12/2016 were used. Suspicious lymph nodes were seen on US and patients subsequently underwent core-biopsy. 64 patients had metastatic lymph nodes. A custom CNN was utilized on 248 US images from 124 patients in the training dataset and tested on 90 US images from 45 patients. The CNN was implemented entirely of 3 × 3 convolutional kernels and linear layers. The 9 convolutional kernels consisted of 6 residual layers, totaling 12 convolutional layers. Feature maps were down-sampled using strided convolutions. Dropout with a 0.5 keep probability and L2 normalization was utilized. Training was implemented by using the Adam optimizer and a final SoftMax score threshold of 0.5 from the average of raw logits from each pixel was used for two class classification (metastasis or not). RESULTS: Our CNN achieved an AUC of 0.72 (SD ± 0.08) in predicting axillary lymph node metastasis from US images in the testing dataset. The model had an accuracy of 72.6% (SD ± 8.4) with a sensitivity and specificity of 65.5% (SD ± 28.6) and 78.9% (SD ± 15.1) respectively. Our algorithm is available to be shared for research use. (https://github.com/stmutasa/MetUS). CONCLUSION: It's feasible to predict axillary lymph node metastasis from US images using a deep learning technique. This can potentially aid nodal staging in patients with breast cancer.

Deciphering musculoskeletal artificial intelligence for clinical applications: how do I get started?

Artificial intelligence (AI) represents a broad category of algorithms for which deep learning is currently the most impactful. When electing to begin the process of building an adequate fundamental knowledge base allowing them to decipher machine learning research and algorithms, clinical musculoskeletal radiologists currently have few options to turn to. In this article, we provide an introduction to the vital terminology to understand, how to make sense of data splits and regularization, an introduction to the statistical analyses used in AI research, a primer on what deep learning can or cannot do, and a brief overview of clinical integration methods. Our goal is to improve the readers' understanding of this field.

AI MSK clinical applications: orthopedic implants.

Artificial intelligence (AI) and deep learning have multiple potential uses in aiding the musculoskeletal radiologist in the radiological evaluation of orthopedic implants. These include identification of implants, characterization of implants according to anatomic type, identification of specific implant models, and evaluation of implants for positioning and complications. In addition, natural language processing (NLP) can aid in the acquisition of clinical information from the medical record that can help with tasks like prepopulating radiology reports. Several proof-of-concept works have been published in the literature describing the application of deep learning toward these various tasks, with performance comparable to that of expert musculoskeletal radiologists. Although much work remains to bring these proof-of-concept algorithms into clinical deployment, AI has tremendous potential toward automating these tasks, thereby augmenting the musculoskeletal radiologist.

Clinical Artificial Intelligence Applications: Musculoskeletal.

We present an overview of current clinical musculoskeletal imaging applications for artificial intelligence, as well as potential future applications and techniques.

Understanding artificial intelligence based radiology studies: CNN architecture.

Artificial intelligence (AI) in radiology has gained wide interest due to the development of neural network architectures with high performance in computer vision related tasks. As AI based software programs become more integrated into the clinical workflow, radiologists can benefit from better understanding the principles of artificial intelligence. This series aims to explain basic concepts of AI and its applications in medical imaging. In this article, we will review the background of neural network architecture and its application in imaging analysis.

Deep Learning of Computed Tomography Virtual Wedge Resection for Prediction of Histologic Usual Interstitial Pneumonitis.

Rationale: The computed tomography (CT) pattern of definite or probable usual interstitial pneumonia (UIP) can be diagnostic of idiopathic pulmonary fibrosis and may obviate the need for invasive surgical biopsy. Few machine-learning studies have investigated the classification of interstitial lung disease (ILD) on CT imaging, but none have used histopathology as a reference standard.Objectives: To predict histopathologic UIP using deep learning of high-resolution computed tomography (HRCT).Methods: Institutional databases were retrospectively searched for consecutive patients with ILD, HRCT, and diagnostic histopathology from 2011 to 2014 (training cohort) and from 2016 to 2017 (testing cohort). A blinded expert radiologist and pulmonologist reviewed all training HRCT scans in consensus and classified HRCT scans based on the 2018 American Thoracic Society/European Respriatory Society/Japanese Respiratory Society/Latin American Thoracic Association diagnostic criteria for idiopathic pulmonary fibrosis. A convolutional neural network (CNN) was built accepting 4 × 4 × 2 cm virtual wedges of peripheral lung on HRCT as input and outputting the UIP histopathologic pattern. The CNN was trained and evaluated on the training cohort using fivefold cross validation and was then tested on the hold-out testing cohort. CNN and human performance were compared in the training cohort. Logistic regression and survival analyses were performed.Results: The CNN was trained on 221 patients (median age 60 yr; interquartile range [IQR], 53-66), including 71 patients (32%) with UIP or probable UIP histopathologic patterns. The CNN was tested on a separate hold-out cohort of 80 patients (median age 66 yr; IQR, 58-69), including 22 patients (27%) with UIP or probable UIP histopathologic patterns. An average of 516 wedges were generated per patient. The percentage of wedges with CNN-predicted UIP yielded a cross validation area under the curve of 74% for histopathological UIP pattern per patient. The optimal cutoff point for classifying patients on the training cohort was 16.5% of virtual lung wedges with CNN-predicted UIP and resulted in sensitivity and specificity of 74% and 58%, respectively, in the testing cohort. CNN-predicted UIP was associated with an increased risk of death or lung transplantation during cross validation (hazard ratio, 1.5; 95% confidence interval, 1.1-2.2; P = 0.03).Conclusions: Virtual lung wedge resection in patients with ILD can be used as an input to a CNN for predicting the histopathologic UIP pattern and transplant-free survival.

Dynamic Changes of Convolutional Neural Network-based Mammographic Breast Cancer Risk Score Among Women Undergoing Chemoprevention Treatment.

INTRODUCTION: We investigated whether our convolutional neural network (CNN)-based breast cancer risk model is modifiable by testing it on women who had undergone risk-reducing chemoprevention treatment. MATERIALS AND METHODS: We conducted a retrospective cohort study of patients diagnosed with atypical hyperplasia, lobular carcinoma in situ, or ductal carcinoma in situ at our institution from 2007 to 2015. The clinical characteristics, chemoprevention use, and mammography images were extracted from the electronic health records. We classified two groups according to chemoprevention use. Mammograms were performed at baseline and subsequent follow-up evaluations for input to our CNN risk model. The 2 chemoprevention groups were compared for the risk score change from baseline to follow-up. The change categories included stayed high risk, stayed low risk, increased from low to high risk, and decreased from high to low risk. Unordered polytomous regression models were used for statistical analysis, with P < .05 considered statistically significant. RESULTS: Of 541 patients, 184 (34%) had undergone chemoprevention treatment (group 1) and 357 (66%) had not (group 2). Using our CNN breast cancer risk score, significantly more women in group 1 had shown a decrease in breast cancer risk compared with group 2 (33.7% vs. 22.9%; P < .01). Significantly fewer women in group 1 had an increase in breast cancer risk compared with group 2 (11.4% vs. 20.2%; P < .01). On multivariate analysis, an increase in breast cancer risk predicted by our model correlated negatively with the use of chemoprevention treatment (P = .02). CONCLUSIONS: Our CNN-based breast cancer risk score is modifiable with potential utility in assessing the efficacy of known chemoprevention agents and testing new chemoprevention strategies.

Development and external validation of a prognostic tool for COVID-19 critical disease.

BACKGROUND: The rapid spread of coronavirus disease 2019 (COVID-19) revealed significant constraints in critical care capacity. In anticipation of subsequent waves, reliable prediction of disease severity is essential for critical care capacity management and may enable earlier targeted interventions to improve patient outcomes. The purpose of this study is to develop and externally validate a prognostic model/clinical tool for predicting COVID-19 critical disease at presentation to medical care. METHODS: This is a retrospective study of a prognostic model for the prediction of COVID-19 critical disease where critical disease was defined as ICU admission, ventilation, and/or death. The derivation cohort was used to develop a multivariable logistic regression model. Covariates included patient comorbidities, presenting vital signs, and laboratory values. Model performance was assessed on the validation cohort by concordance statistics. The model was developed with consecutive patients with COVID-19 who presented to University of California Irvine Medical Center in Orange County, California. External validation was performed with a random sample of patients with COVID-19 at Emory Healthcare in Atlanta, Georgia. RESULTS: Of a total 3208 patients tested in the derivation cohort, 9% (299/3028) were positive for COVID-19. Clinical data including past medical history and presenting laboratory values were available for 29% (87/299) of patients (median age, 48 years [range, 21-88 years]; 64% [36/55] male). The most common comorbidities included obesity (37%, 31/87), hypertension (37%, 32/87), and diabetes (24%, 24/87). Critical disease was present in 24% (21/87). After backward stepwise selection, the following factors were associated with greatest increased risk of critical disease: number of comorbidities, body mass index, respiratory rate, white blood cell count, % lymphocytes, serum creatinine, lactate dehydrogenase, high sensitivity troponin I, ferritin, procalcitonin, and C-reactive protein. Of a total of 40 patients in the validation cohort (median age, 60 years [range, 27-88 years]; 55% [22/40] male), critical disease was present in 65% (26/40). Model discrimination in the validation cohort was high (concordance statistic: 0.94, 95% confidence interval 0.87-1.01). A web-based tool was developed to enable clinicians to input patient data and view likelihood of critical disease. CONCLUSIONS AND RELEVANCE: We present a model which accurately predicted COVID-19 critical disease risk using comorbidities and presenting vital signs and laboratory values, on derivation and validation cohorts from two different institutions. If further validated on additional cohorts of patients, this model/clinical tool may provide useful prognostication of critical care needs.

A novel CNN algorithm for pathological complete response prediction using an I-SPY TRIAL breast MRI database.

PURPOSE: To apply our convolutional neural network (CNN) algorithm to predict neoadjuvant chemotherapy (NAC) response using the I-SPY TRIAL breast MRI dataset. METHODS: From the I-SPY TRIAL breast MRI database, 131 patients from 9 institutions were successfully downloaded for analysis. First post-contrast MRI images were used for 3D segmentation using 3D slicer. Our CNN was implemented entirely of 3 × 3 convolutional kernels and linear layers. The convolutional kernels consisted of 6 residual layers, totaling 12 convolutional layers. Dropout with a 0.5 keep probability and L2 normalization was utilized. Training was implemented by using the Adam optimizer. A 5-fold cross validation was used for performance evaluation. Software code was written in Python using the TensorFlow module on a Linux workstation with one NVidia Titan X GPU. RESULTS: Of 131 patients, 40 patients achieved pCR following NAC (group 1) and 91 patients did not achieve pCR following NAC (group 2). Diagnostic accuracy of our CNN two classification model distinguishing patients with pCR vs non-pCR was 72.5 (SD ± 8.4), with sensitivity 65.5% (SD ± 28.1) and specificity of 78.9% (SD ± 15.2). The area under a ROC Curve (AUC) was 0.72 (SD ± 0.08). CONCLUSION: It is feasible to use our CNN algorithm to predict NAC response in patients using a multi-institution dataset.

Prospective Analysis Using a Novel CNN Algorithm to Distinguish Atypical Ductal Hyperplasia From Ductal Carcinoma in Situ in Breast.

INTRODUCTION: We previously developed a convolutional neural networks (CNN)-based algorithm to distinguish atypical ductal hyperplasia (ADH) from ductal carcinoma in situ (DCIS) using a mammographic dataset. The purpose of this study is to further validate our CNN algorithm by prospectively analyzing an unseen new dataset to evaluate the diagnostic performance of our algorithm. MATERIALS AND METHODS: In this institutional review board-approved study, a new dataset composed of 280 unique mammographic images from 140 patients was used to test our CNN algorithm. All patients underwent stereotactic-guided biopsy of calcifications and underwent surgical excision with available final pathology. The ADH group consisted of 122 images from 61 patients with the highest pathology diagnosis of ADH. The DCIS group consisted of 158 images from 79 patients with the highest pathology diagnosis of DCIS. Two standard mammographic magnification views (craniocaudal and mediolateral/lateromedial) of the calcifications were used for analysis. Calcifications were segmented using an open source software platform 3D slicer and resized to fit a 128 × 128 pixel bounding box. Our previously developed CNN algorithm was used. Briefly, a 15 hidden layer topology was used. The network architecture contained 5 residual layers and dropout of 0.25 after each convolution. Diagnostic performance metrics were analyzed including sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve. The "positive class" was defined as the pure ADH group in this study and thus specificity represents minimizing the amount of falsely labeled pure ADH cases. RESULTS: Area under the receiver operating characteristic curve was 0.90 (95% confidence interval, ± 0.04). Diagnostic accuracy, sensitivity, and specificity was 80.7%, 63.9%, and 93.7%, respectively. CONCLUSION: Prospectively tested on new unseen data, our CNN algorithm distinguished pure ADH from DCIS using mammographic images with high specificity.

Channel width optimized neural networks for liver and vessel segmentation in liver iron quantification.

INTRODUCTION: MRI T2* relaxometry protocols are often used for Liver Iron Quantification in patients with hemochromatosis. Several methods exist to semi-automatically segment parenchyma and exclude vessels for this calculation. PURPOSE: To determine if inclusion of multiple echoes inputs to Convolutional Neural Networks (CNN) improves automated liver and vessel segmentation in MRI T2* relaxometry protocols and to determine if the resultant segmentations agree with manual segmentations for liver iron quantification analysis. METHODS: Multi echo Gradient Recalled Echo (GRE) MRI sequence for T2* relaxometry was performed for 79 exams on 31 patients with hemochromatosis for iron quantification analysis. 275 axial liver slices were manually segmented as ground truth masks. A batch normalized U-Net with variable input width to incorporate multiple echoes is used for segmentation, using DICE as the accuracy metric. ANOVA is used to evaluate significance of channel width changes in segmentation accuracy. Linear regression is used to model the relationship of channel width on segmentation accuracy. Liver segmentations are applied to relaxometry data to calculate liver T2* yielding liver iron concentration(LIC) derived from literature based calibration curves. Manual and CNN based LIC values are compared with Pearson correlation. Bland altman plots are used to visualize differences between manual and CNN based LIC values. RESULTS: Performance metrics are tested on 55 hold out slices. Linear regression indicates that there is a monotonic increase of DICE with increasing channel depth (p = 0.001) with a slope of 3.61e-3. ANOVA indicates a significant increase segmentation accuracy over single channel starting at 3 channels. Incorporation of all channels results in an average DICE of 0.86, an average increase of 0.07 over single channel. The calculated LIC from CNN segmented livers agrees well with manual segmentation (R = 0.998, slope = 0.914, p«0.001), with an average absolute difference 0.27 ± 0.99 mg Fe/g or 1.34 ± 4.3%. CONCLUSION: More input echoes yields higher model accuracy until the noise floor. Echos beyond the first three echo times in GRE based T2* relaxometry do not contribute significant information for segmentation of liver for LIC calculation. Deep learning models with three channel width allow for generalization of model to protocols of more than three echoes, effectively a universal requirement for relaxometry. Deep learning segmentations achieve a good accuracy compared with manual segmentations with minimal preprocessing. Liver iron values calculated from hand segmented liver and Neural network segmented liver were not statistically different from each other.

Advanced Deep Learning Techniques Applied to Automated Femoral Neck Fracture Detection and Classification.

To use deep learning with advanced data augmentation to accurately diagnose and classify femoral neck fractures. A retrospective study of patients with femoral neck fractures was performed. One thousand sixty-three AP hip radiographs were obtained from 550 patients. Ground truth labels of Garden fracture classification were applied as follows: (1) 127 Garden I and II fracture radiographs, (2) 610 Garden III and IV fracture radiographs, and (3) 326 normal hip radiographs. After localization by an initial network, a second CNN classified the images as Garden I/II fracture, Garden III/IV fracture, or no fracture. Advanced data augmentation techniques expanded the training set: (1) generative adversarial network (GAN); (2) digitally reconstructed radiographs (DRRs) from preoperative hip CT scans. In all, 9063 images, real and generated, were available for training and testing. A deep neural network was designed and tuned based on a 20% validation group. A holdout test dataset consisted of 105 real images, 35 in each class. Two class prediction of fracture versus no fracture (AUC 0.92): accuracy 92.3%, sensitivity 0.91, specificity 0.93, PPV 0.96, NPV 0.86. Three class prediction of Garden I/II, Garden III/IV, or normal (AUC 0.96): accuracy 86.0%, sensitivity 0.79, specificity 0.90, PPV 0.80, NPV 0.90. Without any advanced augmentation, the AUC for two-class prediction was 0.80. With DRR as the only advanced augmentation, AUC was 0.91 and with GAN only AUC was 0.87. GANs and DRRs can be used to improve the accuracy of a tool to diagnose and classify femoral neck fractures.

Understanding artificial intelligence based radiology studies: What is overfitting?

Artificial intelligence (AI) is a broad umbrella term used to encompass a wide variety of subfields dedicated to creating algorithms to perform tasks that mimic human intelligence. As AI development grows closer to clinical integration, radiologists will need to become familiar with the principles of artificial intelligence to properly evaluate and use this powerful tool. This series aims to explain certain basic concepts of artificial intelligence, and their applications in medical imaging starting with a concept of overfitting.

Fully Automated Segmentation Algorithm for Perihematomal Edema Volumetry After Spontaneous Intracerebral Hemorrhage.

Background and Purpose- Perihematomal edema (PHE) is a promising surrogate marker of secondary brain injury in patients with spontaneous intracerebral hemorrhage, but it can be challenging to accurately and rapidly quantify. The aims of this study are to derive and internally validate a fully automated segmentation algorithm for volumetric analysis of PHE. Methods- Inpatient computed tomography scans of 400 consecutive adults with spontaneous, supratentorial intracerebral hemorrhage enrolled in the Intracerebral Hemorrhage Outcomes Project (2009-2018) were separated into training (n=360) and test (n=40) datasets. A fully automated segmentation algorithm was derived from manual segmentations in the training dataset using convolutional neural networks, and its performance was compared with that of manual and semiautomated segmentation methods in the test dataset. Results- The mean volumetric dice similarity coefficients for the fully automated segmentation algorithm were 0.838±0.294 and 0.843±0.293 with manual and semiautomated segmentation methods as reference standards, respectively. PHE volumes derived from the fully automated versus manual (r=0.959; P<0.0001), fully automated versus semiautomated (r=0.960; P<0.0001), and semiautomated versus manual (r=0.961; P<0.0001) segmentation methods had strong between-group correlations. The fully automated segmentation algorithm (mean 18.0±1.8 seconds/scan) quantified PHE volumes at a significantly faster rate than both of the manual (mean 316.4±168.8 seconds/scan; P<0.0001) and semiautomated (mean 480.5±295.3 seconds/scan; P<0.0001) segmentation methods. Conclusions- The fully automated segmentation algorithm accurately quantified PHE volumes from computed tomography scans of supratentorial intracerebral hemorrhage patients with high fidelity and greater efficiency compared with manual and semiautomated segmentation methods. External validation of fully automated segmentation for assessment of PHE is warranted.