Deep learning to detect left ventricular structural abnormalities in chest X-rays.

BACKGROUND AND AIMS: Early identification of cardiac structural abnormalities indicative of heart failure is crucial to improving patient outcomes. Chest X-rays (CXRs) are routinely conducted on a broad population of patients, presenting an opportunity to build scalable screening tools for structural abnormalities indicative of Stage B or worse heart failure with deep learning methods. In this study, a model was developed to identify severe left ventricular hypertrophy (SLVH) and dilated left ventricle (DLV) using CXRs. METHODS: A total of 71 589 unique CXRs from 24 689 different patients completed within 1 year of echocardiograms were identified. Labels for SLVH, DLV, and a composite label indicating the presence of either were extracted from echocardiograms. A deep learning model was developed and evaluated using area under the receiver operating characteristic curve (AUROC). Performance was additionally validated on 8003 CXRs from an external site and compared against visual assessment by 15 board-certified radiologists. RESULTS: The model yielded an AUROC of 0.79 (0.76-0.81) for SLVH, 0.80 (0.77-0.84) for DLV, and 0.80 (0.78-0.83) for the composite label, with similar performance on an external data set. The model outperformed all 15 individual radiologists for predicting the composite label and achieved a sensitivity of 71% vs. 66% against the consensus vote across all radiologists at a fixed specificity of 73%. CONCLUSIONS: Deep learning analysis of CXRs can accurately detect the presence of certain structural abnormalities and may be useful in early identification of patients with LV hypertrophy and dilation. As a resource to promote further innovation, 71 589 CXRs with adjoining echocardiographic labels have been made publicly available.

Breast cancer risk prediction combining a convolutional neural network-based mammographic evaluation with clinical factors.

PURPOSE: Deep learning techniques, including convolutional neural networks (CNN), have the potential to improve breast cancer risk prediction compared to traditional risk models. We assessed whether combining a CNN-based mammographic evaluation with clinical factors in the Breast Cancer Surveillance Consortium (BCSC) model improved risk prediction. METHODS: We conducted a retrospective cohort study among 23,467 women, age 35-74, undergoing screening mammography (2014-2018). We extracted electronic health record (EHR) data on risk factors. We identified 121 women who subsequently developed invasive breast cancer at least 1 year after the baseline mammogram. Mammograms were analyzed with a pixel-wise mammographic evaluation using CNN architecture. We used logistic regression models with breast cancer incidence as the outcome and predictors including clinical factors only (BCSC model) or combined with CNN risk score (hybrid model). We compared model prediction performance via area under the receiver operating characteristics curves (AUCs). RESULTS: Mean age was 55.9 years (SD, 9.5) with 9.3% non-Hispanic Black and 36% Hispanic. Our hybrid model did not significantly improve risk prediction compared to the BCSC model (AUC of 0.654 vs 0.624, respectively, p = 0.063). In subgroup analyses, the hybrid model outperformed the BCSC model among non-Hispanic Blacks (AUC 0.845 vs. 0.589; p = 0.026) and Hispanics (AUC 0.650 vs 0.595; p = 0.049). CONCLUSION: We aimed to develop an efficient breast cancer risk assessment method using CNN risk score and clinical factors from the EHR. With future validation in a larger cohort, our CNN model combined with clinical factors may help predict breast cancer risk in a cohort of racially/ethnically diverse women undergoing screening.

Use of a convolutional neural network-based mammographic evaluation to predict breast cancer recurrence among women with hormone receptor-positive operable breast cancer.

PURPOSE: We evaluated whether a novel, fully automated convolutional neural network (CNN)-based mammographic evaluation can predict breast cancer relapse among women with operable hormone receptor (HR)-positive breast cancer. METHODS: We conducted a retrospective cohort study among women with stage I-III, HR-positive unilateral breast cancer diagnosed at Columbia University Medical Center from 2007 to 2017, who received adjuvant endocrine therapy and had at least two mammograms (baseline, annual follow-up) of the contralateral unaffected breast for CNN analysis. We extracted demographics, clinicopathologic characteristics, breast cancer treatments, and relapse status from the electronic health record. Our primary endpoint was change in CNN risk score (range, 0-1). We used two-sample t-tests to assess for difference in mean CNN scores between patients who relapsed vs. remained in remission, and conducted Cox regression analyses to assess for association between change in CNN score and breast cancer-free interval (BCFI), adjusting for known prognostic factors. RESULTS: Among 848 women followed for a median of 59 months, there were 67 (7.9%) breast cancer relapses (36 distant, 25 local, 6 new primaries). There was a significant difference in mean absolute change in CNN risk score from baseline to 1-year follow-up between those who relapsed vs. remained in remission (0.001 vs. - 0.022, p = 0.030). After adjustment for prognostic factors, a 0.01 absolute increase in CNN score at 1-year was significantly associated with BCFI, hazard ratio = 1.05 (95% Confidence Interval 1.01-1.09, p = 0.011). CONCLUSION: Short-term change in the CNN-based breast cancer risk model on adjuvant endocrine therapy predicts breast cancer relapse, and warrants further evaluation in prospective studies.

Deciphering musculoskeletal artificial intelligence for clinical applications: how do I get started?

Artificial intelligence (AI) represents a broad category of algorithms for which deep learning is currently the most impactful. When electing to begin the process of building an adequate fundamental knowledge base allowing them to decipher machine learning research and algorithms, clinical musculoskeletal radiologists currently have few options to turn to. In this article, we provide an introduction to the vital terminology to understand, how to make sense of data splits and regularization, an introduction to the statistical analyses used in AI research, a primer on what deep learning can or cannot do, and a brief overview of clinical integration methods. Our goal is to improve the readers' understanding of this field.

AI MSK clinical applications: orthopedic implants.

Artificial intelligence (AI) and deep learning have multiple potential uses in aiding the musculoskeletal radiologist in the radiological evaluation of orthopedic implants. These include identification of implants, characterization of implants according to anatomic type, identification of specific implant models, and evaluation of implants for positioning and complications. In addition, natural language processing (NLP) can aid in the acquisition of clinical information from the medical record that can help with tasks like prepopulating radiology reports. Several proof-of-concept works have been published in the literature describing the application of deep learning toward these various tasks, with performance comparable to that of expert musculoskeletal radiologists. Although much work remains to bring these proof-of-concept algorithms into clinical deployment, AI has tremendous potential toward automating these tasks, thereby augmenting the musculoskeletal radiologist.

Fully Automated Segmentation Algorithm for Perihematomal Edema Volumetry After Spontaneous Intracerebral Hemorrhage.

Background and Purpose- Perihematomal edema (PHE) is a promising surrogate marker of secondary brain injury in patients with spontaneous intracerebral hemorrhage, but it can be challenging to accurately and rapidly quantify. The aims of this study are to derive and internally validate a fully automated segmentation algorithm for volumetric analysis of PHE. Methods- Inpatient computed tomography scans of 400 consecutive adults with spontaneous, supratentorial intracerebral hemorrhage enrolled in the Intracerebral Hemorrhage Outcomes Project (2009-2018) were separated into training (n=360) and test (n=40) datasets. A fully automated segmentation algorithm was derived from manual segmentations in the training dataset using convolutional neural networks, and its performance was compared with that of manual and semiautomated segmentation methods in the test dataset. Results- The mean volumetric dice similarity coefficients for the fully automated segmentation algorithm were 0.838±0.294 and 0.843±0.293 with manual and semiautomated segmentation methods as reference standards, respectively. PHE volumes derived from the fully automated versus manual (r=0.959; P<0.0001), fully automated versus semiautomated (r=0.960; P<0.0001), and semiautomated versus manual (r=0.961; P<0.0001) segmentation methods had strong between-group correlations. The fully automated segmentation algorithm (mean 18.0±1.8 seconds/scan) quantified PHE volumes at a significantly faster rate than both of the manual (mean 316.4±168.8 seconds/scan; P<0.0001) and semiautomated (mean 480.5±295.3 seconds/scan; P<0.0001) segmentation methods. Conclusions- The fully automated segmentation algorithm accurately quantified PHE volumes from computed tomography scans of supratentorial intracerebral hemorrhage patients with high fidelity and greater efficiency compared with manual and semiautomated segmentation methods. External validation of fully automated segmentation for assessment of PHE is warranted.

Advanced Deep Learning Techniques Applied to Automated Femoral Neck Fracture Detection and Classification.

To use deep learning with advanced data augmentation to accurately diagnose and classify femoral neck fractures. A retrospective study of patients with femoral neck fractures was performed. One thousand sixty-three AP hip radiographs were obtained from 550 patients. Ground truth labels of Garden fracture classification were applied as follows: (1) 127 Garden I and II fracture radiographs, (2) 610 Garden III and IV fracture radiographs, and (3) 326 normal hip radiographs. After localization by an initial network, a second CNN classified the images as Garden I/II fracture, Garden III/IV fracture, or no fracture. Advanced data augmentation techniques expanded the training set: (1) generative adversarial network (GAN); (2) digitally reconstructed radiographs (DRRs) from preoperative hip CT scans. In all, 9063 images, real and generated, were available for training and testing. A deep neural network was designed and tuned based on a 20% validation group. A holdout test dataset consisted of 105 real images, 35 in each class. Two class prediction of fracture versus no fracture (AUC 0.92): accuracy 92.3%, sensitivity 0.91, specificity 0.93, PPV 0.96, NPV 0.86. Three class prediction of Garden I/II, Garden III/IV, or normal (AUC 0.96): accuracy 86.0%, sensitivity 0.79, specificity 0.90, PPV 0.80, NPV 0.90. Without any advanced augmentation, the AUC for two-class prediction was 0.80. With DRR as the only advanced augmentation, AUC was 0.91 and with GAN only AUC was 0.87. GANs and DRRs can be used to improve the accuracy of a tool to diagnose and classify femoral neck fractures.

Fully Automated Segmentation Algorithm for Hematoma Volumetric Analysis in Spontaneous Intracerebral Hemorrhage.

Background and Purpose- Hematoma volume measurements influence prognosis and treatment decisions in patients with spontaneous intracerebral hemorrhage (ICH). The aims of this study are to derive and validate a fully automated segmentation algorithm for ICH volumetric analysis using deep learning methods. Methods- In-patient computed tomography scans of 300 consecutive adults (age ≥18 years) with spontaneous, supratentorial ICH who were enrolled in the ICHOP (Intracerebral Hemorrhage Outcomes Project; 2009-2018) were separated into training (n=260) and test (n=40) datasets. A fully automated segmentation algorithm was derived using convolutional neural networks, and it was trained on manual segmentations from the training dataset. The algorithm's performance was assessed against manual and semiautomated segmentation methods in the test dataset. Results- The mean volumetric Dice similarity coefficients for the fully automated segmentation algorithm when tested against manual and semiautomated segmentation methods were 0.894±0.264 and 0.905±0.254, respectively. ICH volumes derived from fully automated versus manual (R2=0.981; P<0.0001), fully automated versus semiautomated (R2=0.978; P<0.0001), and semiautomated versus manual (R2=0.990; P<0001) segmentation methods had strong between-group correlations. The fully automated segmentation algorithm (mean 12.0±2.7 s/scan) was significantly faster than both of the manual (mean 201.5±92.2 s/scan; P<0.001) and semiautomated (mean 288.58±160.3 s/scan; P<0.001) segmentation methods. Conclusions- The fully automated segmentation algorithm quantified hematoma volumes from computed tomography scans of supratentorial ICH patients with similar accuracy and substantially greater efficiency compared with manual and semiautomated segmentation methods. External validation of the fully automated segmentation algorithm is warranted.

Convolutional Neural Network Using a Breast MRI Tumor Dataset Can Predict Oncotype Dx Recurrence Score.

BACKGROUND: Oncotype Dx is a validated genetic analysis that provides a recurrence score (RS) to quantitatively predict outcomes in patients who meet the criteria of estrogen receptor positive / human epidermal growth factor receptor-2 negative (ER+/HER2-)/node negative invasive breast carcinoma. Although effective, the test is invasive and expensive, which has motivated this investigation to determine the potential role of radiomics. HYPOTHESIS: We hypothesized that convolutional neural network (CNN) can be used to predict Oncotype Dx RS using an MRI dataset. STUDY TYPE: Institutional Review Board (IRB)-approved retrospective study from January 2010 to June 2016. POPULATION: In all, 134 patients with ER+/HER2- invasive ductal carcinoma who underwent both breast MRI and Oncotype Dx RS evaluation. Patients were classified into three groups: low risk (group 1, RS <18), intermediate risk (group 2, RS 18-30), and high risk (group 3, RS >30). FIELD STRENGTH/SEQUENCE: 1.5T and 3.0T. Breast MRI, T1 postcontrast. ASSESSMENT: Each breast tumor underwent 3D segmentation. In all, 1649 volumetric slices in 134 tumors (mean 12.3 slices/tumor) were evaluated. A CNN consisted of four convolutional layers and max-pooling layers. Dropout at 50% was applied to the second to last fully connected layer to prevent overfitting. Three-class prediction (group 1 vs. group 2 vs. group 3) and two-class prediction (group 1 vs. group 2/3) models were performed. STATISTICAL TESTS: A 5-fold crossvalidation test was performed using 80% training and 20% testing. Diagnostic accuracy, sensitivity, specificity, and receiver operating characteristic (ROC) area under the curve (AUC) were evaluated. RESULTS: The CNN achieved an overall accuracy of 81% (95% confidence interval [CI] ± 4%) in three-class prediction with specificity 90% (95% CI ± 5%), sensitivity 60% (95% CI ± 6%), and the area under the ROC curve was 0.92 (SD, 0.01). The CNN achieved an overall accuracy of 84% (95% CI ± 5%) in two-class prediction with specificity 81% (95% CI ± 4%), sensitivity 87% (95% CI ± 5%), and the area under the ROC curve was 0.92 (SD, 0.01). DATA CONCLUSION: It is feasible for current deep CNN architecture to be trained to predict Oncotype DX RS. LEVEL OF EVIDENCE: 4 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;49:518-524.

Global Trend in Artificial Intelligence-Based Publications in Radiology From 2000 to 2018.

OBJECTIVE. The purpose of this study is to evaluate the global trend in artificial intelligence (AI)-based research productivity involving radiology and its subspecialty disciplines. CONCLUSION. The United States is the global leader in AI radiology publication productivity, accounting for almost half of total radiology AI output. Other countries have increased their productivity. Notably, China has increased its productivity exponentially to close to 20% of all AI publications. The top three most productive radiology subspecialties were neuroradiology, body and chest, and nuclear medicine.

Prediction of Lymph Node Maximum Standardized Uptake Value in Patients With Cancer Using a 3D Convolutional Neural Network: A Proof-of-Concept Study.

OBJECTIVE: The purpose of this study is to determine whether a convolutional neural network (CNN) can predict the maximum standardized uptake value (SUVmax) of lymph nodes in patients with cancer using the unenhanced CT images from a PET/CT examination, thus providing a proof of concept for potentially using deep learning to diagnose nodal involvement. MATERIALS AND METHODS: Consecutive initial staging PET/CT scans obtained in 2017 for patients with pathologically proven malignancy were collected. Two blinded radiologists selected one to 10 lymph nodes from the unenhanced CT portion of each PET/CT examination. The SUVmax of the lymph nodes was recorded. Lymph nodes were cropped and used with the primary tumor histology type as input for a novel 3D CNN with predicted SUVmax as the output. The CNN was trained using one cohort and tested using a separate cohort. An SUVmax of 2.5 or greater was defined as FDG avid. Two blinded radiologists separately classified lymph nodes as FDG avid or not FDG avid on the basis of unenhanced CT images and separately using a short-axis measurement cutoff of 1 cm. Logistic regression analysis was performed. RESULTS: A total of 400 lymph nodes (median SUVmax, 6.8 [interquartile range {IQR}, 2.7-11.6]; median short-axis, 1.1 cm [IQR, 0.9-1.6 cm]) in 136 patients were used for training. A total of 164 lymph nodes (median SUVmax, 3.5 [IQR, 1.9-8.6]; median short-axis, 1.0 cm [IQR, 0.7-1.4 cm]) in 49 patients were used for testing. The predicted SUVmax was associated with the real SUVmax (ß estimate = 0.83, p < 0.0001). The predicted SUVmax was associated with FDG avidity (p < 0.0001), with an ROC AUC value of 0.85, and it improved when combined with radiologist qualitative assessment and short-axis criteria. CONCLUSION: A CNN is able to predict with moderate accuracy the SUVmax of lymph nodes, as determined from the unenhanced CT images and tumor histology subtype for patients with cancer.

Accuracy of Distinguishing Atypical Ductal Hyperplasia From Ductal Carcinoma In Situ With Convolutional Neural Network-Based Machine Learning Approach Using Mammographic Image Data.

OBJECTIVE. The purpose of this study was to test the hypothesis that convolutional neural networks can be used to predict which patients with pure atypical ductal hyperplasia (ADH) may be safely monitored rather than undergo surgery. MATERIALS AND METHODS. A total of 298 unique images from 149 patients were used for our convolutional neural network algorithm. A total of 134 images from 67 patients with ADH that had been diagnosed by stereotactic-guided biopsy of calcifications but had not been upgraded to ductal carcinoma in situ or invasive cancer at the time of surgical excision. A total of 164 images from 82 patients with mammographic calcifications indicated that ductal carcinoma in situ was the final diagnosis. Two standard mammographic magnification views of the calcifications (a craniocaudal view and a mediolateral or lateromedial view) were used for analysis. Calcifications were segmented using an open-source software platform and images were resized to fit a bounding box of 128 × 128 pixels. A topology with 15 hidden layers was used to implement the convolutional neural network. The network architecture contained five residual layers and dropout of 0.25 after each convolution. Patients were randomly separated into a training-and-validation set (80% of patients) and a test set (20% of patients). Code was implemented using open-source software on a workstation with an open-source operating system and a graphics card. RESULTS. The AUC value was 0.86 (95% CI, ± 0.03) for the test set. Aggregate sensitivity and specificity were 84.6% (95% CI, ± 4.0%) and 88.2% (95% CI, ± 3.0%), respectively. Diagnostic accuracy was 86.7% (95% CI, ± 2.9). CONCLUSION. It is feasible to apply convolutional neural networks to distinguish pure atypical ductal hyperplasia from ductal carcinoma in situ with the use of mammographic images. A larger dataset will likely result in further improvement of our prediction model.

Fully Automated Convolutional Neural Network Method for Quantification of Breast MRI Fibroglandular Tissue and Background Parenchymal Enhancement.

The aim of this study is to develop a fully automated convolutional neural network (CNN) method for quantification of breast MRI fibroglandular tissue (FGT) and background parenchymal enhancement (BPE). An institutional review board-approved retrospective study evaluated 1114 breast volumes in 137 patients using T1 precontrast, T1 postcontrast, and T1 subtraction images. First, using our previously published method of quantification, we manually segmented and calculated the amount of FGT and BPE to establish ground truth parameters. Then, a novel 3D CNN modified from the standard 2D U-Net architecture was developed and implemented for voxel-wise prediction whole breast and FGT margins. In the collapsing arm of the network, a series of 3D convolutional filters of size 3 × 3 × 3 are applied for standard CNN hierarchical feature extraction. To reduce feature map dimensionality, a 3 × 3 × 3 convolutional filter with stride 2 in all directions is applied; a total of 4 such operations are used. In the expanding arm of the network, a series of convolutional transpose filters of size 3 × 3 × 3 are used to up-sample each intermediate layer. To synthesize features at multiple resolutions, connections are introduced between the collapsing and expanding arms of the network. L2 regularization was implemented to prevent over-fitting. Cases were separated into training (80%) and test sets (20%). Fivefold cross-validation was performed. Software code was written in Python using the TensorFlow module on a Linux workstation with NVIDIA GTX Titan X GPU. In the test set, the fully automated CNN method for quantifying the amount of FGT yielded accuracy of 0.813 (cross-validation Dice score coefficient) and Pearson correlation of 0.975. For quantifying the amount of BPE, the CNN method yielded accuracy of 0.829 and Pearson correlation of 0.955. Our CNN network was able to quantify FGT and BPE within an average of 0.42 s per MRI case. A fully automated CNN method can be utilized to quantify MRI FGT and BPE. Larger dataset will likely improve our model.

Prior to Initiation of Chemotherapy, Can We Predict Breast Tumor Response? Deep Learning Convolutional Neural Networks Approach Using a Breast MRI Tumor Dataset.

We hypothesize that convolutional neural networks (CNN) can be used to predict neoadjuvant chemotherapy (NAC) response using a breast MRI tumor dataset prior to initiation of chemotherapy. An institutional review board-approved retrospective review of our database from January 2009 to June 2016 identified 141 locally advanced breast cancer patients who (1) underwent breast MRI prior to the initiation of NAC, (2) successfully completed adriamycin/taxane-based NAC, and (3) underwent surgical resection with available final surgical pathology data. Patients were classified into three groups based on their NAC response confirmed on final surgical pathology: complete (group 1), partial (group 2), and no response/progression (group 3). A total of 3107 volumetric slices of 141 tumors were evaluated. Breast tumor was identified on first T1 postcontrast dynamic images and underwent 3D segmentation. CNN consisted of ten convolutional layers, four max-pooling layers, and dropout of 50% after a fully connected layer. Dropout, augmentation, and L2 regularization were implemented to prevent overfitting of data. Non-linear functions were modeled by a rectified linear unit (ReLU). Batch normalization was used between the convolutional and ReLU layers to limit drift of layer activations during training. A three-class neoadjuvant prediction model was evaluated (group 1, group 2, or group 3). The CNN achieved an overall accuracy of 88% in three-class prediction of neoadjuvant treatment response. Three-class prediction discriminating one group from the other two was analyzed. Group 1 had a specificity of 95.1% ± 3.1%, sensitivity of 73.9% ± 4.5%, and accuracy of 87.7% ± 0.6%. Group 2 (partial response) had a specificity of 91.6% ± 1.3%, sensitivity of 82.4% ± 2.7%, and accuracy of 87.7% ± 0.6%. Group 3 (no response/progression) had a specificity of 93.4% ± 2.9%, sensitivity of 76.8% ± 5.7%, and accuracy of 87.8% ± 0.6%. It is feasible for current deep CNN architectures to be trained to predict NAC treatment response using a breast MRI dataset obtained prior to initiation of chemotherapy. Larger dataset will likely improve our prediction model.

Predicting Breast Cancer Molecular Subtype with MRI Dataset Utilizing Convolutional Neural Network Algorithm.

To develop a convolutional neural network (CNN) algorithm that can predict the molecular subtype of a breast cancer based on MRI features. An IRB-approved study was performed in 216 patients with available pre-treatment MRIs and immunohistochemical staining pathology data. First post-contrast MRI images were used for 3D segmentation using 3D slicer. A CNN architecture was designed with 14 layers. Residual connections were used in the earlier layers to allow stabilization of gradients during backpropagation. Inception style layers were utilized deeper in the network to allow learned segregation of more complex feature mappings. Extensive regularization was utilized including dropout, L2, feature map dropout, and transition layers. The class imbalance was addressed by doubling the input of underrepresented classes and utilizing a class sensitive cost function. Parameters were tuned based on a 20% validation group. A class balanced holdout set of 40 patients was utilized as the testing set. Software code was written in Python using the TensorFlow module on a Linux workstation with one NVidia Titan X GPU. Seventy-four luminal A, 106 luminal B, 13 HER2+, and 23 basal breast tumors were evaluated. Testing set accuracy was measured at 70%. The class normalized macro area under receiver operating curve (ROC) was measured at 0.853. Non-normalized micro-aggregated AUC was measured at 0.871, representing improved discriminatory power for the highly represented Luminal A and Luminal B subtypes. Aggregate sensitivity and specificity was measured at 0.603 and 0.958. MRI analysis of breast cancers utilizing a novel CNN can predict the molecular subtype of breast cancers. Larger data sets will likely improve our model.

Predicting Post Neoadjuvant Axillary Response Using a Novel Convolutional Neural Network Algorithm.

OBJECTIVES: In the postneoadjuvant chemotherapy (NAC) setting, conventional radiographic complete response (rCR) is a poor predictor of pathologic complete response (pCR) of the axilla. We developed a convolutional neural network (CNN) algorithm to better predict post-NAC axillary response using a breast MRI dataset. METHODS: An institutional review board-approved retrospective study from January 2009 to June 2016 identified 127 breast cancer patients who: (1) underwent breast MRI before the initiation of NAC; (2) successfully completed Adriamycin/Taxane-based NAC; and (3) underwent surgery, including sentinel lymph node evaluation/axillary lymph node dissection with final surgical pathology data. Patients were classified into pathologic complete response (pCR) of the axilla group and non-pCR group based on surgical pathology. Breast MRI performed before NAC was used. Tumor was identified on first T1 postcontrast images underwent 3D segmentation. A total of 2811 volumetric slices of 127 tumors were evaluated. CNN consisted of 10 convolutional layers, 4 max-pooling layers. Dropout, augmentation and L2 regularization were implemented to prevent overfitting of data. RESULTS: On final surgical pathology, 38.6% (49/127) of the patients achieved pCR of the axilla (group 1), and 61.4% (78/127) of the patients did not with residual metastasis detected (group 2). For predicting axillary pCR, our CNN algorithm achieved an overall accuracy of 83% (95% confidence interval [CI] ± 5) with sensitivity of 93% (95% CI ± 6) and specificity of 77% (95% CI ± 4). Area under the ROC curve (0.93, 95% CI ± 0.04). CONCLUSIONS: It is feasible to use CNN architecture to predict post NAC axillary pCR. Larger data set will likely improve our prediction model.

MABAL: a Novel Deep-Learning Architecture for Machine-Assisted Bone Age Labeling.

Bone age assessment (BAA) is a commonly performed diagnostic study in pediatric radiology to assess skeletal maturity. The most commonly utilized method for assessment of BAA is the Greulich and Pyle method (Pediatr Radiol 46.9:1269-1274, 2016; Arch Dis Child 81.2:172-173, 1999) atlas. The evaluation of BAA can be a tedious and time-consuming process for the radiologist. As such, several computer-assisted detection/diagnosis (CAD) methods have been proposed for automation of BAA. Classical CAD tools have traditionally relied on hard-coded algorithmic features for BAA which suffer from a variety of drawbacks. Recently, the advent and proliferation of convolutional neural networks (CNNs) has shown promise in a variety of medical imaging applications. There have been at least two published applications of using deep learning for evaluation of bone age (Med Image Anal 36:41-51, 2017; JDI 1-5, 2017). However, current implementations are limited by a combination of both architecture design and relatively small datasets. The purpose of this study is to demonstrate the benefits of a customized neural network algorithm carefully calibrated to the evaluation of bone age utilizing a relatively large institutional dataset. In doing so, this study will aim to show that advanced architectures can be successfully trained from scratch in the medical imaging domain and can generate results that outperform any existing proposed algorithm. The training data consisted of 10,289 images of different skeletal age examinations, 8909 from the hospital Picture Archiving and Communication System at our institution and 1383 from the public Digital Hand Atlas Database. The data was separated into four cohorts, one each for male and female children above the age of 8, and one each for male and female children below the age of 10. The testing set consisted of 20 radiographs of each 1-year-age cohort from 0 to 1 years to 14-15+ years, half male and half female. The testing set included left-hand radiographs done for bone age assessment, trauma evaluation without significant findings, and skeletal surveys. A 14 hidden layer-customized neural network was designed for this study. The network included several state of the art techniques including residual-style connections, inception layers, and spatial transformer layers. Data augmentation was applied to the network inputs to prevent overfitting. A linear regression output was utilized. Mean square error was used as the network loss function and mean absolute error (MAE) was utilized as the primary performance metric. MAE accuracies on the validation and test sets for young females were 0.654 and 0.561 respectively. For older females, validation and test accuracies were 0.662 and 0.497 respectively. For young males, validation and test accuracies were 0.649 and 0.585 respectively. Finally, for older males, validation and test set accuracies were 0.581 and 0.501 respectively. The female cohorts were trained for 900 epochs each and the male cohorts were trained for 600 epochs. An eightfold cross-validation set was employed for hyperparameter tuning. Test error was obtained after training on a full data set with the selected hyperparameters. Using our proposed customized neural network architecture on our large available data, we achieved an aggregate validation and test set mean absolute errors of 0.637 and 0.536 respectively. To date, this is the best published performance on utilizing deep learning for bone age assessment. Our results support our initial hypothesis that customized, purpose-built neural networks provide improved performance over networks derived from pre-trained imaging data sets. We build on that initial work by showing that the addition of state-of-the-art techniques such as residual connections and inception architecture further improves prediction accuracy. This is important because the current assumption for use of residual and/or inception architectures is that a large pre-trained network is required for successful implementation given the relatively small datasets in medical imaging. Instead we show that a small, customized architecture incorporating advanced CNN strategies can indeed be trained from scratch, yielding significant improvements in algorithm accuracy. It should be noted that for all four cohorts, testing error outperformed validation error. One reason for this is that our ground truth for our test set was obtained by averaging two pediatric radiologist reads compared to our training data for which only a single read was used. This suggests that despite relatively noisy training data, the algorithm could successfully model the variation between observers and generate estimates that are close to the expected ground truth.

Axillary Lymph Node Evaluation Utilizing Convolutional Neural Networks Using MRI Dataset.

The aim of this study is to evaluate the role of convolutional neural network (CNN) in predicting axillary lymph node metastasis, using a breast MRI dataset. An institutional review board (IRB)-approved retrospective review of our database from 1/2013 to 6/2016 identified 275 axillary lymph nodes for this study. Biopsy-proven 133 metastatic axillary lymph nodes and 142 negative control lymph nodes were identified based on benign biopsies (100) and from healthy MRI screening patients (42) with at least 3 years of negative follow-up. For each breast MRI, axillary lymph node was identified on first T1 post contrast dynamic images and underwent 3D segmentation using an open source software platform 3D Slicer. A 32 × 32 patch was then extracted from the center slice of the segmented tumor data. A CNN was designed for lymph node prediction based on each of these cropped images. The CNN consisted of seven convolutional layers and max-pooling layers with 50% dropout applied in the linear layer. In addition, data augmentation and L2 regularization were performed to limit overfitting. Training was implemented using the Adam optimizer, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. Code for this study was written in Python using the TensorFlow module (1.0.0). Experiments and CNN training were done on a Linux workstation with NVIDIA GTX 1070 Pascal GPU. Two class axillary lymph node metastasis prediction models were evaluated. For each lymph node, a final softmax score threshold of 0.5 was used for classification. Based on this, CNN achieved a mean five-fold cross-validation accuracy of 84.3%. It is feasible for current deep CNN architectures to be trained to predict likelihood of axillary lymph node metastasis. Larger dataset will likely improve our prediction model and can potentially be a non-invasive alternative to core needle biopsy and even sentinel lymph node evaluation.

Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses.

Saliency maps are popularly used to "explain" decisions made by modern machine learning models, including deep convolutional neural networks (DCNNs). While the resulting heatmaps purportedly indicate important image features, their "trustworthiness," i.e., utility and robustness, has not been evaluated for musculoskeletal imaging. The purpose of this study was to systematically evaluate the trustworthiness of saliency maps used in disease diagnosis on upper extremity X-ray images. The underlying DCNNs were trained using the Stanford MURA dataset. We studied four trustworthiness criteria-(1) localization accuracy of abnormalities, (2) repeatability, (3) reproducibility, and (4) sensitivity to underlying DCNN weights-across six different gradient-based saliency methods (Grad-CAM (GCAM), gradient explanation (GRAD), integrated gradients (IG), Smoothgrad (SG), smooth IG (SIG), and XRAI). Ground-truth was defined by the consensus of three fellowship-trained musculoskeletal radiologists who each placed bounding boxes around abnormalities on a holdout saliency test set. Compared to radiologists, all saliency methods showed inferior localization (AUPRCs: 0.438 (SG)-0.590 (XRAI); average radiologist AUPRC: 0.816), repeatability (IoUs: 0.427 (SG)-0.551 (IG); average radiologist IOU: 0.613), and reproducibility (IoUs: 0.250 (SG)-0.502 (XRAI); average radiologist IOU: 0.613) on abnormalities such as fractures, orthopedic hardware insertions, and arthritis. Five methods (GCAM, GRAD, IG, SG, XRAI) passed the sensitivity test. Ultimately, no saliency method met all four trustworthiness criteria; therefore, we recommend caution and rigorous evaluation of saliency maps prior to their clinical use.

Deep Learning-Assisted Identification of Femoroacetabular Impingement (FAI) on Routine Pelvic Radiographs.

To use a novel deep learning system to localize the hip joints and detect findings of cam-type femoroacetabular impingement (FAI). A retrospective search of hip/pelvis radiographs obtained in patients to evaluate for FAI yielded 3050 total studies. Each hip was classified separately by the original interpreting radiologist in the following manner: 724 hips had severe cam-type FAI morphology, 962 moderate cam-type FAI morphology, 846 mild cam-type FAI morphology, and 518 hips were normal. The anteroposterior (AP) view from each study was anonymized and extracted. After localization of the hip joints by a novel convolutional neural network (CNN) based on the focal loss principle, a second CNN classified the images of the hip as cam positive, or no FAI. Accuracy was 74% for diagnosing normal vs. abnormal cam-type FAI morphology, with aggregate sensitivity and specificity of 0.821 and 0.669, respectively, at the chosen operating point. The aggregate AUC was 0.736. A deep learning system can be applied to detect FAI-related changes on single view pelvic radiographs. Deep learning is useful for quickly identifying and categorizing pathology on imaging, which may aid the interpreting radiologist.