An Efficient Deep Neural Network to Classify Large 3D Images With Small Objects.

3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they consist of 10x or 100x more pixels than their 2D counterparts. To be trained with high-resolution 3D images, convolutional neural networks resort to downsampling them or projecting them to 2D. We propose an effective alternative, a neural network that enables efficient classification of full-resolution 3D medical images. Compared to off-the-shelf convolutional neural networks, our network, 3D Globally-Aware Multiple Instance Classifier (3D-GMIC), uses 77.98%-90.05% less GPU memory and 91.23%-96.02% less computation. While it is trained only with image-level labels, without segmentation labels, it explains its predictions by providing pixel-level saliency maps. On a dataset collected at NYU Langone Health, including 85,526 patients with full-field 2D mammography (FFDM), synthetic 2D mammography, and 3D mammography, 3D-GMIC achieves an AUC of 0.831 (95% CI: 0.769-0.887) in classifying breasts with malignant findings using 3D mammography. This is comparable to the performance of GMIC on FFDM (0.816, 95% CI: 0.737-0.878) and synthetic 2D (0.826, 95% CI: 0.754-0.884), which demonstrates that 3D-GMIC successfully classified large 3D images despite focusing computation on a smaller percentage of its input compared to GMIC. Therefore, 3D-GMIC identifies and utilizes extremely small regions of interest from 3D images consisting of hundreds of millions of pixels, dramatically reducing associated computational challenges. 3D-GMIC generalizes well to BCS-DBT, an external dataset from Duke University Hospital, achieving an AUC of 0.848 (95% CI: 0.798-0.896).

Improving breast cancer diagnostics with deep learning for MRI.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) has a high sensitivity in detecting breast cancer but often leads to unnecessary biopsies and patient workup. We used a deep learning (DL) system to improve the overall accuracy of breast cancer diagnosis and personalize management of patients undergoing DCE-MRI. On the internal test set (n = 3936 exams), our system achieved an area under the receiver operating characteristic curve (AUROC) of 0.92 (95% CI: 0.92 to 0.93). In a retrospective reader study, there was no statistically significant difference (P = 0.19) between five board-certified breast radiologists and the DL system (mean ΔAUROC, +0.04 in favor of the DL system). Radiologists' performance improved when their predictions were averaged with DL's predictions [mean ΔAUPRC (area under the precision-recall curve), +0.07]. We demonstrated the generalizability of the DL system using multiple datasets from Poland and the United States. An additional reader study on a Polish dataset showed that the DL system was as robust to distribution shift as radiologists. In subgroup analysis, we observed consistent results across different cancer subtypes and patient demographics. Using decision curve analysis, we showed that the DL system can reduce unnecessary biopsies in the range of clinically relevant risk thresholds. This would lead to avoiding biopsies yielding benign results in up to 20% of all patients with BI-RADS category 4 lesions. Last, we performed an error analysis, investigating situations where DL predictions were mostly incorrect. This exploratory work creates a foundation for deployment and prospective analysis of DL-based models for breast MRI.

Differences between human and machine perception in medical diagnosis.

Deep neural networks (DNNs) show promise in image-based medical diagnosis, but cannot be fully trusted since they can fail for reasons unrelated to underlying pathology. Humans are less likely to make such superficial mistakes, since they use features that are grounded on medical science. It is therefore important to know whether DNNs use different features than humans. Towards this end, we propose a framework for comparing human and machine perception in medical diagnosis. We frame the comparison in terms of perturbation robustness, and mitigate Simpson's paradox by performing a subgroup analysis. The framework is demonstrated with a case study in breast cancer screening, where we separately analyze microcalcifications and soft tissue lesions. While it is inconclusive whether humans and DNNs use different features to detect microcalcifications, we find that for soft tissue lesions, DNNs rely on high frequency components ignored by radiologists. Moreover, these features are located outside of the region of the images found most suspicious by radiologists. This difference between humans and machines was only visible through subgroup analysis, which highlights the importance of incorporating medical domain knowledge into the comparison.

Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening.

We present a deep convolutional neural network for breast cancer screening exam classification, trained, and evaluated on over 200000 exams (over 1000000 images). Our network achieves an AUC of 0.895 in predicting the presence of cancer in the breast, when tested on the screening population. We attribute the high accuracy to a few technical advances. 1) Our network's novel two-stage architecture and training procedure, which allows us to use a high-capacity patch-level network to learn from pixel-level labels alongside a network learning from macroscopic breast-level labels. 2) A custom ResNet-based network used as a building block of our model, whose balance of depth and width is optimized for high-resolution medical images. 3) Pretraining the network on screening BI-RADS classification, a related task with more noisy labels. 4) Combining multiple input views in an optimal way among a number of possible choices. To validate our model, we conducted a reader study with 14 readers, each reading 720 screening mammogram exams, and show that our model is as accurate as experienced radiologists when presented with the same data. We also show that a hybrid model, averaging the probability of malignancy predicted by a radiologist with a prediction of our neural network, is more accurate than either of the two separately. To further understand our results, we conduct a thorough analysis of our network's performance on different subpopulations of the screening population, the model's design, training procedure, errors, and properties of its internal representations. Our best models are publicly available at https://github.com/nyukat/breast_cancer_classifier.

Comparison between qualitative and quantitative assessment of background parenchymal enhancement on breast MRI.

BACKGROUND: Potential clinical implications of the level of background parenchymal enhancement (BPE) on breast MRI are increasing. Currently, BPE is typically evaluated subjectively. Tests of concordance between subjective BPE assessment and computer-assisted quantified BPE have not been reported. PURPOSE OR HYPOTHESIS: To compare subjective radiologist assessment of BPE with objective quantified parenchymal enhancement (QPE). STUDY TYPE: Cross-sectional observational study. POPULATION: Between 7/24/2015 and 11/27/2015, 104 sequential patients (ages 23 - 81 years, mean 49 years) without breast cancer underwent breast MRI and were included in this study. FIELD STRENGTH/SEQUENCE: 3T; fat suppressed axial T2, axial T1, and axial fat suppressed T1 before and after intravenous contrast. ASSESSMENT: Four breast imagers graded BPE at 90 and 180 s after contrast injection on a 4-point scale (a-d). Fibroglandular tissue masks were generated using a phantom-validated segmentation algorithm, and were co-registered to pre- and postcontrast fat suppressed images to define the region of interest. QPE was calculated. STATISTICAL TESTS: Receiver operating characteristic (ROC) analyses and kappa coefficients (k) were used to compare subjective BPE with QPE. RESULTS: ROC analyses indicated that subjective BPE at 90 s was best predicted by quantified QPE ≤20.2 = a, 20.3-25.2 = b, 25.3-50.0 = c, >50.0 = d, and at 180 s by quantified QPE ≤ 32.2 = a, 32.3-38.3 = b, 38.4-74.5 = c, >74.5 = d. Agreement between subjective BPE and QPE was slight to fair at 90 s (k = 0.20-0.36) and 180 s (k = 0.19-0.28). At higher levels of QPE, agreement between subjective BPE and QPE significantly decreased for all four radiologists at 90 s (P ≤ 0.004) and for three of four radiologists at 180 s (P ≤ 0.004). DATA CONCLUSION: Radiologists were less consistent with QPE as QPE increased. LEVEL OF EVIDENCE: 3 Technical Efficacy: Stage 3 J. Magn. Reson. Imaging 2018;47:1685-1691.

Background parenchymal enhancement over exam time in patients with and without breast cancer.

PURPOSE: To compare background parenchymal enhancement (BPE) over time in patients with and without breast cancer. MATERIALS AND METHODS: This retrospective Institutional Review Board (IRB)-approved, Health Insurance Portability and Accountability Act (HIPAA)-compliant study included 116 women (25-84 years, mean 54 years) with breast cancer who underwent breast magnetic resonance imaging at 3T between 1/2/2009 and 12/29/2009 and 116 age and date-of-exam-matched women without breast cancer (23-84 years, mean 51 years). Two independent, blinded readers (R1, R2) recorded BPE (minimal, mild, moderate, marked) at three times (100, 210, and 320 seconds postcontrast). Subsequent cancers were diagnosed in 9/96 control patients with follow up (12.6-93.0 months, mean 63.6 months). Exact Mann-Whitney, Fisher's exact, and McNemar tests were performed. RESULTS: Mean BPE was not found to be different between patients with and without breast cancer at any time (P = 0.36-0.64). At time 2 as compared with time 1, there were significantly more patients, both with and without breast cancer, with BPE >minimal (R1: 90 vs. 41 [P < 0.001] and 81 vs. 36 [P < 0.001]; R2: 84 vs. 52 [P < 0.001] and 79 vs. 43 [P < 0.001]) and BPE >mild (R1: 59 vs. 10 [P < 0.001] and 47 vs. 13 [P < 0.001]; R2: 49 vs. 12 [P < 0.001] and 41 vs. 18 [P < 0.001]). BPE changes between times 2 and 3 were not significant (P = 0.083-1.0). Odds ratios for control patients developing breast cancer were significant only for R2 and ranged up to 7.67 (1.49, 39.5; P < 0.01) for BPE >mild at time 2. CONCLUSION: BPE changes between the first and second postcontrast scans and stabilizes thereafter in most patients. Further investigation into the most clinically relevant timepoint for BPE assessment is warranted. LEVEL OF EVIDENCE: 3 J. Magn. Reson. Imaging 2017;45:74-83.

Clinical applicability and relevance of fibroglandular tissue segmentation on routine T1 weighted breast MRI.

PURPOSE: To evaluate clinical applicability of fibroglandular tissue (FGT) segmentation on routine T1 weighted breast MRI and compare FGT quantification with radiologist assessment. METHODS: FGT was segmented on 232 breasts and quantified, and was assessed qualitatively by four breast imagers. RESULTS: FGT segmentation was successful in all 232 breasts. Agreement between radiologists and quantified FGT was moderate to substantial (kappa=0.52-0.67); lower quantified FGT was associated with disagreement between radiologists and quantified FGT (P≤0.002). CONCLUSIONS: FGT segmentation was successful using routine T1 weighted breast MRI. Radiologists were less consistent with quantified results in breasts with lower quantified FGT.

What Do Patients Tweet About Their Mammography Experience?

RATIONALE AND OBJECTIVE: The purpose of this study was to evaluate themes related to patients' experience in undergoing mammography, as expressed on Twitter. METHODS: A total of 464 tweets from July to December 2015 containing the hashtag #mammogram and relating to a patient's experience in undergoing mammography were reviewed. RESULTS: Of the tweets, 45.5% occurred before the mammogram compared to 49.6% that occurred afterward (remainder of tweets indeterminate). However, in patients undergoing their first mammogram, 32.8% occurred before the examination, whereas in those undergoing follow-up mammogram, 53.0% occurred before the examination. Identified themes included breast compression (24.4%), advising other patients to undergo screening (23.9%), recognition of the health importance of the examination (18.8%), the act of waiting (10.1%), relief regarding results (9.7%), reflection that the examination was not that bad (9.1%), generalized apprehension regarding the examination (8.2%), interactions with staff (8.0%), the gown (5.0%), examination costs or access (3.4%), offering or reaching out for online support from other patients (3.2%), perception of screening as a sign of aging (2.4%), and the waiting room or waiting room amenities (1.3%). Of the tweets, 31.9% contained humor, of which 56.1% related to compression. Themes that were more common in patients undergoing their first, rather than follow-up, mammogram included breast compression (16.4% vs 9.1%, respectively) and that the test was not that bad (26.2% vs 7.6%, respectively). CONCLUSION: Online social media provides a platform for women to share their experiences and reactions in undergoing mammography, including humor, positive reflections, and encouragement of others to undergo the examination. Social media thus warrants further evaluation as a potential tool to help foster greater adherence to screening guidelines.

The Patient Experience in Radiology: Observations From Over 3,500 Patient Feedback Reports in a Single Institution.

PURPOSE: To identify factors associated with the patient experience in radiology based on patient feedback reports from a single institution. METHODS: In a departmental patient experience committee initiative, all imaging outpatients are provided names and roles of all departmental employees with whom they interact, along with contact information for providing feedback after their appointment. All resulting feedback was recorded in a web-based database. A total of 3,675 patient comments over a 3-year period were assessed in terms of major themes. Roles of employees recognized within the patient comments were also assessed. RESULTS: Patient feedback comments most commonly related to professional staff behavior (74.5%) and wait times (11.9%), and less commonly related to a spectrum of other issues (comfort during the exam, quality of the facilities, access to information regarding the exam, patient privacy, medical records, the radiology report, billing). The most common attributes relating to staff behavior involved patients' perceptions of staff caring, professionalism, pleasantness, helpfulness, and efficiency. Employees most commonly recognized by the comments were the technologist (50.2%) and receptionist (31.6%) and much less often the radiologist (2.2%). No radiologist was in the top 10% of employees in terms of the number of comments received. CONCLUSION: Patients' comments regarding their experiences in undergoing radiologic imaging were largely influenced by staff behavior and communication (particularly relating to technologists and receptionists), as well as wait times, with radiologists having a far lesser immediate impact. Radiologists are encouraged to engage in activities that promote direct visibility to their patients and thereby combat risks of the perceived "invisible" radiologist.

Comparison of Whole-Body (18)F FDG PET/MR Imaging and Whole-Body (18)F FDG PET/CT in Terms of Lesion Detection and Radiation Dose in Patients with Breast Cancer.

Purpose To compare fluorine 18 ((18)F) fluorodeoxyglucose (FDG) combined positron emission tomography (PET) and magnetic resonance (MR) imaging with (18)F FDG combined PET and computed tomography (CT) in terms of organ-specific metastatic lesion detection and radiation dose in patients with breast cancer. Materials and Methods From July 2012 to October 2013, this institutional review board-approved HIPAA-compliant prospective study included 51 patients with breast cancer (50 women; mean age, 56 years; range, 32-76 years; one man; aged 70 years) who completed PET/MR imaging with diffusion-weighted and contrast material-enhanced sequences after unenhanced PET/CT. Written informed consent for study participation was obtained. Two independent readers for each modality recorded site and number of lesions. Imaging and clinical follow-up, with consensus in two cases, served as the reference standard. Results There were 242 distant metastatic lesions in 30 patients, 18 breast cancers in 17 patients, and 19 positive axillary nodes in eight patients. On a per-patient basis, PET/MR imaging with diffusion-weighted and contrast-enhanced sequences depicted distant (30 of 30 [100%] for readers 1 and 2) and axillary (eight of eight [100%] for reader 1, seven of eight [88%] for reader 2) metastatic disease at rates similar to those of unenhanced PET/CT (distant metastatic disease: 28 of 29 [96%] for readers 3 and 4, P = .50; axillary metastatic disease: seven of eight [88%] for readers 3 and 4, P > .99) and outperformed PET/CT in the detection of breast cancer (17 of 17 [100%] for readers 1 and 2 vs 11 of 17 [65%] for reader 3 and 10 of 17 [59%] for reader 4; P < .001). PET/MR imaging showed increased sensitivity for liver (40 of 40 [100%] for reader 1 and 32 of 40 [80%] for reader 2 vs 30 of 40 [75%] for reader 3 and 28 of 40 [70%] for reader 4; P < .001) and bone (105 of 107 [98%] for reader 1 and 102 of 107 [95%] for reader 2 vs 106 of 107 [99%] for reader 3 and 93 of 107 [87%] for reader 4; P = .012) metastases and revealed brain metastases in five of 51 (10%) patients. PET/CT trended toward increased sensitivity for lung metastases (20 of 23 [87%] for reader 1 and 17 of 23 [74%] for reader 2 vs 23 of 23 [100%] for reader 3 and 22 of 23 [96%] for reader 4; P = .065). Dose reduction averaged 50% (P < .001). Conclusion In patients with breast cancer, PET/MR imaging may yield better sensitivity for liver and possibly bone metastases but not for pulmonary metastases, as compared with that attained with PET/CT, at about half the radiation dose. (©) RSNA, 2016 Online supplemental material is available for this article.

Evaluation of a known breast cancer using an abbreviated breast MRI protocol: Correlation of imaging characteristics and pathology with lesion detection and conspicuity.

OBJECTIVE: This study evaluates use of an abbreviated magnetic resonance imaging protocol with T2-weighted imaging in detecting biopsy-proven unifocal breast cancer. MATERIALS AND METHODS: This is an institutional review board approved retrospective study of patients with biopsy-proven unifocal breast cancer (88% invasive; 12% in situ) undergoing magnetic resonance imaging. In three separate sessions, three breast imagers evaluated (1) T1-weighted non-contrast, post-contrast and post-contrast subtracted images, (2) T1-weighted images with clinical history and prior imaging, and (3) T1-weighted images and T2-weighted images with clinical history and prior imaging. Protocols were compared for cancer detection, reading time and lesion conspicuity. An independent breast radiologist retrospectively analyzed initial enhancement ratio of cancers and retrospectively reviewed lesion morphology and final pathology. RESULTS: All 107 cancers were identified at first protocol by at least one reader; five cancers were missed by either one or two readers. One cancer was missed by one reader at protocols two and three. Mean percentage detection for protocol one was 97.8%; protocol two, 99.4%, protocol three, 99.4%. T2-weighted images did not alter cancer detection but increased lesion conspicuity for 2/3 readers. 3/5 missed lesions were low grade cancers. Initial enhancement ratio was positively associated with increasing tumor grade (p=0.031) and pathology (p=0.002). Reader interpretation time decreased and lesion conspicuity increased as initial enhancement ratio increased. CONCLUSION: Abbreviated magnetic resonance imaging has high rate of detection for known breast cancer and short interpretation time. T2 weighted imaging increased lesion conspicuity without altering detection rate. Initial enhancement ratio correlated with invasive disease and tumor grade.

The service encounter in radiology: acing the "moments of truth" to achieve patient-centered care.

Radiologists are increasingly recognizing their role as direct service providers to patients and seeking to offer an exceptional patient experience as part of high-quality service delivery. Patients' perceptions of service delivery are derived from the chain of numerous individual real-time encounters that occur throughout their visit. These so-called "moments of truth" define the overall experience and form the lasting impression of the given practice in their mind. Providing excellent service can be difficult to achieve in practice given its intangible nature as well as the heterogeneity and unpredictability of the large number of patients, frontline staff, and environmental circumstances that define the patient experience. Thus, broad commitment and team effort among all members of a radiology practice are required. This article explores important areas to be considered by a radiology practice to ensure positive and meaningful patient experiences. Specific ways in which every member within the practice, including schedulers, receptionists, technologists, nurses, and radiologists, can contribute to achieving high-quality patient service are discussed. Examples of patient-oriented language that may be useful in particular scenarios in radiology practice are given. The role of the practice's physical facility, including all aspects of its aesthetics and amenities, as well as of Internet services, in shaping the patient experience is also described. Throughout this work, a proactive approach to promoting a service-oriented organizational culture is provided. By improving the patient experience, these strategies may serve to enhance patients' perceptions of radiology and radiologists.

Inter- and intrareader agreement for categorization of background parenchymal enhancement at baseline and after training.

OBJECTIVE: Background parenchymal enhancement (BPE) refers to enhancing fibroglandular tissue on initial contrast-enhanced MR images. BPE appears to impact the rate of abnormal MRI interpretation and may correlate with breast cancer risk. There are now minimal data as to the uniformity of radiologists' BPE assessments and no data as to whether training improves agreement. Therefore, for this study, we sought to assess interreader agreement for BPE at baseline and after dedicated training. MATERIALS AND METHODS: This study included 119 breast MRI examinations performed in 119 patients (mean age, 47 years; age range, 25-79 years) in 2008. One week before training, four fellowship-trained breast imagers with 2-12 years' experience independently recorded BPE on a 4-point scale as follows: 1 (minimal, ≤ 25%), 2 (mild, 26-50%), 3 (moderate, 51-75%), or 4 (marked, > 75%). The same 119 cases were reread in a new random order within 1 week and at least 3 weeks after training. Interreader agreement and intrareader agreement were assessed using kappa coefficients. RESULTS: With training, interreader agreement increased from fair (κ = 0.36) to moderate (κ = 0.48). Improvement was sustained at 3 weeks after training (κ = 0.45). Intrareader agreement between time points 2 and 3 (κ: mean, 0.79; range, 0.56-0.98) was greater than between time points 1 and 2 (κ: mean, 0.62; range, 0.45-0.84), indicating readers learned and retained. CONCLUSION: Initial interreader agreement for BPE was fair among breast radiologists but achieved sustained improvement with training, highlighting the importance of education and inclusion of standardized BPE categories in a reference atlas.