Understanding metric-related pitfalls in image analysis validation.

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

ROOD-MRI: Benchmarking the robustness of deep learning segmentation models to out-of-distribution and corrupted data in MRI.

Deep artificial neural networks (DNNs) have moved to the forefront of medical image analysis due to their success in classification, segmentation, and detection challenges. A principal challenge in large-scale deployment of DNNs in neuroimage analysis is the potential for shifts in signal-to-noise ratio, contrast, resolution, and presence of artifacts from site to site due to variances in scanners and acquisition protocols. DNNs are famously susceptible to these distribution shifts in computer vision. Currently, there are no benchmarking platforms or frameworks to assess the robustness of new and existing models to specific distribution shifts in MRI, and accessible multi-site benchmarking datasets are still scarce or task-specific. To address these limitations, we propose ROOD-MRI: a novel platform for benchmarking the Robustness of DNNs to Out-Of-Distribution (OOD) data, corruptions, and artifacts in MRI. This flexible platform provides modules for generating benchmarking datasets using transforms that model distribution shifts in MRI, implementations of newly derived benchmarking metrics for image segmentation, and examples for using the methodology with new models and tasks. We apply our methodology to hippocampus, ventricle, and white matter hyperintensity segmentation in several large studies, providing the hippocampus dataset as a publicly available benchmark. By evaluating modern DNNs on these datasets, we demonstrate that they are highly susceptible to distribution shifts and corruptions in MRI. We show that while data augmentation strategies can substantially improve robustness to OOD data for anatomical segmentation tasks, modern DNNs using augmentation still lack robustness in more challenging lesion-based segmentation tasks. We finally benchmark U-Nets and vision transformers, finding robustness susceptibility to particular classes of transforms across architectures. The presented open-source platform enables generating new benchmarking datasets and comparing across models to study model design that results in improved robustness to OOD data and corruptions in MRI.

Sample-Size Determination Methodologies for Machine Learning in Medical Imaging Research: A Systematic Review.

PURPOSE: The required training sample size for a particular machine learning (ML) model applied to medical imaging data is often unknown. The purpose of this study was to provide a descriptive review of current sample-size determination methodologies in ML applied to medical imaging and to propose recommendations for future work in the field. METHODS: We conducted a systematic literature search of articles using Medline and Embase with keywords including "machine learning," "image," and "sample size." The search included articles published between 1946 and 2018. Data regarding the ML task, sample size, and train-test pipeline were collected. RESULTS: A total of 167 articles were identified, of which 22 were included for qualitative analysis. There were only 4 studies that discussed sample-size determination methodologies, and 18 that tested the effect of sample size on model performance as part of an exploratory analysis. The observed methods could be categorized as pre hoc model-based approaches, which relied on features of the algorithm, or post hoc curve-fitting approaches requiring empirical testing to model and extrapolate algorithm performance as a function of sample size. Between studies, we observed great variability in performance testing procedures used for curve-fitting, model assessment methods, and reporting of confidence in sample sizes. CONCLUSIONS: Our study highlights the scarcity of research in training set size determination methodologies applied to ML in medical imaging, emphasizes the need to standardize current reporting practices, and guides future work in development and streamlining of pre hoc and post hoc sample size approaches.

Deformable Registration for Longitudinal Breast MRI Screening.

MRI screening of high-risk patients for breast cancer provides very high sensitivity, but with a high recall rate and negative biopsies. Comparing the current exam to prior exams reduces the number of follow-up procedures requested by radiologists. Such comparison, however, can be challenging due to the highly deformable nature of breast tissues. Automated co-registration of multiple scans has the potential to aid diagnosis by providing 3D images for side-by-side comparison and also for use in CAD systems. Although many deformable registration techniques exist, they generally have a large number of parameters that need to be optimized and validated for each new application. Here, we propose a framework for such optimization and also identify the optimal input parameter set for registration of 3D T1-weighted MRI of breast using Elastix, a widely used and freely available registration tool. A numerical simulation study was first conducted to model the breast tissue and its deformation through finite element (FE) modeling. This model generated the ground truth for evaluating the registration accuracy by providing the deformation of each voxel in the breast volume. An exhaustive search was performed over various values of 7 registration parameters (4050 different combinations of parameters were assessed) and the optimum parameter set was determined. This study showed that there was a large variation in the registration accuracy of different parameter sets ranging from 0.29 mm to 2.50 mm in median registration error and 3.71 mm to 8.90 mm in 95 percentile of the registration error. Mean registration errors of 0.32 mm, 0.29 mm, and 0.30 mm and 95 percentile errors of 3.71 mm, 5.02 mm, and 4.70 mm were obtained by the three best parameter sets. The optimal parameter set was applied to consecutive breast MRI scans of 13 patients. A radiologist identified 113 landmark pairs (~ 11 per patient) which were used to assess registration accuracy. The results demonstrated that using the optimal registration parameter set, a registration accuracy (in mm) of 3.4 [1.8 6.8] was achieved.

Automatic cellularity assessment from post-treated breast surgical specimens.

Neoadjuvant treatment (NAT) of breast cancer (BCa) is an option for patients with the locally advanced disease. It has been compared with standard adjuvant therapy with the aim of improving prognosis and surgical outcome. Moreover, the response of the tumor to the therapy provides useful information for patient management. The pathological examination of the tissue sections after surgery is the gold-standard to estimate the residual tumor and the assessment of cellularity is an important component of tumor burden assessment. In the current clinical practice, tumor cellularity is manually estimated by pathologists on hematoxylin and eosin (H&E) stained slides, the quality, and reliability of which might be impaired by inter-observer variability which potentially affects prognostic power assessment in NAT trials. This procedure is also qualitative and time-consuming. In this paper, we describe a method of automatically assessing cellularity. A pipeline to automatically segment nuclei figures and estimate residual cancer cellularity from within patches and whole slide images (WSIs) of BCa was developed. We have compared the performance of our proposed pipeline in estimating residual cancer cellularity with that of two expert pathologists. We found an intra-class agreement coefficient (ICC) of 0.89 (95% CI of [0.70, 0.95]) between pathologists, 0.74 (95% CI of [0.70, 0.77]) between pathologist #1 and proposed method, and 0.75 (95% CI of [0.71, 0.79]) between pathologist #2 and proposed method. We have also successfully applied our proposed technique on a WSI to locate areas with high concentration of residual cancer. The main advantage of our approach is that it is fully automatic and can be used to find areas with high cellularity in WSIs. This provides a first step in developing an automatic technique for post-NAT tumor response assessment from pathology slides. © 2017 International Society for Advancement of Cytometry.

Improving the Accuracy of Computer-aided Diagnosis for Breast MR Imaging by Differentiating between Mass and Nonmass Lesions.

PURPOSE: To determine suitable features and optimal classifier design for a computer-aided diagnosis (CAD) system to differentiate among mass and nonmass enhancements during dynamic contrast material-enhanced magnetic resonance (MR) imaging of the breast. MATERIALS AND METHODS: Two hundred eighty histologically proved mass lesions and 129 histologically proved nonmass lesions from MR imaging studies were retrospectively collected. The institutional research ethics board approved this study and waived informed consent. Breast Imaging Reporting and Data System classification of mass and nonmass enhancement was obtained from radiologic reports. Image data from dynamic contrast-enhanced MR imaging were extracted and analyzed by using feature selection techniques and binary, multiclass, and cascade classifiers. Performance was assessed by measuring the area under the receiver operating characteristics curve (AUC), sensitivity, and specificity. Bootstrap cross validation was used to predict the best classifier for the classification task of mass and nonmass benign and malignant breast lesions. RESULTS: A total of 176 features were extracted. Feature relevance ranking indicated unequal importance of kinetic, texture, and morphologic features for mass and nonmass lesions. The best classifier performance was a two-stage cascade classifier (mass vs nonmass followed by malignant vs benign classification) (AUC, 0.91; 95% confidence interval (CI): 0.88, 0.94) compared with one-shot classifier (ie, all benign vs malignant classification) (AUC, 0.89; 95% CI: 0.85, 0.92). The AUC was 2% higher for cascade (median percent difference obtained by using paired bootstrapped samples) and was significant (P = .0027). Our proposed two-stage cascade classifier decreases the overall misclassification rate by 12%, with 72 of 409 missed diagnoses with cascade versus 82 of 409 missed diagnoses with one-shot classifier. CONCLUSION: Separately optimizing feature selection and training classifiers for mass and nonmass lesions improves the accuracy of CAD for breast MR imaging. By cascading classifiers, we achieved a significant improvement in performance with respect to the use of a one-shot classifier. Our cascaded classifier may provide an advantage for screening women at high risk for breast cancer, in whom the ability to diagnose cancers at an early stage is of primary importance.

A Metric for Reducing False Positives in the Computer-Aided Detection of Breast Cancer from Dynamic Contrast-Enhanced Magnetic Resonance Imaging Based Screening Examinations of High-Risk Women.

Magnetic resonance imaging (MRI)-enabled cancer screening has been shown to be a highly sensitive method for the early detection of breast cancer. Computer-aided detection systems have the potential to improve the screening process by standardizing radiologists to a high level of diagnostic accuracy. This retrospective study was approved by the institutional review board of Sunnybrook Health Sciences Centre. This study compares the performance of a proposed method for computer-aided detection (based on the second-order spatial derivative of the relative signal intensity) with the signal enhancement ratio (SER) on MRI-based breast screening examinations. Comparison is performed using receiver operating characteristic (ROC) curve analysis as well as free-response receiver operating characteristic (FROC) curve analysis. A modified computer-aided detection system combining the proposed approach with the SER method is also presented. The proposed method provides improvements in the rates of false positive markings over the SER method in the detection of breast cancer (as assessed by FROC analysis). The modified computer-aided detection system that incorporates both the proposed method and the SER method yields ROC results equal to that produced by SER while simultaneously providing improvements over the SER method in terms of false positives per noncancerous exam. The proposed method for identifying malignancies outperforms the SER method in terms of false positives on a challenging dataset containing many small lesions and may play a useful role in breast cancer screening by MRI as part of a computer-aided detection system.

A vector machine formulation with application to the computer-aided diagnosis of breast cancer from DCE-MRI screening examinations.

This study investigates the use of a proposed vector machine formulation with application to dynamic contrast-enhanced magnetic resonance imaging examinations in the context of the computer-aided diagnosis of breast cancer. This paper describes a method for generating feature measurements that characterize a lesion's vascular heterogeneity as well as a supervised learning formulation that represents an improvement over the conventional support vector machine in this application. Spatially varying signal-intensity measures were extracted from the examinations using principal components analysis and the machine learning technique known as the support vector machine (SVM) was used to classify the results. An alternative vector machine formulation was found to improve on the results produced by the established SVM in randomized bootstrap validation trials, yielding a receiver-operating characteristic curve area of 0.82 which represents a statistically significant improvement over the SVM technique in this application.

Feature selection in computer-aided breast cancer diagnosis via dynamic contrast-enhanced magnetic resonance images.

The accuracy of computer-aided diagnosis (CAD) for early detection and classification of breast cancer in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is dependent upon the features used by the CAD classifier. Here, we show that fast orthogonal search (FOS), which provides a more efficient iterative manner of computing stepwise regression feature selection, can select features with predictive value from a set of kinetic and texture candidate features computed from dynamic contrast-enhanced magnetic resonance images. FOS can in minutes search candidate feature sets of millions of terms, which may include cross-products of features up to second-, third- or fourth-order. This method is tested on a set of 83 DCE-MRI images, of which 20 are for cancerous and 63 for benign cases, using a leave-one-out trial. The features selected by FOS were used in a FOS predictor and nearest-neighbour predictor and had an area under the receiver operating curve (AUC) of 0.889 and 0.791 respectively. The FOS predictor AUC is significantly improved over the signal enhancement ratio predictor with an AUC of 0.706 (p = 0.0035 for the difference in the AUCs). Moreover, using FOS-selected features in a support vector machine increased the AUC over that resulting when the features were manually selected.

Removing non-nuclei information from histopathological images: A preprocessing step towards improving nuclei segmentation methods.

Disease interpretation by computer-aided diagnosis systems in digital pathology depends on reliable detection and segmentation of nuclei in hematoxylin and eosin (HE) images. These 2 tasks are challenging since appearance of both cell nuclei and background structures are very variable. This paper presents a method to improve nuclei detection and segmentation in HE images by removing tiles that only contain background information. The method divides each image into smaller patches and uses their projection to the noiselet space to capture different spatial features from non-nuclei background and nuclei structures. The noiselet features are clustered by a K-means algorithm and the resultant partition, defined by the cluster centroids, is herein named the noiselet code-book. A part of an image, a tile, is divided into patches and represented by the histogram of occurrences of the projected patches in the noiselet code-book. Finally, with these histograms, a classifier learns to differentiate between nuclei and non-nuclei tiles. By applying a conventional watershed-marked method to detect and segment nuclei, evaluation consisted in comparing pure watershed method against denoising-plus-watershed in an open database with 8 different types of tissues. The averaged F-score of nuclei detection improved from 0.830 to 0.86 and the dice score after segmentation increased from 0.701 to 0.723.

A radiomic biomarker for prognosis of resected colorectal cancer liver metastases generalizes across MRI contrast agents.

Introduction: Contrast-enhanced MRI is routinely performed as part of preoperative work-up for patients with Colorectal Cancer Liver Metastases (CRLM). Radiomic biomarkers depicting the characteristics of CRLMs in MRI have been associated with overall survival (OS) of patients, but the reproducibility and clinical applicability of these biomarkers are limited due to the variations in MRI protocols between hospitals. Methods: In this work, we propose a generalizable radiomic model for predicting OS of CRLM patients who received preoperative chemotherapy and delayed-phase contrast enhanced (DPCE) MRIs prior to hepatic resection. This retrospective two-center study included three DPCE MRI cohorts (n=221) collected between January 2006 and December 2012. A 10-minute delayed Gd-DO3A-butrol enhanced MRI discovery cohort was used to select features based on robustness across contrast agents, correlation with OS and pairwise Pearson correlation, and to train a logistic regression model that predicts 3-year OS. Results: The model was evaluated on a 10-minute delayed Gd-DO3A-butrol enhanced MRI validation cohort (n=121), a 20-minute delayed Gd-EOB-DTPA (n=72) cohort from the same institute, and a 5-minute delayed Gd-DTPA cohort (n=28) from an independent institute. Two features were selected: minor axis length and dependence variance. The radiomic signature model stratified high-risk and low-risk CRLM groups in the Gd-DO3Abutrol (HR = 6.29, p = .007), Gd-EOB-DTPA (HR = 3.54, p = .003) and Gd-DTPA (HR = 3.16, p = .04) validation cohorts. Discussion: While most existing MRI findings focus on a specific contrast agent, our study shows the potential of MRI features to be generalizable across main-stream contrast agents at delayed phase.

Targeting tumour-associated macrophages in hodgkin lymphoma using engineered extracellular matrix-mimicking cryogels.

Tumour-associated macrophages are linked with poor prognosis and resistance to therapy in Hodgkin lymphoma; however, there are no suitable preclinical models to identify macrophage-targeting therapeutics. We used primary human tumours to guide the development of a mimetic cryogel, wherein Hodgkin (but not Non-Hodgkin) lymphoma cells promoted primary human macrophage invasion. In an invasion inhibitor screen, we identified five drug hits that significantly reduced tumour-associated macrophage invasion: marimastat, batimastat, AS1517499, ruxolitinib, and PD-169316. Importantly, ruxolitinib has demonstrated recent success in Hodgkin lymphoma clinical trials. Both ruxolitinib and PD-169316 (a p38 mitogen-activated protein kinase (p38 MAPK) inhibitor) decreased the percent of M2-like macrophages; however, only PD-169316 enhanced the percentage of M1-like macrophages. We validated p38 MAPK as an anti-invasion drug target with five additional drugs using a high-content imaging platform. With our biomimetic cryogel, we modeled macrophage invasion in Hodgkin lymphoma and then used it for target discovery and drug screening, ultimately identifying potential future therapeutics.

Arterial input function calculation in dynamic contrast-enhanced MRI: an in vivo validation study using co-registered contrast-enhanced ultrasound imaging.

OBJECTIVES: Developing a method of separating intravascular contrast agent concentration to measure the arterial input function (AIF) in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) of tumours, and validating its performance in phantom and in vivo experiments. METHODS: A tissue-mimicking phantom was constructed to model leaky tumour vasculature and DCE-MR images of this phantom were acquired. An in vivo study was performed using tumour-bearing rabbits. Co-registered DCE-MRI and contrast-enhanced ultrasound (CEUS) images were acquired. An independent component analysis (ICA)-based method was developed to separate the intravascular component from DCE-MRI. Results were validated by comparing the time-intensity curves with the actual phantom and in vivo curves. RESULTS: Phantom study: the AIF extracted using ICA correlated well with the true intravascular curve. In vivo study: the AIFs extracted from DCE-MRI using ICA were very close to the true AIF. Intravascular component images were very similar to the CEUS images. The contrast onset times and initial wash-in slope of the ICA-derived AIF showed good agreement with the CEUS curves. CONCLUSION: ICA has the potential to separate the intravascular component from DCE-MRI. This could eliminate the requirement for contrast medium uptake measurements in a major artery and potentially result in more accurate pharmacokinetic parameters. KEY POINTS: • Tumour response to therapy can be inferred from pharmacokinetic parameters. • Arterial input function (AIF) is required for pharmacokinetic modelling of tumours. • Independent component analysis has the potential to measure AIF inside the tumour. • AIF measurement is validated using contrast enhanced ultrasound and phantoms.

BI-RADS BERT and Using Section Segmentation to Understand Radiology Reports.

Radiology reports are one of the main forms of communication between radiologists and other clinicians, and contain important information for patient care. In order to use this information for research and automated patient care programs, it is necessary to convert the raw text into structured data suitable for analysis. State-of-the-art natural language processing (NLP) domain-specific contextual word embeddings have been shown to achieve impressive accuracy for these tasks in medicine, but have yet to be utilized for section structure segmentation. In this work, we pre-trained a contextual embedding BERT model using breast radiology reports and developed a classifier that incorporated the embedding with auxiliary global textual features in order to perform section segmentation. This model achieved 98% accuracy in segregating free-text reports, sentence by sentence, into sections of information outlined in the Breast Imaging Reporting and Data System (BI-RADS) lexicon, which is a significant improvement over the classic BERT model without auxiliary information. We then evaluated whether using section segmentation improved the downstream extraction of clinically relevant information such as modality/procedure, previous cancer, menopausal status, purpose of exam, breast density, and breast MRI background parenchymal enhancement. Using the BERT model pre-trained on breast radiology reports, combined with section segmentation, resulted in an overall accuracy of 95.9% in the field extraction tasks. This is a 17% improvement, compared to an overall accuracy of 78.9% for field extraction with models using classic BERT embeddings and not using section segmentation. Our work shows the strength of using BERT in the analysis of radiology reports and the advantages of section segmentation by identifying the key features of patient factors recorded in breast radiology reports.

Cell segmentation for immunofluorescence multiplexed images using two-stage domain adaptation and weakly labeled data for pre-training.

Cellular profiling with multiplexed immunofluorescence (MxIF) images can contribute to a more accurate patient stratification for immunotherapy. Accurate cell segmentation of the MxIF images is an essential step. We propose a deep learning pipeline to train a Mask R-CNN model (deep network) for cell segmentation using nuclear (DAPI) and membrane (Na+K+ATPase) stained images. We used two-stage domain adaptation by first using a weakly labeled dataset followed by fine-tuning with a manually annotated dataset. We validated our method against manual annotations on three different datasets. Our method yields comparable results to the multi-observer agreement on an ovarian cancer dataset and improves on state-of-the-art performance on a publicly available dataset of mouse pancreatic tissues. Our proposed method, using a weakly labeled dataset for pre-training, showed superior performance in all of our experiments. When using smaller training sample sizes for fine-tuning, the proposed method provided comparable performance to that obtained using much larger training sample sizes. Our results demonstrate that using two-stage domain adaptation with a weakly labeled dataset can effectively boost system performance, especially when using a small training sample size. We deployed the model as a plug-in to CellProfiler, a widely used software platform for cellular image analysis.

Self-supervised driven consistency training for annotation efficient histopathology image analysis.

Training a neural network with a large labeled dataset is still a dominant paradigm in computational histopathology. However, obtaining such exhaustive manual annotations is often expensive, laborious, and prone to inter and intra-observer variability. While recent self-supervised and semi-supervised methods can alleviate this need by learning unsupervised feature representations, they still struggle to generalize well to downstream tasks when the number of labeled instances is small. In this work, we overcome this challenge by leveraging both task-agnostic and task-specific unlabeled data based on two novel strategies: (i) a self-supervised pretext task that harnesses the underlying multi-resolution contextual cues in histology whole-slide images to learn a powerful supervisory signal for unsupervised representation learning; (ii) a new teacher-student semi-supervised consistency paradigm that learns to effectively transfer the pretrained representations to downstream tasks based on prediction consistency with the task-specific unlabeled data. We carry out extensive validation experiments on three histopathology benchmark datasets across two classification and one regression based tasks, i.e., tumor metastasis detection, tissue type classification, and tumor cellularity quantification. Under limited-label data, the proposed method yields tangible improvements, which is close to or even outperforming other state-of-the-art self-supervised and supervised baselines. Furthermore, we empirically show that the idea of bootstrapping the self-supervised pretrained features is an effective way to improve the task-specific semi-supervised learning on standard benchmarks. Code and pretrained models are made available at: https://github.com/srinidhiPY/SSL_CR_Histo.

Missed Breast Cancers on MRI in High-Risk Patients: A Retrospective Case-Control Study.

Purpose: To determine if MRI features and molecular subtype influence the detectability of breast cancers on MRI in high-risk patients. Methods and Materials: Breast cancers in a high-risk population of 104 patients were diagnosed following MRI describing a BI-RADS 4-5 lesion. MRI characteristics at the time of diagnosis were compared with previous MRI, where a BI-RADS 1-2-3 lesion was described. Results: There were 77 false-negative MRIs. A total of 51 cancers were overlooked and 26 were misinterpreted. There was no association found between MRI characteristics, the receptor type and the frequency of missed cancers. The main factors for misinterpreted lesions were multiple breast lesions, prior biopsy/surgery and long-term stability. Lesions were mostly overlooked because of their small size and high background parenchymal enhancement. Among missed lesions, 50% of those with plateau kinetics on initial MRI changed for washout kinetics, and 65% of initially progressively enhancing lesions then showed plateau or washout kinetics. There were more basal-like tumours in BRCA1 carriers (50%) than in non-carriers (13%), p = 0.0001, OR = 6.714, 95% CI = [2.058-21.910]. The proportion of missed cancers was lower in BRCA carriers (59%) versus non-carriers (79%), p < 0.05, OR = 2.621, 95% CI = [1.02-6.74]. Conclusions: MRI characteristics or molecular subtype do not influence breast cancer detectability. Lesions in a post-surgical breast should be assessed with caution. Long-term stability does not rule out malignancy and multimodality evaluation is of added value. Lowering the biopsy threshold for lesions with an interval change in kinetics for a type 2 or 3 curve should be considered. There was a higher rate of interval cancers in BRCA 1 patients attributed to lesions more aggressive in nature.

Learning to segment images with classification labels.

Two of the most common tasks in medical imaging are classification and segmentation. Either task requires labeled data annotated by experts, which is scarce and expensive to collect. Annotating data for segmentation is generally considered to be more laborious as the annotator has to draw around the boundaries of regions of interest, as opposed to assigning image patches a class label. Furthermore, in tasks such as breast cancer histopathology, any realistic clinical application often includes working with whole slide images, whereas most publicly available training data are in the form of image patches, which are given a class label. We propose an architecture that can alleviate the requirements for segmentation-level ground truth by making use of image-level labels to reduce the amount of time spent on data curation. In addition, this architecture can help unlock the potential of previously acquired image-level datasets on segmentation tasks by annotating a small number of regions of interest. In our experiments, we show using only one segmentation-level annotation per class, we can achieve performance comparable to a fully annotated dataset.

Deep neural network models for computational histopathology: A survey.

Histopathological images contain rich phenotypic information that can be used to monitor underlying mechanisms contributing to disease progression and patient survival outcomes. Recently, deep learning has become the mainstream methodological choice for analyzing and interpreting histology images. In this paper, we present a comprehensive review of state-of-the-art deep learning approaches that have been used in the context of histopathological image analysis. From the survey of over 130 papers, we review the field's progress based on the methodological aspect of different machine learning strategies such as supervised, weakly supervised, unsupervised, transfer learning and various other sub-variants of these methods. We also provide an overview of deep learning based survival models that are applicable for disease-specific prognosis tasks. Finally, we summarize several existing open datasets and highlight critical challenges and limitations with current deep learning approaches, along with possible avenues for future research.

Intensity Augmentation to Improve Generalizability of Breast Segmentation Across Different MRI Scan Protocols.

OBJECTIVE: The segmentation of the breast from the chest wall is an important first step in the analysis of breast magnetic resonance images. 3D U-Nets have been shown to obtain high segmentation accuracy and appear to generalize well when trained on one scanner type and tested on another scanner, provided that a very similar MR protocol is used. There has, however, been little work addressing the problem of domain adaptation when image intensities or patient orientation differ markedly between the training set and an unseen test set. In this work we aim to address this domain shift problem. METHOD: We propose to apply extensive intensity augmentation in addition to geometric augmentation during training. We explored both style transfer and a novel intensity remapping approach as intensity augmentation strategies. For our experiments, we trained a 3D U-Net on T1-weighted scans. We tested our network on T2-weighted scans from the same dataset as well as on an additional independent test set acquired with a T1-weighted TWIST sequence and a different coil configuration. RESULTS: By applying intensity augmentation we increased segmentation performance for the T2-weighted scans from a Dice of 0.71 to 0.88. This performance is very close to the baseline performance of training with T2-weighted scans (0.92). On the T1-weighted dataset we obtained a performance increase from 0.77 to 0.85. CONCLUSION: Our results show that the proposed intensity augmentation increases segmentation performance across different datasets. SIGNIFICANCE: The proposed method can improve whole breast segmentation of clinical MR scans acquired with different protocols.