Image-based assessment of extracellular mucin-to-tumor area predicts consensus molecular subtypes (CMS) in colorectal cancer.

The backbone of all colorectal cancer classifications including the consensus molecular subtypes (CMS) highlights microsatellite instability (MSI) as a key molecular pathway. Although mucinous histology (generally defined as >50% extracellular mucin-to-tumor area) is a "typical" feature of MSI, it is not limited to this subgroup. Here, we investigate the association of CMS classification and mucin-to-tumor area quantified using a deep learning algorithm, and  the expression of specific mucins in predicting CMS groups and clinical outcome. A weakly supervised segmentation method was developed to quantify extracellular mucin-to-tumor area in H&E images. Performance was compared to two pathologists' scores, then applied to two cohorts: (1) TCGA (n = 871 slides/412 patients) used for mucin-CMS group correlation and (2) Bern (n = 775 slides/517 patients) for histopathological correlations and next-generation Tissue Microarray construction. TCGA and CPTAC (n = 85 patients) were used to further validate mucin detection and CMS classification by gene and protein expression analysis for MUC2, MUC4, MUC5AC and MUC5B. An excellent inter-observer agreement between pathologists' scores and the algorithm was obtained (ICC = 0.92). In TCGA, mucinous tumors were predominantly CMS1 (25.7%), CMS3 (24.6%) and CMS4 (16.2%). Average mucin in CMS2 was 1.8%, indicating negligible amounts. RNA and protein expression of MUC2, MUC4, MUC5AC and MUC5B were low-to-absent in CMS2. MUC5AC protein expression correlated with aggressive tumor features (e.g., distant metastases (p = 0.0334), BRAF mutation (p < 0.0001), mismatch repair-deficiency (p < 0.0001), and unfavorable 5-year overall survival (44% versus 65% for positive/negative staining). MUC2 expression showed the opposite trend, correlating with less lymphatic (p = 0.0096) and venous vessel invasion (p = 0.0023), no impact on survival.The absence of mucin-expressing tumors in CMS2 provides an important phenotype-genotype correlation. Together with MSI, mucinous histology may help predict CMS classification using only histopathology and should be considered in future image classifiers of molecular subtypes.

Impact of scanner variability on lymph node segmentation in computational pathology.

Computer-aided diagnostics in histopathology are based on the digitization of glass slides. However, heterogeneity between the images generated by different slide scanners can unfavorably affect the performance of computational algorithms. Here, we evaluate the impact of scanner variability on lymph node segmentation due to its clinical importance in colorectal cancer diagnosis. 100 slides containing 276 lymph nodes were digitized using 4 different slide scanners, and 50 of the lymph nodes containing metastatic cancer cells. These 400 scans were subsequently annotated by 2 experienced pathologists to precisely label lymph node boundary. Three different segmentation methods were then applied and compared: Hematoxylin-channel-based thresholding (HCT), Hematoxylin-based active contours (HAC), and a convolution neural network (U-Net). Evaluation of U-Net trained from both a single scanner and an ensemble of all scanners was completed. Mosaic images based on representative tiles from a scanner were used as a reference image to normalize the new data from different test scanners to evaluate the performance of a pre-trained model. Fine-tuning was carried out by using weights of a model trained on one scanner to initialize model weights for other scanners. To evaluate the domain generalization, domain adversarial learning and stain mix-up augmentation were also implemented. Results show that fine-tuning and domain adversarial learning decreased the impact of scanner variability and greatly improved segmentation across scanners. Overall, U-Net with stain mix-up (Matthews correlation coefficient (MCC) = 0.87), domain adversarial learning (MCC = 0.86), and HAC (MCC = 0.87) were shown to outperform HCT (MCC = 0.81) for segmentation of lymph nodes when compared against the ground truth. The findings of this study should be considered for future algorithms applied in diagnostic routines.

Classification of colorectal tissue images from high throughput tissue microarrays by ensemble deep learning methods.

Tissue microarray (TMA) core images are a treasure trove for artificial intelligence applications. However, a common problem of TMAs is multiple sectioning, which can change the content of the intended tissue core and requires re-labelling. Here, we investigate different ensemble methods for colorectal tissue classification using high-throughput TMAs. Hematoxylin and Eosin (H&E) core images of 0.6 mm or 1.0 mm diameter from three international cohorts were extracted from 54 digital slides (n = 15,150 cores). After TMA core extraction and color enhancement, five different flows of independent and ensemble deep learning were applied. Training and testing data with 2144 and 13,006 cores included three classes: tumor, normal or "other" tissue. Ground-truth data were collected from 30 ngTMA slides (n = 8689 cores). A test augmentation is applied to reduce the uncertain prediction. Predictive accuracy of the best method, namely Soft Voting Ensemble of one VGG and one CapsNet models was 0.982, 0.947 and 0.939 for normal, "other" and tumor, which outperformed to independent or ensemble learning with one base-estimator. Our high-accuracy algorithm for colorectal tissue classification in high-throughput TMAs is amenable to images from different institutions, core sizes and stain intensity. It helps to reduce error in TMA core evaluations with previously given labels.

Multi-view convolutional neural networks for automated ocular structure and tumor segmentation in retinoblastoma.

In retinoblastoma, accurate segmentation of ocular structure and tumor tissue is important when working towards personalized treatment. This retrospective study serves to evaluate the performance of multi-view convolutional neural networks (MV-CNNs) for automated eye and tumor segmentation on MRI in retinoblastoma patients. Forty retinoblastoma and 20 healthy-eyes from 30 patients were included in a train/test (N = 29 retinoblastoma-, 17 healthy-eyes) and independent validation (N = 11 retinoblastoma-, 3 healthy-eyes) set. Imaging was done using 3.0 T Fast Imaging Employing Steady-state Acquisition (FIESTA), T2-weighted and contrast-enhanced T1-weighted sequences. Sclera, vitreous humour, lens, retinal detachment and tumor were manually delineated on FIESTA images to serve as a reference standard. Volumetric and spatial performance were assessed by calculating intra-class correlation (ICC) and dice similarity coefficient (DSC). Additionally, the effects of multi-scale, sequences and data augmentation were explored. Optimal performance was obtained by using a three-level pyramid MV-CNN with FIESTA, T2 and T1c sequences and data augmentation. Eye and tumor volumetric ICC were 0.997 and 0.996, respectively. Median [Interquartile range] DSC for eye, sclera, vitreous, lens, retinal detachment and tumor were 0.965 [0.950-0.975], 0.847 [0.782-0.893], 0.975 [0.930-0.986], 0.909 [0.847-0.951], 0.828 [0.458-0.962] and 0.914 [0.852-0.958], respectively. MV-CNN can be used to obtain accurate ocular structure and tumor segmentations in retinoblastoma.

Personalized Anatomic Eye Model From T1-Weighted Volume Interpolated Gradient Echo Magnetic Resonance Imaging of Patients With Uveal Melanoma.

PURPOSE: We present a 3-dimensional patient-specific eye model from magnetic resonance imaging (MRI) for proton therapy treatment planning of uveal melanoma (UM). During MRI acquisition of UM patients, the point fixation can be difficult and, together with physiological blinking, can introduce motion artifacts in the images, thus challenging the model creation. Furthermore, the unclear boundary of the small objects (eg, lens, optic nerve) near the muscle or of the tumors with hemorrhage and tantalum clips can limit model accuracy. METHODS AND MATERIALS: A dataset of 37 subjects, including 30 healthy eyes of volunteers and 7 eyes of UM patients, was investigated. In our previous work, active shape model was successfully applied to retinoblastoma eye segmentation in T1-weighted 3T MRI. Here, we evaluate this method in a more challenging setting, based on 1.5T MRI acquisition and different datasets of awake adult eyes with UM. The lens and cornea together with the sclera, vitreous humor, and optic nerve were automatically segmented and validated against manual delineations of a senior ocular radiation oncologist, in terms of the Dice similarity coefficient and Hausdorff distance. RESULTS: Leave-one-out cross validation (mixing both volunteers and UM patients) yielded median Dice similarity coefficient values (respective of Hausdorff distance) of 94.5% (1.64 mm) for the sclera, 92.2% (1.73 mm) for the vitreous humor, 88.3% (1.09 mm) for the lens, and 81.9% (1.86 mm) for the optic nerve. The average computation time for an eye was 10 seconds. CONCLUSIONS: To our knowledge, our work is the first attempt to automatically segment adult eyes, including patients with UM. Our results show that automated active shape model segmentation can succeed in the presence of motion, tumors, and tantalum clips. These results are promising for inclusion in clinical practice.

Segmentation, Separation and Pose Estimation of Prostate Brachytherapy Seeds in CT Images.

GOAL: In this paper, we address the development of an automatic approach for the computation of pose information (position + orientation) of prostate brachytherapy loose seeds from 3-D CT images. METHODS: From an initial detection of a set of seed candidates in CT images using a threshold and connected component method, the orientation of each individual seed is estimated by using the principal components analysis method. The main originality of this approach is the ability to classify the detected objects based on a priori intensity and volume information and to separate groups of closely spaced seeds using three competing clustering methods: the standard and a modified k-means method and a Gaussian mixture model with an expectation-maximization algorithm. Experiments were carried out on a series of CT images of two phantoms and patients. The fourteen patients correspond to a total of 1063 implanted seeds. Detections are compared to manual segmentation and to related work in terms of detection performance and calculation time. RESULTS: This automatic method has proved to be accurate and fast including the ability to separate groups of seeds in a reliable way and to determine the orientation of each seed. SIGNIFICANCE: Such a method is mandatory to be able to compute precisely the real dose delivered to the patient postoperatively instead of assuming the alignment of seeds along the theoretical insertion direction of the brachytherapy needles.