Multiscale deep learning framework captures systemic immune features in lymph nodes predictive of triple negative breast cancer outcome in large-scale studies.

The suggestion that the systemic immune response in lymph nodes (LNs) conveys prognostic value for triple-negative breast cancer (TNBC) patients has not previously been investigated in large cohorts. We used a deep learning (DL) framework to quantify morphological features in haematoxylin and eosin-stained LNs on digitised whole slide images. From 345 breast cancer patients, 5,228 axillary LNs, cancer-free and involved, were assessed. Generalisable multiscale DL frameworks were developed to capture and quantify germinal centres (GCs) and sinuses. Cox regression proportional hazard models tested the association between smuLymphNet-captured GC and sinus quantifications and distant metastasis-free survival (DMFS). smuLymphNet achieved a Dice coefficient of 0.86 and 0.74 for capturing GCs and sinuses, respectively, and was comparable to an interpathologist Dice coefficient of 0.66 (GC) and 0.60 (sinus). smuLymphNet-captured sinuses were increased in LNs harbouring GCs (p < 0.001). smuLymphNet-captured GCs retained clinical relevance in LN-positive TNBC patients whose cancer-free LNs had on average ≥2 GCs, had longer DMFS (hazard ratio [HR] = 0.28, p = 0.02) and extended GCs' prognostic value to LN-negative TNBC patients (HR = 0.14, p = 0.002). Enlarged smuLymphNet-captured sinuses in involved LNs were associated with superior DMFS in LN-positive TNBC patients in a cohort from Guy's Hospital (multivariate HR = 0.39, p = 0.039) and with distant recurrence-free survival in 95 LN-positive TNBC patients of the Dutch-N4plus trial (HR = 0.44, p = 0.024). Heuristic scoring of subcapsular sinuses in LNs of LN-positive Tianjin TNBC patients (n = 85) cross-validated the association of enlarged sinuses with shorter DMFS (involved LNs: HR = 0.33, p = 0.029 and cancer-free LNs: HR = 0.21 p = 0.01). Morphological LN features reflective of cancer-associated responses are robustly quantifiable by smuLymphNet. Our findings further strengthen the value of assessment of LN properties beyond the detection of metastatic deposits for prognostication of TNBC patients. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Efficient quality control of whole slide pathology images with human-in-the-loop training.

Histopathology whole slide images (WSIs) are being widely used to develop deep learning-based diagnostic solutions, especially for precision oncology. Most of these diagnostic softwares are vulnerable to biases and impurities in the training and test data which can lead to inaccurate diagnoses. For instance, WSIs contain multiple types of tissue regions, at least some of which might not be relevant to the diagnosis. We introduce HistoROI, a robust yet lightweight deep learning-based classifier to segregate WSI into 6 broad tissue regions-epithelium, stroma, lymphocytes, adipose, artifacts, and miscellaneous. HistoROI is trained using a novel human in-the-loop and active learning paradigm that ensures variations in training data for labeling efficient generalization. HistoROI consistently performs well across multiple organs, despite being trained on only a single dataset, demonstrating strong generalization. Further, we have examined the utility of HistoROI in improving the performance of downstream deep learning-based tasks using the CAMELYON breast cancer lymph node and TCGA lung cancer datasets. For the former dataset, the area under the receiver operating characteristic curve (AUC) for metastasis versus normal tissue of a neural network trained using weakly supervised learning increased from 0.88 to 0.92 by filtering the data using HistoROI. Similarly, the AUC increased from 0.88 to 0.93 for the classification between adenocarcinoma and squamous cell carcinoma on the lung cancer dataset. We also found that the performance of the HistoROI improves upon HistoQC for artifact detection on a test dataset of 93 annotated WSIs. The limitations of the proposed model are analyzed, and potential extensions are also discussed.

Author's Reply to "MoNuSAC2020: A Multi-Organ Nuclei Segmentation and Classification Challenge".

We had released MoNuSAC2020 as one of the largest publicly available, manually annotated, curated, multi-class, and multi-instance medical image segmentation datasets. Based on this dataset, we had organized a challenge at the International Symposium on Biomedical Imaging (ISBI) 2020. Along with the challenge participants, we had published an article summarizing the results and findings of the challenge (Verma et al., 2021). Foucart et al. (2022) in their "Analysis of the MoNuSAC 2020 challenge evaluation and results: metric implementation errors" have pointed ways in which the computation of the segmentation performance metric for the challenge can be corrected or improved. After a careful examination of their analysis, we have found a small bug in our code and an erroneous column-header swap in one of our result tables. Here, we present our response to their analysis, and issue an errata. After fixing the bug the challenge rankings remain largely unaffected. On the other hand, two of Foucart et al.'s other suggestions are good for future consideration, but it is not clear that those should be immediately implemented. We thank Foucart et al. for their detailed analysis to help us fix the two errors.

Robust Classification of Histology Images Exploiting Adversarial Auto Encoders.

Deep learning (DL) thrives on the availability of a large number of high quality images with reliable labels. Due to the large size of whole slide images in digital pathology, patches of manageable size are often mined for use in DL models. These patches are variable in quality, weakly supervised, individually less informative, and noisily labelled. To improve classification accuracy even with these noisy inputs and labels in histopathology, we propose a novel method for robust feature generation using an adversarial autoencoder (AAE). We utilize the likelihood of the features in the latent space of AAE as a criterion to weigh the training samples. We propose different weighting schemes for our framework and evaluate the effectiveness of our methods on the publically available BreakHis and BACH histopathology datasets. We observe consistent improvement in AUC scores using our methods, and conclude that robust supervision strategies should be further explored for computational pathology.

MoNuSAC2020: A Multi-Organ Nuclei Segmentation and Classification Challenge.

Detecting various types of cells in and around the tumor matrix holds a special significance in characterizing the tumor micro-environment for cancer prognostication and research. Automating the tasks of detecting, segmenting, and classifying nuclei can free up the pathologists' time for higher value tasks and reduce errors due to fatigue and subjectivity. To encourage the computer vision research community to develop and test algorithms for these tasks, we prepared a large and diverse dataset of nucleus boundary annotations and class labels. The dataset has over 46,000 nuclei from 37 hospitals, 71 patients, four organs, and four nucleus types. We also organized a challenge around this dataset as a satellite event at the International Symposium on Biomedical Imaging (ISBI) in April 2020. The challenge saw a wide participation from across the world, and the top methods were able to match inter-human concordance for the challenge metric. In this paper, we summarize the dataset and the key findings of the challenge, including the commonalities and differences between the methods developed by various participants. We have released the MoNuSAC2020 dataset to the public.

Deep Learning to Estimate Human Epidermal Growth Factor Receptor 2 Status from Hematoxylin and Eosin-Stained Breast Tissue Images.

CONTEXT: Several therapeutically important mutations in cancers are economically detected using immunohistochemistry (IHC), which highlights the overexpression of specific antigens associated with the mutation. However, IHC panels can be imprecise and relatively expensive in low-income settings. On the other hand, although hematoxylin and eosin (H&E) staining used to visualize the general tissue morphology is a routine and low cost, it does not highlight any specific antigen or mutation. AIMS: Using the human epidermal growth factor receptor 2 (HER2) mutation in breast cancer as an example, we strengthen the case for cost-effective detection and screening of overexpression of HER2 protein in H&E-stained tissue. SETTINGS AND DESIGN: We use computational methods that reliably detect subtle morphological changes associated with the over-expression of mutation-specific proteins directly from H&E images. SUBJECTS AND METHODS: We trained a classification pipeline to determine HER2 overexpression status of H&E stained whole slide images. Our training dataset was derived from a single hospital containing 26 (11 HER2+ and 15 HER2-) cases. We tested the classification pipeline on 26 (8 HER2+ and 18 HER2-) held-out cases from the same hospital and 45 independent cases (23 HER2+ and 22 HER2-) from the TCGA-BRCA cohort. The pipeline was composed of a stain separation module and three deep neural network modules in tandem for robustness and interpretability. STATISTICAL ANALYSIS USED: We evaluate our trained model through area under the curve (AUC)-receiver operating characteristic. RESULTS: Our pipeline achieved an AUC of 0.82 (confidence interval [CI]: 0.65-0.98) on held-out cases and an AUC of 0.76 (CI: 0.61-0.89) on the independent dataset from TCGA. We also demonstrate the region-level correspondence of HER2 overexpression between a patient's IHC and H&E serial sections. CONCLUSIONS: Our work strengthens the case for automatically quantifying the overexpression of mutation-specific proteins in H&E-stained digital pathology, and it highlights the importance of multi-stage machine learning pipelines for added robustness and interpretability.