Continual learning strategies for cancer-independent detection of lymph node metastases.

Recently, large, high-quality public datasets have led to the development of convolutional neural networks that can detect lymph node metastases of breast cancer at the level of expert pathologists. Many cancers, regardless of the site of origin, can metastasize to lymph nodes. However, collecting and annotating high-volume, high-quality datasets for every cancer type is challenging. In this paper we investigate how to leverage existing high-quality datasets most efficiently in multi-task settings for closely related tasks. Specifically, we will explore different training and domain adaptation strategies, including prevention of catastrophic forgetting, for breast, colon and head-and-neck cancer metastasis detection in lymph nodes. Our results show state-of-the-art performance on colon and head-and-neck cancer metastasis detection tasks. We show the effectiveness of adaptation of networks from one cancer type to another to obtain multi-task metastasis detection networks. Furthermore, we show that leveraging existing high-quality datasets can significantly boost performance on new target tasks and that catastrophic forgetting can be effectively mitigated.Last, we compare different mitigation strategies.

Human scattered tubular cells represent a heterogeneous population of glycolytic dedifferentiated proximal tubule cells.

Scattered tubular cells (STCs) are a phenotypically distinct cell population in the proximal tubule that increase in number after acute kidney injury. We aimed to characterize the human STC population. Three-dimensional human tissue analysis revealed that STCs are preferentially located within inner bends of the tubule and are barely present in young kidney tissue (<2 years), and their number increases with age. Increased STC numbers were associated with acute tubular injury (kidney injury molecule 1) and interstitial fibrosis (alpha smooth muscle actin). Isolated CD13+ CD24- CD133- proximal tubule epithelial cells (PTECs) and CD13+ CD24+ and CD13+ CD133+ STCs were analyzed using RNA sequencing. Transcriptome analysis revealed an upregulation of nuclear factor κB, tumor necrosis factor alpha, and inflammatory pathways in STCs, whereas metabolism, especially the tricarboxylic acid cycle and oxidative phosphorylation, was downregulated, without showing signs of cellular senescence. Using immunostaining and a publicly available single-cell sequencing database of human kidneys, we demonstrate that STCs represent a heterogeneous population in a transient state. In conclusion, STCs are dedifferentiated PTECs showing a metabolic shift toward glycolysis, which could facilitate cellular survival after kidney injury. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Generalization of Deep Learning in Digital Pathology: Experience in Breast Cancer Metastasis Detection.

Poor generalizability is a major barrier to clinical implementation of artificial intelligence in digital pathology. The aim of this study was to test the generalizability of a pretrained deep learning model to a new diagnostic setting and to a small change in surgical indication. A deep learning model for breast cancer metastases detection in sentinel lymph nodes, trained on CAMELYON multicenter data, was used as a base model, and achieved an AUC of 0.969 (95% CI 0.926-0.998) and FROC of 0.838 (95% CI 0.757-0.913) on CAMELYON16 test data. On local sentinel node data, the base model performance dropped to AUC 0.929 (95% CI 0.800-0.998) and FROC 0.744 (95% CI 0.566-0.912). On data with a change in surgical indication (axillary dissections) the base model performance indicated an even larger drop with a FROC of 0.503 (95%CI 0.201-0.911). The model was retrained with addition of local data, resulting in about a 4% increase for both AUC and FROC for sentinel nodes, and an increase of 11% in AUC and 49% in FROC for axillary nodes. Pathologist qualitative evaluation of the retrained model´s output showed no missed positive slides. False positives, false negatives and one previously undetected micro-metastasis were observed. The study highlights the generalization challenge even when using a multicenter trained model, and that a small change in indication can considerably impact the model´s performance.

Convolutional Neural Networks for the Evaluation of Chronic and Inflammatory Lesions in Kidney Transplant Biopsies.

In kidney transplant biopsies, both inflammation and chronic changes are important features that predict long-term graft survival. Quantitative scoring of these features is important for transplant diagnostics and kidney research. However, visual scoring is poorly reproducible and labor intensive. The goal of this study was to investigate the potential of convolutional neural networks (CNNs) to quantify inflammation and chronic features in kidney transplant biopsies. A structure segmentation CNN and a lymphocyte detection CNN were applied on 125 whole-slide image pairs of periodic acid-Schiff- and CD3-stained slides. The CNN results were used to quantify healthy and sclerotic glomeruli, interstitial fibrosis, tubular atrophy, and inflammation within both nonatrophic and atrophic tubuli, and in areas of interstitial fibrosis. The computed tissue features showed high correlation with Banff lesion scores of five pathologists (A.A., A.Dend., J.H.B., J.K., and T.N.). Analyses on a small subset showed a moderate correlation toward higher CD3+ cell density within scarred regions and higher CD3+ cell count inside atrophic tubuli correlated with long-term change of estimated glomerular filtration rate. The presented CNNs are valid tools to yield objective quantitative information on glomeruli number, fibrotic tissue, and inflammation within scarred and non-scarred kidney parenchyma in a reproducible manner. CNNs have the potential to improve kidney transplant diagnostics and will benefit the community as a novel method to generate surrogate end points for large-scale clinical studies.

Parietal epithelial cells maintain the epithelial cell continuum forming Bowman's space in focal segmental glomerulosclerosis.

In the glomerulus, Bowman's space is formed by a continuum of glomerular epithelial cells. In focal segmental glomerulosclerosis (FSGS), glomeruli show segmental scarring, a result of activated parietal epithelial cells (PECs) invading the glomerular tuft. The segmental scars interrupt the epithelial continuum. However, non-sclerotic segments seem to be preserved even in glomeruli with advanced lesions. We studied the histology of the segmental pattern in Munich Wistar Frömter rats, a model for secondary FSGS. Our results showed that matrix layers lined with PECs cover the sclerotic lesions. These PECs formed contacts with podocytes of the uninvolved tuft segments, restoring the epithelial continuum. Formed Bowman's spaces were still connected to the tubular system. In biopsies of patients with secondary FSGS, we also detected matrix layers formed by PECs, separating the uninvolved from the sclerotic glomerular segments. PECs have a major role in the formation of glomerulosclerosis; we show here that in FSGS they also restore the glomerular epithelial cell continuum that surrounds Bowman's space. This process may be beneficial and indispensable for glomerular filtration in the uninvolved segments of sclerotic glomeruli.

Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks.

Modern pathology diagnostics is being driven toward large scale digitization of microscopic tissue sections. A prerequisite for its safe implementation is the guarantee that all tissue present on a glass slide can also be found back in the digital image. Whole-slide scanners perform a tissue segmentation in a low resolution overview image to prevent inefficient high-resolution scanning of empty background areas. However, currently applied algorithms can fail in detecting all tissue regions. In this study, we developed convolutional neural networks to distinguish tissue from background. We collected 100 whole-slide images of 10 tissue samples-staining categories from five medical centers for development and testing. Additionally, eight more images of eight unfamiliar categories were collected for testing only. We compared our fully-convolutional neural networks to three traditional methods on a range of resolution levels using Dice score and sensitivity. We also tested whether a single neural network can perform equivalently to multiple networks, each specialized in a single resolution. Overall, our solutions outperformed the traditional methods on all the tested resolutions. The resolution-agnostic network achieved average Dice scores between 0.97 and 0.98 across the tested resolution levels, only 0.0069 less than the resolution-specific networks. Finally, its excellent generalization performance was demonstrated by achieving averages of 0.98 Dice score and 0.97 sensitivity on the eight unfamiliar images. A future study should test this network prospectively.

Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology.

Stain variation is a phenomenon observed when distinct pathology laboratories stain tissue slides that exhibit similar but not identical color appearance. Due to this color shift between laboratories, convolutional neural networks (CNNs) trained with images from one lab often underperform on unseen images from the other lab. Several techniques have been proposed to reduce the generalization error, mainly grouped into two categories: stain color augmentation and stain color normalization. The former simulates a wide variety of realistic stain variations during training, producing stain-invariant CNNs. The latter aims to match training and test color distributions in order to reduce stain variation. For the first time, we compared some of these techniques and quantified their effect on CNN classification performance using a heterogeneous dataset of hematoxylin and eosin histopathology images from 4 organs and 9 pathology laboratories. Additionally, we propose a novel unsupervised method to perform stain color normalization using a neural network. Based on our experimental results, we provide practical guidelines on how to use stain color augmentation and stain color normalization in future computational pathology applications.

From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge.

Automated detection of cancer metastases in lymph nodes has the potential to improve the assessment of prognosis for patients. To enable fair comparison between the algorithms for this purpose, we set up the CAMELYON17 challenge in conjunction with the IEEE International Symposium on Biomedical Imaging 2017 Conference in Melbourne. Over 300 participants registered on the challenge website, of which 23 teams submitted a total of 37 algorithms before the initial deadline. Participants were provided with 899 whole-slide images (WSIs) for developing their algorithms. The developed algorithms were evaluated based on the test set encompassing 100 patients and 500 WSIs. The evaluation metric used was a quadratic weighted Cohen's kappa. We discuss the algorithmic details of the 10 best pre-conference and two post-conference submissions. All these participants used convolutional neural networks in combination with pre- and postprocessing steps. Algorithms differed mostly in neural network architecture, training strategy, and pre- and postprocessing methodology. Overall, the kappa metric ranged from 0.89 to -0.13 across all submissions. The best results were obtained with pre-trained architectures such as ResNet. Confusion matrix analysis revealed that all participants struggled with reliably identifying isolated tumor cells, the smallest type of metastasis, with detection rates below 40%. Qualitative inspection of the results of the top participants showed categories of false positives, such as nerves or contamination, which could be targeted for further optimization. Last, we show that simple combinations of the top algorithms result in higher kappa metric values than any algorithm individually, with 0.93 for the best combination.

Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard.

Given the importance of gland morphology in grading prostate cancer (PCa), automatically differentiating between epithelium and other tissues is an important prerequisite for the development of automated methods for detecting PCa. We propose a new deep learning method to segment epithelial tissue in digitised hematoxylin and eosin (H&E) stained prostatectomy slides using immunohistochemistry (IHC) as reference standard. We used IHC to create a precise and objective ground truth compared to manual outlining on H&E slides, especially in areas with high-grade PCa. 102 tissue sections were stained with H&E and subsequently restained with P63 and CK8/18 IHC markers to highlight epithelial structures. Afterwards each pair was co-registered. First, we trained a U-Net to segment epithelial structures in IHC using a subset of the IHC slides that were preprocessed with color deconvolution. Second, this network was applied to the remaining slides to create the reference standard used to train a second U-Net on H&E. Our system accurately segmented both intact glands and individual tumour epithelial cells. The generalisation capacity of our system is shown using an independent external dataset from a different centre. We envision this segmentation as the first part of a fully automated prostate cancer grading pipeline.

1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset.

Background: The presence of lymph node metastases is one of the most important factors in breast cancer prognosis. The most common way to assess regional lymph node status is the sentinel lymph node procedure. The sentinel lymph node is the most likely lymph node to contain metastasized cancer cells and is excised, histopathologically processed, and examined by a pathologist. This tedious examination process is time-consuming and can lead to small metastases being missed. However, recent advances in whole-slide imaging and machine learning have opened an avenue for analysis of digitized lymph node sections with computer algorithms. For example, convolutional neural networks, a type of machine-learning algorithm, can be used to automatically detect cancer metastases in lymph nodes with high accuracy. To train machine-learning models, large, well-curated datasets are needed. Results: We released a dataset of 1,399 annotated whole-slide images (WSIs) of lymph nodes, both with and without metastases, in 3 terabytes of data in the context of the CAMELYON16 and CAMELYON17 Grand Challenges. Slides were collected from five medical centers to cover a broad range of image appearance and staining variations. Each WSI has a slide-level label indicating whether it contains no metastases, macro-metastases, micro-metastases, or isolated tumor cells. Furthermore, for 209 WSIs, detailed hand-drawn contours for all metastases are provided. Last, open-source software tools to visualize and interact with the data have been made available. Conclusions: A unique dataset of annotated, whole-slide digital histopathology images has been provided with high potential for re-use.

PET-CT based automated lung nodule detection.

An automatic method is presented in order to detect lung nodules in PET-CT studies. Using the foreground and background mean ratio independently in every nodule, we can detect the region of the nodules properly. The size and intensity of the lesions do not affect the result of the algorithm, although size constraints are present in the final classification step. The CT image is also used to classify the found lesions built on lung segmentation. We also deal with those cases when nearby and similar nodules are merged into one by a split-up post-processing step. With our method the time of the localization can be decreased from more than one hour to maximum five minutes. The method had been implemented and validated on real clinical cases in Interview Fusion clinical evaluation software (Mediso). Results indicate that our approach is very effective in detecting lung nodules and can be a valuable aid for physicians working in the daily routine of oncology.

Automated material map generation from MRI scout pairs for preclinical PET attenuation correction.

A novel method is presented to perform material map segmentation from preclinical MRI for corresponding PET attenuation correction. MRI does not provide attenuation ratio, hence segmenting a material map from it is challenging. Furthermore the MRI images often suffer from ghost artifacts. On the contrary MRI has no radiation dose. Our method operated with fast spin echo scout pairs that had perpendicular frequency directions. This way the direction of the ghost artifacts were perpendicular as well. Our body-air segmentation method built on this a priori information and successfully erased the ghost artifacts from the final binary mask. Visual and quantitative validation was performed by two preclinical specialists. Results indicate that our method is effective against MRI scout ghost artifacts and that PET attenuation correction based on MRI makes sense even on preclinical images.

New workflows and algorithms of bone scintigraphy based on SPECT-CT.

Gold standard bone scintigraphy workflow contains acquisition of planar anterior and posterior images and if necessary, additional SPECTs as well. Planar acquisitions are time consuming and not enough for accurately locating hotspots. Current paper proposes a novel workflow for fast whole body bone SPECT scintigraphy. We present a novel stitching method to generate a whole body SPECT based on the SPECT projections. Our stitching method is performed on the projection series not on the reconstructed SPECTs, thus stitching artifacts are greatly reduced. Our workflow does not require any anterior-posterior image pairs, since these images are derived from the reconstructed whole body SPECT automatically. Our stitching method has been validated on real clinical data performed by medical physicians. Results show that our method is very effective for whole body SPECT generations leaving no signs of artifacts. Our workflow required overall 16 minutes to acquire a whole body SPECT which is comparable to the 60 minutes acquisition time required for gold standard techniques including planar images and additional SPECT acquisitions.

Automated patient couch removal algorithm on CT images.

The current paper proposes a novel automated patient couch removal method on Computed Tomography (CT) images. Patient couch is often considered to be an unnecessary artifact especially when 3D rendered techniques are used. The method is based on measuring similarity between selected axial slices and the assumption that the bed object is constant on different slices. Due to the weight of the patient the couch could bend which is identifiable as sagittal movement on consecutive axial slices. Therefore the method focuses on finding this movement after an initial segmentation. According to initial validation performed on real medical data, our method is an effective tool to remove patient couch without user interaction.

An extended registration framework for the triple registration of IBZM SPECT, DATSCAN SPECT and MRI brain images to support the evaluation of brain dopamine receptor scintigraphies.

An extended registration model is presented to register medical image triples acquired for brain dopamine receptor scintigraphies. The model operates with rigid and nonlinear transformations in parallel, where all transformation parameters are optimized by one optimization method. The concept of the transformation-sampling-similarity measurement minimizes the memory usage of a real implementation. A partial-fine sampling method is proposed to decrease the processing time of the registration. Real medical data was collected to compare our method with well-known prior ones. The first tests show that the model outperforms the classic registration methods in both speed and accuracy.

Hextuple registration of interim and follow-up PET-CT images for the accurate tracking of patient recovery after therapy.

An extended registration framework is presented to accurately register follow-up PET-CT study triples. Since there are six images to register, sophisticated feature extraction and similarity measurement methods are proposed. An irregular sampling method is introduced to decrease the processing speed of the hextuple registration. The similarity measurement is based on a normalized hybrid extended SSD (Sum of Squared Differences) and and extended NMI (Normalized mutual Information). The method has been tested on a huge amount of simulated data to avoid observer specific results. Based on the validation, our method outperforms prior solutions in both speed and accuracy, hence it should be the subject of further investigations.