The Evolving Role of Artificial Intelligence in Gastrointestinal Histopathology: An Update.

Significant advances in artificial intelligence (AI) over the past decade potentially may lead to dramatic effects on clinical practice. Digitized histology represents an area ripe for AI implementation. We describe several current needs within the world of gastrointestinal histopathology, and outline, using currently studied models, how AI potentially can address them. We also highlight pitfalls as AI makes inroads into clinical practice.

Generative Adversarial Networks in Digital Histopathology: Current Applications, Limitations, Ethical Considerations, and Future Directions.

Generative adversarial networks (GANs) have gained significant attention in the field of image synthesis, particularly in computer vision. GANs consist of a generative model and a discriminative model trained in an adversarial setting to generate realistic and novel data. In the context of image synthesis, the generator produces synthetic images, whereas the discriminator determines their authenticity by comparing them with real examples. Through iterative training, the generator allows the creation of images that are indistinguishable from real ones, leading to high-quality image generation. Considering their success in computer vision, GANs hold great potential for medical diagnostic applications. In the medical field, GANs can generate images of rare diseases, aid in learning, and be used as visualization tools. GANs can leverage unlabeled medical images, which are large in size, numerous in quantity, and challenging to annotate manually. GANs have demonstrated remarkable capabilities in image synthesis and have the potential to significantly impact digital histopathology. This review article focuses on the emerging use of GANs in digital histopathology, examining their applications and potential challenges. Histopathology plays a crucial role in disease diagnosis, and GANs can contribute by generating realistic microscopic images. However, ethical considerations arise because of the reliance on synthetic or pseudogenerated images. Therefore, the manuscript also explores the current limitations and highlights the ethical considerations associated with the use of this technology. In conclusion, digital histopathology has seen an emerging use of GANs for image enhancement, such as color (stain) normalization, virtual staining, and ink/marker removal. GANs offer significant potential in transforming digital pathology when applied to specific and narrow tasks (preprocessing enhancements). Evaluating data quality, addressing biases, protecting privacy, ensuring accountability and transparency, and developing regulation are imperative to ensure the ethical application of GANs.

Scientific and Regulatory Policy Committee Brief Communication: 2019 Survey on Use of Digital Histopathology Systems in Nonclinical Toxicology Studies.

Histopathologic evaluation and peer review using digital whole-slide images (WSIs) is a relatively new medium for assessing nonclinical toxicology studies in Good Laboratory Practice (GLP) environments. To better understand the present and future use of digital pathology in nonclinical toxicology studies, the Society of Toxicologic Pathology (STP) formed a working group to survey STP members with the goal of creating recommendations for implementation. The survey was administered in December 2019, immediately before the COVID-19 pandemic, and the results suggested that the use of digital histopathology for routine GLP histopathology assessment was not widespread. Subsequently, in follow-up correspondence during the pandemic, many responding institutions either began investigating or adopting digital WSI systems to reduce employee exposure to COVID-19. Therefore, the working group presents the survey results as a pre-pandemic baseline data set. Recommendations for use of WSI systems in GLP environments will be the subject of a separate publication.

Medical Image Classification Based on Semi-Supervised Generative Adversarial Network and Pseudo-Labelling.

Deep learning has substantially improved the state-of-the-art in object detection and image classification. Deep learning usually requires large-scale labelled datasets to train the models; however, due to the restrictions in medical data sharing and accessibility and the expensive labelling cost, the application of deep learning in medical image classification has been dramatically hindered. In this study, we propose a novel method that leverages semi-supervised adversarial learning and pseudo-labelling to incorporate the unlabelled images in model learning. We validate the proposed method on two public databases, including ChestX-ray14 for lung disease classification and BreakHis for breast cancer histopathological image diagnosis. The results show that our method achieved highly effective performance with an accuracy of 93.15% while using only 30% of the labelled samples, which is comparable to the state-of-the-art accuracy for chest X-ray classification; it also outperformed the current methods in multi-class breast cancer histopathological image classification with a high accuracy of 96.87%.

Utilizing Whole Slide Images for the Primary Evaluation and Peer Review of a GLP-Compliant Rodent Toxicology Study.

The approach undertaken to deliver a Good Laboratory Practice (GLP) validation of whole slide images (WSIs) and the associated workflow for the digital primary evaluation and peer review of a GLP-compliant rodent inhalation toxicity study is described. The contract research organization (CRO) undertook validation of the slide scanner, scanner software, and associated database software. This provided a GLP validated environment within the database software for the primary histopathologic evaluation using WSI and viewed with the database software web viewer. The CRO also validated a cloud-based digital pathology platform that supported the upload and transfer of WSI and metadata to a cache within the sponsor's local area network. The sponsor undertook a separate GLP validation of the same cloud-based digital pathology platform to cover the download and review of the WSI. The establishment of a fit-for-purpose GLP-compliant workflow for WSI and successful deployment for the digital primary evaluation and peer review of a large GLP toxicology study enabled flexibility in accelerated global working and potential future reuse of digitized data for advanced artificial intelligence and machine learning image analysis.

Histological, immunohistochemical and mRNA gene expression responses in coeliac disease patients challenged with gluten using PAXgene fixed paraffin-embedded duodenal biopsies.

BACKGROUND: There is an unmet need for novel treatments, such as drugs or vaccines, adjunctive to or replacing a burdensome life-long gluten-free diet for coeliac disease. The gold standard for successful treatment is a healed small intestinal mucosa, and therefore, the outcome measures in proof-of-concept studies should be based on evaluation of small intestine biopsies. We here evaluated morphometric, immunohistochemical and messenger RNA (mRNA) expression changes in coeliac disease patients challenged with gluten using PAXgene fixed paraffin-embedded biopsies. METHODS: Fifteen coeliac disease patients were challenged with 4 g of gluten per day for 10 weeks and 24 non-coeliac patients served as disease controls. A wide array of histological and immunohistochemical staining and mRNA-based gene expression tests (RT-qPCR and RNAseq) were carried out. RESULTS: Digital quantitative villous height: crypt depth ratio (VH: CrD) measurements revealed significant duodenal mucosal deterioration in all coeliac disease patients on gluten challenge. In contrast, the Marsh-Oberhuber class worsened in only 80% of coeliac patients. Measuring the intraepithelial CD3+ T-lymphocyte and lamina propria CD138+ plasma cell densities simultaneously proved to be a meaningful new measure of inflammation. Stainings for γδ T cells and IgA deposits, where previously frozen samples have been needed, were successful in PAXgene fixed paraffin-embedded samples. Messenger RNA extraction from the same paraffin-embedded biopsy block was successful and allowed large-scale qRT-PCR and RNAseq analyses for gene expression. Molecular morphometry, using the mRNA expression ratio of villous epithelium-specific gene APOA4 to crypt proliferation gene Ki67, showed a similar significant distinction between paired baseline and post-gluten challenge biopsies as quantitative histomorphometry. CONCLUSION: Rigorous digitally measured histologic and molecular markers suitable for gluten challenge studies can be obtained from a single paraffin-embedded biopsy specimen. Molecular morphometry seems to be a promising new tool that can be used in situations where assessing duodenal mucosal health is of paramount importance. In addition, the diagnostically valuable IgA deposits were now stained in paraffin-embedded specimens making them more accessible in routine clinics.

Utilizing a Pathomics Biomarker to Predict the Effectiveness of Bevacizumab in Ovarian Cancer Treatment.

The purpose of this investigation is to develop and initially assess a quantitative image analysis scheme that utilizes histopathological images to predict the treatment effectiveness of bevacizumab therapy in ovarian cancer patients. As a widely accessible diagnostic tool, histopathological slides contain copious information regarding underlying tumor progression that is associated with tumor prognosis. However, this information cannot be readily identified by conventional visual examination. This study utilizes novel pathomics technology to quantify this meaningful information for treatment effectiveness prediction. Accordingly, a total of 9828 features were extracted from segmented tumor tissue, cell nuclei, and cell cytoplasm, which were categorized into geometric, intensity, texture, and subcellular structure features. Next, the best performing features were selected as the input for SVM (support vector machine)-based prediction models. These models were evaluated on an open dataset containing a total of 78 patients and 288 whole slides images. The results indicated that the sufficiently optimized, best-performing model yielded an area under the receiver operating characteristic (ROC) curve of 0.8312. When examining the best model's confusion matrix, 37 and 25 cases were correctly predicted as responders and non-responders, respectively, achieving an overall accuracy of 0.7848. This investigation initially validated the feasibility of utilizing pathomics techniques to predict tumor responses to chemotherapy at an early stage.

External Validation of a Digital Pathology-based Multimodal Artificial Intelligence Architecture in the NRG/RTOG 9902 Phase 3 Trial.

BACKGROUND: Accurate risk stratification is critical to guide management decisions in localized prostate cancer (PCa). Previously, we had developed and validated a multimodal artificial intelligence (MMAI) model generated from digital histopathology and clinical features. Here, we externally validate this model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial. OBJECTIVE: To externally validate the MMAI model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial. DESIGN, SETTING, AND PARTICIPANTS: Our validation cohort included 318 localized high-risk PCa patients from NRG/RTOG 9902 with available histopathology (337 [85%] of the 397 patients enrolled into the trial had available slides, of which 19 [5.6%] failed due to poor image quality). OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Two previously locked prognostic MMAI models were validated for their intended endpoint: distant metastasis (DM) and PCa-specific mortality (PCSM). Individual clinical factors and the number of National Comprehensive Cancer Network (NCCN) high-risk features served as comparators. Subdistribution hazard ratio (sHR) was reported per standard deviation increase of the score with corresponding 95% confidence interval (CI) using Fine-Gray or Cox proportional hazards models. RESULTS AND LIMITATIONS: The DM and PCSM MMAI algorithms were significantly and independently associated with the risk of DM (sHR [95% CI] = 2.33 [1.60-3.38], p < 0.001) and PCSM, respectively (sHR [95% CI] = 3.54 [2.38-5.28], p < 0.001) when compared against other prognostic clinical factors and NCCN high-risk features. The lower 75% of patients by DM MMAI had estimated 5- and 10-yr DM rates of 4% and 7%, and the highest quartile had average 5- and 10-yr DM rates of 19% and 32%, respectively (p < 0.001). Similar results were observed for the PCSM MMAI algorithm. CONCLUSIONS: We externally validated the prognostic ability of MMAI models previously developed among men with localized high-risk disease. MMAI prognostic models further risk stratify beyond the clinical and pathological variables for DM and PCSM in a population of men already at a high risk for disease progression. This study provides evidence for consistent validation of our deep learning MMAI models to improve prognostication and enable more informed decision-making for patient care. PATIENT SUMMARY: This paper presents a novel approach using images from pathology slides along with clinical variables to validate artificial intelligence (computer-generated) prognostic models. When implemented, clinicians can offer a more personalized and tailored prognostic discussion for men with localized prostate cancer.

Development of a diagnostic support system for the fibrosis of nonalcoholic fatty liver disease using artificial intelligence and deep learning.

Liver fibrosis is a pathological condition characterized by the abnormal proliferation of liver tissue, subsequently able to progress to cirrhosis or possibly hepatocellular carcinoma. The development of artificial intelligence and deep learning have begun to play a significant role in fibrosis detection. This study aimed to develop SMART AI-PATHO, a fully automated assessment method combining quantification of histopathological architectural features, to analyze steatosis and fibrosis in nonalcoholic fatty liver disease (NAFLD) core biopsies and employ Metavir fibrosis staging as standard references and fat assessment grading measurement for comparison with the pathologist interpretations. There were 146 participants enrolled in our study. The correlation of Metavir scoring system interpretation between pathologists and SMART AI-PATHO was significantly correlated (Agreement = 68%, Kappa = 0.59, p-value <0.001), which subgroup analysis of significant fibrosis (Metavir score F2-F4) and nonsignificant fibrosis (Metavir score F0-F1) demonstrated substantial correlated results (agreement = 80%, kappa = 0.61, p-value <0.001), corresponding with the correlation of advanced fibrosis (Metavir score F3-F4) and nonadvanced fibrosis groups (Metavir score F0-F2), (agreement = 89%, kappa = 0.74, p-value <0.001). SMART AI-PATHO, the first pivotal artificially intelligent diagnostic tool for the color-based NAFLD hepatic tissue staging in Thailand, demonstrated satisfactory performance as a pathologist to provide liver fibrosis scoring and steatosis grading. In the future, developing AI algorithms and reliable testing on a larger scale may increase accuracy and contribute to telemedicine consultations for general pathologists in clinical practice.

Imaging the elemental distribution within human malignant melanomas using Laser-Induced Breakdown Spectroscopy.

The diagnosis of malignant melanoma, often an inconspicuous but highly aggressive tumor, is most commonly done by histological examination, while additional diagnostic methods on the level of elements and molecules are constantly being developed. Several studies confirmed differences in the chemical composition of healthy and tumor tissue. Our study presents the potential of the LIBS (Laser-Induced-Breakdown Spectroscopy) technique as a diagnostic tool in malignant melanoma (MM) based on the quantitative changes in elemental composition in cancerous tissue. Our patient group included 17 samples of various types of malignant melanoma and one sample of healthy skin tissue as a control. To achieve a clear perception of results, we have selected two biogenic elements (calcium and magnesium), which showed a dissimilar distribution in cancerous tissue from its healthy surroundings. Moreover, we observed indications of different concentrations of these elements in different subtypes of malignant melanoma, a hypothesis that requires confirmation in a more extensive sample set. The information provided by the LIBS Imaging method could potentially be helpful not only in the diagnostics of tumor tissue but also be beneficial in broadening the knowledge about the tumor itself.

Assessing spatial distribution of bioindicator elements in various cutaneous tumors using correlative imaging with laser-ablation-based analytical methods.

Correlative imaging of cutaneous tumors provides additional information to the standard histopathologic examination. However, the joint progress in the establishment of analytical techniques, such as Laser-Induced Breakdown Spectroscopy (LIBS) and Laser Ablation Inductively Coupled Plasma Mass Spectrometry (LA-ICP-MS) in clinical practice is still limited. Their combination provides complementary information as it is also shown in our study in terms of major biotic (Ca, Mg, and P) and trace (Cu and Zn) elements. To elucidate changes in the elemental composition in tumors, we have compiled a set of malignant tumors (Squamous Cell Carcinoma, Basal Cell Carcinoma, Malignant Melanoma, and Epithelioid Angiosarcoma), one benign tumor (Pigmented Nevus) and one healthy-skin sample. The data processing was based on a methodological pipeline involving binary image registration and affine transformation. Thus, our paper brings a feasibility study of a practical methodological concept that enables us to compare LIBS and LA-ICP-MS results despite the mutual spatial distortion of original elemental images. Moreover, we also show that LIBS could be a sufficient pre-screening method even for a larger number of samples according to the speed and reproducibility of the analyses. Whereas LA-ICP-MS could serve as a ground truth and reference technique for preselected samples.

Robust, credible, and interpretable AI-based histopathological prostate cancer grading.

Background: Prostate cancer (PCa) is among the most common cancers in men and its diagnosis requires the histopathological evaluation of biopsies by human experts. While several recent artificial intelligence-based (AI) approaches have reached human expert-level PCa grading, they often display significantly reduced performance on external datasets. This reduced performance can be caused by variations in sample preparation, for instance the staining protocol, section thickness, or scanner used. Another limiting factor of contemporary AI-based PCa grading is the prediction of ISUP grades, which leads to the perpetuation of human annotation errors. Methods: We developed the prostate cancer aggressiveness index (PCAI), an AI-based PCa detection and grading framework that is trained on objective patient outcome, rather than subjective ISUP grades. We designed PCAI as a clinical application, containing algorithmic modules that offer robustness to data variation, medical interpretability, and a measure of prediction confidence. To train and evaluate PCAI, we generated a multicentric, retrospective, observational trial consisting of six cohorts with 25,591 patients, 83,864 images, and 5 years of median follow-up from 5 different centers and 3 countries. This includes a high-variance dataset of 8,157 patients and 28,236 images with variations in sample thickness, staining protocol, and scanner, allowing for the systematic evaluation and optimization of model robustness to data variation. The performance of PCAI was assessed on three external test cohorts from two countries, comprising 2,255 patients and 9,437 images. Findings: Using our high-variance datasets, we show how differences in sample processing, particularly slide thickness and staining time, significantly reduce the performance of AI-based PCa grading by up to 6.2 percentage points in the concordance index (C-index). We show how a select set of algorithmic improvements, including domain adversarial training, conferred robustness to data variation, interpretability, and a measure of credibility to PCAI. These changes lead to significant prediction improvement across two biopsy cohorts and one TMA cohort, systematically exceeding expert ISUP grading in C-index and AUROC by up to 22 percentage points. Interpretation: Data variation poses serious risks for AI-based histopathological PCa grading, even when models are trained on large datasets. Algorithmic improvements for model robustness, interpretability, credibility, and training on high-variance data as well as outcome-based severity prediction gives rise to robust models with above ISUP-level PCa grading performance.

Predicting clinical endpoints and visual changes with quality-weighted tissue-based renal histological features.

Two common obstacles limiting the performance of data-driven algorithms in digital histopathology classification tasks are the lack of expert annotations and the narrow diversity of datasets. Multi-instance learning (MIL) can address the former challenge for the analysis of whole slide images (WSI), but performance is often inferior to full supervision. We show that the inclusion of weak annotations can significantly enhance the effectiveness of MIL while keeping the approach scalable. An analysis framework was developed to process periodic acid-Schiff (PAS) and Sirius Red (SR) slides of renal biopsies. The workflow segments tissues into coarse tissue classes. Handcrafted and deep features were extracted from these tissues and combined using a soft attention model to predict several slide-level labels: delayed graft function (DGF), acute tubular injury (ATI), and Remuzzi grade components. A tissue segmentation quality metric was also developed to reduce the adverse impact of poorly segmented instances. The soft attention model was trained using 5-fold cross-validation on a mixed dataset and tested on the QUOD dataset containing n = 373 PAS and n = 195 SR biopsies. The average ROC-AUC over different prediction tasks was found to be 0.598 ± 0.011 , significantly higher than using only ResNet50 ( 0.545 ± 0.012 ), only handcrafted features ( 0.542 ± 0.011 ), and the baseline ( 0.532 ± 0.012 ) of state-of-the-art performance. In conjunction with soft attention, weighting tissues by segmentation quality has led to further improvement ( A U C = 0.618 ± 0.010 ) . Using an intuitive visualisation scheme, we show that our approach may also be used to support clinical decision making as it allows pinpointing individual tissues relevant to the predictions.

Automated detection of microscopic placental features indicative of maternal vascular malperfusion using machine learning.

INTRODUCTION: Hypertensive disorders of pregnancy (HDP) and fetal growth restriction (FGR) are common obstetrical complications, often with pathological features of maternal vascular malperfusion (MVM) in the placenta. Currently, clinical placental pathology methods involve a manual visual examination of histology sections, a practice that can be resource-intensive and demonstrates moderate-to-poor inter-pathologist agreement on diagnostic outcomes, dependant on the degree of pathologist sub-specialty training. METHODS: This study aims to apply machine learning (ML) feature extraction methods to classify digital images of placental histopathology specimens, collected from cases of HDP [pregnancy induced hypertension (PIH), preeclampsia (PE), PE + FGR], normotensive FGR, and healthy pregnancies, according to the presence or absence of MVM lesions. 159 digital images were captured from histological placental specimens, manually scored for MVM lesions (MVM- or MVM+) and used to develop a support vector machine (SVM) classifier model, using features extracted from pre-trained ResNet18. The model was trained with data augmentation and shuffling, with the performance assessed for patch-level and image-level classification through measurements of accuracy, precision, and recall using confusion matrices. RESULTS: The SVM model demonstrated accuracies of 70 % and 79 % for patch-level and image-level MVM classification, respectively, with poorest performance observed on images with borderline MVM presence, as determined through post hoc observation. DISCUSSION: The results are promising for the integration of ML methods into the placental histopathological examination process. Using this study as a proof-of-concept will lead our group and others to carry ML models further in placental histopathology.

Standardized CycleGAN training for unsupervised stain adaptation in invasive carcinoma classification for breast histopathology.

Purpose: Generalization is one of the main challenges of computational pathology. Slide preparation heterogeneity and the diversity of scanners lead to poor model performance when used on data from medical centers not seen during training. In order to achieve stain invariance in breast invasive carcinoma patch classification, we implement a stain translation strategy using cycleGANs for unsupervised image-to-image translation. Those models often suffer from a lack of proper metrics to monitor and stop the training at a particular point. We also introduce a method to solve this issue. Approach: We compare three CycleGAN-based approaches to a baseline classification model obtained without any stain invariance strategy. Two of the proposed approaches use CycleGAN's translations at inference or training to build stain-specific classification models. The last method uses them for stain data augmentation during training. This constrains the classification model to learn stain-invariant features. Regarding CycleGANs' training monitoring, we leverage Fréchet inception distance between generated and real samples and use it as a stopping criterion. We compare CycleGANs' models stopped using this criterion and models stopped at a fixed number of epochs. Results: Baseline metrics are set by training and testing the baseline classification model on a reference stain. We assessed performances using three medical centers with H&E and H&E&S staining. Every approach tested in this study improves baseline metrics without needing labels on target stains. The stain augmentation-based approach produced the best results on every stain. Each method's pros and cons are studied and discussed. Moreover, FID stopping criterion proves superiority to methods using a predefined number of training epoch and has the benefit of not requiring any visual inspection of CycleGAN results. Conclusion: We introduce a method to attain stain invariance for breast invasive carcinoma classification by leveraging CycleGAN's abilities to produce realistic translations between various stains. Moreover, we propose a systematical method for scheduling CycleGANs' trainings by using FID as a stopping criterion and prove its superiority to other methods. Finally, we give an insight on the minimal amount of data required for CycleGAN training in a digital histopathology setting.

Prostate cancer radiogenomics reveals proliferative gene expression programs associated with distinct MRI-based hypoxia levels.

BACKGROUND AND PURPOSE: The biology behind individual hypoxia levels in patient tumors is poorly understood. Here, we used radiogenomics to identify associations between magnetic resonance imaging (MRI)-based hypoxia levels and biological processes derived from gene expression data in prostate cancer. MATERIALS AND METHODS: For 85 prostate cancer patients, MRI-based hypoxia images were constructed by combining diffusion-weighted images reflecting oxygen consumption and supply. The ability to differentiate hypoxia levels in these images was verified by comparison with matched biopsy sections stained for the hypoxia marker pimonidazole. For MRI-defined hypoxia levels, corresponding hypoxic fractions were calculated and correlated with biopsy gene expression profiles. Biological processes were predicted by gene set enrichment analysis (GSEA) and validated by immunohistochemistry (Ki67 proliferation marker, reactive stroma grade) and RT-PCR (MYC). RESULTS: Genes with correlation between expression level and hypoxic fraction were identified for 56 MRI-based hypoxia levels. At all levels, GSEA identified proliferation as the predominant biological process enriched among the correlating genes. Two independent proliferative gene signatures were developed. The Peak1 signature, upregulated at moderate/severe hypoxia, reflected MYC upregulation and high Ki67-proliferation index of cancer cells in pimonidazole-positive regions. The Peak2 signature, upregulated at mild to non-hypoxic levels, was associated with fibroblast gene signature and reactive stroma grade. High scores of both Peak1 and Peak2 indicated elevated risk of biochemical recurrence in multiple cohorts. CONCLUSION: Radiogenomics identified two gene expression programs activated at different hypoxia levels, reflecting proliferation of cancer cells and stroma cells. Genes involved in these programs could be candidate targets for intervention.

Imaging phenotypes predict overall survival in glioma more accurate than basic demographic and cell mutation profiles.

BACKGROUND: Magnetic resonance imaging (MRI), digital pathology imaging (PATH), demographics, and IDH mutation status predict overall survival (OS) in glioma. Identifying and characterizing predictive features in the different modalities may improve OS prediction accuracy. PURPOSE: To evaluate the OS prediction accuracy of combinations of prognostic markers in glioma patients. MATERIALS AND METHODS: Multi-contrast MRI, comprising T1-weighted, T1-weighted post-contrast, T2-weighted, T2 fluid-attenuated-inversion-recovery, and pathology images from glioma patients (n = 160) were retrospectively collected (1983-2008) from TCGA alongside age and sex. Phenotypic profiling of tumors was performed by quantifying the radiographic and histopathologic descriptors extracted from the delineated region-of-interest in MRI and PATH images. A Cox proportional hazard model was trained with the MRI and PATH features, IDH mutation status, and basic demographic variables (age and sex) to predict OS. The performance was evaluated in a split-train-test configuration using the concordance-index, computed between the predicted risk score and observed OS. RESULTS: The average age of patients was 51.2years (women: n = 77, age-range=18-84years; men: n = 83, age-range=21-80years). The median OS of the participants was 494.5 (range,3-4752), 481 (range,7-4752), and 524.5 days (range,3-2869), respectively, in complete dataset, training, and test datasets. The addition of MRI or PATH features improved prediction of OS when compared to models based on age, sex, and mutation status alone or their combination (p < 0.001). The full multi-omics model integrated MRI, PATH, clinical, and genetic profiles and predicted the OS best (c-index= 0.87). CONCLUSION: The combination of imaging, genetic, and clinical profiles leads to a more accurate prognosis than the clinical and/or mutation status.

Shedding light on the black box of a neural network used to detect prostate cancer in whole slide images by occlusion-based explainability.

Diagnostic histopathology faces increasing demands due to aging populations and expanding healthcare programs. Semi-automated diagnostic systems employing deep learning methods are one approach to alleviate this pressure. The learning models for histopathology are inherently complex and opaque from the user's perspective. Hence different methods have been developed to interpret their behavior. However, relatively limited attention has been devoted to the connection between interpretation methods and the knowledge of experienced pathologists. The main contribution of this paper is a method for comparing morphological patterns used by expert pathologists to detect cancer with the patterns identified as important for inference of learning models. Given the patch-based nature of processing large-scale histopathological imaging, we have been able to show statistically that the VGG16 model could utilize all the structures that are observable by the pathologist, given the patch size and scan resolution. The results show that the neural network approach to recognizing prostatic cancer is similar to that of a pathologist at medium optical resolution. The saliency maps identified several prevailing histomorphological features characterizing carcinoma, e.g., single-layered epithelium, small lumina, and hyperchromatic nuclei with halo. A convincing finding was the recognition of their mimickers in non-neoplastic tissue. The method can also identify differences, i.e., standard patterns not used by the learning models and new patterns not yet used by pathologists. Saliency maps provide added value for automated digital pathology to analyze and fine-tune deep learning systems and improve trust in computer-based decisions.

Stain transfer using Generative Adversarial Networks and disentangled features.

With the digitization of histopathology, machine learning algorithms have been developed to help pathologists. Color variation in histopathology images degrades the performance of these algorithms. Many models have been proposed to resolve the impact of color variation and transfer histopathology images to a single stain style. Major shortcomings include manual feature extraction, bias on a reference image, being limited to one style to one style transfer, dependence on style labels for source and target domains, and information loss. We propose two models, considering these shortcomings. Our main novelty is using Generative Adversarial Networks (GANs) along with feature disentanglement. The models extract color-related and structural features with neural networks; thus, features are not hand-crafted. Extracting features helps our models do many-to-one stain transformations and require only target-style labels. Our models also do not require a reference image by exploiting GAN. Our first model has one network per stain style transformation, while the second model uses only one network for many-to-many stain style transformations. We compare our models with six state-of-the-art models on the Mitosis-Atypia Dataset. Both proposed models achieved good results, but our second model outperforms other models based on the Histogram Intersection Score (HIS). Our proposed models were applied to three datasets to test their performance. The efficacy of our models was also evaluated on a classification task. Our second model obtained the best results in all the experiments with HIS of 0.88, 0.85, 0.75 for L-channel, a-channel, and b-channel, using the Mitosis-Atypia Dataset and accuracy of 90.3% for classification.

CycleGAN for virtual stain transfer: Is seeing really believing?

Digital Pathology is an area prone to high variation due to multiple factors which can strongly affect diagnostic quality and visual appearance of the Whole-Slide-Images (WSIs). The state-of-the art methods to deal with such variation tend to address this through style-transfer inspired approaches. Usually, these solutions directly apply successful approaches from the literature, potentially with some task-related modifications. The majority of the obtained results are visually convincing, however, this paper shows that this is not a guarantee that such images can be directly used for either medical diagnosis or reducing domain shift.This article shows that slight modification in a stain transfer architecture, such as a choice of normalisation layer, while resulting in a variety of visually appealing results, surprisingly greatly effects the ability of a stain transfer model to reduce domain shift. By extensive qualitative and quantitative evaluations, we confirm that translations resulting from different stain transfer architectures are distinct from each other and from the real samples. Therefore conclusions made by visual inspection or pretrained model evaluation might be misleading.