Open and reusable deep learning for pathology with WSInfer and QuPath.

Digital pathology has seen a proliferation of deep learning models in recent years, but many models are not readily reusable. To address this challenge, we developed WSInfer: an open-source software ecosystem designed to streamline the sharing and reuse of deep learning models for digital pathology. The increased access to trained models can augment research on the diagnostic, prognostic, and predictive capabilities of digital pathology.

OPTIMAL: An OPTimized Imaging Mass cytometry AnaLysis framework for benchmarking segmentation and data exploration.

Analysis of imaging mass cytometry (IMC) data and other low-resolution multiplexed tissue imaging technologies is often confounded by poor single-cell segmentation and suboptimal approaches for data visualization and exploration. This can lead to inaccurate identification of cell phenotypes, states, or spatial relationships compared to reference data from single-cell suspension technologies. To this end we have developed the "OPTimized Imaging Mass cytometry AnaLysis (OPTIMAL)" framework to benchmark any approaches for cell segmentation, parameter transformation, batch effect correction, data visualization/clustering, and spatial neighborhood analysis. Using a panel of 27 metal-tagged antibodies recognizing well-characterized phenotypic and functional markers to stain the same Formalin-Fixed Paraffin Embedded (FFPE) human tonsil sample tissue microarray over 12 temporally distinct batches we tested several cell segmentation models, a range of different arcsinh cofactor parameter transformation values, 5 different dimensionality reduction algorithms, and 2 clustering methods. Finally, we assessed the optimal approach for performing neighborhood analysis. We found that single-cell segmentation was improved by the use of an Ilastik-derived probability map but that issues with poor segmentation were only really evident after clustering and cell type/state identification and not always evident when using "classical" bivariate data display techniques. The optimal arcsinh cofactor for parameter transformation was 1 as it maximized the statistical separation between negative and positive signal distributions and a simple Z-score normalization step after arcsinh transformation eliminated batch effects. Of the five different dimensionality reduction approaches tested, PacMap gave the best data structure with FLOWSOM clustering out-performing phenograph in terms of cell type identification. We also found that neighborhood analysis was influenced by the method used for finding neighboring cells with a "disc" pixel expansion outperforming a "bounding box" approach combined with the need for filtering objects based on size and image-edge location. Importantly, OPTIMAL can be used to assess and integrate with any existing approach to IMC data analysis and, as it creates .FCS files from the segmentation output and allows for single-cell exploration to be conducted using a wide variety of accessible software and algorithms familiar to conventional flow cytometrists.

AimSeg: A machine-learning-aided tool for axon, inner tongue and myelin segmentation.

Electron microscopy (EM) images of axons and their ensheathing myelin from both the central and peripheral nervous system are used for assessing myelin formation, degeneration (demyelination) and regeneration (remyelination). The g-ratio is the gold standard measure of assessing myelin thickness and quality, and traditionally is determined from measurements made manually from EM images-a time-consuming endeavour with limited reproducibility. These measurements have also historically neglected the innermost uncompacted myelin sheath, known as the inner tongue. Nonetheless, the inner tongue has been shown to be important for myelin growth and some studies have reported that certain conditions can elicit its enlargement. Ignoring this fact may bias the standard g-ratio analysis, whereas quantifying the uncompacted myelin has the potential to provide novel insights in the myelin field. In this regard, we have developed AimSeg, a bioimage analysis tool for axon, inner tongue and myelin segmentation. Aided by machine learning classifiers trained on transmission EM (TEM) images of tissue undergoing remyelination, AimSeg can be used either as an automated workflow or as a user-assisted segmentation tool. Validation results on TEM data from both healthy and remyelinating samples show good performance in segmenting all three fibre components, with the assisted segmentation showing the potential for further improvement with minimal user intervention. This results in a considerable reduction in time for analysis compared with manual annotation. AimSeg could also be used to build larger, high quality ground truth datasets to train novel deep learning models. Implemented in Fiji, AimSeg can use machine learning classifiers trained in ilastik. This, combined with a user-friendly interface and the ability to quantify uncompacted myelin, makes AimSeg a unique tool to assess myelin growth.

QuPath Algorithm Accurately Identifies MLH1-Deficient Inflammatory Bowel Disease-Associated Colorectal Cancers in a Tissue Microarray.

Current methods for analysing immunohistochemistry are labour-intensive and often confounded by inter-observer variability. Analysis is time consuming when identifying small clinically important cohorts within larger samples. This study trained QuPath, an open-source image analysis program, to accurately identify MLH1-deficient inflammatory bowel disease-associated colorectal cancers (IBD-CRC) from a tissue microarray containing normal colon and IBD-CRC. The tissue microarray (n = 162 cores) was immunostained for MLH1, digitalised, and imported into QuPath. A small sample (n = 14) was used to train QuPath to detect positive versus no MLH1 and tissue histology (normal epithelium, tumour, immune infiltrates, stroma). This algorithm was applied to the tissue microarray and correctly identified tissue histology and MLH1 expression in the majority of valid cases (73/99, 73.74%), incorrectly identified MLH1 status in one case (1.01%), and flagged 25/99 (25.25%) cases for manual review. Qualitative review found five reasons for flagged cores: small quantity of tissue, diverse/atypical morphology, excessive inflammatory/immune infiltrations, normal mucosa, or weak/patchy immunostaining. Of classified cores (n = 74), QuPath was 100% (95% CI 80.49, 100) sensitive and 98.25% (95% CI 90.61, 99.96) specific for identifying MLH1-deficient IBD-CRC; κ = 0.963 (95% CI 0.890, 1.036) (p < 0.001). This process could be efficiently automated in diagnostic laboratories to examine all colonic tissue and tumours for MLH1 expression.

A biologist's guide to planning and performing quantitative bioimaging experiments.

Technological advancements in biology and microscopy have empowered a transition from bioimaging as an observational method to a quantitative one. However, as biologists are adopting quantitative bioimaging and these experiments become more complex, researchers need additional expertise to carry out this work in a rigorous and reproducible manner. This Essay provides a navigational guide for experimental biologists to aid understanding of quantitative bioimaging from sample preparation through to image acquisition, image analysis, and data interpretation. We discuss the interconnectedness of these steps, and for each, we provide general recommendations, key questions to consider, and links to high-quality open-access resources for further learning. This synthesis of information will empower biologists to plan and execute rigorous quantitative bioimaging experiments efficiently.

MYC sensitises cells to apoptosis by driving energetic demand.

The MYC oncogene is a potent driver of growth and proliferation but also sensitises cells to apoptosis, which limits its oncogenic potential. MYC induces several biosynthetic programmes and primary cells overexpressing MYC are highly sensitive to glutamine withdrawal suggesting that MYC-induced sensitisation to apoptosis may be due to imbalance of metabolic/energetic supply and demand. Here we show that MYC elevates global transcription and translation, even in the absence of glutamine, revealing metabolic demand without corresponding supply. Glutamine withdrawal from MRC-5 fibroblasts depletes key tricarboxylic acid (TCA) cycle metabolites and, in combination with MYC activation, leads to AMP accumulation and nucleotide catabolism indicative of energetic stress. Further analyses reveal that glutamine supports viability through TCA cycle energetics rather than asparagine biosynthesis and that TCA cycle inhibition confers tumour suppression on MYC-driven lymphoma in vivo. In summary, glutamine supports the viability of MYC-overexpressing cells through an energetic rather than a biosynthetic mechanism.

Recent Advances in Pathology: the 2022 Annual Review Issue of The Journal of Pathology.

The 2022 Annual Review Issue of The Journal of Pathology, Recent Advances in Pathology, contains 15 invited reviews on research areas of growing importance in pathology. This year, the articles include those that focus on digital pathology, employing modern imaging techniques and software to enable improved diagnostic and research applications to study human diseases. This subject area includes the ability to identify specific genetic alterations through the morphological changes they induce, as well as integrating digital and computational pathology with 'omics technologies. Other reviews in this issue include an updated evaluation of mutational patterns (mutation signatures) in cancer, the applications of lineage tracing in human tissues, and single cell sequencing technologies to uncover tumour evolution and tumour heterogeneity. The tissue microenvironment is covered in reviews specifically dealing with proteolytic control of epidermal differentiation, cancer-associated fibroblasts, field cancerisation, and host factors that determine tumour immunity. All of the reviews contained in this issue are the work of invited experts selected to discuss the considerable recent progress in their respective fields and are freely available online (https://onlinelibrary.wiley.com/journal/10969896). © 2022 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

Developing image analysis methods for digital pathology.

The potential to use quantitative image analysis and artificial intelligence is one of the driving forces behind digital pathology. However, despite novel image analysis methods for pathology being described across many publications, few become widely adopted and many are not applied in more than a single study. The explanation is often straightforward: software implementing the method is simply not available, or is too complex, incomplete, or dataset-dependent for others to use. The result is a disconnect between what seems already possible in digital pathology based upon the literature, and what actually is possible for anyone wishing to apply it using currently available software. This review begins by introducing the main approaches and techniques involved in analysing pathology images. I then examine the practical challenges inherent in taking algorithms beyond proof-of-concept, from both a user and developer perspective. I describe the need for a collaborative and multidisciplinary approach to developing and validating meaningful new algorithms, and argue that openness, implementation, and usability deserve more attention among digital pathology researchers. The review ends with a discussion about how digital pathology could benefit from interacting with and learning from the wider bioimage analysis community, particularly with regard to sharing data, software, and ideas. © 2022 The Author. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Development of a semi-automated method for tumour budding assessment in colorectal cancer and comparison with manual methods.

AIMS: Tumour budding (TB) is an established prognostic feature in multiple cancers but is not routinely assessed in pathology practice. Efforts to standardise and automate assessment have shifted from haematoxylin and eosin (H&E)-stained images towards cytokeratin immunohistochemistry. The aim of this study was to compare manual H&E and cytokeratin assessment methods with a semi-automated approach built within QuPath open-source software. METHODS AND RESULTS: TB was assessed in cores from the advancing tumour edge in a cohort of stage II/III colon cancers (n = 186). The total numbers of buds detected with each method were as follows: manual H&E, n = 503; manual cytokeratin, n = 2290; and semi-automated, n = 5138. More than four times the number of buds were identified manually with cytokeratin assessment than with H&E assessment. One thousand seven hundred and thirty-four individual buds were identified with both manual and semi-automated assessments applied to cytokeratin images, representing 75.7% of the buds identified manually (n = 2290) and 33.7% of the buds detected with the semi-automated method (n = 5138). Higher semi-automated TB scores were due to any discrete area of cytokeratin immunopositivity within an accepted area range being identified as a bud, regardless of shape or crispness of definition, and to the inclusion of tumour cell clusters within glandular lumina ('luminal pseudobuds'). Although absolute numbers differed, semi-automated and manual bud counts were strongly correlated across cores (ρ = 0.81, P < 0.0001). All methods of TB assessment demonstrated poorer survival associated with higher TB scores. CONCLUSIONS: We present a new QuPath-based approach to TB assessment, which compares favourably with established methods and offers a freely available, rapid and transparent tool that is also applicable to whole slide images.

Developing open-source software for bioimage analysis: opportunities and challenges.

Fast-paced innovations in imaging have resulted in single systems producing exponential amounts of data to be analyzed. Computational methods developed in computer science labs have proven to be crucial for analyzing these data in an unbiased and efficient manner, reaching a prominent role in most microscopy studies. Still, their use usually requires expertise in bioimage analysis, and their accessibility for life scientists has therefore become a bottleneck. Open-source software for bioimage analysis has developed to disseminate these computational methods to a wider audience, and to life scientists in particular. In recent years, the influence of many open-source tools has grown tremendously, helping tens of thousands of life scientists in the process. As creators of successful open-source bioimage analysis software, we here discuss the motivations that can initiate development of a new tool, the common challenges faced, and the characteristics required for achieving success.

Using the R Package Spatstat to Assess Inhibitory Effects of Microregional Hypoxia on the Infiltration of Cancers of the Head and Neck Region by Cytotoxic T Lymphocytes.

(1) Background: The immune system has physiological antitumor activity, which is partially mediated by cytotoxic T lymphocytes (CTL). Tumor hypoxia, which is highly prevalent in cancers of the head and neck region, has been hypothesized to inhibit the infiltration of tumors by CTL. In situ data validating this concept have so far been based solely upon the visual assessment of the distribution of CTL. Here, we have established a set of spatial statistical tools to address this problem mathematically and tested their performance. (2) Patients and Methods: We have analyzed regions of interest (ROI) of 22 specimens of cancers of the head and neck region after 4-plex immunofluorescence staining and whole-slide scanning. Single cell-based segmentation was carried out in QuPath. Specimens were analyzed with the endpoints clustering and interactions between CTL, normoxic, and hypoxic tumor areas, both visually and using spatial statistical tools implemented in the R package Spatstat. (3) Results: Visual assessment suggested clustering of CTL in all instances. The visual analysis also suggested an inhibitory effect between hypoxic tumor areas and CTL in a minority of the whole-slide scans (9 of 22, 41%). Conversely, the objective mathematical analysis in Spatstat demonstrated statistically significant inhibitory interactions between hypoxia and CTL accumulation in a substantially higher number of specimens (16 of 22, 73%). It showed a similar trend in all but one of the remaining samples. (4) Conclusion: Our findings provide non-obvious but statistically rigorous evidence of inhibition of CTL infiltration into hypoxic tumor subregions of cancers of the head and neck. Importantly, these shielded sites may be the origin of tumor recurrences. We provide the methodology for the transfer of our statistical approach to similar questions. We discuss why versions of the Kcross and pcf.cross functions may be the methods of choice among the repertoire of statistical tests in Spatstat for this type of analysis.

Identifying mismatch repair-deficient colon cancer: near-perfect concordance between immunohistochemistry and microsatellite instability testing in a large, population-based series.

AIMS: Establishing the mismatch repair (MMR) status of colorectal cancers is important to enable the detection of underlying Lynch syndrome and inform prognosis and therapy. Current testing typically involves either polymerase chain reaction (PCR)-based microsatellite instability (MSI) testing or MMR protein immunohistochemistry (IHC). The aim of this study was to compare these two approaches in a large, population-based cohort of stage 2 and 3 colon cancer cases in Northern Ireland. METHODS AND RESULTS: The study used the Promega pentaplex assay to determine MSI status and a four-antibody MMR IHC panel. IHC was applied to tumour tissue microarrays with triplicate tumour sampling, and assessed manually. Of 593 cases with available MSI and MMR IHC results, 136 (22.9%) were MSI-high (MSI-H) and 135 (22.8%) showed abnormal MMR IHC. Concordance was extremely high, with 97.1% of MSI-H cases showing abnormal MMR IHC, and 97.8% of cases with abnormal IHC showing MSI-H status. Under-representation of tumour epithelial cells in samples from heavily inflamed tumours resulted in misclassification of several cases with abnormal MMR IHC as microsatellite-stable. MMR IHC revealed rare cases with unusual patterns of MMR protein expression, unusual combinations of expression loss, or secondary clonal loss of expression, as further illustrated by repeat immunostaining on whole tissue sections. CONCLUSIONS: MSI PCR testing and MMR IHC can be considered to be equally proficient tests for establishing MMR/MSI status, when there is awareness of the potential pitfalls of either method. The choice of methodology may depend on available services and expertise.

Deep Learning-Based Segmentation and Quantification in Experimental Kidney Histopathology.

BACKGROUND: Nephropathologic analyses provide important outcomes-related data in experiments with the animal models that are essential for understanding kidney disease pathophysiology. Precision medicine increases the demand for quantitative, unbiased, reproducible, and efficient histopathologic analyses, which will require novel high-throughput tools. A deep learning technique, the convolutional neural network, is increasingly applied in pathology because of its high performance in tasks like histology segmentation. METHODS: We investigated use of a convolutional neural network architecture for accurate segmentation of periodic acid-Schiff-stained kidney tissue from healthy mice and five murine disease models and from other species used in preclinical research. We trained the convolutional neural network to segment six major renal structures: glomerular tuft, glomerulus including Bowman's capsule, tubules, arteries, arterial lumina, and veins. To achieve high accuracy, we performed a large number of expert-based annotations, 72,722 in total. RESULTS: Multiclass segmentation performance was very high in all disease models. The convolutional neural network allowed high-throughput and large-scale, quantitative and comparative analyses of various models. In disease models, computational feature extraction revealed interstitial expansion, tubular dilation and atrophy, and glomerular size variability. Validation showed a high correlation of findings with current standard morphometric analysis. The convolutional neural network also showed high performance in other species used in research-including rats, pigs, bears, and marmosets-as well as in humans, providing a translational bridge between preclinical and clinical studies. CONCLUSIONS: We developed a deep learning algorithm for accurate multiclass segmentation of digital whole-slide images of periodic acid-Schiff-stained kidneys from various species and renal disease models. This enables reproducible quantitative histopathologic analyses in preclinical models that also might be applicable to clinical studies.

Immune status is prognostic for poor survival in colorectal cancer patients and is associated with tumour hypoxia.

BACKGROUND: Immunohistochemical quantification of the immune response is prognostic for colorectal cancer (CRC). Here, we evaluate the suitability of alternative immune classifiers on prognosis and assess whether they relate to biological features amenable to targeted therapy. METHODS: Overall survival by immune (CD3, CD4, CD8, CD20 and FOXP3) and immune-checkpoint (ICOS, IDO-1 and PD-L1) biomarkers in independent CRC cohorts was evaluated. Matched mutational and transcriptomic data were interrogated to identify associated biology. RESULTS: Determination of immune-cold tumours by combined low-density cell counts of CD3, CD4 and CD8 immunohistochemistry constituted the best prognosticator across stage II-IV CRC, particularly in patients with stage IV disease (HR 1.98 [95% CI: 1.47-2.67]). These immune-cold CRCs were associated with tumour hypoxia, confirmed using CAIX immunohistochemistry (P = 0.0009), which may mediate disease progression through common biology (KRAS mutations, CRIS-B subtype and SPP1 mRNA overexpression). CONCLUSIONS: Given the significantly poorer survival of immune-cold CRC patients, these data illustrate that assessment of CD4-expressing cells complements low CD3 and CD8 immunohistochemical quantification in the tumour bulk, potentially facilitating immunophenotyping of patient biopsies to predict prognosis. In addition, we found immune-cold CRCs to associate with a difficult-to-treat, poor prognosis hypoxia signature, indicating that these patients may benefit from hypoxia-targeting clinical trials.

Pan-cancer image-based detection of clinically actionable genetic alterations.

Molecular alterations in cancer can cause phenotypic changes in tumor cells and their micro-environment. Routine histopathology tissue slides - which are ubiquitously available - can reflect such morphological changes. Here, we show that deep learning can consistently infer a wide range of genetic mutations, molecular tumor subtypes, gene expression signatures and standard pathology biomarkers directly from routine histology. We developed, optimized, validated and publicly released a one-stop-shop workflow and applied it to tissue slides of more than 5000 patients across multiple solid tumors. Our findings show that a single deep learning algorithm can be trained to predict a wide range of molecular alterations from routine, paraffin-embedded histology slides stained with hematoxylin and eosin. These predictions generalize to other populations and are spatially resolved. Our method can be implemented on mobile hardware, potentially enabling point-of-care diagnostics for personalized cancer treatment. More generally, this approach could elucidate and quantify genotype-phenotype links in cancer.

Human Pancreatic Carcinoma-Associated Fibroblasts Promote Expression of Co-inhibitory Markers on CD4+ and CD8+ T-Cells.

Carcinoma-associated pancreatic fibroblasts (CAFs) are the major type of cells in the stroma of pancreatic ductal adenocarcinomas and besides their pathological release of extracellular matrix proteins, they are also perceived as key contributors to immune evasion. Despite the known relevance of tumor infiltrating lymphocytes in cancers, the interactions between T-cells and CAFs remain largely unexplored. Here, we found that CAFs isolated from tumors of pancreatic cancer patients undergoing surgical resection (n = 15) expressed higher levels of the PD-1 ligands PD-L1 and PD-L2 compared to primary skin fibroblasts from healthy donors. CAFs strongly inhibited T-cell proliferation in a contact-independent fashion. Blocking the activity of prostaglandin E2 (PGE2) by indomethacin partially restored the proliferative capacity of both CD4+ and CD8+ T-cells. After stimulation, the proportion of proliferating T-cells expressing HLA-DR and the proportion of memory T-cells were decreased when CAFs were present compared to T-cells proliferating in the absence of CAFs. Interestingly, CAFs promoted the expression of TIM-3, PD-1, CTLA-4 and LAG-3 in proliferating T-cells. Immunohistochemistry stainings further showed that T-cells residing within the desmoplastic stromal compartment express PD-1, indicating a role for CAFs on co-inhibitory marker expression also in vivo. We further found that PGE2 promoted the expression of PD-1 and TIM-3 on T-cells. Functional assays showed that proliferating T-cells expressing immune checkpoints produced less IFN-γ, TNF-α, and CD107a after restimulation when CAFs had been present. Thus, this indicates that CAFs induce expression of immune checkpoints on CD4+ and CD8+ T-cells, which contribute to a diminished immune function.

Topography of cancer-associated immune cells in human solid tumors.

Lymphoid and myeloid cells are abundant in the tumor microenvironment, can be quantified by immunohistochemistry and shape the disease course of human solid tumors. Yet, there is no comprehensive understanding of spatial immune infiltration patterns ('topography') across cancer entities and across various immune cell types. In this study, we systematically measure the topography of multiple immune cell types in 965 histological tissue slides from N = 177 patients in a pan-cancer cohort. We provide a definition of inflamed ('hot'), non-inflamed ('cold') and immune excluded patterns and investigate how these patterns differ between immune cell types and between cancer types. In an independent cohort of N = 287 colorectal cancer patients, we show that hot, cold and excluded topographies for effector lymphocytes (CD8) and tumor-associated macrophages (CD163) alone are not prognostic, but that a bivariate classification system can stratify patients. Our study adds evidence to consider immune topographies as biomarkers for patients with solid tumors.

Characterization of a murine mixed neuron-glia model and cellular responses to regulatory T cell-derived factors.

One of the unmet clinical needs in demyelinating diseases such as Multiple Sclerosis (MS) is to provide therapies that actively enhance the process of myelin regeneration (remyelination) in the central nervous system (CNS). Oligodendrocytes, the myelinating cells of the CNS, play a central role in remyelination and originate from oligodendrocyte progenitor cells (OPCs). We recently showed that depletion of regulatory T cells (Treg) impairs remyelination in vivo, and that Treg-secreted factors directly enhance oligodendrocyte differentiation. Here we aim to further characterize the dynamics of Treg-enhanced oligodendrocyte differentiation as well as elucidate the cellular components of a murine mixed neuron-glia model. Murine mixed neuron-glia cultures were generated from P2-7 C57BL/6 mice and characterized for percentage of neuronal and glial cell populations prior to treatment at 7 days in vitro (div) as well as after treatment with Treg-conditioned media at multiple timepoints up to 12 div. Mixed neuron-glia cultures consisted of approximately 30% oligodendroglial lineage cells, 20% neurons and 10% microglia. Furthermore, a full layer of astrocytes, that could not be quantified, was present. Treatment with Treg-conditioned media enhanced the proportion of MBP+ oligodendrocytes and decreased the proportion of PDGFRα+ OPCs, but did not affect OPC proliferation or survival. Treg-enhanced oligodendrocyte differentiation was not caused by Treg polarizing factors, was dependent on the number of activation cycles Treg underwent and was robustly achieved by using 5% conditioned media. These studies provide in-depth characterization of a murine mixed neuron-glia model as well as further insights into the dynamics of Treg-enhanced oligodendrocyte differentiation.