Whole-Slide Imaging, Mutual Information Registration for Multiplex Immunohistochemistry and Immunofluorescence.

Multiplex immunohistochemistry/immunofluorescence (mIHC/mIF) is a developing technology that facilitates the evaluation of multiple, simultaneous protein expressions at single-cell resolution while preserving tissue architecture. These approaches have shown great potential for biomarker discovery, yet many challenges remain. Importantly, streamlined cross-registration of multiplex immunofluorescence images with additional imaging modalities and immunohistochemistry (IHC) can help increase the plex and/or improve the quality of the data generated by potentiating downstream processes such as cell segmentation. To address this problem, a fully automated process was designed to perform a hierarchical, parallelizable, and deformable registration of multiplexed digital whole-slide images (WSIs). We generalized the calculation of mutual information as a registration criterion to an arbitrary number of dimensions, making it well suited for multiplexed imaging. We also used the self-information of a given IF channel as a criterion to select the optimal channels to use for registration. Additionally, as precise labeling of cellular membranes in situ is essential for robust cell segmentation, a pan-membrane immunohistochemical staining method was developed for incorporation into mIF panels or for use as an IHC followed by cross-registration. In this study, we demonstrate this process by registering whole-slide 6-plex/7-color mIF images with whole-slide brightfield mIHC images, including a CD3 and a pan-membrane stain. Our algorithm, WSI, mutual information registration (WSIMIR), performed highly accurate registration allowing the retrospective generation of an 8-plex/9-color, WSI, and outperformed 2 alternative automated methods for cross-registration by Jaccard index and Dice similarity coefficient (WSIMIR vs automated WARPY, P < .01 and P < .01, respectively, vs HALO + transformix, P = .083 and P = .049, respectively). Furthermore, the addition of a pan-membrane IHC stain cross-registered to an mIF panel facilitated improved automated cell segmentation across mIF WSIs, as measured by significantly increased correct detections, Jaccard index (0.78 vs 0.65), and Dice similarity coefficient (0.88 vs 0.79).

The association between the tumor immune microenvironments and clinical outcome in low-grade, early-stage endometrial cancer patients.

Endometrial tumors show substantial heterogeneity in their immune microenvironment. This heterogeneity could be used to improve the accuracy of current outcome prediction tools. We assessed the immune microenvironment of 235 patients diagnosed with low-grade, early-stage endometrial cancer. Multiplex quantitative immunofluorescence was carried out to measure CD8, CD68, FOXP3, PD-1, and PD-L1 markers, as well as cytokeratin (CK), on tissue microarrays. Clustering results revealed five robust immune response patterns, each associated with specific immune populations, cell phenotypes, and cell spatial clustering. Most samples (69%) belonged to the immune-desert subtype, characterized by low immune cell densities. Tumor-infiltrating lymphocyte (TIL)-rich samples (4%) displayed high CD8+ T-cell infiltration, as well as a high percentage of CD8/PD-1+ cells. Immune-exclusion samples (19%) displayed the lowest CD8+ infiltration combined with high PD-L1 expression levels in CK+ tumor cells. In addition, they demonstrated high tumor cell spatial clustering as well as increased spatial proximity of CD8+ /PD-1+ and CK/PD-L1+ cells. FOXP3 and macrophage-rich phenotypes (3% and 4% of total samples) displayed relatively high levels of FOXP3+ regulatory T-cells and CD68+ macrophages, respectively. These phenotypes correlated with clinical outcomes, with immune-exclusion tumors showing an association with tumor relapse. When compared with prediction models built using routine pathological variables, models optimized with immune variables showed increased outcome prediction capacity (AUC = 0.89 versus 0.78) and stratification potential. The improved prediction capacity was independent of mismatch repair protein status and adjuvant radiotherapy treatment. Further, immunofluorescence results could be partially recapitulated using single-marker immunohistochemistry (IHC) performed on whole tissue sections. TIL-rich tumors demonstrated increased CD8+ T-cells by IHC, while immune-exclusion tumors displayed a lack of CD8+ T-cells and frequent expression of PD-L1 in tumor cells. Our results demonstrate the capability of the immune microenvironment to improve standard prediction tools in low-grade, early-stage endometrial carcinomas. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

NMF-RI: blind spectral unmixing of highly mixed multispectral flow and image cytometry data.

MOTIVATION: Recent advances in multiplex immunostaining and multispectral cytometry have opened the door to simultaneously visualizing an unprecedented number of biomarkers both in liquid and solid samples. Properly unmixing fluorescent emissions is a challenging task, which normally requires the characterization of the individual fluorochromes from control samples. As the number of fluorochromes increases, the cost in time and use of reagents becomes prohibitively high. Here, we present a fully unsupervised blind spectral unmixing method for the separation of fluorescent emissions in highly mixed spectral data, without the need for control samples. To this end, we extend an existing method based on non-negative Matrix Factorization, and introduce several critical improvements: initialization based on the theoretical spectra, automated selection of 'sparse' data and use of a re-initialized multilayer optimizer. RESULTS: Our algorithm is exhaustively tested using synthetic data to study its robustness against different levels of colocalization, signal to noise ratio, spectral resolution and the effect of errors in the initialization of the algorithm. Then, we compare the performance of our method to that of traditional spectral unmixing algorithms using novel multispectral flow and image cytometry systems. In all cases, we show that our blind unmixing algorithm performs robust unmixing of highly spatially and spectrally mixed data with an unprecedently low computational cost. In summary, we present the first use of a blind unmixing method in multispectral flow and image cytometry, opening the door to the widespread use of our method to efficiently pre-process multiplex immunostaining samples without the need of experimental controls. AVAILABILITY AND IMPLEMENTATION: https://github.com/djimenezsanchez/Blind_Unmixing_NMF_RI/ contains the source code and all datasets used in this manuscript. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Spatially resolved tissue imaging to analyze the tumor immune microenvironment: beyond cell-type densities.

INTRODUCTION: The tissue immune microenvironment is associated with key aspects of tumor biology. The interaction between the immune system and cancer cells has predictive and prognostic potential across different tumor types. Spatially resolved tissue-based technologies allowed researchers to simultaneously quantify different immune populations in tumor samples. However, bare quantification fails to harness the spatial nature of tissue-based technologies. Tumor-immune interactions are associated with specific spatial patterns that can be measured. In recent years, several computational tools have been developed to increase our understanding of these spatial patterns. TOPICS COVERED: In this review, we cover standard techniques as well as new advances in the field of spatial analysis of the immune microenvironment. We focused on marker quantification, spatial intratumor heterogeneity analysis, cell‒cell spatial interaction studies and neighborhood analyses.

Synplex: In Silico Modeling of the Tumor Microenvironment From Multiplex Images.

Multiplex immunofluorescence is a novel, high-content imaging technique that allows simultaneous in situ labeling of multiple tissue antigens. This technique is of growing relevance in the study of the tumor microenvironment, and the discovery of biomarkers of disease progression or response to immune-based therapies. Given the number of markers and the potential complexity of the spatial interactions involved, the analysis of these images requires the use of machine learning tools that rely for their training on the availability of large image datasets, extremely laborious to annotate. We present Synplex, a computer simulator of multiplexed immunofluorescence images from user-defined parameters: i. cell phenotypes, defined by the level of expression of markers and morphological parameters; ii. cellular neighborhoods based on the spatial association of cell phenotypes; and iii. interactions between cellular neighborhoods. We validate Synplex by generating synthetic tissues that accurately simulate real cancer cohorts with underlying differences in the composition of their tumor microenvironment and show proof-of-principle examples of how Synplex could be used for data augmentation when training machine learning models, and for the in silico selection of clinically relevant biomarkers. Synplex is publicly available at https://github.com/djimenezsanchez/Synplex.

Weakly supervised deep learning to predict recurrence in low-grade endometrial cancer from multiplexed immunofluorescence images.

Predicting recurrence in low-grade, early-stage endometrial cancer (EC) is both challenging and clinically relevant. We present a weakly-supervised deep learning framework, NaroNet, that can learn, without manual expert annotation, the complex tumor-immune interrelations at three levels: local phenotypes, cellular neighborhoods, and tissue areas. It uses multiplexed immunofluorescence for the simultaneous visualization and quantification of CD68 + macrophages, CD8 + T cells, FOXP3 + regulatory T cells, PD-L1/PD-1 protein expression, and tumor cells. We used 489 tumor cores from 250 patients to train a multilevel deep-learning model to predict tumor recurrence. Using a tenfold cross-validation strategy, our model achieved an area under the curve of 0.90 with a 95% confidence interval of 0.83-0.95. Our model predictions resulted in concordance for 96,8% of cases (κ = 0.88). This method could accurately assess the risk of recurrence in EC, outperforming current prognostic factors, including molecular subtyping.

NaroNet: Discovery of tumor microenvironment elements from highly multiplexed images.

Understanding the spatial interactions between the elements of the tumor microenvironment -i.e. tumor cells. fibroblasts, immune cells- and how these interactions relate to the diagnosis or prognosis of a tumor is one of the goals of computational pathology. We present NaroNet, a deep learning framework that models the multi-scale tumor microenvironment from multiplex-stained cancer tissue images and provides patient-level interpretable predictions using a seamless end-to-end learning pipeline. Trained only with multiplex-stained tissue images and their corresponding patient-level clinical labels, NaroNet unsupervisedly learns which cell phenotypes, cell neighborhoods, and neighborhood interactions have the highest influence to predict the correct label. To this end, NaroNet incorporates several novel and state-of-the-art deep learning techniques, such as patch-level contrastive learning, multi-level graph embeddings, a novel max-sum pooling operation, or a metric that quantifies the relevance that each microenvironment element has in the individual predictions. We validate NaroNet using synthetic data simulating multiplex-immunostained images where a patient label is artificially associated to the -adjustable- probabilistic incidence of different microenvironment elements. We then apply our model to two sets of images of human cancer tissues: 336 seven-color multiplex-immunostained images from 12 high-grade endometrial cancer patients; and 382 35-plex mass cytometry images from 215 breast cancer patients. In both synthetic and real datasets, NaroNet provides outstanding predictions of relevant clinical information while associating those predictions to the presence of specific microenvironment elements.

Neutrophil Extracellular Traps, Local IL-8 Expression, and Cytotoxic T-Lymphocyte Response in the Lungs of Patients With Fatal COVID-19.

BACKGROUND: Excessive inflammation is pathogenic in the pneumonitis associated with severe COVID-19. Neutrophils are among the most abundantly present leukocytes in the inflammatory infiltrates and may form neutrophil extracellular traps (NETs) under the local influence of cytokines. NETs constitute a defense mechanism against bacteria, but have also been shown to mediate tissue damage in a number of diseases. RESEARCH QUESTION: Could NETs and their tissue-damaging properties inherent to neutrophil-associated functions play a role in the respiratory failure seen in patients with severe COVID-19, and how does this relate to the SARS-CoV-2 viral loads, IL-8 (CXCL8) chemokine expression, and cytotoxic T-lymphocyte infiltrates? STUDY DESIGN AND METHODS: Sixteen lung biopsy samples obtained immediately after death were analyzed methodically as exploratory and validation cohorts. NETs were analyzed quantitatively by multiplexed immunofluorescence and were correlated with local levels of IL-8 messenger RNA (mRNA) and the density of CD8+ T-cell infiltration. SARS-CoV-2 presence in tissue was quantified by reverse-transcriptase polymerase chain reaction and immunohistochemistry analysis. RESULTS: NETs were found in the lung interstitium and surrounding the bronchiolar epithelium with interindividual and spatial heterogeneity. NET density did not correlate with SARS-CoV-2 tissue viral load. NETs were associated with local IL-8 mRNA levels. NETs were also detected in pulmonary thrombi and in only one of eight liver tissues. NET focal presence correlated negatively with CD8+ T-cell infiltration in the lungs. INTERPRETATION: Abundant neutrophils undergoing NETosis are found in the lungs of patients with fatal COVID-19, but no correlation was found with viral loads. The strong association between NETs and IL-8 points to this chemokine as a potentially causative factor. The function of cytotoxic T-lymphocytes in the immune responses against SARS-CoV-2 may be interfered with by the presence of NETs.