Spatial heterogeneity of cancer associated protein expression in immunohistochemically stained images as an improved prognostic biomarker.

The identification of new tumor biomarkers for patient stratification before therapy, for monitoring of disease progression, and for characterization of tumor biology plays a crucial role in cancer research. The status of these biomarkers is mostly scored manually by a pathologist and such scores typically, do not consider the spatial heterogeneity of the protein's expression in the tissue. Using advanced image analysis methods, marker expression can be determined quantitatively with high accuracy and reproducibility on a per-cell level. To aggregate such per-cell marker expressions on a patient level, the expression values for single cells are usually averaged for the whole tissue. However, averaging neglects the spatial heterogeneity of the marker expression in the tissue. We present two novel approaches for quantitative scoring of spatial marker expression heterogeneity. The first approach is based on a co-occurrence analysis of the marker expression in neighboring cells. The second approach accounts for the local variability of the protein's expression by tiling the tissue with a regular grid and assigning local spatial heterogeneity phenotypes per tile. We apply our novel scores to quantify the spatial expression of four different membrane markers, i.e., HER2, CMET, CD44, and EGFR in immunohistochemically (IHC) stained tissue sections of colorectal cancer patients. We evaluate the prognostic relevance of our spatial scores in this cohort and show that the spatial heterogeneity scores clearly outperform the marker expression average as a prognostic factor (CMET: p-value=0.01 vs. p-value=0.3).

Computational Tumor Infiltration Phenotypes Enable the Spatial and Genomic Analysis of Immune Infiltration in Colorectal Cancer.

Cancer immunotherapy has led to significant therapeutic progress in the treatment of metastatic and formerly untreatable tumors. However, drug response rates are variable and often only a subgroup of patients will show durable response to a treatment. Biomarkers that help to select those patients that will benefit the most from immunotherapy are thus of crucial importance. Here, we aim to identify such biomarkers by investigating the tumor microenvironment, i.e., the interplay between different cell types like immune cells, stromal cells and malignant cells within the tumor and developed a computational method that determines spatial tumor infiltration phenotypes. Our method is based on spatial point pattern analysis of immunohistochemically stained colorectal cancer tumor tissue and accounts for the intra-tumor heterogeneity of immune infiltration. We show that, compared to base-line models, tumor infiltration phenotypes provide significant additional support for the prediction of established biomarkers in a colorectal cancer patient cohort (n = 80). Integration of tumor infiltration phenotypes with genetic and genomic data from the same patients furthermore revealed significant associations between spatial infiltration patterns and common mutations in colorectal cancer and gene expression signatures. Based on these associations, we computed novel gene signatures that allow one to predict spatial tumor infiltration patterns from gene expression data only and validated this approach in a separate dataset from the Cancer Genome Atlas.

Analysis of Spatial Organization of Suppressive Myeloid Cells and Effector T Cells in Colorectal Cancer-A Potential Tool for Discovering Prognostic Biomarkers in Clinical Research.

The development and progression of solid tumors such as colorectal cancer (CRC) are known to be affected by the immune system and cell types such as T cells, natural killer (NK) cells, and natural killer T (NKT) cells are emerging as interesting targets for immunotherapy and clinical biomarker research. In addition, CD3+ and CD8+ T cell distribution in tumors has shown positive prognostic value in stage I-III CRC. Recent developments in digital computational pathology support not only classical cell density based tumor characterization, but also a more comprehensive analysis of the spatial cell organization in the tumor immune microenvironment (TiME). Leveraging that methodology in the current study, we tried to address the question of how the distribution of myeloid derived suppressor cells in TiME of primary CRC affects the function and location of cytotoxic T cells. We applied multicolored immunohistochemistry to identify monocytic (CD11b+CD14+) and granulocytic (CD11b+CD15+) myeloid cell populations together with proliferating and non-proliferating cytotoxic T cells (CD8+Ki67+/-). Through automated object detection and image registration using HALO software (IndicaLabs), we applied dedicated spatial statistics to measure the extent of overlap between the areas occupied by myeloid and T cells. With this approach, we observed distinct spatial organizational patterns of immune cells in tumors obtained from 74 treatment-naive CRC patients. Detailed analysis of inter-cell distances and myeloid-T cell spatial overlap combined with integrated gene expression data allowed to stratify patients irrespective of their mismatch repair (MMR) status or consensus molecular subgroups (CMS) classification. In addition, generation of cell distance-derived gene signatures and their mapping to the TCGA data set revealed associations between spatial immune cell distribution in TiME and certain subsets of CD8+ and CD4+ T cells. The presented study sheds a new light on myeloid and T cell interactions in TiME in CRC patients. Our results show that CRC tumors present distinct distribution patterns of not only T effector cells but also tumor resident myeloid cells, thus stressing the necessity of more comprehensive characterization of TiME in order to better predict cancer prognosis. This research emphasizes the importance of a multimodal approach by combining computational pathology with its detailed spatial statistics and gene expression profiling. Finally, our study presents a novel approach to cancer patients' characterization that can potentially be used to develop new immunotherapy strategies, not based on classical biomarkers related to CRC biology.

Topological Tumor Graphs: A Graph-Based Spatial Model to Infer Stromal Recruitment for Immunosuppression in Melanoma Histology.

Despite the advent of immunotherapy, metastatic melanoma represents an aggressive tumor type with a poor survival outcome. The successful application of immunotherapy requires in-depth understanding of the biological basis and immunosuppressive mechanisms within the tumor microenvironment. In this study, we conducted spatially explicit analyses of the stromal-immune interface across 400 melanoma hematoxylin and eosin (H&E) specimens from The Cancer Genome Atlas. A computational pathology pipeline (CRImage) was used to classify cells in the H&E specimen into stromal, immune, or cancer cells. The estimated proportions of these cell types were validated by independent measures of tumor purity, pathologists' estimate of lymphocyte density, imputed immune cell subtypes, and pathway analyses. Spatial interactions between these cell types were computed using a graph-based algorithm (topological tumor graphs, TTG). This approach identified two stromal features, namely stromal clustering and stromal barrier, which represented the melanoma stromal microenvironment. Tumors with increased stromal clustering and barrier were associated with reduced intratumoral lymphocyte distribution and poor overall survival independent of existing prognostic factors. To explore the genomic basis of these TTG-derived stromal phenotypes, we used a deep learning approach integrating genomic (copy number) and transcriptomic data, thereby inferring a compressed representation of copy number-driven alterations in gene expression. This integrative analysis revealed that tumors with high stromal clustering and barrier had reduced expression of pathways involved in naïve CD4 signaling, MAPK, and PI3K signaling. Taken together, our findings support the immunosuppressive role of stromal cells and T-cell exclusion within the vicinity of melanoma cells. SIGNIFICANCE: Computational histology-based stromal phenotypes within the tumor microenvironment are significantly associated with prognosis and immune exclusion in melanoma.

Superpixel-Based Conditional Random Fields (SuperCRF): Incorporating Global and Local Context for Enhanced Deep Learning in Melanoma Histopathology.

Computational pathology-based cell classification algorithms are revolutionizing the study of the tumor microenvironment and can provide novel predictive/prognosis biomarkers crucial for the delivery of precision oncology. Current algorithms used on hematoxylin and eosin slides are based on individual cell nuclei morphology with limited local context features. Here, we propose a novel multi-resolution hierarchical framework (SuperCRF) inspired by the way pathologists perceive regional tissue architecture to improve cell classification and demonstrate its clinical applications. We develop SuperCRF by training a state-of-art deep learning spatially constrained- convolution neural network (SC-CNN) to detect and classify cells from 105 high-resolution (20×) H&E-stained slides of The Cancer Genome Atlas melanoma dataset and subsequently, a conditional random field (CRF) by combining cellular neighborhood with tumor regional classification from lower resolution images (5, 1.25×) given by a superpixel-based machine learning framework. SuperCRF led to an 11.85% overall improvement in the accuracy of the state-of-art deep learning SC-CNN cell classifier. Consistent with a stroma-mediated immune suppressive microenvironment, SuperCRF demonstrated that (i) a high ratio of lymphocytes to all lymphocytes within the stromal compartment (p = 0.026) and (ii) a high ratio of stromal cells to all cells (p < 0.0001 compared to p = 0.039 for SC-CNN only) are associated with poor survival in patients with melanoma. SuperCRF improves cell classification by introducing global and local context-based information and can be implemented in combination with any single-cell classifier. SuperCRF provides valuable tools to study the tumor microenvironment and identify predictors of survival and response to therapy.

Clustering of samples with a tree-shaped dependence structure, with an application to microscopic time lapse imaging.

MOTIVATION: Recent imaging technologies allow for high-throughput tracking of cells as they migrate, divide, express fluorescent markers and change their morphology. The interpretation of these data requires unbiased, efficient statistical methods that model the dynamics of cell phenotypes. RESULTS: We introduce treeHFM, a probabilistic model which generalizes the theory of hidden Markov models to tree structured data. While accounting for the entire genealogy of a cell, treeHFM categorizes cells according to their primary phenotypic features. It models all relevant events in a cell's life, including cell division, and thereby enables the analysis of event order and cell fate heterogeneity. Simulations show higher accuracy in predicting correct state labels when modeling the more complex, tree-shaped dependency of samples over standard HMM modeling. Applying treeHFM to time lapse images of hematopoietic progenitor cell differentiation, we demonstrate that progenitor cells undergo a well-ordered sequence of differentiation events. AVAILABILITY AND IMPLEMENTATION: The treeHFM is implemented in C++. We provide wrapper functions for the programming languages R (CRAN package, https://CRAN.R-project.org/package=treeHFM) and Matlab (available at Mathworks Central, http://se.mathworks.com/matlabcentral/fileexchange/57575-treehfml). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Interfaces of Malignant and Immunologic Clonal Dynamics in Ovarian Cancer.

High-grade serous ovarian cancer (HGSC) exhibits extensive malignant clonal diversity with widespread but non-random patterns of disease dissemination. We investigated whether local immune microenvironment factors shape tumor progression properties at the interface of tumor-infiltrating lymphocytes (TILs) and cancer cells. Through multi-region study of 212 samples from 38 patients with whole-genome sequencing, immunohistochemistry, histologic image analysis, gene expression profiling, and T and B cell receptor sequencing, we identified three immunologic subtypes across samples and extensive within-patient diversity. Epithelial CD8+ TILs negatively associated with malignant diversity, reflecting immunological pruning of tumor clones inferred by neoantigen depletion, HLA I loss of heterozygosity, and spatial tracking between T cell and tumor clones. In addition, combinatorial prognostic effects of mutational processes and immune properties were observed, illuminating how specific genomic aberration types associate with immune response and impact survival. We conclude that within-patient spatial immune microenvironment variation shapes intraperitoneal malignant spread, provoking new evolutionary perspectives on HGSC clonal dispersion.

MowJoe: a method for automated-high throughput dissected leaf phenotyping.

BACKGROUND: Accurate and automated phenotyping of leaf images is necessary for high throughput studies of leaf form like genome-wide association analysis and other forms of quantitative trait locus mapping. Dissected leaves (also referred to as compound) that are subdivided into individual units are an attractive system to study diversification of form. However, there are only few software tools for their automated analysis. Thus, high-throughput image processing algorithms are needed that can partition these leaves in their phenotypically relevant units and calculate morphological features based on these units. RESULTS: We have developed MowJoe, an image processing algorithm that dissects a dissected leaf into leaflets, petiolule, rachis and petioles. It employs image skeletonization to convert leaves into graphs, and thereafter applies algorithms operating on graph structures. This partitioning of a leaf allows the derivation of morphological features such as leaf size, or eccentricity of leaflets. Furthermore, MowJoe automatically places landmarks onto the terminal leaflet that can be used for further leaf shape analysis. It generates specific output files that can directly be imported into downstream shape analysis tools. We applied the algorithm to two accessions of Cardamine hirsuta and show that our features are able to robustly discriminate between these accessions. CONCLUSION: MowJoe is a tool for the semi-automated, quantitative high throughput shape analysis of dissected leaf images. It provides the statistical power for the detection of the genetic basis of quantitative morphological variations.

Continuous single cell imaging reveals sequential steps of plasmacytoid dendritic cell development from common dendritic cell progenitors.

Functionally distinct plasmacytoid and conventional dendritic cells (pDC and cDC) shape innate and adaptive immunity. They are derived from common dendritic cell progenitors (CDPs) in the murine bone marrow, which give rise to CD11c+ MHCII- precursors with early commitment to DC subpopulations. In this study, we dissect pDC development from CDP into an ordered sequence of differentiation events by monitoring the expression of CD11c, MHC class II, Siglec H and CCR9 in CDP cultures by continuous single cell imaging and tracking. Analysis of CDP genealogies revealed a stepwise differentiation of CDPs into pDCs in a part of the CDP colonies. This developmental pathway involved an early CD11c+ SiglecH- pre-DC stage and a Siglec H+ CCR9low precursor stage, which was followed rapidly by upregulation of CCR9 indicating final pDC differentiation. In the majority of the remaining CDP pedigrees however the Siglec H+ CCR9low precursor state was maintained for several generations. Thus, although a fraction of CDPs transits through precursor stages rapidly to give rise to a first wave of pDCs, the majority of CDP progeny differentiate more slowly and give rise to longer lived precursor cells which are poised to differentiate on demand.

Factor graph analysis of live cell-imaging data reveals mechanisms of cell fate decisions.

MOTIVATION: Cell fate decisions have a strong stochastic component. The identification of the underlying mechanisms therefore requires a rigorous statistical analysis of large ensembles of single cells that were tracked and phenotyped over time. RESULTS: We introduce a probabilistic framework for testing elementary hypotheses on dynamic cell behavior using time-lapse cell-imaging data. Factor graphs, probabilistic graphical models, are used to properly account for cell lineage and cell phenotype information. Our model is applied to time-lapse movies of murine granulocyte-macrophage progenitor (GMP) cells. It decides between competing hypotheses on the mechanisms of their differentiation. Our results theoretically substantiate previous experimental observations that lineage instruction, not selection is the cause for the differentiation of GMP cells into mature monocytes or neutrophil granulocytes. AVAILABILITY AND IMPLEMENTATION: The Matlab source code is available at http://treschgroup.de/Genealogies.html.

Unsupervised automated high throughput phenotyping of RNAi time-lapse movies.

BACKGROUND: Gene perturbation experiments in combination with fluorescence time-lapse cell imaging are a powerful tool in reverse genetics. High content applications require tools for the automated processing of the large amounts of data. These tools include in general several image processing steps, the extraction of morphological descriptors, and the grouping of cells into phenotype classes according to their descriptors. This phenotyping can be applied in a supervised or an unsupervised manner. Unsupervised methods are suitable for the discovery of formerly unknown phenotypes, which are expected to occur in high-throughput RNAi time-lapse screens. RESULTS: We developed an unsupervised phenotyping approach based on Hidden Markov Models (HMMs) with multivariate Gaussian emissions for the detection of knockdown-specific phenotypes in RNAi time-lapse movies. The automated detection of abnormal cell morphologies allows us to assign a phenotypic fingerprint to each gene knockdown. By applying our method to the Mitocheck database, we show that a phenotypic fingerprint is indicative of a gene's function. CONCLUSION: Our fully unsupervised HMM-based phenotyping is able to automatically identify cell morphologies that are specific for a certain knockdown. Beyond the identification of genes whose knockdown affects cell morphology, phenotypic fingerprints can be used to find modules of functionally related genes.

Global analysis of eukaryotic mRNA degradation reveals Xrn1-dependent buffering of transcript levels.

The rates of mRNA synthesis and degradation determine cellular mRNA levels and can be monitored by comparative dynamic transcriptome analysis (cDTA) that uses nonperturbing metabolic RNA labeling. Here we present cDTA data for 46 yeast strains lacking genes involved in mRNA degradation and metabolism. In these strains, changes in mRNA degradation rates are generally compensated by changes in mRNA synthesis rates, resulting in a buffering of mRNA levels. We show that buffering of mRNA levels requires the RNA exonuclease Xrn1. The buffering is rapidly established when mRNA synthesis is impaired, but is delayed when mRNA degradation is impaired, apparently due to Xrn1-dependent transcription repressor induction. Cluster analysis of the data defines the general mRNA degradation machinery, reveals different substrate preferences for the two mRNA deadenylase complexes Ccr4-Not and Pan2-Pan3, and unveils an interwoven cellular mRNA surveillance network.

Semi-automated 3D leaf reconstruction and analysis of trichome patterning from light microscopic images.

Trichomes are leaf hairs that are formed by single cells on the leaf surface. They are known to be involved in pathogen resistance. Their patterning is considered to emerge from a field of initially equivalent cells through the action of a gene regulatory network involving trichome fate promoting and inhibiting factors. For a quantitative analysis of single and double mutants or the phenotypic variation of patterns in different ecotypes, it is imperative to statistically evaluate the pattern reliably on a large number of leaves. Here we present a method that enables the analysis of trichome patterns at early developmental leaf stages and the automatic analysis of various spatial parameters. We focus on the most challenging young leaf stages that require the analysis in three dimensions, as the leaves are typically not flat. Our software TrichEratops reconstructs 3D surface models from 2D stacks of conventional light-microscope pictures. It allows the GUI-based annotation of different stages of trichome development, which can be analyzed with respect to their spatial distribution to capture trichome patterning events. We show that 3D modeling removes biases of simpler 2D models and that novel trichome patterning features increase the sensitivity for inter-accession comparisons.

Learning gene network structure from time laps cell imaging in RNAi Knock downs.

MOTIVATION: As RNA interference is becoming a standard method for targeted gene perturbation, computational approaches to reverse engineer parts of biological networks based on measurable effects of RNAi become increasingly relevant. The vast majority of these methods use gene expression data, but little attention has been paid so far to other data types. RESULTS: Here we present a method, which can infer gene networks from high-dimensional phenotypic perturbation effects on single cells recorded by time-lapse microscopy. We use data from the Mitocheck project to extract multiple shape, intensity and texture features at each frame. Features from different cells and movies are then aligned along the cell cycle time. Subsequently we use Dynamic Nested Effects Models (dynoNEMs) to estimate parts of the network structure between perturbed genes via a Markov Chain Monte Carlo approach. Our simulation results indicate a high reconstruction quality of this method. A reconstruction based on 22 gene knock downs yielded a network, where all edges could be explained via the biological literature. AVAILABILITY: The implementation of dynoNEMs is part of the Bioconductor R-package nem.

Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling.

Solid tumors are heterogeneous tissues composed of a mixture of cancer and normal cells, which complicates the interpretation of their molecular profiles. Furthermore, tissue architecture is generally not reflected in molecular assays, rendering this rich information underused. To address these challenges, we developed a computational approach based on standard hematoxylin and eosin-stained tissue sections and demonstrated its power in a discovery and validation cohort of 323 and 241 breast tumors, respectively. To deconvolute cellular heterogeneity and detect subtle genomic aberrations, we introduced an algorithm based on tumor cellularity to increase the comparability of copy number profiles between samples. We next devised a predictor for survival in estrogen receptor-negative breast cancer that integrated both image-based and gene expression analyses and significantly outperformed classifiers that use single data types, such as microarray expression signatures. Image processing also allowed us to describe and validate an independent prognostic factor based on quantitative analysis of spatial patterns between stromal cells, which are not detectable by molecular assays. Our quantitative, image-based method could benefit any large-scale cancer study by refining and complementing molecular assays of tumor samples.