A general algorithm for consensus 3D cell segmentation from 2D segmented stacks.

Cell segmentation is the fundamental task. Only by segmenting, can we define the quantitative spatial unit for collecting measurements to draw biological conclusions. Deep learning has revolutionized 2D cell segmentation, enabling generalized solutions across cell types and imaging modalities. This has been driven by the ease of scaling up image acquisition, annotation and computation. However 3D cell segmentation, which requires dense annotation of 2D slices still poses significant challenges. Labelling every cell in every 2D slice is prohibitive. Moreover it is ambiguous, necessitating cross-referencing with other orthoviews. Lastly, there is limited ability to unambiguously record and visualize 1000's of annotated cells. Here we develop a theory and toolbox, u-Segment3D for 2D-to-3D segmentation, compatible with any 2D segmentation method. Given optimal 2D segmentations, u-Segment3D generates the optimal 3D segmentation without data training, as demonstrated on 11 real life datasets, >70,000 cells, spanning single cells, cell aggregates and tissue.

Multiplexed 3D Analysis of Immune States and Niches in Human Tissue.

Tissue homeostasis and the emergence of disease are controlled by changes in the proportions of resident and recruited cells, their organization into cellular neighbourhoods, and their interactions with acellular tissue components. Highly multiplexed tissue profiling (spatial omics) 1 makes it possible to study this microenvironment in situ , usually in 4-5 micron thick sections (the standard histopathology format) 2 . Microscopy-based tissue profiling is commonly performed at a resolution sufficient to determine cell types but not to detect subtle morphological features associated with cytoskeletal reorganisation, juxtracrine signalling, or membrane trafficking 3 . Here we describe a high-resolution 3D imaging approach able to characterize a wide variety of organelles and structures at sub-micron scale while simultaneously quantifying millimetre-scale spatial features. This approach combines cyclic immunofluorescence (CyCIF) imaging 4 of over 50 markers with confocal microscopy of archival human tissue thick enough (30-40 microns) to fully encompass two or more layers of intact cells. 3D imaging of entire cell volumes substantially improves the accuracy of cell phenotyping and allows cell proximity to be scored using plasma membrane apposition, not just nuclear position. In pre-invasive melanoma in situ 5 , precise phenotyping shows that adjacent melanocytic cells are plastic in state and participate in tightly localised niches of interferon signalling near sites of initial invasion into the underlying dermis. In this and metastatic melanoma, mature and precursor T cells engage in an unexpectedly diverse array of juxtracrine and membrane-membrane interactions as well as looser "neighbourhood" associations 6 whose morphologies reveal functional states. These data provide new insight into the transitions occurring during early tumour formation and immunoediting and demonstrate the potential for phenotyping of tissues at a level of detail previously restricted to cultured cells and organoids.

Cell Spotter (CSPOT): A machine-learning approach to automated cell spotting and quantification of highly multiplexed tissue images.

Highly multiplexed tissue imaging and in situ spatial profiling aim to extract single-cell data from specimens containing closely packed cells of diverse morphology. This is challenging due to the difficulty of accurately assigning boundaries between cells (segmentation) and then generating per-cell staining intensities. Existing methods use gating to convert per-cell intensity data to positive and negative scores; this is a common approach in flow cytometry, but one that is problematic in imaging. In contrast, human experts identify cells in crowded environments using morphological, neighborhood, and intensity information. Here we describe a computational approach (Cell Spotter or CSPOT) that uses supervised machine learning in combination with classical segmentation to perform automated cell type calling. CSPOT is robust to artifacts that commonly afflict tissue imaging and can replace conventional gating. The end-to-end Python implementation of CSPOT can be integrated into cloud-based image processing pipelines to substantially improve the speed, accuracy, and reproducibility of single-cell spatial data.

2D and 3D multiplexed subcellular profiling of nuclear instability in human cancer.

Nuclear atypia, including altered nuclear size, contour, and chromatin organization, is ubiquitous in cancer cells. Atypical primary nuclei and micronuclei can rupture during interphase; however, the frequency, causes, and consequences of nuclear rupture are unknown in most cancers. We demonstrate that nuclear envelope rupture is surprisingly common in many human cancers, particularly glioblastoma. Using highly-multiplexed 2D and super-resolution 3D-imaging of glioblastoma tissues and patient-derived xenografts and cells, we link primary nuclear rupture with reduced lamin A/C and micronuclear rupture with reduced lamin B1. Moreover, ruptured glioblastoma cells activate cGAS-STING-signaling involved in innate immunity. We observe that local patterning of cell states influences tumor spatial organization and is linked to both lamin expression and rupture frequency, with neural-progenitor-cell-like states exhibiting the lowest lamin A/C levels and greatest susceptibility to primary nuclear rupture. Our study reveals that nuclear instability is a core feature of cancer, and links nuclear integrity, cell state, and immune signaling.

Addressing persistent challenges in digital image analysis of cancerous tissues.

The National Cancer Institute (NCI) supports many research programs and consortia, many of which use imaging as a major modality for characterizing cancerous tissue. A trans-consortia Image Analysis Working Group (IAWG) was established in 2019 with a mission to disseminate imaging-related work and foster collaborations. In 2022, the IAWG held a virtual hackathon focused on addressing challenges of analyzing high dimensional datasets from fixed cancerous tissues. Standard image processing techniques have automated feature extraction, but the next generation of imaging data requires more advanced methods to fully utilize the available information. In this perspective, we discuss current limitations of the automated analysis of multiplexed tissue images, the first steps toward deeper understanding of these limitations, what possible solutions have been developed, any new or refined approaches that were developed during the Image Analysis Hackathon 2022, and where further effort is required. The outstanding problems addressed in the hackathon fell into three main themes: 1) challenges to cell type classification and assessment, 2) translation and visual representation of spatial aspects of high dimensional data, and 3) scaling digital image analyses to large (multi-TB) datasets. We describe the rationale for each specific challenge and the progress made toward addressing it during the hackathon. We also suggest areas that would benefit from more focus and offer insight into broader challenges that the community will need to address as new technologies are developed and integrated into the broad range of image-based modalities and analytical resources already in use within the cancer research community.

High-plex immunofluorescence imaging and traditional histology of the same tissue section for discovering image-based biomarkers.

Precision medicine is critically dependent on better methods for diagnosing and staging disease and predicting drug response. Histopathology using hematoxylin and eosin (H&E)-stained tissue (not genomics) remains the primary diagnostic method in cancer. Recently developed highly multiplexed tissue imaging methods promise to enhance research studies and clinical practice with precise, spatially resolved single-cell data. Here, we describe the 'Orion' platform for collecting H&E and high-plex immunofluorescence images from the same cells in a whole-slide format suitable for diagnosis. Using a retrospective cohort of 74 colorectal cancer resections, we show that immunofluorescence and H&E images provide human experts and machine learning algorithms with complementary information that can be used to generate interpretable, multiplexed image-based models predictive of progression-free survival. Combining models of immune infiltration and tumor-intrinsic features achieves a 10- to 20-fold discrimination between rapid and slow (or no) progression, demonstrating the ability of multimodal tissue imaging to generate high-performance biomarkers.

Multiplexed 3D atlas of state transitions and immune interaction in colorectal cancer.

Advanced solid cancers are complex assemblies of tumor, immune, and stromal cells characterized by high intratumoral variation. We use highly multiplexed tissue imaging, 3D reconstruction, spatial statistics, and machine learning to identify cell types and states underlying morphological features of known diagnostic and prognostic significance in colorectal cancer. Quantitation of these features in high-plex marker space reveals recurrent transitions from one tumor morphology to the next, some of which are coincident with long-range gradients in the expression of oncogenes and epigenetic regulators. At the tumor invasive margin, where tumor, normal, and immune cells compete, T cell suppression involves multiple cell types and 3D imaging shows that seemingly localized 2D features such as tertiary lymphoid structures are commonly interconnected and have graded molecular properties. Thus, while cancer genetics emphasizes the importance of discrete changes in tumor state, whole-specimen imaging reveals large-scale morphological and molecular gradients analogous to those in developing tissues.

UnMICST: Deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues.

Upcoming technologies enable routine collection of highly multiplexed (20-60 channel), subcellular resolution images of mammalian tissues for research and diagnosis. Extracting single cell data from such images requires accurate image segmentation, a challenging problem commonly tackled with deep learning. In this paper, we report two findings that substantially improve image segmentation of tissues using a range of machine learning architectures. First, we unexpectedly find that the inclusion of intentionally defocused and saturated images in training data substantially improves subsequent image segmentation. Such real augmentation outperforms computational augmentation (Gaussian blurring). In addition, we find that it is practical to image the nuclear envelope in multiple tissues using an antibody cocktail thereby better identifying nuclear outlines and improving segmentation. The two approaches cumulatively and substantially improve segmentation on a wide range of tissue types. We speculate that the use of real augmentations will have applications in image processing outside of microscopy.

Multiplexed and reproducible high content screening of live and fixed cells using Dye Drop.

High-throughput measurement of cells perturbed using libraries of small molecules, gene knockouts, or different microenvironmental factors is a key step in functional genomics and pre-clinical drug discovery. However, it remains difficult to perform accurate single-cell assays in 384-well plates, limiting many studies to well-average measurements (e.g., CellTiter-Glo®). Here we describe a public domain Dye Drop method that uses sequential density displacement and microscopy to perform multi-step assays on living cells. We use Dye Drop cell viability and DNA replication assays followed by immunofluorescence imaging to collect single-cell dose-response data for 67 investigational and clinical-grade small molecules in 58 breast cancer cell lines. By separating the cytostatic and cytotoxic effects of drugs computationally, we uncover unexpected relationships between the two. Dye Drop is rapid, reproducible, customizable, and compatible with manual or automated laboratory equipment. Dye Drop improves the tradeoff between data content and cost, enabling the collection of information-rich perturbagen-response datasets.

Spatial intra-tumor heterogeneity is associated with survival of lung adenocarcinoma patients.

Intra-tumor heterogeneity (ITH) of human tumors is important for tumor progression, treatment response, and drug resistance. However, the spatial distribution of ITH remains incompletely understood. Here, we present spatial analysis of ITH in lung adenocarcinomas from 147 patients using multi-region mass spectrometry of >5,000 regions, single-cell copy number sequencing of ~2,000 single cells, and cyclic immunofluorescence of >10 million cells. We identified two distinct spatial patterns among tumors, termed clustered and random geographic diversification (GD). These patterns were observed in the same samples using both proteomic and genomic data. The random proteomic GD pattern, which is characterized by decreased cell adhesion and lower levels of tumor-interacting endothelial cells, was significantly associated with increased risk of recurrence or death in two independent patient cohorts. Our study presents comprehensive spatial mapping of ITH in lung adenocarcinoma and provides insights into the mechanisms and clinical consequences of GD.

A multi-omic analysis of MCF10A cells provides a resource for integrative assessment of ligand-mediated molecular and phenotypic responses.

The phenotype of a cell and its underlying molecular state is strongly influenced by extracellular signals, including growth factors, hormones, and extracellular matrix proteins. While these signals are normally tightly controlled, their dysregulation leads to phenotypic and molecular states associated with diverse diseases. To develop a detailed understanding of the linkage between molecular and phenotypic changes, we generated a comprehensive dataset that catalogs the transcriptional, proteomic, epigenomic and phenotypic responses of MCF10A mammary epithelial cells after exposure to the ligands EGF, HGF, OSM, IFNG, TGFB and BMP2. Systematic assessment of the molecular and cellular phenotypes induced by these ligands comprise the LINCS Microenvironment (ME) perturbation dataset, which has been curated and made publicly available for community-wide analysis and development of novel computational methods ( synapse.org/LINCS_MCF10A ). In illustrative analyses, we demonstrate how this dataset can be used to discover functionally related molecular features linked to specific cellular phenotypes. Beyond these analyses, this dataset will serve as a resource for the broader scientific community to mine for biological insights, to compare signals carried across distinct molecular modalities, and to develop new computational methods for integrative data analysis.

Single cell spatial analysis reveals the topology of immunomodulatory purinergic signaling in glioblastoma.

How the glioma immune microenvironment fosters tumorigenesis remains incompletely defined. Here, we use single-cell RNA-sequencing and multiplexed tissue-imaging to characterize the composition, spatial organization, and clinical significance of extracellular purinergic signaling in glioma. We show that microglia are the predominant source of CD39, while tumor cells principally express CD73. In glioblastoma, CD73 is associated with EGFR amplification, astrocyte-like differentiation, and increased adenosine, and is linked to hypoxia. Glioblastomas enriched for CD73 exhibit inflammatory microenvironments, suggesting that purinergic signaling regulates immune adaptation. Spatially-resolved single-cell analyses demonstrate a strong spatial correlation between tumor-CD73 and microglial-CD39, with proximity associated with poor outcomes. Similar spatial organization is present in pediatric high-grade gliomas including H3K27M-mutant diffuse midline glioma. These data reveal that purinergic signaling in gliomas is shaped by genotype, lineage, and functional state, and that core enzymes expressed by tumor and myeloid cells are organized to promote adenosine-rich microenvironments potentially amenable to therapeutic targeting.

Pathogenic variants damage cell composition and single cell transcription in cardiomyopathies.

Pathogenic variants in genes that cause dilated cardiomyopathy (DCM) and arrhythmogenic cardiomyopathy (ACM) convey high risks for the development of heart failure through unknown mechanisms. Using single-nucleus RNA sequencing, we characterized the transcriptome of 880,000 nuclei from 18 control and 61 failing, nonischemic human hearts with pathogenic variants in DCM and ACM genes or idiopathic disease. We performed genotype-stratified analyses of the ventricular cell lineages and transcriptional states. The resultant DCM and ACM ventricular cell atlas demonstrated distinct right and left ventricular responses, highlighting genotype-associated pathways, intercellular interactions, and differential gene expression at single-cell resolution. Together, these data illuminate both shared and distinct cellular and molecular architectures of human heart failure and suggest candidate therapeutic targets.

Stitching and registering highly multiplexed whole-slide images of tissues and tumors using ASHLAR.

MOTIVATION: Stitching microscope images into a mosaic is an essential step in the analysis and visualization of large biological specimens, particularly human and animal tissues. Recent approaches to highly multiplexed imaging generate high-plex data from sequential rounds of lower-plex imaging. These multiplexed imaging methods promise to yield precise molecular single-cell data and information on cellular neighborhoods and tissue architecture. However, attaining mosaic images with single-cell accuracy requires robust image stitching and image registration capabilities that are not met by existing methods. RESULTS: We describe the development and testing of ASHLAR, a Python tool for coordinated stitching and registration of 103 or more individual multiplexed images to generate accurate whole-slide mosaics. ASHLAR reads image formats from most commercial microscopes and slide scanners, and we show that it performs better than existing open-source and commercial software. ASHLAR outputs standard OME-TIFF images that are ready for analysis by other open-source tools and recently developed image analysis pipelines. AVAILABILITY AND IMPLEMENTATION: ASHLAR is written in Python and is available under the MIT license at https://github.com/labsyspharm/ashlar. The newly published data underlying this article are available in Sage Synapse at https://dx.doi.org/10.7303/syn25826362; the availability of other previously published data re-analyzed in this article is described in Supplementary Table S4. An informational website with user guides and test data is available at https://labsyspharm.github.io/ashlar/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

The Spatial Landscape of Progression and Immunoediting in Primary Melanoma at Single-Cell Resolution.

Cutaneous melanoma is a highly immunogenic malignancy that is surgically curable at early stages but life-threatening when metastatic. Here we integrate high-plex imaging, 3D high-resolution microscopy, and spatially resolved microregion transcriptomics to study immune evasion and immunoediting in primary melanoma. We find that recurrent cellular neighborhoods involving tumor, immune, and stromal cells change significantly along a progression axis involving precursor states, melanoma in situ, and invasive tumor. Hallmarks of immunosuppression are already detectable in precursor regions. When tumors become locally invasive, a consolidated and spatially restricted suppressive environment forms along the tumor-stromal boundary. This environment is established by cytokine gradients that promote expression of MHC-II and IDO1, and by PD1-PDL1-mediated cell contacts involving macrophages, dendritic cells, and T cells. A few millimeters away, cytotoxic T cells synapse with melanoma cells in fields of tumor regression. Thus, invasion and immunoediting can coexist within a few millimeters of each other in a single specimen. SIGNIFICANCE: The reorganization of the tumor ecosystem in primary melanoma is an excellent setting in which to study immunoediting and immune evasion. Guided by classic histopathology, spatial profiling of proteins and mRNA reveals recurrent morphologic and molecular features of tumor evolution that involve localized paracrine cytokine signaling and direct cell-cell contact. This article is highlighted in the In This Issue feature, p. 1397.

MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging.

Highly multiplexed tissue imaging makes detailed molecular analysis of single cells possible in a preserved spatial context. However, reproducible analysis of large multichannel images poses a substantial computational challenge. Here, we describe a modular and open-source computational pipeline, MCMICRO, for performing the sequential steps needed to transform whole-slide images into single-cell data. We demonstrate the use of MCMICRO on tissue and tumor images acquired using multiple imaging platforms, thereby providing a solid foundation for the continued development of tissue imaging software.

Age-dependent regulation of SARS-CoV-2 cell entry genes and cell death programs correlates with COVID-19 severity.

Novel coronavirus disease 2019 (COVID-19) severity is highly variable, with pediatric patients typically experiencing less severe infection than adults and especially the elderly. The basis for this difference is unclear. We find that mRNA and protein expression of angiotensin-converting enzyme 2 (ACE2), the cell entry receptor for the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that causes COVID-19, increases with advancing age in distal lung epithelial cells. However, in humans, ACE2 expression exhibits high levels of intra- and interindividual heterogeneity. Further, cells infected with SARS-CoV-2 experience endoplasmic reticulum stress, triggering an unfolded protein response and caspase-mediated apoptosis, a natural host defense system that halts virion production. Apoptosis of infected cells can be selectively induced by treatment with apoptosis-modulating BH3 mimetic drugs. Notably, epithelial cells within young lungs and airways are more primed to undergo apoptosis than those in adults, which may naturally hinder virion production and support milder COVID-19 severity.

Age-dependent regulation of SARS-CoV-2 cell entry genes and cell death programs correlates with COVID-19 disease severity.

Angiotensin-converting enzyme 2 (ACE2) maintains cardiovascular and renal homeostasis but also serves as the entry receptor for the novel severe acute respiratory syndrome coronavirus (SARS-CoV-2), the causal agent of novel coronavirus disease 2019 (COVID-19). COVID-19 disease severity is typically lower in pediatric patients than adults (particularly the elderly), but higher rates of hospitalizations requiring intensive care are observed in infants than in older children - the reasons for these differences are unknown. ACE2 is expressed in several adult tissues and cells, including alveolar type 2 cells of the distal lung epithelium, but expression at other ages is largely unexplored. Here we show that ACE2 transcripts are expressed in the lung and trachea shortly after birth, downregulated during childhood, and again expressed at high levels in late adulthood. Notably, the repertoire of cells expressing ACE2 protein in the mouse lung and airways shifts during key phases of lung maturation. In particular, podoplanin-positive cells, which are likely alveolar type I cells responsible for gas exchange, express ACE2 only in advanced age. Similar patterns of expression were evident in analysis of human lung tissue from over 100 donors, along with extreme inter- and intra-individual heterogeneity in ACE2 protein expression in epithelial cells. Furthermore, we find that apoptosis, which is a natural host defense system against viral infection, is dynamically regulated during lung maturation, resulting in periods of heightened apoptotic priming and dependence on pro-survival BCL-2 family proteins including MCL-1. Infection of human lung cells with SARS-CoV-2 triggers an unfolded protein stress response and upregulation of the endogenous MCL-1 inhibitor Noxa; in young individuals, MCL-1 inhibition is sufficient to trigger apoptosis in lung epithelial cells and may thus limit virion production and inflammatory signaling. Overall, we identify strong and distinct correlates of COVID-19 disease severity across lifespan and advance our understanding of the regulation of ACE2 and cell death programs in the mammalian lung. Furthermore, our work provides the framework for translation of apoptosis modulating drugs as novel treatments for COVID-19.

Author Correction: Immunogenomic profiling determines responses to combined PARP and PD-1 inhibition in ovarian cancer.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.