DEPICTER: Deep representation clustering for histology annotation.

Automatic segmentation of histopathology whole-slide images (WSI) usually involves supervised training of deep learning models with pixel-level labels to classify each pixel of the WSI into tissue regions such as benign or cancerous. However, fully supervised segmentation requires large-scale data manually annotated by experts, which can be expensive and time-consuming to obtain. Non-fully supervised methods, ranging from semi-supervised to unsupervised, have been proposed to address this issue and have been successful in WSI segmentation tasks. But these methods have mainly been focused on technical advancements in algorithmic performance rather than on the development of practical tools that could be used by pathologists or researchers in real-world scenarios. In contrast, we present DEPICTER (Deep rEPresentatIon ClusTERing), an interactive segmentation tool for histopathology annotation that produces a patch-wise dense segmentation map at WSI level. The interactive nature of DEPICTER leverages self- and semi-supervised learning approaches to allow the user to participate in the segmentation producing reliable results while reducing the workload. DEPICTER consists of three steps: first, a pretrained model is used to compute embeddings from image patches. Next, the user selects a number of benign and cancerous patches from the multi-resolution image. Finally, guided by the deep representations, label propagation is achieved using our novel seeded iterative clustering method or by directly interacting with the embedding space via feature space gating. We report both real-time interaction results with three pathologists and evaluate the performance on three public cancer classification dataset benchmarks through simulations. The code and demos of DEPICTER are publicly available at https://github.com/eduardchelebian/depicter.

Harmonizing the Generation and Pre-publication Stewardship of FAIR Image data.

Together with the molecular knowledge of genes and proteins, biological images promise to significantly enhance the scientific understanding of complex cellular systems and to advance predictive and personalized therapeutic products for human health. For this potential to be realized, quality-assured image data must be shared among labs at a global scale to be compared, pooled, and reanalyzed, thus unleashing untold potential beyond the original purpose for which the data was generated. There are two broad sets of requirements to enable image data sharing in the life sciences. One set of requirements is articulated in the companion White Paper entitled "Enabling Global Image Data Sharing in the Life Sciences," which is published in parallel and addresses the need to build the cyberinfrastructure for sharing the digital array data (arXiv:2401.13023 [q-bio.OT], https://doi.org/10.48550/arXiv.2401.13023). In this White Paper, we detail a broad set of requirements, which involves collecting, managing, presenting, and propagating contextual information essential to assess the quality, understand the content, interpret the scientific implications, and reuse image data in the context of the experimental details. We start by providing an overview of the main lessons learned to date through international community activities, which have recently made considerable progress toward generating community standard practices for imaging Quality Control (QC) and metadata. We then provide a clear set of recommendations for amplifying this work. The driving goal is to address remaining challenges, and democratize access to common practices and tools for a spectrum of biomedical researchers, regardless of their expertise, access to resources, and geographical location.

PACMan: A software package for automated single-cell chlorophyll fluorometry.

Microalgae, small photosynthetic unicells, are of great interest to ecology, ecotoxicology and biotechnology and there is a growing need to investigate the ability of cells to photosynthesize under variable conditions. Current strategies involve hand-operated pulse-amplitude-modulated (PAM) chlorophyll fluorimeters, which can provide detailed insights into the photophysiology of entire populations- or individual cells of microalgae but are typically limited in their throughput. To increase the throughput of a commercially available MICROSCOPY-PAM system, we present the PAM Automation Control Manager ('PACMan'), an open-source Python software package that automates image acquisition, microscopy stage control and the triggering of external hardware components. PACMan comes with a user-friendly graphical user interface and is released together with a stand-alone tool (PAMalysis) for the automated calculation of per-cell maximum quantum efficiencies (= Fv /Fm ). Using these two software packages, we successfully tracked the photophysiology of >1000 individual cells of green algae (Chlamydomonas reinhardtii) and dinoflagellates (genus Symbiodiniaceae) within custom-made microfluidic devices. Compared to the manual operation of MICROSCOPY-PAM systems, this represents a 10-fold increase in throughput. During experiments, PACMan coordinated the movement of the microscope stage and triggered the MICROSCOPY-PAM system to repeatedly capture high-quality image data across multiple positions. Finally, we analyzed single-cell Fv /Fm with the manufacturer-supplied software and PAMalysis, demonstrating a median difference <0.5% between both methods. We foresee that PACMan, and its auxiliary software package will help increase the experimental throughput in a range of microalgae studies currently relying on hand-operated MICROSCOPY-PAM technologies.

Label-free deep learning-based species classification of bacteria imaged by phase-contrast microscopy.

Reliable detection and classification of bacteria and other pathogens in the human body, animals, food, and water is crucial for improving and safeguarding public health. For instance, identifying the species and its antibiotic susceptibility is vital for effective bacterial infection treatment. Here we show that phase contrast time-lapse microscopy combined with deep learning is sufficient to classify four species of bacteria relevant to human health. The classification is performed on living bacteria and does not require fixation or staining, meaning that the bacterial species can be determined as the bacteria reproduce in a microfluidic device, enabling parallel determination of susceptibility to antibiotics. We assess the performance of convolutional neural networks and vision transformers, where the best model attained a class-average accuracy exceeding 98%. Our successful proof-of-principle results suggest that the methods should be challenged with data covering more species and clinically relevant isolates for future clinical use.

Evaluating the utility of brightfield image data for mechanism of action prediction.

Fluorescence staining techniques, such as Cell Painting, together with fluorescence microscopy have proven invaluable for visualizing and quantifying the effects that drugs and other perturbations have on cultured cells. However, fluorescence microscopy is expensive, time-consuming, labor-intensive, and the stains applied can be cytotoxic, interfering with the activity under study. The simplest form of microscopy, brightfield microscopy, lacks these downsides, but the images produced have low contrast and the cellular compartments are difficult to discern. Nevertheless, by harnessing deep learning, these brightfield images may still be sufficient for various predictive purposes. In this study, we compared the predictive performance of models trained on fluorescence images to those trained on brightfield images for predicting the mechanism of action (MoA) of different drugs. We also extracted CellProfiler features from the fluorescence images and used them to benchmark the performance. Overall, we found comparable and largely correlated predictive performance for the two imaging modalities. This is promising for future studies of MoAs in time-lapse experiments for which using fluorescence images is problematic. Explorations based on explainable AI techniques also provided valuable insights regarding compounds that were better predicted by one modality over the other.

TissUUmaps 3: Improvements in interactive visualization, exploration, and quality assessment of large-scale spatial omics data.

Background and objectives: Spatially resolved techniques for exploring the molecular landscape of tissue samples, such as spatial transcriptomics, often result in millions of data points and images too large to view on a regular desktop computer, limiting the possibilities in visual interactive data exploration. TissUUmaps is a free, open-source browser-based tool for GPU-accelerated visualization and interactive exploration of 107+ data points overlaying tissue samples. Methods: Herein we describe how TissUUmaps 3 provides instant multiresolution image viewing and can be customized, shared, and also integrated into Jupyter Notebooks. We introduce new modules where users can visualize markers and regions, explore spatial statistics, perform quantitative analyses of tissue morphology, and assess the quality of decoding in situ transcriptomics data. Results: We show that thanks to targeted optimizations the time and cost associated with interactive data exploration were reduced, enabling TissUUmaps 3 to handle the scale of today's spatial transcriptomics methods. Conclusion: TissUUmaps 3 provides significantly improved performance for large multiplex datasets as compared to previous versions. We envision TissUUmaps to contribute to broader dissemination and flexible sharing of largescale spatial omics data.

Roadmap on Deep Learning for Microscopy.

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

A topographic atlas defines developmental origins of cell heterogeneity in the human embryonic lung.

The lung contains numerous specialized cell types with distinct roles in tissue function and integrity. To clarify the origins and mechanisms generating cell heterogeneity, we created a comprehensive topographic atlas of early human lung development. Here we report 83 cell states and several spatially resolved developmental trajectories and predict cell interactions within defined tissue niches. We integrated single-cell RNA sequencing and spatially resolved transcriptomics into a web-based, open platform for interactive exploration. We show distinct gene expression programmes, accompanying sequential events of cell differentiation and maturation of the secretory and neuroendocrine cell types in proximal epithelium. We define the origin of airway fibroblasts associated with airway smooth muscle in bronchovascular bundles and describe a trajectory of Schwann cell progenitors to intrinsic parasympathetic neurons controlling bronchoconstriction. Our atlas provides a rich resource for further research and a reference for defining deviations from homeostatic and repair mechanisms leading to pulmonary diseases.

Visualization and quality control tools for large-scale multiplex tissue analysis in TissUUmaps3.

Large-scale multiplex tissue analysis aims to understand processes such as development and tumor formation by studying the occurrence and interaction of cells in local environments in, for example, tissue samples from patient cohorts. A typical procedure in the analysis is to delineate individual cells, classify them into cell types, and analyze their spatial relationships. All steps come with a number of challenges, and to address them and identify the bottlenecks of the analysis, it is necessary to include quality control tools in the analysis workflow. This makes it possible to optimize the steps and adjust settings in order to get better and more precise results. Additionally, the development of automated approaches for tissue analysis requires visual verification to reduce skepticism with regard to the accuracy of the results. Quality control tools could be used to build users' trust in automated approaches. In this paper, we present three plugins for visualization and quality control in large-scale multiplex tissue analysis of microscopy images. The first plugin focuses on the quality of cell staining, the second one was made for interactive evaluation and comparison of different cell classification results, and the third one serves for reviewing interactions of different cell types.

De novo spatiotemporal modelling of cell-type signatures in the developmental human heart using graph convolutional neural networks.

With the emergence of high throughput single cell techniques, the understanding of the molecular and cellular diversity of mammalian organs have rapidly increased. In order to understand the spatial organization of this diversity, single cell data is often integrated with spatial data to create probabilistic cell maps. However, targeted cell typing approaches relying on existing single cell data achieve incomplete and biased maps that could mask the true diversity present in a tissue slide. Here we applied a de novo technique to spatially resolve and characterize cellular diversity of in situ sequencing data during human heart development. We obtained and made accessible well defined spatial cell-type maps of fetal hearts from 4.5 to 9 post conception weeks, not biased by probabilistic cell typing approaches. With our analysis, we could characterize previously unreported molecular diversity within cardiomyocytes and epicardial cells and identified their characteristic expression signatures, comparing them with specific subpopulations found in single cell RNA sequencing datasets. We further characterized the differentiation trajectories of epicardial cells, identifying a clear spatial component on it. All in all, our study provides a novel technique for conducting de novo spatial-temporal analyses in developmental tissue samples and a useful resource for online exploration of cell-type differentiation during heart development at sub-cellular image resolution.

Automated detection of vascular remodeling in tumor-draining lymph nodes by the deep-learning tool HEV-finder.

Vascular remodeling is common in human cancer and has potential as future biomarkers for prediction of disease progression and tumor immunity status. It can also affect metastatic sites, including the tumor-draining lymph nodes (TDLNs). Dilation of the high endothelial venules (HEVs) within TDLNs has been observed in several types of cancer. We recently demonstrated that it is a premetastatic effect that can be linked to tumor invasiveness in breast cancer. Manual visual assessment of changes in vascular morphology is a tedious and difficult task, limiting high-throughput analysis. Here we present a fully automated approach for detection and classification of HEV dilation. By using 12,524 manually classified HEVs, we trained a deep-learning model and created a graphical user interface for visualization of the results. The tool, named the HEV-finder, selectively analyses HEV dilation in specific regions of the lymph nodes. We evaluated the HEV-finder's ability to detect and classify HEV dilation in different types of breast cancer compared to manual annotations. Our results constitute a successful example of large-scale, fully automated, and user-independent, image-based quantitative assessment of vascular remodeling in human pathology and lay the ground for future exploration of HEV dilation in TDLNs as a biomarker. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

SimSearch: A Human-in-The-Loop Learning Framework for Fast Detection of Regions of Interest in Microscopy Images.

OBJECTIVE: Large-scale microscopy-based experiments often result in images with rich but sparse information content. An experienced microscopist can visually identify regions of interest (ROIs), but this becomes a cumbersome task with large datasets. Here we present SimSearch, a framework for quick and easy user-guided training of a deep neural model aimed at fast detection of ROIs in large-scale microscopy experiments. METHODS: The user manually selects a small number of patches representing different classes of ROIs. This is followed by feature extraction using a pre-trained deep-learning model, and interactive patch selection pruning, resulting in a smaller set of clean (user approved) and larger set of noisy (unapproved) training patches of ROIs and background. The pre-trained deep-learning model is thereafter first trained on the large set of noisy patches, followed by refined training using the clean patches. RESULTS: The framework is evaluated on fluorescence microscopy images from a large-scale drug screening experiment, brightfield images of immunohistochemistry-stained patient tissue samples, and malaria-infected human blood smears, as well as transmission electron microscopy images of cell sections. Compared to state-of-the-art and manual/visual assessment, the results show similar performance with maximal flexibility and minimal a priori information and user interaction. CONCLUSIONS: SimSearch quickly adapts to different data sets, which demonstrates the potential to speed up many microscopy-based experiments based on a small amount of user interaction. SIGNIFICANCE: SimSearch can help biologists quickly extract informative regions and perform analyses on large datasets helping increase the throughput in a microscopy experiment.

Spatial Statistics for Understanding Tissue Organization.

Interpreting tissue architecture plays an important role in gaining a better understanding of healthy tissue development and disease. Novel molecular detection and imaging techniques make it possible to locate many different types of objects, such as cells and/or mRNAs, and map their location across the tissue space. In this review, we present several methods that provide quantification and statistical verification of observed patterns in the tissue architecture. We categorize these methods into three main groups: Spatial statistics on a single type of object, two types of objects, and multiple types of objects. We discuss the methods in relation to four hypotheses regarding the methods' capability to distinguish random and non-random distributions of objects across a tissue sample, and present a number of openly available tools where these methods are provided. We also discuss other spatial statistics methods compatible with other types of input data.

Regular Use of Depot Medroxyprogesterone Acetate Causes Thinning of the Superficial Lining and Apical Distribution of Human Immunodeficiency Virus Target Cells in the Human Ectocervix.

BACKGROUND: The hormonal contraceptive depot medroxyprogesterone acetate (DMPA) may be associated with an increased risk of acquiring human immunodeficiency virus (HIV). We hypothesize that DMPA use influences the ectocervical tissue architecture and HIV target cell localization. METHODS: Quantitative image analysis workflows were developed to assess ectocervical tissue samples collected from DMPA users and control subjects not using hormonal contraception. RESULTS: Compared to controls, the DMPA group exhibited a significantly thinner apical ectocervical epithelial layer and a higher proportion of CD4+CCR5+ cells with a more superficial location. This localization corresponded to an area with a nonintact E-cadherin net structure. CD4+Langerin+ cells were also more superficially located in the DMPA group, although fewer in number compared to the controls. Natural plasma progesterone levels did not correlate with any of these parameters, whereas estradiol levels were positively correlated with E-cadherin expression and a more basal location for HIV target cells of the control group. CONCLUSIONS: DMPA users have a less robust epithelial layer and a more apical distribution of HIV target cells in the human ectocervix, which could confer a higher risk of HIV infection. Our results highlight the importance of assessing intact genital tissue samples to gain insights into HIV susceptibility factors.

Morphological Features Extracted by AI Associated with Spatial Transcriptomics in Prostate Cancer.

Prostate cancer is a common cancer type in men, yet some of its traits are still under-explored. One reason for this is high molecular and morphological heterogeneity. The purpose of this study was to develop a method to gain new insights into the connection between morphological changes and underlying molecular patterns. We used artificial intelligence (AI) to analyze the morphology of seven hematoxylin and eosin (H&E)-stained prostatectomy slides from a patient with multi-focal prostate cancer. We also paired the slides with spatially resolved expression for thousands of genes obtained by a novel spatial transcriptomics (ST) technique. As both spaces are highly dimensional, we focused on dimensionality reduction before seeking associations between them. Consequently, we extracted morphological features from H&E images using an ensemble of pre-trained convolutional neural networks and proposed a workflow for dimensionality reduction. To summarize the ST data into genetic profiles, we used a previously proposed factor analysis. We found that the regions were automatically defined, outlined by unsupervised clustering, associated with independent manual annotations, in some cases, finding further relevant subdivisions. The morphological patterns were also correlated with molecular profiles and could predict the spatial variation of individual genes. This novel approach enables flexible unsupervised studies relating morphological and genetic heterogeneity using AI to be carried out.

Genes in human obesity loci are causal obesity genes in C. elegans.

Obesity and its associated metabolic syndrome are a leading cause of morbidity and mortality. Given the disease's heavy burden on patients and the healthcare system, there has been increased interest in identifying pharmacological targets for the treatment and prevention of obesity. Towards this end, genome-wide association studies (GWAS) have identified hundreds of human genetic variants associated with obesity. The next challenge is to experimentally define which of these variants are causally linked to obesity, and could therefore become targets for the treatment or prevention of obesity. Here we employ high-throughput in vivo RNAi screening to test for causality 293 C. elegans orthologs of human obesity-candidate genes reported in GWAS. We RNAi screened these 293 genes in C. elegans subject to two different feeding regimens: (1) regular diet, and (2) high-fructose diet, which we developed and present here as an invertebrate model of diet-induced obesity (DIO). We report 14 genes that promote obesity and 3 genes that prevent DIO when silenced in C. elegans. Further, we show that knock-down of the 3 DIO genes not only prevents excessive fat accumulation in primary and ectopic fat depots but also improves the health and extends the lifespan of C. elegans overconsuming fructose. Importantly, the direction of the association between expression variants in these loci and obesity in mice and humans matches the phenotypic outcome of the loss-of-function of the C. elegans ortholog genes, supporting the notion that some of these genes would be causally linked to obesity across phylogeny. Therefore, in addition to defining causality for several genes so far merely correlated with obesity, this study demonstrates the value of model systems compatible with in vivo high-throughput genetic screening to causally link GWAS gene candidates to human diseases.

Artificial Intelligence for Diagnosis and Gleason Grading of Prostate Cancer in Biopsies-Current Status and Next Steps.

Diagnosis and Gleason grading of prostate cancer in biopsies are critical for the clinical management of men with prostate cancer. Despite this, the high grading variability among pathologists leads to the potential for under- and overtreatment. Artificial intelligence (AI) systems have shown promise in assisting pathologists to perform Gleason grading, which could help address this problem. In this mini-review, we highlight studies reporting on the development of AI systems for cancer detection and Gleason grading, and discuss the progress needed for widespread clinical implementation, as well as anticipated future developments. PATIENT SUMMARY: This mini-review summarizes the evidence relating to the validation of artificial intelligence (AI)-assisted cancer detection and Gleason grading of prostate cancer in biopsies, and highlights the remaining steps required prior to its widespread clinical implementation. We found that, although there is strong evidence to show that AI is able to perform Gleason grading on par with experienced uropathologists, more work is needed to ensure the accuracy of results from AI systems in diverse settings across different patient populations, digitization platforms, and pathology laboratories.

Machine learning for cell classification and neighborhood analysis in glioma tissue.

Multiplexed and spatially resolved single-cell analyses that intend to study tissue heterogeneity and cell organization invariably face as a first step the challenge of cell classification. Accuracy and reproducibility are important for the downstream process of counting cells, quantifying cell-cell interactions, and extracting information on disease-specific localized cell niches. Novel staining techniques make it possible to visualize and quantify large numbers of cell-specific molecular markers in parallel. However, due to variations in sample handling and artifacts from staining and scanning, cells of the same type may present different marker profiles both within and across samples. We address multiplexed immunofluorescence data from tissue microarrays of low-grade gliomas and present a methodology using two different machine learning architectures and features insensitive to illumination to perform cell classification. The fully automated cell classification provides a measure of confidence for the decision and requires a comparably small annotated data set for training, which can be created using freely available tools. Using the proposed method, we reached an accuracy of 83.1% on cell classification without the need for standardization of samples. Using our confidence measure, cells with low-confidence classifications could be excluded, pushing the classification accuracy to 94.5%. Next, we used the cell classification results to search for cell niches with an unsupervised learning approach based on graph neural networks. We show that the approach can re-detect specialized tissue niches in previously published data, and that our proposed cell classification leads to niche definitions that may be relevant for sub-groups of glioma, if applied to larger data sets.

Deep-learning models for lipid nanoparticle-based drug delivery.

Background: Early prediction of time-lapse microscopy experiments enables intelligent data management and decision-making. Aim: Using time-lapse data of HepG2 cells exposed to lipid nanoparticles loaded with mRNA for expression of GFP, the authors hypothesized that it is possible to predict in advance whether a cell will express GFP. Methods: The first modeling approach used a convolutional neural network extracting per-cell features at early time points. These features were then combined and explored using either a long short-term memory network (approach 2) or time series feature extraction and gradient boosting machines (approach 3). Results: Accounting for the temporal dynamics significantly improved performance. Conclusion: The results highlight the benefit of accounting for temporal dynamics when studying drug delivery using high-content imaging.

ImageJ and CellProfiler: Complements in Open-Source Bioimage Analysis.

ImageJ and CellProfiler have long been leading open-source platforms in the field of bioimage analysis. ImageJ's traditional strength is in single-image processing and investigation, while CellProfiler is designed for building large-scale, modular analysis pipelines. Although many image analysis problems can be well solved with one or the other, using these two platforms together in a single workflow can be powerful. Here, we share two pipelines demonstrating mechanisms for productively and conveniently integrating ImageJ and CellProfiler for (1) studying cell morphology and migration via tracking, and (2) advanced stitching techniques for handling large, tiled image sets to improve segmentation. No single platform can provide all the key and most efficient functionality needed for all studies. While both programs can be and are often used separately, these pipelines demonstrate the benefits of using them together for image analysis workflows. ImageJ and CellProfiler are both committed to interoperability between their platforms, with ongoing development to improve how both are leveraged from the other. © 2021 Wiley Periodicals LLC. Basic Protocol 1: Studying cell morphology and cell migration in time-lapse datasets using TrackMate (Fiji) and CellProfiler Basic Protocol 2: Creating whole plate montages to easily assess adaptability of segmentation parameters.