BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets.

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. We observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

Automated detection of apoptotic bodies and cells in label-free time-lapse high-throughput video microscopy using deep convolutional neural networks.

MOTIVATION: Reliable label-free methods are needed for detecting and profiling apoptotic events in time-lapse cell-cell interaction assays. Prior studies relied on fluorescent markers of apoptosis, e.g. Annexin-V, that provide an inconsistent and late indication of apoptotic onset for human melanoma cells. Our motivation is to improve the detection of apoptosis by directly detecting apoptotic bodies in a label-free manner. RESULTS: Our trained ResNet50 network identified nanowells containing apoptotic bodies with 92% accuracy and predicted the onset of apoptosis with an error of one frame (5 min/frame). Our apoptotic body segmentation yielded an IoU accuracy of 75%, allowing associative identification of apoptotic cells. Our method detected apoptosis events, 70% of which were not detected by Annexin-V staining. AVAILABILITY AND IMPLEMENTATION: Open-source code and sample data provided at https://github.com/kwu14victor/ApoBDproject.

Salicylic acid inducible nucleocytoplasmic shuttling of NPR1 fusion proteins in human cells.

Ligand inducible proteins that enable precise and reversible control of nuclear translocation of passenger proteins have broad applications ranging from genetic studies in mammals to therapeutics that target diseases such as cancer and diabetes. One of the drawbacks of the current translocation systems is that the ligands used to control nuclear localization are either toxic or prone to crosstalk with endogenous protein cascades within live animals. We sought to take advantage of salicylic acid (SA), a small molecule that has been extensively used in humans. In plants, SA functions as a hormone that can mediate immunity and is sensed by the nonexpressor of pathogenesis-related (NPR) proteins. Although it is well recognized that nuclear translocation of NPR1 is essential to promoting immunity in plants, the exact subdomain of Arabidopsis thaliana NPR1 (AtNPR1) essential for SA-mediated nuclear translocation is controversial. Here, we utilized the fluorescent protein mCherry as the reporter to investigate the ability of SA to induce nuclear translocation of the full-length NPR1 protein or its C-terminal transactivation (TAD) domain using HEK293 cells as a heterologous system. HEK293 cells lack accessory plant proteins including NPR3/NPR4 and are thus ideally suited for studying the impact of SA-induced changes in NPR1. Our results obtained using a stable expression system show that the TAD of AtNPR1 is sufficient to enable the reversible SA-mediated nuclear translocation of mCherry. Our studies advance a basic understanding of nuclear translocation mediated by the TAD of AtNPR1 and uncover a biotechnological tool for SA-mediated nuclear localization.

Artificial intelligence and machine learning in nephropathology.

Artificial intelligence (AI) for the purpose of this review is an umbrella term for technologies emulating a nephropathologist's ability to extract information on diagnosis, prognosis, and therapy responsiveness from native or transplant kidney biopsies. Although AI can be used to analyze a wide variety of biopsy-related data, this review focuses on whole slide images traditionally used in nephropathology. AI applications in nephropathology have recently become available through several advancing technologies, including (i) widespread introduction of glass slide scanners, (ii) data servers in pathology departments worldwide, and (iii) through greatly improved computer hardware to enable AI training. In this review, we explain how AI can enhance the reproducibility of nephropathology results for certain parameters in the context of precision medicine using advanced architectures, such as convolutional neural networks, that are currently the state of the art in machine learning software for this task. Because AI applications in nephropathology are still in their infancy, we show the power and potential of AI applications mostly in the example of oncopathology. Moreover, we discuss the technological obstacles as well as the current stakeholder and regulatory concerns about developing AI applications in nephropathology from the perspective of nephropathologists and the wider nephrology community. We expect the gradual introduction of these technologies into routine diagnostics and research for selective tasks, suggesting that this technology will enhance the performance of nephropathologists rather than making them redundant.

TIMING 2.0: high-throughput single-cell profiling of dynamic cell-cell interactions by time-lapse imaging microscopy in nanowell grids.

MOTIVATION: Automated profiling of cell-cell interactions from high-throughput time-lapse imaging microscopy data of cells in nanowell grids (TIMING) has led to fundamental insights into cell-cell interactions in immunotherapy. This application note aims to enable widespread adoption of TIMING by (i) enabling the computations to occur on a desktop computer with a graphical processing unit instead of a server; (ii) enabling image acquisition and analysis to occur in the laboratory avoiding network data transfers to/from a server and (iii) providing a comprehensive graphical user interface. RESULTS: On a desktop computer, TIMING 2.0 takes 5 s/block/image frame, four times faster than our previous method on the same computer, and twice as fast as our previous method (TIMING) running on a Dell PowerEdge server. The cell segmentation accuracy (f-number = 0.993) is superior to our previous method (f-number = 0.821). A graphical user interface provides the ability to inspect the video analysis results, make corrective edits efficiently (one-click editing of an entire nanowell video sequence in 5-10 s) and display a summary of the cell killing efficacy measurements. AVAILABILITY AND IMPLEMENTATION: Open source Python software (GPL v3 license), instruction manual, sample data and sample results are included with the Supplement (https://github.com/RoysamLab/TIMING2). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Morphologically constrained spectral unmixing by dictionary learning for multiplex fluorescence microscopy.

MOTIVATION: Current spectral unmixing methods for multiplex fluorescence microscopy have a limited ability to cope with high spectral overlap as they only analyze spectral information over individual pixels. Here, we present adaptive Morphologically Constrained Spectral Unmixing (MCSU) algorithms that overcome this limitation by exploiting morphological differences between sub-cellular structures, and their local spatial context. RESULTS: The proposed method was effective at improving spectral unmixing performance by exploiting: (i) a set of dictionary-based models for object morphologies learned from the image data; and (ii) models of spatial context learned from the image data using a total variation algorithm. The method was evaluated on multi-spectral images of multiplex-labeled pancreatic ductal adenocarcinoma (PDAC) tissue samples. The former constraint ensures that neighbouring pixels correspond to morphologically similar structures, and the latter constraint ensures that neighbouring pixels have similar spectral signatures. The average Mean Squared Error (MSE) and Signal Reconstruction Error (SRE) ratio of the proposed method was 39.6% less and 8% more, respectively, compared to the best of all other algorithms that do not exploit these spatial constraints. AVAILABILITY AND IMPLEMENTATION: Open source software (MATLAB). CONTACT: broysam@central.uh.edu. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Automated profiling of individual cell-cell interactions from high-throughput time-lapse imaging microscopy in nanowell grids (TIMING).

MOTIVATION: There is a need for effective automated methods for profiling dynamic cell-cell interactions with single-cell resolution from high-throughput time-lapse imaging data, especially, the interactions between immune effector cells and tumor cells in adoptive immunotherapy. RESULTS: Fluorescently labeled human T cells, natural killer cells (NK), and various target cells (NALM6, K562, EL4) were co-incubated on polydimethylsiloxane arrays of sub-nanoliter wells (nanowells), and imaged using multi-channel time-lapse microscopy. The proposed cell segmentation and tracking algorithms account for cell variability and exploit the nanowell confinement property to increase the yield of correctly analyzed nanowells from 45% (existing algorithms) to 98% for wells containing one effector and a single target, enabling automated quantification of cell locations, morphologies, movements, interactions, and deaths without the need for manual proofreading. Automated analysis of recordings from 12 different experiments demonstrated automated nanowell delineation accuracy >99%, automated cell segmentation accuracy >95%, and automated cell tracking accuracy of 90%, with default parameters, despite variations in illumination, staining, imaging noise, cell morphology, and cell clustering. An example analysis revealed that NK cells efficiently discriminate between live and dead targets by altering the duration of conjugation. The data also demonstrated that cytotoxic cells display higher motility than non-killers, both before and during contact. CONTACT: broysam@central.uh.edu or nvaradar@central.uh.edu SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Population-scale three-dimensional reconstruction and quantitative profiling of microglia arbors.

MOTIVATION: The arbor morphologies of brain microglia are important indicators of cell activation. This article fills the need for accurate, robust, adaptive and scalable methods for reconstructing 3-D microglial arbors and quantitatively mapping microglia activation states over extended brain tissue regions. RESULTS: Thick rat brain sections (100-300 µm) were multiplex immunolabeled for IBA1 and Hoechst, and imaged by step-and-image confocal microscopy with automated 3-D image mosaicing, producing seamless images of extended brain regions (e.g. 5903 × 9874 × 229 voxels). An over-complete dictionary-based model was learned for the image-specific local structure of microglial processes. The microglial arbors were reconstructed seamlessly using an automated and scalable algorithm that exploits microglia-specific constraints. This method detected 80.1 and 92.8% more centered arbor points, and 53.5 and 55.5% fewer spurious points than existing vesselness and LoG-based methods, respectively, and the traces were 13.1 and 15.5% more accurate based on the DIADEM metric. The arbor morphologies were quantified using Scorcioni's L-measure. Coifman's harmonic co-clustering revealed four morphologically distinct classes that concord with known microglia activation patterns. This enabled us to map spatial distributions of microglial activation and cell abundances. AVAILABILITY AND IMPLEMENTATION: Experimental protocols, sample datasets, scalable open-source multi-threaded software implementation (C++, MATLAB) in the electronic supplement, and website (www.farsight-toolkit.org). http://www.farsight-toolkit.org/wiki/Population-scale_Three-dimensional_Reconstruction_and_Quanti-tative_Profiling_of_Microglia_Arbors CONTACT: broysam@central.uh.edu SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Antibody Fc engineering improves frequency and promotes kinetic boosting of serial killing mediated by NK cells.

The efficacy of most therapeutic monoclonal antibodies (mAbs) targeting tumor antigens results primarily from their ability to elicit potent cytotoxicity through effector-mediated functions. We have engineered the fragment crystallizable (Fc) region of the immunoglobulin G (IgG) mAb, HuM195, targeting the leukemic antigen CD33, by introducing the triple mutation Ser293Asp/Ala330Leu/Ile332Glu (DLE), and developed Time-lapse Imaging Microscopy in Nanowell Grids to analyze antibody-dependent cell-mediated cytotoxicity kinetics of thousands of individual natural killer (NK) cells and mAb-coated target cells. We demonstrate that the DLE-HuM195 antibody increases both the quality and the quantity of NK cell-mediated antibody-dependent cytotoxicity by endowing more NK cells to participate in cytotoxicity via accrued CD16-mediated signaling and by increasing serial killing of target cells. NK cells encountering targets coated with DLE-HuM195 induce rapid target cell apoptosis by promoting simultaneous conjugates to multiple target cells and induce apoptosis in twice the number of target cells within the same period as the wild-type mAb. Enhanced target killing was also associated with increased frequency of NK cells undergoing apoptosis, but this effect was donor-dependent. Antibody-based therapies targeting tumor antigens will benefit from a better understanding of cell-mediated tumor elimination, and our work opens further opportunities for the therapeutic targeting of CD33 in the treatment of acute myeloid leukemia.

Biological imaging software tools.

Few technologies are more widespread in modern biological laboratories than imaging. Recent advances in optical technologies and instrumentation are providing hitherto unimagined capabilities. Almost all these advances have required the development of software to enable the acquisition, management, analysis and visualization of the imaging data. We review each computational step that biologists encounter when dealing with digital images, the inherent challenges and the overall status of available software for bioimage informatics, focusing on open-source options.

Preexisting epithelial diversity in normal human livers: a tissue-tethered cytometric analysis in portal/periportal epithelial cells.

UNLABELLED: Routine light microscopy identifies two distinct epithelial cell populations in normal human livers: hepatocytes and biliary epithelial cells (BECs). Considerable epithelial diversity, however, arises during disease states when a variety of hepatocyte-BEC hybrid cells appear. This has been attributed to activation and differentiation of putative hepatic progenitor cells (HPC) residing in the canals of Hering and/or metaplasia of preexisting mature epithelial cells. A novel analytic approach consisting of multiplex labeling, high-resolution whole-slide imaging (WSI), and automated image analysis was used to determine if more complex epithelial cell phenotypes preexist in normal adult human livers, which might provide an alternative explanation for disease-induced epithelial diversity. "Virtually digested" WSI enabled quantitative cytometric analyses of individual cells displayed in a variety of formats (e.g., scatterplots) while still tethered to the WSI and tissue structure. We employed biomarkers specifically associated with mature epithelial forms (HNF4α for hepatocytes, CK19 and HNF1ß for BEC) and explored for the presence of cells with hybrid biomarker phenotypes. The results showed abundant hybrid cells in portal bile duct BEC, canals of Hering, and immediate periportal hepatocytes. These bipotential cells likely serve as a reservoir for the epithelial diversity of ductular reactions, appearance of hepatocytes in bile ducts, and the rapid and fluid transition of BEC to hepatocytes, and vice versa. CONCLUSION: Novel imaging and computational tools enable increased information extraction from tissue samples and quantify the considerable preexistent hybrid epithelial diversity in normal human liver. This computationally enabled tissue analysis approach offers much broader potential beyond the results presented here.

Visual Prompting Based Incremental Learning for Semantic Segmentation of Multiplex Immuno-Flourescence Microscopy Imagery.

Deep learning approaches are state-of-the-art for semantic segmentation of medical images, but unlike many deep learning applications, medical segmentation is characterized by small amounts of annotated training data. Thus, while mainstream deep learning approaches focus on performance in domains with large training sets, researchers in the medical imaging field must apply new methods in creative ways to meet the more constrained requirements of medical datasets. We propose a framework for incrementally fine-tuning a multi-class segmentation of a high-resolution multiplex (multi-channel) immuno-flourescence image of a rat brain section, using a minimal amount of labelling from a human expert. Our framework begins with a modified Swin-UNet architecture that treats each biomarker in the multiplex image separately and learns an initial "global" segmentation (pre-training). This is followed by incremental learning and refinement of each class using a very limited amount of additional labeled data provided by a human expert for each region and its surroundings. This incremental learning utilizes the multi-class weights as an initialization and uses the additional labels to steer the network and optimize it for each region in the image. In this way, an expert can identify errors in the multi-class segmentation and rapidly correct them by supplying the model with additional annotations hand-picked from the region. In addition to increasing the speed of annotation and reducing the amount of labelling, we show that our proposed method outperforms a traditional multi-class segmentation by a large margin.

Cellular data extraction from multiplexed brain imaging data using self-supervised Dual-loss Adaptive Masked Autoencoder.

Reliable large-scale cell detection and segmentation is the fundamental first step to understanding biological processes in the brain. The ability to phenotype cells at scale can accelerate preclinical drug evaluation and system-level brain histology studies. The impressive advances in deep learning offer a practical solution to cell image detection and segmentation. Unfortunately, categorizing cells and delineating their boundaries for training deep networks is an expensive process that requires skilled biologists. This paper presents a novel self-supervised Dual-Loss Adaptive Masked Autoencoder (DAMA) for learning rich features from multiplexed immunofluorescence brain images. DAMA's objective function minimizes the conditional entropy in pixel-level reconstruction and feature-level regression. Unlike existing self-supervised learning methods based on a random image masking strategy, DAMA employs a novel adaptive mask sampling strategy to maximize mutual information and effectively learn brain cell data. To the best of our knowledge, this is the first effort to develop a self-supervised learning method for multiplexed immunofluorescence brain images. Our extensive experiments demonstrate that DAMA features enable superior cell detection, segmentation, and classification performance without requiring many annotations. In addition, to examine the generalizability of DAMA, we also experimented on TissueNet, a multiplexed imaging dataset comprised of two-channel fluorescence images from six distinct tissue types, captured using six different imaging platforms. Our code is publicly available at https://github.com/hula-ai/DAMA.

Automated image computing reshapes computational neuroscience.

We briefly identify several critical issues in current computational neuroscience, and present our opinions on potential solutions based on bioimage informatics, especially automated image computing.

Computational prediction of neural progenitor cell fates.

Understanding how stem and progenitor cells choose between alternative cell fates is a major challenge in developmental biology. Efforts to tackle this problem have been hampered by the scarcity of markers that can be used to predict cell division outcomes. Here we present a computational method, based on algorithmic information theory, to analyze dynamic features of living cells over time. Using this method, we asked whether rat retinal progenitor cells (RPCs) display characteristic phenotypes before undergoing mitosis that could foretell their fate. We predicted whether RPCs will undergo a self-renewing or terminal division with 99% accuracy, or whether they will produce two photoreceptors or another combination of offspring with 87% accuracy. Our implementation can segment, track and generate predictions for 40 cells simultaneously on a standard computer at 5 min per frame. This method could be used to isolate cell populations with specific developmental potential, enabling previously impossible investigations.

Visual Prompting based Incremental Learning for Semantic Segmentation of Multiplex Immuno-Flourescence Microscopy Imagery.

Deep learning approaches are state-of-the-art for semantic segmentation of medical images, but unlike many deep learning applications, medical segmentation is characterized by small amounts of annotated training data. Thus, while mainstream deep learning approaches focus on performance in domains with large training sets, researchers in the medical imaging field must apply new methods in creative ways to meet the more constrained requirements of medical datasets. We propose a framework for incrementally fine-tuning a multi-class segmentation of a high-resolution multiplex (multi-channel) immuno-flourescence image of a rat brain section, using a minimal amount of labelling from a human expert. Our framework begins with a modified Swin-UNet architecture that treats each biomarker in the multiplex image separately and learns an initial "global" segmentation (pre-training). This is followed by incremental learning and refinement of each class using a very limited amount of additional labeled data provided by a human expert for each region and its surroundings. This incremental learning utilizes the multi-class weights as an initialization and uses the additional labels to steer the network and optimize it for each region in the image. In this way, an expert can identify errors in the multi-class segmentation and rapidly correct them by supplying the model with additional annotations hand-picked from the region. In addition to increasing the speed of annotation and reducing the amount of labelling, we show that our proposed method outperforms a traditional multi-class segmentation by a large margin.

NetMets: software for quantifying and visualizing errors in biological network segmentation.

One of the major goals in biomedical image processing is accurate segmentation of networks embedded in volumetric data sets. Biological networks are composed of a meshwork of thin filaments that span large volumes of tissue. Examples of these structures include neurons and microvasculature, which can take the form of both hierarchical trees and fully connected networks, depending on the imaging modality and resolution. Network function depends on both the geometric structure and connectivity. Therefore, there is considerable demand for algorithms that segment biological networks embedded in three-dimensional data. While a large number of tracking and segmentation algorithms have been published, most of these do not generalize well across data sets. One of the major reasons for the lack of general-purpose algorithms is the limited availability of metrics that can be used to quantitatively compare their effectiveness against a pre-constructed ground-truth. In this paper, we propose a robust metric for measuring and visualizing the differences between network models. Our algorithm takes into account both geometry and connectivity to measure network similarity. These metrics are then mapped back onto an explicit model for visualization.

A hyperfused mitochondrial state achieved at G1-S regulates cyclin E buildup and entry into S phase.

Mitochondria undergo fission-fusion events that render these organelles highly dynamic in cells. We report a relationship between mitochondrial form and cell cycle control at the G(1)-S boundary. Mitochondria convert from isolated, fragmented elements into a hyperfused, giant network at G(1)-S transition. The network is electrically continuous and has greater ATP output than mitochondria at any other cell cycle stage. Depolarizing mitochondria at early G(1) to prevent these changes causes cell cycle progression into S phase to be blocked. Inducing mitochondrial hyperfusion by acute inhibition of dynamin-related protein-1 (DRP1) causes quiescent cells maintained without growth factors to begin replicating their DNA and coincides with buildup of cyclin E, the cyclin responsible for G(1)-to-S phase progression. Prolonged or untimely formation of hyperfused mitochondria, through chronic inhibition of DRP1, causes defects in mitotic chromosome alignment and S-phase entry characteristic of cyclin E overexpression. These findings suggest a hyperfused mitochondrial system with specialized properties at G(1)-S is linked to cyclin E buildup for regulation of G(1)-to-S progression.

Impaired Experience-Dependent Refinement of Place Cells in a Rat Model of Alzheimer's Disease.

BACKGROUND: Hippocampal place cells play an integral role in generating spatial maps. Impaired spatial memory is a characteristic pathology of Alzheimer's disease (AD), yet it remains unclear how AD influences the properties of hippocampal place cells. OBJECTIVE: To record electrophysiological activity in hippocampal CA1 neurons in freely-moving 18-month-old male TgF344-AD and age-matched wild-type (WT) littermates to examine place cell properties. METHODS: We implanted 32-channel electrode arrays into the CA1 subfield of 18-month-old male WT and TgF344-AD (n = 6/group) rats. Ten days after implantation, single unit activity in an open field arena was recorded across days. The spatial information content, in-field firing rate, and stability of each place cell was compared across groups. Pathology was assessed by immunohistochemical staining, and a deep neural network approach was used to count cell profiles. RESULTS: Aged TgF344-AD rats exhibited hippocampal amyloid-ß deposition, and a significant increase in Iba1 immunoreactivity and microglia cell counts. Place cells from WT and TgF344-AD rat showed equivalent spatial information, in-field firing rates, and place field stability when initially exposed to the arena. However, by day 3, the place cells in aged WT rats showed characteristic spatial tuning as evidenced by higher spatial information content, stability, and in-field firing rates, an effect not seen in TgF344-AD rats. CONCLUSION: These findings support the notion that altered electrophysiological properties of place cells may contribute to the learning and memory deficits observed in AD.

Quantifying subcellular distribution of fluorescent fusion proteins in cells migrating within tissues.

The movement of proteins within cells can provide dynamic indications of cell signaling and cell polarity, but methods are needed to track and quantify subcellular protein movement within tissue environments. Here we present a semiautomated approach to quantify subcellular protein location for hundreds of migrating cells within intact living tissue using retrovirally expressed fluorescent fusion proteins and time-lapse two-photon microscopy of intact thymic lobes. We have validated the method using GFP-PKCζ, a marker for cell polarity, and LAT-GFP, a marker for T-cell receptor signaling, and have related the asymmetric distribution of these proteins to the direction and speed of cell migration. These approaches could be readily adapted to other fluorescent fusion proteins, tissues and biological questions.