DeepSeeded: Volumetric Segmentation of Dense Cell Populations with a Cascade of Deep Neural Networks in Bacterial Biofilm Applications.

Accurate and automatic segmentation of individual cell instances in microscopy images is a vital step for quantifying the cellular attributes, which can subsequently lead to new discoveries in biomedical research. In recent years, data-driven deep learning techniques have shown promising results in this task. Despite the success of these techniques, many fail to accurately segment cells in microscopy images with high cell density and low signal-to-noise ratio. In this paper, we propose a novel 3D cell segmentation approach DeepSeeded, a cascaded deep learning architecture that estimates seeds for a classical seeded watershed segmentation. The cascaded architecture enhances the cell interior and border information using Euclidean distance transforms and detects the cell seeds by performing voxel-wise classification. The data-driven seed estimation process proposed here allows segmenting touching cell instances from a dense, intensity-inhomogeneous microscopy image volume. We demonstrate the performance of the proposed method in segmenting 3D microscopy images of a particularly dense cell population called bacterial biofilms. Experimental results on synthetic and two real biofilm datasets suggest that the proposed method leads to superior segmentation results when compared to state-of-the-art deep learning methods and a classical method.

A Semantic and Motion-Aware Spatiotemporal Transformer Network for Action Detection.

This paper presents a novel spatiotemporal transformer network that introduces several original components to detect actions in untrimmed videos. First, the multi-feature selective semantic attention model calculates the correlations between spatial and motion features to model spatiotemporal interactions between different action semantics properly. Second, the motion-aware network encodes the locations of action semantics in video frames utilizing the motion-aware 2D positional encoding algorithm. Such a motion-aware mechanism memorizes the dynamic spatiotemporal variations in action frames that current methods cannot exploit. Third, the sequence-based temporal attention model captures the heterogeneous temporal dependencies in action frames. In contrast to standard temporal attention used in natural language processing, primarily aimed at finding similarities between linguistic words, the proposed sequence-based temporal attention is designed to determine both the differences and similarities between video frames that jointly define the meaning of actions. The proposed approach outperforms the state-of-the-art solutions on four spatiotemporal action datasets: AVA 2.2, AVA 2.1, UCF101-24, and EPIC-Kitchens.

Efficient Learning of Transform-Domain LMS Filter Using Graph Laplacian.

Transform-domain least mean squares (TDLMS) adaptive filters encompass the class of learning algorithms where the input data are subjected to a data-independent unitary transform followed by a power normalization stage as preprocessing steps. Because conventional transformations are not data-dependent, this preconditioning procedure was shown theoretically to improve the convergence of the least mean squares (LMS) filter only for certain classes of input data. So, one can tailor the transformation to the class of data. However, in reality, if the class of input data is not known beforehand, it is difficult to decide which transformation to use. Thus, there is a need to devise a learning framework to obtain such a preconditioning transformation using input data prior to applying on the input data. It is hypothesized that the underlying topology of the data affects the selection of the transformation. With the input modeled as a weighted finite graph, our method, called preconditioning using graph (PrecoG), adaptively learns the desired transform by recursive estimation of the graph Laplacian matrix. We show the efficacy of the transform as a generalized split preconditioner on a linear system of equations and in Hebbian-LMS learning models. In terms of the improvement of the condition number after applying the transformation, PrecoG performs significantly better than the existing state-of-the-art techniques that involve unitary and nonunitary transforms.

BCM3D 2.0: accurate segmentation of single bacterial cells in dense biofilms using computationally generated intermediate image representations.

Accurate detection and segmentation of single cells in three-dimensional (3D) fluorescence time-lapse images is essential for observing individual cell behaviors in large bacterial communities called biofilms. Recent progress in machine-learning-based image analysis is providing this capability with ever-increasing accuracy. Leveraging the capabilities of deep convolutional neural networks (CNNs), we recently developed bacterial cell morphometry in 3D (BCM3D), an integrated image analysis pipeline that combines deep learning with conventional image analysis to detect and segment single biofilm-dwelling cells in 3D fluorescence images. While the first release of BCM3D (BCM3D 1.0) achieved state-of-the-art 3D bacterial cell segmentation accuracies, low signal-to-background ratios (SBRs) and images of very dense biofilms remained challenging. Here, we present BCM3D 2.0 to address this challenge. BCM3D 2.0 is entirely complementary to the approach utilized in BCM3D 1.0. Instead of training CNNs to perform voxel classification, we trained CNNs to translate 3D fluorescence images into intermediate 3D image representations that are, when combined appropriately, more amenable to conventional mathematical image processing than a single experimental image. Using this approach, improved segmentation results are obtained even for very low SBRs and/or high cell density biofilm images. The improved cell segmentation accuracies in turn enable improved accuracies of tracking individual cells through 3D space and time. This capability opens the door to investigating time-dependent phenomena in bacterial biofilms at the cellular level.

3D GAN image synthesis and dataset quality assessment for bacterial biofilm.

MOTIVATION: Data-driven deep learning techniques usually require a large quantity of labeled training data to achieve reliable solutions in bioimage analysis. However, noisy image conditions and high cell density in bacterial biofilm images make 3D cell annotations difficult to obtain. Alternatively, data augmentation via synthetic data generation is attempted, but current methods fail to produce realistic images. RESULTS: This article presents a bioimage synthesis and assessment workflow with application to augment bacterial biofilm images. 3D cyclic generative adversarial networks (GAN) with unbalanced cycle consistency loss functions are exploited in order to synthesize 3D biofilm images from binary cell labels. Then, a stochastic synthetic dataset quality assessment (SSQA) measure that compares statistical appearance similarity between random patches from random images in two datasets is proposed. Both SSQA scores and other existing image quality measures indicate that the proposed 3D Cyclic GAN, along with the unbalanced loss function, provides a reliably realistic (as measured by mean opinion score) 3D synthetic biofilm image. In 3D cell segmentation experiments, a GAN-augmented training model also presents more realistic signal-to-background intensity ratio and improved cell counting accuracy. AVAILABILITY AND IMPLEMENTATION: https://github.com/jwang-c/DeepBiofilm. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

CAT: Centerness-Aware Anchor-Free Tracker.

Accurate and robust scale estimation in visual object tracking is a challenging task. To obtain a scale estimation of the target object, most methods rely either on a multi-scale searching scheme or on refining a set of predefined anchor boxes. These methods require heuristically selected parameters, such as scale factors of the multi-scale searching scheme, or sizes and aspect ratios of the predefined candidate anchor boxes. On the contrary, a centerness-aware anchor-free tracker (CAT) is designed in this work. First, the location and scale of the target object are predicted in an anchor-free fashion by decomposing tracking into parallel classification and regression problems. The proposed anchor-free design obviates the need for hyperparameters related to the anchor boxes, making CAT more generic and flexible. Second, the proposed centerness-aware classification branch can identify the foreground from the background while predicting the normalized distance from the location within the foreground to the target center, i.e., the centerness. The proposed centerness-aware classification branch improves the tracking accuracy and robustness significantly by suppressing low-quality state estimates. The experiments show that our centerness-aware anchor-free tracker, with its appealing features, achieves salient performance in a wide variety of tracking scenarios.

Graph-Theoretic Post-Processing of Segmentation With Application to Dense Biofilms.

Recent deep learning methods have provided successful initial segmentation results for generalized cell segmentation in microscopy. However, for dense arrangements of small cells with limited ground truth for training, the deep learning methods produce both over-segmentation and under-segmentation errors. Post-processing attempts to balance the trade-off between the global goal of cell counting for instance segmentation, and local fidelity to the morphology of identified cells. The need for post-processing is especially evident for segmenting 3D bacterial cells in densely-packed communities called biofilms. A graph-based recursive clustering approach, m-LCuts, is proposed to automatically detect collinearly structured clusters and applied to post-process unsolved cells in 3D bacterial biofilm segmentation. Construction of outlier-removed graphs to extract the collinearity feature in the data adds additional novelty to m-LCuts. The superiority of m-LCuts is observed by the evaluation in cell counting with over 90% of cells correctly identified, while a lower bound of 0.8 in terms of average single-cell segmentation accuracy is maintained. This proposed method does not need manual specification of the number of cells to be segmented. Furthermore, the broad adaptation for working on various applications, with the presence of data collinearity, also makes m-LCuts stand out from the other approaches.

Closed-Loop Low-Rank Echocardiographic Artifact Removal.

Echocardiographic image sequences are frequently corrupted by quasi-static artifacts ("clutter") superimposed on the moving myocardium. Conventionally, localized blind source separation methods exploiting local correlation in the clutter have proven effective in the suppression of these artifacts. These methods use the spectral characteristics to distinguish the clutter from tissue and background noise and are applied exhaustively over the data set. The exhaustive application results in high computational complexity and a loss of useful tissue signal. In this article, we develop a closed-loop algorithm in which the clutter is first detected using an adaptively determined weighting function and then removed using low-rank estimation methods. We show that our method is adaptable to different low-rank estimators, by presenting two such estimators: sparse coding in the principal component domain and nuclear norm minimization. We compare the performance of our proposed method (CLEAR) with two methods: singular value filtering (SVF) and morphological component analysis (MCA). The performance was quantified in silico by measuring the error with respect to a known "ground truth" data set with no clutter for different combinations of moving clutter and tissue. Our method retains more tissue with a lower error of 3.88 ± 0.093 dB (sparse coding) and 3.47 ± 0.78 (nuclear norm) compared with the benchmark methods 8.5 ± 0.7 dB (SVF) and 9.3 ± 0.5 dB (MCA) particularly in instances where the rate of tissue motion and artifact motion is small (≤0.25 periods of center frequency per frame) while producing comparable clutter reduction performance. CLEAR was also validated in vivo by quantifying the tracking error over the cardiac cycle on five mouse heart data sets with synthetic clutter. CLEAR reduced the error by approximately 50%, compared with 25% for the SVF.

Microglial Calcium Waves During the Hyperacute Phase of Ischemic Stroke.

BACKGROUND AND PURPOSE: Ischemic injury triggers multiple pathological responses in the brain tissue, including spreading depolarizations across the cerebral cortex (cortical spreading depolarizations [CSD]). Microglia have been recently shown to play a significant role in the propagation of CSD. However, the intracellular responses of myeloid cells during ischemic stroke have not been investigated. METHODS: We have studied intracellular calcium activity in cortical microglia in the stroke model of the middle cerebral artery occlusion, using the murine Polr2a-based and Cre-dependent GCaMP5 and tdTomato reporter (PC::G5-tdT). High-speed 2-photon microscopy through cranial windows was employed to record signals from genetically encoded indicators of calcium. Inflammatory stimuli and pharmacological inhibition were used to modulate microglial calcium responses in the somatosensory cortex. RESULTS: In vivo imaging revealed periodical calcium activity in microglia during the hyperacute phase of ischemic stroke. This activity was more frequent during the first 6 hours after occlusion, but the amplitudes of calcium transients became larger at later time points. Consistent with CSD nature of these events, we reproducibly triggered comparable calcium transients with microinjections of potassium chloride (KCl) into adjacent cortical areas. Furthermore, lipopolysaccharide-induced peripheral inflammation, mimicking sterile inflammation during ischemic stroke, produced significantly greater microglial calcium transients during CSD. Finally, in vivo pharmacological analysis with CRAC (calcium release-activated channel) inhibitor CM-EX-137 demonstrated that CSD-associated microglial calcium transients after KCl microinjections are mediated at least in part by the CRAC mechanism. CONCLUSIONS: Our findings demonstrate that microglia participate in ischemic brain injury via previously undetected mechanisms, which may provide new avenues for therapeutic interventions.

Non-invasive single-cell morphometry in living bacterial biofilms.

Fluorescence microscopy enables spatial and temporal measurements of live cells and cellular communities. However, this potential has not yet been fully realized for investigations of individual cell behaviors and phenotypic changes in dense, three-dimensional (3D) bacterial biofilms. Accurate cell detection and cellular shape measurement in densely packed biofilms are challenging because of the limited resolution and low signal to background ratios (SBRs) in fluorescence microscopy images. In this work, we present Bacterial Cell Morphometry 3D (BCM3D), an image analysis workflow that combines deep learning with mathematical image analysis to accurately segment and classify single bacterial cells in 3D fluorescence images. In BCM3D, deep convolutional neural networks (CNNs) are trained using simulated biofilm images with experimentally realistic SBRs, cell densities, labeling methods, and cell shapes. We systematically evaluate the segmentation accuracy of BCM3D using both simulated and experimental images. Compared to state-of-the-art bacterial cell segmentation approaches, BCM3D consistently achieves higher segmentation accuracy and further enables automated morphometric cell classifications in multi-population biofilms.

NeuroPath2Path: Classification and elastic morphing between neuronal arbors using path-wise similarity.

Neuron shape and connectivity affect function. Modern imaging methods have proven successful at extracting morphological information. One potential path to achieve analysis of this morphology is through graph theory. Encoding by graphs enables the use of high throughput informatic methods to extract and infer brain function. However, the application of graph-theoretic methods to neuronal morphology comes with certain challenges in term of complex subgraph matching and the difficulty in computing intermediate shapes in between two imaged temporal samples. Here we report a novel, efficacious graph-theoretic method that rises to the challenges. The morphology of a neuron, which consists of its overall size, global shape, local branch patterns, and cell-specific biophysical properties, can vary significantly with the cell's identity, location, as well as developmental and physiological state. Various algorithms have been developed to customize shape based statistical and graph related features for quantitative analysis of neuromorphology, followed by the classification of neuron cell types using the features. Unlike the classical feature extraction based methods from imaged or 3D reconstructed neurons, we propose a model based on the rooted path decomposition from the soma to the dendrites of a neuron and extract morphological features from each constituent path. We hypothesize that measuring the distance between two neurons can be realized by minimizing the cost of continuously morphing the set of all rooted paths of one neuron to another. To validate this claim, we first establish the correspondence of paths between two neurons using a modified Munkres algorithm. Next, an elastic deformation framework that employs the square root velocity function is established to perform the continuous morphing, which, as an added benefit, provides an effective visualization tool. We experimentally show the efficacy of NeuroPath2Path, NeuroP2P, over the state of the art.

SITUP: Scale Invariant Tracking using Average Peak-to-Correlation Energy.

Robust and accurate scale estimation of a target object is a challenging task in visual object tracking. Most existing tracking methods cannot accommodate large scale variation in complex image sequences and thus result in inferior performance. In this paper, we propose to incorporate a novel criterion called the average peak-to-correlation energy into the multi-resolution translation filter framework to obtain robust and accurate scale estimation. The resulting system is named SITUP: Scale Invariant Tracking using Average Peak-to-Correlation Energy. SITUP effectively tackles the problem of fixed template size in standard discriminative correlation filter based trackers. Extensive empirical evaluation on the publicly available tracking benchmark datasets demonstrates that the proposed scale searching framework meets the demands of scale variation challenges effectively while providing superior performance over other scale adaptive variants of standard discriminative correlation filter based trackers. Also, SITUP obtains favorable performance compared to state-of-the-art trackers for various scenarios while operating in real-time on a single CPU.

Calcium Imaging of Microglial Network Activity in Stroke.

Calcium signaling plays a significant role in microglial activation. Genetically encoded calcium indicators (GECI) have been widely used for calcium imaging studies in many brain cell types, including neurons, astrocytes, and oligodendrocytes. However, microglial calcium imaging approaches have been hampered by idiosyncrasies of their gene expression and malleable cell properties. The generation of PC::G5-tdT, a Polr2a locus-based conditional mouse reporter of calcium, facilitated the deployment of GECI in microglia. When crossed with the Iba1(Aif1)-IRES-Cre line, all brain microglia of the progeny are labeled with the calcium indicator variant GCaMP5G and the red fluorescent protein tdTomato. This reporter system has enabled in vivo studies of intracellular calcium in large microglial cell populations in cerebral pathologies such as ischemic stroke. In this chapter, we outline specific guidelines for genetic, surgical, imaging, and data analysis aspects of microglial calcium monitoring of the ischemic cortex following middle cerebral artery occlusion.

Content-Aware Enhancement of Images With Filamentous Structures.

In this paper, we describe a novel enhancement method for images containing filamentous structures. Our method combines a gradient sparsity constraint with a filamentous structure constraint for the effective removal of clutter and noise from the background. The method is applied and evaluated on three types of data: 1) confocal microscopy images of neurons; 2) calcium imaging data; and 3) images of road pavement. We found that the images enhanced by our method preserve both the structure and the intensity details of the original object. In the case of neuron microscopy, we find that the neurons enhanced by our method are better correlated with the original structure intensities than the neurons enhanced by well-known vessel enhancement methods. Experiments on simulated calcium imaging data indicate that both the number of detected neurons and the accuracy of the derived calcium activity are improved. Applying our method to real calcium data, more regions exhibiting calcium activity in the full field of view were found. In road pavement crack detection, smaller or milder cracks were detected after using our enhancement method.

Speckle Removal Using Diffusion Potential for Optical Coherence Tomography Images.

We propose a fast and accurate solution to speckle reduction targeted specifically at optical coherence tomography images. The proposed speckle removing filter is designed using a novel potential function based on the gradient of the local variance of intensity. After filtering, the spatially neighboring pixels with close values of intensities converge to uniform gray values, while the edges remain intact. This filtering process results in removal of speckle without destroying the edges of the desired object. The proposed filter also prevents the generation of any false edges. Detailed experimental analysis shows at least 1-dB improvement in the peak signal-to-noise ratio for spectral domain optical coherence tomography images. The method also shows superior edge preservation, contrast, and speed compared to the state of the art in speckle removing filters.

Publisher Correction: Functional aspects of meningeal lymphatics in ageing and Alzheimer's disease.

Change history: In this Article, Extended Data Fig. 9 was appearing as Fig. 2 in the HTML, and in Fig. 2, the panel labels 'n' and 'o' overlapped the figure; these errors have been corrected online.

MRI analysis to map interstitial flow in the brain tumor microenvironment.

Glioblastoma (GBM), a highly aggressive form of brain tumor, is a disease marked by extensive invasion into the surrounding brain. Interstitial fluid flow (IFF), or the movement of fluid within the spaces between cells, has been linked to increased invasion of GBM cells. Better characterization of IFF could elucidate underlying mechanisms driving this invasion in vivo. Here, we develop a technique to noninvasively measure interstitial flow velocities in the glioma microenvironment of mice using dynamic contrast-enhanced magnetic resonance imaging (MRI), a common clinical technique. Using our in vitro model as a phantom "tumor" system and in silico models of velocity vector fields, we show we can measure average velocities and accurately reconstruct velocity directions. With our combined MR and analysis method, we show that velocity magnitudes are similar across four human GBM cell line xenograft models and the direction of fluid flow is heterogeneous within and around the tumors, and not always in the outward direction. These values were not linked to the tumor size. Finally, we compare our flow velocity magnitudes and the direction of flow to a classical marker of vessel leakage and bulk fluid drainage, Evans blue. With these data, we validate its use as a marker of high and low IFF rates and IFF in the outward direction from the tumor border in implanted glioma models. These methods show, for the first time, the nature of interstitial fluid flow in models of glioma using a technique that is translatable to clinical and preclinical models currently using contrast-enhanced MRI.

Functional aspects of meningeal lymphatics in ageing and Alzheimer's disease.

Ageing is a major risk factor for many neurological pathologies, but its mechanisms remain unclear. Unlike other tissues, the parenchyma of the central nervous system (CNS) lacks lymphatic vasculature and waste products are removed partly through a paravascular route. (Re)discovery and characterization of meningeal lymphatic vessels has prompted an assessment of their role in waste clearance from the CNS. Here we show that meningeal lymphatic vessels drain macromolecules from the CNS (cerebrospinal and interstitial fluids) into the cervical lymph nodes in mice. Impairment of meningeal lymphatic function slows paravascular influx of macromolecules into the brain and efflux of macromolecules from the interstitial fluid, and induces cognitive impairment in mice. Treatment of aged mice with vascular endothelial growth factor C enhances meningeal lymphatic drainage of macromolecules from the cerebrospinal fluid, improving brain perfusion and learning and memory performance. Disruption of meningeal lymphatic vessels in transgenic mouse models of Alzheimer's disease promotes amyloid-ß deposition in the meninges, which resembles human meningeal pathology, and aggravates parenchymal amyloid-ß accumulation. Meningeal lymphatic dysfunction may be an aggravating factor in Alzheimer's disease pathology and in age-associated cognitive decline. Thus, augmentation of meningeal lymphatic function might be a promising therapeutic target for preventing or delaying age-associated neurological diseases.

SDL: Saliency-Based Dictionary Learning Framework for Image Similarity.

In image classification, obtaining adequate data to learn a robust classifier has often proven to be difficult in several scenarios. Classification of histological tissue images for health care analysis is a notable application in this context due to the necessity of surgery, biopsy or autopsy. To adequately exploit limited training data in classification, we propose a saliency guided dictionary learning method and subsequently an image similarity technique for histo-pathological image classification. Salient object detection from images aids in the identification of discriminative image features. We leverage the saliency values for the local image regions to learn a dictionary and respective sparse codes for an image, such that the more salient features are reconstructed with smaller error. The dictionary learned from an image gives a compact representation of the image itself and is capable of representing images with similar content, with comparable sparse codes. We employ this idea to design a similarity measure between a pair of images, where local image features of one image, are encoded with the dictionary learned from the other and vice versa. To effectively utilize the learned dictionary, we take into account the contribution of each dictionary atom in the sparse codes to generate a global image representation for image comparison. The efficacy of the proposed method was evaluated using three tissue data sets that consist of mammalian kidney, lung and spleen tissue, breast cancer, and colon cancer tissue images. From the experiments, we observe that our methods outperform the state of the art with an increase of 14.2% in the average classification accuracy over all data sets.

MISTICA: Minimum Spanning Tree-Based Coarse Image Alignment for Microscopy Image Sequences.

Registration of an in vivo microscopy image sequence is necessary in many significant studies, including studies of atherosclerosis in large arteries and the heart. Significant cardiac and respiratory motion of the living subject, occasional spells of focal plane changes, drift in the field of view, and long image sequences are the principal roadblocks. The first step in such a registration process is the removal of translational and rotational motion. Next, a deformable registration can be performed. The focus of our study here is to remove the translation and/or rigid body motion that we refer to here as coarse alignment. The existing techniques for coarse alignment are unable to accommodate long sequences often consisting of periods of poor quality images (as quantified by a suitable perceptual measure). Many existing methods require the user to select an anchor image to which other images are registered. We propose a novel method for coarse image sequence alignment based on minimum weighted spanning trees (MISTICA) that overcomes these difficulties. The principal idea behind MISTICA is to reorder the images in shorter sequences, to demote nonconforming or poor quality images in the registration process, and to mitigate the error propagation. The anchor image is selected automatically making MISTICA completely automated. MISTICA is computationally efficient. It has a single tuning parameter that determines graph width, which can also be eliminated by the way of additional computation. MISTICA outperforms existing alignment methods when applied to microscopy image sequences of mouse arteries.