Immersive and interactive visualization of 3D spatio-temporal data using a space time hypercube: Application to cell division and morphogenesis analysis.

The analysis of multidimensional time-varying datasets faces challenges, notably regarding the representation of the data and the visualization of temporal variations. We propose an extension of the well-known Space-Time Cube (STC) visualization technique in order to visualize time-varying 3D spatial data, taking advantage of the interaction capabilities of Virtual Reality (VR). First, we propose the Space-Time Hypercube (STH) as an abstraction for 3D temporal data, extended from the STC concept. Second, through the example of embryo development imaging dataset, we detail the construction and visualization of a STC based on a user-driven projection of the spatial and temporal information. This projection yields a 3D STC visualization, which can also encode additional numerical and categorical data. Additionally, we propose a set of tools allowing the user to filter and manipulate the 3D STC which benefits the visualization, exploration and interaction possibilities offered by VR. Finally, we evaluated the proposed visualization method in the context of 3D temporal cell imaging data analysis, through a user study (n = 5) reporting the feedback from five biologists. These domain experts also accompanied the application design as consultants, providing insights on how the STC visualization could be used for the exploration of complex 3D temporal morphogenesis data.

Sparse denoising and adaptive estimation enhances the resolution and contrast of fluorescence emission difference microscopy based on an array detector.

An array detector allows a resolution gain for confocal microscopy by combining images sensed by a set of photomultipliers tubes (or sub-detectors). Several methods have been proposed to reconstruct a high-resolution image by linearly combining sub-detector images, especially the fluorescence emission difference (FED) technique. To improve the resolution and contrast of FED microscopy based on an array detector, we propose to associate sparse denoising with spatial adaptive estimation. We show on both calibration slides and real data that our approach applied to the full stack of spatially reassigned detector signals, enables us to achieve a higher reconstruction performance in terms of resolution, image contrast, and noise reduction.

SPITFIR(e): a supermaneuverable algorithm for fast denoising and deconvolution of 3D fluorescence microscopy images and videos.

Modern fluorescent microscopy imaging is still limited by the optical aberrations and the photon budget available in the specimen. A direct consequence is the necessity to develop flexible and "off-road" algorithms in order to recover structural details and improve spatial resolution, which is critical when restraining the illumination to low levels in order to limit photo-damages. Here, we report SPITFIR(e) a flexible method designed to accurately and quickly restore 2D-3D fluorescence microscopy images and videos (4D images). We designed a generic sparse-promoting regularizer to subtract undesirable out-of-focus background and we developed a primal-dual algorithm for fast optimization. SPITFIR(e) is a "swiss-knife" method for practitioners as it adapts to any microscopy techniques, to various sources of signal degradation (noise, blur), to variable image contents, as well as to low signal-to-noise ratios. Our method outperforms existing state-of-the-art algorithms, and is more flexible than supervised deep-learning methods requiring ground truth datasets. The performance, the flexibility, and the ability to push the spatiotemporal resolution limit of sub-diffracted fluorescence microscopy techniques are demonstrated on experimental datasets acquired with various microscopy techniques from 3D spinning-disk confocal up to lattice light sheet microscopy.

BayesTICS: Local temporal image correlation spectroscopy and Bayesian simulation technique for sparse estimation of diffusion in fluorescence imaging.

The dynamics and fusion of vesicles during the last steps of exocytosis are not well established yet in cell biology. An open issue is the characterization of the diffusion process at the plasma membrane. Total internal reflection fluorescence microscopy (TIRFM) has been successfully used to analyze the coordination of proteins involved in this mechanism. It enables to capture dynamics of proteins with high frame rate and reasonable signal-to-noise values. Nevertheless, methodological approaches that can analyze and estimate diffusion in local small areas at the scale of a single diffusing spot within cells, are still lacking. To address this issue, we propose a novel correlation-based method for local diffusion estimation. As a starting point, we consider Fick's second law of diffusion that relates the diffusive flux to the gradient of the concentration. Then, we derive an explicit parametric model which is further fitted to time-correlation signals computed from regions of interest (ROI) containing individual spots. Our modeling and Bayesian estimation framework are well appropriate to represent isolated diffusion events and are robust to noise, ROI sizes, and localization of spots in ROIs. The performance of BayesTICS is shown on both synthetic and real TIRFM images depicting Transferrin Receptor proteins.

A generative model to simulate spatiotemporal dynamics of biomolecules in cells.

Generators of space-time dynamics in bioimaging have become essential to build ground truth datasets for image processing algorithm evaluation such as biomolecule detectors and trackers, as well as to generate training datasets for deep learning algorithms. In this contribution, we leverage a stochastic model, called birth-death-move (BDM) point process, in order to generate joint dynamics of biomolecules in cells. This particle-based stochastic simulation method is very flexible and can be seen as a generalization of well-established standard particle-based generators. In comparison, our approach allows us: (1) to model a system of particles in motion, possibly in interaction, that can each possibly switch from a motion regime (e.g., Brownian) to another (e.g., a directed motion); (2) to take into account finely the appearance over time of new trajectories and their disappearance, these events possibly depending on the cell regions but also on the current spatial configuration of all existing particles. This flexibility enables to generate more realistic dynamics than standard particle-based simulation procedures, by for example accounting for the colocalization phenomena often observed between intracellular vesicles. We explain how to specify all characteristics of a BDM model, with many practical examples that are relevant for bioimaging applications. As an illustration, based on real fluorescence microscopy datasets, we finally calibrate our model to mimic the joint dynamics of Langerin and Rab11 proteins near the plasma membrane, including the well-known colocalization occurrence between these two types of vesicles. We show that the resulting synthetic sequences exhibit comparable features as those observed in real microscopy image sequences.

Challenges of intracellular visualization using virtual and augmented reality.

Microscopy image observation is commonly performed on 2D screens, which limits human capacities to grasp volumetric, complex, and discrete biological dynamics. With the massive production of multidimensional images (3D + time, multi-channels) and derived images (e.g., restored images, segmentation maps, and object tracks), scientists need appropriate visualization and navigation methods to better apprehend the amount of information in their content. New modes of visualization have emerged, including virtual reality (VR)/augmented reality (AR) approaches which should allow more accurate analysis and exploration of large time series of volumetric images, such as those produced by the latest 3D + time fluorescence microscopy. They include integrated algorithms that allow researchers to interactively explore complex spatiotemporal objects at the scale of single cells or multicellular systems, almost in a real time manner. In practice, however, immersion of the user within 3D + time microscopy data represents both a paradigm shift in human-image interaction and an acculturation challenge, for the concerned community. To promote a broader adoption of these approaches by biologists, further dialogue is needed between the bioimaging community and the VR&AR developers.

DCT2net: An Interpretable Shallow CNN for Image Denoising.

This work tackles the issue of noise removal from images, focusing on the well-known DCT image denoising algorithm. The latter, stemming from signal processing, has been well studied over the years. Though very simple, it is still used in crucial parts of state-of-the-art "traditional" denoising algorithms such as BM3D. For a few years however, deep convolutional neural networks (CNN), especially DnCNN, have outperformed their traditional counterparts, making signal processing methods less attractive. In this paper, we demonstrate that a DCT denoiser can be seen as a shallow CNN and thereby its original linear transform can be tuned through gradient descent in a supervised manner, improving considerably its performance. This gives birth to a fully interpretable CNN called DCT2net. To deal with remaining artifacts induced by DCT2net, an original hybrid solution between DCT and DCT2net is proposed combining the best that these two methods can offer; DCT2net is selected to process non-stationary image patches while DCT is optimal for piecewise smooth patches. Experiments on artificially noisy images demonstrate that two-layer DCT2net provides comparable results to BM3D and is as fast as DnCNN algorithm.

STracking: a free and open-source Python library for particle tracking and analysis.

SUMMARY: Analysis of intra- and extracellular dynamic like vesicles transport involves particle tracking algorithms. The design of a particle tracking pipeline is a routine but tedious task. Therefore, particle dynamics analysis is often performed by combining several pieces of software (filtering, detection, tracking, etc.) requiring many manual operations, and thus leading to poorly reproducible results. Given the new segmentation tools based on deep learning, modularity and interoperability between software have become essential in particle tracking algorithms. A good synergy between a particle detector and a tracker is of paramount importance. In addition, a user-friendly interface to control the quality of estimated trajectories is necessary. To address these issues, we developed STracking, a Python library that allows combining algorithms into standardized particle tracking pipelines. AVAILABILITY AND IMPLEMENTATION: STracking is available as a Python library using 'pip install' and the source code is publicly available on GitHub (https://github.com/sylvainprigent/stracking). A graphical interface is available using two napari plugins: napari-stracking and napari-tracks-reader. These napari plugins can be installed via the napari plugins menu or using 'pip install'. The napari plugin source codes are available on GitHub (https://github.com/sylvainprigent/napari-tracks-reader, https://github.com/sylvainprigent/napari-stracking). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Deep learning improves macromolecule identification in 3D cellular cryo-electron tomograms.

Cryogenic electron tomography (cryo-ET) visualizes the 3D spatial distribution of macromolecules at nanometer resolution inside native cells. However, automated identification of macromolecules inside cellular tomograms is challenged by noise and reconstruction artifacts, as well as the presence of many molecular species in the crowded volumes. Here, we present DeepFinder, a computational procedure that uses artificial neural networks to simultaneously localize multiple classes of macromolecules. Once trained, the inference stage of DeepFinder is faster than template matching and performs better than other competitive deep learning methods at identifying macromolecules of various sizes in both synthetic and experimental datasets. On cellular cryo-ET data, DeepFinder localized membrane-bound and cytosolic ribosomes (roughly 3.2 MDa), ribulose 1,5-bisphosphate carboxylase-oxygenase (roughly 560 kDa soluble complex) and photosystem II (roughly 550 kDa membrane complex) with an accuracy comparable to expert-supervised ground truth annotations. DeepFinder is therefore a promising algorithm for the semiautomated analysis of a wide range of molecular targets in cellular tomograms.

Quantitative dSTORM super-resolution microscopy localizes Aurora kinase A/AURKA in the mitochondrial matrix.

BACKGROUND INFORMATION: Mitochondria are dynamic organelles playing essential metabolic and signaling functions in cells. Their ultrastructure has largely been investigated with electron microscopy (EM) techniques. However, quantifying protein-protein proximities using EM is extremely challenging. Super-resolution microscopy techniques as direct stochastic optical reconstruction microscopy (dSTORM) now provide a fluorescent-based, quantitative alternative to EM. Recently, super-resolution microscopy approaches including dSTORM led to valuable advances in our knowledge of mitochondrial ultrastructure, and in linking it with new insights in organelle functions. Nevertheless, dSTORM is mostly used to image integral mitochondrial proteins, and there is little or no information on proteins transiently present at this compartment. The cancer-related Aurora kinase A/AURKA is a protein localized at various subcellular locations, including mitochondria. RESULTS: We first demonstrate that dSTORM coupled to GcoPS can resolve protein proximities within individual submitochondrial compartments. Then, we show that dSTORM provides sufficient spatial resolution to visualize and quantify the most abundant pool of endogenous AURKA in the mitochondrial matrix, as previously shown for overexpressed AURKA. In addition, we uncover a smaller pool of AURKA localized at the OMM, which could have a potential functional readout. We conclude by demonstrating that aldehyde-based fixatives are more specific for the OMM pool of the kinase instead. CONCLUSIONS: Our results indicate that dSTORM coupled to GcoPS colocalization analysis is a suitable approach to explore the compartmentalization of non-integral mitochondrial proteins as AURKA, in a qualitative and quantitative manner. This method also opens up the possibility of analyzing the proximity between AURKA and its multiple mitochondrial partners with exquisite spatial resolution, thereby allowing novel insights into the mitochondrial functions controlled by AURKA. SIGNIFICANCE: Probing and quantifying the presence of endogenous AURKA - a cell cycle-related protein localized at mitochondria - in the different organelle subcompartments, using quantitative dSTORM super-resolution microscopy.

The coordination of spindle-positioning forces during the asymmetric division of the Caenorhabditis elegans zygote.

In Caenorhabditis elegans zygote, astral microtubules generate forces essential to position the mitotic spindle, by pushing against and pulling from the cortex. Measuring microtubule dynamics there, we revealed the presence of two populations, corresponding to pulling and pushing events. It offers a unique opportunity to study, under physiological conditions, the variations of both spindle-positioning forces along space and time. We propose a threefold control of pulling force, by polarity, spindle position and mitotic progression. We showed that the sole anteroposterior asymmetry in dynein on-rate, encoding pulling force imbalance, is sufficient to cause posterior spindle displacement. The positional regulation, reflecting the number of microtubule contacts in the posterior-most region, reinforces this imbalance only in late anaphase. Furthermore, we exhibited the first direct proof that dynein processivity increases along mitosis. It reflects the temporal control of pulling forces, which strengthens at anaphase onset following mitotic progression and independently from chromatid separation. In contrast, the pushing force remains constant and symmetric and contributes to maintaining the spindle at the cell centre during metaphase.

A Monte Carlo framework for missing wedge restoration and noise removal in cryo-electron tomography.

We propose a statistical method to address an important issue in cryo-electron tomography image analysis: reduction of a high amount of noise and artifacts due to the presence of a missing wedge (MW) in the spectral domain. The method takes as an input a 3D tomogram derived from limited-angle tomography, and gives as an output a 3D denoised and artifact compensated volume. The artifact compensation is achieved by filling up the MW with meaningful information. To address this inverse problem, we compute a Minimum Mean Square Error (MMSE) estimator of the uncorrupted image. The underlying high-dimensional integral is computed by applying a dedicated Markov Chain Monte-Carlo (MCMC) sampling procedure based on the Metropolis-Hasting (MH) algorithm. The proposed MWR (Missing Wedge Restoration) algorithm can be used to enhance visualization or as a pre-processing step for image analysis, including segmentation and classification of macromolecules. Results are presented for both synthetic data and real 3D cryo-electron images.

Ends and middle: Global force balance and septum location in fission yeast.

The fission yeast cell is shaped as a very regular cylinder ending by hemi-spheres at both cell ends. Its conserved phenotypes are often used as read-outs for classifying interacting genes and protein networks. Using Pascal and Young-Laplace laws, we proposed a framework where scaling arguments predicted shapes. Here we probed quantitatively one of these relations which predicts that the division site would be located closer to the cell end with the larger radius of curvature. By combining genetics and quantitative imaging, we tested experimentally whether altered shapes of cell end correlate with a displaced division site, leading to asymmetric cell division. Our results show that the division site position depends on the radii of curvatures of both ends. This new geometrical mechanism for the proper division plane positioning could be essential to achieve even partitioning of cellular material at each cell division.

Determination of Microtubule Lattice Parameters from Cryo-electron Microscope Images Using TubuleJ.

The α-ß tubulin heterodimer undergoes subtle conformational changes during microtubule assembly. These can be modulated by external factors, whose effects on microtubule structure can be characterized on 2D views obtained by cryo-electron microscopy. Analysis of microtubule images is facilitated if they are straight enough to interpret and filter their image Fourier transform, which provide useful information concerning the arrangement of tubulin molecules inside the microtubule lattice. Here, we describe the use of the TubuleJ software to straighten microtubules and determine their lattice parameters. Basic 3D reconstructions can be performed to evaluate the relevance of these parameters. This approach can be used to analyze the effects of nucleotide analogues, drugs or MAPs on microtubule structure, or to select microtubule images prior to high-resolution 3D reconstructions.

Testing independence between two random sets for the analysis of colocalization in bioimaging.

Colocalization aims at characterizing spatial associations between two fluorescently tagged biomolecules by quantifying the co-occurrence and correlation between the two channels acquired in fluorescence microscopy. Colocalization is presented either as the degree of overlap between the two channels or the overlays of the red and green images, with areas of yellow indicating colocalization of the molecules. This problem remains an open issue in diffraction-limited microscopy and raises new challenges with the emergence of superresolution imaging, a microscopic technique awarded by the 2014 Nobel prize in chemistry. We propose GcoPS, for Geo-coPositioning System, an original method that exploits the random sets structure of the tagged molecules to provide an explicit testing procedure. Our simulation study shows that GcoPS unequivocally outperforms the best competitive methods in adverse situations (noise, irregularly shaped fluorescent patterns, and different optical resolutions). GcoPS is also much faster, a decisive advantage to face the huge amount of data in superresolution imaging. We demonstrate the performances of GcoPS on two biological real data sets, obtained by conventional diffraction-limited microscopy technique and by superresolution technique, respectively.

A sequential algorithm to detect diffusion switching along intracellular particle trajectories.

MOTIVATION: Recent advances in molecular biology and fluorescence microscopy imaging have made possible the inference of the dynamics of single molecules in living cells. Changes of dynamics can occur along a trajectory. Then, an issue is to estimate the temporal change-points that is the times at which a change of dynamics occurs. The number of points in the trajectory required to detect such changes will depend on both the magnitude and type of the motion changes. Here, the number of points per trajectory is of the order of 102, even if in practice dramatic motion changes can be detected with less points. RESULTS: We propose a non-parametric procedure based on test statistics computed on local windows along the trajectory to detect the change-points. This algorithm controls the number of false change-point detections in the case where the trajectory is fully Brownian. We also develop a strategy for aggregating the detections obtained with different window sizes so that the window size is no longer a parameter to optimize. A Monte Carlo study is proposed to demonstrate the performances of the method and also to compare the procedure to two competitive algorithms. At the end, we illustrate the efficacy of the method on real data in 2D and 3D, depicting the motion of mRNA complexes-called mRNA-binding proteins-in neuronal dendrites, Galectin-3 endocytosis and trafficking within the cell. AVAILABILITY AND IMPLEMENTATION: A user-friendly Matlab package containing examples and the code of the simulations used in the paper is available at http://serpico.rennes.inria.fr/doku.php? id=software:cpanalysis:index. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

An overview of diffusion models for intracellular dynamics analysis.

We present an overview of diffusion models commonly used for quantifying the dynamics of intracellular particles (e.g. biomolecules) inside eukaryotic living cells. It is established that inference on the modes of mobility of molecules is central in cell biology since it reflects interactions between structures and determines functions of biomolecules in the cell. In that context, Brownian motion is a key component in short distance transportation (e.g. connectivity for signal transduction). Another dynamical process that has been heavily studied in the past decade is the motor-mediated transport (e.g. dynein, kinesin and myosin) of molecules. Primarily supported by actin filament and microtubule network, it ensures spatial organization and temporal synchronization in the intracellular mechanisms and structures. Nevertheless, the complexity of internal structures and molecular processes in the living cell influence the molecular dynamics and prevent the systematic application of pure Brownian or directed motion modeling. On the one hand, cytoskeleton density will hinder the free displacement of the particle, a phenomenon called subdiffusion. On the other hand, the cytoskeleton elasticity combined with thermal bending can contribute a phenomenon called superdiffusion. This paper discusses the basics of diffusion modes observed in eukariotic cells, by introducing the essential properties of these processes. Applications of diffusion models include protein trafficking and transport and membrane diffusion.

3D flow field estimation and assessment for live cell fluorescence microscopy.

MOTIVATION: The revolution in light sheet microscopy enables the concurrent observation of thousands of dynamic processes, from single molecules to cellular organelles, with high spatiotemporal resolution. However, challenges in the interpretation of multidimensional data requires the fully automatic measurement of those motions to link local processes to cellular functions. This includes the design and the implementation of image processing pipelines able to deal with diverse motion types, and 3D visualization tools adapted to the human visual system. RESULTS: Here, we describe a new method for 3D motion estimation that addresses the aforementioned issues. We integrate 3D matching and variational approach to handle a diverse range of motion without any prior on the shape of moving objects. We compare different similarity measures to cope with intensity ambiguities and demonstrate the effectiveness of the Census signature for both stages. Additionally, we present two intuitive visualization approaches to adapt complex 3D measures into an interpretable 2D view, and a novel way to assess the quality of flow estimates in absence of ground truth. AVAILABILITY AND IMPLEMENTATION: https://team.inria.fr/serpico/data/3d-optical-flow-data/. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.