Sensitivity-Aware Density Estimation in Multiple Dimensions.

We formulate an optimization problem to estimate probability densities in the context of multidimensional problems that are sampled with uneven probability. It considers detector sensitivity as an heterogeneous density and takes advantage of the computational speed and flexible boundary conditions offered by splines on a grid. We choose to regularize the Hessian of the spline via the nuclear norm to promote sparsity. As a result, the method is spatially adaptive and stable against the choice of the regularization parameter, which plays the role of the bandwidth. We test our computational pipeline on standard densities and provide software.We also present a new approach to PET rebinning as an application of our framework.

Coupled Splines for Sparse Curve Fitting.

We formulate as an inverse problem the construction of sparse parametric continuous curve models that fit a sequence of contour points. Our prior is incorporated as a regularization term that encourages rotation invariance and sparsity. We prove that an optimal solution to the inverse problem is a closed curve with spline components. We then show how to efficiently solve the task using B-splines as basis functions. We extend our problem formulation to curves made of two distinct components with complementary smoothness properties and solve it using hybrid splines. We illustrate the performance of our model on contours of different smoothness. Our experimental results show that we can faithfully reconstruct any general contour using few parameters, even in the presence of imprecisions in the measurements.

Artifacts in optical projection tomography due to refractive-index mismatch: model and correction.

Optical projection tomography (OPT) is a powerful tool for three-dimensional (3D) imaging of mesoscopic samples. While it is able to achieve resolution of a few tens of microns over a sample volume of several cubic centimeters, the reconstructed images often suffer from artifacts caused by inaccurate calibration. In this work, we focus on the refractive-index mismatch between the sample and the surrounding medium. We derive a 3D cone-beam forward model of OPT that approximates the effect of refractive-index mismatch. We then implement a fast and efficient reconstruction method to correct for the induced seagull-shaped artifacts on experimental images of fluorescent beads.

Steer'n'Detect: fast 2D template detection with accurate orientation estimation.

MOTIVATION: Rotated template matching is an efficient and versatile algorithm to analyze microscopy images, as it automates the detection of stereotypical structures, such as organelles that can appear at any orientation. Its performance however quickly degrades in noisy image data. RESULTS: We introduce Steer'n'Detect, an ImageJ plugin implementing a recently published algorithm to detect patterns of interest at any orientation with high accuracy from a single template in 2D images. Steer'n'Detect provides a faster and more robust substitute to template matching. By adapting to the statistics of the image background, it guarantees accurate results even in the presence of noise. The plugin comes with an intuitive user interface facilitating results analysis and further post-processing. AVAILABILITY AND IMPLEMENTATION: https://github.com/Biomedical-Imaging-Group/Steer-n-Detect. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

A constrained method for lensless coherent imaging of thin samples.

Lensless inline holography can produce high-resolution images over a large field of view (FoV). In a previous work [Appl. Opt.60, B38 (2021)APOPAI0003-693510.1364/AO.414976], we showed that (i) the actual FoV can be extrapolated outside of the camera FoV and (ii) the effective resolution of the setup can be several times higher than the resolution of the camera. In this paper, we present a reconstruction method to recover high resolution with an extrapolated FoV image of the phase and the amplitude of a sample from aliased intensity measurements taken at a lower resolution.

DeepImageJ: A user-friendly environment to run deep learning models in ImageJ.

DeepImageJ is a user-friendly solution that enables the generic use of pre-trained deep learning models for biomedical image analysis in ImageJ. The deepImageJ environment gives access to the largest bioimage repository of pre-trained deep learning models (BioImage Model Zoo). Hence, nonexperts can easily perform common image processing tasks in life-science research with deep learning-based tools including pixel and object classification, instance segmentation, denoising or virtual staining. DeepImageJ is compatible with existing state of the art solutions and it is equipped with utility tools for developers to include new models. Very recently, several training frameworks have adopted the deepImageJ format to deploy their work in one of the most used softwares in the field (ImageJ). Beyond its direct use, we expect deepImageJ to contribute to the broader dissemination and reuse of deep learning models in life sciences applications and bioimage informatics.

Robust Phase Unwrapping via Deep Image Prior for Quantitative Phase Imaging.

Quantitative phase imaging (QPI) is an emerging label-free technique that produces images containing morphological and dynamical information without contrast agents. Unfortunately, the phase is wrapped in most imaging system. Phase unwrapping is the computational process that recovers a more informative image. It is particularly challenging with thick and complex samples such as organoids. Recent works that rely on supervised training show that deep learning is a powerful method to unwrap the phase; however, supervised approaches require large and representative datasets which are difficult to obtain for complex biological samples. Inspired by the concept of deep image priors, we propose a deep-learning-based method that does not need any training set. Our framework relies on an untrained convolutional neural network to accurately unwrap the phase while ensuring the consistency of the measurements. We experimentally demonstrate that the proposed method faithfully recovers the phase of complex samples on both real and simulated data. Our work paves the way to reliable phase imaging of thick and complex samples with QPI.

Active Subdivision Surfaces for the Semiautomatic Segmentation of Biomedical Volumes.

We present a new family of active surfaces for the semiautomatic segmentation of volumetric objects in 3D biomedical images. We represent our deformable model by a subdivision surface encoded by a small set of control points and generated through a geometric refinement process. The subdivision operator confers important properties to the surface such as smoothness, reproduction of desirable shapes and interpolation of the control points. We deform the subdivision surface through the minimization of suitable gradient-based and region-based energy terms that we have designed for that purpose. In addition, we provide an easy way to combine these energies with convolutional neural networks. Our active subdivision surface satisfies the property of multiresolution, which allows us to adopt a coarse-to-fine optimization strategy. This speeds up the computations and decreases its dependence on initialization compared to singleresolution active surfaces. Performance evaluations on both synthetic and real biomedical data show that our active subdivision surface is robust in the presence of noise and outperforms current state-of-the-art methods. In addition, we provide a software that gives full control over the active subdivision surface via an intuitive manipulation of the control points.

Time-Dependent Deep Image Prior for Dynamic MRI.

We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. We introduce a generalized version of the deep-image-prior approach, which optimizes the weights of a reconstruction network to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional data. In particular, for cardiac images, it does not require the marking of heartbeats or the reordering of spokes. The key ingredients of our method are threefold: 1) a fixed low-dimensional manifold that encodes the temporal variations of images; 2) a network that maps the manifold into a more expressive latent space; and 3) a convolutional neural network that generates a dynamic series of MRI images from the latent variables and that favors their consistency with the measurements in k -space. Our method outperforms the state-of-the-art methods quantitatively and qualitatively in both retrospective and real fetal cardiac datasets. To the best of our knowledge, this is the first unsupervised deep-learning-based method that can reconstruct the continuous variation of dynamic MRI sequences with high spatial resolution.

Deep Learning Enables Individual Xenograft Cell Classification in Histological Images by Analysis of Contextual Features.

Patient-Derived Xenografts (PDXs) are the preclinical models which best recapitulate inter- and intra-patient complexity of human breast malignancies, and are also emerging as useful tools to study the normal breast epithelium. However, data analysis generated with such models is often confounded by the presence of host cells and can give rise to data misinterpretation. For instance, it is important to discriminate between xenografted and host cells in histological sections prior to performing immunostainings. We developed Single Cell Classifier (SCC), a data-driven deep learning-based computational tool that provides an innovative approach for automated cell species discrimination based on a multi-step process entailing nuclei segmentation and single cell classification. We show that human and murine cell contextual features, more than cell-intrinsic ones, can be exploited to discriminate between cell species in both normal and malignant tissues, yielding up to 96% classification accuracy. SCC will facilitate the interpretation of H&E- and DAPI-stained histological sections of xenografted human-in-mouse tissues and it is open to new in-house built models for further applications. SCC is released as an open-source plugin in ImageJ/Fiji available at the following link: https://github.com/Biomedical-Imaging-Group/SingleCellClassifier .

Principled Design and Implementation of Steerable Detectors.

We provide a complete pipeline for the detection of patterns of interest in an image. In our approach, the patterns are assumed to be adequately modeled by a known template, and are located at unknown positions and orientations that we aim at retrieving. We propose a continuous-domain additive image model, where the analyzed image is the sum of the patterns to localize and a background with self-similar isotropic power-spectrum. We are then able to compute the optimal filter fulfilling the SNR criterion based on one single template and background pair: it strongly responds to the template while being optimally decoupled from the background model. In addition, we constrain our filter to be steerable, which allows for a fast template detection together with orientation estimation. In practice, the implementation requires to discretize a continuous-domain formulation on polar grids, which is performed using quadratic radial B-splines. We demonstrate the practical usefulness of our method on a variety of template approximation and pattern detection experiments. We show that the detection performance drastically improves when we exploit the statistics of the background via its power-spectrum decay, which we refer to as spectral-shaping. The proposed scheme outperforms state-of-the-art steerable methods by up to 50% of absolute detection performance.

Joint Angular Refinement and Reconstruction for Single-Particle Cryo-EM.

Single-particle cryo-electron microscopy (cryo-EM) reconstructs the three-dimensional (3D) structure of biomolecules from a large set of 2D projection images with random and unknown orientations. A crucial step in the single-particle cryo-EM pipeline is 3D refinement, which resolves a highresolution 3D structure from an initial approximate volume by refining the estimation of the orientation of each projection. In this work, we propose a new approach that refines the projection angles on the continuum. We formulate the optimization problem over the density map and the orientations jointly. The density map is updated using the efficient alternating-direction method of multipliers, while the orientations are updated through a semicoordinate- wise gradient descent for which we provide an explicit derivation of the gradient. Our method eliminates the requirement for a fine discretization of the orientation space and does away with the classical but computationally expensive templatematching step. Numerical results demonstrate the feasibility and performance of our approach compared to several baselines.

Support and approximation properties of Hermite splines.

In this paper, we formally investigate two mathematical aspects of Hermite splines that are relevant to practical applications. We first demonstrate that Hermite splines are maximally localized, in the sense that the size of their support is minimal among pairs of functions with identical reproduction properties. Then, we precisely quantify the approximation power of Hermite splines for the reconstruction of functions and their derivatives. It is known that the Hermite and B-spline approximation schemes have the same approximation order. More precisely, their approximation error vanishes as O ( T 4 ) when the step size T goes to zero. In this work, we show that they actually have the same asymptotic approximation error constants, too. Therefore, they have identical asymptotic approximation properties. Hermite splines combine optimal localization and excellent approximation power, while retaining interpolation properties and closed-form expression, in contrast to existing similar functions. These findings shed a new light on the convenience of Hermite splines in the context of computer graphics and geometrical design.

Deep-learning projector for optical diffraction tomography.

Optical diffraction tomography is an effective tool to estimate the refractive indices of unknown objects. It proceeds by solving an ill-posed inverse problem for which the wave equation governs the scattering events. The solution has traditionally been derived by the minimization of an objective function in which the data-fidelity term encourages measurement consistency while the regularization term enforces prior constraints. In this work, we propose to train a convolutional neural network (CNN) as the projector in a projected-gradient-descent method. We iteratively produce high-quality estimates and ensure measurement consistency, thus keeping the best of CNN-based and regularization-based worlds. Our experiments on two-dimensional-simulated and real data show an improvement over other conventional or deep-learning-based methods. Furthermore, our trained CNN projector is general enough to accommodate various forward models for the handling of multiple-scattering events.

Publisher Correction: Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software.

In the version of this paper originally published, Figure 4a contained errors that were introduced during typesetting. The bottom 11° ThunderSTORM image is an xz view but was incorrectly labeled as xy, and the low x-axis value in the four line profiles was incorrectly set as -60 instead of -50. These errors have been corrected in the PDF and HTML versions of the paper.

Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software.

With the widespread uptake of two-dimensional (2D) and three-dimensional (3D) single-molecule localization microscopy (SMLM), a large set of different data analysis packages have been developed to generate super-resolution images. In a large community effort, we designed a competition to extensively characterize and rank the performance of 2D and 3D SMLM software packages. We generated realistic simulated datasets for popular imaging modalities-2D, astigmatic 3D, biplane 3D and double-helix 3D-and evaluated 36 participant packages against these data. This provides the first broad assessment of 3D SMLM software and provides a holistic view of how the latest 2D and 3D SMLM packages perform in realistic conditions. This resource allows researchers to identify optimal analytical software for their experiments, allows 3D SMLM software developers to benchmark new software against the current state of the art, and provides insight into the current limits of the field.

Imaging cellular ultrastructures using expansion microscopy (U-ExM).

Determining the structure and composition of macromolecular assemblies is a major challenge in biology. Here we describe ultrastructure expansion microscopy (U-ExM), an extension of expansion microscopy that allows the visualization of preserved ultrastructures by optical microscopy. This method allows for near-native expansion of diverse structures in vitro and in cells; when combined with super-resolution microscopy, it unveiled details of ultrastructural organization, such as centriolar chirality, that could otherwise be observed only by electron microscopy.

Multimodal imaging and high-throughput image-processing for drug screening on living organisms on-chip.

A major step for the validation of medical drugs is the screening on whole organisms, which gives the systemic information that is missing when using cellular models. Caenorhabditis elegans is a soil worm that catches the interest of researchers who study systemic physiopathology (e.g., metabolic and neurodegenerative diseases) because: (1) its large genetic homology with humans supports translational analysis; (2) worms are much easier to handle and grow in large amounts compared with rodents, for which (3) the costs and (4) the ethical concerns are substantial. Here, we demonstrate how multimodal optical imaging on such an organism can provide high-content information relevant to the drug development pipeline (e.g., mode-of-action identification, dose-response analysis), especially when combined with on-chip multiplexing capability. After designing a microfluidic array to select small separated populations of C. elegans, we combine fluorescence and bright-field imaging along with high-throughput feature recognition and signal detection to enable the identification of the mode-of-action of an antibiotic. For this purpose, we use a genetically encoded fluorescence reporter of mitochondrial stress, which we studied in living specimens during their entire development. Furthermore, we demonstrate real-time, very large field-of-view capability on multiplexed motility assays for the assessment of the dose-response relation of an anesthetic.

Imaging neural activity in the ventral nerve cord of behaving adult Drosophila.

To understand neural circuits that control limbs, one must measure their activity during behavior. Until now this goal has been challenging, because limb premotor and motor circuits have been largely inaccessible for large-scale recordings in intact, moving animals-a constraint that is true for both vertebrate and invertebrate models. Here, we introduce a method for 2-photon functional imaging from the ventral nerve cord (VNC) of behaving adult Drosophila melanogaster. We use this method to reveal patterns of activity across nerve cord populations during grooming and walking and to uncover the functional encoding of moonwalker ascending neurons (MANs), moonwalker descending neurons (MDNs), and a previously uncharacterized class of locomotion-associated A1 descending neurons. Finally, we develop a genetic reagent to destroy the indirect flight muscles and to facilitate experimental access to the VNC. Taken together, these approaches enable the direct investigation of circuits associated with complex limb movements.

Fast multiscale reconstruction for Cryo-EM.

We present a multiscale reconstruction framework for single-particle analysis (SPA). The representation of three-dimensional (3D) objects with scaled basis functions permits the reconstruction of volumes at any desired scale in the real-space. This multiscale approach generates interesting opportunities in SPA for the stabilization of the initial volume problem or the 3D iterative refinement procedure. In particular, we show that reconstructions performed at coarse scale are more robust to angular errors and permit gains in computational speed. A key component of the proposed iterative scheme is its fast implementation. The costly step of reconstruction, which was previously hindering the use of advanced iterative methods in SPA, is formulated as a discrete convolution with a cost that does not depend on the number of projection directions. The inclusion of the contrast transfer function inside the imaging matrix is also done at no extra computational cost. By permitting full 3D regularization, the framework is by itself a robust alternative to direct methods for performing reconstruction in adverse imaging conditions (e.g., heavy noise, large angular misassignments, low number of projections). We present reconstructions obtained at different scales from a dataset of the 2015/2016 EMDataBank Map Challenge. The algorithm has been implemented in the Scipion package.