In Toto Imaging and Reconstruction of Post-Implantation Mouse Development at the Single-Cell Level.

The mouse embryo has long been central to the study of mammalian development; however, elucidating the cell behaviors governing gastrulation and the formation of tissues and organs remains a fundamental challenge. A major obstacle is the lack of live imaging and image analysis technologies capable of systematically following cellular dynamics across the developing embryo. We developed a light-sheet microscope that adapts itself to the dramatic changes in size, shape, and optical properties of the post-implantation mouse embryo and captures its development from gastrulation to early organogenesis at the cellular level. We furthermore developed a computational framework for reconstructing long-term cell tracks, cell divisions, dynamic fate maps, and maps of tissue morphogenesis across the entire embryo. By jointly analyzing cellular dynamics in multiple embryos registered in space and time, we built a dynamic atlas of post-implantation mouse development that, together with our microscopy and computational methods, is provided as a resource. VIDEO ABSTRACT.

A Preferred Curvature-Based Continuum Mechanics Framework for Modeling Embryogenesis.

Mechanics plays a key role in the development of higher organisms. However, understanding this relationship is complicated by the difficulty of modeling the link between local forces generated at the subcellular level and deformations observed at the tissue and whole-embryo levels. Here we propose an approach first developed for lipid bilayers and cell membranes, in which force-generation by cytoskeletal elements enters a continuum mechanics formulation for the full system in the form of local changes in preferred curvature. This allows us to express and solve the system using only tissue strains. Locations of preferred curvature are simply related to products of gene expression. A solution, in that context, means relaxing the system's mechanical energy to yield global morphogenetic predictions that accommodate a tendency toward the local preferred curvature, without a need to explicitly model force-generation mechanisms at the molecular level. Our computational framework, which we call SPHARM-MECH, extends a 3D spherical harmonics parameterization known as SPHARM to combine this level of abstraction with a sparse shape representation. The integration of these two principles allows computer simulations to be performed in three dimensions on highly complex shapes, gene expression patterns, and mechanical constraints. We demonstrate our approach by modeling mesoderm invagination in the fruit-fly embryo, where local forces generated by the acto-myosin meshwork in the region of the future mesoderm lead to formation of a ventral tissue fold. The process is accompanied by substantial changes in cell shape and long-range cell movements. Applying SPHARM-MECH to whole-embryo live imaging data acquired with light-sheet microscopy reveals significant correlation between calculated and observed tissue movements. Our analysis predicts the observed cell shape anisotropy on the ventral side of the embryo and suggests an active mechanical role of mesoderm invagination in supporting the onset of germ-band extension.

Whole-animal functional and developmental imaging with isotropic spatial resolution.

Imaging fast cellular dynamics across large specimens requires high resolution in all dimensions, high imaging speeds, good physical coverage and low photo-damage. To meet these requirements, we developed isotropic multiview (IsoView) light-sheet microscopy, which rapidly images large specimens via simultaneous light-sheet illumination and fluorescence detection along four orthogonal directions. Combining these four views by means of high-throughput multiview deconvolution yields images with high resolution in all three dimensions. We demonstrate whole-animal functional imaging of Drosophila larvae at a spatial resolution of 1.1-2.5 µm and temporal resolution of 2 Hz for several hours. We also present spatially isotropic whole-brain functional imaging in Danio rerio larvae and spatially isotropic multicolor imaging of fast cellular dynamics across gastrulating Drosophila embryos. Compared with conventional light-sheet microscopy, IsoView microscopy improves spatial resolution at least sevenfold and decreases resolution anisotropy at least threefold. Compared with existing high-resolution light-sheet techniques, IsoView microscopy effectively doubles the penetration depth and provides subsecond temporal resolution for specimens 400-fold larger than could previously be imaged.

Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data.

The comprehensive reconstruction of cell lineages in complex multicellular organisms is a central goal of developmental biology. We present an open-source computational framework for the segmentation and tracking of cell nuclei with high accuracy and speed. We demonstrate its (i) generality by reconstructing cell lineages in four-dimensional, terabyte-sized image data sets of fruit fly, zebrafish and mouse embryos acquired with three types of fluorescence microscopes, (ii) scalability by analyzing advanced stages of development with up to 20,000 cells per time point at 26,000 cells min(-1) on a single computer workstation and (iii) ease of use by adjusting only two parameters across all data sets and providing visualization and editing tools for efficient data curation. Our approach achieves on average 97.0% linkage accuracy across all species and imaging modalities. Using our system, we performed the first cell lineage reconstruction of early Drosophila melanogaster nervous system development, revealing neuroblast dynamics throughout an entire embryo.

Efficient Bayesian-based multiview deconvolution.

Light-sheet fluorescence microscopy is able to image large specimens with high resolution by capturing the samples from multiple angles. Multiview deconvolution can substantially improve the resolution and contrast of the images, but its application has been limited owing to the large size of the data sets. Here we present a Bayesian-based derivation of multiview deconvolution that drastically improves the convergence time, and we provide a fast implementation using graphics hardware.

Cortical representations of olfactory input by trans-synaptic tracing.

In the mouse, each class of olfactory receptor neurons expressing a given odorant receptor has convergent axonal projections to two specific glomeruli in the olfactory bulb, thereby creating an odour map. However, it is unclear how this map is represented in the olfactory cortex. Here we combine rabies-virus-dependent retrograde mono-trans-synaptic labelling with genetics to control the location, number and type of 'starter' cortical neurons, from which we trace their presynaptic neurons. We find that individual cortical neurons receive input from multiple mitral cells representing broadly distributed glomeruli. Different cortical areas represent the olfactory bulb input differently. For example, the cortical amygdala preferentially receives dorsal olfactory bulb input, whereas the piriform cortex samples the whole olfactory bulb without obvious bias. These differences probably reflect different functions of these cortical areas in mediating innate odour preference or associative memory. The trans-synaptic labelling method described here should be widely applicable to mapping connections throughout the mouse nervous system.

Quantitative high-speed imaging of entire developing embryos with simultaneous multiview light-sheet microscopy.

Live imaging of large biological specimens is fundamentally limited by the short optical penetration depth of light microscopes. To maximize physical coverage, we developed the SiMView technology framework for high-speed in vivo imaging, which records multiple views of the specimen simultaneously. SiMView consists of a light-sheet microscope with four synchronized optical arms, real-time electronics for long-term sCMOS-based image acquisition at 175 million voxels per second, and computational modules for high-throughput image registration, segmentation, tracking and real-time management of the terabytes of multiview data recorded per specimen. We developed one-photon and multiphoton SiMView implementations and recorded cellular dynamics in entire Drosophila melanogaster embryos with 30-s temporal resolution throughout development. We furthermore performed high-resolution long-term imaging of the developing nervous system and followed neuroblast cell lineages in vivo. SiMView data sets provide quantitative morphological information even for fast global processes and enable accurate automated cell tracking in the entire early embryo.

Fast and robust optical flow for time-lapse microscopy using super-voxels.

MOTIVATION: Optical flow is a key method used for quantitative motion estimation of biological structures in light microscopy. It has also been used as a key module in segmentation and tracking systems and is considered a mature technology in the field of computer vision. However, most of the research focused on 2D natural images, which are small in size and rich in edges and texture information. In contrast, 3D time-lapse recordings of biological specimens comprise up to several terabytes of image data and often exhibit complex object dynamics as well as blurring due to the point-spread-function of the microscope. Thus, new approaches to optical flow are required to improve performance for such data. RESULTS: We solve optical flow in large 3D time-lapse microscopy datasets by defining a Markov random field (MRF) over super-voxels in the foreground and applying motion smoothness constraints between super-voxels instead of voxel-wise. This model is tailored to the specific characteristics of light microscopy datasets: super-voxels help registration in textureless areas, the MRF over super-voxels efficiently propagates motion information between neighboring cells and the background subtraction and super-voxels reduce the dimensionality of the problem by an order of magnitude. We validate our approach on large 3D time-lapse datasets of Drosophila and zebrafish development by analyzing cell motion patterns. We show that our approach is, on average, 10 × faster than commonly used optical flow implementations in the Insight Tool-Kit (ITK) and reduces the average flow end point error by 50% in regions with complex dynamic processes, such as cell divisions. AVAILABILITY: Source code freely available in the Software section at http://janelia.org/lab/keller-lab.

Towards comprehensive cell lineage reconstructions in complex organisms using light-sheet microscopy.

Understanding the development of complex multicellular organisms as a function of the underlying cell behavior is one of the most fundamental goals of developmental biology. The ability to quantitatively follow cell dynamics in entire developing embryos is an indispensable step towards such a system-level understanding. In recent years, light-sheet fluorescence microscopy has emerged as a particularly promising strategy for recording the in vivo data required to realize this goal. Using light-sheet fluorescence microscopy, entire complex organisms can be rapidly imaged in three dimensions at sub-cellular resolution, achieving high temporal sampling and excellent signal-to-noise ratio without damaging the living specimen or bleaching fluorescent markers. The resulting datasets allow following individual cells in vertebrate and higher invertebrate embryos over up to several days of development. However, the complexity and size of these multi-terabyte recordings typically preclude comprehensive manual analyses. Thus, new computational approaches are required to automatically segment cell morphologies, accurately track cell identities and systematically analyze cell behavior throughout embryonic development. We review current efforts in light-sheet microscopy and bioimage informatics towards this goal, and argue that comprehensive cell lineage reconstructions are finally within reach for many key model organisms, including fruit fly, zebrafish and mouse.

Analysis of the intact surface layer of Caulobacter crescentus by cryo-electron tomography.

The surface layers (S layers) of those bacteria and archaea that elaborate these crystalline structures have been studied for 40 years. However, most structural analysis has been based on electron microscopy of negatively stained S-layer fragments separated from cells, which can introduce staining artifacts and allow rearrangement of structures prone to self-assemble. We present a quantitative analysis of the structure and organization of the S layer on intact growing cells of the Gram-negative bacterium Caulobacter crescentus using cryo-electron tomography (CET) and statistical image processing. Instead of the expected long-range order, we observed different regions with hexagonally organized subunits exhibiting short-range order and a broad distribution of periodicities. Also, areas of stacked double layers were found, and these increased in extent when the S-layer protein (RsaA) expression level was elevated by addition of multiple rsaA copies. Finally, we combined high-resolution amino acid residue-specific Nanogold labeling and subtomogram averaging of CET volumes to improve our understanding of the correlation between the linear protein sequence and the structure at the 2-nm level of resolution that is presently available. The results support the view that the U-shaped RsaA monomer predicted from negative-stain tomography proceeds from the N terminus at one vertex, corresponding to the axis of 3-fold symmetry, to the C terminus at the opposite vertex, which forms the prominent 6-fold symmetry axis. Such information will help future efforts to analyze subunit interactions and guide selection of internal sites for display of heterologous protein segments.

Subtomogram alignment by adaptive Fourier coefficient thresholding.

In the past few years, three-dimensional (3D) subtomogram alignment has become an important tool in cryo-electron tomography (CET). This technique allows one to produce higher resolution images of structures which can not be reconstructed using single-particle methods. Building on previous work, we present a new dissimilarity measure between subtomograms that works well for the noisy images that often occur in CET images. A technique that is more robust to noise provides the ability to analyze macromolecules in thicker samples such as whole cells or lower the defocus in thinner samples to push the first zero of the Contrast Transfer Function (CTF). Our method, Threshold Constrained Cross-Correlation (TCCC), uses statistics of the noise to automatically select only a small percentage of the Fourier coefficients to compute the cross-correlation, which has two main advantages: first, it reduces the influence of the noise by looking at only those peaks dominated by signal; and second, it avoids the missing wedge normalization problem since we consider the same number of coefficients for all possible pairs of subtomograms. We present results with synthetic and real data to compare our approach with other existing methods under different SNR and missing wedge conditions, and show that TCCC improves alignment results for datasets with SNR<0.1. We have made our source code freely available for the community.

3D segmentation of cell boundaries from whole cell cryogenic electron tomography volumes.

Cryogenic electron tomography (cryo-ET) has gained increasing interest in recent years due to its ability to image whole cells and subcellular structures in 3D at nanometer resolution in their native environment. However, due to dose restrictions and the inability to acquire high tilt angle images, the reconstructed volumes are noisy and have missing information. Thus, features are unreliable, and precision extraction of the cell boundary is difficult, manual and time intensive. This paper presents an efficient recursive algorithm called BLASTED (Boundary Localization using Adaptive Shape and Texture Discovery) to automatically extract the cell boundary using a conditional random field (CRF) framework in which boundary points and shape are jointly inferred. The algorithm learns the texture of the boundary region progressively, and uses a global shape model and shape-dependent features to propose candidate boundary points on a slice of the membrane. It then updates the shape of that slice by accepting the appropriate candidate points using local spatial clustering, the global shape model, and trained boosted texture classifiers. The BLASTED algorithm segmented the cell membrane over an average of 93% of the length of the cell in 19 difficult cryo-ET datasets.

Markov random field based automatic image alignment for electron tomography.

We present a method for automatic full-precision alignment of the images in a tomographic tilt series. Full-precision automatic alignment of cryo electron microscopy images has remained a difficult challenge to date, due to the limited electron dose and low image contrast. These facts lead to poor signal to noise ratio (SNR) in the images, which causes automatic feature trackers to generate errors, even with high contrast gold particles as fiducial features. To enable fully automatic alignment for full-precision reconstructions, we frame the problem probabilistically as finding the most likely particle tracks given a set of noisy images, using contextual information to make the solution more robust to the noise in each image. To solve this maximum likelihood problem, we use Markov Random Fields (MRF) to establish the correspondence of features in alignment and robust optimization for projection model estimation. The resulting algorithm, called Robust Alignment and Projection Estimation for Tomographic Reconstruction, or RAPTOR, has not needed any manual intervention for the difficult datasets we have tried, and has provided sub-pixel alignment that is as good as the manual approach by an expert user. We are able to automatically map complete and partial marker trajectories and thus obtain highly accurate image alignment. Our method has been applied to challenging cryo electron tomographic datasets with low SNR from intact bacterial cells, as well as several plastic section and X-ray datasets.

Real-Time Three-Dimensional Cell Segmentation in Large-Scale Microscopy Data of Developing Embryos.

We present the Real-time Accurate Cell-shape Extractor (RACE), a high-throughput image analysis framework for automated three-dimensional cell segmentation in large-scale images. RACE is 55-330 times faster and 2-5 times more accurate than state-of-the-art methods. We demonstrate the generality of RACE by extracting cell-shape information from entire Drosophila, zebrafish, and mouse embryos imaged with confocal and light-sheet microscopes. Using RACE, we automatically reconstructed cellular-resolution tissue anisotropy maps across developing Drosophila embryos and quantified differences in cell-shape dynamics in wild-type and mutant embryos. We furthermore integrated RACE with our framework for automated cell lineaging and performed joint segmentation and cell tracking in entire Drosophila embryos. RACE processed these terabyte-sized datasets on a single computer within 1.4 days. RACE is easy to use, as it requires adjustment of only three parameters, takes full advantage of state-of-the-art multi-core processors and graphics cards, and is available as open-source software for Windows, Linux, and Mac OS.

Light sheet-based imaging and analysis of early embryogenesis in the fruit fly.

The fruit fly is an excellent model system for investigating the sequence of epithelial tissue invaginations constituting the process of gastrulation. By combining recent advancements in light sheet fluorescence microscopy (LSFM) and image processing, the three-dimensional fly embryo morphology and relevant gene expression patterns can be accurately recorded throughout the entire process of embryogenesis. LSFM provides exceptionally high imaging speed, high signal-to-noise ratio, low level of photoinduced damage, and good optical penetration depth. This powerful combination of capabilities makes LSFM particularly suitable for live imaging of the fly embryo.The resulting high-information-content image data are subsequently processed to obtain the outlines of cells and cell nuclei, as well as the geometry of the whole embryo tissue by image segmentation. Furthermore, morphodynamics information is extracted by computationally tracking objects in the image. Towards that goal we describe the successful implementation of a fast fitting strategy of Gaussian mixture models.The data obtained by image processing is well-suited for hypothesis testing of the detailed biomechanics of the gastrulating embryo. Typically this involves constructing computational mechanics models that consist of an objective function providing an estimate of strain energy for a given morphological configuration of the tissue, and a numerical minimization mechanism of this energy, achieved by varying morphological parameters.In this chapter, we provide an overview of in vivo imaging of fruit fly embryos using LSFM, computational tools suitable for processing the resulting images, and examples of computational biomechanical simulations of fly embryo gastrulation.

Efficient processing and analysis of large-scale light-sheet microscopy data.

Light-sheet microscopy is a powerful method for imaging the development and function of complex biological systems at high spatiotemporal resolution and over long time scales. Such experiments typically generate terabytes of multidimensional image data, and thus they demand efficient computational solutions for data management, processing and analysis. We present protocols and software to tackle these steps, focusing on the imaging-based study of animal development. Our protocols facilitate (i) high-speed lossless data compression and content-based multiview image fusion optimized for multicore CPU architectures, reducing image data size 30-500-fold; (ii) automated large-scale cell tracking and segmentation; and (iii) visualization, editing and annotation of multiterabyte image data and cell-lineage reconstructions with tens of millions of data points. These software modules are open source. They provide high data throughput using a single computer workstation and are readily applicable to a wide spectrum of biological model systems.

Automated reconstruction of neuronal morphology based on local geometrical and global structural models.

Digital reconstruction of neurons from microscope images is an important and challenging problem in neuroscience. In this paper, we propose a model-based method to tackle this problem. We first formulate a model structure, then develop an algorithm for computing it by carefully taking into account morphological characteristics of neurons, as well as the image properties under typical imaging protocols. The method has been tested on the data sets used in the DIADEM competition and produced promising results for four out of the five data sets.

Alignment of cryo-electron tomography datasets.

Data acquisition of cryo-electron tomography (CET) samples described in previous chapters involves relatively imprecise mechanical motions: the tilt series has shifts, rotations, and several other distortions between projections. Alignment is the procedure of correcting for these effects in each image and requires the estimation of a projection model that describes how points from the sample in three-dimensions are projected to generate two-dimensional images. This estimation is enabled by finding corresponding common features between images. This chapter reviews several software packages that perform alignment and reconstruction tasks completely automatically (or with minimal user intervention) in two main scenarios: using gold fiducial markers as high contrast features or using relevant biological structures present in the image (marker-free). In particular, we emphasize the key decision points in the process that users should focus on in order to obtain high-resolution reconstructions.