A novel deep learning-based 3D cell segmentation framework for future image-based disease detection.

Cell segmentation plays a crucial role in understanding, diagnosing, and treating diseases. Despite the recent success of deep learning-based cell segmentation methods, it remains challenging to accurately segment densely packed cells in 3D cell membrane images. Existing approaches also require fine-tuning multiple manually selected hyperparameters on the new datasets. We develop a deep learning-based 3D cell segmentation pipeline, 3DCellSeg, to address these challenges. Compared to the existing methods, our approach carries the following novelties: (1) a robust two-stage pipeline, requiring only one hyperparameter; (2) a light-weight deep convolutional neural network (3DCellSegNet) to efficiently output voxel-wise masks; (3) a custom loss function (3DCellSeg Loss) to tackle the clumped cell problem; and (4) an efficient touching area-based clustering algorithm (TASCAN) to separate 3D cells from the foreground masks. Cell segmentation experiments conducted on four different cell datasets show that 3DCellSeg outperforms the baseline models on the ATAS (plant), HMS (animal), and LRP (plant) datasets with an overall accuracy of 95.6%, 76.4%, and 74.7%, respectively, while achieving an accuracy comparable to the baselines on the Ovules (plant) dataset with an overall accuracy of 82.2%. Ablation studies show that the individual improvements in accuracy is attributable to 3DCellSegNet, 3DCellSeg Loss, and TASCAN, with the 3DCellSeg demonstrating robustness across different datasets and cell shapes. Our results suggest that 3DCellSeg can serve a powerful biomedical and clinical tool, such as histo-pathological image analysis, for cancer diagnosis and grading.

Size control of the inner ear via hydraulic feedback.

Animals make organs of precise size, shape, and symmetry but how developing embryos do this is largely unknown. Here, we combine quantitative imaging, physical theory, and physiological measurement of hydrostatic pressure and fluid transport in zebrafish to study size control of the developing inner ear. We find that fluid accumulation creates hydrostatic pressure in the lumen leading to stress in the epithelium and expansion of the otic vesicle. Pressure, in turn, inhibits fluid transport into the lumen. This negative feedback loop between pressure and transport allows the otic vesicle to change growth rate to control natural or experimentally-induced size variation. Spatiotemporal patterning of contractility modulates pressure-driven strain for regional tissue thinning. Our work connects molecular-driven mechanisms, such as osmotic pressure driven strain and actomyosin tension, to the regulation of tissue morphogenesis via hydraulic feedback to ensure robust control of organ size. Editorial note: This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (see decision letter).

Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution.

Optical and electron microscopy have made tremendous inroads toward understanding the complexity of the brain. However, optical microscopy offers insufficient resolution to reveal subcellular details, and electron microscopy lacks the throughput and molecular contrast to visualize specific molecular constituents over millimeter-scale or larger dimensions. We combined expansion microscopy and lattice light-sheet microscopy to image the nanoscale spatial relationships between proteins across the thickness of the mouse cortex or the entire Drosophila brain. These included synaptic proteins at dendritic spines, myelination along axons, and presynaptic densities at dopaminergic neurons in every fly brain region. The technology should enable statistically rich, large-scale studies of neural development, sexual dimorphism, degree of stereotypy, and structural correlations to behavior or neural activity, all with molecular contrast.

Lamellar projections in the endolymphatic sac act as a relief valve to regulate inner ear pressure.

The inner ear is a fluid-filled closed-epithelial structure whose function requires maintenance of an internal hydrostatic pressure and fluid composition. The endolymphatic sac (ES) is a dead-end epithelial tube connected to the inner ear whose function is unclear. ES defects can cause distended ear tissue, a pathology often seen in hearing and balance disorders. Using live imaging of zebrafish larvae, we reveal that the ES undergoes cycles of slow pressure-driven inflation followed by rapid deflation. Absence of these cycles in lmx1bb mutants leads to distended ear tissue. Using serial-section electron microscopy and adaptive optics lattice light-sheet microscopy, we find a pressure relief valve in the ES comprised of partially separated apical junctions and dynamic overlapping basal lamellae that separate under pressure to release fluid. We propose that this lmx1-dependent pressure relief valve is required to maintain fluid homeostasis in the inner ear and other fluid-filled cavities.

Feedback between tissue packing and neurogenesis in the zebrafish neural tube.

Balancing the rate of differentiation and proliferation in developing tissues is essential to produce organs of robust size and composition. Although many molecular regulators have been established, how these connect to physical and geometrical aspects of tissue architecture is poorly understood. Here, using high-resolution timelapse imaging, we find that changes to cell geometry associated with dense tissue packing play a significant role in regulating differentiation rate in the zebrafish neural tube. Specifically, progenitors that are displaced away from the apical surface due to crowding, tend to differentiate in a Notch-dependent manner. Using simulations we show that interplay between progenitor density, cell shape and changes in differentiation rate could naturally result in negative-feedback control on progenitor cell number. Given these results, we suggest a model whereby differentiation rate is regulated by density dependent effects on cell geometry to: (1) correct variability in cell number; and (2) balance the rates of proliferation and differentiation over development to 'fill' the available space.

Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms.

True physiological imaging of subcellular dynamics requires studying cells within their parent organisms, where all the environmental cues that drive gene expression, and hence the phenotypes that we actually observe, are present. A complete understanding also requires volumetric imaging of the cell and its surroundings at high spatiotemporal resolution, without inducing undue stress on either. We combined lattice light-sheet microscopy with adaptive optics to achieve, across large multicellular volumes, noninvasive aberration-free imaging of subcellular processes, including endocytosis, organelle remodeling during mitosis, and the migration of axons, immune cells, and metastatic cancer cells in vivo. The technology reveals the phenotypic diversity within cells across different organisms and developmental stages and may offer insights into how cells harness their intrinsic variability to adapt to different physiological environments.

Recovery of shape and size in a developing organ pair.

BACKGROUND: Paired organs in animals are largely bilaterally symmetric despite inherent noise in most biological processes. How is precise organ shape and size achieved during development despite this noise? Examining paired organ development is a challenge because it requires repeated quantification of two structures in parallel within living embryos. Here we combine bilateral quantification of morphology through time with asymmetric perturbations to study regulation of organ shape, size, and symmetry in developing organ pairs. RESULTS: We present quantitative live imaging tools to measure the shape and size of the developing inner ears on both the left and right side simultaneously over time. By quantifying variation between the left and right inner ear (intrinsic noise) and between different individuals (extrinsic noise), we find that initial variability decreases over time in normal development to achieve symmetry. Early asymmetry is increased by environmental stress, but symmetry is still recovered over subsequent developmental time. Using multiple unilateral perturbations including Fgf signaling and ultraviolet light, we find that growth can be adjusted to compensate for a range of initial size and shape differences. CONCLUSIONS: We propose that symmetry in developmental systems does not emerge through precise deterministic bilateral development, but rather through feedback mechanisms that adjust morphogenesis rates to account for variation. Developmental Dynamics 246:451-465, 2016. © 2017 Wiley Periodicals, Inc.

Improved Long-Term Imaging of Embryos with Genetically Encoded α-Bungarotoxin.

Rapid advances in microscopy and genetic labeling strategies have created new opportunities for time-lapse imaging of embryonic development. However, methods for immobilizing embryos for long periods while maintaining normal development have changed little. In zebrafish, current immobilization techniques rely on the anesthetic tricaine. Unfortunately, prolonged tricaine treatment at concentrations high enough to immobilize the embryo produces undesirable side effects on development. We evaluate three alternative immobilization strategies: combinatorial soaking in tricaine and isoeugenol, injection of α-bungarotoxin protein, and injection of α-bungarotoxin mRNA. We find evidence for co-operation between tricaine and isoeugenol to give immobility with improved health. However, even in combination these anesthetics negatively affect long-term development. α-bungarotoxin is a small protein from snake venom that irreversibly binds and inactivates acetylcholine receptors. We find that α-bungarotoxin either as purified protein from snakes or endogenously expressed in zebrafish from a codon-optimized synthetic gene can immobilize embryos for extended periods of time with few health effects or developmental delays. Using α-bungarotoxin mRNA injection we obtain complete movies of zebrafish embryogenesis from the 1-cell stage to 3 days post fertilization, with normal health and no twitching. These results demonstrate that endogenously expressed α-bungarotoxin provides unprecedented immobility and health for time-lapse microscopy.

Interplay of cell shape and division orientation promotes robust morphogenesis of developing epithelia.

Epithelial cells acquire functionally important shapes (e.g., squamous, cuboidal, columnar) during development. Here, we combine theory, quantitative imaging, and perturbations to analyze how tissue geometry, cell divisions, and mechanics interact to shape the presumptive enveloping layer (pre-EVL) on the zebrafish embryonic surface. We find that, under geometrical constraints, pre-EVL flattening is regulated by surface cell number changes following differentially oriented cell divisions. The division pattern is, in turn, determined by the cell shape distribution, which forms under geometrical constraints by cell-cell mechanical coupling. An integrated mathematical model of this shape-division feedback loop recapitulates empirical observations. Surprisingly, the model predicts that cell shape is robust to changes of tissue surface area, cell volume, and cell number, which we confirm in vivo. Further simulations and perturbations suggest the parameter linking cell shape and division orientation contributes to epithelial diversity. Together, our work identifies an evolvable design logic that enables robust cell-level regulation of tissue-level development.

Specified neural progenitors sort to form sharp domains after noisy Shh signaling.

Sharply delineated domains of cell types arise in developing tissues under instruction of inductive signal (morphogen) gradients, which specify distinct cell fates at different signal levels. The translation of a morphogen gradient into discrete spatial domains relies on precise signal responses at stable cell positions. However, cells in developing tissues undergoing morphogenesis and proliferation often experience complex movements, which may affect their morphogen exposure, specification, and positioning. How is a clear pattern achieved with cells moving around? Using in toto imaging of the zebrafish neural tube, we analyzed specification patterns and movement trajectories of neural progenitors. We found that specified progenitors of different fates are spatially mixed following heterogeneous Sonic Hedgehog signaling responses. Cell sorting then rearranges them into sharply bordered domains. Ectopically induced motor neuron progenitors also robustly sort to correct locations. Our results reveal that cell sorting acts to correct imprecision of spatial patterning by noisy inductive signals.

An efficient, scalable, and adaptable framework for solving generic systems of level-set PDEs.

In the last decade, level-set methods have been actively developed for applications in image registration, segmentation, tracking, and reconstruction. However, the development of a wide variety of level-set PDEs and their numerical discretization schemes, coupled with hybrid combinations of PDE terms, stopping criteria, and reinitialization strategies, has created a software logistics problem. In the absence of an integrative design, current toolkits support only specific types of level-set implementations which restrict future algorithm development since extensions require significant code duplication and effort. In the new NIH/NLM Insight Toolkit (ITK) v4 architecture, we implemented a level-set software design that is flexible to different numerical (continuous, discrete, and sparse) and grid representations (point, mesh, and image-based). Given that a generic PDE is a summation of different terms, we used a set of linked containers to which level-set terms can be added or deleted at any point in the evolution process. This container-based approach allows the user to explore and customize terms in the level-set equation at compile-time in a flexible manner. The framework is optimized so that repeated computations of common intensity functions (e.g., gradient and Hessians) across multiple terms is eliminated. The framework further enables the evolution of multiple level-sets for multi-object segmentation and processing of large datasets. For doing so, we restrict level-set domains to subsets of the image domain and use multithreading strategies to process groups of subdomains or level-set functions. Users can also select from a variety of reinitialization policies and stopping criteria. Finally, we developed a visualization framework that shows the evolution of a level-set in real-time to help guide algorithm development and parameter optimization. We demonstrate the power of our new framework using confocal microscopy images of cells in a developing zebrafish embryo.

ACME: automated cell morphology extractor for comprehensive reconstruction of cell membranes.

The quantification of cell shape, cell migration, and cell rearrangements is important for addressing classical questions in developmental biology such as patterning and tissue morphogenesis. Time-lapse microscopic imaging of transgenic embryos expressing fluorescent reporters is the method of choice for tracking morphogenetic changes and establishing cell lineages and fate maps in vivo. However, the manual steps involved in curating thousands of putative cell segmentations have been a major bottleneck in the application of these technologies especially for cell membranes. Segmentation of cell membranes while more difficult than nuclear segmentation is necessary for quantifying the relations between changes in cell morphology and morphogenesis. We present a novel and fully automated method to first reconstruct membrane signals and then segment out cells from 3D membrane images even in dense tissues. The approach has three stages: 1) detection of local membrane planes, 2) voting to fill structural gaps, and 3) region segmentation. We demonstrate the superior performance of the algorithms quantitatively on time-lapse confocal and two-photon images of zebrafish neuroectoderm and paraxial mesoderm by comparing its results with those derived from human inspection. We also compared with synthetic microscopic images generated by simulating the process of imaging with fluorescent reporters under varying conditions of noise. Both the over-segmentation and under-segmentation percentages of our method are around 5%. The volume overlap of individual cells, compared to expert manual segmentation, is consistently over 84%. By using our software (ACME) to study somite formation, we were able to segment touching cells with high accuracy and reliably quantify changes in morphogenetic parameters such as cell shape and size, and the arrangement of epithelial and mesenchymal cells. Our software has been developed and tested on Windows, Mac, and Linux platforms and is available publicly under an open source BSD license (https://github.com/krm15/ACME).