Toward scalable reuse of vEM data: OME-Zarr to the rescue.

The growing size of EM volumes is a significant barrier to findable, accessible, interoperable, and reusable (FAIR) sharing. Storage, sharing, visualization and processing are challenging for large datasets. Here we discuss a recent development toward the standardized storage of volume electron microscopy (vEM) data which addresses many of the issues that researchers face. The OME-Zarr format splits data into more manageable, performant chunks enabling streaming-based access, and unifies important metadata such as multiresolution pyramid descriptions. The file format is designed for centralized and remote storage (e.g., cloud storage or file system) and is therefore ideal for sharing large data. By coalescing on a common, community-wide format, these benefits will expand as ever more data is made available to the scientific community.

The big warp: Registration of disparate retinal imaging modalities and an example overlay of ultrawide-field photos and en-face OCTA images.

PURPOSE: To develop an algorithm and scripts to combine disparate multimodal imaging modalities and show its use by overlaying en-face optical coherence tomography angiography (OCTA) images and Optos ultra-widefield (UWF) retinal images using the Fiji (ImageJ) plugin BigWarp. METHODS: Optos UWF images and Heidelberg en-face OCTA images were collected from various patients as part of their routine care. En-face OCTA images were generated and ten (10) images at varying retinal depths were exported. The Fiji plugin BigWarp was used to transform the Optos UWF image onto the en-face OCTA image using matching reference points in the retinal vasculature surrounding the macula. The images were then overlayed and stacked to create a series of ten combined Optos UWF and en-face OCTA images of increasing retinal depths. The first algorithm was modified to include two scripts that automatically aligned all the en-face OCTA images. RESULTS: The Optos UWF image could easily be transformed to the en-face OCTA images using BigWarp with common vessel branch point landmarks in the vasculature. The resulting warped Optos image was then successfully superimposed onto the ten Optos UWF images. The scripts more easily allowed for automatic overlay of the images. CONCLUSIONS: Optos UWF images can be successfully superimposed onto en-face OCTA images using freely available software that has been applied to ocular use. This synthesis of multimodal imaging may increase their potential diagnostic value. Script A is publicly available at https://doi.org/10.6084/m9.figshare.16879591.v1 and Script B is available at https://doi.org/10.6084/m9.figshare.17330048.

An unbiased template of the Drosophila brain and ventral nerve cord.

The fruit fly Drosophila melanogaster is an important model organism for neuroscience with a wide array of genetic tools that enable the mapping of individual neurons and neural subtypes. Brain templates are essential for comparative biological studies because they enable analyzing many individuals in a common reference space. Several central brain templates exist for Drosophila, but every one is either biased, uses sub-optimal tissue preparation, is imaged at low resolution, or does not account for artifacts. No publicly available Drosophila ventral nerve cord template currently exists. In this work, we created high-resolution templates of the Drosophila brain and ventral nerve cord using the best-available technologies for imaging, artifact correction, stitching, and template construction using groupwise registration. We evaluated our central brain template against the four most competitive, publicly available brain templates and demonstrate that ours enables more accurate registration with fewer local deformations in shorter time.

Correlative three-dimensional super-resolution and block-face electron microscopy of whole vitreously frozen cells.

Within cells, the spatial compartmentalization of thousands of distinct proteins serves a multitude of diverse biochemical needs. Correlative super-resolution (SR) fluorescence and electron microscopy (EM) can elucidate protein spatial relationships to global ultrastructure, but has suffered from tradeoffs of structure preservation, fluorescence retention, resolution, and field of view. We developed a platform for three-dimensional cryogenic SR and focused ion beam-milled block-face EM across entire vitreously frozen cells. The approach preserves ultrastructure while enabling independent SR and EM workflow optimization. We discovered unexpected protein-ultrastructure relationships in mammalian cells including intranuclear vesicles containing endoplasmic reticulum-associated proteins, web-like adhesions between cultured neurons, and chromatin domains subclassified on the basis of transcriptional activity. Our findings illustrate the value of a comprehensive multimodal view of ultrastructural variability across whole cells.

A Complete Electron Microscopy Volume of the Brain of Adult Drosophila melanogaster.

Drosophila melanogaster has a rich repertoire of innate and learned behaviors. Its 100,000-neuron brain is a large but tractable target for comprehensive neural circuit mapping. Only electron microscopy (EM) enables complete, unbiased mapping of synaptic connectivity; however, the fly brain is too large for conventional EM. We developed a custom high-throughput EM platform and imaged the entire brain of an adult female fly at synaptic resolution. To validate the dataset, we traced brain-spanning circuitry involving the mushroom body (MB), which has been extensively studied for its role in learning. All inputs to Kenyon cells (KCs), the intrinsic neurons of the MB, were mapped, revealing a previously unknown cell type, postsynaptic partners of KC dendrites, and unexpected clustering of olfactory projection neurons. These reconstructions show that this freely available EM volume supports mapping of brain-spanning circuits, which will significantly accelerate Drosophila neuroscience. VIDEO ABSTRACT.

Image-based correction of continuous and discontinuous non-planar axial distortion in serial section microscopy.

Motivation: Serial section microscopy is an established method for detailed anatomy reconstruction of biological specimen. During the last decade, high resolution electron microscopy (EM) of serial sections has become the de-facto standard for reconstruction of neural connectivity at ever increasing scales (EM connectomics). In serial section microscopy, the axial dimension of the volume is sampled by physically removing thin sections from the embedded specimen and subsequently imaging either the block-face or the section series. This process has limited precision leading to inhomogeneous non-planar sampling of the axial dimension of the volume which, in turn, results in distorted image volumes. This includes that section series may be collected and imaged in unknown order. Results: We developed methods to identify and correct these distortions through image-based signal analysis without any additional physical apparatus or measurements. We demonstrate the efficacy of our methods in proof of principle experiments and application to real world problems. Availability and Implementation: We made our work available as libraries for the ImageJ distribution Fiji and for deployment in a high performance parallel computing environment. Our sources are open and available at http://github.com/saalfeldlab/section-sort, http://github.com/saalfeldlab/z-spacing and http://github.com/saalfeldlab/z-spacing-spark. Contact: saalfelds@janelia.hhmi.org. Supplementary information: Supplementary data are available at Bioinformatics online.

Automated cerebellar lobule segmentation with application to cerebellar structural analysis in cerebellar disease.

The cerebellum plays an important role in both motor control and cognitive function. Cerebellar function is topographically organized and diseases that affect specific parts of the cerebellum are associated with specific patterns of symptoms. Accordingly, delineation and quantification of cerebellar sub-regions from magnetic resonance images are important in the study of cerebellar atrophy and associated functional losses. This paper describes an automated cerebellar lobule segmentation method based on a graph cut segmentation framework. Results from multi-atlas labeling and tissue classification contribute to the region terms in the graph cut energy function and boundary classification contributes to the boundary term in the energy function. A cerebellar parcellation is achieved by minimizing the energy function using the α-expansion technique. The proposed method was evaluated using a leave-one-out cross-validation on 15 subjects including both healthy controls and patients with cerebellar diseases. Based on reported Dice coefficients, the proposed method outperforms two state-of-the-art methods. The proposed method was then applied to 77 subjects to study the region-specific cerebellar structural differences in three spinocerebellar ataxia (SCA) genetic subtypes. Quantitative analysis of the lobule volumes shows distinct patterns of volume changes associated with different SCA subtypes consistent with known patterns of atrophy in these genetic subtypes.

A Multiple Object Geometric Deformable Model for Image Segmentation.

Deformable models are widely used for image segmentation, most commonly to find single objects within an image. Although several methods have been proposed to segment multiple objects using deformable models, substantial limitations in their utility remain. This paper presents a multiple object segmentation method using a novel and efficient object representation for both two and three dimensions. The new framework guarantees object relationships and topology, prevents overlaps and gaps, enables boundary-specific speeds, and has a computationally efficient evolution scheme that is largely independent of the number of objects. Maintaining object relationships and straightforward use of object-specific and boundary-specific smoothing and advection forces enables the segmentation of objects with multiple compartments, a critical capability in the parcellation of organs in medical imaging. Comparing the new framework with previous approaches shows its superior performance and scalability.

Approaching expert results using a hierarchical cerebellum parcellation protocol for multiple inexpert human raters.

Volumetric measurements obtained from image parcellation have been instrumental in uncovering structure-function relationships. However, anatomical study of the cerebellum is a challenging task. Because of its complex structure, expert human raters have been necessary for reliable and accurate segmentation and parcellation. Such delineations are time-consuming and prohibitively expensive for large studies. Therefore, we present a three-part cerebellar parcellation system that utilizes multiple inexpert human raters that can efficiently and expediently produce results nearly on par with those of experts. This system includes a hierarchical delineation protocol, a rapid verification and evaluation process, and statistical fusion of the inexpert rater parcellations. The quality of the raters' and fused parcellations was established by examining their Dice similarity coefficient, region of interest (ROI) volumes, and the intraclass correlation coefficient of region volume. The intra-rater ICC was found to be 0.93 at the finest level of parcellation.

System for integrated neuroimaging analysis and processing of structure.

Mapping brain structure in relation to neurological development, function, plasticity, and disease is widely considered to be one of the most essential challenges for opening new lines of neuro-scientific inquiry. Recent developments with MRI analysis of structural connectivity, anatomical brain segmentation, cortical surface parcellation, and functional imaging have yielded fantastic advances in our ability to probe the neurological structure-function relationship in vivo. To date, the image analysis efforts in each of these areas have typically focused on a single modality. Here, we extend the cortical reconstruction using implicit surface evolution (CRUISE) methodology to perform efficient, consistent, and topologically correct analyses in a natively multi-parametric manner. This effort combines and extends state-of-the-art techniques to simultaneously consider and analyze structural and diffusion information alongside quantitative and functional imaging data. Robust and consistent estimates of the cortical surface extraction, cortical labeling, diffusion-inferred contrasts, diffusion tractography, and subcortical parcellation are demonstrated in a scan-rescan paradigm. Accompanying this demonstration, we present a fully automated software system complete with validation data.

Automatic Cell Segmentation in Fluorescence Images of Confluent Cell Monolayers Using Multi-object Geometric Deformable Model.

With the rapid development of microscopy for cell imaging, there is a strong and growing demand for image analysis software to quantitatively study cell morphology. Automatic cell segmentation is an important step in image analysis. Despite substantial progress, there is still a need to improve the accuracy, efficiency, and adaptability to different cell morphologies. In this paper, we propose a fully automatic method for segmenting cells in fluorescence images of confluent cell monolayers. This method addresses several challenges through a combination of ideas. 1) It realizes a fully automatic segmentation process by first detecting the cell nuclei as initial seeds and then using a multi-object geometric deformable model (MGDM) for final segmentation. 2) To deal with different defects in the fluorescence images, the cell junctions are enhanced by applying an order-statistic filter and principal curvature based image operator. 3) The final segmentation using MGDM promotes robust and accurate segmentation results, and guarantees no overlaps and gaps between neighboring cells. The automatic segmentation results are compared with manually delineated cells, and the average Dice coefficient over all distinguishable cells is 0.88.

SEGMENTATION OF THE COMPLETE SUPERIOR CEREBELLAR PEDUNCLES USING A MULTI-OBJECT GEOMETRIC DEFORMABLE MODEL.

The superior cerebellar peduncles (SCPs) are white matter tracts that serve as the major efferent pathways from the cerebellum to the thalamus. With diffusion tensor images (DTI), tractography algorithms or volumetric segmentation methods have been able to reconstruct part of the SCPs. However, when the fibers cross, the primary eigenvector (PEV) no longer represents the primary diffusion direction. Therefore, at the crossing of the left and right SCP, known as the decussation of the SCPs (dSCP), fiber tracts propagate incorrectly. To our knowledge, previous methods have not been able to segment the SCPs correctly. In this work, we explore the diffusion properties and seek to volumetrically segment the complete SCPs. The non-crossing SCPs and dSCP are modeled as different objects. A multi-object geometric deformable model is employed to define the boundaries of each piece of the SCPs, with the forces derived from diffusion properties as well as the PEV. We tested our method on a software phantom and real subjects. Results indicate that our method is able to the resolve the crossing and segment the complete SCPs with repeatability.

Parcellation of the Thalamus Using Diffusion Tensor Images and a Multi-object Geometric Deformable Model.

The thalamus is a sub-cortical gray matter structure that relays signals between the cerebral cortex and midbrain. It can be parcellated into the thalamic nuclei which project to different cortical regions. The ability to automatically parcellate the thalamic nuclei could lead to enhanced diagnosis or prognosis in patients with some brain disease. Previous works have used diffusion tensor images (DTI) to parcellate the thalamus, using either tensor similarity or cortical connectivity as information driving the parcellation. In this paper, we propose a method that uses the diffusion tensors in a different way than previous works to guide a multiple object geometric deformable model (MGDM) for parcellation. The primary eigenvector (PEV) is used to indicate the homogeneity of fiber orientations. To remove the ambiguity due to the fact that the PEV is an orientation, we map the PEV into a 5D space known as the Knutsson space. An edge map is then generated from the 5D vector to show divisions between regions of aligned PEV's. The generalized gradient vector flow (GGVF) calculated from the edge map drives the evolution of the boundary of each nucleus. Region based force, balloon force, and curvature force are also employed to refine the boundaries. Experiments have been carried out on five real subjects. Quantitative measures show that the automated parcellation agrees with the manual delineation of an expert under a published protocol.

Automated segmentation of the cerebellar lobules using boundary specific classification and evolution.

The cerebellum is instrumental in coordinating many vital functions ranging from speech and balance to eye movement. The effect of cerebellar pathology on these functions is frequently examined using volumetric studies that depend on consistent and accurate delineation, however, no existing automated methods adequately delineate the cerebellar lobules. In this work, we describe a method we call the Automatic Classification of Cerebellar Lobules Algorithm using Implicit Multi-boundary evolution (ACCLAIM). A multiple object geometric deformable model (MGDM) enables each boundary surface of each individual lobule to be evolved under different level set speeds. An important innovation described in this work is that the speed for each lobule boundary is derived from a classifier trained specifically to identify that boundary. We compared our method to segmentations obtained using the atlas-based and multi-atlas fusion techniques, and demonstrate ACCLAIM's superior performance.

Magnetic resonance connectome automated pipeline: an overview.

This article presents a novel, tightly integrated pipeline for estimating a connectome. The pipeline utilizes magnetic resonance (MR) imaging (MRI) data to produce a high-level estimate of the structural connectivity in the human brain. The MR connectome automated pipeline (MRCAP) is efficient, and its modular construction allows researchers to modify algorithms to meet their specific requirements. The pipeline has been validated, and more than 200 connectomes have been processed and analyzed to date.

Foibles, follies, and fusion: web-based collaboration for medical image labeling.

Labels that identify specific anatomical and functional structures within medical images are essential to the characterization of the relationship between structure and function in many scientific and clinical studies. Automated methods that allow for high throughput have not yet been developed for all anatomical targets or validated for exceptional anatomies, and manual labeling remains the gold standard in many cases. However, manual placement of labels within a large image volume such as that obtained using magnetic resonance imaging (MRI) is exceptionally challenging, resource intensive, and fraught with intra- and inter-rater variability. The use of statistical methods to combine labels produced by multiple raters has grown significantly in popularity, in part, because it is thought that by estimating and accounting for rater reliability estimates of the true labels will be more accurate. This paper demonstrates the performance of a class of these statistical label combination methodologies using real-world data contributed by minimally trained human raters. The consistency of the statistical estimates, the accuracy compared to the individual observations, and the variability of both the estimates and the individual observations with respect to the number of labels are presented. It is demonstrated that statistical fusion successfully combines label information using data from online (Internet-based) collaborations among minimally trained raters. This first successful demonstration of a statistically based approach using minimally trained raters opens numerous possibilities for very large scale efforts in collaboration. Extension and generalization of these technologies for new applications will certainly present fascinating areas for continuing research.

Resolution of crossing fibers with constrained compressed sensing using diffusion tensor MRI.

Diffusion tensor imaging (DTI) is widely used to characterize tissue micro-architecture and brain connectivity. In regions of crossing fibers, however, the tensor model fails because it cannot represent multiple, independent intra-voxel orientations. Most of the methods that have been proposed to resolve this problem require diffusion magnetic resonance imaging (MRI) data that comprise large numbers of angles and high b-values, making them problematic for routine clinical imaging and many scientific studies. We present a technique based on compressed sensing that can resolve crossing fibers using diffusion MRI data that can be rapidly and routinely acquired in the clinic (30 directions, b-value equal to 700 s/mm2). The method assumes that the observed data can be well fit using a sparse linear combination of tensors taken from a fixed collection of possible tensors each having a different orientation. A fast algorithm for computing the best orientations based on a hierarchical compressed sensing algorithm and a novel metric for comparing estimated orientations are also proposed. The performance of this approach is demonstrated using both simulations and in vivo images. The method is observed to resolve crossing fibers using conventional data as well as a standard q-ball approach using much richer data that requires considerably more image acquisition time.

Robust statistical fusion of image labels.

Image labeling and parcellation (i.e., assigning structure to a collection of voxels) are critical tasks for the assessment of volumetric and morphometric features in medical imaging data. The process of image labeling is inherently error prone as images are corrupted by noise and artifacts. Even expert interpretations are subject to subjectivity and the precision of the individual raters. Hence, all labels must be considered imperfect with some degree of inherent variability. One may seek multiple independent assessments to both reduce this variability and quantify the degree of uncertainty. Existing techniques have exploited maximum a posteriori statistics to combine data from multiple raters and simultaneously estimate rater reliabilities. Although quite successful, wide-scale application has been hampered by unstable estimation with practical datasets, for example, with label sets with small or thin objects to be labeled or with partial or limited datasets. As well, these approaches have required each rater to generate a complete dataset, which is often impossible given both human foibles and the typical turnover rate of raters in a research or clinical environment. Herein, we propose a robust approach to improve estimation performance with small anatomical structures, allow for missing data, account for repeated label sets, and utilize training/catch trial data. With this approach, numerous raters can label small, overlapping portions of a large dataset, and rater heterogeneity can be robustly controlled while simultaneously estimating a single, reliable label set and characterizing uncertainty. The proposed approach enables many individuals to collaborate in the construction of large datasets for labeling tasks (e.g., human parallel processing) and reduces the otherwise detrimental impact of rater unavailability.

Direct segmentation of the major white matter tracts in diffusion tensor images.

Diffusion-weighted images of the human brain are acquired more and more routinely in clinical research settings, yet segmenting and labeling white matter tracts in these images is still challenging. We present in this paper a fully automated method to extract many anatomical tracts at once on diffusion tensor images, based on a Markov random field model and anatomical priors. The approach provides a direct voxel labeling, models explicitly fiber crossings and can handle white matter lesions. Experiments on simulations and repeatability studies show robustness to noise and reproducibility of the algorithm, which has been made publicly available.