SmartEM: machine-learning guided electron microscopy.

Connectomics provides essential nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole circuit or brain reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a singlebeam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM allocates the proper imaging time for each region of interest - scanning all pixels equally rapidly, then re-scanning small subareas more slowly where a higher quality signal is required to achieve accurate segmentability, in significantly less time. We demonstrate that this pipeline achieves a 7-fold acceleration of image acquisition time for connectomics using a commercial single-beam SEM. We apply SmartEM to reconstruct a portion of mouse cortex with the same accuracy as traditional microscopy but in less time.

Extensive soma-soma plate-like contact sites (ephapses) connect suprachiasmatic nucleus neurons.

The hypothalamic suprachiasmatic nucleus (SCN) is the central pacemaker for mammalian circadian rhythms. As such, this ensemble of cell-autonomous neuronal oscillators with divergent periods must maintain coordinated oscillations. To investigate ultrastructural features enabling such synchronization, 805 coronal ultrathin sections of mouse SCN tissue were imaged with electron microscopy and aligned into a volumetric stack, from which selected neurons within the SCN core were reconstructed in silico. We found that clustered SCN core neurons were physically connected to each other via multiple large soma-to-soma plate-like contacts. In some cases, a sliver of a glial process was interleaved. These contacts were large, covering on average ∼21% of apposing neuronal somata. It is possible that contacts may be the electrophysiological substrate for synchronization between SCN neurons. Such plate-like contacts may explain why the synchronization of SCN neurons is maintained even when chemical synaptic transmission or electrical synaptic transmission via gap junctions is blocked. Such ephaptic contact-mediated synchronization among nearby neurons may therefore contribute to the wave-like oscillations of circadian core clock genes and calcium signals observed in the SCN.

Three­dimensional reconstruction of SCN tissue via serial electron microscopy revealed a novel structural feature of SCN neurons that may account for interneuronal synchronization that persists even when the predominant mechanisms of neuronal communication are blocked. We found that SCN core neurons are connected by multiple soma­soma contact specializations, ultrastructural elements that could enable synchronization of tightly packed neurons organized in clustered networks. This extensive network of plate­like soma­soma contacts among clustered SCN neurons may provide insight into how ∼20,000 autonomous neuronal oscillators with a broad range of intrinsic periods remain synchronized in the absence of ordinary communication modalities, thereby conferring the resilience required for the SCN to function as the mammalian circadian pacemaker.

Two-stage error detection to improve electron microscopy image mosaicking.

Large-scale electron microscopy (EM) has enabled the reconstruction of brain connectomes at the synaptic level by serially scanning over massive areas of sample sections. The acquired big EM data sets raise the great challenge of image mosaicking at high accuracy. Currently, it simply follows the conventional algorithms designed for natural images, which are usually composed of only a few tiles, using a single type of keypoint feature that would sacrifice speed for stronger performance. Even so, in the process of stitching hundreds of thousands of tiles for large EM data, errors are still inevitable and diverse. Moreover, there has not yet been an appropriate metric to quantitatively evaluate the stitching of biomedical EM images. Here we propose a two-stage error detection method to improve the EM image mosaicking. It firstly uses point-based error detection in combination with a hybrid feature framework to expedite the stitching computation while maintaining high accuracy. Following is the second detection of unresolved errors with a newly designed metric of EM stitched image quality assessment (EMSIQA). The novel detection-based mosaicking pipeline is tested on large EM data sets and proven to be more effective and as accurate when compared with existing methods.

A petavoxel fragment of human cerebral cortex reconstructed at nanoscale resolution.

To fully understand how the human brain works, knowledge of its structure at high resolution is needed. Presented here is a computationally intensive reconstruction of the ultrastructure of a cubic millimeter of human temporal cortex that was surgically removed to gain access to an underlying epileptic focus. It contains about 57,000 cells, about 230 millimeters of blood vessels, and about 150 million synapses and comprises 1.4 petabytes. Our analysis showed that glia outnumber neurons 2:1, oligodendrocytes were the most common cell, deep layer excitatory neurons could be classified on the basis of dendritic orientation, and among thousands of weak connections to each neuron, there exist rare powerful axonal inputs of up to 50 synapses. Further studies using this resource may bring valuable insights into the mysteries of the human brain.

Sample preparation methods for volume electron microscopy in mollusc Berghia stephanieae.

Creating a high-resolution brain atlas in diverse species offers crucial insights into general principles underlying brain function and development. A volume electron microscopy approach to generate such neural maps has been gaining importance due to advances in imaging, data storage capabilities, and data analysis protocols. Sample preparation remains challenging and is a crucial step to accelerate the imaging and data processing pipeline. Here, we introduce several replicable methods for processing the brains of the gastropod mollusc, Berghia stephanieae for volume electron microscopy. Although high-pressure freezing is the most reliable method, the depth of cryopreservation is a severe limitation for large tissue samples. We introduce a BROPA-based method using pyrogallol and methods to rapidly process samples that can save hours at the bench. This is the first report on sample preparation and imaging pipeline for volume electron microscopy in a gastropod mollusc, opening up the potential for connectomic analysis and comparisons with other phyla.

A large-scale volumetric correlated light and electron microscopy study localizes Alzheimer's disease-related molecules in the hippocampus.

Connectomics is a nascent neuroscience field to map and analyze neuronal networks. It provides a new way to investigate abnormalities in brain tissue, including in models of Alzheimer's disease (AD). This age-related disease is associated with alterations in amyloid-ß (Aß) and phosphorylated tau (pTau). These alterations correlate with AD's clinical manifestations, but causal links remain unclear. Therefore, studying these molecular alterations within the context of the local neuronal and glial milieu may provide insight into disease mechanisms. Volume electron microscopy (vEM) is an ideal tool for performing connectomics studies at the ultrastructural level, but localizing specific biomolecules within large-volume vEM data has been challenging. Here we report a volumetric correlated light and electron microscopy (vCLEM) approach using fluorescent nanobodies as immuno-probes to localize Alzheimer's disease-related molecules in a large vEM volume. Three molecules (pTau, Aß, and a marker for activated microglia (CD11b)) were labeled without the need for detergents by three nanobody probes in a sample of the hippocampus of the 3xTg Alzheimer's disease model mouse. Confocal microscopy followed by vEM imaging of the same sample allowed for registration of the location of the molecules within the volume. This dataset revealed several ultrastructural abnormalities regarding the localizations of Aß and pTau in novel locations. For example, two pTau-positive post-synaptic spine-like protrusions innervated by axon terminals were found projecting from the axon initial segment of a pyramidal cell. Three pyramidal neurons with intracellular Aß or pTau were 3D reconstructed. Automatic synapse detection, which is necessary for connectomics analysis, revealed the changes in density and volume of synapses at different distances from an Aß plaque. This vCLEM approach is useful to uncover molecular alterations within large-scale volume electron microscopy data, opening a new connectomics pathway to study Alzheimer's disease and other types of dementia.

Probing Molecular Diversity and Ultrastructure of Brain Cells with Fluorescent Aptamers.

Detergent-free immunolabeling has been proven feasible for correlated light and electron microscopy, but its application is restricted by the availability of suitable affinity reagents. Here we introduce CAptVE, a method using slow off-rate modified aptamers for cell fluorescence labeling on ultrastructurally reconstructable electron micrographs. CAptVE provides labeling for a wide range of biomarkers, offering a pathway to integrate molecular analysis into recent approaches to delineate neural circuits via connectomics.

A Scalable Staining Strategy for Whole-Brain Connectomics.

Mapping the complete synaptic connectivity of a mammalian brain would be transformative, revealing the pathways underlying perception, behavior, and memory. Serial section electron microscopy, via membrane staining using osmium tetroxide, is ideal for visualizing cells and synaptic connections but, in whole brain samples, faces significant challenges related to chemical treatment and volume changes. These issues can adversely affect both the ultrastructural quality and macroscopic tissue integrity. By leveraging time-lapse X-ray imaging and brain proxies, we have developed a 12-step protocol, ODeCO, that effectively infiltrates osmium throughout an entire mouse brain while preserving ultrastructure without any cracks or fragmentation, a necessary prerequisite for constructing the first comprehensive mouse brain connectome.

Preserving extracellular space for high-quality optical and ultrastructural studies of whole mammalian brains.

Analysis of brain structure, connectivity, and molecular diversity relies on effective tissue fixation. Conventional tissue fixation causes extracellular space (ECS) loss, complicating the segmentation of cellular objects from electron microscopy datasets. Previous techniques for preserving ECS in mammalian brains utilizing high-pressure perfusion can give inconsistent results owing to variations in the hydrostatic pressure within the vasculature. A more reliable fixation protocol that uniformly preserves the ECS throughout whole brains would greatly benefit a wide range of neuroscience studies. Here, we report a straightforward transcardial perfusion strategy that preserves ECS throughout the whole rodent brain. No special setup is needed besides sequential solution changes, and the protocol offers excellent reproducibility. In addition to better capturing tissue ultrastructure, preservation of ECS has many downstream advantages such as accelerating heavy-metal staining for electron microscopy, improving detergent-free immunohistochemistry for correlated light and electron microscopy, and facilitating lipid removal for tissue clearing.

Multiplexed volumetric CLEM enabled by antibody derivatives provides new insights into the cytology of the mouse cerebellar cortex.

Mapping neuronal networks that underlie behavior has become a central focus in neuroscience. While serial section electron microscopy (ssEM) can reveal the fine structure of neuronal networks (connectomics), it does not provide the molecular information that helps identify cell types or their functional properties. Volumetric correlated light and electron microscopy (vCLEM) combines ssEM and volumetric fluorescence microscopy to incorporate molecular labeling into ssEM datasets. We developed an approach that uses small fluorescent single-chain variable fragment (scFv) immuno-probes to perform multiplexed detergent-free immuno-labeling and ssEM on the same samples. We generated eight such fluorescent scFvs that targeted useful markers for brain studies (green fluorescent protein, glial fibrillary acidic protein, calbindin, parvalbumin, voltage-gated potassium channel subfamily A member 2, vesicular glutamate transporter 1, postsynaptic density protein 95, and neuropeptide Y). To test the vCLEM approach, six different fluorescent probes were imaged in a sample of the cortex of a cerebellar lobule (Crus 1), using confocal microscopy with spectral unmixing, followed by ssEM imaging of the same sample. The results show excellent ultrastructure with superimposition of the multiple fluorescence channels. Using this approach we could document a poorly described cell type in the cerebellum, two types of mossy fiber terminals, and the subcellular localization of one type of ion channel. Because scFvs can be derived from existing monoclonal antibodies, hundreds of such probes can be generated to enable molecular overlays for connectomic studies.

Connectomics of the Octopus vulgaris vertical lobe provides insight into conserved and novel principles of a memory acquisition network.

Here, we present the first analysis of the connectome of a small volume of the Octopus vulgaris vertical lobe (VL), a brain structure mediating the acquisition of long-term memory in this behaviorally advanced mollusk. Serial section electron microscopy revealed new types of interneurons, cellular components of extensive modulatory systems, and multiple synaptic motifs. The sensory input to the VL is conveyed via~1.8 × 106 axons that sparsely innervate two parallel and interconnected feedforward networks formed by the two types of amacrine interneurons (AM), simple AMs (SAMs) and complex AMs (CAMs). SAMs make up 89.3% of the~25 × 106VL cells, each receiving a synaptic input from only a single input neuron on its non-bifurcating primary neurite, suggesting that each input neuron is represented in only~12 ± 3.4SAMs. This synaptic site is likely a 'memory site' as it is endowed with LTP. The CAMs, a newly described AM type, comprise 1.6% of the VL cells. Their bifurcating neurites integrate multiple inputs from the input axons and SAMs. While the SAM network appears to feedforward sparse 'memorizable' sensory representations to the VL output layer, the CAMs appear to monitor global activity and feedforward a balancing inhibition for 'sharpening' the stimulus-specific VL output. While sharing morphological and wiring features with circuits supporting associative learning in other animals, the VL has evolved a unique circuit that enables associative learning based on feedforward information flow.

Multiplexed volumetric CLEM enabled by antibody derivatives provides new insights into the cytology of the mouse cerebellar cortex.

Mapping neuronal networks that underlie behavior has become a central focus in neuroscience. While serial section electron microscopy (ssEM) can reveal the fine structure of neuronal networks (connectomics), it does not provide the molecular information that helps identify cell types or their functional properties. Volumetric correlated light and electron microscopy (vCLEM) combines ssEM and volumetric fluorescence microscopy to incorporate molecular labeling into ssEM datasets. We developed an approach that uses small fluorescent single-chain variable fragment (scFv) immuno-probes to perform multiplexed detergent-free immuno-labeling and ssEM on the same samples. We generated eight such fluorescent scFvs that targeted useful markers for brain studies (green fluorescent protein, glial fibrillary acidic protein, calbindin, parvalbumin, voltage-gated potassium channel subfamily A member 2, vesicular glutamate transporter 1, postsynaptic density protein 95, and neuropeptide Y). To test the vCLEM approach, six different fluorescent probes were imaged in a sample of the cortex of a cerebellar lobule (Crus 1), using confocal microscopy with spectral unmixing, followed by ssEM imaging of the same sample. The results show excellent ultrastructure with superimposition of the multiple fluorescence channels. Using this approach we could document a poorly described cell type in the cerebellum, two types of mossy fiber terminals, and the subcellular localization of one type of ion channel. Because scFvs can be derived from existing monoclonal antibodies, hundreds of such probes can be generated to enable molecular overlays for connectomic studies.

Immersion Fixation and Staining of Multicubic Millimeter Volumes for Electron Microscopy-Based Connectomics of Human Brain Biopsies.

Connectomics allows mapping of cells and their circuits at the nanometer scale in volumes of approximately 1 mm3. Given that the human cerebral cortex can be 3 mm in thickness, larger volumes are required. Larger-volume circuit reconstructions of human brain are limited by 1) the availability of fresh biopsies; 2) the need for excellent preservation of ultrastructure, including extracellular space; and 3) the requirement of uniform staining throughout the sample, among other technical challenges. Cerebral cortical samples from neurosurgical patients are available owing to lead placement for deep brain stimulation. Described here is an immersion fixation, heavy metal staining, and tissue processing method that consistently provides excellent ultrastructure throughout human and rodent surgical brain samples of volumes 2 × 2 × 2 mm3 and up to 37 mm3 with one dimension ≤2 mm. This method should allow synapse-level circuit analysis in samples from patients with psychiatric and neurologic disorders.

Post-embryonic remodeling of the C. elegans motor circuit.

During development, animals can maintain behavioral output even as underlying circuitry structurally remodels. After hatching, C. elegans undergoes substantial motor neuron expansion and synapse rewiring while the animal continuously moves with an undulatory pattern. To understand how the circuit transitions from its juvenile to mature configuration without interrupting functional output, we reconstructed the C. elegans motor circuit by electron microscopy across larval development. We observed the following: First, embryonic motor neurons transiently interact with the developing post-embryonic motor neurons prior to remodeling of their juvenile wiring. Second, post-embryonic neurons initiate synapse development with their future partners as their neurites navigate through the juvenile nerve cords. Third, embryonic and post-embryonic neurons sequentially build structural machinery needed for the adult circuit before the embryonic neurons relinquish their roles to post-embryonic neurons. Fourth, this transition is repeated region by region along the body in an anterior-to-posterior sequence, following the birth order of neurons. Through this orchestrated and programmed rewiring, the motor circuit gradually transforms from asymmetric to symmetric wiring. These maturation strategies support the continuous maintenance of motor patterns as the juvenile circuit develops into the adult configuration.

Functional and ultrastructural analysis of reafferent mechanosensation in larval zebrafish.

All animals need to differentiate between exafferent stimuli, which are caused by the environment, and reafferent stimuli, which are caused by their own movement. In the case of mechanosensation in aquatic animals, the exafferent inputs are water vibrations in the animal's proximity, which need to be distinguishable from the reafferent inputs arising from fluid drag due to locomotion. Both of these inputs are detected by the lateral line, a collection of mechanosensory organs distributed along the surface of the body. In this study, we characterize in detail how hair cells-the receptor cells of the lateral line-in zebrafish larvae discriminate between such reafferent and exafferent signals. Using dye labeling of the lateral line nerve, we visualize two parallel descending inputs that can influence lateral line sensitivity. We combine functional imaging with ultra-structural EM circuit reconstruction to show that cholinergic signals originating from the hindbrain transmit efference copies (copies of the motor command that cancel out self-generated reafferent stimulation during locomotion) and that dopaminergic signals from the hypothalamus may have a role in threshold modulation, both in response to locomotion and salient stimuli. We further gain direct mechanistic insight into the core components of this circuit by loss-of-function perturbations using targeted ablations and gene knockouts. We propose that this simple circuit is the core implementation of mechanosensory reafferent suppression in these young animals and that it might form the first instantiation of state-dependent modulation found at later stages in development.

Connectomes across development reveal principles of brain maturation.

An animal's nervous system changes as its body grows from birth to adulthood and its behaviours mature1-8. The form and extent of circuit remodelling across the connectome is unknown3,9-15. Here we used serial-section electron microscopy to reconstruct the full brain of eight isogenic Caenorhabditis elegans individuals across postnatal stages to investigate how it changes with age. The overall geometry of the brain is preserved from birth to adulthood, but substantial changes in chemical synaptic connectivity emerge on this consistent scaffold. Comparing connectomes between individuals reveals substantial differences in connectivity that make each brain partly unique. Comparing connectomes across maturation reveals consistent wiring changes between different neurons. These changes alter the strength of existing connections and create new connections. Collective changes in the network alter information processing. During development, the central decision-making circuitry is maintained, whereas sensory and motor pathways substantially remodel. With age, the brain becomes progressively more feedforward and discernibly modular. Thus developmental connectomics reveals principles that underlie brain maturation.

Two Stream Active Query Suggestion for Active Learning in Connectomics.

For large-scale vision tasks in biomedical images, the labeled data is often limited to train effective deep models. Active learning is a common solution, where a query suggestion method selects representative unlabeled samples for annotation, and the new labels are used to improve the base model. However, most query suggestion models optimize their learnable parameters only on the limited labeled data and consequently become less effective for the more challenging unlabeled data. To tackle this, we propose a two-stream active query suggestion approach. In addition to the supervised feature extractor, we introduce an unsupervised one optimized on all raw images to capture diverse image features, which can later be improved by fine-tuning on new labels. As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics. With the framework, we curate, to our best knowledge, the largest connectomics dataset with dense synapses and mitochondria annotation. On this new dataset, our method outperforms previous state-of-the-art methods by 3.1% for synapse and 3.8% for mitochondria in terms of region-of-interest proposal accuracy. We also apply our method to image classification, where it outperforms previous approaches on CIFAR-10 under the same limited annotation budget. The project page is https://zudi-lin.github.io/projects/#two_stream_active.

A connectomics approach to understanding a retinal disease.

Macular telangiectasia type 2 (MacTel), a late-onset macular degeneration, has been linked to a loss in the retina of Müller glial cells and the amino acid serine, synthesized by the Müller cells. The disease is confined mainly to a central retinal region called the MacTel zone. We have used electron microscopic connectomics techniques, optimized for disease analysis, to study the retina from a 48-y-old woman suffering from MacTel. The major observations made were specific changes in mitochondrial structure within and outside the MacTel zone that were present in all retinal cell types. We also identified an abrupt boundary of the MacTel zone that coincides with the loss of Müller cells and macular pigment. Since Müller cells synthesize retinal serine, we propose that a deficiency of serine, required for mitochondrial maintenance, causes mitochondrial changes that underlie MacTel development.

Developmental Rewiring between Cerebellar Climbing Fibers and Purkinje Cells Begins with Positive Feedback Synapse Addition.

During postnatal development, cerebellar climbing fibers alter their innervation strengths onto supernumerary Purkinje cell targets, generating a one-to-few connectivity pattern in adulthood. To get insight about the processes responsible for this remapping, we reconstructed serial electron microscopy datasets from mice during the first postnatal week. Between days 3 and 7, individual climbing fibers selectively add many synapses onto a subset of Purkinje targets in a positive-feedback manner, without pruning synapses from other targets. Active zone sizes of synapses associated with powerful versus weak inputs are indistinguishable. Changes in synapse number are thus the predominant form of early developmental plasticity. Finally, the numbers of climbing fibers and Purkinje cells in a local region nearly match. Initial over-innervation of Purkinje cells by climbing fibers is therefore economical: the number of axons entering a region is enough to assure that each ultimately retains a postsynaptic target and that none branched there in vain.

Serial-section electron microscopy using automated tape-collecting ultramicrotome (ATUM).

The Automated Tape-Collecting Ultramicrotome (ATUM) is a tape-reeling device that is placed in a water-filled diamond knife boat to collect serial sections as they are cut by a conventional ultramicrotome. The ATUM can collect thousands of sections of many different shapes and sizes, which are subsequently imaged by a scanning electron microscope. This method has been used for large-scale connectomics projects of mouse brain, and is well suited for other smaller-scale studies of tissues, cells, and organisms. Here, we describe basic procedures for preparing a block for ATUM sectioning, handling of the ATUM, tape preparation, post-treatment of sections, and considerations for mapping, imaging, and aligning the serial sections.