Very-High Dynamic Range, 10,000 Frames/Second Pixel Array Detector for Electron Microscopy.

Precision and accuracy of quantitative scanning transmission electron microscopy (STEM) methods such as ptychography, and the mapping of electric, magnetic, and strain fields depend on the dose. Reasonable acquisition time requires high beam current and the ability to quantitatively detect both large and minute changes in signal. A new hybrid pixel array detector (PAD), the second-generation Electron Microscope Pixel Array Detector (EMPAD-G2), addresses this challenge by advancing the technology of a previous generation PAD, the EMPAD. The EMPAD-G2 images continuously at a frame-rates up to 10 kHz with a dynamic range that spans from low-noise detection of single electrons to electron beam currents exceeding 180 pA per pixel, even at electron energies of 300 keV. The EMPAD-G2 enables rapid collection of high-quality STEM data that simultaneously contain full diffraction information from unsaturated bright-field disks to usable Kikuchi bands and higher-order Laue zones. Test results from 80 to 300 keV are presented, as are first experimental results demonstrating ptychographic reconstructions, strain and polarization maps. We introduce a new information metric, the maximum usable imaging speed (MUIS), to identify when a detector becomes electron-starved, saturated or its pixel count is mismatched with the beam current.

Sparse Scanning Electron Microscopy Data Acquisition and Deep Neural Networks for Automated Segmentation in Connectomics.

With the growing importance of three-dimensional and very large field of view imaging, acquisition time becomes a serious bottleneck. Additionally, dose reduction is of importance when imaging material like biological tissue that is sensitive to electron radiation. Random sparse scanning can be used in the combination with image reconstruction techniques to reduce the acquisition time or electron dose in scanning electron microscopy. In this study, we demonstrate a workflow that includes data acquisition on a scanning electron microscope, followed by a sparse image reconstruction based on compressive sensing or alternatively using neural networks. Neuron structures are automatically segmented from the reconstructed images using deep learning techniques. We show that the average dwell time per pixel can be reduced by a factor of 2-3, thereby providing a real-life confirmation of previous results on simulated data in one of the key segmentation applications in connectomics and thus demonstrating the feasibility and benefit of random sparse scanning techniques for a specific real-world scenario.

SmartEM: machine-learning guided electron microscopy.

Connectomics provides essential nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole circuit or brain reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a singlebeam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM allocates the proper imaging time for each region of interest - scanning all pixels equally rapidly, then re-scanning small subareas more slowly where a higher quality signal is required to achieve accurate segmentability, in significantly less time. We demonstrate that this pipeline achieves a 7-fold acceleration of image acquisition time for connectomics using a commercial single-beam SEM. We apply SmartEM to reconstruct a portion of mouse cortex with the same accuracy as traditional microscopy but in less time.

How should a fixed budget of dwell time be spent in scanning electron microscopy to optimize image quality?

In scanning electron microscopy, the achievable image quality is often limited by a maximum feasible acquisition time per dataset. Particularly with regard to three-dimensional or large field-of-view imaging, a compromise must be found between a high amount of shot noise, which leads to a low signal-to-noise ratio, and excessive acquisition times. Assuming a fixed acquisition time per frame, we compared three different strategies for algorithm-assisted image acquisition in scanning electron microscopy. We evaluated (1) raster scanning with a reduced dwell time per pixel followed by a state-of-the-art Denoising algorithm, (2) raster scanning with a decreased resolution in conjunction with a state-of-the-art Super Resolution algorithm, and (3) a sparse scanning approach where a fixed percentage of pixels is visited by the beam in combination with state-of-the-art inpainting algorithms. Additionally, we considered increased beam currents for each of the strategies. The experiments showed that sparse scanning using an appropriate reconstruction technique was superior to the other strategies.