High-speed 4-dimensional scanning transmission electron microscopy using compressive sensing techniques.

Here we show that compressive sensing allows 4-dimensional (4-D) STEM data to be obtained and accurately reconstructed with both high-speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4-D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal-to-noise ratio in the recovered phase using 0.3% of the total data. Lay abstract: Four-dimensional scanning transmission electron microscopy (4-D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2-dimensional signal is acquired at each 2-D probe location, equating to a 4-D dataset. Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4-D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256^2 probe locations can take on the order of 30s, approximately 100-1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4-D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination. Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10-100 times. The acquired data is then processed to fill-in or inpaint the missing data, taking advantage of the inherently low-complex signals which can be linearly combined to recover the information. In this work, similar methods are demonstrated for the acquisition of 4-D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4-D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100-300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.

The Potential of Subsampling and Inpainting for Fast Low-Dose Cryo FIB-SEM Imaging.

Traditional image acquisition for cryo focused ion-beam scanning electron microscopy (FIB-SEM) tomography often sees thousands of images being captured over a period of many hours, with immense data sets being produced. When imaging beam sensitive materials, these images are often compromised by additional constraints related to beam damage and the devitrification of the material during imaging, which renders data acquisition both costly and unreliable. Subsampling and inpainting are proposed as solutions for both of these aspects, allowing fast and low-dose imaging to take place in the Focused ion-beam scanning electron microscopy FIB-SEM without an appreciable loss in image quality. In this work, experimental data are presented which validate subsampling and inpainting as a useful tool for convenient and reliable data acquisition in a FIB-SEM, with new methods of handling three-dimensional data being employed in the context of dictionary learning and inpainting algorithms using a newly developed microscope control software and data recovery algorithm.

Cryo-electron ptychography: Applications and potential in biological characterisation.

There is a clear need for developments in characterisation techniques that provide detailed information about structure-function relationships in biology. Using electron microscopy to achieve high resolution while maintaining a broad field of view remains a challenge, particularly for radiation-sensitive specimens where the signal-to-noise ratio required to maintain structural integrity is limited by low electron fluence. In this review, we explore the potential of cryogenic electron ptychography as an alternative method for characterising biological systems under low-fluence conditions. Using this method with increased information content from multiple sampled regions of interest potentially allows 3D reconstruction with far fewer particles than required in conventional cryo-electron microscopy. This is important for achieving higher resolution in systems where distributions of homogeneous single particles are difficult to obtain. We discuss the progress, limitations, and potential areas for future development of this approach for both single particle analysis and applications to heterogeneous large objects.

Mapping Nanoscale Electrostatic Field Fluctuations around Graphene Dislocation Cores Using Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM).

Defects in crystalline lattices cause modulation of the atomic density, and this leads to variations in the associated electrostatics at the nanoscale. Mapping these spatially varying charge fluctuations using transmission electron microscopy has typically been challenging due to complicated contrast transfer inherent to conventional phase contrast imaging. To overcome this, we used four-dimensional scanning transmission electron microscopy (4D-STEM) to measure electrostatic fields near point dislocations in a monolayer. The asymmetry of the atomic density in a (1,0) edge dislocation core in graphene yields a local enhancement of the electric field in part of the dislocation core. Through experiment and simulation, the increased electric field magnitude is shown to arise from "long-range" interactions from beyond the nearest atomic neighbor. These results provide insights into the use of 4D-STEM to quantify electrostatics in thin materials and map out the lateral potential variations that are important for molecular and atomic bonding through Coulombic interactions.

Cryogenic electron ptychographic single particle analysis with wide bandwidth information transfer.

Advances in cryogenic transmission electron microscopy have revolutionised the determination of many macromolecular structures at atomic or near-atomic resolution. This method is based on conventional defocused phase contrast imaging. However, it has limitations of weaker contrast for small biological molecules embedded in vitreous ice, in comparison with cryo-ptychography, which shows increased contrast. Here we report a single-particle analysis based on the use of ptychographic reconstruction data, demonstrating that three dimensional reconstructions with a wide information transfer bandwidth can be recovered by Fourier domain synthesis. Our work suggests future applications in otherwise challenging single particle analyses, including small macromolecules and heterogeneous or flexible particles. In addition structure determination in situ within cells without the requirement for protein purification and expression may be possible.

Towards real-time STEM simulations through targeted subsampling strategies.

Scanning transmission electron microscopy images can be complex to interpret on the atomic scale as the contrast is sensitive to multiple factors such as sample thickness, composition, defects and aberrations. Simulations are commonly used to validate or interpret real experimental images, but they come at a cost of either long computation times or specialist hardware such as graphics processing units. Recent works in compressive sensing for experimental STEM images have shown that it is possible to significantly reduce the amount of acquired signal and still recover the full image without significant loss of image quality, and therefore it is proposed here that similar methods can be applied to STEM simulations. In this paper, we demonstrate a method that can significantly increase the efficiency of STEM simulations through a targeted sampling strategy, along with a new approach to independently subsample each frozen phonon layer. We show the effectiveness of this method by simulating a SrTiO3 grain boundary and monolayer 2H-MoS2 containing a sulphur vacancy using the abTEM software. We also show how this method is not limited to only traditional multislice methods, but also increases the speed of the PRISM simulation method. Furthermore, we discuss the possibility for STEM simulations to seed the acquisition of real data, to potentially lead the way to self-driving (correcting) STEM.

Three-dimensional electron ptychography of organic-inorganic hybrid nanostructures.

Three dimensional scaffolded DNA origami with inorganic nanoparticles has been used to create tailored multidimensional nanostructures. However, the image contrast of DNA is poorer than those of the heavy nanoparticles in conventional transmission electron microscopy at high defocus so that the biological and non-biological components in 3D scaffolds cannot be simultaneously resolved using tomography of samples in a native state. We demonstrate the use of electron ptychography to recover high contrast phase information from all components in a DNA origami scaffold without staining. We further quantitatively evaluate the enhancement of contrast in comparison with conventional transmission electron microscopy. In addition, We show that for ptychography post-reconstruction focusing simplifies the workflow and reduces electron dose and beam damage.

Trainable segmentation for transmission electron microscope images of inorganic nanoparticles.

We present a trainable segmentation method implemented within the python package ParticleSpy. The method takes user labelled pixels, which are used to train a classifier and segment images of inorganic nanoparticles from transmission electron microscope images. This implementation is based on the trainable Waikato Environment for Knowledge Analysis (WEKA) segmentation, but is written in python, allowing a large degree of flexibility and meaning it can be easily expanded using other python packages. We find that trainable segmentation offers better accuracy than global or local thresholding methods and requires as few as 100 user-labelled pixels to produce an accurate segmentation. Trainable segmentation presents a balance of accuracy and training time between global/local thresholding and neural networks, when used on transmission electron microscope images of nanoparticles. We also quantitatively investigate the effectiveness of the components of trainable segmentation, its filter kernels and classifiers, in order to demonstrate the use cases for the different filter kernels in ParticleSpy and the most accurate classifiers for different data types. A set of filter kernels is identified that are effective in distinguishing particles from background but that retain dissimilar features. In terms of classifiers, we find that different classifiers perform optimally for different image contrast; specifically, a random forest classifier performs best for high-contrast ADF images, but that QDA and Gaussian Naïve Bayes classifiers perform better for low-contrast TEM images.

Measurement of the size, shape and composition of nanoparticles is routinely performed using transmission electron microscopy and related techniques. Typically, distinguishing particles from the background in an image is performed using the intensity of each pixel, creating two sets of pixels to separate particles from background. However, this separation of intensity can be difficult if the contrast in an image is low, or if the intensity of the background varies significantly. In this study, an approach that takes into account additional image features (such as boundaries and texture) was investigated to study electron microscope images of metallic nanoparticles. In this 'trainable segmentation' approach, the user labels examples of particle and background pixels in order to train a machine learning algorithm to distinguish between particles and background. The performance of different machine learning algorithms was investigated, in addition to the effect of using different features to aid the segmentation. Overall, a trainable segmentation approach was found to perform better than use of an intensity threshold to distinguish between particles and background in electron microscope images.

Direct Visualisation of the Surface Atomic Active Sites of Carbon-Supported Co3 O4 Nanocrystals via High-Resolution Phase Restoration.

The atomic arrangement of the terminating facets on spinel Co3 O4 nanocrystals is strongly linked to their catalytic performance. However, the spinel crystal structure offers multiple possible surface terminations depending on the synthesis. Thus, understanding the terminating surface atomic structure is essential in developing high-performance Co3 O4 nanocrystals. In this work, we present direct atomic-scale observation of the surface terminations of Co3 O4 nanoparticles supported on hollow carbon spheres (HCSs) using exit wavefunction reconstruction from aberration-corrected transmission electron microscopy focal-series. The restored high-resolution phases show distinct resolved oxygen and cobalt atomic columns. The data show that the structure of {100}, {110}, and {111} facets of spinel Co3 O4 exhibit characteristic active sites for carbon monoxide (CO) adsorption, in agreement with density functional theory calculations. Of these facets, the {100} and {110} surface terminations are better suited for CO adsorption than the {111}. However, the presence of oxygen on the {111} surface termination indicates this facet also plays an essential role in CO adsorption. Our results demonstrate direct evidence of the surface termination atomic structure beyond the assumed stoichiometry of the surface.

Applications of deep learning in electron microscopy.

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.

Quantifying the performance of a hybrid pixel detector with GaAs:Cr sensor for transmission electron microscopy.

Hybrid pixel detectors (HPDs) have been shown to be highly effective for diffraction-based and time-resolved studies in transmission electron microscopy, but their performance is limited by the fact that high-energy electrons scatter over long distances in their thick Si sensors. An advantage of HPDs compared to monolithic active pixel sensors is that their sensors do not need to be fabricated from Si. We have compared the performance of the Medipix3 HPD with a Si sensor and a GaAs:Cr sensor using primary electrons in the energy range of 60-300 keV. We describe the measurement and calculation of the detectors' modulation transfer function (MTF) and detective quantum efficiency (DQE), which show that the performance of the GaAs:Cr device is markedly superior to that of the Si device for high-energy electrons.

Atomic resolution HOLZ-STEM imaging of atom position modulation in oxide heterostructures.

It is shown that higher order Laue zone (HOLZ) rings in high energy electron diffraction are specific to individual columns of atoms, and show different strengths, structure and radii for different atom columns along the same projection in a structure. An atomic resolution 4-dimensional STEM dataset is recorded from a <110> direction in a perovskite trilayer, where only the central LaFeO3 layer should show a period doubling that gives rise to an extra HOLZ ring. Careful comparison between experiment and multislice simulations is used to understand the origins of all features in the patterns. A strong HOLZ ring is seen for the La-O columns, indicating strong La position modulation along this direction, whereas a weaker ring is seen along the O columns, and a very weak ring is seen along the Fe columns. This demonstrates that atomic resolution HOLZ-STEM is a feasible method for investigating the 3D periodicity of crystalline materials with atomic resolution.

Oleylamine Aging of PtNi Nanoparticles Giving Enhanced Functionality for the Oxygen Reduction Reaction.

We report a rapid solution-phase strategy to synthesize alloyed PtNi nanoparticles which demonstrate outstanding functionality for the oxygen reduction reaction (ORR). This one-pot coreduction colloidal synthesis results in a monodisperse population of single-crystal nanoparticles of rhombic dodecahedral morphology with Pt-enriched edges and compositions close to Pt1Ni2. We use nanoscale 3D compositional analysis to reveal for the first time that oleylamine (OAm)-aging of the rhombic dodecahedral Pt1Ni2 particles results in Ni leaching from surface facets, producing aged particles with concave faceting, an exceptionally high surface area, and a composition of Pt2Ni1. We show that the modified atomic nanostructures catalytically outperform the original PtNi rhombic dodecahedral particles by more than two-fold and also yield improved cycling durability. Their functionality for the ORR far exceeds commercially available Pt/C nanoparticle electrocatalysts, both in terms of mass-specific activities (up to a 25-fold increase) and intrinsic area-specific activities (up to a 27-fold increase).

Contrast transfer and noise considerations in focused-probe electron ptychography.

Electron ptychography is a 4-D STEM phase-contrast imaging technique with applications to light-element and beam-sensitive materials. Although the electron dose (electrons incident per unit area on the sample) is the primary figure of merit for imaging beam-sensitive materials, it is also necessary to consider the contrast transfer properties of the imaging technique. Here, we explore the contrast transfer properties of electron ptychography. The contrast transfer of focused-probe, non-iterative electron ptychography using the single-side-band (SSB) method is demonstrated experimentally. The band-pass nature of the phase-contrast transfer function (PCTF) for SSB ptychography places strict limitations on the probe convergence semi-angles required to resolve specific sample features with high contrast. The PCTF of the extended ptychographic iterative engine (ePIE) is broader than that for SSB ptychography, although when both high and low spatial frequencies are transferred, band-pass filtering is required to remove image artefacts. Normalisation of the transfer function with respect to the noise level shows that the transfer window is increased while avoiding noise amplification. Avoiding algorithms containing deconvolution steps may also increase the dose-efficiency of ptychographic phase reconstructions.

Transforming carbon dioxide into jet fuel using an organic combustion-synthesized Fe-Mn-K catalyst.

With mounting concerns over climate change, the utilisation or conversion of carbon dioxide into sustainable, synthetic hydrocarbons fuels, most notably for transportation purposes, continues to attract worldwide interest. This is particularly true in the search for sustainable or renewable aviation fuels. These offer considerable potential since, instead of consuming fossil crude oil, the fuels are produced from carbon dioxide using sustainable renewable hydrogen and energy. We report here a synthetic protocol to the fixation of carbon dioxide by converting it directly into aviation jet fuel using novel, inexpensive iron-based catalysts. We prepare the Fe-Mn-K catalyst by the so-called Organic Combustion Method, and the catalyst shows a carbon dioxide conversion through hydrogenation to hydrocarbons in the aviation jet fuel range of 38.2%, with a yield of 17.2%, and a selectivity of 47.8%, and with an attendant low carbon monoxide (5.6%) and methane selectivity (10.4%). The conversion reaction also produces light olefins ethylene, propylene, and butenes, totalling a yield of 8.7%, which are important raw materials for the petrochemical industry and are presently also only obtained from fossil crude oil. As this carbon dioxide is extracted from air, and re-emitted from jet fuels when combusted in flight, the overall effect is a carbon-neutral fuel. This contrasts with jet fuels produced from hydrocarbon fossil sources where the combustion process unlocks the fossil carbon and places it into the atmosphere, in longevity, as aerial carbon - carbon dioxide.

Low-dose phase retrieval of biological specimens using cryo-electron ptychography.

Cryo-electron microscopy is an essential tool for high-resolution structural studies of biological systems. This method relies on the use of phase contrast imaging at high defocus to improve information transfer at low spatial frequencies at the expense of higher spatial frequencies. Here we demonstrate that electron ptychography can recover the phase of the specimen with continuous information transfer across a wide range of the spatial frequency spectrum, with improved transfer at lower spatial frequencies, and as such is more efficient for phase recovery than conventional phase contrast imaging. We further show that the method can be used to study frozen-hydrated specimens of rotavirus double-layered particles and HIV-1 virus-like particles under low-dose conditions (5.7 e/Å2) and heterogeneous objects in an Adenovirus-infected cell over large fields of view (1.14 × 1.14 µm), thus making it suitable for studies of many biologically important structures.