Ptychographic phase retrieval via a deep-learning-assisted iterative algorithm.

Ptychography is a powerful computational imaging technique with microscopic imaging capability and adaptability to various specimens. To obtain an imaging result, it requires a phase-retrieval algorithm whose performance directly determines the imaging quality. Recently, deep neural network (DNN)-based phase retrieval has been proposed to improve the imaging quality from the ordinary model-based iterative algorithms. However, the DNN-based methods have some limitations because of the sensitivity to changes in experimental conditions and the difficulty of collecting enough measured specimen images for training the DNN. To overcome these limitations, a ptychographic phase-retrieval algorithm that combines model-based and DNN-based approaches is proposed. This method exploits a DNN-based denoiser to assist an iterative algorithm like ePIE in finding better reconstruction images. This combination of DNN and iterative algorithms allows the measurement model to be explicitly incorporated into the DNN-based approach, improving its robustness to changes in experimental conditions. Furthermore, to circumvent the difficulty of collecting the training data, it is proposed that the DNN-based denoiser be trained without using actual measured specimen images but using a formula-driven supervised approach that systemically generates synthetic images. In experiments using simulation based on a hard X-ray ptychographic measurement system, the imaging capability of the proposed method was evaluated by comparing it with ePIE and rPIE. These results demonstrated that the proposed method was able to reconstruct higher-spatial-resolution images with half the number of iterations required by ePIE and rPIE, even for data with low illumination intensity. Also, the proposed method was shown to be robust to its hyperparameters. In addition, the proposed method was applied to ptychographic datasets of a Simens star chart and ink toner particles measured at SPring-8 BL24XU, which confirmed that it can successfully reconstruct images from measurement scans with a lower overlap ratio of the illumination regions than is required by ePIE and rPIE.

Fourier Ptychographic Coherent Anti-Stokes Raman Scattering Microscopy with Point-Scanning for Super-Resolution Imaging.

Fourier ptychography (FP) is a high resolution wide-field imaging method based on the extended aperture in the Fourier space, which is synthesized from raw images with varying illumination angles. If FP is extended to coherent nonlinear optical imaging, the resolution could be further improved due to the increase of the cutoff frequency of the synthesized coherent optical transfer function (C-OTF) with respect to the order of nonlinear optical processes. However, there is a fundamental conflict between wide-field FP and nonlinear optical imaging, whereby the nonlinear optical imaging typically requires a focused excitation laser beam with high power density. To tackle the problem, in this work, a unique point-scanning FP (PS-FP) method is presented for super-resolution nonlinear optical imaging, in which the nonlinear optical signal is obtained by using focused laser beam, while the conventional FP algorithm can still be used to retrieve the super-resolution image. PS-FP coherent anti-Stokes Raman scattering (PS-FP-CARS) imaging on a variety of samples, where a 1.8-fold expansion of the OTF is achieved experimentally for enhancing vibrational imaging. Further theoretical calculation shows that the C-OTF of PS-FP higher-order CARS (PS-FP-HO-CARS) can be expanded up to ≈4.9-fold, thereby improving the spatial resolution by ≈3-fold in comparison with conventional point-scanning CARS with under tightly focused beams. The generality of PS-FP method developed in this work can be adapted to other coherent nonlinear optical imaging modalities for super-resolution imaging in tissue and cells.

Three-Dimensional Nanoscale Imaging of SiO2 Nanofiller in Styrene-Butadiene Rubber with High-Resolution and High-Sensitivity Ptychographic X-ray Computed Tomography.

SiO2 aggregates in styrene-butadiene rubber (SBR) were observed using ptychographic X-ray computed tomography (PXCT). The rubber composites were illuminated with X-rays focused by total reflection focusing mirrors, and the ptychographic diffraction patterns were collected using a CITIUS detector in the range of -75° to +75° angle of incidence. The projection images of the rubber composites were reconstructed with a two-dimensional resolution of 76 nm, and no significant structural changes were observed during the PXCT measurements. A three-dimensional image of the rubber composite was reconstructed with an isotropic resolution of 98 nm. Segmentation of SiO2 from the SBR, based on a histogram analysis of the phase shift, revealed a fragmented network structure of interconnected SiO2 aggregates.

Low-voltage single-atom electron microscopy with carbon-based nanomaterials.

The properties of materials are strongly correlated with their atomic scale structures. Achieving a comprehensive understanding of the atomic-scale structure-property relationship requires advancements of imaging and spectroscopy techniques. Aberration-corrected scanning transmission electron microscopy (STEM) has seen rapid development over the past decades and is now routinely employed for atomic-scale characterization. However, quantitative STEM imaging and spectroscopy analysis at the single-atom level is challenging due to the extremely weak signals generated from individual atom, thus imposing stringent requirements for analysis sensitivity. This review discusses the development and application of low-voltage STEM techniques with single-atom sensitivity, primarily based on recent research presented on an invited talk at the 5th 2D23 SALVE Symposium, including annular dark-field (ADF) imaging, functional imaging and electron energy-loss spectroscopy (EELS) analysis. Carbon-based nanomaterials were chosen as model systems for demonstrating the capabilities of single-atom STEM imaging and EELS analysis, due to their structural stability under low accelerating voltages and their rich physical and chemical properties. Moreover, this review summarizes recent advancements and applications of low-voltage single-atom STEM imaging and spectroscopy in the study of functional materials and discusses prospects for future developments.

New achievements in orbital angular momentum beam characterization using a Hartmann wavefront sensor and the Kirkpatrick-Baez active optical system KAOS.

Advances in physics have been significantly driven by state-of-the-art technology, and in photonics and X-ray science this calls for the ability to manipulate the characteristics of optical beams. Orbital angular momentum (OAM) beams hold substantial promise in various domains such as ultra-high-capacity optical communication, rotating body detection, optical tweezers, laser processing, super-resolution imaging etc. Hence, the advancement of OAM beam-generation technology and the enhancement of its technical proficiency and characterization capabilities are of paramount importance. These endeavours will not only facilitate the use of OAM beams in the aforementioned sectors but also extend the scope of applications in diverse fields related to OAM beams. At the FERMI Free-Electron Laser (Trieste, Italy), OAM beams are generated either by tailoring the emission process on the undulator side or, in most cases, by coupling a spiral zone plate (SZP) in tandem with the refocusing Kirkpatrick-Baez active optic system (KAOS). To provide a robust and reproducible workflow to users, a Hartmann wavefront sensor (WFS) is used for both optics tuning and beam characterization. KAOS is capable of delivering both tightly focused and broad spots, with independent control over vertical and horizontal magnification. This study explores a novel non-conventional `near collimation' operational mode aimed at generating beams with OAM that employs the use of a lithographically manufactured SZP to achieve this goal. The article evaluates the mirror's performance through Hartmann wavefront sensing, offers a discussion of data analysis methodologies, and provides a quantitative analysis of these results with ptychographic reconstructions.

A distributed software system for integrating data-intensive imaging methods in a hard X-ray nanoprobe beamline at the SSRF.

The development of hard X-ray nanoprobe techniques has given rise to a number of experimental methods, like nano-XAS, nano-XRD, nano-XRF, ptychography and tomography. Each method has its own unique data processing algorithms. With the increase in data acquisition rate, the large amount of generated data is now a big challenge to these algorithms. In this work, an intuitive, user-friendly software system is introduced to integrate and manage these algorithms; by taking advantage of the loosely coupled, component-based design approach of the system, the data processing speed of the imaging algorithm is enhanced through optimization of the parallelism efficiency. This study provides meaningful solutions to tackle complexity challenges faced in synchrotron data processing.

Morphology of Copper Nanofoams for Radiation Hydrodynamics and Fusion Applications Investigated by 3D Ptychotomography.

The performance of metal and polymer foams used in inertial confinement fusion (ICF), inertial fusion energy (IFE), and high-energy-density (HED) experiments is currently limited by our understanding of their nanostructure and its variation in bulk material. We utilized an X-ray-free electron laser (XFEL) together with lensless X-ray imaging techniques to probe the 3D morphology of copper foams at nanoscale resolution (28 nm). The observed morphology of the thin shells is more varied than expected from previous characterizations, with a large number of them distorted, merged, or open, and a targeted mass density 14% less than calculated. This nanoscale information can be used to directly inform and improve foam modeling and fabrication methods to create a tailored material response for HED experiments.

Advancing Single-Particle Analysis in Synthetic Chemical Systems: A Forward-Looking Discussion.

Single-particle analysis (SPA) is a fundamental method of cryo-electron microscopy developed to resolve the structures of biological macromolecules. This method has seen significant success in structural biology, yet its potential applications in synthetic chemical systems remain underexplored. In this perspective article, SPA and associated electron microscopy techniques are first briefly introduced. It is then proposed that SPA is well-suited for structural analysis of chemical systems where discrete, identical macromolecules can be readily obtained. Applicable systems include various clusters such as coinage metal clusters, metal-oxo/sulfur clusters, metal-organic clusters, and supramolecular compounds like coordination cages and metallo-supramolecular cages. When high-quality large single crystals are unattainable, SPA provides an alternative method for determining their structures. Beyond these end products, it is suggested that SPA can be instrumental in studying synthetic intermediates of materials with specific building units, such as metal-organic frameworks and zeolites. Given that various intermediates coexist in the reaction system, a purification step is necessary before conducting SPA, which can be facilitated by soft-landing electrospray ionization mass spectrometry.

Subgradient-projection-based stable phase-retrieval algorithm for X-ray ptychography.

X-ray ptychography is a lensless imaging technique that visualizes the nano-structure of a thick specimen which cannot be observed with an electron microscope. It reconstructs a complex-valued refractive index of the specimen from observed diffraction patterns. This reconstruction problem is called phase retrieval (PR). For further improvement in the imaging capability, including expansion of the depth of field, various PR algorithms have been proposed. Since a high-quality PR method is built upon a base PR algorithm such as ePIE, developing a well performing base PR algorithm is important. This paper proposes an improved iterative algorithm named CRISP. It exploits subgradient projection which allows adaptive step size and can be expected to avoid yielding a poor image. The proposed algorithm was compared with ePIE, which is a simple and fast-convergence algorithm, and its modified algorithm, rPIE. The experiments confirmed that the proposed method improved the reconstruction performance for both simulation and real data.

Imaging Negative Charge around Single Vanadium Dopant Atoms in Monolayer Tungsten Diselenide Using 4D Scanning Transmission Electron Microscopy.

There has been extensive activity exploring the doping of semiconducting two-dimensional (2D) transition metal dichalcogenides in order to tune their electronic and magnetic properties. The outcome of doping depends on various factors, including the intrinsic properties of the host material, the nature of the dopants used, their spatial distribution, as well as their interactions with other types of defects. A thorough atomic-level analysis is essential to fully understand these mechanisms. In this work, the vanadium-doped WSe2 monolayer grown by molecular beam epitaxy is investigated using four-dimensional scanning transmission electron microscopy (4D-STEM). Through center-of-mass-based reconstruction, atomic-scale maps are produced, allowing the visualization of both the electric field and the electrostatic potential around individual V atoms. To provide quantitative insights, these results are successfully compared to multislice image simulations based on ab initio calculations, accounting for lens aberrations. Finally, a negative charge around the V dopants is detected as a drop in the electrostatic potential, unambiguously demonstrating that 4D-STEM can be used to detect and to accurately analyze single-dopant charge states in semiconducting 2D materials.

Simultaneous Multifocal Plane Fourier Ptychographic Microscopy Utilizing a Standard RGB Camera.

Fourier ptychographic microscopy (FPM) is a computational imaging technology that can acquire high-resolution large-area images for applications ranging from biology to microelectronics. In this study, we utilize multifocal plane imaging to enhance the existing FPM technology. Using an RGB light emitting diode (LED) array to illuminate the sample, raw images are captured using a color camera. Then, exploiting the basic optical principle of wavelength-dependent focal length variation, three focal plane images are extracted from the raw image through simple R, G, and B channel separation. Herein, a single aspherical lens with a numerical aperture (NA) of 0.15 was used as the objective lens, and the illumination NA used for FPM image reconstruction was 0.08. Therefore, simultaneous multifocal plane FPM with a synthetic NA of 0.23 was achieved. The multifocal imaging performance of the enhanced FPM system was then evaluated by inspecting a transparent organic light-emitting diode (OLED) sample. The FPM system was able to simultaneously inspect the individual OLED pixels as well as the surface of the encapsulating glass substrate by separating R, G, and B channel images from the raw image, which was taken in one shot.

Multi-scale structural characterization of ceramic-based photonic glasses for structural colors.

Structural colors arise from selective light interaction with (nano)structures, which give them advantages over pigmented colors such as resistance to fading and possibility to be fabricated out of traditional low-cost and non-toxic materials. Since the color arises from the photonic (nano)structures, different structural features can impact their photonic response and thus, their color. Therefore, the detailed characterization of their structural features is crucial for further improvement of structural colors. In this work, we present a detailed multi-scale structural characterization of ceramic-based photonic glasses by using a combination of high-resolution ptychographic X-ray computed tomography and small angle X-ray scattering. Our results uncover the structure-processing-properties' relationships of such nanoparticles-based photonic glasses and point out to the need of a review of the structural features used in simulation models concomitantly with the need for further investigations by experimentalists, where we point out exactly which structural features need to be improved.

Near-field electron ptychography using full-field structured illumination.

A new configuration for near-field ptychography using a full-field illumination with a structured electron beam is proposed. A structured electron beam illuminating the entire field of view is scanned over the specimen, and a series of in-line holograms formed in the near-field region below the specimen are collected. The structured beam is generated by a conductive film with random openings, which ensures high stability and coherence of the beam. Observation in the near-field region reduces the beam concentration that occurs in the far-field region, which contributes to accurate recording of the beam intensity with a finite dynamic range of the detectors. The use of full-field illumination prevents the accumulation of errors caused by concatenating the local structures, which is the method used in conventional reconstruction. Since all holograms are obtained from the entire field of view, they have uniform multiplicity in terms of specimen information within the field of view. This contributes to robust and efficient reconstruction for a large field of view. The proposed method was tested using both simulated and experimental holograms. For the simulated holograms, the reconstruction of the specimen transmission function was achieved with an error less than 1/3485 of the wavelength. The method was further validated using experimental holograms obtained from MgO particles. The reconstructed phase transmission function of the specimen was consistent with the specimen structure and was equivalent to a mean inner potential of V on the MgO particle, which is in close agreement with previously reported values.

Enhanced tissue slide imaging in the complex domain via cross-explainable GAN for Fourier ptychographic microscopy.

Achieving microscopy with large space-bandwidth products plays a key role in diagnostic imaging and is widely significant in the overall field of clinical practice. Among quantitative microscopy techniques, Fourier Ptychography (FP) provides a wide field of view and high-resolution images, suitable to the histopathological field, but onerous in computational terms. Artificial intelligence can be a solution in this sense. In particular, this research delves into the application of Generative Adversarial Networks (GAN) for the dual-channel complex FP image enhancement of human kidney samples. The study underscores the GANs' efficacy in promoting biological architectures in FP domain, thereby still guaranteeing high resolution and visibility of detailed microscopic structures. We demonstrate successful GAN-based enhanced reconstruction through two strategies: cross-explainability and expert survey. The cross-explainability is evaluated through the comparison of explanation maps for both real and imaginary components underlining its robustness. This comparison further shows that their interplay is pivotal for accurate reconstruction without hallucinations. Secondly, the enhanced reconstruction accuracy and effectiveness in a clinical workflow are confirmed through a two-step survey conducted with nephrologists.

A Methodology for Robust Multislice Ptychography.

While multislice electron ptychography can provide thermal vibration limited resolution and structural information in 3D, it relies on properly selecting many intertwined acquisition and computational parameters. Here, we outline a methodology for selecting acquisition parameters to enable robust ptychographic reconstructions. We develop two physically informed metrics, areal oversampling and Ronchigram magnification, to describe the selection of these parameters in multislice ptychography. Through simulations, we comprehensively evaluate the validity of these two metrics over a broad range of conditions and show that they accurately guide reconstruction success. Further, we validate these conclusions with experimental ptychographic data and demonstrate close agreement between trends in simulated and experimental data. Using these metrics, we achieve experimental multislice reconstructions at a scan step of 2.1Å/px, enabling large field-of-view, data-efficient reconstructions. These experimental design principles enable the routine and robust use of multislice ptychography for 3D characterization of materials at the atomic scale.

X-Ray Multibeam Ptychography at up to 20 keV: Nano-Lithography Enhances X-Ray Nano-Imaging.

Hard X-rays are needed for non-destructive nano-imaging of solid matter. Synchrotron radiation facilities (SRF) provide the highest-quality images with single-digit nm resolution using advanced techniques such as X-ray ptychography. However, the resolution or field of view is ultimately constrained by the available coherent flux. To address this, the beam's incoherent fraction can be exploited using multiple parallel beams in an X-ray multibeam ptychography (MBP) approach. This expands the domain of X-ray ptychography to larger samples or more rapid measurements. Both qualities favor the study of complex composite or functional samples, such as catalysts, energy materials, or electronic devices. The challenge of performing ptychography at high energy and with many parallel beams must be overcome to extract the full advantages for extended samples while minimizing beam attenuation. Here, that challenge is overcome by creating a lens array using cutting-edge laser printing technology and applying it to perform scanning with MBP with up to 12 beams and at photon energies of 13 and 20 keV. This exceeds the measurement limits of conventional hard X-ray ptychography without compromising image quality for various samples: Siemens star test pattern, Ni/Al2O3 catalyst, microchip, and gold nano-crystal clusters.

High-speed 4-dimensional scanning transmission electron microscopy using compressive sensing techniques.

Here we show that compressive sensing allows 4-dimensional (4-D) STEM data to be obtained and accurately reconstructed with both high-speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4-D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal-to-noise ratio in the recovered phase using 0.3% of the total data. Lay abstract: Four-dimensional scanning transmission electron microscopy (4-D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2-dimensional signal is acquired at each 2-D probe location, equating to a 4-D dataset. Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4-D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256^2 probe locations can take on the order of 30s, approximately 100-1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4-D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination. Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10-100 times. The acquired data is then processed to fill-in or inpaint the missing data, taking advantage of the inherently low-complex signals which can be linearly combined to recover the information. In this work, similar methods are demonstrated for the acquisition of 4-D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4-D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100-300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.

First X-ray spectral ptychography and resonant ptychographic computed tomography experiments at the SWING beamline from Synchrotron SOLEIL.

X-ray ptychography and ptychographic computed tomography have seen a rapid rise since the advent of fourth-generation synchrotrons with a high degree of coherent radiation. In addition to quantitative multiscale structural analysis, ptychography with spectral capabilities has been developed, allowing for spatial-localized multiscale structural and spectral information of samples. The SWING beamline of Synchrotron SOLEIL has recently developed a nanoprobe setup where the endstation's first spectral and resonant ptychographic measurements have been successfully conducted. A metallic nickel wire sample was measured using 2D spectral ptychography in XANES mode and resonant ptychographic tomography. From the 2D spectral ptychography measurements, the spectra of the components of the sample's complex-valued refractive index, δ and ß, were extracted, integrated along the sample thickness. By performing resonance ptychographic tomography at two photon energies, 3D maps of the refractive index decrement, δ, were obtained at the Ni K-edge energy and another energy above the edge. These maps allowed the detection of impurities in the Ni wire. The significance of accounting for the atomic scattering factor is demonstrated in the calculation of electron density near a resonance through the use of the δ values. These results indicate that at the SWING beamline it is possible to conduct state-of-the-art spectral and resonant ptychography experiments using the nanoprobe setup.

Multimodal 2D and 3D microscopic mapping of growth cartilage by computational imaging techniques - a short review including new research.

Being able to image the microstructure of growth cartilage is important for understanding the onset and progression of diseases such as osteochondrosis and osteoarthritis, as well as for developing new treatments and implants. Studies of cartilage using conventional optical brightfield microscopy rely heavily on histological staining, where the added chemicals provide tissue-specific colours. Other microscopy contrast mechanisms include polarization, phase- and scattering contrast, enabling non-stained or 'label-free' imaging that significantly simplifies the sample preparation, thereby also reducing the risk of artefacts. Traditional high-performance microscopes tend to be both bulky and expensive.Computational imagingdenotes a range of techniques where computers with dedicated algorithms are used as an integral part of the image formation process. Computational imaging offers many advantages like 3D measurements, aberration correction and quantitative phase contrast, often combined with comparably cheap and compact hardware. X-ray microscopy is also progressing rapidly, in certain ways trailing the development of optical microscopy. In this study, we first briefly review the structures of growth cartilage and relevant microscopy characterization techniques, with an emphasis on Fourier ptychographic microscopy (FPM) and advanced x-ray microscopies. We next demonstrate with our own results computational imaging through FPM and compare the images with hematoxylin eosin and saffron (HES)-stained histology. Zernike phase contrast, and the nonlinear optical microscopy techniques of second harmonic generation (SHG) and two-photon excitation fluorescence (TPEF) are explored. Furthermore, X-ray attenuation-, phase- and diffraction-contrast computed tomography (CT) images of the very same sample are presented for comparisons. Future perspectives on the links to artificial intelligence, dynamic studies andin vivopossibilities conclude the article.

Aberration Correction for Large-Angle Illumination Scanning Transmission Electron Microscopy by Using Iterative Electron Ptychography Algorithms.

Modern aberration correctors in the scanning transmission electron microscope (STEM) have dramatically improved the attainable spatial resolution and enabled atomical structure and spectroscopic analysis even at low acceleration voltages (≤80 kV). For a large-angle illumination, achieving successful aberration correction to high angles is challenging with an aberration corrector, which limits further improvements in applications such as super-resolution, three-dimensional atomic depth resolution, or atomic surface morphology analyses. Electron ptychography based on four-dimensional STEM can provide a postprocessing strategy to overcome the current technological limitations. In this work, we have demonstrated that aberration correction for large-angle illumination is feasible by pushing the capabilities of regularized ptychographic iterative engine algorithms to reconstruct 4D data sets acquired using a relatively low-efficiency complementary metal oxide semiconductor camera. We report super resolution (0.71 Å) with large-angle illumination (50-60 mrad) and under 60 kV accelerating voltage.