Realizing smart scanning transmission electron microscopy using high performance computing.

Scanning Transmission Electron Microscopy (STEM) coupled with Electron Energy Loss Spectroscopy (EELS) presents a powerful platform for detailed material characterization via rich imaging and spectroscopic data. Modern electron microscopes can access multiple length scales and sampling rates far beyond human perception and reaction time. Recent advancements in machine learning (ML) offer a promising avenue to enhance these capabilities by integrating ML algorithms into the STEM-EELS framework, fostering an environment of active learning. This work enables the seamless integration of STEM with High-Performance Computing (HPC) systems. This integration is facilitated by our developed server software, written in Python, which acts as a wrapper over DigitalMicrograph (version 3.5) hardware modules to enable remote computer interactions. We present several implemented workflows that exemplify this integration. These workflows include sophisticated techniques such as object finding and deep kernel learning. Through these developments, we demonstrate how the fusion of STEM-EELS with ML and HPC enhances the efficiency and scope of material characterization for all of STEM available globally having Gatan, Inc. image filter installed on them. The codes are available on GitHub.

Integration of scanning probe microscope with high-performance computing: Fixed-policy and reward-driven workflows implementation.

The rapid development of computation power and machine learning algorithms has paved the way for automating scientific discovery with a scanning probe microscope (SPM). The key elements toward operationalization of the automated SPM are the interface to enable SPM control from Python codes, availability of high computing power, and development of workflows for scientific discovery. Here, we build a Python interface library that enables controlling an SPM from either a local computer or a remote high-performance computer, which satisfies the high computation power need of machine learning algorithms in autonomous workflows. We further introduce a general platform to abstract the operations of SPM in scientific discovery into fixed-policy or reward-driven workflows. Our work provides a full infrastructure to build automated SPM workflows for both routine operations and autonomous scientific discovery with machine learning.

Evolution of Ferroelectric Properties in SmxBi1-xFeO3 via Automated Piezoresponse Force Microscopy across combinatorial spread libraries.

Combinatorial spread libraries offer an approach to explore the evolution of material properties over broad concentration, temperature, and growth parameter spaces. However, the traditional limitation of this approach is the requirement for the read-out of functional properties across the library. Here we develop automated piezoresponse force microscopy (PFM) for the exploration of combinatorial spread libraries and demonstrate its application in the SmxBi1-xFeO3 system with the ferroelectric-antiferroelectric morphotropic phase boundary. This approach relies on the synergy of the quantitative nature of PFM and the implementation of automated experiments that allow PFM-based sampling of macroscopic samples. The concentration dependence of pertinent ferroelectric parameters was determined and used to develop the mathematical framework based on the Ginzburg-Landau theory describing the evolution of these properties across the concentration space. We pose that a combination of automated scanning probe microscope and combinatorial spread library approach will emerge as an efficient research paradigm to close the characterization gap in high-throughput materials discovery. We make the data sets open to the community, and we hope that this will stimulate other efforts to interpret and understand the physics of these systems.

Coexistence and Interplay of Two Ferroelectric Mechanisms in Zn1-xMgxO.

Ferroelectric materials promise exceptional attributes including low power dissipation, fast operational speeds, enhanced endurance, and superior retention to revolutionize information technology. However, the practical application of ferroelectric-semiconductor memory devices has been significantly challenged by the incompatibility of traditional perovskite oxide ferroelectrics with metal-oxide-semiconductor technology. Recent discoveries of ferroelectricity in binary oxides such as Zn1-xMgxO and Hf1-xZrxO have been a focal point of research in ferroelectric information technology. This work investigates the ferroelectric properties of Zn1-xMgxO utilizing automated band excitation piezoresponse force microscopy. This findings reveal the coexistence of two ferroelectric subsystems within Zn1-xMgxO. A "fringing-ridge mechanism" of polarization switching is proposed that is characterized by initial lateral expansion of nucleation without significant propagation in depth, contradicting the conventional domain growth process observed in ferroelectrics. This unique polarization dynamics in Zn1-xMgxO suggests a new understanding of ferroelectric behavior, contributing to both the fundamental science of ferroelectrics and their application in information technology.

Bayesian Conavigation: Dynamic Designing of the Material Digital Twins via Active Learning.

Scientific advancement is universally based on the dynamic interplay between theoretical insights, modeling, and experimental discoveries. However, this feedback loop is often slow, including delayed community interactions and the gradual integration of experimental data into theoretical frameworks. This challenge is particularly exacerbated in domains dealing with high-dimensional object spaces, such as molecules and complex microstructures. Hence, the integration of theory within automated and autonomous experimental setups, or theory in the loop-automated experiment, is emerging as a crucial objective for accelerating scientific research. The critical aspect is to use not only theory but also on-the-fly theory updates during the experiment. Here, we introduce a method for integrating theory into the loop through Bayesian conavigation of theoretical model space and experimentation. Our approach leverages the concurrent development of surrogate models for both simulation and experimental domains at the rates determined by latencies and costs of experiments and computation, alongside the adjustment of control parameters within theoretical models to minimize epistemic uncertainty over the experimental object spaces. This methodology facilitates the creation of digital twins of material structures, encompassing both the surrogate model of behavior that includes the correlative part and the theoretical model itself. While being demonstrated here within the context of functional responses in ferroelectric materials, our approach holds promise for broader applications, such as the exploration of optical properties in nanoclusters, microstructure-dependent properties in complex materials, and properties of molecular systems.

Nanoscale core-shell structure and recrystallization of swift heavy ion tracks in SrTiO3.

It is widely accepted that the interaction of swift heavy ions with many complex oxides is predominantly governed by the electronic energy loss that gives rise to nanoscale amorphous ion tracks along the penetration direction. The question of how electronic excitation and electron-phonon coupling affect the atomic system through defect production, recrystallization, and strain effects has not yet been fully clarified. To advance the knowledge of the atomic structure of ion tracks, we irradiated single crystalline SrTiO3 with 629 MeV Xe ions and performed comprehensive electron microscopy investigations complemented by molecular dynamics simulations. This study shows discontinuous ion-track formation along the ion penetration path, comprising an amorphous core and a surrounding few monolayer thick shell of strained/defective crystalline SrTiO3. Using machine-learning-aided analysis of atomic-scale images, we demonstrate the presence of 4-8% strain in the disordered region interfacing with the amorphous core in the initially formed ion tracks. Under constant exposure of the electron beam during imaging, the amorphous part of the ion tracks readily recrystallizes radially inwards from the crystalline-amorphous interface under the constant electron-beam irradiation during the imaging. Cation strain in the amorphous region is observed to be significantly recovered, while the oxygen sublattice remains strained even under the electron irradiation due to the present oxygen vacancies. The molecular dynamics simulations support this observation and suggest that local transient heating and annealing facilitate recrystallization process of the amorphous phase and drive Sr and Ti sublattices to rearrange. In contrast, the annealing of O atoms is difficult, thus leaving a remnant of oxygen vacancies and strain even after recrystallization. This work provides insights for creating and transforming novel interfaces and nanostructures for future functional applications.

Direct Fabrication of Atomically Defined Pores in MXenes Using Feedback-Driven STEM.

Controlled fabrication of nanopores in 2D materials offer the means to create robust membranes needed for ion transport and nanofiltration. Techniques for creating nanopores have relied upon either plasma etching or direct irradiation; however, aberration-corrected scanning transmission electron microscopy (STEM) offers the advantage of combining a sub-Å sized electron beam for atomic manipulation along with atomic resolution imaging. Here, a method for automated nanopore fabrication is utilized with real-time atomic visualization to enhance the mechanistic understanding of beam-induced transformations. Additionally, an electron beam simulation technique, Electron-Beam Simulator (E-BeamSim) is developed to observe the atomic movements and interactions resulting from electron beam irradiation. Using the MXene Ti3C2Tx, the influence of temperature on nanopore fabrication is explored by tracking atomic transformations and find that at room temperature the electron beam irradiation induces random displacement and results in titanium pileups at the nanopore edge, which is confirmed by E-BeamSim. At elevated temperatures, after removal of the surface functional groups and with the increased mobility of atoms results in atomic transformations that lead to the selective removal of atoms layer by layer. This work can lead to the development of defect engineering techniques within functionalized MXene layers and other 2D materials.

Explainability and human intervention in autonomous scanning probe microscopy.

The broad adoption of machine learning (ML)-based autonomous experiments (AEs) in material characterization and synthesis requires strategies development for understanding and intervention in the experimental workflow. Here, we introduce and realize a post-experimental analysis strategy for deep kernel learning-based autonomous scanning probe microscopy. This approach yields real-time and post-experimental indicators for the progression of an active learning process interacting with an experimental system. We further illustrate how this approach can be applied to human-in-the-loop AEs, where human operators make high-level decisions at high latencies setting the policies for AEs, and the ML algorithm performs low-level, fast decisions. The proposed approach is universal and can be extended to other techniques and applications such as combinatorial library analysis.

Ferroelectricity in hafnia controlled via surface electrochemical state.

Ferroelectricity in binary oxides including hafnia and zirconia has riveted the attention of the scientific community due to the highly unconventional physical mechanisms and the potential for the integration of these materials into semiconductor workflows. Over the last decade, it has been argued that behaviours such as wake-up phenomena and an extreme sensitivity to electrode and processing conditions suggest that ferroelectricity in these materials is strongly influenced by other factors, including electrochemical boundary conditions and strain. Here we argue that the properties of these materials emerge due to the interplay between the bulk competition between ferroelectric and structural instabilities, similar to that in classical antiferroelectrics, coupled with non-local screening mediated by the finite density of states at surfaces and internal interfaces. Via the decoupling of electrochemical and electrostatic controls, realized via environmental and ultra-high vacuum piezoresponse force microscopy, we show that these materials demonstrate a rich spectrum of ferroic behaviours including partial-pressure-induced and temperature-induced transitions between ferroelectric and antiferroelectric behaviours. These behaviours are consistent with an antiferroionic model and suggest strategies for hafnia-based device optimization.

Disentangling Electronic Transport and Hysteresis at Individual Grain Boundaries in Hybrid Perovskites via Automated Scanning Probe Microscopy.

Underlying the rapidly increasing photovoltaic efficiency and stability of metal halide perovskites (MHPs) is the advancement in the understanding of the microstructure of polycrystalline MHP thin film. Over the past decade, intense efforts have been aimed at understanding the effect of microstructures on MHP properties, including chemical heterogeneity, strain disorder, phase impurity, etc. It has been found that grain and grain boundary (GB) are tightly related to lots of microscale and nanoscale behavior in MHP thin films. Atomic force microscopy (AFM) is widely used to observe grain and boundary structures in topography and subsequently to study the correlative surface potential and conductivity of these structures. For now, most AFM measurements have been performed in imaging mode to study the static behavior; in contrast, AFM spectroscopy mode allows us to investigate the dynamic behavior of materials, e.g., conductivity under sweeping voltage. However, a major limitation of AFM spectroscopy measurements is that they require manual operation by human operators, and as such only limited data can be obtained, hindering systematic investigations of these microstructures. In this work, we designed a workflow combining the conductive AFM measurement with a machine learning (ML) algorithm to systematically investigate grain boundaries in MHPs. The trained ML model can extract GBs locations from the topography image, and the workflow drives the AFM probe to each GB location to perform a current-voltage (IV) curve automatically. Then, we are able to have IV curves at all GB locations, allowing us to systematically understand the property of GBs. Using this method, we discovered that the GB junction points are less conductive, potentially more photoactive, and can play critical roles in MHP stability, while most previous works only focused on the difference between GB and grains.

Roadmap on Deep Learning for Microscopy.

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Autonomous scanning probe microscopy with hypothesis learning: Exploring the physics of domain switching in ferroelectric materials.

Using hypothesis-learning-driven automated scanning probe microscopy (SPM), we explore the bias-induced transformations that underpin the functionality of broad classes of devices and materials from batteries and memristors to ferroelectrics and antiferroelectrics. Optimization and design of these materials require probing the mechanisms of these transformations on the nanometer scale as a function of a broad range of control parameters, leading to experimentally intractable scenarios. Meanwhile, often these behaviors are understood within potentially competing theoretical hypotheses. Here, we develop a hypothesis list covering possible limiting scenarios for domain growth in ferroelectric materials, including thermodynamic, domain-wall pinning, and screening limited. The hypothesis-driven SPM autonomously identifies the mechanisms of bias-induced domain switching, and the results indicate that domain growth is ruled by kinetic control. We note that the hypothesis learning can be broadly used in other automated experiment settings.

Learning and Predicting Photonic Responses of Plasmonic Nanoparticle Assemblies via Dual Variational Autoencoders.

The application of machine learning is demonstrated for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. It is shown that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that hyperspectral darkfield microscopy can be used to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.