Autonomous Scanning Tunneling Microscopy Imaging via Deep Learning.

Scanning tunneling microscopy (STM) is a powerful technique that provides the ability to manipulate and characterize individual atoms and molecules with atomic-level precision. However, the processes of scanning samples, operating the probe, and analyzing data are typically labor-intensive and subjective. Deep learning (DL) techniques have shown immense potential in automating complex tasks and solving high-dimensional problems. In this study, we developed an autonomous STM framework powered by DL to enable autonomous operations of the STM without human interventions. Our framework employs a convolutional neural network (CNN) for real-time evaluation of STM image quality, a U-net model for identifying bare surfaces, and a deep Q-learning network (DQN) agent for autonomous probe conditioning. Additionally, we integrated an object recognition model for the automated recognition of different adsorbates. This autonomous framework enables the acquisition of space-averaging information using STM techniques without compromising the high-resolution molecular imaging. We achieved measuring an area of approximately 1.9 µm2 within 48 h of continuous measurement and automatedly generated the statistics on the molecular species present within the mesoscopic area. We demonstrate the high robustness of the framework by conducting measurements at the liquid nitrogen temperature (∼78 K). We envision that the integration of DL techniques and high-resolution microscopy will not only extend the functionality and capability of scanning probe microscopes but also accelerate the understanding and discovery of new materials.

Automated Generation and Analysis of Molecular Images Using Generative Artificial Intelligence Models.

The development of scanning probe microscopy (SPM) has enabled unprecedented scientific discoveries through high-resolution imaging. Simulations and theoretical analysis of SPM images are equally important as obtaining experimental images since their comparisons provide fruitful understandings of the structures and physical properties of the investigated systems. So far, SPM image simulations are conventionally based on quantum mechanical theories, which can take several days in tasks of large-scale systems. Here, we have developed a scanning tunneling microscopy (STM) molecular image simulation and analysis framework based on a generative adversarial model, CycleGAN. It allows efficient translations between STM data and molecular models. Our CycleGAN-based framework introduces an approach for high-fidelity STM image simulation, outperforming traditional quantum mechanical methods in efficiency and accuracy. We envision that the integration of generative networks and high-resolution molecular imaging opens avenues in materials discovery relying on SPM technologies.

Predicting Molecular Self-Assembly on Metal Surfaces Using Graph Neural Networks Based on Experimental Data Sets.

The application of supramolecular chemistry on solid surfaces has received extensive attention in the past few decades. To date, combining experiments with quantum mechanical or molecular dynamic methods represents the key strategy to explore the molecular self-assembled structures, which is, however, often laborious. Recently, machine learning (ML) has become one of the most exciting tools in material research, allowing for both efficiency and accuracy in predicting molecular properties. In this work, we constructed a graph neural network to predict the self-assembly of functional polycyclic aromatic hydrocarbons (PAHs) on metal surfaces. Using scanning tunneling microscopy (STM), we characterized the self-assembled nanostructures of a homologous series of PAH molecules on different metal surfaces to construct an experimental data set for model training. Compared with traditional ML algorithms, our model exhibits better predictive performance. Finally, the generalization of the model is further verified by comparing the ML predictions and experimental results of different functionalized molecule. Our results demonstrate training experimental data sets to produce a predictive ML model of molecular self-assembly with generalization performance, which allows for the predictive design of nanostructures with functional molecules.

Applying a Deep-Learning-Based Keypoint Detection in Analyzing Surface Nanostructures.

Scanning tunneling microscopy (STM) imaging has been routinely applied in studying surface nanostructures owing to its capability of acquiring high-resolution molecule-level images of surface nanostructures. However, the image analysis still heavily relies on manual analysis, which is often laborious and lacks uniform criteria. Recently, machine learning has emerged as a powerful tool in material science research for the automatic analysis and processing of image data. In this paper, we propose a method for analyzing molecular STM images using computer vision techniques. We develop a lightweight deep learning framework based on the YOLO algorithm by labeling molecules with its keypoints. Our framework achieves high efficiency while maintaining accuracy, enabling the recognitions of molecules and further statistical analysis. In addition, the usefulness of this model is exemplified by exploring the length of polyphenylene chains fabricated from on-surface synthesis. We foresee that computer vision methods will be frequently used in analyzing image data in the field of surface chemistry.