Roadmap on emerging hardware and technology for machine learning.

Recent progress in artificial intelligence is largely attributed to the rapid development of machine learning, especially in the algorithm and neural network models. However, it is the performance of the hardware, in particular the energy efficiency of a computing system that sets the fundamental limit of the capability of machine learning. Data-centric computing requires a revolution in hardware systems, since traditional digital computers based on transistors and the von Neumann architecture were not purposely designed for neuromorphic computing. A hardware platform based on emerging devices and new architecture is the hope for future computing with dramatically improved throughput and energy efficiency. Building such a system, nevertheless, faces a number of challenges, ranging from materials selection, device optimization, circuit fabrication and system integration, to name a few. The aim of this Roadmap is to present a snapshot of emerging hardware technologies that are potentially beneficial for machine learning, providing the Nanotechnology readers with a perspective of challenges and opportunities in this burgeoning field.

Nanoscale Mapping of the Double Layer Potential at the Graphene-Electrolyte Interface.

The electrical double layer (EDL) governs the operation of multiple electrochemical devices, determines reaction potentials, and conditions ion transport through cellular membranes in living organisms. The few existing methods of EDL probing have low spatial resolution, usually only providing spatially averaged information. On the other hand, traditional Kelvin probe force microscopy (KPFM) is capable of mapping potential with nanoscale lateral resolution but cannot be used in electrolytes with concentrations higher than several mmol/L. Here, we resolve this experimental impediment by combining KPFM with graphene-capped electrolytic cells to quantitatively measure the potential drop across the EDL in aqueous electrolytes of decimolar and molar concentrations with a high lateral resolution. The surface potential of graphene in contact with deionized water and 0.1 mol/L solutions of CuSO4 and MgSO4 as a function of counter electrode voltage is reported. The measurements are supported by numerical modeling to reveal the role of the graphene membrane in potential screening and to determine the EDL potential drop. The proposed approach proves to be especially useful for imaging spatially inhomogeneous systems, such as nanoparticles submerged in an electrolyte solution. It could be suitable for in operando and in vivo measurements of the potential drop in the EDL on the surfaces of nanocatalysts and biological cells in equilibrium with liquid solutions.

High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm.

Using memristive properties common for titanium dioxide thin film devices, we designed a simple write algorithm to tune device conductance at a specific bias point to 1% relative accuracy (which is roughly equivalent to seven-bit precision) within its dynamic range even in the presence of large variations in switching behavior. The high precision state is nonvolatile and the results are likely to be sustained for nanoscale memristive devices because of the inherent filamentary nature of the resistive switching. The proposed functionality of memristive devices is especially attractive for analog computing with low precision data. As one representative example we demonstrate hybrid circuitry consisting of an integrated circuit summing amplifier and two memristive devices to perform the analog multiply-and-add (dot-product) computation, which is a typical bottleneck operation in information processing.

Gradient Decomposition Methods for Training Neural Networks With Non-ideal Synaptic Devices.

While promising for high-capacity machine learning accelerators, memristor devices have non-idealities that prevent software-equivalent accuracies when used for online training. This work uses a combination of Mini-Batch Gradient Descent (MBGD) to average gradients, stochastic rounding to avoid vanishing weight updates, and decomposition methods to keep the memory overhead low during mini-batch training. Since the weight update has to be transferred to the memristor matrices efficiently, we also investigate the impact of reconstructing the gradient matrixes both internally (rank-seq) and externally (rank-sum) to the memristor array. Our results show that streaming batch principal component analysis (streaming batch PCA) and non-negative matrix factorization (NMF) decomposition algorithms can achieve near MBGD accuracy in a memristor-based multi-layer perceptron trained on the MNIST (Modified National Institute of Standards and Technology) database with only 3 to 10 ranks at significant memory savings. Moreover, NMF rank-seq outperforms streaming batch PCA rank-seq at low-ranks making it more suitable for hardware implementation in future memristor-based accelerators.

Room Temperature Skyrmions in Strain-Engineered FeGe thin films.

Skyrmions hold great promise for low-energy consumption and stable high density information storage, and stabilization of the skyrmion lattice (SkX) phase at or above room temperature is greatly desired for practical use. The topological Hall effect can be used to identify candidate systems above room temperature, a challenging regime for direct observation by Lorentz electron microscopy. Atomically ordered FeGe thin films are grown epitaxially on Ge(111) substrates with ~ 4 % tensile strain. Magnetic characterization reveals enhancement of Curie temperature to 350 K due to strain, well above the bulk value of 278 K. Strong topological Hall effect was observed between 10 K and 330 K, with a significant increase in magnitude observed at 330 K. The increase in magnitude occurs just below the Curie temperature, a similar relative temperature position as the onset of Skx phase in bulk FeGe. The results suggest that strained FeGe films may host a SkX phase above room temperature when significant tensile strain is applied.

Streaming Batch Eigenupdates for Hardware Neural Networks.

Neural networks based on nanodevices, such as metal oxide memristors, phase change memories, and flash memory cells, have generated considerable interest for their increased energy efficiency and density in comparison to graphics processing units (GPUs) and central processing units (CPUs). Though immense acceleration of the training process can be achieved by leveraging the fact that the time complexity of training does not scale with the network size, it is limited by the space complexity of stochastic gradient descent, which grows quadratically. The main objective of this work is to reduce this space complexity by using low-rank approximations of stochastic gradient descent. This low spatial complexity combined with streaming methods allows for significant reductions in memory and compute overhead, opening the door for improvements in area, time and energy efficiency of training. We refer to this algorithm and architecture to implement it as the streaming batch eigenupdate (SBE) approach.

Spontaneous current constriction in threshold switching devices.

Threshold switching devices are of increasing importance for a number of applications including solid-state memories and neuromorphic circuits. Their non-linear characteristics are thought to be associated with a spontaneous (occurring without an apparent external stimulus) current flow constriction but the extent and the underlying mechanism are a subject of debate. Here we use Scanning Joule Expansion Microscopy to demonstrate that, in functional layers with thermally activated electrical conductivity, the current spontaneously and gradually constricts when a device is biased into the negative differential resistance region. We also show that the S-type negative differential resistance I-V characteristics are only a subset of possible solutions and it is possible to have multiple current density distributions corresponding to the same value of the device voltage. In materials with steep dependence of current on temperature the current constriction can occur in nanoscale devices, making this effect relevant for computing applications.

Publisher Correction: Stateful characterization of resistive switching TiO2 with electron beam induced currents.

The original version of this Article contained an error in Eq. 1. The arrows between the symbols "T" and "B", and "B" and "T", were written "↔" but should have been "→", and incorrectly read: IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT↔B+IESEEB↔T The correct from of the Eq. 1 is as follows:IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT→B+IESEEB→T This has now been corrected in both the PDF and HTML versions of the article.

Stateful characterization of resistive switching TiO2 with electron beam induced currents.

Metal oxide resistive switches are increasingly important as possible artificial synapses in next-generation neuromorphic networks. Nevertheless, there is still no codified set of tools for studying properties of the devices. To this end, we demonstrate electron beam-induced current measurements as a powerful method to monitor the development of local resistive switching in TiO2-based devices. By comparing beam energy-dependent electron beam-induced currents with Monte Carlo simulations of the energy absorption in different device layers, it is possible to deconstruct the origins of filament image formation and relate this to both morphological changes and the state of the switch. By clarifying the contrast mechanisms in electron beam-induced current microscopy, it is possible to gain new insights into the scaling of the resistive switching phenomenon and observe the formation of a current leakage region around the switching filament. Additionally, analysis of symmetric device structures reveals propagating polarization domains.

Modeling and Experimental Demonstration of a Hopfield Network Analog-to-Digital Converter with Hybrid CMOS/Memristor Circuits.

The purpose of this work was to demonstrate the feasibility of building recurrent artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memristor circuits. To do so, we modeled a Hopfield network implementing an analog-to-digital converter (ADC) with up to 8 bits of precision. Major shortcomings affecting the ADC's precision, such as the non-ideal behavior of CMOS circuitry and the specific limitations of memristors, were investigated and an effective solution was proposed, capitalizing on the in-field programmability of memristors. The theoretical work was validated experimentally by demonstrating the successful operation of a 4-bit ADC circuit implemented with discrete Pt/TiO2- x /Pt memristors and CMOS integrated circuit components.

Resistive switching and its suppression in Pt/Nb:SrTiO3 junctions.

Oxide-based resistive switching devices are promising candidates for new memory and computing technologies. Poor understanding of the defect-based mechanisms that give rise to resistive switching is a major impediment for engineering reliable and reproducible devices. Here we identify an unintentional interface layer as the origin of resistive switching in Pt/Nb:SrTiO3 junctions. We clarify the microscopic mechanisms by which the interface layer controls the resistive switching. We show that appropriate interface processing can eliminate this contribution. These findings are an important step towards engineering more reliable resistive switching devices.