Roadmap on emerging hardware and technology for machine learning.

Recent progress in artificial intelligence is largely attributed to the rapid development of machine learning, especially in the algorithm and neural network models. However, it is the performance of the hardware, in particular the energy efficiency of a computing system that sets the fundamental limit of the capability of machine learning. Data-centric computing requires a revolution in hardware systems, since traditional digital computers based on transistors and the von Neumann architecture were not purposely designed for neuromorphic computing. A hardware platform based on emerging devices and new architecture is the hope for future computing with dramatically improved throughput and energy efficiency. Building such a system, nevertheless, faces a number of challenges, ranging from materials selection, device optimization, circuit fabrication and system integration, to name a few. The aim of this Roadmap is to present a snapshot of emerging hardware technologies that are potentially beneficial for machine learning, providing the Nanotechnology readers with a perspective of challenges and opportunities in this burgeoning field.

The missing memristor found.

Anyone who ever took an electronics laboratory class will be familiar with the fundamental passive circuit elements: the resistor, the capacitor and the inductor. However, in 1971 Leon Chua reasoned from symmetry arguments that there should be a fourth fundamental element, which he called a memristor (short for memory resistor). Although he showed that such an element has many interesting and valuable circuit properties, until now no one has presented either a useful physical model or an example of a memristor. Here we show, using a simple analytical example, that memristance arises naturally in nanoscale systems in which solid-state electronic and ionic transport are coupled under an external bias voltage. These results serve as the foundation for understanding a wide range of hysteretic current-voltage behaviour observed in many nanoscale electronic devices that involve the motion of charged atomic or molecular species, in particular certain titanium dioxide cross-point switches.

Mechanical control of electroresistive switching.

Hysteretic metal-insulator transitions (MIT) mediated by ionic dynamics or ferroic phase transitions underpin emergent applications for nonvolatile memories and logic devices. The vast majority of applications and studies have explored the MIT coupled to the electric field or temperarture. Here, we argue that MIT coupled to ionic dynamics should be controlled by mechanical stimuli, the behavior we refer to as the piezochemical effect. We verify this effect experimentally and demonstrate that it allows both studying materials physics and enabling novel data storage technologies with mechanical writing and current-based readout.

Recent Advances and Future Prospects for Memristive Materials, Devices, and Systems.

Memristive technology has been rapidly emerging as a potential alternative to traditional CMOS technology, which is facing fundamental limitations in its development. Since oxide-based resistive switches were demonstrated as memristors in 2008, memristive devices have garnered significant attention due to their biomimetic memory properties, which promise to significantly improve power consumption in computing applications. Here, we provide a comprehensive overview of recent advances in memristive technology, including memristive devices, theory, algorithms, architectures, and systems. In addition, we discuss research directions for various applications of memristive technology including hardware accelerators for artificial intelligence, in-sensor computing, and probabilistic computing. Finally, we provide a forward-looking perspective on the future of memristive technology, outlining the challenges and opportunities for further research and innovation in this field. By providing an up-to-date overview of the state-of-the-art in memristive technology, this review aims to inform and inspire further research in this field.

Corrigendum: Applications and techniques for fast machine learning in science.

[This corrects the article DOI: 10.3389/fdata.2022.787421.].

High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm.

Using memristive properties common for titanium dioxide thin film devices, we designed a simple write algorithm to tune device conductance at a specific bias point to 1% relative accuracy (which is roughly equivalent to seven-bit precision) within its dynamic range even in the presence of large variations in switching behavior. The high precision state is nonvolatile and the results are likely to be sustained for nanoscale memristive devices because of the inherent filamentary nature of the resistive switching. The proposed functionality of memristive devices is especially attractive for analog computing with low precision data. As one representative example we demonstrate hybrid circuitry consisting of an integrated circuit summing amplifier and two memristive devices to perform the analog multiply-and-add (dot-product) computation, which is a typical bottleneck operation in information processing.

Four-dimensional address topology for circuits with stacked multilayer crossbar arrays.

We present a topological framework that provides a simple yet powerful electronic circuit architecture for constructing and using multilayer crossbar arrays, allowing a significantly increased integration density of memristive crosspoint devices beyond the scaling limits of lateral feature sizes. The truly remarkable feature of such circuits, which is an extension of the CMOL (Cmos + MOLecular-scale devices) concept for an area-like interface to a three-dimensional system, is that a large-feature-size complimentary metal-oxide-semiconductor (CMOS) substrate can provide high-density interconnects to multiple crossbar layers through a single set of vertical vias. The physical locations of the memristive devices are mapped to a four-dimensional logical address space such that unique access from the CMOS substrate is provided to every device in a stacked array of crossbars. This hybrid architecture is compatible with digital memories, field-programmable gate arrays, and biologically inspired adaptive networks and with state-of-the-art integrated circuit foundries.

Applications and Techniques for Fast Machine Learning in Science.

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science-the concept of integrating powerful ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.

The switching location of a bipolar memristor: chemical, thermal and structural mapping.

Memristors are memory resistors promising a rapid integration into future memory technologies. However, progress is still critically limited by a lack of understanding of the physical processes occurring at the nanoscale. Here we correlate device electrical characteristics with local atomic structure, chemistry and temperature. We resolved a single conducting channel that is made up of a reduced phase of the as-deposited titanium oxide. Moreover, we observed sufficient Joule heating to induce a crystallization of the oxide surrounding the channel, with a peculiar pattern that finite element simulations correlated with the existence of a hot spot close to the bottom electrode, thus identifying the switching location. This work reports direct observations in all three dimensions of the internal structure of titanium oxide memristors.

Memristor-CMOS hybrid integrated circuits for reconfigurable logic.

Hybrid reconfigurable logic circuits were fabricated by integrating memristor-based crossbars onto a foundry-built CMOS (complementary metal-oxide-semiconductor) platform using nanoimprint lithography, as well as materials and processes that were compatible with the CMOS. Titanium dioxide thin-film memristors served as the configuration bits and switches in a data routing network and were connected to gate-level CMOS components that acted as logic elements, in a manner similar to a field programmable gate array. We analyzed the chips using a purpose-built testing system, and demonstrated the ability to configure individual devices, use them to wire up various logic gates and a flip-flop, and then reconfigure devices.

Coupled ionic and electronic transport model of thin-film semiconductor memristive behavior.

The memristor, the fourth passive circuit element, was predicted theoretically nearly 40 years ago, but we just recently demonstrated both an intentional material system and an analytical model that exhibited the properties of such a device. Here we provide a more physical model based on numerical solutions of coupled drift-diffusion equations for electrons and ions with appropriate boundary conditions. We simulate the dynamics of a two-terminal memristive device based on a semiconductor thin film with mobile dopants that are partially compensated by a small amount of immobile acceptors. We examine the mobile ion distributions, zero-bias potentials, and current-voltage characteristics of the model for both steady-state bias conditions and for dynamical switching to obtain physical insight into the transport processes responsible for memristive behavior in semiconductor films.

Capacity, Fidelity, and Noise Tolerance of Associative Spatial-Temporal Memories Based on Memristive Neuromorphic Networks.

We have calculated key characteristics of associative (content-addressable) spatial-temporal memories based on neuromorphic networks with restricted connectivity-"CrossNets." Such networks may be naturally implemented in nanoelectronic hardware using hybrid memristive circuits, which may feature extremely high energy efficiency, approaching that of biological cortical circuits, at much higher operation speed. Our numerical simulations, in some cases confirmed by analytical calculations, show that the characteristics depend substantially on the method of information recording into the memory. Of the four methods we have explored, two methods look especially promising-one based on the quadratic programming, and the other one being a specific discrete version of the gradient descent. The latter method provides a slightly lower memory capacity (at the same fidelity) then the former one, but it allows local recording, which may be more readily implemented in nanoelectronic hardware. Most importantly, at the synchronous retrieval, both methods provide a capacity higher than that of the well-known Ternary Content-Addressable Memories with the same number of nonvolatile memory cells (e.g., memristors), though the input noise immunity of the CrossNet memories is lower.

High-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays.

Potential advantages of analog- and mixed-signal nanoelectronic circuits, based on floating-gate devices with adjustable conductance, for neuromorphic computing had been realized long time ago. However, practical realizations of this approach suffered from using rudimentary floating-gate cells of relatively large area. Here, we report a prototype $28\times28$ binary-input, ten-output, three-layer neuromorphic network based on arrays of highly optimized embedded nonvolatile floating-gate cells, redesigned from a commercial 180-nm nor flash memory. All active blocks of the circuit, including 101 780 floating-gate cells, have a total area below 1 mm2. The network has shown a 94.7% classification fidelity on the common Modified National Institute of Standards and Technology benchmark, close to the 96.2% obtained in simulation. The classification of one pattern takes a sub-1- $\mu \text{s}$ time and a sub-20-nJ energy-both numbers much better than in the best reported digital implementations of the same task. Estimates show that a straightforward optimization of the hardware and its transfer to the already available 55-nm technology may increase this advantage to more than $10^{2}\times $ in speed and $10^{4}\times $ in energy efficiency.

Publisher Correction: Stateful characterization of resistive switching TiO2 with electron beam induced currents.

The original version of this Article contained an error in Eq. 1. The arrows between the symbols "T" and "B", and "B" and "T", were written "↔" but should have been "→", and incorrectly read: IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT↔B+IESEEB↔T The correct from of the Eq. 1 is as follows:IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT→B+IESEEB→T This has now been corrected in both the PDF and HTML versions of the article.

Defect-tolerant architectures for nanoelectronic crossbar memories.

We have calculated the maximum useful bit density that may be achieved by the synergy of bad bit exclusion and advanced (BCH) error correcting codes in prospective crossbar nanoelectronic memories, as a function of defective memory cell fraction. While our calculations are based on a particular ("CMOL") memory topology, with naturally segmented nanowires and an area-distributed nano/CMOS interface, for realistic parameters our results are also applicable to "global" crossbar memories with peripheral interfaces. The results indicate that the crossbar memories with a nano/CMOS pitch ratio close to 1/3 (which is typical for the current, initial stage of the nanoelectronics development) may overcome purely semiconductor memories in useful bit density if the fraction of nanodevice defects (stuck-on-faults) is below approximately 15%, even under rather tough, 30 ns upper bound on the total access time. Moreover, as the technology matures, and the pitch ratio approaches an order of magnitude, the crossbar memories may be far superior to the densest semiconductor memories by providing, e.g., a 1 Tbit/cm2 density even for a plausible defect fraction of 2%. These highly encouraging results are much better than those reported in literature earlier, including our own early work, mostly due to more advanced error correcting codes.

Stateful characterization of resistive switching TiO2 with electron beam induced currents.

Metal oxide resistive switches are increasingly important as possible artificial synapses in next-generation neuromorphic networks. Nevertheless, there is still no codified set of tools for studying properties of the devices. To this end, we demonstrate electron beam-induced current measurements as a powerful method to monitor the development of local resistive switching in TiO2-based devices. By comparing beam energy-dependent electron beam-induced currents with Monte Carlo simulations of the energy absorption in different device layers, it is possible to deconstruct the origins of filament image formation and relate this to both morphological changes and the state of the switch. By clarifying the contrast mechanisms in electron beam-induced current microscopy, it is possible to gain new insights into the scaling of the resistive switching phenomenon and observe the formation of a current leakage region around the switching filament. Additionally, analysis of symmetric device structures reveals propagating polarization domains.

Flexible three-dimensional artificial synapse networks with correlated learning and trainable memory capability.

If a three-dimensional physical electronic system emulating synapse networks could be built, that would be a significant step toward neuromorphic computing. However, the fabrication complexity of complementary metal-oxide-semiconductor architectures impedes the achievement of three-dimensional interconnectivity, high-device density, or flexibility. Here we report flexible three-dimensional artificial chemical synapse networks, in which two-terminal memristive devices, namely, electronic synapses (e-synapses), are connected by vertically stacking crossbar electrodes. The e-synapses resemble the key features of biological synapses: unilateral connection, long-term potentiation/depression, a spike-timing-dependent plasticity learning rule, paired-pulse facilitation, and ultralow-power consumption. The three-dimensional artificial synapse networks enable a direct emulation of correlated learning and trainable memory capability with strong tolerances to input faults and variations, which shows the feasibility of using them in futuristic electronic devices and can provide a physical platform for the realization of smart memories and machine learning and for operation of the complex algorithms involving hierarchical neural networks.High-density information storage calls for the development of modern electronics with multiple stacking architectures that increase the complexity of three-dimensional interconnectivity. Here, Wu et al. build a stacked yet flexible artificial synapse network using layer-by-layer solution processing.

Modeling and Experimental Demonstration of a Hopfield Network Analog-to-Digital Converter with Hybrid CMOS/Memristor Circuits.

The purpose of this work was to demonstrate the feasibility of building recurrent artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memristor circuits. To do so, we modeled a Hopfield network implementing an analog-to-digital converter (ADC) with up to 8 bits of precision. Major shortcomings affecting the ADC's precision, such as the non-ideal behavior of CMOS circuitry and the specific limitations of memristors, were investigated and an effective solution was proposed, capitalizing on the in-field programmability of memristors. The theoretical work was validated experimentally by demonstrating the successful operation of a 4-bit ADC circuit implemented with discrete Pt/TiO2- x /Pt memristors and CMOS integrated circuit components.