Bulk-Switching Memristor-Based Compute-In-Memory Module for Deep Neural Network Training.

The constant drive to achieve higher performance in deep neural networks (DNNs) has led to the proliferation of very large models. Model training, however, requires intensive computation time and energy. Memristor-based compute-in-memory (CIM) modules can perform vector-matrix multiplication (VMM) in place and in parallel, and have shown great promises in DNN inference applications. However, CIM-based model training faces challenges due to non-linear weight updates, device variations, and low-precision. In this work, a mixed-precision training scheme is experimentally implemented to mitigate these effects using a bulk-switching memristor-based CIM module. Low-precision CIM modules are used to accelerate the expensive VMM operations, with high-precision weight updates accumulated in digital units. Memristor devices are only changed when the accumulated weight update value exceeds a pre-defined threshold. The proposed scheme is implemented with a system-onchip of fully integrated analog CIM modules and digital sub-systems, showing fast convergence of LeNet training to 97.73%. The efficacy of training larger models is evaluated using realistic hardware parameters and verifies that CIM modules can enable efficient mix-precision DNN training with accuracy comparable to full-precision software-trained models. Additionally, models trained on chip are inherently robust to hardware variations, allowing direct mapping to CIM inference chips without additional re-training.

Memristive technologies for data storage, computation, encryption, and radio-frequency communication.

Memristive devices, which combine a resistor with memory functions such that voltage pulses can change their resistance (and hence their memory state) in a nonvolatile manner, are beginning to be implemented in integrated circuits for memory applications. However, memristive devices could have applications in many other technologies, such as non-von Neumann in-memory computing in crossbar arrays, random number generation for data security, and radio-frequency switches for mobile communications. Progress toward the integration of memristive devices in commercial solid-state electronic circuits and other potential applications will depend on performance and reliability challenges that still need to be addressed, as described here.

Investigating Selectorless Property within Niobium Devices for Storage Applications.

Resistive random-access memory (RRAM) crossbar arrays have shown significant promise as drivers of neuromorphic computing, in-memory computing, and high-density storage-class memory applications. However, leakage current through parasitic sneak paths is one of the dominant obstacles for large-scale commercial deployment of RRAM arrays. To overcome this issue without compromising on the structural simplicity, the use of inherent selectors native to switching is one of the most promising ways to reduce sneak path currents without sacrificing density associated with the simple two-electrode structure. In this study, niobium oxide (NbOx) was chosen as the resistive switching layer since it co-exhibits non-volatile memory and metal-insulator-transition selector behavior. Experimental results demonstrate abnormal phenomena in the reset process: a rapid decrease in current, followed by an increase when reset from the on state. The current conduction mechanism was examined through statistical analysis, and a conduction filament physical model was developed to explain the abnormal phenomenon. Under optimized operation conditions, non-linearity of ∼500 and fast switching speeds of 30 ns set and 50 ns reset were obtained. The switching behaviors with the intrinsic selector property make the NbOx device an attractive candidate for future memory and in-memory computing applications.

Adaptive Synaptic Memory via Lithium Ion Modulation in RRAM Devices.

Biologically plausible computing systems require fine-grain tuning of analog synaptic characteristics. In this study, lithium-doped silicate resistive random access memory with a titanium nitride (TiN) electrode mimicking biological synapses is demonstrated. Biological plausibility of this RRAM device is thought to occur due to the low ionization energy of lithium ions, which enables controllable forming and filamentary retraction spontaneously or under an applied voltage. The TiN electrode can effectively store lithium ions, a principle widely adopted from battery construction, and allows state-dependent decay to be reliably achieved. As a result, this device offers multi-bit functionality and synaptic plasticity for simulating various strengths in neuronal connections. Both short-term memory and long-term memory are emulated across dynamical timescales. Spike-timing-dependent plasticity and paired-pulse facilitation are also demonstrated. These mechanisms are capable of self-pruning to generate efficient neural networks. Time-dependent resistance decay is observed for different conductance values, which mimics both biological and artificial memory pruning and conforms to the trend of the biological brain that prunes weak synaptic connections. By faithfully emulating learning rules that exist in human's higher cortical areas from STDP to synaptic pruning, the device has the capacity to drive forward the development of highly efficient neuromorphic computing systems.

Filament-Free Bulk Resistive Memory Enables Deterministic Analogue Switching.

Digital computing is nearing its physical limits as computing needs and energy consumption rapidly increase. Analogue-memory-based neuromorphic computing can be orders of magnitude more energy efficient at data-intensive tasks like deep neural networks, but has been limited by the inaccurate and unpredictable switching of analogue resistive memory. Filamentary resistive random access memory (RRAM) suffers from stochastic switching due to the random kinetic motion of discrete defects in the nanometer-sized filament. In this work, this stochasticity is overcome by incorporating a solid electrolyte interlayer, in this case, yttria-stabilized zirconia (YSZ), toward eliminating filaments. Filament-free, bulk-RRAM cells instead store analogue states using the bulk point defect concentration, yielding predictable switching because the statistical ensemble behavior of oxygen vacancy defects is deterministic even when individual defects are stochastic. Both experiments and modeling show bulk-RRAM devices using TiO2- X switching layers and YSZ electrolytes yield deterministic and linear analogue switching for efficient inference and training. Bulk-RRAM solves many outstanding issues with memristor unpredictability that have inhibited commercialization, and can, therefore, enable unprecedented new applications for energy-efficient neuromorphic computing. Beyond RRAM, this work shows how harnessing bulk point defects in ionic materials can be used to engineer deterministic nanoelectronic materials and devices.

Memristor networks for real-time neural activity analysis.

The ability to efficiently analyze the activities of biological neural networks can significantly promote our understanding of neural communications and functionalities. However, conventional neural signal analysis approaches need to transmit and store large amounts of raw recording data, followed by extensive processing offline, posing significant challenges to the hardware and preventing real-time analysis and feedback. Here, we demonstrate a memristor-based reservoir computing (RC) system that can potentially analyze neural signals in real-time. We show that the perovskite halide-based memristor can be directly driven by emulated neural spikes, where the memristor state reflects temporal features in the neural spike train. The RC system is successfully used to recognize neural firing patterns, monitor the transition of the firing patterns, and identify neural synchronization states among different neurons. Advanced neuroelectronic systems with such memristor networks can enable efficient neural signal analysis with high spatiotemporal precision, and possibly closed-loop feedback control.

Stabilization of Mode-Dependent Impulsive Hybrid Systems Driven by DFA With Mixed-Mode Effects.

This paper is concerned with mode-dependent impulsive hybrid systems driven by deterministic finite automaton (DFA) with mixed-mode effects. In the hybrid systems, a complex phenomenon called mixed mode, caused in time-varying delay switching systems, is considered explicitly. Furthermore, mode-dependent impulses, which can exist not only at the instants coinciding with mode switching but also at the instants when there is no system switching, are also taken into consideration. First, we establish a rigorous mathematical equation expression of this class of hybrid systems. Then, several criteria of stabilization of this class of hybrid systems are presented based on semi-tensor product (STP) techniques, multiple Lyapunov-Krasovskii functionals, as well as the average dwell time approach. Finally, an example is simulated to illustrate the effectiveness of the obtained results.

In Situ Nano-thermomechanical Experiment Reveals Brittle to Ductile Transition in Silicon Nanowires.

Silicon (Si) nanostructures are widely used in microelectronics and nanotechnology. Brittle to ductile transition in nanoscale Si is of great scientific and technological interest but this phenomenon and its underlying mechanism remain elusive. By conducting in situ temperature-controlled nanomechanical testing inside a transmission electron microscope (TEM), here we show that the crystalline Si nanowires under tension are brittle at room temperature but exhibit ductile behavior with dislocation-mediated plasticity at elevated temperatures. We find that reducing the nanowire diameter promotes the dislocation-mediated responses, as shown by 78 Si nanowires tested between room temperature and 600 K. In situ high-resolution TEM imaging and atomistic reaction pathway modeling reveal that the unconventional 1/2⟨110⟩{001} dislocations become highly active with increasing temperature and thus play a critical role in the formation of deformation bands, leading to transition from brittle fracture to dislocation-mediated failure in Si nanowires at elevated temperatures. This study provides quantitative characterization and mechanistic insight for the brittle to ductile transition in Si nanostructures.

Charge Transition of Oxygen Vacancies during Resistive Switching in Oxide-Based RRAM.

Resistive random-access memory (RRAM) devices have attracted broad interest as promising building blocks for high-density nonvolatile memory and neuromorphic computing applications. Atomic level thermodynamic and kinetic descriptions of resistive switching (RS) processes are essential for continued device design and optimization but are relatively lacking for oxide-based RRAMs. It is generally accepted that RS occurs due to the redistribution of charged oxygen vacancies driven by an external electric field. However, this assumption contradicts the experimentally observed stable filaments, where the high vacancy concentration should lead to a strong Coulomb repulsion and filament instability. In this work, through predictive atomistic calculations in combination with experimental measurements, we attempt to understand the interactions between oxygen vacancies and the microscopic processes that are required for stable RS in a Ta2O5-based RRAM. We propose a model based on a series of charge transition processes that explains the drift and aggregation of vacancies during RS. The model was validated by experimental measurements where illuminated devices exhibit accelerated RS behaviors during SET and RESET. The activation energies of ion migration and charge transition were further experimentally determined through a transient current measurement, consistent with the modeling results. Our results help provide comprehensive understanding on the internal dynamics of RS and will benefit device optimization and applications.

Ionic modulation and ionic coupling effects in MoS2 devices for neuromorphic computing.

Coupled ionic-electronic effects present intriguing opportunities for device and circuit development. In particular, layered two-dimensional materials such as MoS2 offer highly anisotropic ionic transport properties, facilitating controlled ion migration and efficient ionic coupling among devices. Here, we report reversible modulation of MoS2 films that is consistent with local 2H-1T' phase transitions by controlling the migration of Li+ ions with an electric field, where an increase/decrease in the local Li+ ion concentration leads to the transition between the 2H (semiconductor) and 1T' (metal) phases. The resulting devices show excellent memristive behaviour and can be directly coupled with each other through local ionic exchange, naturally leading to synaptic competition and synaptic cooperation effects observed in biology. These results demonstrate the potential of direct modulation of two-dimensional materials through field-driven ionic processes, and can lead to future electronic and energy devices based on coupled ionic-electronic effects and biorealistic implementation of artificial neural networks.

MoS2 Memristors Exhibiting Variable Switching Characteristics toward Biorealistic Synaptic Emulation.

Memristors based on 2D layered materials could provide biorealistic ionic interactions and potentially enable construction of energy-efficient artificial neural networks capable of faithfully emulating neuronal interconnections in human brains. To build reliable 2D-material-based memristors suitable for constructing working neural networks, the memristive switching mechanisms in such memristors need to be systematically analyzed. Here, we present a study on the switching characteristics of the few-layer MoS2 memristors made by mechanical printing. First, two types of dc-programmed switching characteristics, termed rectification-mediated and conductance-mediated behaviors, are observed among different MoS2 memristors, which are attributed to the modulation of MoS2/metal Schottky barriers and redistribution of vacancies, respectively. We also found that an as-fabricated MoS2 memristor initially exhibits an analog pulse-programmed switching behavior, but it can be converted to a quasi-binary memristor with an abrupt switching behavior through an electrical stress process. Such a transition of switching characteristics is attributed to field-induced agglomeration of vacancies at MoS2/metal interfaces. The additional Kelvin probe force microscopy, Auger electron spectroscopy analysis, and electronic characterization results support this hypothesis. Finally, we fabricated a testing device consisting of two adjacent MoS2 memristors and demonstrated that these two memristors can be ionically coupled to each other. This device interconnection scheme could be exploited to build neural networks for emulating ionic interactions among neurons. This work advances the device physics for understanding the memristive properties of 2D-material-based memristors and serves as a critical foundation for building biorealistic neuromorphic computing systems based on such memristors.

K-means Data Clustering with Memristor Networks.

Memristor-based neuromorphic networks have been actively studied as a promising candidate to overcome the von-Neumann bottleneck in future computing applications. Several recent studies have demonstrated memristor network's capability to perform supervised as well as unsupervised learning, where features inherent in the input are identified and analyzed by comparing with features stored in the memristor network. However, even though in some cases the stored feature vectors can be normalized so that the winning neurons can be directly found by the (input) vector-(stored) vector dot-products, in many other cases, normalization of the feature vectors is not trivial or practically feasible, and calculation of the actual Euclidean distance between the input vector and the stored vector is required. Here we report experimental implementation of memristor crossbar hardware systems that can allow direct comparison of the Euclidean distances without normalizing the weights. The experimental system enables unsupervised K-means clustering algorithm through online learning, and produces high classification accuracy (93.3%) for the standard IRIS data set. The approaches and devices can be used in other unsupervised learning systems, and significantly broaden the range of problems a memristor-based network can solve.

Optogenetics-Inspired Tunable Synaptic Functions in Memristors.

Two-terminal memristors with internal Ca2+-like dynamics can be used to faithfully emulate biological synaptic functions and have been intensively studied for neural network implementations. Inspired by the optogenetic technique that utilizes light to tune the Ca2+ dynamics and subsequently the synaptic plasticity, we develop a CH3NH3PbI3 (MAPbI3)-based memristor that exhibits light-tunable synaptic behaviors. Specifically, we show that by increasing the formation energy of iodine vacancy (VI·/VI×), light illumination can be used to control the VI·/VI× generation and annihilation dynamics, resembling light-controlled Ca2+ influx in biological synapses. We demonstrate that the memory formation and memory loss behaviors in the memristors can be modified by controlling the intensity and the wavelength of the illuminated light. Coincidence detection of electrical and light stimulations is also implemented in the memristive device with real-time (≤20 ms) response to light illumination. These results open options to modify the synaptic plasticity effects in memristor-based neuromorphic systems and can lead to the development of electronic systems that can faithfully emulate diverse biological processes.

On-Demand Reconfiguration of Nanomaterials: When Electronics Meets Ionics.

Rapid advances in the semiconductor industry, driven largely by device scaling, are now approaching fundamental physical limits and face severe power, performance, and cost constraints. Multifunctional materials and devices may lead to a paradigm shift toward new, intelligent, and efficient computing systems, and are being extensively studied. Herein examines how, by controlling the internal ion distribution in a solid-state film, a material's chemical composition and physical properties can be reversibly reconfigured using an applied electric field, at room temperature and after device fabrication. Reconfigurability is observed in a wide range of materials, including commonly used dielectric films, and has led to the development of new device concepts such as resistive random-access memory. Physical reconfigurability further allows memory and logic operations to be merged in the same device for efficient in-memory computing and neuromorphic computing systems. By directly changing the chemical composition of the material, coupled electrical, optical, and magnetic effects can also be obtained. A survey of recent fundamental material and device studies that reveal the dynamic ionic processes is included, along with discussions on systematic modeling efforts, device and material challenges, and future research directions.

Reservoir computing using dynamic memristors for temporal information processing.

Reservoir computing systems utilize dynamic reservoirs having short-term memory to project features from the temporal inputs into a high-dimensional feature space. A readout function layer can then effectively analyze the projected features for tasks, such as classification and time-series analysis. The system can efficiently compute complex and temporal data with low-training cost, since only the readout function needs to be trained. Here we experimentally implement a reservoir computing system using a dynamic memristor array. We show that the internal ionic dynamic processes of memristors allow the memristor-based reservoir to directly process information in the temporal domain, and demonstrate that even a small hardware system with only 88 memristors can already be used for tasks, such as handwritten digit recognition. The system is also used to experimentally solve a second-order nonlinear task, and can successfully predict the expected output without knowing the form of the original dynamic transfer function.

Iodine Vacancy Redistribution in Organic-Inorganic Halide Perovskite Films and Resistive Switching Effects.

Organic-inorganic halide perovskite (OHP) materials, for example, CH3 NH3 PbI3 (MAPbI3 ), have attracted significant interest for applications such as solar cells, photodectors, light-emitting diodes, and lasers. Previous studies have shown that charged defects can migrate in perovskites under an electric field and/or light illumination, potentially preventing these devices from practical applications. Understanding and control of the defect generation and movement will not only lead to more stable devices but also new device concepts. Here, it is shown that the formation/annihilation of iodine vacancies (VI 's) in MAPbI3 films, driven by electric fields and light illumination, can induce pronounced resistive switching effects. Due to a low diffusion energy barrier (≈0.17 eV), the VI 's can readily drift under an electric field, and spontaneously diffuse with a concentration gradient. It is shown that the VI diffusion process can be suppressed by controlling the affinity of the contact electrode material to I- ions, or by light illumination. An electrical-write and optical-erase memory element is further demonstrated by coupling ion migration with electric fields and light illumination. These results provide guidance toward improved stability and performance of perovskite-based optoelectronic systems, and can lead to the development of solid-state devices that couple ionics, electronics, and optics.

Sparse coding with memristor networks.

Sparse representation of information provides a powerful means to perform feature extraction on high-dimensional data and is of broad interest for applications in signal processing, computer vision, object recognition and neurobiology. Sparse coding is also believed to be a key mechanism by which biological neural systems can efficiently process a large amount of complex sensory data while consuming very little power. Here, we report the experimental implementation of sparse coding algorithms in a bio-inspired approach using a 32 × 32 crossbar array of analog memristors. This network enables efficient implementation of pattern matching and lateral neuron inhibition and allows input data to be sparsely encoded using neuron activities and stored dictionary elements. Different dictionary sets can be trained and stored in the same system, depending on the nature of the input signals. Using the sparse coding algorithm, we also perform natural image processing based on a learned dictionary.

Experimental Demonstration of Feature Extraction and Dimensionality Reduction Using Memristor Networks.

Memristors have been considered as a leading candidate for a number of critical applications ranging from nonvolatile memory to non-Von Neumann computing systems. Feature extraction, which aims to transform input data from a high-dimensional space to a space with fewer dimensions, is an important technique widely used in machine learning and pattern recognition applications. Here, we experimentally demonstrate that memristor arrays can be used to perform principal component analysis, one of the most commonly used feature extraction techniques, through online, unsupervised learning. Using Sanger's rule, that is, the generalized Hebbian algorithm, the principal components were obtained as the memristor conductances in the network after training. The network was then used to analyze sensory data from a standard breast cancer screening database with high classification success rate (97.1%).

Electronic and optical properties of oxygen vacancies in amorphous Ta2O5 from first principles.

Oxygen vacancies are important defects considered to play a central role in the electronic and optical properties of tantalum pentoxide (Ta2O5) films and devices. Despite extensive experimental studies on oxygen vacancies in Ta2O5, the reported defect states are ambiguously identified due to the absence of accurate and conclusive theoretical evidence. Here we investigate the thermodynamic, electronic, and optical properties of oxygen vacancies in amorphous Ta2O5 by first-principles calculations based on hybrid-functional density functional theory (DFT). The calculated thermodynamic and optical transition levels are in good agreement with a broad range of diverse measured properties with various experimental methods, providing conclusive evidence for the identification of the defect states observed in experiments as originating from oxygen vacancies. Our calculations also predict the formation of spin-polarized polarons. Our results elucidate the fundamental atomistic properties of oxygen vacancies in various oxidation states as a function of growth conditions and provide guidance to control the properties of Ta2O5 films/devices.

Emulation of synaptic metaplasticity in memristors.

Recent studies have shown that nanoionic-based memristors can offer rich internal dynamics during ion movement that enables these solid-state devices to emulate various synaptic functions in biological systems naturally. The experimental observations can be explained within the 2nd-order memristor theoretical framework, which states that the device conductance (weight) can be determined by multiple internal state variables that can be modulated at different time scales and lead to different activity-dependent synaptic behaviors. Here, we show experimentally that not only the synaptic weight, but also synaptic plasticity (i.e. polarity and the rate of weight change) depends on the history of the input activities. This "plasticity of plasticity" resembles metaplasticity effects observed in biological systems, which have been found to facilitate neuron competition and stability. Specifically, we show that the memristor device may exhibit the same apparent weight (conductance) after experiencing different history of activities, but when subjected to additional, identical stimulation conditions, the device will however exhibit very different responses including the polarity and rate of weight (conductance) change. These findings serve to further our knowledge of fundamental physical mechanisms in memristors, and help advance adaptive artificial neuromorphic systems based on these emerging devices.