Formation and Retrieval of Cell Assemblies in a Biologically Realistic Spiking Neural Network Model of Area CA3 in the Mouse Hippocampus.

The hippocampal formation is critical for episodic memory, with area Cornu Ammonis 3 (CA3) a necessary substrate for auto-associative pattern completion. Recent theoretical and experimental evidence suggests that the formation and retrieval of cell assemblies enable these functions. Yet, how cell assemblies are formed and retrieved in a full-scale spiking neural network (SNN) of CA3 that incorporates the observed diversity of neurons and connections within this circuit is not well understood. Here, we demonstrate that a data-driven SNN model quantitatively reflecting the neuron type-specific population sizes, intrinsic electrophysiology, connectivity statistics, synaptic signaling, and long-term plasticity of the mouse CA3 is capable of robust auto-association and pattern completion via cell assemblies. Our results show that a broad range of assembly sizes could successfully and systematically retrieve patterns from heavily incomplete or corrupted cues after a limited number of presentations. Furthermore, performance was robust with respect to partial overlap of assemblies through shared cells, substantially enhancing memory capacity. These novel findings provide computational evidence that the specific biological properties of the CA3 circuit produce an effective neural substrate for associative learning in the mammalian brain.

Hardware-Mappable Cellular Neural Networks for Distributed Wavefront Detection in Next-Generation Cardiac Implants.

Artificial intelligence algorithms are being adopted to analyze medical data, promising faster interpretation to support doctors' diagnostics. The next frontier is to bring these powerful algorithms to implantable medical devices. Herein, a closed-loop solution is proposed, where a cellular neural network is used to detect abnormal wavefronts and wavebrakes in cardiac signals recorded in human tissue is trained to achieve >96% accuracy, >92% precision, >99% specificity, and >93% sensitivity, when floating point precision weights are assumed. Unfortunately, the current hardware technologies for floating point precision are too bulky or energy intensive for compact standalone applications in medical implants. Emerging device technologies, such as memristors, can provide the compact and energy-efficient hardware fabric to support these efforts and can be reliably embedded with existing sensor and actuator platforms in implantable devices. A distributed design that considers the hardware limitations in terms of overhead and limited bit precision is also discussed. The proposed distributed solution can be easily adapted to other medical technologies that require compact and efficient computing, like wearable devices and lab-on-chip platforms.

Data-driven RRAM device models using Kriging interpolation.

A two-tier Kriging interpolation approach is proposed to model jump tables for resistive switches. Originally developed for mining and geostatistics, its locality of the calculation makes this approach particularly powerful for modeling electronic devices with complex behavior landscape and switching noise, like RRAM. In this paper, a first Kriging model is used to model and predict the mean in the signal, followed up by a second Kriging step used to model the standard deviation of the switching noise. We use 36 synthetic datasets covering a broad range of different mean and standard deviation Gaussian distributions to test the validity of our approach. We also show the applicability to experimental data obtained from TiOx devices and compare the predicted vs. the experimental test distributions using Kolmogorov-Smirnov and maximum mean discrepancy tests. Our results show that the proposed Kriging approach can predict both the mean and standard deviation in the switching more accurately than typical binning model. Kriging-based jump tables can be used to realistically model the behavior of RRAM and other non-volatile analog device populations and the impact of the weight dispersion in neural network simulations.

Gradient Decomposition Methods for Training Neural Networks With Non-ideal Synaptic Devices.

While promising for high-capacity machine learning accelerators, memristor devices have non-idealities that prevent software-equivalent accuracies when used for online training. This work uses a combination of Mini-Batch Gradient Descent (MBGD) to average gradients, stochastic rounding to avoid vanishing weight updates, and decomposition methods to keep the memory overhead low during mini-batch training. Since the weight update has to be transferred to the memristor matrices efficiently, we also investigate the impact of reconstructing the gradient matrixes both internally (rank-seq) and externally (rank-sum) to the memristor array. Our results show that streaming batch principal component analysis (streaming batch PCA) and non-negative matrix factorization (NMF) decomposition algorithms can achieve near MBGD accuracy in a memristor-based multi-layer perceptron trained on the MNIST (Modified National Institute of Standards and Technology) database with only 3 to 10 ranks at significant memory savings. Moreover, NMF rank-seq outperforms streaming batch PCA rank-seq at low-ranks making it more suitable for hardware implementation in future memristor-based accelerators.

Streaming Batch Eigenupdates for Hardware Neural Networks.

Neural networks based on nanodevices, such as metal oxide memristors, phase change memories, and flash memory cells, have generated considerable interest for their increased energy efficiency and density in comparison to graphics processing units (GPUs) and central processing units (CPUs). Though immense acceleration of the training process can be achieved by leveraging the fact that the time complexity of training does not scale with the network size, it is limited by the space complexity of stochastic gradient descent, which grows quadratically. The main objective of this work is to reduce this space complexity by using low-rank approximations of stochastic gradient descent. This low spatial complexity combined with streaming methods allows for significant reductions in memory and compute overhead, opening the door for improvements in area, time and energy efficiency of training. We refer to this algorithm and architecture to implement it as the streaming batch eigenupdate (SBE) approach.

Challenges hindering memristive neuromorphic hardware from going mainstream.

Memristive devices have elicited intense research in the past decade thanks to their inherent low voltage operation, multi-bit storage and cost-effective manufacturability. Nonetheless, several outstanding performance and manufacturability challenges have prevented the widespread industry adoption of redox-based memristive matrices. Here, we discuss these challenges in terms of key metrics and propose a roadmap towards realizing competitive memristive-based neuromorphic processing systems.

Publisher Correction: Stateful characterization of resistive switching TiO2 with electron beam induced currents.

The original version of this Article contained an error in Eq. 1. The arrows between the symbols "T" and "B", and "B" and "T", were written "↔" but should have been "→", and incorrectly read: IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT↔B+IESEEB↔T The correct from of the Eq. 1 is as follows:IEBIC=IEBAC+ISEE+I(e↔h)+IEBICT→B+IESEEB→T This has now been corrected in both the PDF and HTML versions of the article.

Stateful characterization of resistive switching TiO2 with electron beam induced currents.

Metal oxide resistive switches are increasingly important as possible artificial synapses in next-generation neuromorphic networks. Nevertheless, there is still no codified set of tools for studying properties of the devices. To this end, we demonstrate electron beam-induced current measurements as a powerful method to monitor the development of local resistive switching in TiO2-based devices. By comparing beam energy-dependent electron beam-induced currents with Monte Carlo simulations of the energy absorption in different device layers, it is possible to deconstruct the origins of filament image formation and relate this to both morphological changes and the state of the switch. By clarifying the contrast mechanisms in electron beam-induced current microscopy, it is possible to gain new insights into the scaling of the resistive switching phenomenon and observe the formation of a current leakage region around the switching filament. Additionally, analysis of symmetric device structures reveals propagating polarization domains.