An Event-based Neural Compressive Telemetry with >11× Loss-less Data Reduction for High-bandwidth Intracortical Brain Computer Interfaces.

Intracortical brain-computer interfaces offer superior spatial and temporal resolutions, but face challenges as the increasing number of recording channels introduces high amounts of data to be transferred. This requires power-hungry data serialization and telemetry, leading to potential tissue damage risks. To address this challenge, this paper introduces an event-based neural compressive telemetry (NCT) consisting of 8 channel-rotating Δ-ADCs, an event-driven serializer supporting a proposed ternary address event representation protocol, and an event-based LVDS driver. Leveraging a high sparsity of extracellular spikes and high spatial correlation of the high-density recordings, the proposed NCT achieves a compression ratio of >11.4×, while consumes only 1 µW per channel, which is 127× more efficient than state of the art. The NCT well preserves the spike waveform fidelity, and has a low normalized RMS error <23% even with a spike amplitude down to only 31 µV.

Vector Symbolic Finite State Machines in Attractor Neural Networks.

Hopfield attractor networks are robust distributed models of human memory, but they lack a general mechanism for effecting state-dependent attractor transitions in response to input. We propose construction rules such that an attractor network may implement an arbitrary finite state machine (FSM), where states and stimuli are represented by high-dimensional random vectors and all state transitions are enacted by the attractor network's dynamics. Numerical simulations show the capacity of the model, in terms of the maximum size of implementable FSM, to be linear in the size of the attractor network for dense bipolar state vectors and approximately quadratic for sparse binary state vectors. We show that the model is robust to imprecise and noisy weights, and so a prime candidate for implementation with high-density but unreliable devices. By endowing attractor networks with the ability to emulate arbitrary FSMs, we propose a plausible path by which FSMs could exist as a distributed computational primitive in biological neural networks.

Finding the gap: neuromorphic motion-vision in dense environments.

Animals have evolved mechanisms to travel safely and efficiently within different habitats. On a journey in dense terrains animals avoid collisions and cross narrow passages while controlling an overall course. Multiple hypotheses target how animals solve challenges faced during such travel. Here we show that a single mechanism enables safe and efficient travel. We developed a robot inspired by insects. It has remarkable capabilities to travel in dense terrain, avoiding collisions, crossing gaps and selecting safe passages. These capabilities are accomplished by a neuromorphic network steering the robot toward regions of low apparent motion. Our system leverages knowledge about vision processing and obstacle avoidance in insects. Our results demonstrate how insects might safely travel through diverse habitats. We anticipate our system to be a working hypothesis to study insects' travels in dense terrains. Furthermore, it illustrates that we can design novel hardware systems by understanding the underlying mechanisms driving behaviour.

Impact of spiking neurons leakages and network recurrences on event-based spatio-temporal pattern recognition.

Spiking neural networks coupled with neuromorphic hardware and event-based sensors are getting increased interest for low-latency and low-power inference at the edge. However, multiple spiking neuron models have been proposed in the literature with different levels of biological plausibility and different computational features and complexities. Consequently, there is a need to define the right level of abstraction from biology in order to get the best performance in accurate, efficient and fast inference in neuromorphic hardware. In this context, we explore the impact of synaptic and membrane leakages in spiking neurons. We confront three neural models with different computational complexities using feedforward and recurrent topologies for event-based visual and auditory pattern recognition. Our results showed that, in terms of accuracy, leakages are important when there are both temporal information in the data and explicit recurrence in the network. Additionally, leakages do not necessarily increase the sparsity of spikes flowing in the network. We also investigated the impact of heterogeneity in the time constant of leakages. The results showed a slight improvement in accuracy when using data with a rich temporal structure, thereby validating similar findings obtained in previous studies. These results advance our understanding of the computational role of the neural leakages and network recurrences, and provide valuable insights for the design of compact and energy-efficient neuromorphic hardware for embedded systems.

Skin-Inspired Flexible and Stretchable Electrospun Carbon Nanofiber Sensors for Neuromorphic Sensing.

During the past few decades, a significant amount of research effort has been dedicated toward developing skin-inspired sensors for real-time human motion monitoring and next-generation robotic devices. Although several flexible and wearable sensors have been developed in the past, the need of the hour is developing accurate, reliable, sophisticated, facile yet inexpensive flexible sensors coupled with neuromorphic systems or spiking neural networks to encode tactile information without the need for complex digital architectures, thus achieving true skin-like sensing with limited resources. In this work, we propose an approach entailing carbon nanofiber-polydimethylsiloxane composite-based piezoresistive sensors, coupled with spiking neural networks, to mimic skin-like sensing. The strain and pressure sensors have been combined with appropriately designed neural networks to encode analog voltages to spikes to recreate bioinspired tactile sensing and proprioception. To further validate the proprioceptive capability of the system, a gesture tracking smart glove, combined with a spiking neural network, was demonstrated. Wearable and flexible sensors with accompanying neural networks such as the ones proposed in this work will pave the way for a future generation of skin-mimetic sensors for advanced prosthetic devices, apparel integrable smart sensors for human motion monitoring, and human-machine interfaces.

An Event-Based Digital Time Difference Encoder Model Implementation for Neuromorphic Systems.

Neuromorphic systems are a viable alternative to conventional systems for real-time tasks with constrained resources. Their low power consumption, compact hardware realization, and low-latency response characteristics are the key ingredients of such systems. Furthermore, the event-based signal processing approach can be exploited for reducing the computational load and avoiding data loss due to its inherently sparse representation of sensed data and adaptive sampling time. In event-based systems, the information is commonly coded by the number of spikes within a specific temporal window. However, the temporal information of event-based signals can be difficult to extract when using rate coding. In this work, we present a novel digital implementation of the model, called time difference encoder (TDE), for temporal encoding on event-based signals, which translates the time difference between two consecutive input events into a burst of output events. The number of output events along with the time between them encodes the temporal information. The proposed model has been implemented as a digital circuit with a configurable time constant, allowing it to be used in a wide range of sensing tasks that require the encoding of the time difference between events, such as optical flow-based obstacle avoidance, sound source localization, and gas source localization. This proposed bioinspired model offers an alternative to the Jeffress model for the interaural time difference estimation, which is validated in this work with a sound source lateralization proof-of-concept system. The model was simulated and implemented on a field-programmable gate array (FPGA), requiring 122 slice registers of hardware resources and less than 1 mW of power consumption.

An Implantable Neuromorphic Sensing System Featuring Near-sensor Computation and Send-on-Delta Transmission for Wireless Neural Sensing of Peripheral Nerves.

This paper presents a bio-inspired event-driven neuromorphic sensing system (NSS) capable of performing on-chip feature extraction and "send-on-delta" pulse-based transmission, targeting peripheral-nerve neural recording applications. The proposed NSS employs event-based sampling which, by leveraging the sparse nature of electroneurogram (ENG) signals, achieves a data compression ratio of >125×, while maintaining a low normalized RMS error of 4% after reconstruction. The proposed NSS consists of three sub-circuits. A clockless level-crossing (LC) ADC with background offset calibration has been employed to reduce the data rate, while maintaining a high signal to quantization noise ratio. A fully synthesized spiking neural network (SNN) extracts temporal features of compound action potential signals consumes only 13 µW. An event-driven pulse-based body channel communication (Pulse-BCC) with serialized address-event representation encoding (AER) schemes minimizes transmission energy and form factor. The prototype is fabricated in 40-nm CMOS occupying a 0.32-mm2 active area and consumes in total 28.2 µW and 50 µW power in feature extraction and full diagnosis mode, respectively. The presented NSS also extracts temporal features of compound action potential signals with 10-µs precision.

ED-BioRob: A Neuromorphic Robotic Arm With FPGA-Based Infrastructure for Bio-Inspired Spiking Motor Controllers.

Compared to classic robotics, biological nervous systems respond to stimuli in a fast and efficient way regarding the body motor actions. Decision making, once the sensory information arrives to the brain, is in the order of ms, while the whole process from sensing to movement requires tens of ms. Classic robotic systems usually require complex computational abilities. Key differences between biological systems and robotic machines lie in the way information is coded and transmitted. A neuron is the "basic" element that constitutes biological nervous systems. Neurons communicate in an event-driven way through small currents or ionic pulses (spikes). When neurons are arranged in networks, they allow not only for the processing of sensory information, but also for the actuation over the muscles in the same spiking manner. This paper presents the application of a classic motor control model (proportional-integral-derivative) developed with the biological spike processing principle, including the motor actuation with time enlarged spikes instead of the classic pulse-width-modulation. This closed-loop control model, called spike-based PID controller (sPID), was improved and adapted for a dual FPGA-based system to control the four joints of a bioinspired light robot (BioRob X5), called event-driven BioRob (ED-BioRob). The use of spiking signals allowed the system to achieve a current consumption bellow 1A for the entire 4 DoF working at the same time. Furthermore, the robot joints commands can be received from a population of silicon-neurons running on the Dynap-SE platform. Thus, our proposal aims to bridge the gap between a general purpose processing analog neuromorphic hardware and the spiking actuation of a robotic platform.

Efficient Processing of Spatio-Temporal Data Streams With Spiking Neural Networks.

Spiking neural networks (SNNs) are potentially highly efficient models for inference on fully parallel neuromorphic hardware, but existing training methods that convert conventional artificial neural networks (ANNs) into SNNs are unable to exploit these advantages. Although ANN-to-SNN conversion has achieved state-of-the-art accuracy for static image classification tasks, the following subtle but important difference in the way SNNs and ANNs integrate information over time makes the direct application of conversion techniques for sequence processing tasks challenging. Whereas all connections in SNNs have a certain propagation delay larger than zero, ANNs assign different roles to feed-forward connections, which immediately update all neurons within the same time step, and recurrent connections, which have to be rolled out in time and are typically assigned a delay of one time step. Here, we present a novel method to obtain highly accurate SNNs for sequence processing by modifying the ANN training before conversion, such that delays induced by ANN rollouts match the propagation delays in the targeted SNN implementation. Our method builds on the recently introduced framework of streaming rollouts, which aims for fully parallel model execution of ANNs and inherently allows for temporal integration by merging paths of different delays between input and output of the network. The resulting networks achieve state-of-the-art accuracy for multiple event-based benchmark datasets, including N-MNIST, CIFAR10-DVS, N-CARS, and DvsGesture, and through the use of spatio-temporal shortcut connections yield low-latency approximate network responses that improve over time as more of the input sequence is processed. In addition, our converted SNNs are consistently more energy-efficient than their corresponding ANNs.

Event-Based Eccentric Motion Detection Exploiting Time Difference Encoding.

Attentional selectivity tends to follow events considered as interesting stimuli. Indeed, the motion of visual stimuli present in the environment attract our attention and allow us to react and interact with our surroundings. Extracting relevant motion information from the environment presents a challenge with regards to the high information content of the visual input. In this work we propose a novel integration between an eccentric down-sampling of the visual field, taking inspiration from the varying size of receptive fields (RFs) in the mammalian retina, and the Spiking Elementary Motion Detector (sEMD) model. We characterize the system functionality with simulated data and real world data collected with bio-inspired event driven cameras, successfully implementing motion detection along the four cardinal directions and diagonally.

Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain.

Neuromorphic engineering (NE) encompasses a diverse range of approaches to information processing that are inspired by neurobiological systems, and this feature distinguishes neuromorphic systems from conventional computing systems. The brain has evolved over billions of years to solve difficult engineering problems by using efficient, parallel, low-power computation. The goal of NE is to design systems capable of brain-like computation. Numerous large-scale neuromorphic projects have emerged recently. This interdisciplinary field was listed among the top 10 technology breakthroughs of 2014 by the MIT Technology Review and among the top 10 emerging technologies of 2015 by the World Economic Forum. NE has two-way goals: one, a scientific goal to understand the computational properties of biological neural systems by using models implemented in integrated circuits (ICs); second, an engineering goal to exploit the known properties of biological systems to design and implement efficient devices for engineering applications. Building hardware neural emulators can be extremely useful for simulating large-scale neural models to explain how intelligent behavior arises in the brain. The principal advantages of neuromorphic emulators are that they are highly energy efficient, parallel and distributed, and require a small silicon area. Thus, compared to conventional CPUs, these neuromorphic emulators are beneficial in many engineering applications such as for the porting of deep learning algorithms for various recognitions tasks. In this review article, we describe some of the most significant neuromorphic spiking emulators, compare the different architectures and approaches used by them, illustrate their advantages and drawbacks, and highlight the capabilities that each can deliver to neural modelers. This article focuses on the discussion of large-scale emulators and is a continuation of a previous review of various neural and synapse circuits (Indiveri et al., 2011). We also explore applications where these emulators have been used and discuss some of their promising future applications.

Mimicking biological neurons with a nanoscale ferroelectric transistor.

Neuron is the basic computing unit in brain-inspired neural networks. Although a multitude of excellent artificial neurons realized with conventional transistors have been proposed, they might not be energy and area efficient in large-scale networks. The recent discovery of ferroelectricity in hafnium oxide (HfO2) and the related switching phenomena at the nanoscale might provide a solution. This study employs the newly reported accumulative polarization reversal in nanoscale HfO2-based ferroelectric field-effect transistors (FeFETs) to implement two key neuronal dynamics: the integration of action potentials and the subsequent firing according to the biologically plausible all-or-nothing law. We show that by carefully shaping electrical excitations based on the particular nucleation-limited switching kinetics of the ferroelectric layer further neuronal behaviors can be emulated, such as firing activity tuning, arbitrary refractory period and the leaky effect. Finally, we discuss the advantages of an FeFET-based neuron, highlighting its transferability to advanced scaling technologies and the beneficial impact it may have in reducing the complexity of neuromorphic circuits.

Corrigendum: Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain.

[This corrects the article DOI: 10.3389/fnins.2018.00891.].

Modeling latency code processing in the electric sense: from the biological template to its VLSI implementation.

Understanding the coding of sensory information under the temporal constraints of natural behavior is not yet well resolved. There is a growing consensus that spike timing or latency coding can maximally exploit the timing of neural events to make fast computing elements and that such mechanisms are essential to information processing functions in the brain. The electric sense of mormyrid fish provides a convenient biological model where this coding scheme can be studied. The sensory input is a physically ordered spatial pattern of current densities, which is coded in the precise timing of primary afferent spikes. The neural circuits of the processing pathway are well known and the system exhibits the best known illustration of corollary discharge, which provides the reference to decoding the sensory afferent latency pattern. A theoretical model has been constructed from available electrophysiological and neuroanatomical data to integrate the principal traits of the neural processing structure and to study sensory interaction with motor-command-driven corollary discharge signals. This has been used to explore neural coding strategies at successive stages in the network and to examine the simulated network capacity to reproduce output neuron responses. The model shows that the network has the ability to resolve primary afferent spike timing differences in the sub-millisecond range, and that this depends on the coincidence of sensory and corollary discharge-driven gating signals. In the integrative and output stages of the network, corollary discharge sets up a proactive background filter, providing temporally structured excitation and inhibition within the network whose balance is then modulated locally by sensory input. This complements the initial gating mechanism and contributes to amplification of the input pattern of latencies, conferring network hyperacuity. These mechanisms give the system a robust capacity to extract behaviorally meaningful features of the electric image with high sensitivity over a broad working range. Since the network largely depends on spike timing, we finally discuss its suitability for implementation in robotic applications based on neuromorphic hardware.

Tunnel junction based memristors as artificial synapses.

We prepared magnesia, tantalum oxide, and barium titanate based tunnel junction structures and investigated their memristive properties. The low amplitudes of the resistance change in these types of junctions are the major obstacle for their use. Here, we increased the amplitude of the resistance change from 10% up to 100%. Utilizing the memristive properties, we looked into the use of the junction structures as artificial synapses. We observed analogs of long-term potentiation, long-term depression and spike-time dependent plasticity in these simple two terminal devices. Finally, we suggest a possible pathway of these devices toward their integration in neuromorphic systems for storing analog synaptic weights and supporting the implementation of biologically plausible learning mechanisms.

Synthesizing cognition in neuromorphic electronic systems.

The quest to implement intelligent processing in electronic neuromorphic systems lacks methods for achieving reliable behavioral dynamics on substrates of inherently imprecise and noisy neurons. Here we report a solution to this problem that involves first mapping an unreliable hardware layer of spiking silicon neurons into an abstract computational layer composed of generic reliable subnetworks of model neurons and then composing the target behavioral dynamics as a "soft state machine" running on these reliable subnets. In the first step, the neural networks of the abstract layer are realized on the hardware substrate by mapping the neuron circuit bias voltages to the model parameters. This mapping is obtained by an automatic method in which the electronic circuit biases are calibrated against the model parameters by a series of population activity measurements. The abstract computational layer is formed by configuring neural networks as generic soft winner-take-all subnetworks that provide reliable processing by virtue of their active gain, signal restoration, and multistability. The necessary states and transitions of the desired high-level behavior are then easily embedded in the computational layer by introducing only sparse connections between some neurons of the various subnets. We demonstrate this synthesis method for a neuromorphic sensory agent that performs real-time context-dependent classification of motion patterns observed by a silicon retina.

A robust sound perception model suitable for neuromorphic implementation.

We have recently demonstrated the emergence of dynamic feature sensitivity through exposure to formative stimuli in a real-time neuromorphic system implementing a hybrid analog/digital network of spiking neurons. This network, inspired by models of auditory processing in mammals, includes several mutually connected layers with distance-dependent transmission delays and learning in the form of spike timing dependent plasticity, which effects stimulus-driven changes in the network connectivity. Here we present results that demonstrate that the network is robust to a range of variations in the stimulus pattern, such as are found in naturalistic stimuli and neural responses. This robustness is a property critical to the development of realistic, electronic neuromorphic systems. We analyze the variability of the response of the network to "noisy" stimuli which allows us to characterize the acuity in information-theoretic terms. This provides an objective basis for the quantitative comparison of networks, their connectivity patterns, and learning strategies, which can inform future design decisions. We also show, using stimuli derived from speech samples, that the principles are robust to other challenges, such as variable presentation rate, that would have to be met by systems deployed in the real world. Finally we demonstrate the potential applicability of the approach to real sounds.

Emergent Auditory Feature Tuning in a Real-Time Neuromorphic VLSI System.

Many sounds of ecological importance, such as communication calls, are characterized by time-varying spectra. However, most neuromorphic auditory models to date have focused on distinguishing mainly static patterns, under the assumption that dynamic patterns can be learned as sequences of static ones. In contrast, the emergence of dynamic feature sensitivity through exposure to formative stimuli has been recently modeled in a network of spiking neurons based on the thalamo-cortical architecture. The proposed network models the effect of lateral and recurrent connections between cortical layers, distance-dependent axonal transmission delays, and learning in the form of Spike Timing Dependent Plasticity (STDP), which effects stimulus-driven changes in the pattern of network connectivity. In this paper we demonstrate how these principles can be efficiently implemented in neuromorphic hardware. In doing so we address two principle problems in the design of neuromorphic systems: real-time event-based asynchronous communication in multi-chip systems, and the realization in hybrid analog/digital VLSI technology of neural computational principles that we propose underlie plasticity in neural processing of dynamic stimuli. The result is a hardware neural network that learns in real-time and shows preferential responses, after exposure, to stimuli exhibiting particular spectro-temporal patterns. The availability of hardware on which the model can be implemented, makes this a significant step toward the development of adaptive, neurobiologically plausible, spike-based, artificial sensory systems.