Growth rules for the repair of Asynchronous Irregular neuronal networks after peripheral lesions.

Several homeostatic mechanisms enable the brain to maintain desired levels of neuronal activity. One of these, homeostatic structural plasticity, has been reported to restore activity in networks disrupted by peripheral lesions by altering their neuronal connectivity. While multiple lesion experiments have studied the changes in neurite morphology that underlie modifications of synapses in these networks, the underlying mechanisms that drive these changes are yet to be explained. Evidence suggests that neuronal activity modulates neurite morphology and may stimulate neurites to selective sprout or retract to restore network activity levels. We developed a new spiking network model of peripheral lesioning and accurately reproduced the characteristics of network repair after deafferentation that are reported in experiments to study the activity dependent growth regimes of neurites. To ensure that our simulations closely resemble the behaviour of networks in the brain, we model deafferentation in a biologically realistic balanced network model that exhibits low frequency Asynchronous Irregular (AI) activity as observed in cerebral cortex. Our simulation results indicate that the re-establishment of activity in neurons both within and outside the deprived region, the Lesion Projection Zone (LPZ), requires opposite activity dependent growth rules for excitatory and inhibitory post-synaptic elements. Analysis of these growth regimes indicates that they also contribute to the maintenance of activity levels in individual neurons. Furthermore, in our model, the directional formation of synapses that is observed in experiments requires that pre-synaptic excitatory and inhibitory elements also follow opposite growth rules. Lastly, we observe that our proposed structural plasticity growth rules and the inhibitory synaptic plasticity mechanism that also balances our AI network both contribute to the restoration of the network to pre-deafferentation stable activity levels.

Feed-forward and noise-tolerant detection of feature homogeneity in spiking networks with a latency code.

In studies of the visual system as well as in computer vision, the focus is often on contrast edges. However, the primate visual system contains a large number of cells that are insensitive to spatial contrast and, instead, respond to uniform homogeneous illumination of their visual field. The purpose of this information remains unclear. Here, we propose a mechanism that detects feature homogeneity in visual areas, based on latency coding and spike time coincidence, in a purely feed-forward and therefore rapid manner. We demonstrate how homogeneity information can interact with information on contrast edges to potentially support rapid image segmentation. Furthermore, we analyze how neuronal crosstalk (noise) affects the mechanism's performance. We show that the detrimental effects of crosstalk can be partly mitigated through delayed feed-forward inhibition that shapes bi-phasic post-synaptic events. The delay of the feed-forward inhibition allows effectively controlling the size of the temporal integration window and, thereby, the coincidence threshold. The proposed model is based on single-spike latency codes in a purely feed-forward architecture that supports low-latency processing, making it an attractive scheme of computation in spiking neuronal networks where rapid responses and low spike counts are desired.

An unsupervised neuromorphic clustering algorithm.

Brains perform complex tasks using a fraction of the power that would be required to do the same on a conventional computer. New neuromorphic hardware systems are now becoming widely available that are intended to emulate the more power efficient, highly parallel operation of brains. However, to use these systems in applications, we need "neuromorphic algorithms" that can run on them. Here we develop a spiking neural network model for neuromorphic hardware that uses spike timing-dependent plasticity and lateral inhibition to perform unsupervised clustering. With this model, time-invariant, rate-coded datasets can be mapped into a feature space with a specified resolution, i.e., number of clusters, using exclusively neuromorphic hardware. We developed and tested implementations on the SpiNNaker neuromorphic system and on GPUs using the GeNN framework. We show that our neuromorphic clustering algorithm achieves results comparable to those of conventional clustering algorithms such as self-organizing maps, neural gas or k-means clustering. We then combine it with a previously reported supervised neuromorphic classifier network to demonstrate its practical use as a neuromorphic preprocessing module.

A neuromorphic network for generic multivariate data classification.

Computational neuroscience has uncovered a number of computational principles used by nervous systems. At the same time, neuromorphic hardware has matured to a state where fast silicon implementations of complex neural networks have become feasible. En route to future technical applications of neuromorphic computing the current challenge lies in the identification and implementation of functional brain algorithms. Taking inspiration from the olfactory system of insects, we constructed a spiking neural network for the classification of multivariate data, a common problem in signal and data analysis. In this model, real-valued multivariate data are converted into spike trains using "virtual receptors" (VRs). Their output is processed by lateral inhibition and drives a winner-take-all circuit that supports supervised learning. VRs are conveniently implemented in software, whereas the lateral inhibition and classification stages run on accelerated neuromorphic hardware. When trained and tested on real-world datasets, we find that the classification performance is on par with a naïve Bayes classifier. An analysis of the network dynamics shows that stable decisions in output neuron populations are reached within less than 100 ms of biological time, matching the time-to-decision reported for the insect nervous system. Through leveraging a population code, the network tolerates the variability of neuronal transfer functions and trial-to-trial variation that is inevitably present on the hardware system. Our work provides a proof of principle for the successful implementation of a functional spiking neural network on a configurable neuromorphic hardware system that can readily be applied to real-world computing problems.

Automatic segmentation of odor maps in the mouse olfactory bulb using regularized non-negative matrix factorization.

Segmentation of functional parts in image series of functional activity is a common problem in neuroscience. Here we apply regularized non-negative matrix factorization (rNMF) to extract glomeruli in intrinsic optical signal (IOS) images of the olfactory bulb. Regularization allows us to incorporate prior knowledge about the spatio-temporal characteristics of glomerular signals. We demonstrate how to identify suitable regularization parameters on a surrogate dataset. With appropriate regularization segmentation by rNMF is more resilient to noise and requires fewer observations than conventional spatial independent component analysis (sICA). We validate our approach in experimental data using anatomical outlines of glomeruli obtained by 2-photon imaging of resting synapto-pHluorin fluorescence. Taken together, we show that rNMF provides a straightforward method for problem tailored source separation that enables reliable automatic segmentation of functional neural images, with particular benefit in situations with low signal-to-noise ratio as in IOS imaging.

Mapping odorant sensitivities reveals a sparse but structured representation of olfactory chemical space by sensory input to the mouse olfactory bulb.

In olfactory systems, convergence of sensory neurons onto glomeruli generates a map of odorant receptor identity. How glomerular maps relate to sensory space remains unclear. We sought to better characterize this relationship in the mouse olfactory system by defining glomeruli in terms of the odorants to which they are most sensitive. Using high-throughput odorant delivery and ultrasensitive imaging of sensory inputs, we imaged responses to 185 odorants presented at concentrations determined to activate only one or a few glomeruli across the dorsal olfactory bulb. The resulting datasets defined the tuning properties of glomeruli - and, by inference, their cognate odorant receptors - in a low-concentration regime, and yielded consensus maps of glomerular sensitivity across a wide range of chemical space. Glomeruli were extremely narrowly tuned, with ~25% responding to only one odorant, and extremely sensitive, responding to their effective odorants at sub-picomolar to nanomolar concentrations. Such narrow tuning in this concentration regime allowed for reliable functional identification of many glomeruli based on a single diagnostic odorant. At the same time, the response spectra of glomeruli responding to multiple odorants was best predicted by straightforward odorant structural features, and glomeruli sensitive to distinct odorants with common structural features were spatially clustered. These results define an underlying structure to the primary representation of sensory space by the mouse olfactory system.

Machine Learning Approaches to Classify Anatomical Regions in Rodent Brain from High Density Recordings.

Identifying different functional regions during a brain surgery is a challenging task usually performed by highly specialized neurophysiologists. Progress in this field may be used to improve in situ brain navigation and will serve as an important building block to minimize the number of animals in preclinical brain research required by properly positioning implants intraoperatively. The study at hand aims to correlate recorded extracellular signals with the volume of origin by deep learning methods. Our work establishes connections between the position in the brain and recorded high-density neural signals. This was achieved by evaluating the performance of BLSTM, BGRU, QRNN and CNN neural network architectures on multisite electrophysiological data sets. All networks were able to successfully distinguish cortical and thalamic brain regions according to their respective neural signals. The BGRU provides the best results with an accuracy of 88.6 % and demonstrates that this classification task might be solved in higher detail while minimizing complex preprocessing steps.

The similarity between odors and their binary mixtures in Drosophila.

How are odor mixtures perceived? We take a behavioral approach toward this question, using associative odor-recognition experiments in Drosophila. We test how strongly flies avoid a binary mixture after punishment training with one of its constituent elements and how much, in turn, flies avoid an odor element if it had been a component of a previously punished binary mixture. A distinguishing feature of our approach is that we first adjust odors for task-relevant behavioral potency, that is, for equal learnability. Doing so, we find that 1) generalization between mixture and elements is symmetrical and partial, 2) elements are equally similar to all mixtures containing it and that 3) mixtures are equally similar to both their constituent elements. As boundary conditions for the applicability of these rules, we note that first, although variations in learnability are small and remain below statistical cut-off, these variations nevertheless correlate with the elements' perceptual "weight" in the mixture; thus, even small differences in learnability between the elements have the potential to feign mixture asymmetries. Second, the more distant the elements of a mixture are to each other in terms of their physicochemical properties, the more distant the flies regard the elements from the mixture. Thus, titrating for task-relevant behavioral potency and taking into account physicochemical relatedness of odors reveals rules of mixture perception that, maybe surprisingly, appear to be fairly simple.

A behavioral odor similarity "space" in larval Drosophila.

To provide a behavior-based estimate of odor similarity in larval Drosophila, we use 4 recognition-type experiments: 1) We train larvae to associate an odor with food and then test whether they would regard another odor as the same as the trained one. 2) We train larvae to associate an odor with food and test whether they prefer the trained odor against a novel nontrained one. 3) We train larvae differentially to associate one odor with food, but not the other one, and test whether they prefer the rewarded against the nonrewarded odor. 4) In an experiment like (3), we test the larvae after a 30-min break. This yields a combined task-independent estimate of perceived difference between odor pairs. Comparing these perceived differences to published measures of physicochemical difference reveals a weak correlation. A notable exception are 3-octanol and benzaldehyde, which are distinct in published accounts of chemical similarity and in terms of their published sensory representation but nevertheless are consistently regarded as the most similar of the 10 odor pairs employed. It thus appears as if at least some aspects of olfactory perception are "computed" in postreceptor circuits on the basis of sensory signals rather than being immediately given by them.

Resolving Fast Gas Transients with Metal Oxide Sensors.

Electronic olfaction can help detect and localize harmful gases and pollutants, but the turbulence of the natural environment presents a particular challenge: odor encounters are intermittent, and an effective electronic nose must therefore be able to resolve short odor pulses. The slow responses of the widely used metal oxide (MOX) gas sensors complicate the task. Here, we combine high-resolution data acquisition with a processing method based on Kalman filtering and absolute-deadband sampling to extract fast onset events. We find that our system can resolve the onset time of odor encounters with enough precision for source direction estimation with a pair of MOX sensors in a stereo-osmic configuration.

Event-Based Sensing and Signal Processing in the Visual, Auditory, and Olfactory Domain: A Review.

The nervous systems converts the physical quantities sensed by its primary receptors into trains of events that are then processed in the brain. The unmatched efficiency in information processing has long inspired engineers to seek brain-like approaches to sensing and signal processing. The key principle pursued in neuromorphic sensing is to shed the traditional approach of periodic sampling in favor of an event-driven scheme that mimicks sampling as it occurs in the nervous system, where events are preferably emitted upon the change of the sensed stimulus. In this paper we highlight the advantages and challenges of event-based sensing and signal processing in the visual, auditory and olfactory domains. We also provide a survey of the literature covering neuromorphic sensing and signal processing in all three modalities. Our aim is to facilitate research in event-based sensing and signal processing by providing a comprehensive overview of the research performed previously as well as highlighting conceptual advantages, current progress and future challenges in the field.

Processing and classification of chemical data inspired by insect olfaction.

The chemical sense of insects has evolved to encode and classify odorants. Thus, the neural circuits in their olfactory system are likely to implement an efficient method for coding, processing, and classifying chemical information. Here, we describe a computational method to process molecular representations and classify molecules. The three-step approach mimics neurocomputational principles observed in olfactory systems. In the first step, the original stimulus space is sampled by "virtual receptors," which are chemotopically arranged by a self-organizing map. In the second step, the signals from the virtual receptors are decorrelated via correlation-based lateral inhibition. Finally, in the third step, olfactory scent perception is modeled by a machine learning classifier. We found that signal decorrelation during the second stage significantly increases the accuracy of odorant classification. Moreover, our results suggest that the proposed signal transform is capable of dimensionality reduction and is more robust against overdetermined representations than principal component scores. Our olfaction-inspired method was successfully applied to predicting bioactivities of pharmaceutically active compounds with high accuracy. It represents a way to efficiently connect chemical structure with biological activity space.

Computational exploration of molecular receptive fields in the olfactory bulb reveals a glomerulus-centric chemical map.

Progress in olfactory research is currently hampered by incomplete knowledge about chemical receptive ranges of primary receptors. Moreover, the chemical logic underlying the arrangement of computational units in the olfactory bulb has still not been resolved. We undertook a large-scale approach at characterising molecular receptive ranges (MRRs) of glomeruli in the dorsal olfactory bulb (dOB) innervated by the MOR18-2 olfactory receptor, also known as Olfr78, with human ortholog OR51E2. Guided by an iterative approach that combined biological screening and machine learning, we selected 214 odorants to characterise the response of MOR18-2 and its neighbouring glomeruli. We found that a combination of conventional physico-chemical and vibrational molecular descriptors performed best in predicting glomerular responses using nonlinear Support-Vector Regression. We also discovered several previously unknown odorants activating MOR18-2 glomeruli, and obtained detailed MRRs of MOR18-2 glomeruli and their neighbours. Our results confirm earlier findings that demonstrated tunotopy, that is, glomeruli with similar tuning curves tend to be located in spatial proximity in the dOB. In addition, our results indicate chemotopy, that is, a preference for glomeruli with similar physico-chemical MRR descriptions being located in spatial proximity. Together, these findings suggest the existence of a partial chemical map underlying glomerular arrangement in the dOB. Our methodology that combines machine learning and physiological measurements lights the way towards future high-throughput studies to deorphanise and characterise structure-activity relationships in olfaction.

Sparse coding with a somato-dendritic rule.

Cortical neurons are silent most of the time: sparse activity enables low-energy computation in the brain, and promises to do the same in neuromorphic hardware. Beyond power efficiency, sparse codes have favourable properties for associative learning, as they can store more information than local codes but are easier to read out than dense codes. Auto-encoders with a sparse constraint can learn sparse codes, and so can single-layer networks that combine recurrent inhibition with unsupervised Hebbian learning. But the latter usually require fast homeostatic plasticity, which could lead to catastrophic forgetting in embodied agents that learn continuously. Here we set out to explore whether plasticity at recurrent inhibitory synapses could take up that role instead, regulating both the population sparseness and the firing rates of individual neurons. We put the idea to the test in a network that employs compartmentalised inputs to solve the task: rate-based dendritic compartments integrate the feedforward input, while spiking integrate-and-fire somas compete through recurrent inhibition. A somato-dendritic learning rule allows somatic inhibition to modulate nonlinear Hebbian learning in the dendrites. Trained on MNIST digits and natural images, the network discovers independent components that form a sparse encoding of the input and support linear decoding. These findings confirm that intrinsic homeostatic plasticity is not strictly required for regulating sparseness: inhibitory synaptic plasticity can have the same effect. Our work illustrates the usefulness of compartmentalised inputs, and makes the case for moving beyond point neuron models in artificial spiking neural networks.

Optimized Particle Swarm Optimization (OPSO) and its application to artificial neural network training.

BACKGROUND: Particle Swarm Optimization (PSO) is an established method for parameter optimization. It represents a population-based adaptive optimization technique that is influenced by several "strategy parameters". Choosing reasonable parameter values for the PSO is crucial for its convergence behavior, and depends on the optimization task. We present a method for parameter meta-optimization based on PSO and its application to neural network training. The concept of the Optimized Particle Swarm Optimization (OPSO) is to optimize the free parameters of the PSO by having swarms within a swarm. We assessed the performance of the OPSO method on a set of five artificial fitness functions and compared it to the performance of two popular PSO implementations. RESULTS: Our results indicate that PSO performance can be improved if meta-optimized parameter sets are applied. In addition, we could improve optimization speed and quality on the other PSO methods in the majority of our experiments. We applied the OPSO method to neural network training with the aim to build a quantitative model for predicting blood-brain barrier permeation of small organic molecules. On average, training time decreased by a factor of four and two in comparison to the other PSO methods, respectively. By applying the OPSO method, a prediction model showing good correlation with training-, test- and validation data was obtained. CONCLUSION: Optimizing the free parameters of the PSO method can result in performance gain. The OPSO approach yields parameter combinations improving overall optimization performance. Its conceptual simplicity makes implementing the method a straightforward task.

Comparing Neuromorphic Solutions in Action: Implementing a Bio-Inspired Solution to a Benchmark Classification Task on Three Parallel-Computing Platforms.

Neuromorphic computing employs models of neuronal circuits to solve computing problems. Neuromorphic hardware systems are now becoming more widely available and "neuromorphic algorithms" are being developed. As they are maturing toward deployment in general research environments, it becomes important to assess and compare them in the context of the applications they are meant to solve. This should encompass not just task performance, but also ease of implementation, speed of processing, scalability, and power efficiency. Here, we report our practical experience of implementing a bio-inspired, spiking network for multivariate classification on three different platforms: the hybrid digital/analog Spikey system, the digital spike-based SpiNNaker system, and GeNN, a meta-compiler for parallel GPU hardware. We assess performance using a standard hand-written digit classification task. We found that whilst a different implementation approach was required for each platform, classification performances remained in line. This suggests that all three implementations were able to exercise the model's ability to solve the task rather than exposing inherent platform limits, although differences emerged when capacity was approached. With respect to execution speed and power consumption, we found that for each platform a large fraction of the computing time was spent outside of the neuromorphic device, on the host machine. Time was spent in a range of combinations of preparing the model, encoding suitable input spiking data, shifting data, and decoding spike-encoded results. This is also where a large proportion of the total power was consumed, most markedly for the SpiNNaker and Spikey systems. We conclude that the simulation efficiency advantage of the assessed specialized hardware systems is easily lost in excessive host-device communication, or non-neuronal parts of the computation. These results emphasize the need to optimize the host-device communication architecture for scalability, maximum throughput, and minimum latency. Moreover, our results indicate that special attention should be paid to minimize host-device communication when designing and implementing networks for efficient neuromorphic computing.

A Probabilistic Model for Estimating the Depth and Threshold Temperature of C-fiber Nociceptors.

The subjective experience of thermal pain follows the detection and encoding of noxious stimuli by primary afferent neurons called nociceptors. However, nociceptor morphology has been hard to access and the mechanisms of signal transduction remain unresolved. In order to understand how heat transducers in nociceptors are activated in vivo, it is important to estimate the temperatures that directly activate the skin-embedded nociceptor membrane. Hence, the nociceptor's temperature threshold must be estimated, which in turn will depend on the depth at which transduction happens in the skin. Since the temperature at the receptor cannot be accessed experimentally, such an estimation can currently only be achieved through modeling. However, the current state-of-the-art model to estimate temperature at the receptor suffers from the fact that it cannot account for the natural stochastic variability of neuronal responses. We improve this model using a probabilistic approach which accounts for uncertainties and potential noise in system. Using a data set of 24 C-fibers recorded in vitro, we show that, even without detailed knowledge of the bio-thermal properties of the system, the probabilistic model that we propose here is capable of providing estimates of threshold and depth in cases where the classical method fails.