The Amyloid Precursor Protein Modulates the Position and Length of the Axon Initial Segment.

The amyloid precursor protein (APP) is linked to the genetics and pathogenesis of Alzheimer's disease (AD). It is the parent protein of the ß-amyloid (Aß) peptide, the main constituent of the amyloid plaques found in an AD brain. The pathways from APP to Aß are intensively studied, yet the normal functions of APP itself have generated less interest. We report here that glutamate stimulation of neuronal activity leads to a rapid increase in App gene expression. In mouse and human neurons, elevated APP protein changes the structure of the axon initial segment (AIS) where action potentials are initiated. The AIS is shortened in length and shifts away from the cell body. The GCaMP8f Ca2+ reporter confirms the predicted decrease in neuronal activity. NMDA antagonists or knockdown of App block the glutamate effects. The actions of APP on the AIS are cell-autonomous; exogenous Aß, either fibrillar or oligomeric, has no effect. In culture, APPSwe (a familial AD mutation) induces larger AIS changes than wild type APP. Ankyrin G and ßIV-spectrin, scaffolding proteins of the AIS, both physically associate with APP, more so in AD brains. Finally, in humans with sporadic AD or in the R1.40 AD mouse model, both females and males, neurons have elevated levels of APP protein that invade the AIS. In vivo as in vitro, this increased APP is associated with a significant shortening of the AIS. The findings outline a new role for the APP and encourage a reconsideration of its relationship to AD.SIGNIFICANCE STATEMENT While the amyloid precursor protein (APP) has long been associated with Alzheimer's disease (AD), the normal functions of the full-length Type I membrane protein have been largely unexplored. We report here that the levels of APP protein increase with neuronal activity. In vivo and in vitro, modest amounts of excess APP alter the properties of the axon initial segment. The ß-amyloid peptide derived from APP is without effect. Consistent with the observed changes in the axon initial segment which would be expected to decrease action potential firing, we show that APP expression depresses neuronal activity. In mouse AD models and human sporadic AD, APP physically associates with the scaffolding proteins of the axon initial segment, suggesting a relationship with AD dementia.

Performance of closed-loop resuscitation in a pig model of haemorrhagic shock with fluid alone or in combination with norepinephrine, a pilot study.

We evaluated the performance of a new device to control the administration of fluid alone or co-administration of fluid and norepinephrine in a pig model of haemorrhagic shock in two sets of experiments. In the first one, resuscitation was guided using continuous arterial pressure measurements (three groups: resuscitation with fluid by a physician, CL resuscitation with fluid, and CL resuscitation with fluid and norepinephrine). In the second one, resuscitation was guided using discontinuous arterial pressure measurements (three groups: CL resuscitation with fluid alone, CL resuscitation with fluid and moderate dose norepinephrine, and CL resuscitation with fluid and a high dose of norepinephrine). Pigs were resuscitated for 1 h. In the first set of experiments, proportion of time spent in the target area of 78-88 mmHg of systolic arterial pressure was not statistically different between the three groups: manual, 71.2% (39.1-80.1); CL with fluid, 87.8% (68.3-97.4); and CL with fluid and norepinephrine, 78.1% (59.2-83.6), p = 0.151. In the second set of experiments, performance of CL resuscitation with fluid or with combination of fluid and high or moderate dose of norepinephrine was not significantly different (p = 0.543 for time in target). Pigs resuscitated with norepinephrine required less fluid and had less haemodilution than pigs resuscitated with fluid alone. Performance of CL resuscitation using continuous arterial pressure measurement was not significantly different than optimised manual treatment by a dedicated physician. Performance of CL resuscitation was reduced with discontinuous arterial pressure measurements in comparison with continuous arterial pressure measurements.

Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA.

Neuromorphic vision sensors detect changes in luminosity taking inspiration from mammalian retina and providing a stream of events with high temporal resolution, also known as Dynamic Vision Sensors (DVS). This continuous stream of events can be used to extract spatio-temporal patterns from a scene. A time-surface represents a spatio-temporal context for a given spatial radius around an incoming event from a sensor at a specific time history. Time-surfaces can be organized in a hierarchical way to extract features from input events using the Hierarchy Of Time-Surfaces algorithm, hereinafter HOTS. HOTS can be organized in consecutive layers to extract combination of features in a similar way as some deep-learning algorithms do. This work introduces a novel FPGA architecture for accelerating HOTS network. This architecture is mainly based on block-RAM memory and the non-restoring square root algorithm, requiring basic components and enabling it for low-power low-latency embedded applications. The presented architecture has been tested on a Zynq 7100 platform at 100 MHz. The results show that the latencies are in the range of 1 µ s to 6.7 µ s, requiring a maximum dynamic power consumption of 77 mW. This system was tested with a gesture recognition dataset, obtaining an accuracy loss for 16-bit precision of only 1.2% with respect to the original software HOTS.

Asynchronous Event-Based Motion Processing: From Visual Events to Probabilistic Sensory Representation.

In this work, we propose a two-layered descriptive model for motion processing from retina to the cortex, with an event-based input from the asynchronous time-based image sensor (ATIS) camera. Spatial and spatiotemporal filtering of visual scenes by motion energy detectors has been implemented in two steps in a simple layer of a lateral geniculate nucleus model and a set of three-dimensional Gabor kernels, eventually forming a probabilistic population response. The high temporal resolution of independent and asynchronous local sensory pixels from the ATIS provides a realistic stimulation to study biological motion processing, as well as developing bio-inspired motion processors for computer vision applications. Our study combines two significant theories in neuroscience: event-based stimulation and probabilistic sensory representation. We have modeled how this might be done at the vision level, as well as suggesting this framework as a generic computational principle among different sensory modalities.

Effects of pupillary responses to luminance and attention on visual spatial discrimination.

The optic quality of the eyes is, at least in part, determined by pupil size. Large pupils let more light enter the eyes, but degrade the point spread function, and thus the spatial resolution that can be achieved (Campbell & Gregory, 1960). In natural conditions, the pupil is mainly driven by the luminance (and possibly the color and contrast) at the gazed location, but is also modulated by attention and cognitive factors. Whether changes in eyes' optics related to pupil size modulation by luminance and attention impacts visual processing was assessed in two experiments. In Experiment 1, we measured pupil size using a constantly visible display made of four disks with different luminance levels, with no other task than fixating the disks in succession. The results confirmed that pupil size depends on the luminance of the gazed stimulus. Experiment 2, using similar settings as Experiment 1, used a two-interval forced-choice design to test whether discriminating high spatial frequencies that requires covert attention to parafoveal stimuli is better during the fixation of bright disks that entails a small pupil size, and hence better eyes' optics, as compared to fixating dark disks that entails a large pupil size, and hence poorer eyes' optics. As in Experiment 1, we observed large modulations of pupil size depending on the luminance of the gazed stimulus, but pupil dynamics was more variable, with marked pupil dilation during stimulus encoding, presumably because the demanding spatial frequency discrimination task engaged attention. However, discrimination performance and mean pupil size were not correlated. Despite this lack of correlation, the slopes of pupil dilation during stimulus encoding were correlated to performance, while the slopes of pupil dilation during decision-making were not. We discuss these results regarding the possible functional roles of pupil size modulations.

Full-field OCT technique for high speed event-based optical flow and particle tracking.

This article introduces a method to extract the speed and density of microparticles in real time at several kHz using an asynchronous event-based camera mounted on a full-field optical coherence tomography (FF-OCT) setup. These cameras detect significant amplitude changes, allowing scene-driven acquisitions. They are composed of an array of autonomously operating pixels. Events are triggered when an illuminance change at the pixel level is significant at 1µs time precision. The event-driven FF-OCT algorithm relies on a time-based optical flow computation to operate directly on incoming events and updates the estimation of velocity, direction and density while reducing both computation and data load. We show that for fast moving microparticles in a range of 0.4 - 6.5mm/s, the method performs faster and more efficiently than existing techniques in real time. The target application of this work is to evaluate erythrocyte dynamics at the microvascular level in vivo with a high temporal resolution.

STICK: Spike Time Interval Computational Kernel, a Framework for General Purpose Computation Using Neurons, Precise Timing, Delays, and Synchrony.

There has been significant research over the past two decades in developing new platforms for spiking neural computation. Current neural computers are primarily developed to mimic biology. They use neural networks, which can be trained to perform specific tasks to mainly solve pattern recognition problems. These machines can do more than simulate biology; they allow us to rethink our current paradigm of computation. The ultimate goal is to develop brain-inspired general purpose computation architectures that can breach the current bottleneck introduced by the von Neumann architecture. This work proposes a new framework for such a machine. We show that the use of neuron-like units with precise timing representation, synaptic diversity, and temporal delays allows us to set a complete, scalable compact computation framework. The framework provides both linear and nonlinear operations, allowing us to represent and solve any function. We show usability in solving real use cases from simple differential equations to sets of nonlinear differential equations leading to chaotic attractors.

Visual tracking using neuromorphic asynchronous event-based cameras.

This letter presents a novel computationally efficient and robust pattern tracking method based on a time-encoded, frame-free visual data. Recent interdisciplinary developments, combining inputs from engineering and biology, have yielded a novel type of camera that encodes visual information into a continuous stream of asynchronous, temporal events. These events encode temporal contrast and intensity locally in space and time. We show that the sparse yet accurately timed information is well suited as a computational input for object tracking. In this letter, visual data processing is performed for each incoming event at the time it arrives. The method provides a continuous and iterative estimation of the geometric transformation between the model and the events representing the tracked object. It can handle isometry, similarities, and affine distortions and allows for unprecedented real-time performance at equivalent frame rates in the kilohertz range on a standard PC. Furthermore, by using the dimension of time that is currently underexploited by most artificial vision systems, the method we present is able to solve ambiguous cases of object occlusions that classical frame-based techniques handle poorly.

What can neuromorphic event-driven precise timing add to spike-based pattern recognition?

This letter introduces a study to precisely measure what an increase in spike timing precision can add to spike-driven pattern recognition algorithms. The concept of generating spikes from images by converting gray levels into spike timings is currently at the basis of almost every spike-based modeling of biological visual systems. The use of images naturally leads to generating incorrect artificial and redundant spike timings and, more important, also contradicts biological findings indicating that visual processing is massively parallel, asynchronous with high temporal resolution. A new concept for acquiring visual information through pixel-individual asynchronous level-crossing sampling has been proposed in a recent generation of asynchronous neuromorphic visual sensors. Unlike conventional cameras, these sensors acquire data not at fixed points in time for the entire array but at fixed amplitude changes of their input, resulting optimally sparse in space and time-pixel individually and precisely timed only if new, (previously unknown) information is available (event based). This letter uses the high temporal resolution spiking output of neuromorphic event-based visual sensors to show that lowering time precision degrades performance on several recognition tasks specifically when reaching the conventional range of machine vision acquisition frequencies (30-60 Hz). The use of information theory to characterize separability between classes for each temporal resolution shows that high temporal acquisition provides up to 70% more information that conventional spikes generated from frame-based acquisition as used in standard artificial vision, thus drastically increasing the separability between classes of objects. Experiments on real data show that the amount of information loss is correlated with temporal precision. Our information-theoretic study highlights the potentials of neuromorphic asynchronous visual sensors for both practical applications and theoretical investigations. Moreover, it suggests that representing visual information as a precise sequence of spike times as reported in the retina offers considerable advantages for neuro-inspired visual computations.

Cell specific electrodes for neuronal network reconstruction and monitoring.

Direct interfacing of neurons with electronic devices has been investigated for both prosthetic and neuro-computing applications. In vitro neuronal networks provide great tools not only for improving neuroprostheses but also to take advantage of their computing abilities. However, it is often difficult to organize neuronal networks according to specific cell distributions. Our aim was to develop a cell-type specific immobilization of neurons on individual electrodes to produce organized in vitro neuronal networks on multi-electrode arrays (MEAs). We demonstrate the selective capture of retinal neurons on antibody functionalized surfaces following the formation of self-assembled monolayers from protein-thiol conjugates by simple contact and protein-polypyrrole deposits by electrochemical functionalization. This neuronal selection was achieved on gold for either cone photoreceptors or retinal ganglion neurons using a PNA lectin or a Thy1 antibody, respectively. Anti-fouling of un-functionalized gold surfaces was optimized to increase the capture efficiencies. The technique was extended to electrode arrays by addressing electropolymerization of pyrrole monomers and pyrrole-protein conjugates to active electrodes. Retinal ganglion cell recording on the array further demonstrated the integrity of these neurons following their selection on polypyrrole-coated electrodes. Therefore, this protein-polypyrrole electrodeposition could provide a new approach to generate organized in vitro neuronal networks.

A robust event-driven approach to always-on object recognition.

We propose a neuromimetic architecture capable of always-on pattern recognition, i.e. at any time during processing. To achieve this, we have extended an existing event-based algorithm (Lagorce et al., 2017), which introduced novel spatio-temporal features as a Hierarchy Of Time-Surfaces (HOTS). Built from asynchronous events captured by a neuromorphic camera, these time surfaces allow to encode the local dynamics of a visual scene and to create an efficient event-based pattern recognition architecture. Inspired by neuroscience, we have extended this method to improve its performance. First, we add a homeostatic gain control on the activity of neurons to improve the learning of spatio-temporal patterns (Grimaldi et al., 2021). We also provide a new mathematical formalism that allows an analogy to be drawn between the HOTS algorithm and Spiking Neural Networks (SNN). Following this analogy, we transform the offline pattern categorization method into an online and event-driven layer. This classifier uses the spiking output of the network to define new time surfaces and we then perform the online classification with a neuromimetic implementation of a multinomial logistic regression. These improvements not only consistently increase the performance of the network, but also bring this event-driven pattern recognition algorithm fully online. The results have been validated on different datasets: Poker-DVS (Serrano-Gotarredona and Linares-Barranco, 2015), N-MNIST (Orchard, Jayawant et al., 2015) and DVS Gesture (Amir et al., 2017). This demonstrates the efficiency of this bio-realistic SNN for ultra-fast object recognition through an event-by-event categorization process.

Real-Time High Speed Motion Prediction Using Fast Aperture-Robust Event-Driven Visual Flow.

Optical flow is a crucial component of the feature space for early visual processing of dynamic scenes especially in new applications such as self-driving vehicles, drones and autonomous robots. The dynamic vision sensors are well suited for such applications because of their asynchronous, sparse and temporally precise representation of the visual dynamics. Many algorithms proposed for computing visual flow for these sensors suffer from the aperture problem as the direction of the estimated flow is governed by the curvature of the object rather than the true motion direction. Some methods that do overcome this problem by temporal windowing under-utilize the true precise temporal nature of the dynamic sensors. In this paper, we propose a novel multi-scale plane fitting based visual flow algorithm that is robust to the aperture problem and also computationally fast and efficient. Our algorithm performs well in many scenarios ranging from fixed camera recording simple geometric shapes to real world scenarios such as camera mounted on a moving car and can successfully perform event-by-event motion estimation of objects in the scene to allow for predictions of upto 500 ms i.e., equivalent to 10 to 25 frames with traditional cameras.

Assessment of optogenetically-driven strategies for prosthetic restoration of cortical vision in large-scale neural simulation of V1.

The neural encoding of visual features in primary visual cortex (V1) is well understood, with strong correlates to low-level perception, making V1 a strong candidate for vision restoration through neuroprosthetics. However, the functional relevance of neural dynamics evoked through external stimulation directly imposed at the cortical level is poorly understood. Furthermore, protocols for designing cortical stimulation patterns that would induce a naturalistic perception of the encoded stimuli have not yet been established. Here, we demonstrate a proof of concept by solving these issues through a computational model, combining (1) a large-scale spiking neural network model of cat V1 and (2) a virtual prosthetic system transcoding the visual input into tailored light-stimulation patterns which drive in situ the optogenetically modified cortical tissue. Using such virtual experiments, we design a protocol for translating simple Fourier contrasted stimuli (gratings) into activation patterns of the optogenetic matrix stimulator. We then quantify the relationship between spatial configuration of the imposed light pattern and the induced cortical activity. Our simulations in the absence of visual drive (simulated blindness) show that optogenetic stimulation with a spatial resolution as low as 100 [Formula: see text]m, and light intensity as weak as [Formula: see text] photons/s/cm[Formula: see text] is sufficient to evoke activity patterns in V1 close to those evoked by normal vision.

Autonomous Flying With Neuromorphic Sensing.

Autonomous flight for large aircraft appears to be within our reach. However, launching autonomous systems for everyday missions still requires an immense interdisciplinary research effort supported by pointed policies and funding. We believe that concerted endeavors in the fields of neuroscience, mathematics, sensor physics, robotics, and computer science are needed to address remaining crucial scientific challenges. In this paper, we argue for a bio-inspired approach to solve autonomous flying challenges, outline the frontier of sensing, data processing, and flight control within a neuromorphic paradigm, and chart directions of research needed to achieve operational capabilities comparable to those we observe in nature. One central problem of neuromorphic computing is learning. In biological systems, learning is achieved by adaptive and relativistic information acquisition characterized by near-continuous information retrieval with variable rates and sparsity. This results in both energy and computational resource savings being an inspiration for autonomous systems. We consider pertinent features of insect, bat and bird flight behavior as examples to address various vital aspects of autonomous flight. Insects exhibit sophisticated flight dynamics with comparatively reduced complexity of the brain. They represent excellent objects for the study of navigation and flight control. Bats and birds enable more complex models of attention and point to the importance of active sensing for conducting more complex missions. The implementation of neuromorphic paradigms for autonomous flight will require fundamental changes in both traditional hardware and software. We provide recommendations for sensor hardware and processing algorithm development to enable energy efficient and computationally effective flight control.

Optogenetic therapy: high spatiotemporal resolution and pattern discrimination compatible with vision restoration in non-human primates.

Vision restoration is an ideal medical application for optogenetics, because the eye provides direct optical access to the retina for stimulation. Optogenetic therapy could be used for diseases involving photoreceptor degeneration, such as retinitis pigmentosa or age-related macular degeneration. We describe here the selection, in non-human primates, of a specific optogenetic construct currently tested in a clinical trial. We used the microbial opsin ChrimsonR, and showed that the AAV2.7m8 vector had a higher transfection efficiency than AAV2 in retinal ganglion cells (RGCs) and that ChrimsonR fused to tdTomato (ChR-tdT) was expressed more efficiently than ChrimsonR. Light at 600 nm activated RGCs transfected with AAV2.7m8 ChR-tdT, from an irradiance of 1015 photons.cm-2.s-1. Vector doses of 5 × 1010 and 5 × 1011 vg/eye transfected up to 7000 RGCs/mm2 in the perifovea, with no significant immune reaction. We recorded RGC responses from a stimulus duration of 1 ms upwards. When using the recorded activity to decode stimulus information, we obtained an estimated visual acuity of 20/249, above the level of legal blindness (20/400). These results lay the groundwork for the ongoing clinical trial with the AAV2.7m8 - ChR-tdT vector for vision restoration in patients with retinitis pigmentosa.

Event-Based Face Detection and Tracking Using the Dynamics of Eye Blinks.

We present the first purely event-based method for face detection using the high temporal resolution properties of an event-based camera to detect the presence of a face in a scene using eye blinks. Eye blinks are a unique and stable natural dynamic temporal signature of human faces across population that can be fully captured by event-based sensors. We show that eye blinks have a unique temporal signature over time that can be easily detected by correlating the acquired local activity with a generic temporal model of eye blinks that has been generated from a wide population of users. In a second stage once a face has been located it becomes possible to apply a probabilistic framework to track its spatial location for each incoming event while using eye blinks to correct for drift and tracking errors. Results are shown for several indoor and outdoor experiments. We also release an annotated data set that can be used for future work on the topic.

Event-Based Gesture Recognition With Dynamic Background Suppression Using Smartphone Computational Capabilities.

In this paper, we introduce a framework for dynamic gesture recognition with background suppression operating on the output of a moving event-based camera. The system is developed to operate in real-time using only the computational capabilities of a mobile phone. It introduces a new development around the concept of time-surfaces. It also presents a novel event-based methodology to dynamically remove backgrounds that uses the high temporal resolution properties of event-based cameras. To our knowledge, this is the first Android event-based framework for vision-based recognition of dynamic gestures running on a smartphone without off-board processing. We assess the performances by considering several scenarios in both indoors and outdoors, for static and dynamic conditions, in uncontrolled lighting conditions. We also introduce a new event-based dataset for gesture recognition with static and dynamic backgrounds (made publicly available). The set of gestures has been selected following a clinical trial to allow human-machine interaction for the visually impaired and older adults. We finally report comparisons with prior work that addressed event-based gesture recognition reporting comparable results, without the use of advanced classification techniques nor power greedy hardware.

Event-Based Computation for Touch Localization Based on Precise Spike Timing.

Precise spike timing and temporal coding are used extensively within the nervous system of insects and in the sensory periphery of higher order animals. However, conventional Artificial Neural Networks (ANNs) and machine learning algorithms cannot take advantage of this coding strategy, due to their rate-based representation of signals. Even in the case of artificial Spiking Neural Networks (SNNs), identifying applications where temporal coding outperforms the rate coding strategies of ANNs is still an open challenge. Neuromorphic sensory-processing systems provide an ideal context for exploring the potential advantages of temporal coding, as they are able to efficiently extract the information required to cluster or classify spatio-temporal activity patterns from relative spike timing. Here we propose a neuromorphic model inspired by the sand scorpion to explore the benefits of temporal coding, and validate it in an event-based sensory-processing task. The task consists in localizing a target using only the relative spike timing of eight spatially-separated vibration sensors. We propose two different approaches in which the SNNs learns to cluster spatio-temporal patterns in an unsupervised manner and we demonstrate how the task can be solved both analytically and through numerical simulation of multiple SNN models. We argue that the models presented are optimal for spatio-temporal pattern classification using precise spike timing in a task that could be used as a standard benchmark for evaluating event-based sensory processing models based on temporal coding.

Behavioural responses to a photovoltaic subretinal prosthesis implanted in non-human primates.

Retinal dystrophies and age-related macular degeneration related to photoreceptor degeneration can cause blindness. In blind patients, although the electrical activation of the residual retinal circuit can provide useful artificial visual perception, the resolutions of current retinal prostheses have been limited either by large electrodes or small numbers of pixels. Here we report the evaluation, in three awake non-human primates, of a previously reported near-infrared-light-sensitive photovoltaic subretinal prosthesis. We show that multipixel stimulation of the prosthesis within radiation safety limits enabled eye tracking in the animals, that they responded to stimulations directed at the implant with repeated saccades and that the implant-induced responses were present two years after device implantation. Our findings pave the way for the clinical evaluation of the prosthesis in patients affected by dry atrophic age-related macular degeneration.

A Theory for Sparse Event-Based Closed Loop Control.

Most dynamic systems are controlled by discrete time controllers. One of the main challenges faced during the design of a digital control law is the selection of the appropriate sampling time. A small sampling time will increase the accuracy of the controlled output at the expense of heavy computations. In contrast, a large sampling time will decrease the computational power needed to update the control law at the expense of a smaller stability region. In addition, once the setpoint is reached, the controlled input is still updated, making the overall controlled system not energetically efficient. To be more efficient, one can update the control law based on a significant fixed change of the controlled signal (send-on-delta or event-based controller). Like for time-based discretization, the amplitude of the significant change must be chosen carefully to avoid oscillations around the setpoint (e.g., if the setpoint is in between two samples) or an unnecessary increase of the samples number needed to reach the setpoint with a given accuracy. This paper proposes a novel non-linear event-based discretization method based on inter-events duration. We demonstrate that our new method reaches an arbitrary accuracy independently of the setpoint amplitude without increasing the network data transmission bandwidth. The method decreases the overall number of samples needed to estimate the states of a dynamical system and the update rate of an actuator, making it more energetically efficient.