Closing the loop: High-speed robotics with accelerated neuromorphic hardware.

The BrainScaleS-2 system is an established analog neuromorphic platform with versatile applications in the diverse fields of computational neuroscience and spike-based machine learning. In this work, we extend the system with a configurable realtime event interface that enables a tight coupling of its distinct analog network core to external sensors and actuators. The 1,000-fold acceleration of the emulated nerve cells allows us to target high-speed robotic applications that require precise timing on a microsecond scale. As a showcase, we present a closed-loop setup for commuting brushless DC motors: we utilize PyTorch to train a spiking neural network emulated on the analog substrate to control an electric motor from a sensory event stream. The presented system enables research in the area of event-driven controllers for high-speed robotics, including self-supervised and biologically inspired online learning for such applications.

Simulated Dopamine Modulation of a Neurorobotic Model of the Basal Ganglia.

The vertebrate basal ganglia play an important role in action selection-the resolution of conflicts between alternative motor programs. The effective operation of basal ganglia circuitry is also known to rely on appropriate levels of the neurotransmitter dopamine. We investigated reducing or increasing the tonic level of simulated dopamine in a prior model of the basal ganglia integrated into a robot control architecture engaged in a foraging task inspired by animal behaviour. The main findings were that progressive reductions in the levels of simulated dopamine caused slowed behaviour and, at low levels, an inability to initiate movement. These states were partially relieved by increased salience levels (stronger sensory/motivational input). Conversely, increased simulated dopamine caused distortion of the robot's motor acts through partially expressed motor activity relating to losing actions. This could also lead to an increased frequency of behaviour switching. Levels of simulated dopamine that were either significantly lower or higher than baseline could cause a loss of behavioural integration, sometimes leaving the robot in a 'behavioral trap'. That some analogous traits are observed in animals and humans affected by dopamine dysregulation suggests that robotic models could prove useful in understanding the role of dopamine neurotransmission in basal ganglia function and dysfunction.

NeoSLAM: Long-Term SLAM Using Computational Models of the Brain.

Simultaneous Localization and Mapping (SLAM) is a fundamental problem in the field of robotics, enabling autonomous robots to navigate and create maps of unknown environments. Nevertheless, the SLAM methods that use cameras face problems in maintaining accurate localization over extended periods across various challenging conditions and scenarios. Following advances in neuroscience, we propose NeoSLAM, a novel long-term visual SLAM, which uses computational models of the brain to deal with this problem. Inspired by the human neocortex, NeoSLAM is based on a hierarchical temporal memory model that has the potential to identify temporal sequences of spatial patterns using sparse distributed representations. Being known to have a high representational capacity and high tolerance to noise, sparse distributed representations have several properties, enabling the development of a novel neuroscience-based loop-closure detector that allows for real-time performance, especially in resource-constrained robotic systems. The proposed method has been thoroughly evaluated in terms of environmental complexity by using a wheeled robot deployed in the field and demonstrated that the accuracy of loop-closure detection was improved compared with the traditional RatSLAM system.

Elucidating multifinal and equifinal pathways to developmental disorders by constructing real-world neurorobotic models.

Vigorous research has been conducted to accumulate biological and theoretical knowledge about neurodevelopmental disorders, including molecular, neural, computational, and behavioral characteristics; however, these findings remain fragmentary and do not elucidate integrated mechanisms. An obstacle is the heterogeneity of developmental pathways causing clinical phenotypes. Additionally, in symptom formations, the primary causes and consequences of developmental learning processes are often indistinguishable. Herein, we review developmental neurorobotic experiments tackling problems related to the dynamic and complex properties of neurodevelopmental disorders. Specifically, we focus on neurorobotic models under predictive processing lens for the study of developmental disorders. By constructing neurorobotic models with predictive processing mechanisms of learning, perception, and action, we can simulate formations of integrated causal relationships among neurodynamical, computational, and behavioral characteristics in the robot agents while considering developmental learning processes. This framework has the potential to bind neurobiological hypotheses (excitation-inhibition imbalance and functional disconnection), computational accounts (unusual encoding of uncertainty), and clinical symptoms. Developmental neurorobotic approaches may serve as a complementary research framework for integrating fragmented knowledge and overcoming the heterogeneity of neurodevelopmental disorders.

Recovery of Volitional Motor Control and Overground Walking in Participants With Chronic Clinically Motor Complete Spinal Cord Injury: Restoration of Rehabilitative Function With Epidural Spinal Stimulation (RESTORES) Trial-A Preliminary Study.

Spinal cord injury (SCI) is damage to any part of the spinal cord resulting in paralysis, bowel and/or bladder incontinence, and loss of sensation and other bodily functions. Current treatments for chronic SCI are focused on managing symptoms and preventing further damage to the spinal cord with limited neuro-restorative interventions. Recent research and independent clinical trials of spinal cord stimulation (SCS) or intensive neuro-rehabilitation including neuro-robotics in participants with SCI have suggested potential malleability of the neuronal networks for neurological recovery. We hypothesize that epidural electrical stimulation (EES) delivered via SCS in conjunction with mental imagery practice and robotic neuro-rehabilitation can synergistically improve volitional motor function below the level of injury in participants with chronic clinically motor-complete SCI. In our pilot clinical RESTORES trial (RESToration Of Rehabilitative function with Epidural spinal Stimulation), we investigate the feasibility of this combined multi-modal approach in restoring volitional motor control and achieving independent overground locomotion in participants with chronic motor complete thoracic SCI. Secondary aims are to assess the safety of this combination therapy including the off-label SCS usage as well as improving functional outcome measures. To our knowledge, this is the first clinical trial that investigates the combined impact of this multi-modal EES and rehabilitation strategy in participants with chronic motor complete SCI. Two participants with chronic motor-complete thoracic SCI were recruited for this pilot trial. Both participants have successfully regained volitional motor control below their level of SCI injury and achieved independent overground walking within a month of post-operative stimulation and rehabilitation. There were no adverse events noted in our trial and there was an improvement in post-operative truncal stability score. Results from this pilot study demonstrates the feasibility of combining EES, mental imagery practice and robotic rehabilitation in improving volitional motor control below level of SCI injury and restoring independent overground walking for participants with chronic motor-complete SCI. Our team believes that this provides very exciting promise in a field currently devoid of disease-modifying therapies.

Neurorobotic reinforcement learning for domains with parametrical uncertainty.

Neuromorphic hardware paired with brain-inspired learning strategies have enormous potential for robot control. Explicitly, these advantages include low energy consumption, low latency, and adaptability. Therefore, developing and improving learning strategies, algorithms, and neuromorphic hardware integration in simulation is a key to moving the state-of-the-art forward. In this study, we used the neurorobotics platform (NRP) simulation framework to implement spiking reinforcement learning control for a robotic arm. We implemented a force-torque feedback-based classic object insertion task ("peg-in-hole") and controlled the robot for the first time with neuromorphic hardware in the loop. We therefore provide a solution for training the system in uncertain environmental domains by using randomized simulation parameters. This leads to policies that are robust to real-world parameter variations in the target domain, filling the sim-to-real gap.To the best of our knowledge, it is the first neuromorphic implementation of the peg-in-hole task in simulation with the neuromorphic Loihi chip in the loop, and with scripted accelerated interactive training in the Neurorobotics Platform, including randomized domains.

Embodied bidirectional simulation of a spiking cortico-basal ganglia-cerebellar-thalamic brain model and a mouse musculoskeletal body model distributed across computers including the supercomputer Fugaku.

Embodied simulation with a digital brain model and a realistic musculoskeletal body model provides a means to understand animal behavior and behavioral change. Such simulation can be too large and complex to conduct on a single computer, and so distributed simulation across multiple computers over the Internet is necessary. In this study, we report our joint effort on developing a spiking brain model and a mouse body model, connecting over the Internet, and conducting bidirectional simulation while synchronizing them. Specifically, the brain model consisted of multiple regions including secondary motor cortex, primary motor and somatosensory cortices, basal ganglia, cerebellum and thalamus, whereas the mouse body model, provided by the Neurorobotics Platform of the Human Brain Project, had a movable forelimb with three joints and six antagonistic muscles to act in a virtual environment. Those were simulated in a distributed manner across multiple computers including the supercomputer Fugaku, which is the flagship supercomputer in Japan, while communicating via Robot Operating System (ROS). To incorporate models written in C/C++ in the distributed simulation, we developed a C++ version of the rosbridge library from scratch, which has been released under an open source license. These results provide necessary tools for distributed embodied simulation, and demonstrate its possibility and usefulness toward understanding animal behavior and behavioral change.

An Integrated Neurorobotics Model of the Cerebellar-Basal Ganglia Circuitry.

This work presents a neurorobotics model of the brain that integrates the cerebellum and the basal ganglia regions to coordinate movements in a humanoid robot. This cerebellar-basal ganglia circuitry is well known for its relevance to the motor control used by most mammals. Other computational models have been designed for similar applications in the robotics field. However, most of them completely ignore the interplay between neurons from the basal ganglia and cerebellum. Recently, neuroscientists indicated that neurons from both regions communicate not only at the level of the cerebral cortex but also at the subcortical level. In this work, we built an integrated neurorobotics model to assess the capacity of the network to predict and adjust the motion of the hands of a robot in real time. Our model was capable of performing different movements in a humanoid robot by respecting the sensorimotor loop of the robot and the biophysical features of the neuronal circuitry. The experiments were executed in simulation and the real world. We believe that our proposed neurorobotics model can be an important tool for new studies on the brain and a reference toward new robot motor controllers.

From Brain Models to Robotic Embodied Cognition: How Does Biological Plausibility Inform Neuromorphic Systems?

We examine the challenging "marriage" between computational efficiency and biological plausibility-A crucial node in the domain of spiking neural networks at the intersection of neuroscience, artificial intelligence, and robotics. Through a transdisciplinary review, we retrace the historical and most recent constraining influences that these parallel fields have exerted on descriptive analysis of the brain, construction of predictive brain models, and ultimately, the embodiment of neural networks in an enacted robotic agent. We study models of Spiking Neural Networks (SNN) as the central means enabling autonomous and intelligent behaviors in biological systems. We then provide a critical comparison of the available hardware and software to emulate SNNs for investigating biological entities and their application on artificial systems. Neuromorphics is identified as a promising tool to embody SNNs in real physical systems and different neuromorphic chips are compared. The concepts required for describing SNNs are dissected and contextualized in the new no man's land between cognitive neuroscience and artificial intelligence. Although there are recent reviews on the application of neuromorphic computing in various modules of the guidance, navigation, and control of robotic systems, the focus of this paper is more on closing the cognition loop in SNN-embodied robotics. We argue that biologically viable spiking neuronal models used for electroencephalogram signals are excellent candidates for furthering our knowledge of the explainability of SNNs. We complete our survey by reviewing different robotic modules that can benefit from neuromorphic hardware, e.g., perception (with a focus on vision), localization, and cognition. We conclude that the tradeoff between symbolic computational power and biological plausibility of hardware can be best addressed by neuromorphics, whose presence in neurorobotics provides an accountable empirical testbench for investigating synthetic and natural embodied cognition. We argue this is where both theoretical and empirical future work should converge in multidisciplinary efforts involving neuroscience, artificial intelligence, and robotics.

A neurorobotics approach to behaviour selection based on human activity recognition.

Behaviour selection has been an active research topic for robotics, in particular in the field of human-robot interaction. For a robot to interact autonomously and effectively with humans, the coupling between techniques for human activity recognition and robot behaviour selection is of paramount importance. However, most approaches to date consist of deterministic associations between the recognised activities and the robot behaviours, neglecting the uncertainty inherent to sequential predictions in real-time applications. In this paper, we address this gap by presenting an initial neurorobotics model that embeds, in a simulated robot, computational models of parts of the mammalian brain that resembles neurophysiological aspects of the basal ganglia-thalamus-cortex (BG-T-C) circuit, coupled with human activity recognition techniques. A robotics simulation environment was developed for assessing the model, where a mobile robot accomplished tasks by using behaviour selection in accordance with the activity being performed by the inhabitant of an intelligent home. Initial results revealed that the initial neurorobotics model is advantageous, especially considering the coupling between the most accurate activity recognition approaches and the computational models of more complex animals.

SNS-Toolbox: An Open Source Tool for Designing Synthetic Nervous Systems and Interfacing Them with Cyber-Physical Systems.

One developing approach for robotic control is the use of networks of dynamic neurons connected with conductance-based synapses, also known as Synthetic Nervous Systems (SNS). These networks are often developed using cyclic topologies and heterogeneous mixtures of spiking and non-spiking neurons, which is a difficult proposition for existing neural simulation software. Most solutions apply to either one of two extremes, the detailed multi-compartment neural models in small networks, and the large-scale networks of greatly simplified neural models. In this work, we present our open-source Python package SNS-Toolbox, which is capable of simulating hundreds to thousands of spiking and non-spiking neurons in real-time or faster on consumer-grade computer hardware. We describe the neural and synaptic models supported by SNS-Toolbox, and provide performance on multiple software and hardware backends, including GPUs and embedded computing platforms. We also showcase two examples using the software, one for controlling a simulated limb with muscles in the physics simulator Mujoco, and another for a mobile robot using ROS. We hope that the availability of this software will reduce the barrier to entry when designing SNS networks, and will increase the prevalence of SNS networks in the field of robotic control.

The Efficacity of the NeuroAssist Robotic System for Motor Rehabilitation of the Upper Limb-Promising Results from a Pilot Study.

The research aimed to evaluate the efficacy of the NeuroAssist, a parallel robotic system comprised of three robotic modules equipped with human-robot interaction capabilities, an internal sensor system for torque monitoring, and an external sensor system for real-time patient monitoring for the motor rehabilitation of the shoulder, elbow, and wrist. The study enrolled 10 consecutive patients with right upper limb paresis caused by stroke, traumatic spinal cord disease, or multiple sclerosis admitted to the Neurology I Department of Cluj-Napoca Emergency County Hospital. The patients were evaluated clinically and electrophysiologically before (T1) and after the intervention (T2). The intervention consisted of five consecutive daily sessions of 30-45 min each of 30 passive repetitive movements performed with the robot. There were significant differences (Wilcoxon signed-rank test) between baseline and end-point clinical parameters, specifically for the Barthel Index (53.00 ± 37.72 vs. 60.50 ± 36.39, p = 0.016) and Activities of Daily Living Index (4.70 ± 3.43 vs. 5.50 ± 3.80, p = 0.038). The goniometric parameters improved: shoulder flexion (70.00 ± 56.61 vs. 80.00 ± 63.59, p = 0.026); wrist flexion/extension (34.00 ± 28.75 vs. 42.50 ± 33.7, p = 0.042)/(30.00 ± 22.97 vs. 41.00 ± 30.62, p = 0.042); ulnar deviation (23.50 ± 19.44 vs. 33.50 ± 24.15, p = 0.027); and radial deviation (17.50 ± 18.14 vs. 27.00 ± 24.85, p = 0.027). There was a difference in muscle activation of the extensor digitorum communis muscle (1.00 ± 0.94 vs. 1.40 ± 1.17, p = 0.046). The optimized and dependable NeuroAssist Robotic System improved shoulder and wrist range of motion and functional scores, regardless of the cause of the motor deficit. However, further investigations are necessary to establish its definite role in motor recovery.

Dynamic balance of a bipedal robot using neural network training with simulated annealing.

This work proposes using an evolutionary optimization method known as simulated annealing to train artificial neural networks. These neural networks are used to control posture stabilization of a humanoid robot in a simulation. A total of eight multilayer perceptron neural networks are used. Although the control is used mainly for posture stabilization and not displacement, we propose a posture set to achieve this, including right leg lift in sagittal plane and right leg lift in frontal plane. At the beginning, tests are carried out only considering gravitational force and reaction force between the floor and the humanoid; then tests are carried out with two disturbances: tilted ground and adding a mass to the humanoid. We found that using simulated annealing the robot maintains its stability at all times, decreasing the number of epochs needed to converge, and also, showing flexibility and adaptability to disturbances. The way neural networks learn is analyzed; videos of the movements made, and the model for further experimentation are provided.

Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics.

Experience replay is widely used in AI to bootstrap reinforcement learning (RL) by enabling an agent to remember and reuse past experiences. Classical techniques include shuffled-, reversed-ordered- and prioritized-memory buffers, which have different properties and advantages depending on the nature of the data and problem. Interestingly, recent computational neuroscience work has shown that these techniques are relevant to model hippocampal reactivations recorded during rodent navigation. Nevertheless, the brain mechanisms for orchestrating hippocampal replay are still unclear. In this paper, we present recent neurorobotics research aiming to endow a navigating robot with a neuro-inspired RL architecture (including different learning strategies, such as model-based (MB) and model-free (MF), and different replay techniques). We illustrate through a series of numerical simulations how the specificities of robotic experimentation (e.g., autonomous state decomposition by the robot, noisy perception, state transition uncertainty, non-stationarity) can shed new lights on which replay techniques turn out to be more efficient in different situations. Finally, we close the loop by raising new hypotheses for neuroscience from such robotic models of hippocampal replay.

Evaluating Muscle Synergies With EMG Data and Physics Simulation in the Neurorobotics Platform.

Although we can measure muscle activity and analyze their activation patterns, we understand little about how individual muscles affect the joint torque generated. It is known that they are controlled by circuits in the spinal cord, a system much less well-understood than the cortex. Knowing the contribution of the muscles toward a joint torque would improve our understanding of human limb control. We present a novel framework to examine the control of biomechanics using physics simulations informed by electromyography (EMG) data. These signals drive a virtual musculoskeletal model in the Neurorobotics Platform (NRP), which we then use to evaluate resulting joint torques. We use our framework to analyze raw EMG data collected during an isometric knee extension study to identify synergies that drive a musculoskeletal lower limb model. The resulting knee torques are used as a reference for genetic algorithms (GA) to generate new simulated activation patterns. On the platform the GA finds solutions that generate torques matching those observed. Possible solutions include synergies that are similar to those extracted from the human study. In addition, the GA finds activation patterns that are different from the biological ones while still producing the same knee torque. The NRP forms a highly modular integrated simulation platform allowing these in silico experiments. We argue that our framework allows for research of the neurobiomechanical control of muscles during tasks, which would otherwise not be possible.