A computational neural model that incorporates both intrinsic dynamics and sensory feedback in the Aplysia feeding network.

Studying the nervous system underlying animal motor control can shed light on how animals can adapt flexibly to a changing environment. We focus on the neural basis of feeding control in Aplysia californica. Using the Synthetic Nervous System framework, we developed a model of Aplysia feeding neural circuitry that balances neurophysiological plausibility and computational complexity. The circuitry includes neurons, synapses, and feedback pathways identified in existing literature. We organized the neurons into three layers and five subnetworks according to their functional roles. Simulation results demonstrate that the circuitry model can capture the intrinsic dynamics at neuronal and network levels. When combined with a simplified peripheral biomechanical model, it is sufficient to mediate three animal-like feeding behaviors (biting, swallowing, and rejection). The kinematic, dynamic, and neural responses of the model also share similar features with animal data. These results emphasize the functional roles of sensory feedback during feeding.

SNS-Toolbox: An Open Source Tool for Designing Synthetic Nervous Systems and Interfacing Them with Cyber-Physical Systems.

One developing approach for robotic control is the use of networks of dynamic neurons connected with conductance-based synapses, also known as Synthetic Nervous Systems (SNS). These networks are often developed using cyclic topologies and heterogeneous mixtures of spiking and non-spiking neurons, which is a difficult proposition for existing neural simulation software. Most solutions apply to either one of two extremes, the detailed multi-compartment neural models in small networks, and the large-scale networks of greatly simplified neural models. In this work, we present our open-source Python package SNS-Toolbox, which is capable of simulating hundreds to thousands of spiking and non-spiking neurons in real-time or faster on consumer-grade computer hardware. We describe the neural and synaptic models supported by SNS-Toolbox, and provide performance on multiple software and hardware backends, including GPUs and embedded computing platforms. We also showcase two examples using the software, one for controlling a simulated limb with muscles in the physics simulator Mujoco, and another for a mobile robot using ROS. We hope that the availability of this software will reduce the barrier to entry when designing SNS networks, and will increase the prevalence of SNS networks in the field of robotic control.

Modular Design of a Polymer-Bilayer-Based Mechanically Compliant Worm-Like Robot.

Soft earthworm-like robots that exhibit mechanical compliance can, in principle, navigate through uneven terrains and constricted spaces that are inaccessible to traditional legged and wheeled robots. However, unlike the biological originals that they mimic, most of the worm-like robots reported to date contain rigid components that limit their compliance, such as electromotors or pressure-driven actuation systems. Here, a mechanically compliant worm-like robot with a fully modular body that is based on soft polymers is reported. The robot is composed of strategically assembled, electrothermally activated polymer bilayer actuators, which are based on a semicrystalline polyurethane with an exceptionally large nonlinear thermal expansion coefficient. The segments are designed on the basis of a modified Timoshenko model, and finite element analysis simulation is used to describe their performance. Upon electrical activation of the segments with basic waveform patterns, the robot can move through repeatable peristaltic locomotion on exceptionally slippery or sticky surfaces and it can be oriented in any direction. The soft body enables the robot to wriggle through openings and tunnels that are much smaller than its cross-section.

Biomechanical and Sensory Feedback Regularize the Behavior of Different Locomotor Central Pattern Generators.

This work presents an in-depth numerical investigation into a hypothesized two-layer central pattern generator (CPG) that controls mammalian walking and how different parameter choices might affect the stepping of a simulated neuromechanical model. Particular attention is paid to the functional role of features that have not received a great deal of attention in previous work: the weak cross-excitatory connectivity within the rhythm generator and the synapse strength between the two layers. Sensitivity evaluations of deafferented CPG models and the combined neuromechanical model are performed. Locomotion frequency is increased in two different ways for both models to investigate whether the model's stability can be predicted by trends in the CPG's phase response curves (PRCs). Our results show that the weak cross-excitatory connection can make the CPG more sensitive to perturbations and that increasing the synaptic strength between the two layers results in a trade-off between forced phase locking and the amount of phase delay that can exist between the two layers. Additionally, although the models exhibit these differences in behavior when disconnected from the biomechanical model, these differences seem to disappear with the full neuromechanical model and result in similar behavior despite a variety of parameter combinations. This indicates that the neural variables do not have to be fixed precisely for stable walking; the biomechanical entrainment and sensory feedback may cancel out the strengths of excitatory connectivity in the neural circuit and play a critical role in shaping locomotor behavior. Our results support the importance of including biomechanical models in the development of computational neuroscience models that control mammalian locomotion.

Analyzing Modeled Torque Profiles to Understand Scale-Dependent Active Muscle Responses in the Hip Joint.

Animal locomotion is influenced by a combination of constituent joint torques (e.g., due to limb inertia and passive viscoelasticity), which determine the necessary muscular response to move the limb. Across animal size-scales, the relative contributions of these constituent joint torques affect the muscular response in different ways. We used a multi-muscle biomechanical model to analyze how passive torque components change due to an animal's size-scale during locomotion. By changing the size-scale of the model, we characterized emergent muscular responses at the hip as a result of the changing constituent torque profile. Specifically, we found that activation phases between extensor and flexor torques to be opposite between small and large sizes for the same kinematic motion. These results suggest general principles of how animal size affects neural control strategies. Our modeled torque profiles show a strong agreement with documented hindlimb torque during locomotion and can provide insights into the neural organization and muscle activation behavior of animals whose motion has not been extensively documented.

Biologically Inspired Optimal Terminal Iterative Learning Control for the Swing Phase of Gait in a Hybrid Neuroprosthesis: A Modeling Study.

(1) Background: An iterative learning control (ILC) strategy was developed for a "Muscle First" Motor-Assisted Hybrid Neuroprosthesis (MAHNP). The MAHNP combines a backdrivable exoskeletal brace with neural stimulation technology to enable persons with paraplegia due to spinal cord injury (SCI) to execute ambulatory motions and walk upright. (2) Methods: The ILC strategy was developed to swing the legs in a biologically inspired ballistic fashion. It maximizes muscular recruitment and activates the motorized exoskeletal bracing to assist the motion as needed. The control algorithm was tested using an anatomically realistic three-dimensional musculoskeletal model of the lower leg and pelvis suitably modified to account for exoskeletal inertia. The model was developed and tested with the OpenSim biomechanical modeling suite. (3) Results: Preliminary data demonstrate the efficacy of the controller in swing-leg simulations and its ability to learn to balance muscular and motor contributions to improve performance and accomplish consistent stepping. In particular, the controller took 15 iterations to achieve the desired outcome with 0.3% error.

Internal state effects on behavioral shifts in freely behaving praying mantises (Tenodera sinensis).

How we interact with our environment largely depends on both the external cues presented by our surroundings and the internal state from within. Internal states are the ever-changing physiological conditions that communicate the immediate survival needs and motivate the animal to behaviorally fulfill them. Satiety level constitutes such a state, and therefore has a dynamic influence on the output behaviors of an animal. In predatory insects like the praying mantis, hunting tactics, grooming, and mating have been shown to change hierarchical organization of behaviors depending on satiety. Here, we analyze behavior sequences of freely hunting praying mantises (Tenodera sinensis) to explore potential differences in sequential patterning of behavior as a correlate of satiety. First, our data supports previous work that showed starved praying mantises were not just more often attentive to prey, but also more often attentive to further prey. This was indicated by the increased time fraction spent in attentive bouts such as prey monitoring, head turns (to track prey), translations (closing the distance to the prey), and more strike attempts. With increasing satiety, praying mantises showed reduced time in these behaviors and exhibited them primarily towards close-proximity prey. Furthermore, our data demonstrates that during states of starvation, the praying mantis exhibits a stereotyped pattern of behavior that is highly motivated by prey capture. As satiety increased, the sequenced behaviors became more variable, indicating a shift away from the necessity of prey capture to more fluid presentations of behavior assembly.

Neural Networks Trained via Reinforcement Learning Stabilize Walking of a Three-Dimensional Biped Model With Exoskeleton Applications.

Our group is developing a cyber-physical walking system (CPWS) for people paralyzed by spinal cord injuries (SCI). The current CPWS consists of a functional neuromuscular stimulation (FNS) system and a powered lower-limb exoskeleton for walking with leg movements in the sagittal plane. We are developing neural control systems that learn to assist the user of this CPWS to walk with stability. In a previous publication (Liu et al., Biomimetics, 2019, 4, 28), we showed a neural controller that stabilized a simulated biped in the sagittal plane. We are considering adding degrees of freedom to the CPWS to allow more natural walking movements and improved stability. Thus, in this paper, we present a new neural network enhanced control system that stabilizes a three-dimensional simulated biped model of a human wearing an exoskeleton. Results show that it stabilizes human/exoskeleton models and is robust to impact disturbances. The simulated biped walks at a steady pace in a range of typical human ambulatory speeds from 0.7 to 1.3 m/s, follows waypoints at a precision of 0.3 m, remains stable, and continues walking forward despite impact disturbances and adapts its speed to compensate for persistent external disturbances. Furthermore, the neural network controller stabilizes human models of different statures from 1.4 to 2.2 m tall without any changes to the control parameters. Please see videos at the following link: 3D biped walking control.

A computational model of insect campaniform sensilla predicts encoding of forces during walking.

Control of forces is essential in both animals and walking machines. Insects measure forces as strains in their exoskeletons via campaniform sensilla (CS). Deformations of cuticular caps embedded in the exoskeleton excite afferents that project to the central nervous system. CS afferent firing frequency (i.e. 'discharge') is highly dynamic, correlating with the rate of change of the force. Discharges adapt over time to tonic forces and exhibit hysteresis during cyclic loading.In this study we characterized a phenomenological model that predicts CS discharge, in which discharge is proportional to the instantaneous stimulus force relative to an adaptive variable. In contrast to previous studies of sensory adaptation, our model (1) is nonlinear and (2) reproduces the characteristic power-law adaptation with first order dynamics only (i.e. no 'fractional derivatives' are required to explain dynamics). We solve the response of the system analytically in multiple cases and use these solutions to derive the dynamics of the adaptive variable. We show that the model can reproduce responses of insect CS to many different force stimuli after being tuned to reproduce only one response, suggesting that the model captures the underlying dynamics of the system. We show that adaptation to tonic forces, rate-sensitivity, and hysteresis are different manifestations of the same underlying mechanism: the adaptive variable. We tune the model to replicate the dynamics of three different CS groups from two insects (cockroach and stick insect), demonstrating that it is generalizable. We also invert the model to estimate the stimulus force given the discharge recording from the animal. We discuss the adaptive neural and mechanical processes that the model may mimic and the model's use for understanding the role of load feedback in insect motor control. A preliminary model and results were previously published in the proceedings of the Conference on Biohybrid and Biomimetic Systems.

Estimating Center of Mass Kinematics During Perturbed Human Standing Using Accelerometers.

Estimating center of mass (COM) through sensor measurements is done to maintain walking and standing stability with exoskeletons. The authors present a method for estimating COM kinematics through an artificial neural network, which was trained by minimizing the mean squared error between COM displacements measured by a gold-standard motion capture system and recorded acceleration signals from body-mounted accelerometers. A total of 5 able-bodied participants were destabilized during standing through: (1) unexpected perturbations caused by 4 linear actuators pulling on the waist and (2) volitionally moving weighted jars on a shelf. Each movement type was averaged across all participants. The algorithm's performance was quantified by the root mean square error and coefficient of determination (R2) calculated from both the entire trial and during each perturbation type. Throughout the trials and movement types, the average coefficient of determination was 0.83, with 89% of the movements with R2 > .70, while the average root mean square error ranged between 7.3% and 22.0%, corresponding to 0.5- and 0.94-cm error in both the coronal and sagittal planes. COM can be estimated in real time for balance control of exoskeletons for individuals with a spinal cord injury, and the procedure can be generalized for other gait studies.

An Analysis of Peristaltic Locomotion for Maximizing Velocity or Minimizing Cost of Transport of Earthworm-Like Robots.

Earthworm-like peristaltic locomotion has been implemented in >50 robots, with many potential applications in otherwise inaccessible terrain. Design guidelines for peristaltic locomotion have come from observations of biology, but robots have empirically explored different structures, actuators, and control waveform shapes than those observed in biological organisms. In this study, we suggest a template analysis based on simplified segments undergoing beam deformations. This analysis enables calculation of the minimum power required by the structure for locomotion and maximum speed of locomotion. Thus, design relationships are shown that apply to peristaltic robots and potentially to earthworms. Specifically, although speed is maximized by moving as many segments as possible, cost of transport (COT) is optimized by moving fewer segments. Furthermore, either soft or relatively stiff segments are possible, but the anisotropy of the stiffnesses is important. Experimentally, we show on our earthworm robot that this method predicts which control waveforms (equivalent to different gaits) correspond to least input power or to maximum velocity. We extend our analysis to 150 segments (similar to that of earthworms) to show that reducing COT is an alternate explanation for why earthworms have so few moving segments. The mathematical relationships developed here between structural properties, actuation power, and waveform shape will enable the design of future robots with more segments and limited onboard power.

Extending the Functional Subnetwork Approach to a Generalized Linear Integrate-and-Fire Neuron Model.

Engineering neural networks to perform specific tasks often represents a monumental challenge in determining network architecture and parameter values. In this work, we extend our previously-developed method for tuning networks of non-spiking neurons, the "Functional subnetwork approach" (FSA), to the tuning of networks composed of spiking neurons. This extension enables the direct assembly and tuning of networks of spiking neurons and synapses based on the network's intended function, without the use of global optimization or machine learning. To extend the FSA, we show that the dynamics of a generalized linear integrate and fire (GLIF) neuron model have fundamental similarities to those of a non-spiking leaky integrator neuron model. We derive analytical expressions that show functional parallels between: (1) A spiking neuron's steady-state spiking frequency and a non-spiking neuron's steady-state voltage in response to an applied current; (2) a spiking neuron's transient spiking frequency and a non-spiking neuron's transient voltage in response to an applied current; and (3) a spiking synapse's average conductance during steady spiking and a non-spiking synapse's conductance. The models become more similar as additional spiking neurons are added to each population "node" in the network. We apply the FSA to model a neuromuscular reflex pathway two different ways: Via non-spiking components and then via spiking components. These results provide a concrete example of how a single non-spiking neuron may model the average spiking frequency of a population of spiking neurons. The resulting model also demonstrates that by using the FSA, models can be constructed that incorporate both spiking and non-spiking units. This work facilitates the construction of large networks of spiking neurons and synapses that perform specific functions, for example, those implemented with neuromorphic computing hardware, by providing an analytical method for directly tuning their parameters without time-consuming optimization or learning.

A neuromechanical model exploring the role of the common inhibitor motor neuron in insect locomotion.

In this work, we analyze a simplified, dynamical, closed-loop, neuromechanical simulation of insect joint control. We are specifically interested in two elements: (1) how slow muscle fibers may serve as temporal integrators of sensory feedback and (2) the role of common inhibitory (CI) motor neurons in resetting this integration when the commanded position changes, particularly during steady-state walking. Despite the simplicity of the model, we show that slow muscle fibers increase the accuracy of limb positioning, even for motions much shorter than the relaxation time of the fiber; this increase in accuracy is due to the slow dynamics of the fibers; the CI motor neuron plays a critical role in accelerating muscle relaxation when the limb moves to a new position; as in the animal, this architecture enables the control of the stance phase speed, independent of swing phase amplitude or duration, by changing the gain of sensory feedback to the stance phase muscles. We discuss how this relates to other models, and how it could be applied to robotic control.

Design and Actuation of a Fabric-Based Worm-Like Robot.

Soft-bodied animals, such as earthworms, are capable of contorting their body to squeeze through narrow spaces, create or enlarge burrows, and move on uneven ground. In many applications such as search and rescue, inspection of pipes and medical procedures, it may be useful to have a hollow-bodied robot with skin separating inside and outside. Textiles can be key to such skins. Inspired by earthworms, we developed two new robots: FabricWorm and MiniFabricWorm. We explored the application of fabric in soft robotics and how textile can be integrated along with other structural elements, such as three-dimensional (3D) printed parts, linear springs, and flexible nylon tubes. The structure of FabricWorm consists of one third the number of rigid pieces as compared to its predecessor Compliant Modular Mesh Worm-Steering (CMMWorm-S), while the structure of MiniFabricWorm consists of no rigid components. This article presents the design of such a mesh and its limitations in terms of structural softness. We experimentally measured the stiffness properties of these robots and compared them directly to its predecessors. FabricWorm and MiniFabricWorm are capable of peristaltic locomotion with a maximum speed of 33 cm/min (0.49 body-lengths/min) and 13.8 cm/min (0.25 body-lengths/min), respectively.

Neuromechanical Model of Rat Hindlimb Walking with Two-Layer CPGs.

This work demonstrates a neuromechanical model of rat hindlimb locomotion undergoing nominal walking with perturbations. In the animal, two types of responses to perturbations are observed: resetting and non-resetting deletions. This suggests that the animal locomotor system contains a memory-like organization. To model this phenomenon, we built a synthetic nervous system that uses separate rhythm generator and pattern formation layers to activate antagonistic muscle pairs about each joint in the sagittal plane. Our model replicates the resetting and non-resetting deletions observed in the animal. In addition, in the intact (i.e., fully afferented) rat walking simulation, we observe slower recovery after perturbation, which is different from the deafferented animal experiment. These results demonstrate that our model is a biologically feasible description of some of the neural circuits in the mammalian spinal cord that control locomotion, and the difference between our simulation and fictive motion shows the importance of sensory feedback on motor output. This model also demonstrates how the pattern formation network can activate muscle synergies in a coordinated way to produce stable walking, which motivates the use of more complex synergies activating more muscles in the legs for three-dimensional limb motion.

Implementation of Deep Deterministic Policy Gradients for Controlling Dynamic Bipedal Walking.

A control system for bipedal walking in the sagittal plane was developed in simulation. The biped model was built based on anthropometric data for a 1.8 m tall male of average build. At the core of the controller is a deep deterministic policy gradient (DDPG) neural network that was trained in GAZEBO, a physics simulator, to predict the ideal foot placement to maintain stable walking despite external disturbances. The complexity of the DDPG network was decreased through carefully selected state variables and a distributed control system. Additional controllers for the hip joints during their stance phases and the ankle joint during toe-off phase help to stabilize the biped during walking. The simulated biped can walk at a steady pace of approximately 1 m/s, and during locomotion it can maintain stability with a 30 kg·m/s impulse applied forward on the torso or a 40 kg·m/s impulse applied rearward. It also maintains stable walking with a 10 kg backpack or a 25 kg front pack. The controller was trained on a 1.8 m tall model, but also stabilizes models 1.4-2.3 m tall with no changes.

Leg-local neural mechanisms for searching and learning enhance robotic locomotion.

Adapting motor output based on environmental forces is critical for successful locomotion in the real world. Arthropods use at least two neural mechanisms to adjust muscle activation while walking based on detected forces. Mechanism 1 uses negative feedback of leg depressor force to ensure that each stance leg supports an appropriate amount of the body's weight. Mechanism 2 encourages searching for ground contact if the leg supports no body weight. We expand the neural controller for MantisBot, a robot based upon a praying mantis, to include these mechanisms by incorporating leg-local memory and command neurons, as observed in arthropods. We present results from MantisBot transitioning between searching and stepping, mimicking data from animals as reported in the literature.

A Functional Subnetwork Approach to Designing Synthetic Nervous Systems That Control Legged Robot Locomotion.

A dynamical model of an animal's nervous system, or synthetic nervous system (SNS), is a potentially transformational control method. Due to increasingly detailed data on the connectivity and dynamics of both mammalian and insect nervous systems, controlling a legged robot with an SNS is largely a problem of parameter tuning. Our approach to this problem is to design functional subnetworks that perform specific operations, and then assemble them into larger models of the nervous system. In this paper, we present networks that perform addition, subtraction, multiplication, division, differentiation, and integration of incoming signals. Parameters are set within each subnetwork to produce the desired output by utilizing the operating range of neural activity, R, the gain of the operation, k, and bounds based on biological values. The assembly of large networks from functional subnetworks underpins our recent results with MantisBot.

A neural network with central pattern generators entrained by sensory feedback controls walking of a bipedal model.

A neuromechanical simulation of a planar, bipedal walking robot has been developed. It is constructed as a simplified, planar musculoskeletal model of the biomechanics of the human lower body. The controller consists of a dynamic neural network with central pattern generators (CPGs) entrained by force and movement sensory feedback to generate appropriate muscle forces for walking. The CPG model is a two-level architecture, which consists of separate rhythm generator and pattern formation networks. The biped model walks stably in the sagittal plane without inertial sensors or a centralized posture controller or a 'baby walker' to help overcome gravity. Its gait is similar to humans' and it walks at speeds from 0.850 m s-1 up to 1.289 m s-1 with leg length of 0.84 m. The model walks over small unknown steps (6% of leg length) and up and down 5° slopes without any additional higher level control actions.

A muscle-driven approach to restore stepping with an exoskeleton for individuals with paraplegia.

BACKGROUND: Functional neuromuscular stimulation, lower limb orthosis, powered lower limb exoskeleton, and hybrid neuroprosthesis (HNP) technologies can restore stepping in individuals with paraplegia due to spinal cord injury (SCI). However, a self-contained muscle-driven controllable exoskeleton approach based on an implanted neural stimulator to restore walking has not been previously demonstrated, which could potentially result in system use outside the laboratory and viable for long term use or clinical testing. In this work, we designed and evaluated an untethered muscle-driven controllable exoskeleton to restore stepping in three individuals with paralysis from SCI. METHODS: The self-contained HNP combined neural stimulation to activate the paralyzed muscles and generate joint torques for limb movements with a controllable lower limb exoskeleton to stabilize and support the user. An onboard controller processed exoskeleton sensor signals, determined appropriate exoskeletal constraints and stimulation commands for a finite state machine (FSM), and transmitted data over Bluetooth to an off-board computer for real-time monitoring and data recording. The FSM coordinated stimulation and exoskeletal constraints to enable functions, selected with a wireless finger switch user interface, for standing up, standing, stepping, or sitting down. In the stepping function, the FSM used a sensor-based gait event detector to determine transitions between gait phases of double stance, early swing, late swing, and weight acceptance. RESULTS: The HNP restored stepping in three individuals with motor complete paralysis due to SCI. The controller appropriately coordinated stimulation and exoskeletal constraints using the sensor-based FSM for subjects with different stimulation systems. The average range of motion at hip and knee joints during walking were 8.5°-20.8° and 14.0°-43.6°, respectively. Walking speeds varied from 0.03 to 0.06 m/s, and cadences from 10 to 20 steps/min. CONCLUSIONS: A self-contained muscle-driven exoskeleton was a feasible intervention to restore stepping in individuals with paraplegia due to SCI. The untethered hybrid system was capable of adjusting to different individuals' needs to appropriately coordinate exoskeletal constraints with muscle activation using a sensor-driven FSM for stepping. Further improvements for out-of-the-laboratory use should include implantation of plantar flexor muscles to improve walking speed and power assist as needed at the hips and knees to maintain walking as muscles fatigue.