Open-ended movements structure sensorimotor information in early human development.

Human behaviors, with whole-body coordination, involve large-scale sensorimotor interaction. Spontaneous bodily movements in the early developmental stage potentially lead toward acquisition of such coordinated behavior. These movements presumably contribute to the structuration of sensorimotor interaction, providing specific regularities in bidirectional information among muscle activities and proprioception. Whether and how spontaneous movements, despite being task-free, structure and organize sensorimotor interactions in the entire body during early development remain unknown. Herein, to address these issues, we gained insights into the structuration process of the sensorimotor interaction in neonates and 3-mo-old infants. By combining detailed motion capture and musculoskeletal simulation, sensorimotor information flows among muscle activities and proprioception throughout the body were obtained. Subsequently, we extracted spatial modules and temporal state in sensorimotor information flows. Our approach demonstrated that early spontaneous movements elicited body-dependent sensorimotor modules, revealing age-related changes in them, depending on the combination or direction. The sensorimotor interactions also displayed temporal non-random fluctuations analogous to those seen in spontaneous activities in the cerebral cortex and spinal cord. Furthermore, we found recurring state sequence patterns across multiple participants, characterized by a substantial increase in infants compared to the patterns in neonates. Therefore, early spontaneous movements induce the spatiotemporal structuration in sensorimotor interactions and subsequent developmental changes. These results implicated that early open-ended movements, emerging from a certain neural substrate, regulate the sensorimotor interactions through embodiment and contribute to subsequent coordinated behaviors. Our findings also provide a conceptual linkage between early spontaneous movements and spontaneous neuronal activity in terms of spatiotemporal characteristics.

Spike-induced ordering: Stochastic neural spikes provide immediate adaptability to the sensorimotor system.

Most biological neurons exhibit stochastic and spiking action potentials. However, the benefits of stochastic spikes versus continuous signals other than noise tolerance and energy efficiency remain largely unknown. In this study, we provide an insight into the potential roles of stochastic spikes, which may be beneficial for producing on-site adaptability in biological sensorimotor agents. We developed a platform that enables parametric modulation of the stochastic and discontinuous output of a stochastically spiking neural network (sSNN) to the rate-coded smooth output. This platform was applied to a complex musculoskeletal-neural system of a bipedal walker, and we demonstrated how stochastic spikes may help improve on-site adaptability of a bipedal walker to slippery surfaces or perturbation of random external forces. We further applied our sSNN platform to more general and simple sensorimotor agents and demonstrated four basic functions provided by an sSNN: 1) synchronization to a natural frequency, 2) amplification of the resonant motion in a natural frequency, 3) basin enlargement of the behavioral goal state, and 4) rapid complexity reduction and regular motion pattern formation. We propose that the benefits of sSNNs are not limited to musculoskeletal dynamics. Indeed, a wide range of the stability and adaptability of biological systems may arise from stochastic spiking dynamics.

A 3D printed hydrostatic skeleton for an earthworm-inspired soft burrowing robot.

Moving through soil is challenging for robots, particularly for soft robots. Herein, we propose a support structure, based on the hydrostatic skeleton of earthworms, to overcome this problem. To create extremely flexible, thin-walled, worm-sized deformed segments, a specialized 3D printer for low-hardness rubber was utilized. To obtain large radial deformation, we investigated the properties of the soft materials for 3D printing and the geometry of the segments. Notably, segments are deformed with multiply-wound shape memory alloy wires. We constructed an earthworm robot by connecting shape memory alloy-driven segments in series and experimentally demonstrated that this robot could propel in the soil. The proposed robot is unique in that it has a small diameter of 10 mm and exhibits a peristaltic motion in soil.

Direct evidence of EEG coherence in alleviating phantom limb pain by virtual referred sensation: Case report.

Virtual reality (VR) systems have been integrated into rehabilitation techniques for phantom limb pain (PLP). In this case report, we used electroencephalography (EEG) to analyze corticocortical coherence between the bilateral sensorimotor cortices during vibrotactile stimulation in conjunction with VR rehabilitation in two PLP patients. As a result, we observed PLP alleviation and increased alpha wave coherence during VR rehabilitation when stimulation was delivered to the cheek and shoulder (referred sensation areas) of the affected side. Vibrotactile stimulation with VR rehabilitation may enhance the awareness and movement of the phantom hand.

Stapedial reflex threshold predicts individual loudness tolerance for people with autistic spectrum disorders.

People with autism spectrum disorder (ASD) frequently show the symptoms of oversensitivity to sound (hyperacusis). Although the previous studies have investigated methods for quantifying hyperacusis in ASD, appropriate physiological signs for quantifying hyperacusis in ASD remain poorly understood. Here, we investigated the relationship of loudness tolerance with the threshold of the stapedial reflex and with contralateral suppression of the distortion product otoacoustic emissions, which has been suggested to be related to hyperacusis in people without ASD. We tested an ASD group and a neurotypical group. The results revealed that only the stapedial reflex threshold was significantly correlated with loudness tolerance in both groups. In addition to reduced loudness tolerance, people with lower stapedial reflex thresholds also exhibited higher scores on the Social Responsiveness Scale-2.

Developmental changes in intralimb coordination during spontaneous movements of human infants from 2 to 3 months of age.

Human infants show a variety of spontaneous movements in the first few months of life. Although the pattern of spontaneous movements changes at approximately 2 months of age, the precise mechanism that governs the developmental changes in intralimb coordination remains unclear. In the present study, we focused on knee-ankle coordination during spontaneous movements of human infants from 2 to 3 months of age. Multiple attitude sensors were used to measure three-dimensional angular motion of knee and ankle joint motions. We acquired a one-dimensional time series of the knee joint angle around the putative hinge joint and a two-dimensional time series of ankle motions on the putative sagittal and frontal plane. First, we found that 3-month-old infants show a significant predominance to extend their knee joints, remarkably so on the left side. To quantify dissociated motions of the knee and ankle, we calculated the temporal correlation and the regression slope between them. We observed that 3-month-old infants moved their ankle joints more independently of knee motions than 2-month-old infants. Finally, we found that dissociated motions of the knee and ankle simultaneously develop with knee extension predominance. The developmental change from synchronization to dissociation of intralimb joint movements during spontaneous movements suggests that the development of the cortical and/or subcortical mechanism may mediate selective activation and inhibition of joint motions at approximately 2 months of age.

Tactile feedback for relief of deafferentation pain using virtual reality system: a pilot study.

BACKGROUND: Previous studies have tried to relieve deafferentation pain (DP) by using virtual reality rehabilitation systems. However, the effectiveness of multimodal sensory feedback was not validated. The objective of this study is to relieve DP by neurorehabilitation using a virtual reality system with multimodal sensory feedback and to validate the efficacy of tactile feedback on immediate pain reduction. METHODS: We have developed a virtual reality rehabilitation system with multimodal sensory feedback and applied it to seven patients with DP caused by brachial plexus avulsion or arm amputation. The patients executed a reaching task using the virtual phantom limb manipulated by their real intact limb. The reaching task was conducted under two conditions: one with tactile feedback on the intact hand and one without. The pain intensity was evaluated through a questionnaire. RESULTS: We found that the task with the tactile feedback reduced DP more (41.8 ± 19.8 %) than the task without the tactile feedback (28.2 ± 29.5 %), which was supported by a Wilcoxon signed-rank test result (p < 0.05). CONCLUSIONS: Overall, our findings indicate that the tactile feedback improves the immediate pain intensity through rehabilitation using our virtual reality system.

Emergence of integrated behaviors through direct optimization for homeostasis.

Homeostasis is a self-regulatory process, wherein an organism maintains a specific internal physiological state. Homeostatic reinforcement learning (RL) is a framework recently proposed in computational neuroscience to explain animal behavior. Homeostatic RL organizes the behaviors of autonomous embodied agents according to the demands of the internal dynamics of their bodies, coupled with the external environment. Thus, it provides a basis for real-world autonomous agents, such as robots, to continually acquire and learn integrated behaviors for survival. However, prior studies have generally explored problems pertaining to limited size, as the agent must handle observations of such coupled dynamics. To overcome this restriction, we developed an advanced method to realize scaled-up homeostatic RL using deep RL. Furthermore, several rewards for homeostasis have been proposed in the literature. We identified that the reward definition that uses the difference in drive function yields the best results. We created two benchmark environments for homeostasis and performed a behavioral analysis. The analysis showed that the trained agents in each environment changed their behavior based on their internal physiological states. Finally, we extended our method to address vision using deep convolutional neural networks. The analysis of a trained agent revealed that it has visual saliency rooted in the survival environment and internal representations resulting from multimodal input.

Embedding Bifurcations into Pneumatic Artificial Muscle.

Harnessing complex body dynamics has long been a challenge in robotics, particularly when dealing with soft dynamics, which exhibit high complexity in interacting with the environment. Recent studies indicate that these dynamics can be used as a computational resource, exemplified by the McKibben pneumatic artificial muscle, a common soft actuator. This study demonstrates that bifurcations, including periodic and chaotic dynamics, can be embedded into the pneumatic artificial muscle, with the entire bifurcation structure using the framework of physical reservoir computing. These results suggest that dynamics not present in training data can be embedded through bifurcation embedment, implying the capability to incorporate various qualitatively different patterns into pneumatic artificial muscle without the need to design and learn all required patterns explicitly. Thus, this study introduces a novel approach to simplify robotic devices and control training by reducing reliance on external pattern generators and the amount and types of training data needed for control.

Respiratory entrainment of the locus coeruleus modulates arousal level to avoid physical risks from external vibration.

Slow rocking chairs can easily put people to sleep, while violent shaking, such as during earthquakes, may lead to rapid awakening. However, the influence of external body vibrations on arousal remains unclear. Herein, a computational model of a locus coeruleus (LC)-norepinephrine (NE) system and cardio-respiratory system were used to show that respiratory entrainment of the LC modulates arousal levels, which is an adaptation to avoid physical risks from external vibration. External vibrations of sinusoidal waves with different frequencies ranging from 0.1 to 20 [Hz] were applied to the LC based on the results of previous studies. We found that respiratory entrainment of the LC decreased the breathing rate (BR) and heart rate (HR) to maintain the HR within its normal range. Furthermore, 1:1 phase locking enhanced arousal level while phase-amplitude coupling decreased it for larger vibration stimuli. These findings suggest that respiratory entrainment of the LC might automatically modulate cardio-respiratory system homeostasis and arousal levels for performance readiness (fight/flight or freeze) to avoid physical risks from larger external vibrations.

Curriculum-reinforcement learning on simulation platform of tendon-driven high-degree of freedom underactuated manipulator.

A high degree of freedom (DOF) benefits manipulators by presenting various postures when reaching a target. Using a tendon-driven system with an underactuated structure can provide flexibility and weight reduction to such manipulators. The design and control of such a composite system are challenging owing to its complicated architecture and modeling difficulties. In our previous study, we developed a tendon-driven, high-DOF underactuated manipulator inspired from an ostrich neck referred to as the Robostrich arm. This study particularly focused on the control problems and simulation development of such a tendon-driven high-DOF underactuated manipulator. We proposed a curriculum-based reinforcement-learning approach. Inspired by human learning, progressing from simple to complex tasks, the Robostrich arm can obtain manipulation abilities by step-by-step reinforcement learning ranging from simple position control tasks to practical application tasks. In addition, an approach was developed to simulate tendon-driven manipulation with a complicated structure. The results show that the Robostrich arm can continuously reach various targets and simultaneously maintain its tip at the desired orientation while mounted on a mobile platform in the presence of perturbation. These results show that our system can achieve flexible manipulation ability even if vibrations are presented by locomotion.

Characterization of continuum robot arms under reinforcement learning and derived improvements.

In robotics, soft continuum robot arms are a promising prospect owing to their redundancy and passivity; however, no comprehensive study exists that examines their characteristics compared to rigid manipulators. In this study, we examined the advantages of a continuum robot arm as compared to a typical and rigid seven-degree-of-freedom (7-DoF) robot manipulator in terms of performing various tasks through reinforcement learning. We conducted simulations for tasks with different characteristics that require control over position and force. Common tasks in robot manipulators, such as reaching, crank rotation, object throwing, and peg-in-hole were considered. The initial conditions of the robot and environment were randomized, aiming for evaluations including robustness. The results indicate that the continuum robot arm excels in the crank-rotation task, which is characterized by uncertainty in environmental conditions and cumulative rewards. However, the rigid robot arm learned better motions for the peg-in-hole task than the other tasks, which requires fine motion control of the end-effector. In the throwing task, the continuum robot arm scored well owing to its good handling of anisotropy. Moreover, we developed a reinforcement-learning method based on the comprehensive experimental results. The proposed method successfully improved the motion learning of a continuum robot arm by adding a technique to regulate the initial state of the robot. To the best of our knowledge, ours is the first reinforcement-learning experiment with multiple tasks on a single continuum robot arm and is the first report of a comparison between a single continuum robot arm and rigid manipulator on a wide range of tasks. This simulation study can make a significant contribution to the design of continuum arms and specification of their applications, and development of control and reinforcement learning methods.

Active pole bending effect in pole support phase.

In pole vaulting, model analysis is one of the key methods to increase vaulting height. To date, the effects of athletes' motions during 'pole support phase' have been measured and modelled to improve and set new world records. The motions were extracted based on the context of pole bending interaction and parameters to improve vaulting height were investigated. However, due to experimental, mechanical, and sensing restrictions, ranges and interactions of the parameters were poorly addressed. To investigate further, a parameter space must be globally explored. Here, we show parameter sensitivities and interactive effects between initial velocity, pole length, bending amplitude and switching time. From the simulation studies, we found that active pole bending enabled successful pole vaulting with lower initial velocity and longer poles. Vaulting height had a local maximum point at a specific initial velocity and positive bending could control conditions to deliver the local maximum height. Positive bending controls the rising-up speed of the pole and contributes to the verticalisation of the vaulting angle. Negative bending increases the vaulting speed and contributes to the robustness of the vaulting angle. Our results demonstrate how these parameters affect the vaulting performances and suggest how athletes should activate their bodies.

Skeletonizing the Dynamics of Soft Continuum Body from Video.

Soft continuum bodies have demonstrated their effectiveness in generating flexible and adaptive functionalities by capitalizing on the rich deformability of soft material. Compared with a rigid-body robot, it is in general difficult to model and emulate the morphology dynamics of a soft continuum body. In addition, a soft continuum body potentially has an infinite degree of freedom, requiring considerable labor to manually annotate its dynamics from external sensory data such as video. In this study, we propose a novel noninvasive framework for automatically extracting the skeletal dynamics from video of a soft continuum body and show the applications and effectiveness of our framework. First, we demonstrate that our framework can extract skeletal dynamics from animal videos, which can be effectively utilized for the analysis of soft continuum body including animal motion. Next, we focus on a soft continuum arm, a commonly used platform in soft robotics, and evaluate the potential information-processing capability. Normally, to control such a high-dimensional system, it is necessary to introduce many sensors to completely capture the motion dynamics, causing the deterioration of the material's softness. We illustrate that the evaluation of the memory capacity and sensory reconstruction error enables us to verify the minimum number of sensors sufficient for fully grasping the state dynamics, which is highly useful in designing a sensor arrangement for a soft robot. Also, we release the software developed in this study as open source for biology and soft robotics communities, which contributes to automating the annotation process required for the motion analysis of soft continuum bodies.

Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware.

Ever-growing demand for artificial intelligence has motivated research on unconventional computation based on physical devices. While such computation devices mimic brain-inspired analog information processing, the learning procedures still rely on methods optimized for digital processing such as backpropagation, which is not suitable for physical implementation. Here, we present physical deep learning by extending a biologically inspired training algorithm called direct feedback alignment. Unlike the original algorithm, the proposed method is based on random projection with alternative nonlinear activation. Thus, we can train a physical neural network without knowledge about the physical system and its gradient. In addition, we can emulate the computation for this training on scalable physical hardware. We demonstrate the proof-of-concept using an optoelectronic recurrent neural network called deep reservoir computer. We confirmed the potential for accelerated computation with competitive performance on benchmarks. Our results provide practical solutions for the training and acceleration of neuromorphic computation.

Blower-Powered Soft Inflatable Joints for Physical Human-Robot Interaction.

Inflatables are safe and lightweight structures even at the human scale. Inflatable robots are expected to be applied to physical human-robot interaction (pHRI). Although active joint mechanisms are essential for developing inflatable robots, the existing mechanisms are complex in structure and it is difficult to integrate actuators, which diminish the advantages of inflatables. This study proposes blower-powered soft inflatable joints that are easy to fabricate and contain enough space for an actuation inside. The joints are driven by tendon wires pulled by linear actuators. We derived a theoretical model for both unilateral and bilateral joints and demonstrated a hugging robot with multiple joints as an application of the proposed joint mechanism. The novelty of the proposed joint mechanism and the inflatable robot is that rigid parts have been thoroughly eliminated and the tendons for actuation have been successfully hidden inside. Moreover, the active control of the internal pressure makes inflatables resistant to punctures. We expect that the contact safety of inflatable robots will facilitate advancement of the pHRI field.

Reward sensitivity differs depending on global self-esteem in value-based decision-making.

Global self-esteem is a component of individual personality that impacts decision-making. Many studies have discussed the different preferences for decision-making in response to threats to a person's self-confidence, depending on global self-esteem. However, studies about global self-esteem and non-social decision-making have indicated that decisions differ due to reward sensitivity. Here, reward sensitivity refers to the extent to which rewards change decisions. We hypothesized that individuals with lower global self-esteem have lower reward sensitivity and investigated the relationship between self-esteem and reward sensitivity using a computational model. We first examined the effect of expected value and maximum value in learning under uncertainties because some studies have shown the possibility of saliency (e.g. maximum value) and relative value (e.g. expected value) affecting decisions, respectively. In our learning task, expected value affected decisions, but there was no significant effect of maximum value. Therefore, we modelled participants' choices under the condition of different expected value without considering maximum value. We used the Q-learning model, which is one of the traditional computational models in explaining experiential learning decisions. Global self-esteem correlated positively with reward sensitivity. Our results suggest that individual reward sensitivity affects decision-making depending on one's global self-esteem.

Designing spontaneous behavioral switching via chaotic itinerancy.

Chaotic itinerancy is a frequently observed phenomenon in high-dimensional nonlinear dynamical systems and is characterized by itinerant transitions among multiple quasi-attractors. Several studies have pointed out that high-dimensional activity in animal brains can be observed to exhibit chaotic itinerancy, which is considered to play a critical role in the spontaneous behavior generation of animals. Thus, how to design desired chaotic itinerancy is a topic of great interest, particularly for neurorobotics researchers who wish to understand and implement autonomous behavioral controls. However, it is generally difficult to gain control over high-dimensional nonlinear dynamical systems. In this study, we propose a method for implementing chaotic itinerancy reproducibly in a high-dimensional chaotic neural network. We demonstrate that our method enables us to easily design both the trajectories of quasi-attractors and the transition rules among them simply by adjusting the limited number of system parameters and by using the intrinsic high-dimensional chaos.

Cross-modal and scale-free action representations through enaction.

Embodied action representation and action understanding are the first steps to understand what it means to communicate. We present a biologically plausible mechanism to the representation and the recognition of actions in a neural network with spiking neurons based on the learning mechanism of spike-timing-dependent plasticity (STDP). We show how grasping is represented through the multi-modal integration between the vision and tactile maps across multiple temporal scales. The network evolves into a small-world organization with scale-free dynamics promoting efficient inter-modal binding of the neural assemblies with accurate timing. Finally, it acquires the qualitative properties of the mirror neuron system to trigger an observed action performed by someone else.

Fusing autonomy and sociality via embodied emergence and development of behaviour and cognition from fetal period.

Human-centred AI/Robotics are quickly becoming important. Their core claim is that AI systems or robots must be designed and work for the benefits of humans with no harm or uneasiness. It essentially requires the realization of autonomy, sociality and their fusion at all levels of system organization, even beyond programming or pre-training. The biologically inspired core principle of such a system is described as the emergence and development of embodied behaviour and cognition. The importance of embodiment, emergence and continuous autonomous development is explained in the context of developmental robotics and dynamical systems view of human development. We present a hypothetical early developmental scenario that fills in the very beginning part of the comprehensive scenarios proposed in developmental robotics. Then our model and experiments on emergent embodied behaviour are presented. They consist of chaotic maps embedded in sensory-motor loops and coupled via embodiment. Behaviours that are consistent with embodiment and adaptive to environmental structure emerge within a few seconds without any external reward or learning. Next, our model and experiments on human fetal development are presented. A precise musculo-skeletal fetal body model is placed in a uterus model. Driven by spinal nonlinear oscillator circuits coupled together via embodiment, somatosensory signals are evoked and learned by a model of the cerebral cortex with 2.6 million neurons and 5.3 billion synapses. The model acquired cortical representations of self-body and multi-modal sensory integration. This work is important because it models very early autonomous development in realistic detailed human embodiment. Finally, discussions toward human-like cognition are presented including other important factors such as motivation, emotion, internal organs and genetic factors. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.