Simultaneous Optimization of Discrete and Continuous Parameters Defining a Robot Morphology and Controller.

The morphology and controller design of robots is often a labor-intensive task performed by experienced and intuitive engineers. Automatic robot design using machine learning is attracting increasing attention in the hope that it will reduce the design workload and result in better-performing robots. Most robots are created by joining several rigid parts and then mounting actuators and their controllers. Many studies limit the possible types of rigid parts to a finite set to reduce the computational burden. However, this not only limits the search space, but also prohibits the use of powerful optimization techniques. To find a robot closer to the global optimal design, a method that explores a richer set of robots is desirable. In this article, we propose a novel method to efficiently search for various robot designs. The method combines three different optimization methods with different characteristics. We apply proximal policy optimization (PPO) or soft actor-critic (SAC) as the controller, the REINFORCE algorithm to determine the lengths and other numerical parameters of the rigid parts, and a newly proposed method to determine the number and layout of the rigid parts and joints. Experiments with physical simulations confirm that when this method is used to handle two types of tasks-walking and manipulation-it performs better than simple combinations of existing methods. The source code and videos of our experiments are available online (https://github.com/r-koike/eagent).

Finite-Level Quantized Min-Consensus Control Based on Encoding-Decoding.

This article studies the min-consensus control of continuous-time real-valued multiagent systems, with sampled information, quantized communication, and switching topologies. Due to the limited bandwidth of the digital communication network, only finite-bit binary symbolic sequence can be exchanged among the agents. In order to realize the min-consensus control with quantized communication and limited bandwidth, a novel finite-level biased quantizer and a nonstrict decreasing scaling function are designed, and correspondingly a set of switching encoders and decoders are constructed. By means of the proposed encoders and decoders, the according sampled-data min-consensus control inputs are carefully constructed, and the memory variables are introduced into the control inputs and are monotonically decreasing no matter how the communication topology is switched. The proposed encoding-decoding-based control scheme can achieve accurate min-consensus with limited bandwidth, as long as the communication graphs are jointly strongly connected. The numerical simulations show the effectiveness of the proposed control scheme.

Fore-Aft Asymmetry Improves the Stability of Trotting in the Transverse Plane: A Modeling Study.

Quadrupedal mammals have fore-aft asymmetry in their body structure, which affects their walking and running dynamics. However, the effects of asymmetry, particularly in the transverse plane, remain largely unclear. In this study, we examined the effects of fore-aft asymmetry on quadrupedal trotting in the transverse plane from a dynamic viewpoint using a simple model, which consists of two rigid bodies connected by a torsional joint with a torsional spring and four spring legs. Specifically, we introduced fore-aft asymmetry into the model by changing the physical parameters between the fore and hind parts of the model based on dogs, which have a short neck, and horses, which have a long neck. We numerically searched the periodic solutions for trotting and investigated the obtained solutions and their stability. We found that three types of periodic solutions with different foot patterns appeared that depended on the asymmetry. Additionally, the asymmetry improved gait stability. Our findings improve our understanding of gait dynamics in quadrupeds with fore-aft asymmetry.

Three Characteristics of Cheetah Galloping Improve Running Performance Through Spinal Movement: A Modeling Study.

Cheetahs are the fastest land animal. Their galloping shows three characteristics: small vertical movement of their center of mass, small whole-body pitching movement, and large spine bending movement. We hypothesize that these characteristics lead to enhanced gait performance in cheetahs, including higher gait speed. In this study, we used a simple model with a spine joint and torsional spring, which emulate the body flexibility, to verify our hypothesis from a dynamic perspective. Specifically, we numerically searched periodic solutions and evaluated what extent each solution shows the three characteristics. We then evaluated the gait performance and found that the solutions with the characteristics achieve high performances. This result supports our hypothesis. Furthermore, we revealed the mechanism for the high performances through the dynamics of the spine movement. These findings extend the current understanding of the dynamic mechanisms underlying high-speed locomotion in cheetahs.

Center of Mass Offset Enhances the Selection of Transverse Gallop in High-Speed Running by Horses: A Modeling Study.

Horses use the transverse gallop in high-speed running. However, different animals use different gaits, and the gait preference of horses remains largely unclear. Horses have fore-aft asymmetry in their body structure and their center of mass (CoM) is anteriorly located far from the center of the body. Since such a CoM offset affects the running dynamics, we hypothesize that the CoM offset of horses is important in gait selection. In order to verify our hypothesis and clarify the gait selection mechanisms by horses from a dynamic viewpoint, we developed a simple model with CoM offset and investigated its effects on running. Specifically, we numerically obtained periodic solutions and classified these solutions into six types of gaits, including the transverse gallop, based on the footfall pattern. Our results show that the transverse gallop is optimal when the CoM offset is located at the position estimated in horses. Our findings provide useful insight into the gait selection mechanisms in high-speed running of horses.

Path Integral Policy Improvement With Population Adaptation.

Path integral policy improvement (PI2) is known to be an efficient reinforcement learning algorithm, particularly, if the target system is a high-dimensional dynamical system. However, PI2, and its existing extensions, have adjustable parameters, on which the efficiency depends significantly. This article proposes an extension of PI2 that adjusts all of the critical parameters automatically. Motion acquisition tasks for three different types of simulated legged robots were performed to test the efficacy of the proposed algorithm. The results show that the proposed method cannot only eliminate the burden on the user to set the parameters appropriately but also improve the optimization performance significantly. For one of the acquired motions, a real robot experiment was conducted to show the validity of the motion.

Generation of Direct-, Retrograde-, and Source-Wave Gaits in Multi-Legged Locomotion in a Decentralized Manner via Embodied Sensorimotor Interaction.

Multi-legged animals show several types of ipsilateral interlimb coordination. Millipedes use a direct-wave gait, in which the swing leg movements propagate from posterior to anterior. In contrast, centipedes use a retrograde-wave gait, in which the swing leg movements propagate from anterior to posterior. Interestingly, when millipedes walk in a specific way, both direct and retrograde waves of the swing leg movements appear with the waves' source, which we call the source-wave gait. However, the gait generation mechanism is still unclear because of the complex nature of the interaction between neural control and dynamic body systems. The present study used a simple model to understand the mechanism better, primarily how local sensory feedback affects multi-legged locomotion. The model comprises a multi-legged body and its locomotion control system using biologically inspired oscillators with local sensory feedback, phase resetting. Each oscillator controls each leg independently. Our simulation produced the above three types of animal gaits. These gaits are not predesigned but emerge through the interaction between the neural control and dynamic body systems through sensory feedback (embodied sensorimotor interaction) in a decentralized manner. The analytical description of these gaits' solution and stability clarifies the embodied sensorimotor interaction's functional roles in the interlimb coordination.

Dynamical determinants enabling two different types of flight in cheetah gallop to enhance speed through spine movement.

Cheetahs use a galloping gait in their fastest speed range. It has been reported that cheetahs achieve high-speed galloping by performing two types of flight through spine movement (gathered and extended). However, the dynamic factors that enable cheetahs to incorporate two types of flight while galloping remain unclear. To elucidate this issue from a dynamical viewpoint, we developed a simple analytical model. We derived possible periodic solutions with two different flight types (like cheetah galloping), and others with only one flight type (unlike cheetah galloping). The periodic solutions provided two criteria to determine the flight type, related to the position and magnitude of ground reaction forces entering the body. The periodic solutions and criteria were verified using measured cheetah data, and provided a dynamical mechanism by which galloping with two flight types enhances speed. These findings extend current understanding of the dynamical mechanisms underlying high-speed locomotion in cheetahs.

Haptic vs. Visual Neurofeedback for Brain Training: A Proof-of-Concept Study.

The current practice of administering neurofeedback using the patients' visual and/or auditory channel(s) is known to cause fatigue, excessive boredom, and restricted mobility during prolonged therapy sessions. This paper proposes haptics as an alternative means to provide neurofeedback and investigates its effectiveness by conducting two user studies (Study- I & II) using a novel compact wearable haptic device that provides vibrotactile feedback to the user's neck. Each user study has three neurofeedback modes: visual-only, haptics-only, and visual-and-haptics combined. Study- I examines the participant's performance in a brain-training task by measuring their attention level (AL) and the task completion time (CT). Study- II, in addition to the brain-training task, investigates the participants' ability to perform a secondary task (playing a shape-sorting game) while receiving the neurofeedback. Results show that users performed similarly well in brain-training with haptics-only and visual-only feedback. However, when engaged in a secondary task, the users performed significantly better (AL and CT improved around 11% and 17%, respectively) with haptics, indicating a clear advantage of haptics over visual neurofeedback. Being able to perform routine activities during brain-training would likely increase user adherence to longer therapy sessions. In the future, we plan to verify these findings by conducting experiments on ADHD-patients.

Unified Approach to the Motion Design for a Snake Robot Negotiating Complicated Pipe Structures.

A unified method for designing the motion of a snake robot negotiating complicated pipe structures is presented. Such robots moving inside pipes must deal with various "obstacles," such as junctions, bends, diameter changes, shears, and blockages. To surmount these obstacles, we propose a method that enables the robot to adapt to multiple pipe structures in a unified way. This method also applies to motion that is necessary to pass between the inside and the outside of a pipe. We designed the target form of the snake robot using two helices connected by an arbitrary shape. This method can be applied to various obstacles by designing a part of the target form specifically for given obstacles. The robot negotiates obstacles under shift control by employing a rolling motion. Considering the slip between the robot and the pipe, the model expands the method to cover cases where two helices have different properties. We demonstrated the effectiveness of the proposed method in various experiments.

Body torsional flexibility effects on stability during trotting and pacing based on a simple analytical model.

Quadruped animals use not only their legs but also their trunks during walking and running. Although many previous studies have investigated the flexion, extension, and lateral bending of the trunk, few studies have investigated the body torsion, and its dynamic effects on locomotion thus remain unclear. In this study, we investigated the effects of body torsion on gait stability during trotting and pacing. Specifically, we constructed a simple model consisting of two rigid bodies connected via a torsional joint that has a torsional spring and four leg springs. We then derived periodic solutions for trotting and pacing and evaluated the stabilities of these motion types using a Poincaré map. We found that the moments of inertia of the bodies and the spring constant ratio of the torsional spring and the leg springs determine the stability of these periodic solutions. We then determined the stability conditions for these parameters and elucidated the relevant mechanisms. In addition, we clarified the importance of the body torsion to the gait stability by comparison with a rigid model. Finally, we analyzed the biological relevance of our findings and provided a design principle for development of quadruped robots.

Behavioral responses to colony-level properties affect disturbance resistance of red harvester ant colonies.

Self-organizing biological systems, such as colonies of social insects, are characterized by their decentralized control and flexible responses to changing environments, often likened to swarm intelligence. Although decentralized control is well known to be a product of local interactions among agents, without the need for a bird's-eye view, indirect knowledge of properties that indicate the current states of the entire system also helps each agent to respond to changes, thereby leading to a more adaptive system. In this study, we analyze the rules that govern workers' behavioral responses to colony-level properties and assess whether they contribute to adaptive flexibility in social insect colonies. We focus on task allocation among red harvester ants (Pogonomyrmex barbatus) as a model system and develop an ordinary differential equation model to describe the system of task allocation among workers. We simulate 12 scenarios specifying how workers respond to changes in the colony-level properties of colony size and nutritional state. We found that when workers decrease their contact rates in response to increasing colony size, they enable achievement of a larger colony size, similar to that of P. barbatus colonies in nature, and when workers increase their foraging levels in response to decreasing colony-wide nutritional levels, they increase resilience to environmental disturbances. These negative feedback rules governing the response to colony-level properties are consistent with previous reports on ants and honeybees.

Simple analytical model reveals the functional role of embodied sensorimotor interaction in hexapod gaits.

Insects have various gaits with specific characteristics and can change their gaits smoothly in accordance with their speed. These gaits emerge from the embodied sensorimotor interactions that occur between the insect's neural control and body dynamic systems through sensory feedback. Sensory feedback plays a critical role in coordinated movements such as locomotion, particularly in stick insects. While many previously developed insect models can generate different insect gaits, the functional role of embodied sensorimotor interactions in the interlimb coordination of insects remains unclear because of their complexity. In this study, we propose a simple physical model that is amenable to mathematical analysis to explain the functional role of these interactions clearly. We focus on a foot contact sensory feedback called phase resetting, which regulates leg retraction timing based on touchdown information. First, we used a hexapod robot to determine whether the distributed decoupled oscillators used for legs with the sensory feedback generate insect-like gaits through embodied sensorimotor interactions. The robot generated two different gaits and one had similar characteristics to insect gaits. Next, we proposed the simple model as a minimal model that allowed us to analyze and explain the gait mechanism through the embodied sensorimotor interactions. The simple model consists of a rigid body with massless springs acting as legs, where the legs are controlled using oscillator phases with phase resetting, and the governed equations are reduced such that they can be explained using only the oscillator phases with some approximations. This simplicity leads to analytical solutions for the hexapod gaits via perturbation analysis, despite the complexity of the embodied sensorimotor interactions. This is the first study to provide an analytical model for insect gaits under these interaction conditions. Our results clarified how this specific foot contact sensory feedback contributes to generation of insect-like ipsilateral interlimb coordination during hexapod locomotion.

Adaptive Control Strategies for Interlimb Coordination in Legged Robots: A Review.

Walking animals produce adaptive interlimb coordination during locomotion in accordance with their situation. Interlimb coordination is generated through the dynamic interactions of the neural system, the musculoskeletal system, and the environment, although the underlying mechanisms remain unclear. Recently, investigations of the adaptation mechanisms of living beings have attracted attention, and bio-inspired control systems based on neurophysiological findings regarding sensorimotor interactions are being developed for legged robots. In this review, we introduce adaptive interlimb coordination for legged robots induced by various factors (locomotion speed, environmental situation, body properties, and task). In addition, we show characteristic properties of adaptive interlimb coordination, such as gait hysteresis and different time-scale adaptations. We also discuss the underlying mechanisms and control strategies to achieve adaptive interlimb coordination and the design principle for the control system of legged robots.

Task-Space Synchronization of Networked Mechanical Systems With Uncertain Parameters and Communication Delays.

This paper addresses the adaptive synchronization problem of networked mechanical systems in task space with time-varying communication delays, where both kinematic and dynamic uncertainties are considered and the information flow in the networks is represented by a directed graph. Based on a novel coordination auxiliary system, we first extend existing feedback architecture to achieve synchronization of networked mechanical systems in task space with slow-varying delays. Given that abrupt turns arise for the delays sometimes, we then propose a delay-independent adaptive synchronization control scheme which removes the requirement of the slow-varying condition. Both of the two control schemes are established with time-domain approaches by using Lyapunov-Krasovskii functions. Simulation results are provided to demonstrate the effectiveness of the proposed control schemes.

A galloping quadruped model using left-right asymmetry in touchdown angles.

Among quadrupedal gaits, the galloping gait has specific characteristics in terms of locomotor behavior. In particular, it shows a left-right asymmetry in gait parameters such as touchdown angle and the relative phase of limb movements. In addition, asymmetric gait parameters show a characteristic dependence on locomotion speed. There are two types of galloping gaits in quadruped animals: the transverse gallop, often observed in horses; and the rotary gallop, often observed in dogs and cheetahs. These two gaits have different footfall sequences. Although these specific characteristics in quadrupedal galloping gaits have been observed and described in detail, the underlying mechanisms remain unclear. In this paper, we use a simple physical model with a rigid body and four massless springs and incorporate the left-right asymmetry of touchdown angles. Our simulation results show that our model produces stable galloping gaits for certain combinations of model parameters and explains these specific characteristics observed in the quadrupedal galloping gait. The results are then evaluated in comparison with the measured data of quadruped animals and the gait mechanisms are clarified from the viewpoint of dynamics, such as the roles of the left-right touchdown angle difference in the generation of galloping gaits and energy transfer during one gait cycle to produce two different galloping gaits.

A novel EOG/EEG hybrid human-machine interface adopting eye movements and ERPs: application to robot control.

This study presents a novel human-machine interface (HMI) based on both electrooculography (EOG) and electroencephalography (EEG). This hybrid interface works in two modes: an EOG mode recognizes eye movements such as blinks, and an EEG mode detects event related potentials (ERPs) like P300. While both eye movements and ERPs have been separately used for implementing assistive interfaces, which help patients with motor disabilities in performing daily tasks, the proposed hybrid interface integrates them together. In this way, both the eye movements and ERPs complement each other. Therefore, it can provide a better efficiency and a wider scope of application. In this study, we design a threshold algorithm that can recognize four kinds of eye movements including blink, wink, gaze, and frown. In addition, an oddball paradigm with stimuli of inverted faces is used to evoke multiple ERP components including P300, N170, and VPP. To verify the effectiveness of the proposed system, two different online experiments are carried out. One is to control a multifunctional humanoid robot, and the other is to control four mobile robots. In both experiments, the subjects can complete tasks effectively by using the proposed interface, whereas the best completion time is relatively short and very close to the one operated by hand.