A Multi-Agent Reinforcement Learning Method for Omnidirectional Walking of Bipedal Robots.

Achieving omnidirectional walking for bipedal robots is considered one of the most challenging tasks in robotics technology. Reinforcement learning (RL) methods have proved effective in bipedal walking tasks. However, most existing methods use state machines to switch between multiple policies and achieve omnidirectional gait, which results in shaking during the policy switching process for bipedal robots. To achieve a seamless transition between omnidirectional gait and transient motion for full-size bipedal robots, we propose a novel multi-agent RL method. Firstly, a multi-agent RL algorithm based on the actor-critic framework is designed, and policy entropy is introduced to improve exploration efficiency. By learning agents with parallel initial state distributions, we minimize reliance on gait planner effectiveness in the Robot Operating System (ROS). Additionally, we design a novel heterogeneous policy experience replay mechanism based on Euclidean distance. Secondly, considering the periodicity of bipedal robot walking, we develop a new periodic gait function. Including periodic objectives in the policy can accelerate the convergence speed of training periodic gait functions. Finally, to enhance the robustness of the policy, we construct a novel curriculum learning method by discretizing Gaussian distribution and incorporate it into the robot's training task. Our method is validated in a simulation environment, and the results show that our method can achieve multiple gaits through a policy network and achieve smooth transitions between different gaits.

Incorporating structural plasticity into self-organization recurrent networks for sequence learning.

Introduction: Spiking neural networks (SNNs), inspired by biological neural networks, have received a surge of interest due to its temporal encoding. Biological neural networks are driven by multiple plasticities, including spike timing-dependent plasticity (STDP), structural plasticity, and homeostatic plasticity, making network connection patterns and weights to change continuously during the lifecycle. However, it is unclear how these plasticities interact to shape neural networks and affect neural signal processing. Method: Here, we propose a reward-modulated self-organization recurrent network with structural plasticity (RSRN-SP) to investigate this issue. Specifically, RSRN-SP uses spikes to encode information, and incorporate multiple plasticities including reward-modulated spike timing-dependent plasticity (R-STDP), homeostatic plasticity, and structural plasticity. On the one hand, combined with homeostatic plasticity, R-STDP is presented to guide the updating of synaptic weights. On the other hand, structural plasticity is utilized to simulate the growth and pruning of synaptic connections. Results and discussion: Extensive experiments for sequential learning tasks are conducted to demonstrate the representational ability of the RSRN-SP, including counting task, motion prediction, and motion generation. Furthermore, the simulations also indicate that the characteristics arose from the RSRN-SP are consistent with biological observations.

Design, Control, and Validation of a Symmetrical Hip and Straight-Legged Vertically-Compliant Bipedal Robot.

This paper presents the development, modeling, and control of L03, an underactuated 3D bipedal robot with symmetrical hips and straight legs. This innovative design requires only five actuators, two for the legs and three for the hips. This paper is divided into three parts: (1) mechanism design and kinematic analysis; (2) trajectory planning for the center of mass and foot landing points based on the Divergent Component of Motion (DCM), enabling lateral and forward walking capabilities for the robot; and (3) gait stability analysis through prototype experiments. The primary focus of this study is to explore the application of underactuated symmetrical designs and determine the number of motors required to achieve omnidirectional movement of a bipedal robot. Our simulation and experimental results demonstrate that L03 achieves simple walking with a stable and consistent gait. Due to its lightweight construction, low leg inertia, and straight-legged design, L03 can achieve ground perception and gentle ground contact without the need for force sensors. Compared to existing bipedal robots, L03 closely adheres to the characteristics of the linear inverted pendulum model, making it an invaluable platform for future algorithm research.

Design, Modeling, and Control of a New Multi-Motion Mobile Robot Based on Spoked Mecanum Wheels.

This paper presents an exciting and meaningful design to make mobile robots capable of adapting to various terrains. We designed a relatively simple and novel composite motion mechanism called the flexible spoked mecanum (FSM) wheel and created a mobile robot, LZ-1, with multiple motion modes based on the FSM wheel. Based on the motion analysis of the FSM wheel, we designed an omnidirectional motion mode for this robot, allowing it to move flexibly in all directions and successfully traverse rugged terrains. In addition, we designed a crawl motion mode for this robot, which can climb stairs effectively. We used a multilayer control method to move the robot according to the designed motion modes. Multiple experiments showed that these two motion modes for the robot are effective on various terrains.

Improving imbalance classification via ensemble learning based on two-stage learning.

The excellent performance of deep neural networks on image classification tasks depends on a large-scale high-quality dataset. However, the datasets collected from the real world are typically biased in their distribution, which will lead to a sharp decline in model performance, mainly because an imbalanced distribution results in the prior shift and covariate shift. Recent studies have typically used a two-stage learning method consisting of two rebalancing strategies to solve these problems, but the combination of partial rebalancing strategies will damage the representational ability of the networks. In addition, the two-stage learning method is of little help in addressing the problem of covariate shift. To solve the above two issues, we first propose a sample logit-aware reweighting method called (SLA), which can not only repair the weights of majority class hard samples and minority class samples but will also integrate with logit adjustment to form a stable two-stage learning strategy. Second, to solve the covariate shift problem, inspired by ensemble learning, we propose a multi-domain expert specialization model, which can achieve a more comprehensive decision by averaging expert classification results from multiple different domains. Finally, we combine SLA and logit adjustment into a two-stage learning method and apply our model to the CIFAR-LT and ImageNet-LT datasets. Compared with the most advanced methods, our experimental results show excellent performance.

Interactive Natural Language Grounding via Referring Expression Comprehension and Scene Graph Parsing.

Natural language provides an intuitive and effective interaction interface between human beings and robots. Currently, multiple approaches are presented to address natural language visual grounding for human-robot interaction. However, most of the existing approaches handle the ambiguity of natural language queries and achieve target objects grounding via dialogue systems, which make the interactions cumbersome and time-consuming. In contrast, we address interactive natural language grounding without auxiliary information. Specifically, we first propose a referring expression comprehension network to ground natural referring expressions. The referring expression comprehension network excavates the visual semantics via a visual semantic-aware network, and exploits the rich linguistic contexts in expressions by a language attention network. Furthermore, we combine the referring expression comprehension network with scene graph parsing to achieve unrestricted and complicated natural language grounding. Finally, we validate the performance of the referring expression comprehension network on three public datasets, and we also evaluate the effectiveness of the interactive natural language grounding architecture by conducting extensive natural language query groundings in different household scenarios.

Intention-Related Natural Language Grounding via Object Affordance Detection and Intention Semantic Extraction.

Similar to specific natural language instructions, intention-related natural language queries also play an essential role in our daily life communication. Inspired by the psychology term "affordance" and its applications in Human-Robot interaction, we propose an object affordance-based natural language visual grounding architecture to ground intention-related natural language queries. Formally, we first present an attention-based multi-visual features fusion network to detect object affordances from RGB images. While fusing deep visual features extracted from a pre-trained CNN model with deep texture features encoded by a deep texture encoding network, the presented object affordance detection network takes into account the interaction of the multi-visual features, and reserves the complementary nature of the different features by integrating attention weights learned from sparse representations of the multi-visual features. We train and validate the attention-based object affordance recognition network on a self-built dataset in which a large number of images originate from MSCOCO and ImageNet. Moreover, we introduce an intention semantic extraction module to extract intention semantics from intention-related natural language queries. Finally, we ground intention-related natural language queries by integrating the detected object affordances with the extracted intention semantics. We conduct extensive experiments to validate the performance of the object affordance detection network and the intention-related natural language queries grounding architecture.

Bifurcation and chaos in the simple passive dynamic walking model with upper body.

We present some rich new complex gaits in the simple walking model with upper body by Wisse et al. in [Robotica 22, 681 (2004)]. We first show that the stable gait found by Wisse et al. may become chaotic via period-doubling bifurcations. Such period-doubling routes to chaos exist for all parameters, such as foot mass, upper body mass, body length, hip spring stiffness, and slope angle. Then, we report three new gaits with period 3, 4, and 6; for each gait, there is also a period-doubling route to chaos. Finally, we show a practical method for finding a topological horseshoe in 3D Poincaré map, and present a rigorous verification of chaos from these gaits.

New bifurcations in the simplest passive walking model.

This paper uncovers several new stable periodic gaits in the simplest passive walking bipedal model proposed in the literature. It is demonstrated that the model has period-3 to period-7 gaits beside the period-1 gaits found by Garcia et al. By simulations, this paper shows that each of these new gaits leads to chaos via period-doubling bifurcation and loses its stability by cyclic-fold bifurcation. This interesting phenomenon suggests a series of new bifurcation scenarios that have not been observed before. To confirm the new gaits and their bifurcations, this paper presents computer assisted proofs on the existence and stability of each periodic gait and its period-doubling gaits with the interval Newton method. To verify that the routes indeed lead to chaos, computer-assisted proofs are also given by means of topological horseshoes theory.

Chaotic dynamics in a class of three dimensional glass networks.

In this paper, we study chaotic dynamics of a class of three-dimensional Glass networks with different decay constants, illustrate how the horseshoe is generated, and present a rigorous computer-assisted verification of chaoticity by virtue of interval analysis and topological horseshoe theory.