Biped Robots Control in Gusty Environments with Adaptive Exploration Based DDPG.

As technology rapidly evolves, the application of bipedal robots in various environments has widely expanded. These robots, compared to their wheeled counterparts, exhibit a greater degree of freedom and a higher complexity in control, making the challenge of maintaining balance and stability under changing wind speeds particularly intricate. Overcoming this challenge is critical as it enables bipedal robots to sustain more stable gaits during outdoor tasks, thereby increasing safety and enhancing operational efficiency in outdoor settings. To transcend the constraints of existing methodologies, this research introduces an adaptive bio-inspired exploration framework for bipedal robots facing wind disturbances, which is based on the Deep Deterministic Policy Gradient (DDPG) approach. This framework allows the robots to perceive their bodily states through wind force inputs and adaptively modify their exploration coefficients. Additionally, to address the convergence challenges posed by sparse rewards, this study incorporates Hindsight Experience Replay (HER) and a reward-reshaping strategy to provide safer and more effective training guidance for the agents. Simulation outcomes reveal that robots utilizing this advanced method can more swiftly explore behaviors that contribute to stability in complex conditions, and demonstrate improvements in training speed and walking distance over traditional DDPG algorithms.

Design and sequential jumping experimental validation of a musculoskeletal bipedal robot based on the spring-loaded inverted pendulum model.

To effectively control a robot's motion, it is common to employ a simplified model that approximates the robot's dynamics. Nevertheless, discrepancies between the actual mechanical properties of the robot and the simplified model can result in motion failures. To address this issue, this study introduces a pneumatic-driven bipedal musculoskeletal robot designed to closely match the mechanical characteristics of a simplified spring-loaded inverted pendulum (SLIP) model. The SLIP model is widely utilized in robotics due to its passive stability and dynamic properties resembling human walking patterns. A musculoskeletal bipedal robot was designed and manufactured to concentrate its center of mass within a compact body around the hip joint, featuring low leg inertia in accordance with SLIP model principles. Furthermore, we validated that the robot exhibits similar dynamic characteristics to the SLIP model through a sequential jumping experiment and by comparing its performance to SLIP model simulation.

Parametric Design and Prototyping of a Low-Power Planar Biped Robot.

This study proposes a design approach and the development of a low-power planar biped robot named YU-Bibot. The kinematic structure of the robot consists of six independently driven axes, and it weighs approximately 20 kg. Based on biomimetics, the robot dimensions were selected as the average anthropomorphic dimensions of the human lower extremities. The optimization of the mechanical design and actuator selection of the robot was based on the results of parametric simulations. The natural human walking gait was mimicked as a walking pattern in these simulations. As a result of the optimization, a low power-to-weight ratio of 30 W/kg was obtained. The drive system of the robot joints consists of servo-controlled brushless DC motors with reduction gears and additional bevel gears at the knee and ankle joints. The robot features spring-supported knee and ankle joints that counteract the robot's weight and compensate for the backlash present in these joints. The robot is constrained to move only in the sagittal plane by using a lateral support structure. The robot's feet are equipped with low-cost, force-sensitive resistor (FSR)-type sensors for monitoring ground contact and zero-moment point (ZMP) criterion. The experimental results indicate that the proposed robot mechanism can follow the posture commands accurately and demonstrate locomotion at moderate stability. The proposed parametric natural gait simulation-based design approach and the resulting biped robot design with a low power/weight ratio are the main contributions of this study.

A parallel heterogeneous policy deep reinforcement learning algorithm for bipedal walking motion design.

Considering the dynamics and non-linear characteristics of biped robots, gait optimization is an extremely challenging task. To tackle this issue, a parallel heterogeneous policy Deep Reinforcement Learning (DRL) algorithm for gait optimization is proposed. Firstly, the Deep Deterministic Policy Gradient (DDPG) algorithm is used as the main architecture to run multiple biped robots in parallel to interact with the environment. And the network is shared to improve the training efficiency. Furthermore, heterogeneous experience replay is employed instead of the traditional experience replay mechanism to optimize the utilization of experience. Secondly, according to the walking characteristics of biped robots, a biped robot periodic gait is designed with reference to sinusoidal curves. The periodic gait takes into account the effects of foot lift height, walking period, foot lift speed and ground contact force of the biped robot. Finally, different environments and different biped robot models pose challenges for different optimization algorithms. Thus, a unified gait optimization framework for biped robots based on the RoboCup3D platform is established. Comparative experiments were conducted using the unified gait optimization framework, and the experimental results show that the method outlined in this paper can make the biped robot walk faster and more stably.

Central Pattern Generator with Defined Pulse Signals for Compliant-Resistant Control of Biped Robots.

For biped robots, the ability to maintain balance under external forces is an essential requirement. Inspired by human beings' behaviors to resist external forces, a compliant-resistant balance-control method is proposed to keep the biped robot balance subjected to an external force. A model-free trajectory generator is designed based on the central pattern generator (CPG) to generate compliant-resistant human-like behavior. The CPG pattern generator generates the desired pulse signal utilizing Matsuoka's CPG. The signal modulator applies the defined signal to the robot's center of mass (CoM) to generate the workspace trajectory when standing on double feet. Moreover, when standing on single foot, the output signal of the CPG will directly act on the hip joint of the robot to generate the joint space trajectory. Furthermore, the motion engine calculates the workspace trajectory into joint sequence values. The proposed control strategy can generate defined pulse signals to realize compliant-resistant balance control for biped robots. The control strategy proposed in this paper is verified in the NAO simulation environment.

Hybrid Momentum Compensation Control by Using Arms for Bipedal Dynamic Walking.

Biped robots swing their legs alternately to achieve highly dynamic walking, which is the basic ability required for them to perform tasks. However, swinging of the swinging leg in the air will disturb the interaction between the supporting leg and the ground and affect the upper body's balance during dynamic walking. To allow the robot to use its own intrinsic motion characteristics to maintain stable movement like a human when its lower limbs are affected by unknown disturbances during dynamic walking, the ability to use its arms to resist disturbances is essential. This article presents a hybrid momentum compensation control method for torque-controlled biped robots to adapt to unknown disturbances during dynamic walking. First, a hybrid angular momentum and linear momentum regulator is designed to compensate for the disturbance caused by the swinging leg. Second, based on real-time dynamic state changes of the legs, a mixed-momentum quadratic programming controller is designed to realize stable dynamic walking. The proposed method allows the force-controlled robot to maintain its balance while walking down an unknown platform, and it maintains good straightness in the forward direction of dynamic motion. The proposed method's effectiveness is verified experimentally on the BHR-B2 force-controlled biped robot platform.

Online whole-stage gait planning method for biped robots based on improved Variable Spring-Loaded Inverted Pendulum with Finite-sized Foot (VSLIP-FF) model.

Environmental adaptability and real-time control are significant to the actual application of biped robots. The current Spring-Loaded Inverted Pendulum (SLIP) walking exhibits the compliant interaction with environments. However, the movability and controllability of this model is limited owing to the lack of ankles. Moreover, complicated nonlinear optimization problems in gait generation bring difficulties to real-time control. To overcome these problems, this study proposes an online whole-stage gait planning method to enhance the bipedal walking performance. Firstly, considering the role of ankles, this study applies the proposed template model called Variable Spring-Loaded Inverted Pendulum with Finite-sized Foot (VSLIP-FF) model. Then a Finite State Machine (FSM)-based gait pattern including the corresponding bio-inspired gait strategies is established, which extends the single cyclic gait to the whole-stage gait. Secondly, to realize real-time gait planning, an online gait generator based on a neural network is applied to reduce the calculational burden. Finally, the method is applied on the simulation prototype and real robot platform for verification. Experimental results validate that the proposed method can achieve an autonomous gait with the online planning time of 0.01s, and the step length range is expanded by 37.52% compared with the traditional SLIP model.

Balanced Standing on One Foot of Biped Robot Based on Three-Particle Model Predictive Control.

Balancing is a fundamental task in the motion control of bipedal robots. Compared to two-foot balancing, one-foot balancing introduces new challenges, such as a smaller supporting polygon and control difficulty coming from the kinematic coupling between the center of mass (CoM) and the swinging leg. Although nonlinear model predictive control (NMPC) may solve this problem, it is not feasible to implement it on the actual robot because of its large amount of calculation. This paper proposes the three-particle model predictive control (TP-MPC) approach. It combines with the hierarchical whole-body control (WBC) to solve the one-leg balancing problem in real time. The bipedal robot's torso and two legs are modeled as three separate particles without inertia. The TP-MPC generates feasible swing leg trajectories, followed by the WBC to adjust the robot's center of mass. Since the three-particle model is linear, the TP-MPC requires less computational cost, which implies real-time execution on an actual robot. The proposed method is verified in simulation. Simulation results show that our method can resist much larger external disturbance than the WBC-only control scheme.

Design and simulation analysis of a bionic ostrich robot.

To look for the reason why the biped animal in nature can run with such high speed and to design a bionic biped prototype which can behave the high speed running and jumping ability, this paper takes the fastest bipedal animal in nature: ostrich as the research subject. Firstly, the body structure and motion characteristics of ostrich are investigated. Secondly, a simple mechanical structure of bionic ostrich robot is designed based on the above biological investigated results. The robot is under-actuated with one actuator each leg, with a spring on the tarsometatarsus and a torsion spring on the metatarsophalangeal joint at the foot end. And then the mechanical design of leg structure is optimized. Finally, the high-speed running and jumping running gait is planned, and comparative simulations are implemented with different design requirements among pure rigid and rigid-flexible coupling scheme, which are rigid, only with spring, only with torsion spring, and with spring and torsion spring both, in detail. Simulation results show that the rigid-flexible coupling design scheme and whole body motion coordination can achieve better high speed performance. It provides an insight for the design and control of legged robots.

Kid-size robot humanoid walking with heel-contact and toe-off motion.

Human-like features, like toe-off, heel-strike can enhance the performance of bipedal robots. However, few studies have considered the anthropomorphism of walking planning. Fewer studies have achieved their toe-off, heel-strike gait planning framework in a child-sized humanoid robot platform. This paper presents a human-like walking control framework based on the Divergent Component of Motion (DCM) com planning method that enables a child-sized humanoid robot to walk with a humanoid pattern with a speed of 0.6 s per step a strike of 30 cm. The control framework consists of three parts: the human-like gait generation of the center of mass (CoM) and swings foot trajectory, the dynamic replan in phase switch and the upper body stabilization controller. The dynamic replanning of the CoM and foot trajectory can efficiently decrease the vibration in the step-phase switch. The up-body stabilization controller can reduce the up-body swing in walking and increase the robot's stability while walking. The robot uses a mems-based inertial measurement unit (IMU) and joint position encoders to estimate the current state of the robot and use force-sensitive resistors (FSR) on the robot foot to identify the actual step phase of the robot. None of these solutions is high-cost or difficult to integrate with a child-size robot. Software simulations and walking experiments are using to verify the motion control algorithm. The effectiveness of the pattern generation and the controller can realize more human-like walking styles in a child-size robot are confirmed.

Modeling and Control of a Wheeled Biped Robot.

It is difficult to realize the stable control of a wheeled biped robot (WBR), as it is an underactuated nonlinear system. To improve the balance and dynamic locomotion capabilities of a WBR, a decoupled control framework is proposed. First, the WBR is decoupled into a variable-length wheeled inverted pendulum and a five-link multi-rigid body system. Then, for the above two simplified models, a time-varying linear quadratic regulator and a model predictive controller are designed, respectively. In addition, in order to improve the accuracy of the feedback information of the robot, the Kalman filter is used to optimally estimate the system state. The control framework can enable the WBR to realize changing height, resisting external disturbances, velocity tracking and jumping. The results obtained by simulations and physical experiments verify the effectiveness of the framework.

Torque Curve Optimization of Ankle Push-Off in Walking Bipedal Robots Using Genetic Algorithm.

Ankle push-off occurs when muscle-tendon units about the ankle joint generate a burst of positive power at the end of stance phase in human walking. Ankle push-off mainly contributes to both leg swing and center of mass (CoM) acceleration. Humans use the amount of ankle push-off to induce speed changes. Thus, this study focuses on determining the faster walking speed and the lowest energy efficiency of biped robots by using ankle push-off. The real-time-space trajectory method is used to provide reference positions for the hip and knee joints. The torque curve during ankle push-off, composed of three quintic polynomial curves, is applied to the ankle joint. With the walking distance and the mechanical cost of transport (MCOT) as the optimization goals, the genetic algorithm (GA) is used to obtain the optimal torque curve during ankle push-off. The results show that the biped robot achieved a maximum speed of 1.3 m/s, and the ankle push-off occurs at 41.27-48.34% of the gait cycle. The MCOT of the bipedal robot corresponding to the high economy gait is 0.70, and the walking speed is 0.54 m/s. This study may further prompt the design of the ankle joint and identify the important implications of ankle push-off for biped robots.

Stability Control of a Biped Robot on a Dynamic Platform Based on Hybrid Reinforcement Learning.

In this work, we introduced a novel hybrid reinforcement learning scheme to balance a biped robot (NAO) on an oscillating platform, where the rotation of the platform is considered as the external disturbance to the robot. The platform had two degrees of freedom in rotation, pitch and roll. The state space comprised the position of center of pressure, and joint angles and joint velocities of two legs. The action space consisted of the joint angles of ankles, knees, and hips. By adding the inverse kinematics techniques, the dimension of action space was significantly reduced. Then, a model-based system estimator was employed during the offline training procedure to estimate the dynamics model of the system by using novel hierarchical Gaussian processes, and to provide initial control inputs, after which the reduced action space of each joint was obtained by minimizing the cost of reaching the desired stable state. Finally, a model-free optimizer based on DQN (λ) was introduced to fine tune the initial control inputs, where the optimal control inputs were obtained for each joint at any state. The proposed reinforcement learning not only successfully avoided the distribution mismatch problem, but also improved the sample efficiency. Simulation results showed that the proposed hybrid reinforcement learning mechanism enabled the NAO robot to balance on an oscillating platform with different frequencies and magnitudes. Both control performance and robustness were guaranteed during the experiments.

Analysis and control of biped robot with variable stiffness ankle joints.

BACKGROUND: Biped robot locomotion is an active topic of research, and the walking stability is one of the research objectives. OBJECTIVE: This paper discusses the variable stiffness ankle joint and the walking control of a biped robot. METHODS: A design is introduced that achieves the ankle joint variable stiffness by using a pneumatic unit. The actuation system of the biped robot is based on the hybrid use of electric and pneumatic. The locomotion control architecture has been proposed to exploit natural leg dynamics in order to improve the biped robot walking stability. We also present a dynamic simulation which matches the biped robot and experiments with the real biped robot. RESULTS: The simulation and experiments result that introducing the variable stiffness ankle joint and the controller achieve a significant improvement in foot-ground impact and walking stability of the biped robot. CONCLUSION: The biped robot with variable stiffness ankle joints has a better walking performance under the control method.

Simulation Analysis of Impulsive Ankle Push-Off on the Walking Speed of a Planar Biped Robot.

Ankle push-off generates more than 80% positive power at the end of the stance phase during human walking. In this paper, the influence of impulsive ankle push-off on the walking speed of a biped robot is studied by simulation. When the push-off height of the ankle joint is 13 cm based on the ground (the height of the ankle joint of the swing leg) and the ankle push-off torque increases from 17 to 20.8 N·m, the duration of the swinging leg actually decreases from 50 to 30% of the gait cycle, the fluctuation amplitude of the COM (center of mass) instantaneous speed of the robot decreases from 95 to 35% of the maximum speed, and the walking speed increases from 0.51 to 1.14 m/s. The results demonstrate that impulsive ankle push-off can effectively increase the walking speed of the planar biped robot by accelerating the swing leg and reducing the fluctuation of the COM instantaneous speed. Finally, a comparison of the joint kinematics of the simulation robot and the human at a normal walking speed shows similar motion patterns.

A Variable Stiffness Actuator Module With Favorable Mass Distribution for a Bio-inspired Biped Robot.

Achieving human-like locomotion with humanoid platforms often requires the use of variable stiffness actuators (VSAs) in multi-degree-of-freedom robotic joints. VSAs possess 2 motors for the control of both stiffness and equilibrium position. Hence, they add mass and mechanical complexity to the design of humanoids. Mass distribution of the legs is an important design parameter, because it can have detrimental effects on the cost of transport. This work presents a novel VSA module, designed to be implemented in a bio-inspired humanoid robot, Binocchio, that houses all components on the same side of the actuated joint. This feature allowed to place the actuator's mass to more proximal locations with respect to the actuated joint instead of concentrating it at the joint level, creating a more favorable mass distribution in the humanoid. Besides, it also facilitated it's usage in joints with centralized multi-degree of freedom (DoF) joints instead of cascading single DoF modules. The design of the VSA module is presented, including it's integration in the multi-DoFs joints of Binocchio. Experiments validated the static characteristics of the VSA module to accurately estimate the output torque and stiffness. The dynamic responses of the driving and stiffening mechanisms are shown. Finally, experiments show the ability of the actuation system to replicate the envisioned human-like kinematic, torque and stiffness profiles for Binocchio.

Energy efficient walking control for biped robots using interval type-2 fuzzy logic systems and optimized iteration algorithm.

An energy efficient approach is proposed for the walking control of bipedal robots. To compensate the ZMP error caused by model uncertainties and external disturbances, we design a new walking controller in this paper. Different from currently available control schemes for cancelling ZMP error, our newly proposed one additionally incorporates a fuzzy logic systems(FLSs) mechanism and an iterative mechanism. By employing FLSs to deduce Center of Mass(CoM) correction according to ZMP error and designing iterative mechanism to compute the optimal joint position, the newly proposed controller exhibits an excellent performance. To tackle the control difficulties arising from physical constraints of actuators and hard-to-stabilization of biped robot, an optimized control algorithm is included in the iterative mechanism to guarantee the convergence to the optimal solution. Moreover, the interval type-2 FLSs are adopted to handle the uncertainties. Finally, the experiment results are provided to validate the proposed control scheme.

Design and control of a pneumatic musculoskeletal biped robot.

BACKGROUND: Pneumatic artificial muscles are quite promising actuators for humanoid robots owing to their similar characteristics with human muscles. Moreover, biologically inspired musculoskeletal systems are particularly important for humanoid robots to perform versatile dynamic tasks. OBJECTIVE: This study aims to develop a pneumatic musculoskeletal biped robot, and its controller, to realize human-like walking. METHODS: According to the simplified musculoskeletal structure of human lower limbs, each leg of the biped robot is driven by nine muscles, including three pairs of monoarticular muscles which are arranged in the flexor-extensor form, as well as three biarticular muscles which span two joints. To lower cost, high-speed on/off solenoid valves rather than proportional valves are used to control the muscles. The joint trajectory tracking controller based on PID control method is designed to achieve the desired motion. Considering the complex characteristics of pneumatic artificial muscles, the control model is obtained through parameter identification experiments. RESULTS: Preliminary experimental results demonstrate that the biped robot is able to walk with this control strategy. CONCLUSION: The proposed musculoskeletal structure and control strategy are effective for the biped robot to achieve human-like walking.

Robust disturbance rejection control of a biped robotic system using high-order extended state observer.

This study addressed the problem of robust control of a biped robot based on disturbance estimation. Active disturbance rejection control was the paradigm used for controlling the biped robot by direct active estimation. A robust controller was developed to implement disturbance cancelation based on a linear extended state observer of high gain class. A robust high-gain scheme was proposed for developing a state estimator of the biped robot despite poor knowledge of the plant and the presence of uncertainties. The estimated states provided by the state estimator were used to implement a feedback controller that was effective in actively rejecting the perturbations as well as forcing the trajectory tracking error to within a small vicinity of the origin. The theoretical convergence of the tracking error was proven using the Lyapunov theory. The controller was implemented by numerical simulations that showed the convergence of the tracking error. A comparison with a high-order sliding-mode-observer-based controller confirmed the superior performance of the controller using the robust observer introduced in this study. Finally, the proposed controller was implemented on an actual biped robot using an embedded hardware-in-the-loop strategy.

Design of a biped robot actuated by pneumatic artificial muscles.

High compliant legs are essential for the efficient versatile locomotion and shock absorbency of humans. This study proposes a biped robot actuated by pneumatic artificial muscles to mimic human locomotion. On the basis of the musculoskeletal architecture of human lower limbs, each leg of the biped robot is modeled as a system of three segments, namely, hip joint, knee joint, and ankle joint, and eleven muscles, including both monoarticular and biarticular muscles. Each rotational joint is driven by a pair of antagonistic muscles, enabling joint compliance to be tuned by operating the pressure inside the muscles. Biarticular muscles play an important role in transferring power between joints. Walking simulations verify that biarticular muscles contribute to joint compliance and can absorb impact energy when the robot makes an impact upon ground contact.