Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks.

In the field of quadruped robots, the most classic motion control algorithm is based on model prediction control (MPC). However, this method poses challenges as it necessitates the precise construction of the robot's dynamics model, making it difficult to achieve agile movements similar to those of a biological dog. Due to these limitations, researchers are increasingly turning to model-free learning methods, which significantly reduce the difficulty of modeling and engineering debugging and simultaneously reduce real-time optimization computational burden. Inspired by the growth process of humans and animals, from learning to walk to fluent movements, this article proposes a hierarchical reinforcement learning framework for the motion controller to learn some higher-level tasks. First, some basic motion skills can be learned from motion data captured from a biological dog. Then, with these learned basic motion skills as a foundation, the quadruped robot can focus on learning higher-level tasks without starting from low-level kinematics, which saves redundant training time. By utilizing domain randomization techniques during the training process, the trained policy function can be directly transferred to a physical robot without modification, and the resulting controller can perform more biomimetic movements. By implementing the method proposed in this article, the agility and adaptability of the quadruped robot can be maximally utilized to achieve efficient operations in complex terrains.

Hybrid Bipedal Locomotion Based on Reinforcement Learning and Heuristics.

Locomotion control has long been vital to legged robots. Agile locomotion can be implemented through either model-based controller or reinforcement learning. It is proven that robust controllers can be obtained through model-based methods and learning-based policies have advantages in generalization. This paper proposed a hybrid framework of locomotion controller that combines deep reinforcement learning and simple heuristic policy and assigns them to different activation phases, which provides guidance for adaptive training without producing conflicts between heuristic knowledge and learned policies. The training in simulation follows a step-by-step stochastic curriculum to guarantee success. Domain randomization during training and assistive extra feedback loops on real robot are also adopted to smooth the transition to the real world. Comparison experiments are carried out on both simulated and real Wukong-IV humanoid robots, and the proposed hybrid approach matches the canonical end-to-end approaches with higher rate of success, faster converging speed, and 60% less tracking error in velocity tracking tasks.

Multi-expert learning of adaptive legged locomotion.

Achieving versatile robot locomotion requires motor skills that can adapt to previously unseen situations. We propose a multi-expert learning architecture (MELA) that learns to generate adaptive skills from a group of representative expert skills. During training, MELA is first initialized by a distinct set of pretrained experts, each in a separate deep neural network (DNN). Then, by learning the combination of these DNNs using a gating neural network (GNN), MELA can acquire more specialized experts and transitional skills across various locomotion modes. During runtime, MELA constantly blends multiple DNNs and dynamically synthesizes a new DNN to produce adaptive behaviors in response to changing situations. This approach leverages the advantages of trained expert skills and the fast online synthesis of adaptive policies to generate responsive motor skills during the changing tasks. Using one unified MELA framework, we demonstrated successful multiskill locomotion on a real quadruped robot that performed coherent trotting, steering, and fall recovery autonomously and showed the merit of multi-expert learning generating behaviors that can adapt to unseen scenarios.

Cortico-Hippocampal Computational Modeling Using Quantum-Inspired Neural Networks.

Many current computational models that aim to simulate cortical and hippocampal modules of the brain depend on artificial neural networks. However, such classical or even deep neural networks are very slow, sometimes taking thousands of trials to obtain the final response with a considerable amount of error. The need for a large number of trials at learning and the inaccurate output responses are due to the complexity of the input cue and the biological processes being simulated. This article proposes a computational model for an intact and a lesioned cortico-hippocampal system using quantum-inspired neural networks. This cortico-hippocampal computational quantum-inspired (CHCQI) model simulates cortical and hippocampal modules by using adaptively updated neural networks entangled with quantum circuits. The proposed model is used to simulate various classical conditioning tasks related to biological processes. The output of the simulated tasks yielded the desired responses quickly and efficiently compared with other computational models, including the recently published Green model.

Cortico-Hippocampal Computational Modeling Using Quantum Neural Networks to Simulate Classical Conditioning Paradigms.

Most existing cortico-hippocampal computational models use different artificial neural network topologies. These conventional approaches, which simulate various biological paradigms, can get slow training and inadequate conditioned responses for two reasons: increases in the number of conditioned stimuli and in the complexity of the simulated biological paradigms in different phases. In this paper, a cortico-hippocampal computational quantum (CHCQ) model is proposed for modeling intact and lesioned systems. The CHCQ model is the first computational model that uses the quantum neural networks for simulating the biological paradigms. The model consists of two entangled quantum neural networks: an adaptive single-layer feedforward quantum neural network and an autoencoder quantum neural network. The CHCQ model adaptively updates all the weights of its quantum neural networks using quantum instar, outstar, and Widrow-Hoff learning algorithms. Our model successfully simulated several biological processes and maintained the output-conditioned responses quickly and efficiently. Moreover, the results were consistent with prior biological studies.

Green model to adapt classical conditioning learning in the hippocampus.

Compared with the biological paradigms of classical conditioning, non-adaptive computational models are not capable of realistically simulating the biological behavioural functions of the hippocampal regions, because of their implausible requirement for a large number of learning trials, which can be on the order of hundreds. Additionally, these models did not attain a unified, final stable state even after hundreds of learning trials. Conversely, the output response has a different threshold for similar tasks in various models with prolonged transient response of unspecified status via the training or even testing phases. Accordingly, a green model is a combination of adaptive neuro-computational hippocampal and cortical models that is proposed by adaptively updating the whole weights in all layers for both intact networks and lesion networks using instar and outstar learning rules with adaptive resonance theory (ART). The green model sustains and expands the classical conditioning biological paradigms of the non-adaptive models. The model also overcomes the irregular output response behaviour by using the proposed feature of adaptivity. Further, the model successfully simulates the hippocampal regions without passing the final output response back to the whole network, which is considered to be biologically implausible. The results of the Green model showed a significant improvement confirmed by empirical studies of different tasks. In addition, the results indicated that the model outperforms the previously published models. All the obtained results successfully and quickly attained a stable, desired final state (with a unified concluding state of either "1" or "0") with a significantly shorter transient duration.

[Epidemiological investigation on an outbreak of amoebic dysentery in Jiangshan City of Zhejiang Province].

The investigation was carried out from 15 May to 10 June 2006 among diarrhea patients of two schools and periphery residents in Hecun Town, Jiangshan City. Stool samples were examined for Entamoeba histolytica. Water samples were taken for microbial analysis. 31 cases with E. histolytica were found,with 74.2% (23 cases) of students and preschool children. 9 cases were found in Liuyi kindergarten and 8 cases in Hecun central primary school with a prevalence of 7.4% and 0.65%, respectively. Among 594 asymptomatic close contactors, 9 cases (1.5%) were carriers of cysts. Of the 31 cases, 22 were found with no habit of handwashing before eating or after defecation, and 14 cases had a close contact to the patients. No amoebic cysts or trophozoites were found from 12 water samples collected from schools or patient's houses, but the Escherichia coli level exceeded the national standard in 7 samples.