High-Accuracy 3D Gaze Estimation with Efficient Recalibration for Head-Mounted Gaze Tracking Systems.

The problem of 3D gaze estimation can be viewed as inferring the visual axes from eye images. It remains a challenge especially for the head-mounted gaze tracker (HMGT) with a simple camera setup due to the complexity of the human visual system. Although the mainstream regression-based methods could establish the mapping relationship between eye image features and the gaze point to calculate the visual axes, it may lead to inadequate fitting performance and appreciable extrapolation errors. Moreover, regression-based methods suffer from a degraded user experience because of the increased burden in recalibration procedures when slippage occurs between HMGT and head. To address these issues, a high-accuracy 3D gaze estimation method along with an efficient recalibration approach is proposed with head pose tracking in this paper. The two key parameters, eyeball center and camera optical center, are estimated in head frame with geometry-based method, so that a mapping relationship between two direction features is proposed to calculate the direction of the visual axis. As the direction features are formulated with the accurately estimated parameters, the complexity of mapping relationship could be reduced and a better fitting performance can be achieved. To prevent the noticeable extrapolation errors, direction features with uniform angular intervals for fitting the mapping are retrieved over human's field of view. Additionally, an efficient single-point recalibration method is proposed with an updated eyeball coordinate system, which reduces the burden of calibration procedures significantly. Our experiment results show that the calibration and recalibration methods could improve the gaze estimation accuracy by 35 percent (from a mean error of 2.00 degrees to 1.31 degrees) and 30 percent (from a mean error of 2.00 degrees to 1.41 degrees), respectively, compared with the state-of-the-art methods.

Anthropomorphic motion planning for multi-degree-of-freedom arms.

With the development of technology, the humanoid robot is no longer a concept, but a practical partner with the potential to assist people in industry, healthcare and other daily scenarios. The basis for the success of humanoid robots is not only their appearance, but more importantly their anthropomorphic behaviors, which is crucial for the human-robot interaction. Conventionally, robots are designed to follow meticulously calculated and planned trajectories, which typically rely on predefined algorithms and models, resulting in the inadaptability to unknown environments. Especially when faced with the increasing demand for personalized and customized services, predefined motion planning cannot be adapted in time to adapt to personal behavior. To solve this problem, anthropomorphic motion planning has become the focus of recent research with advances in biomechanics, neurophysiology, and exercise physiology which deepened the understanding of the body for generating and controlling movement. However, there is still no consensus on the criteria by which anthropomorphic motion is accurately generated and how to generate anthropomorphic motion. Although there are articles that provide an overview of anthropomorphic motion planning such as sampling-based, optimization-based, mimicry-based, and other methods, these methods differ only in the nature of the planning algorithms and have not yet been systematically discussed in terms of the basis for extracting upper limb motion characteristics. To better address the problem of anthropomorphic motion planning, the key milestones and most recent literature have been collated and summarized, and three crucial topics are proposed to achieve anthropomorphic motion, which are motion redundancy, motion variation, and motion coordination. The three characteristics are interrelated and interdependent, posing the challenge for anthropomorphic motion planning system. To provide some insights for the research on anthropomorphic motion planning, and improve the anthropomorphic motion ability, this article proposes a new taxonomy based on physiology, and a more complete system of anthropomorphic motion planning by providing a detailed overview of the existing methods and their contributions.

Design and evaluation of a symmetric amplification mechanism based anthropomorphic shoulder.

Shoulder joints determine the motion range of the upper limb. Thus, the compact and well-stiffened spherical parallel mechanism (SPM) has emerged as the mainstream shoulder prosthesis design approaches. However, the SPM's moving pairs of redundant motions impose excessive constraints that limit its workspace. Therefore, amplifying the workspace of the SPM to cover the motion range required by human daily activities is a pressing problem in shoulder prosthesis design. To address this challenge, this study proposed a workspace amplification approach through the kinematic analysis of a symmetrically arranged 2 degrees of freedom (DoFs) passive mechanism, together with the designed and optimized 3-RRR SPM, to construct an anthropomorphic shoulder. The effectiveness and reliability of the proposed mechanism was verified through thorough analyses. Simulation and experiment results demonstrated that the workspace amplification mechanism could significantly increase the maximum motion match index between the shoulder prosthesis and the daily workspace of the human shoulder from only 26.3% to 94.79%, solving the problem that the traditional SPM-based prostheses cannot satisfy the motion range required by daily activities. Moreover, the proposed mechanism has the potential to amplify the workspace of most parallel mechanisms with multiple DoFs after proper modification.

Non-back-drivable clutch based self-locking mechanism of prosthetic joint to improve manipulation stability.

During activities of daily living (ADLs), the wrist is mainly engaged in positioning and directing the hand. Researches have demonstrated that restoring wrist mobility can significantly enhance the manipulation ability, reduce body distortion caused by motion compensation, and improve the quality of life for amputees. However, most daily activities, particularly the delicate ones, place high demands on the ability of wrist to maintain a certain rotation angle, also known as non-back-drivable ability, which poses a challenge to the design of prosthetic wrists. To address this issue, various solutions have been proposed, including motor holding brakes, high reduction ratio reducers, and worm gears. However, the motor holding brake only functions after a power outage and cannot continuously prevent torque from the load end. The latter two solutions may alter the transmission ratio, resulting in reduced movement speed and transmission efficiency. Therefore, how to design a miniaturized non-back-drivable mechanism without changing the transmission ratio so that the forearm rotational freedom can be locked at any position for any duration is a problem to be solved in the research of prosthetic wrist designs. This paper presents a line-contact based non-back-drivable clutch (NBDC) that does not cause changes in the transmission ratio, ensuring the motion performance of the prosthetic limb. At the same time, it does not introduce additional friction in the forward transmission process, guaranteeing the overall efficiency. Most importantly, it only allows the torque transmitting from the motor to the load, prevents the load reversely from driving back even in a power failure condition, significantly improving the stability, safety, and comfort. Detailed kinematic and static analyses of the working process has been conducted, and transient dynamics simulation has been performed to verify its effectiveness. Through experiments, it is demonstrated that the self-locking torque of the output end could reach approximately 600 Nmm, and the unlocking torque of the input end is about 80 Nmm, which can be effectively integrated in prosthetic wrist rotation joints, contributing to the performance, safety and energy saving of prosthetic joint systems.

Torque modulation mechanism of the knee joint during balance recovery.

Exploring the torque modulation mechanisms of human joints is critical for analyzing the human balance control system and developing natural human-machine interactions for balance support. However, the knee joint is often overlooked in biomechanical models because of its limited range of motion during balance recovery. This poses a challenge in establishing mathematical models for the knee joint's torque modulation mechanisms using computer simulations based on the inverted pendulum model. This study aims to provide a simplified linear feedback model inspired by sensorimotor transformation theory to reveal the torque modulation mechanism of the knee joint. The model was validated using data from experiments involving support-surface translation perturbations. The goodness-of-fit metrics of the model, including R2 values and root mean square errors (RMSE), demonstrated strong explanatory power (R2 ranged from 0.77 to 0.90) and low error (RMSE ranging from 0.035 to 0.072) across different perturbation magnitudes and directions. Through pooling samples across various perturbation conditions and conducting multiple fits, this model revealed that knee torque is modulated using a direction-specific strategy with adaptable feedback gains. These results suggest that the proposed simplified linear model can be used to develop assistive systems and retrieve insights on balance recovery mechanisms.

Influences of dynamic load phase shifts on the energetics and biomechanics of humans.

Using load-suspended backpacks to reduce vertical peak dynamic load exerted on humans can reduce metabolic costs. However, is it possible to further reduce metabolic cost by modulating dynamic load phase shift? If so, is anti-phase better than the others? In this study, we investigated the biomechanics, energetics and trunk response under phase shifts. Nine subjects wearing an active backpack with 19.4 kg loads walked on a treadmill at 5 km h-1 with four phase shift trials (T1-T4) and a load-locked trial (LK). Our results show that anti-phase trial (T3) assists ankle more and reduces the moment and gastrocnemius medialis activity, while T4 assists knee more and reduces the moment and rectus femoris activity. Due to the load injecting more mechanical energy into human in T3 and T4, the positive centre-of-mass work is significantly reduced. However, the gross metabolic rate is lowest in T4 and 4.43% lower than in T2, which may be because the activations of erector spinae and gluteus maximus are reduced in T4. In addition, T3 increases trunk extensor effort, which may weaken the metabolic advantage. This study provides guidance for improving assistance strategies and human-load interfaces and deepens the understanding of the energetics and biomechanics of human loaded walking.

A review of the design of load-carrying exoskeletons.

The increasing necessity of load-carrying activities has led to greater human musculoskeletal damage and an increased metabolic cost. With the rise of exoskeleton technology, researchers have begun exploring different approaches to developing wearable robots to augment human load-carrying ability. However, there is a lack of systematic discussion on biomechanics, mechanical designs, and augmentation performance. To achieve this, extensive studies have been reviewed and 108 references are selected mainly from 2013 to 2022 to address the most recent development. Other earlier 20 studies are selected to present the origin of different design principles. In terms of the way to achieve load-carrying augmentation, the exoskeletons reviewed in this paper are sorted by four categories based on the design principles, namely load-suspended backpacks, lower-limb exoskeletons providing joint torques, exoskeletons transferring load to the ground and exoskeletons transferring load between body segments. Specifically, the driving modes of active and passive, the structure of rigid and flexible, the conflict between assistive performance and the mass penalty of the exoskeleton, and the autonomy are discussed in detail in each section to illustrate the advances, challenges, and future trends of exoskeletons designed to carry loads.

Modulating Energy Among Foot-Ankle Complex With an Unpowered Exoskeleton Improves Human Walking Economy.

Over the course of both evolution and development, the human musculoskeletal system has been well shaped for the cushion function of the foot during foot-strike and the impulsive function of the ankle joint during push-off. Nevertheless, an efficient energy interaction between foot structure and ankle joint is still lacking in the human body itself, which may limit the further potential of economical walking. Here we showed the metabolic expenditure of walking can be lessened by an unpowered exoskeleton robot that modulates energy among the foot-ankle complex towards a more effective direction. The unpowered exoskeleton recycles negative mechanical energy of the foot that is normally dissipated in heel-strike, retains the stored energy before mid-stance, and then transfers the energy to the ankle joint to assist the push-off. The modulation process of the exoskeleton consumes no input energy, yet reduces the metabolic cost of walking by 8.19 ± 0.96 % (mean ± s.e.m) for healthy subjects. The electromyography measurements demonstrate the activities of target ankle plantarflexors decreased significantly without added effort for the antagonistic muscle, suggesting the exoskeleton enhanced the subjects' energy efficiency of the foot-ankle complex in a natural manner. Furthermore, the exoskeleton also provides cushion assistance for walking, which leads to significantly decreased activity of the quadriceps muscle during heel-strike. Rather than strengthening the functions of existing biological structures, developing the complementary energy loop that does not exist in the human body itself also shows its potential for gait assistance.

Characterization and Categorization of Various Human Lower Limb Movements Based on Kinematic Synergies.

A proper movement categorization reduces the complexity of understanding or reproducing human movements in fields such as physiology, rehabilitation, and robotics, through partitioning a wide variety of human movements into representative sub-motion groups. However, how to establish a categorization (especially a quantitative categorization) for various human lower limb movements is rarely investigated in literature and remains challenging due to the diversity and complexity of the lower limb movements (diverse gait modes and interaction styles with the environment). Here we present a quantitative categorization for the various lower limb movements. To this end, a similarity measure between movements was first built based on limb kinematic synergies that provide a unified and physiologically meaningful framework for evaluating the similarities among different types of movements. Then, a categorization was established via hierarchical cluster analysis for thirty-four lower limb movements, including walking, running, hopping, sitting-down-standing-up, and turning in different environmental conditions. According to the movement similarities, the various movements could be divided into three distinct clusters (cluster 1: walking, running, and sitting-down-standing-up; cluster 2: hopping; cluster 3: turning). In each cluster, cluster-specific movement synergies were required. Besides the uniqueness of each cluster, similarities were also found among part of the synergies employed by these different clusters, perhaps related to common behavioral goals in these clusters. The mix of synergies shared across the clusters and synergies for specific clusters thus suggests the coexistence of the conservation and augmentation of the kinematic synergies underlying the construction of the diverse and complex motor behaviors. Overall, the categorization presented here yields a quantitative and hierarchical representation of the various lower limb movements, which can serve as a basis for the understanding of the formation mechanisms of human locomotion and motor function assessment and reproduction in related fields.

A portable exotendon assisting hip and knee joints reduces muscular burden during walking.

Assistive devices are used to reduce human effort during locomotion with increasing success. More assistance strategies are worth exploring, so we aimed to design a lightweight biarticular device with well-chosen parameters to reduce muscle effort. Based on the experience of previous success, we designed an exotendon to assist in swing leg deceleration. Then we conducted experiments to test the performance of the exotendon with different spring stiffness during walking. With the assistance of the exotendon, peak activation of semitendinosus decreased, with the largest reduction of 12.3% achieved with the highest spring stiffness (p = 0.004). The peak activations of other measured muscles were not significantly different (p = 0.15-0.92). The biological hip extension and knee flexion moments likewise significantly decreased with the spring stiffness (p < 0.01). The joint angle was altered during the assisted phases with decreased hip flexion and knee extension. Meanwhile, the step frequency and the step length were also altered, while the step width remained unaffected. Gait variability changed only in the frontal plane, exhibiting lower step width variability. We conclude that passive devices assisting hip extension and knee flexion can significantly reduce the burden on the hamstring muscles, while the kinematics is easily altered.

Common kinematic synergies of various human locomotor behaviours.

Humans show a variety of locomotor behaviours in daily living, varying in locomotor modes and interaction styles with the external environment. However, how this excellent motor ability is formed, whether there are some invariants underlying various locomotor behaviours and simplifying their generation, and what factors contribute to the invariants remain unclear. Here, we find three common kinematic synergies that form the six joint motions of one lower limb during walking, running, hopping and sitting-down-standing-up (movement variance accounted for greater than 90%), through identifying the coordination characteristics of 36 lower limb motor tasks in diverse environments. This finding supports the notion that humans simplify the generation of various motor behaviours through re-using several basic motor modules, rather than developing entirely new modules for each behaviour. Moreover, a potential link is also found between these synergies and the unique biomechanical characteristics of the human musculoskeletal system (muscular-articular connective architecture and bone shape), and the patterns of inter-joint coordination are consistent with the energy-saving mechanism in locomotion by using biarticular muscles as efficient mechanical energy transducers between joints. Altogether, our work helps understand the formation mechanisms of human locomotion from a holistic viewpoint and evokes inspirations for the development of artificial limbs imitating human motor ability.

A science-driven method for determining morphological parameters of prosthetic hands.

The unique morphological bases of human hands, which are distinct from other primates, endow them with excellent grasping and manipulative abilities. However, the lack of understanding of human hand morphology and its parametric features is a major obstacle in the scientific design of prosthetic hands. Existing designs of prosthetic hand morphologies mostly adopt engineering-based methods, which depend on human experience, direct measurements of human hands, or numerical simulation/optimization. This paper explores for the first time a science-driven design method for prosthetic hand morphology, aiming to facilitate the development of prosthetic hands with human-level dexterity. We first use human morphological, movement, and postural data to quantitatively cognize general morphological characteristics of human hands in static, dynamic, functional, and non-functional perspectives. Taking these cognitions as bases, we develop a method able to quickly transfer human morphological parameters to prosthetic hands and endow the prosthetic hands with great grasping/manipulative potential at the same time. We apply this method to the design of an advanced prosthetic hand (called X-hand II) embedded with compact actuating systems. The human-size prosthetic hand can reach wide grasping/manipulative ranges close to those of human hands, replicate various daily grasping types and even execute dexterous in-hand manipulation. This science-driven method may also inspire other artificial limb and bionic robot designs.