Experiment-free exoskeleton assistance via learning in simulation.

Exoskeletons have enormous potential to improve human locomotive performance1-3. However, their development and broad dissemination are limited by the requirement for lengthy human tests and handcrafted control laws2. Here we show an experiment-free method to learn a versatile control policy in simulation. Our learning-in-simulation framework leverages dynamics-aware musculoskeletal and exoskeleton models and data-driven reinforcement learning to bridge the gap between simulation and reality without human experiments. The learned controller is deployed on a custom hip exoskeleton that automatically generates assistance across different activities with reduced metabolic rates by 24.3%, 13.1% and 15.4% for walking, running and stair climbing, respectively. Our framework may offer a generalizable and scalable strategy for the rapid development and widespread adoption of a variety of assistive robots for both able-bodied and mobility-impaired individuals.

Delayed center of mass feedback in elderly humans leads to greater muscle co-contraction and altered balance strategy under perturbed balance: A predictive musculoskeletal simulation study.

Falls are one of the leading causes of non-disease death and injury in the elderly, often due to delayed sensory neural feedback essential for balance. This delay, challenging to measure or manipulate in human studies, necessitates exploration through neuromusculoskeletal modeling to reveal its intricate effects on balance. In this study, we developed a novel three-way muscle feedback control approach, including muscle length feedback, muscle force feedback, and enter of mass feedback, for balancing and investigated specifically the effects of center of mass feedback delay on elderly people's balance strategies. We conducted simulations of cyclic perturbed balance at different magnitudes ranging from 0 to 80 mm and with three center of mass feedback delays (100, 150 & 200 ms). The results reveal two key points: 1) Longer center of mass feedback delays resulted in increased muscle activations and co-contraction, 2) Prolonged center of mass feedback delays led to noticeable shifts in balance strategies during perturbed standing. Under low-amplitude perturbations, the ankle strategy was predominantly used, while higher amplitude disturbances saw more frequent employment of hip and knee strategies. Additionally, prolonged center of mass delays altered balance strategies across different phases of perturbation, with a noticeable increase in overall ankle strategy usage. These findings underline the adverse effects of prolonged feedback delays on an individual's stability, necessitating greater muscle co-contraction and balance strategy adjustment to maintain balance under perturbation. Our findings advocate for the development of training programs tailored to enhance balance reactions and mitigate muscle feedback delays within clinical or rehabilitation settings for fall prevention in elderly people.

Robust walking control of a lower limb rehabilitation exoskeleton coupled with a musculoskeletal model via deep reinforcement learning.

BACKGROUND: Few studies have systematically investigated robust controllers for lower limb rehabilitation exoskeletons (LLREs) that can safely and effectively assist users with a variety of neuromuscular disorders to walk with full autonomy. One of the key challenges for developing such a robust controller is to handle different degrees of uncertain human-exoskeleton interaction forces from the patients. Consequently, conventional walking controllers either are patient-condition specific or involve tuning of many control parameters, which could behave unreliably and even fail to maintain balance. METHODS: We present a novel, deep neural network, reinforcement learning-based robust controller for a LLRE based on a decoupled offline human-exoskeleton simulation training with three independent networks, which aims to provide reliable walking assistance against various and uncertain human-exoskeleton interaction forces. The exoskeleton controller is driven by a neural network control policy that acts on a stream of the LLRE's proprioceptive signals, including joint kinematic states, and subsequently predicts real-time position control targets for the actuated joints. To handle uncertain human interaction forces, the control policy is trained intentionally with an integrated human musculoskeletal model and realistic human-exoskeleton interaction forces. Two other neural networks are connected with the control policy network to predict the interaction forces and muscle coordination. To further increase the robustness of the control policy to different human conditions, we employ domain randomization during training that includes not only randomization of exoskeleton dynamics properties but, more importantly, randomization of human muscle strength to simulate the variability of the patient's disability. Through this decoupled deep reinforcement learning framework, the trained controller of LLREs is able to provide reliable walking assistance to patients with different degrees of neuromuscular disorders without any control parameter tuning. RESULTS AND CONCLUSION: A universal, RL-based walking controller is trained and virtually tested on a LLRE system to verify its effectiveness and robustness in assisting users with different disabilities such as passive muscles (quadriplegic), muscle weakness, or hemiplegic conditions without any control parameter tuning. Analysis of the RMSE for joint tracking, CoP-based stability, and gait symmetry shows the effectiveness of the controller. An ablation study also demonstrates the strong robustness of the control policy under large exoskeleton dynamic property ranges and various human-exoskeleton interaction forces. The decoupled network structure allows us to isolate the LLRE control policy network for testing and sim-to-real transfer since it uses only proprioception information of the LLRE (joint sensory state) as the input. Furthermore, the controller is shown to be able to handle different patient conditions without the need for patient-specific control parameter tuning.

Artificial Neural Network-Based Activities Classification, Gait Phase Estimation, and Prediction.

Gait patterns are critical to health monitoring, gait impairment assessment, and wearable device control. Unrhythmic gait pattern detection under community-based conditions is a new frontier in this area. The present paper describes a high-accuracy gait phase estimation and prediction algorithm built on a two-stage artificial neural network. This work targets to develop an algorithm that can estimate and predict the gait cycle in real time using a portable controller with only two IMU sensors (one on each thigh) in the community setting. Our algorithm can detect the gait phase in unrhythmic conditions during walking, stair ascending, and stair descending, and classify these activities with standing. Moreover, our algorithm is able to predict both future intra- and inter-stride gait phases, offering a potential means to improve wearable device controller performance. The proposed data-driven algorithm is based on a dataset consisting of 5 able-bodied subjects and validated on 3 different able-bodied subjects. Under unrhythmic activity situations, validation shows that the algorithm can accurately identify multiple activities with 99.55% accuracy, and estimate ([Formula: see text]: 6.3%) and predict 200-ms-ahead ([Formula: see text]: 8.6%) the gait phase percentage in real time, which are on average 57.7 and 54.0% smaller than the error from the event-based method in the same conditions. This study showcases a solution to estimate and predict gait status for multiple unrhythmic activities, which may be deployed to controllers for wearable robots or health monitoring devices.

A Simple Grading Scale for Predicting Symptomatic Intracranial Hemorrhage after Mechanical Thrombectomy.

INTRODUCTION: Hemorrhagic transformation, especially symptomatic intracranial hemorrhage (sICH), is a common complication after mechanical embolectomy. This study explored a grading scale based on clinical and radiological parameters to predict sICH after mechanical embolectomy. METHODS: Demographic and clinical data were retrospectively collected from patients with acute ischemic stroke treated with mechanical embolectomy at West China Hospital. Clinical and radiological factors associated with sICH were identified and used to develop the "STBA" grading scale. This score was then validated using data from an independent sample at the First Affiliated Hospital of Kunming Medical University. RESULTS: We analyzed 268 patients with acute ischemic stroke who were treated with mechanical embolectomy at West China Hospital, of whom 30 (11.2%) had sICH. Patients were rated on an "STBA" score ranging from 0 to 6 based on whether systolic blood pressure was ≥145 mm Hg at admission (yes = 2 points; no = 0 points), time from acute ischemic stroke until groin puncture was ≥300 min (yes = 1; no = 0), blood glucose was ≥8.8 mmol/L (yes = 1; no = 0), and the Alberta Stroke Program Early Computed Tomography score at admission was 0-5 (2 points), 6-7 (1 point), or 8-10 (0 points). The STBA score showed good discrimination in the derivation sample (area under the receiver operating characteristic curve = 0.858) and in the validation sample (area = 0.814). CONCLUSIONS: The STBA score may be a reliable clinical scoring system to predict sICH in acute ischemic stroke patients treated with mechanical embolectomy.

Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance.

A significant challenge for the control of a robotic lower extremity rehabilitation exoskeleton is to ensure stability and robustness during programmed tasks or motions, which is crucial for the safety of the mobility-impaired user. Due to various levels of the user's disability, the human-exoskeleton interaction forces and external perturbations are unpredictable and could vary substantially and cause conventional motion controllers to behave unreliably or the robot to fall down. In this work, we propose a new, reinforcement learning-based, motion controller for a lower extremity rehabilitation exoskeleton, aiming to perform collaborative squatting exercises with efficiency, stability, and strong robustness. Unlike most existing rehabilitation exoskeletons, our exoskeleton has ankle actuation on both sagittal and front planes and is equipped with multiple foot force sensors to estimate center of pressure (CoP), an important indicator of system balance. This proposed motion controller takes advantage of the CoP information by incorporating it in the state input of the control policy network and adding it to the reward during the learning to maintain a well balanced system state during motions. In addition, we use dynamics randomization and adversary force perturbations including large human interaction forces during the training to further improve control robustness. To evaluate the effectiveness of the learning controller, we conduct numerical experiments with different settings to demonstrate its remarkable ability on controlling the exoskeleton to repetitively perform well balanced and robust squatting motions under strong perturbations and realistic human interaction forces.

Model-Based Comparison of Passive and Active Assistance Designs in an Occupational Upper Limb Exoskeleton for Overhead Lifting.

OCCUPATIONAL APPLICATIONSIn recent years, various upper limb exoskeletons have been developed aiming to support industrial workers for a range of tasks and reduce risks of work-related musculoskeletal disorders. Most commercially available upper limb exoskeletons are passive systems that use compliant elements such as springs or elastic components to store and release energy to assist the user's motion. In contrast, many active exoskeletons, which are typically comprised of one or more powered actuators to provide joint assistance, are still in the research and development stages. Nevertheless, the functions and efficacy of various exoskeleton systems need to be further compared and assessed. This study presents a model-based approach to evaluate different designs of passive and active assistance and demonstrates the benefits of both assistance methods in an overhead lifting task. In addition, the modeling and simulation indicate the potential advantages of using the active assistance, based on electromyography.

TECHNICAL ABSTRACTBackground: In the literature, efficacy of passive upper limb exoskeletons has been demonstrated in reduced activity of involved muscles during overhead occupational tasks. However, there are fewer studies that have investigated the efficacy of active upper limb exoskeletons or compared them with their passive counterparts.Purpose: We aimed to use an approach simulating human-exoskeleton interactions to compare several passive and active assistance methods in an upper limb exoskeleton and to evaluate how different assistance types affect musculoskeletal loadings during overhead lifting.Methods: An upper-extremity musculoskeletal model was integrated with a five degree-of-freedom exoskeleton for virtual human-in-the-loop evaluation of exoskeleton design and control. Different assistance methods were evaluated, including spring-based activation zones and active control based on EMG, to examine their biomechanical effects on musculoskeletal loadings including interaction forces and moments, muscle activations, and joint moments and reaction forces.Results: Our modeling and simulation results suggest the effectiveness of the proposed passive and active assistance methods in reducing biomechanical loadings­the upper-limb exoskeletons could reduce maximum loading on the shoulder joint by up to 46% compared to the no-exoskeleton situation. Active assistance was found to outperform the passive assistance approach. Specifically, EMG-based active assistance could assist over the whole lifting range and had a larger capability to reduce deltoid muscle activation and shoulder joint reaction force.Conclusions: We used a modeling and simulation approach to virtually evaluate various exoskeleton assistance methods without testing multiple physical prototypes and to investigate the effects of these methods on musculoskeletal loadings that cannot be measured directly or noninvasively. Our findings offer new approaches for testing methods and improving exoskeleton designs with "smart" controls. More research is planned to further optimize the exoskeleton control strategies and validate the simulated results in a real-life implementation.

Automated medical avatar animation for warfighter mission simulation.

BACKGROUND: Virtual representations of human internal anatomy are important for military applications such as protective equipment design, injury severity prediction, thermal analysis, and physiological simulations. High-fidelity volumetric models based on imaging data are typically in static postures and difficult to use in simulations of realistic mission scenarios. This study aimed to investigate a hybrid approach to reposition medical avatars that preserves internal anatomy but allows rapid repositioning of full three-dimensional (3D) meshes. METHODS: A software framework that accepts a medical avatar in a 3D tetrahedral mesh format representing 72 organs and tissues with an articulated skeleton was developed. The skeleton is automatically resized and associated to the avatar using rigging and skinning algorithms inspired by computer animation techniques. Military relevant motions were used for animations. A motion retargeting algorithm was implemented to apply animation to avatars of various sizes, and a motion blending algorithm was implemented to smoothly transition between movements. These algorithms were incorporated into a path generation tool that accepts initial, intermediate, and final coordinates of a multisegment action along with the specific motion for each segment to synthesize a realistic compound set of movements comprising the animation. RESULTS: The developed pipeline for dynamic repositioning of medical avatars was demonstrated. Various complex motions were automatically animated. Retargeting was demonstrated on models of varying sizes. Movements along a path were animated to demonstrate smooth motion transitions. Animation of the full 3D avatar mesh ran in real time on a standard desktop personal computer. The repositioning algorithm successfully preserved the shape and volume of rigid structures such as bone. CONCLUSION: The developed software leverages techniques from various disciplines to create a hybrid approach enabling real-time 3D mesh repositioning appropriate for use in simulated military missions using avatars containing a complete anatomy representation. The workflow is largely automated, enabling rapid evaluation of new mission scenarios.

A quasi-3D model of the whole lung: airway extension to the tracheobronchial limit using the constrained constructive optimization and alveolar modeling, using a sac-trumpet model.

Existing computational models used for simulating the flow and species transport in the human airways are zero-dimensional (0D) compartmental, three-dimensional (3D) computational fluid dynamics (CFD), or the recently developed quasi-3D (Q3D) models. Unlike compartmental models, the full CFD and Q3D models are physiologically and anatomically consistent in the mouth and the upper airways, since the starting point of these models is the mouth-lung surface geometry, typically created from computed tomography (CT) scans. However, the current resolution of CT scans limits the airway detection between the 3rd-4th and 7th-9th generations. Consequently, CFD and the Q3D models developed using these scans are generally limited to these generations. In this study, we developed a method to extend the conducting airways from the end of the truncated Q3D lung to the tracheobronchial (TB) limit. We grew the lung generations within the closed lung lobes using the modified constrained constructive optimization, creating an aerodynamically optimized network aiming to produce equal pressure at the distal ends of the terminal segments. This resulted in a TB volume and lateral area of ∼165 cc and ∼2000 cm2, respectively. We created a "sac-trumpet" model at each of the TB outlets to represent the alveoli. The volumes of the airways and the individual alveolar generations match the anatomical values by design: with the functional residual capacity at 2611 cc. Lateral surface areas were scaled to match the physiological values. These generated Q3D whole lung models can be efficiently used for conducting multiple breathing cycles of drug transport and deposition simulations.

Design and Evaluation of Torque Compensation Controllers for a Lower Extremity Exoskeleton.

In this article, we present an integrated human-in-the-loop simulation paradigm for the design and evaluation of a lower extremity exoskeleton that is elastically strapped onto human lower limbs. The exoskeleton has three rotational DOFs on each side and weighs 23 kg. Two torque compensation controllers of the exoskeleton are introduced, aiming to minimize interference and maximize assistance, respectively. Their effects on the wearer's biomechanical loadings are studied with a running motion and predicted ground reaction forces (GRFs). It is found that the added weight of the passive exoskeleton substantially increases the wearer's musculoskeletal loadings. The maximizing assistance controller reduces the knee joint torque by 31% when compared with the normal running (without exoskeleton) and by 50% when compared with the passive exoskeleton case. When compared with the normal running, this controller also reduces the hip flexion and extension torques by 31% and 38%, respectively. As a result, the peak activations of the biceps short head, gluteus maximus, and rectus femoris muscles are reduced by more than a half. Nonetheless, the axial knee joint reaction force increases for all exoskeleton cases due to the added weight and higher ground reaction forces. In summary, the results provide sound evidence of the efficacy of the proposed controllers on reducing the wearer's musculoskeletal loadings. And it is shown that the human-in-the-loop simulation paradigm presented here can be used for virtual design and evaluation of powered exoskeletons and pave the way for building optimized exoskeleton prototypes for experimental evaluation.

Spatial coordinate corrected motion tracking for optical coherence elastography.

We develop a spatial coordinate corrected (SCC) motion tracking method for optical coherence elastography. SCC motion tracking refers the instantaneous velocity field extracted from optical coherence tomography (OCT) data to the laboratory coordinate system and accurately reconstructs the displacement field established during the mechanical excitation (compression) process. We acquired image data from compression OCE experiments on human breast tissue specimens, and reconstructed the displacement field through Doppler analysis of OCT data. Our results suggested that SCC tracking enables accurate reconstruction of displacement field, and enables effective identification mechanical heterogeneity that can be used as a biomarker for cancer diagnosis and tumor margin assessment.

Neck musculoskeletal model generation through anthropometric scaling.

A new methodology was developed to quickly generate whole body models with detailed neck musculoskeletal architecture that are properly scaled in terms of anthropometry and muscle strength. This method was implemented in an anthropometric model generation software that allows users to interactively generate any new male or female musculoskeletal models with adjustment of anthropometric parameters (such as height, weight, neck circumference, and neck length) without the need of subject-specific motion capture or medical images. 50th percentile male and female models were developed based on the 2012 US Army Anthropometric Survey (ANSUR II) database and optimized with a novel bilevel optimization method to have strengths comparable to experimentally measured values in the literature. Other percentile models (ranging from the 1st to 99th percentile) were generated based on anthropometric scaling of the 50th percentile models and compared. The resultant models are reasonably accurate in terms of both musculoskeletal geometry and neck strength, demonstrating the effectiveness of the developed methodology for interactive neck model generation with anthropometric scaling.

Spatial coordinate corrected motion tracking for optical coherence elastography.

We investigate a spatial coordinate correction (SCC) method to track motion with high accuracy for optical coherence elastography (OCE). Through SCC, we refer the displacement field tracked by optical coherence tomography (OCT) in the loaded sample to individual material points defined in a fixed coordinate system. SCC allows OCE to perform spatially and temporally unambiguous tracking of displacement and enables accurate mechanical characterization of biological tissue for cancer diagnosis and tumor margin assessment. In this study, we validated the effectiveness of motion tracking based on SCC using experimental OCE data obtained from ex vivo human breast tissues.

Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm.

Embedding computational models in the physical world is a critical step towards constraining their behavior and building practical applications. Here we aim to drive a realistic musculoskeletal arm model using a biomimetic cortical spiking model, and make a robot arm reproduce the same trajectories in real time. Our cortical model consisted of a 3-layered cortex, composed of several hundred spiking model-neurons, which display physiologically realistic dynamics. We interconnected the cortical model to a two-joint musculoskeletal model of a human arm, with realistic anatomical and biomechanical properties. The virtual arm received muscle excitations from the neuronal model, and fed back proprioceptive information, forming a closed-loop system. The cortical model was trained using spike timing-dependent reinforcement learning to drive the virtual arm in a 2D reaching task. Limb position was used to simultaneously control a robot arm using an improved network interface. Virtual arm muscle activations responded to motoneuron firing rates, with virtual arm muscles lengths encoded via population coding in the proprioceptive population. After training, the virtual arm performed reaching movements which were smoother and more realistic than those obtained using a simplistic arm model. This system provided access to both spiking network properties and to arm biophysical properties, including muscle forces. The use of a musculoskeletal virtual arm and the improved control system allowed the robot arm to perform movements which were smoother than those reported in our previous paper using a simplistic arm. This work provides a novel approach consisting of bidirectionally connecting a cortical model to a realistic virtual arm, and using the system output to drive a robotic arm in real time. Our techniques are applicable to the future development of brain neuroprosthetic control systems, and may enable enhanced brain-machine interfaces with the possibility for finer control of limb prosthetics.

Patient-specific wall stress analysis in cerebral aneurysms using inverse shell model.

Stress analyses of patient-specific vascular structures commonly assume that the reconstructed in vivo configuration is stress free although it is in a pre-deformed state. We submit that this assumption can be obviated using an inverse approach, thus increasing accuracy of stress estimates. In this paper, we introduce an inverse approach of stress analysis for cerebral aneurysms modeled as nonlinear thin shell structures, and demonstrate the method using a patient-specific aneurysm. A lesion surface derived from medical images, which corresponds to the deformed configuration under the arterial pressure, is taken as the input. The wall stress in the given deformed configuration, together with the unstressed initial configuration, are predicted by solving the equilibrium equations as opposed to traditional approach where the deformed geometry is assumed stress free. This inverse approach also possesses a unique advantage, that is, for some lesions it enables us to predict the wall stress without accurate knowledge of the wall elastic property. In this study, we also investigate the sensitivity of the wall stress to material parameters. It is found that the in-plane component of the wall stress is indeed insensitive to the material model.

Inverse method of stress analysis for cerebral aneurysms.

We present a method for predicting the wall stress in a class of cerebral aneurysms. The method hinges on an inverse formulation of the elastostatic equilibrium problem; it takes as the input a deformed configuration and the corresponding pressure, and predicts the wall stress in the given deformed state. For a membrane structure, the inverse formulation possesses a remarkable feature, that is, it can practically determine the wall tension without accurate knowledge of the wall elastic properties. In this paper, we present a finite element formulation for the inverse membrane problem and perform material sensitivity studies on idealized lesions and an image-based cerebral aneurysm model.

Inverse elastostatic stress analysis in pre-deformed biological structures: Demonstration using abdominal aortic aneurysms.

In stress analysis of membrane-like biological structures, the geometry constructed from in vivo image, which often corresponds to a deformed state, is routinely taken as the initial stress-free geometry. In this paper, we show that this limitation can be completely removed using an inverse elastostatic approach, namely, a method for finding the initial geometry of an elastic body from a given deformed state. We demonstrate the utility of the inverse approach using a patient-specific abdominal aortic aneurysm model, and identify the scope of error in stress estimation in the conventional approach within a realistic range of material parameter variations.

[Three-dimensional finite element study on middle face advancement with distraction osteogenesis].

Based on the anatomic structure of a girl with class III skelet al malocclusion, a three-dimensional finite element biomechanical model of facial soft tissue was established. With the use of this model, three surgery plans of distraction osteogenesis, LeFort I, II, III maxillary complex advancement in the direction of functional occlusal plane, were simulated. As a result, the facial soft tissue deformation was predicted and the ratio of the facial location deformation to the free bone advancement was calculated. The facial shape after surgery could be viewed in 3D. In addition, the location of center of resistance was investigated when the free bone was protracted forward in the process of LeFort I maxillary complex advancement; it was located at a site about 30 mm posterior to the soft tissue A point. The research result indicates that three-dimensional finite element research on distraction osteogenesis can provide instruction for setting the suitable protraction point and direction of the protraction force in surgery, and by predicting the facial soft tissue deformation, it also can provide the surgeon and patient with information on the options and reference to the surgery plans.

[Study on the 3D mathematical mode of the muscle groups applied to human mandible by a linear programming method].

Four types of 3D mathematical mode of the muscle groups applied to the human mandible have been developed. One is based on electromyography (EMG) and the others are based on linear programming with different objective function. Each model contains 26 muscle forces and two joint forces, allowing simulation of static bite forces and concomitant joint reaction forces for various bite point locations and mandibular positions. In this paper, the method of image processing to measure the position and direction of muscle forces according to 3D CAD model was built with CT data. Matlab optimization toolbox is applied to solve the three modes based on linear programming. Results show that the model with an objective function requiring a minimum sum of the tensions in the muscles is reasonable and agrees very well with the normal physiology activity.