Estimating 3D kinematics and kinetics from virtual inertial sensor data through musculoskeletal movement simulations.

Portable measurement systems using inertial sensors enable motion capture outside the lab, facilitating longitudinal and large-scale studies in natural environments. However, estimating 3D kinematics and kinetics from inertial data for a comprehensive biomechanical movement analysis is still challenging. Machine learning models or stepwise approaches performing Kalman filtering, inverse kinematics, and inverse dynamics can lead to inconsistencies between kinematics and kinetics. We investigated the reconstruction of 3D kinematics and kinetics of arbitrary running motions from inertial sensor data using optimal control simulations of full-body musculoskeletal models. To evaluate the feasibility of the proposed method, we used marker tracking simulations created from optical motion capture data as a reference and for computing virtual inertial data such that the desired solution was known exactly. We generated the inertial tracking simulations by formulating optimal control problems that tracked virtual acceleration and angular velocity while minimizing effort without requiring a task constraint or an initial state. To evaluate the proposed approach, we reconstructed three trials each of straight running, curved running, and a v-cut of 10 participants. We compared the estimated inertial signals and biomechanical variables of the marker and inertial tracking simulations. The inertial data was tracked closely, resulting in low mean root mean squared deviations for pelvis translation (≤20.2 mm), angles (≤1.8 deg), ground reaction forces (≤1.1 BW%), joint moments (≤0.1 BWBH%), and muscle forces (≤5.4 BW%) and high mean coefficients of multiple correlation for all biomechanical variables (≥0.99). Accordingly, our results showed that optimal control simulations tracking 3D inertial data could reconstruct the kinematics and kinetics of individual trials of all running motions. The simulations led to mutually and dynamically consistent kinematics and kinetics, which allows researching causal chains, for example, to analyze anterior cruciate ligament injury prevention. Our work proved the feasibility of the approach using virtual inertial data. When using the approach in the future with measured data, the sensor location and alignment on the segment must be estimated, and soft-tissue artifacts are potential error sources. Nevertheless, we demonstrated that optimal control simulation tracking inertial data is highly promising for estimating 3D kinematics and kinetics for a comprehensive biomechanical analysis.

Contributing Components of Metabolic Energy Models to Metabolic Cost Estimations in Gait.

OBJECTIVE: As metabolic cost is a primary factor influencing humans' gait, we want to deepen our understanding of metabolic energy expenditure models. Therefore, this paper identifies the parameters and input variables, such as muscle or joint states, that contribute to accurate metabolic cost estimations. METHODS: We explored the parameters of four metabolic energy expenditure models in a Monte Carlo sensitivity analysis. Then, we analysed the model parameters by their calculated sensitivity indices, physiological context, and the resulting metabolic rates during the gait cycle. The parameter combination with the highest accuracy in the Monte Carlo simulations represented a quasi-optimized model. In the second step, we investigated the importance of input parameters and variables by analysing the accuracy of neural networks trained with different input features. RESULTS: Power-related parameters were most influential in the sensitivity analysis and the neural network-based feature selection. We observed that the quasi-optimized models produced negative metabolic rates, contradicting muscle physiology. Neural network-based models showed promising abilities but have been unable to match the accuracy of traditional metabolic energy expenditure models. CONCLUSION: We showed that power-related metabolic energy expenditure model parameters and inputs are most influential during gait. Furthermore, our results suggest that neural network-based metabolic energy expenditure models are viable. However, bigger datasets are required to achieve better accuracy. SIGNIFICANCE: As there is a need for more accurate metabolic energy expenditure models, we explored which musculoskeletal parameters are essential when developing a model to estimate metabolic energy.

Methods for integrating postural control into biomechanical human simulations: a systematic review.

Understanding of the human body's internal processes to maintain balance is fundamental to simulate postural control behaviour. The body uses multiple sensory systems' information to obtain a reliable estimate about the current body state. This information is used to control the reactive behaviour to maintain balance. To predict a certain motion behaviour with knowledge of the muscle forces, forward dynamic simulations of biomechanical human models can be utilized. We aim to use predictive postural control simulations to give therapy recommendations to patients suffering from postural disorders in the future. It is important to know which types of modelling approaches already exist to apply such predictive forward dynamic simulations. Current literature provides different models that aim to simulate human postural control. We conducted a systematic literature research to identify the different approaches of postural control models. The different approaches are discussed regarding their applied biomechanical models, sensory representation, sensory integration, and control methods in standing and gait simulations. We searched on Scopus, Web of Science and PubMed using a search string, scanned 1253 records, and found 102 studies to be eligible for inclusion. The included studies use different ways for sensory representation and integration, although underlying neural processes still remain unclear. We found that for postural control optimal control methods like linear quadratic regulators and model predictive control methods are used less, when models' level of details is increasing, and nonlinearities become more important. Considering musculoskeletal models, reflex-based and PD controllers are mainly applied and show promising results, as they aim to create human-like motion behaviour considering physiological processes.

Change the direction: 3D optimal control simulation by directly tracking marker and ground reaction force data.

Optimal control simulations of musculoskeletal models can be used to reconstruct motions measured with optical motion capture to estimate joint and muscle kinematics and kinetics. These simulations are mutually and dynamically consistent, in contrast to traditional inverse methods. Commonly, optimal control simulations are generated by tracking generalized coordinates in combination with ground reaction forces. The generalized coordinates are estimated from marker positions using, for example, inverse kinematics. Hence, inaccuracies in the estimated coordinates are tracked in the simulation. We developed an approach to reconstruct arbitrary motions, such as change of direction motions, using optimal control simulations of 3D full-body musculoskeletal models by directly tracking marker and ground reaction force data. For evaluation, we recorded three trials each of straight running, curved running, and a v-cut for 10 participants. We reconstructed the recordings with marker tracking simulations, coordinate tracking simulations, and inverse kinematics and dynamics. First, we analyzed the convergence of the simulations and found that the wall time increased three to four times when using marker tracking compared to coordinate tracking. Then, we compared the marker trajectories, ground reaction forces, pelvis translations, joint angles, and joint moments between the three reconstruction methods. Root mean squared deviations between measured and estimated marker positions were smallest for inverse kinematics (e.g., 7.6 ± 5.1 mm for v-cut). However, measurement noise and soft tissue artifacts are likely also tracked in inverse kinematics, meaning that this approach does not reflect a gold standard. Marker tracking simulations resulted in slightly higher root mean squared marker deviations (e.g., 9.5 ± 6.2 mm for v-cut) than inverse kinematics. In contrast, coordinate tracking resulted in deviations that were nearly twice as high (e.g., 16.8 ± 10.5 mm for v-cut). Joint angles from coordinate tracking followed the estimated joint angles from inverse kinematics more closely than marker tracking (e.g., root mean squared deviation of 1.4 ± 1.8 deg vs. 3.5 ± 4.0 deg for v-cut). However, we did not have a gold standard measurement of the joint angles, so it is unknown if this larger deviation means the solution is less accurate. In conclusion, we showed that optimal control simulations of change of direction running motions can be created by tracking marker and ground reaction force data. Marker tracking considerably improved marker accuracy compared to coordinate tracking. Therefore, we recommend reconstructing movements by directly tracking marker data in the optimal control simulation when precise marker tracking is required.

Perspective on "in the wild" movement analysis using machine learning.

Recent advances in wearable sensing and machine learning have created ample opportunities for "in the wild" movement analysis in sports, since the combination of both enables real-time feedback to be provided to athletes and coaches, as well as long-term monitoring of movements. The potential for real-time feedback is useful for performance enhancement or technique analysis, and can be achieved by training efficient models and implementing them on dedicated hardware. Long-term monitoring of movement can be used for injury prevention, among others. Such applications are often enabled by training a machine learned model from large datasets that have been collected using wearable sensors. Therefore, in this perspective paper, we provide an overview of approaches for studies that aim to analyze sports movement "in the wild" using wearable sensors and machine learning. First, we discuss how a measurement protocol can be set up by answering six questions. Then, we discuss the benefits and pitfalls and provide recommendations for effective training of machine learning models from movement data, focusing on data pre-processing, feature calculation, and model selection and tuning. Finally, we highlight two application domains where "in the wild" data recording was combined with machine learning for injury prevention and technique analysis, respectively.

Predictive Simulations to Replicate Human Gait Adaptations and Energetics With Exoskeletons.

Robotic exoskeletons have the potential to restore and enhance human mobility. However, optimally controlling these devices, to work in concert with human users, is challenging. Accurate model simulations of the interaction between exoskeletons and users may expedite the design process and improve control. Here, as a proof of principle, we tested if we could use predictive simulations to replicate human gait adaptations and changes in energy expenditure from an experiment where participants walked with exoskeletons. We recreated a past experimental paradigm, where robotic exoskeletons were used to shift people's energetically optimal step frequency to frequencies higher and lower than normally preferred. To match the experimental controller, we modelled knee-worn exoskeletons that applied resistive torques, either proportional or inversely proportional to step frequency-decreasing or increasing the energy optimal step frequency, respectively. We were able to replicate the experiment, finding higher and lower optimal step frequencies than in natural walking under each respective condition. Our simulated resistive torques and objective landscapes resembled the measured experimental resistive torque and energy landscapes. Individual muscle energetics revealed distinct coordination strategies consistent with each exoskeleton controller condition. Increasing the accuracy of step frequency and energetic predictions was best achieved by increasing the number of virtual participants (varying whole-body anthropometrics), rather than the number of muscle parameter sets (varying muscle anthropometrics). In future, our approach can be used to design controllers in advance of human testing, to help identify reasonable solution spaces or tailor design to individual users.

Antagonistic co-contraction can minimize muscular effort in systems with uncertainty.

Muscular co-contraction of antagonistic muscle pairs is often observed in human movement, but it is considered inefficient and it can currently not be predicted in simulations where muscular effort or metabolic energy are minimized. Here, we investigated the relationship between minimizing effort and muscular co-contraction in systems with random uncertainty to see if muscular co-contraction can minimize effort in such system. We also investigated the effect of time delay in the muscle, by varying the time delay in the neural control as well as the activation time constant. We solved optimal control problems for a one-degree-of-freedom pendulum actuated by two identical antagonistic muscles, using forward shooting, to find controller parameters that minimized muscular effort while the pendulum remained upright in the presence of noise added to the moment at the base of the pendulum. We compared a controller with and without feedforward control. Task precision was defined by bounding the root mean square deviation from the upright position, while different perturbation levels defined task difficulty. We found that effort was minimized when the feedforward control was nonzero, even when feedforward control was not necessary to perform the task, which indicates that co-contraction can minimize effort in systems with uncertainty. We also found that the optimal level of co-contraction increased with time delay, both when the activation time constant was increased and when neural time delay was added. Furthermore, we found that for controllers with a neural time delay, a different trajectory was optimal for a controller with feedforward control than for one without, which indicates that simulation trajectories are dependent on the controller architecture. Future movement predictions should therefore account for uncertainty in dynamics and control, and carefully choose the controller architecture. The ability of models to predict co-contraction from effort or energy minimization has important clinical and sports applications. If co-contraction is undesirable, one should aim to remove the cause of co-contraction rather than the co-contraction itself.

Adaptation Strategies for Personalized Gait Neuroprosthetics.

Personalization of gait neuroprosthetics is paramount to ensure their efficacy for users, who experience severe limitations in mobility without an assistive device. Our goal is to develop assistive devices that collaborate with and are tailored to their users, while allowing them to use as much of their existing capabilities as possible. Currently, personalization of devices is challenging, and technological advances are required to achieve this goal. Therefore, this paper presents an overview of challenges and research directions regarding an interface with the peripheral nervous system, an interface with the central nervous system, and the requirements of interface computing architectures. The interface should be modular and adaptable, such that it can provide assistance where it is needed. Novel data processing technology should be developed to allow for real-time processing while accounting for signal variations in the human. Personalized biomechanical models and simulation techniques should be developed to predict assisted walking motions and interactions between the user and the device. Furthermore, the advantages of interfacing with both the brain and the spinal cord or the periphery should be further explored. Technological advances of interface computing architecture should focus on learning on the chip to achieve further personalization. Furthermore, energy consumption should be low to allow for longer use of the neuroprosthesis. In-memory processing combined with resistive random access memory is a promising technology for both. This paper discusses the aforementioned aspects to highlight new directions for future research in gait neuroprosthetics.

Efficient trajectory optimization for curved running using a 3D musculoskeletal model with implicit dynamics.

Trajectory optimization with musculoskeletal models can be used to reconstruct measured movements and to predict changes in movements in response to environmental changes. It enables an exhaustive analysis of joint angles, joint moments, ground reaction forces, and muscle forces, among others. However, its application is still limited to simplified problems in two dimensional space or straight motions. The simulation of movements with directional changes, e.g. curved running, requires detailed three dimensional models which lead to a high-dimensional solution space. We extended a full-body three dimensional musculoskeletal model to be specialized for running with directional changes. Model dynamics were implemented implicitly and trajectory optimization problems were solved with direct collocation to enable efficient computation. Standing, straight running, and curved running were simulated starting from a random initial guess to confirm the capabilities of our model and approach: efficacy, tracking and predictive power. Altogether the simulations required 1 h 17 min and corresponded well to the reference data. The prediction of curved running using straight running as tracking data revealed the necessity of avoiding interpenetration of body segments. In summary, the proposed formulation is able to efficiently predict a new motion task while preserving dynamic consistency. Hence, labor-intensive and thus costly experimental studies could be replaced by simulations for movement analysis and virtual product design.

Exploring the Contribution of Proprioceptive Reflexes to Balance Control in Perturbed Standing.

Humans control balance using different feedback loops involving the vestibular system, the visual system, and proprioception. In this article, we focus on proprioception and explore the contribution of reflexes based on force and length feedback to standing balance. In particular, we address the questions of how much proprioception alone could explain balance control, and whether one modality, force or length feedback, is more important than the other. A sagittal plane neuro-musculoskeletal model was developed with six degrees of freedom and nine muscles in each leg. A controller was designed using proprioceptive reflexes and a dead zone. No feedback control was applied inside the dead zone. Reflexes were active once the center of mass moved outside the dead zone. Controller parameters were found by solving an optimization problem, where effort was minimized while the neuro-musculoskeletal model should remain standing upright on a perturbed platform. The ground was perturbed with random square pulses in the sagittal plane with different amplitudes and durations. The optimization was solved for three controllers: using force and length feedback (base model), using only force feedback, and using only length feedback. Simulations were compared to human data from previous work, where an experiment with the same perturbation signal was performed. The optimized controller yielded a similar posture, since average joint angles were within 5 degrees of the experimental average joint angles. The joint angles of the base model, the length only model, and the force only model correlated weakly (ankle) to moderately with the experimental joint angles. The ankle moment correlated weakly to moderately with the experimental ankle moment, while the hip and knee moment were only weakly correlated, or not at all. The time series of the joint angles showed that the length feedback model was better able to explain the experimental joint angles than the force feedback model. Changes in time delay affected the correlation of the joint angles and joint moments. The objective of effort minimization yielded lower joint moments than in the experiment, suggesting that other objectives are also important in balance control, which cause an increase in effort and thus larger joint moments.

CNN-Based Estimation of Sagittal Plane Walking and Running Biomechanics From Measured and Simulated Inertial Sensor Data.

Machine learning is a promising approach to evaluate human movement based on wearable sensor data. A representative dataset for training data-driven models is crucial to ensure that the model generalizes well to unseen data. However, the acquisition of sufficient data is time-consuming and often infeasible. We present a method to create realistic inertial sensor data with corresponding biomechanical variables by 2D walking and running simulations. We augmented a measured inertial sensor dataset with simulated data for the training of convolutional neural networks to estimate sagittal plane joint angles, joint moments, and ground reaction forces (GRFs) of walking and running. When adding simulated data, the root mean square error (RMSE) of the test set of hip, knee, and ankle joint angles decreased up to 17%, 27% and 23%, the RMSE of knee and ankle joint moments up to 6% and the RMSE of anterior-posterior and vertical GRF up to 2 and 6%. Simulation-aided estimation of joint moments and GRFs was limited by inaccuracies of the biomechanical model. Improving the physics-based model and domain adaptation learning may further increase the benefit of simulated data. Future work can exploit biomechanical simulations to connect different data sources in order to create representative datasets of human movement. In conclusion, machine learning can benefit from available domain knowledge on biomechanical simulations to supplement cumbersome data collections.

A solution method for predictive simulations in a stochastic environment.

Predictive gait simulations currently do not account for environmental or internal noise. We describe a method to solve predictive simulations of human movements in a stochastic environment using a collocation method. The optimization is performed over multiple noisy episodes of the trajectory, instead of a single episode in a deterministic environment. Each episode used the same control parameters. The method was verified on a torque-driven pendulum swing-up problem. A different optimal trajectory was found in a stochastic environment than in the deterministic environment. Next, it was applied to gait to show its application in predictive simulation of human movement. We show that, unlike in a deterministic model, a nonzero minimum foot clearance during swing is predicted by a minimum-effort criterion in a stochastic environment. The predicted amount of foot clearance increased with the noise amplitude.

Metabolic cost calculations of gait using musculoskeletal energy models, a comparison study.

This paper compares predictions of metabolic energy expenditure in gait using seven metabolic energy expenditure models to assess their correlation with experimental data. Ground reaction forces, marker data, and pulmonary gas exchange data were recorded for six walking trials at combinations of two speeds, 0.8 m/s and 1.3 m/s, and three inclines, -8% (downhill), level, and 8% (uphill). The metabolic cost, calculated with the metabolic energy models was compared to the metabolic cost from the pulmonary gas exchange rates. A repeated measures correlation showed that all models correlated well with experimental data, with correlations of at least 0.9. The model by Bhargava et al. (J Biomech, 2004: 81-88) and the model by Lichtwark and Wilson (J Exp Biol, 2005: 2831-3843) had the highest correlation, 0.95. The model by Margaria (Int Z Angew Physiol Einschl Arbeitsphysiol, 1968: 339-351) predicted the increase in metabolic cost following a change in dynamics best in absolute terms.

A metabolic energy expenditure model with a continuous first derivative and its application to predictive simulations of gait.

Whether humans minimize metabolic energy in gait is unknown. Gradient-based optimization could be used to predict gait without using walking data but requires a twice differentiable metabolic energy model. Therefore, the metabolic energy model of Umberger et al. ( 2003 ) was adapted to be twice differentiable. Predictive simulations of a reaching task and gait were solved using this continuous model and by minimizing effort. The reaching task simulation showed that energy minimization predicts unrealistic movements when compared to effort minimization. The predictive gait simulations showed that objectives other than metabolic energy are also important in gait.

Joint contact forces can be reduced by improving joint moment symmetry in below-knee amputee gait simulations.

Despite having a fully functional knee and hip in both legs, asymmetries in joint moments of the knee and hip are often seen in gait of persons with a unilateral transtibial amputation (TTA), possibly resulting in excessive joint loading. We hypothesize that persons with a TTA can walk with more symmetric joint moments at the cost of increased effort or abnormal kinematics. The hypothesis was tested using predictive simulations of gait. Open loop controls of one gait cycle were found by solving an optimization problem that minimizes a combination of walking effort and tracking error in joint angles, ground reaction force and gait cycle duration. A second objective was added to penalize joint moment asymmetry, creating a multi-objective optimization problem. A Pareto front was constructed by changing the weights of the objectives and three solutions were analyzed to study the effect of increasing joint moment symmetry. When the optimization placed more weight on moment symmetry, walking effort increased and kinematics became less normal, confirming the hypothesis. TTA gait improved with a moderate increase in joint moment symmetry. At a small cost of effort and abnormal kinematics, the peak hip extension moment in the intact leg was decreased significantly, and so was the joint contact force in the knee and hip. Additional symmetry required a significant increase in walking effort and the joint contact forces in both hips became significantly higher than in able-bodied gait.