A Data-Driven Approach to Predict Fatigue in Exercise Based on Motion Data from Wearable Sensors or Force Plate.

Fatigue increases the risk of injury during sports training and rehabilitation. Early detection of fatigue during exercises would help adapt the training in order to prevent over-training and injury. This study lays the foundation for a data-driven model to automatically predict the onset of fatigue and quantify consequent fatigue changes using a force plate (FP) or inertial measurement units (IMUs). The force plate and body-worn IMUs were used to capture movements associated with exercises (squats, high knee jacks, and corkscrew toe-touch) to estimate participant-specific fatigue levels in a continuous fashion using random forest (RF) regression and convolutional neural network (CNN) based regression models. Analysis of unseen data showed high correlation (up to 89%, 93%, and 94% for the squat, jack, and corkscrew exercises, respectively) between the predicted fatigue levels and self-reported fatigue levels. Predictions using force plate data achieved similar performance as those with IMU data; the best results in both cases were achieved with a convolutional neural network. The displacement of the center of pressure (COP) was found to be correlated with fatigue compared to other commonly used features of the force plate. Bland-Altman analysis also confirmed that the predicted fatigue levels were close to the true values. These results contribute to the field of human motion recognition by proposing a deep neural network model that can detect fairly small changes of motion data in a continuous process and quantify the movement. Based on the successful findings with three different exercises, the general nature of the methodology is potentially applicable to a variety of other forms of exercises, thereby contributing to the future adaptation of exercise programs and prevention of over-training and injury as a result of excessive fatigue.

Adaptive auditory assistance for stride length cadence modification in older adults and people with Parkinson's.

Gait rehabilitation using auditory cues can help older adults and people with Parkinson's improve walking performance. While auditory cues are convenient and can reliably modify gait cadence, it is not clear if auditory cues can reliably modify stride length (SL), another key gait performance metric. Existing algorithms also do not address habituation or fluctuation in motor capability, and have not been evaluated with target populations or under dual-task conditions. In this study, we develop an adaptive auditory cueing framework that aims to modulate SL and cadence. The framework monitors the gait parameters and learns a personalized cue-response model to relate the gait parameters to the input cues. The cue-response model is represented using a multi-output Gaussian Process (MOGP) and is used during optimization to select the cue to provide. The adaptive cueing approach is benchmarked against the fixed approach, where cues are provided at a fixed cadence. The two approaches are tested under single and dual-task conditions with 13 older adults (OA) and 8 people with Parkinson's (PwP). The results show that more than half of the OA and PwP in the study can change both SL and cadence using auditory cues. The fixed approach is best at changing people's gait without secondary task, however, the addition of the secondary task significantly degrades effectiveness at changing SL. The adaptive approach can maintain the same level of SL change regardless of the presence of the secondary task. A separate analysis is conducted to identify factors that influence the performance of the adaptive framework. Gait information from the previous time step, along with the previous input cue, can improve its prediction accuracy. More diversity in the initialization data can also improve the GP model. Finally, we did not find a strong correlation between stride length and cadence when the parameters are contingent upon input cues.

A biomechanics and energetics dataset of neurotypical adults walking with and without kinematic constraints.

Numerous studies have explored the biomechanics and energetics of human walking, offering valuable insights into how we walk. However, prior studies focused on changing external factors (e.g., walking speed) and examined group averages and trends rather than individual adaptations in the presence of internal constraints (e.g., injury-related muscle weakness). To address this gap, this paper presents an open dataset of human walking biomechanics and energetics collected from 21 neurotypical young adults. To investigate the effects of internal constraints (reduced joint range of motion), the participants are both the control group (free walking) and the intervention group (constrained walking - left knee fully extended using a passive orthosis). Each subject walked on a dual-belt treadmill at three speeds (0.4, 0.8, and 1.1 m/s) and five step frequencies ( - 10% to 20% of their preferred frequency) for a total of 30 test conditions. The dataset includes raw and segmented data featuring ground reaction forces, joint motion, muscle activity, and metabolic data. Additionally, a sample code is provided for basic data manipulation and visualisation.

Experimental evaluation of accuracy and efficiency of two control strategies for a novel foot commanded robotic laparoscope holders with surgeons.

The implementation of a laparoscope-holding robot in minimally invasive surgery enhances the efficiency and safety of the operation. However, the extra robot control task can increase the cognitive load on surgeons. A suitable interface may simplify the control task and reduce the surgeon load. Foot interfaces are commonly used for commanding laparoscope-holding robots, with two control strategies available: decoupled control permits only one Cartesian axis actuation, known as decoupled commands; hybrid control allows for both decoupled commands and multiple axes actuation, known as coupled commands. This paper aims to determine the optimal control strategy for foot interfaces by investigating two common assumptions in the literature: (1) Decoupled control is believed to result in better predictability of the final laparoscopic view orientation, and (2) Hybrid control has the efficiency advantage in laparoscope control. Our user study with 11 experienced and trainee surgeons shows that decoupled control has better predictability than hybrid control, while both approaches are equally efficient. In addition, using two surgery-like tasks in a simulator, users' choice of decoupled and coupled commands is analysed based on their level of surgical experience and the nature of the movement. Results show that trainee surgeons tend to issue more commands than the more experienced participants. Single decoupled commands were frequently used in small view adjustments, while a mixture of coupled and decoupled commands was preferred in larger view adjustments. A guideline for foot interface control strategy selection is provided.

Effect of Tibial Component Misalignment on the Lower Limb Joint Kinematics During Squat.

The component orientation of the total knee replacement is critical to surgical outcomes. There have been many studies focused on knee movement for different component rotations. However, the effect of component misalignment on a dynamic movement, especially which requires high knee flexion, is not widely studied. The aim of this study is to investigate the effect of tibial component misalignment on a squat motion by predictive simulation. Squat motions with different replacement component alignments were predicted by formulating an optimal control problem. The result indicates that component misalignment on coronal and horizontal planes reduces peak joint flexion angles and the external rotation on the horizontal plane has the most negative impact. Misalignment in external rotation resulted in the greatest reduction of peak joint flexion angles. The simulation was validated by comparison with experimental data, which showed a high level of correlation with the predicted motion.Clinical relevance- The predictive simulation presented in this study can predict the dynamic post-surgery movement of TKR. It has the potential to help surgeons and clinicians at the preoperative planning stage.

Auditory cueing strategy for stride length and cadence modification: a feasibility study with healthy adults.

People with Parkinson's Disease experience gait impairments that significantly impact their quality of life. Visual, auditory, and tactile cues can alleviate gait impairments, but they can become less effective due to the progressive nature of the disease and changes in people's motor capability. In this study, we develop a human-in-the-loop (HIL) framework that monitors two key gait parameters, stride length and cadence, and continuously learns a person-specific model of how the parameters change in response to the feedback. The model is then used in an optimization algorithm to improve the gait parameters. This feasibility study examines whether auditory cues can be used to influence stride length in people without gait impairments. The results demonstrate the benefits of the HIL framework in maintaining people's stride length in the presence of a secondary task.Clinical relevance- This paper proposes a gait rehabilitation framework that provides a personalized cueing strategy based on the person's real-time response to cues. The proposed approach has potential application to people with Parkinson's Disease.

Adaptive cueing strategy for gait modification: A case study using auditory cues.

People with Parkinson's (PwP) experience gait impairments that can be improved through cue training, where visual, auditory, or haptic cues are provided to guide the walker's cadence or step length. There are two types of cueing strategies: open and closed-loop. Closed-loop cueing may be more effective in addressing habituation and cue dependency, but has to date been rarely validated with PwP. In this study, we adapt a human-in-the-loop framework to conduct preliminary analysis with four PwP. The closed-loop framework learns an individualized model of the walker's responsiveness to cues and generates an optimized cue based on the model. In this feasibility study, we determine whether participants in early stages of Parkinson's can respond to the novel cueing framework, and compare the performance of the framework to two alternative cueing strategies (fixed/proportional approaches) in changing the participant's cadence to two target cadences (speed up/slow down). The preliminary results show that the selection of the target cadence has an impact on the participant's gait performance. With the appropriate target, the framework and the fixed approaches perform similarly in slowing the participants' cadence. However, the proposed framework demonstrates better efficiency, explainability, and robustness across participants. Participants also have the highest retention rate in the absence of cues with the proposed framework. Finally, there is no clear benefit of using the proportional approach.

Modelling Physical Human-Robot Interface with Different Users, Cuffs, and Strapping Pressures: A Case Study.

Assisting persons during physical therapy or augmenting their performance often requires precise delivery of an intervention. Robotic devices are perfectly placed to do so, but their intervention highly depends on the physical human-robot connection. The inherent compliance in the connection leads to delays and losses in bi-directional power transmission and can lead to human-robot joint axes misalignment. This is often neglected in the literature by assuming a rigid connection and has a negative impact on the intervention's effectiveness and robustness. This paper presents the preliminary results of a study that aims to close that gap. The study investigates what model forms and parameters best capture human-robot connection dynamics across different persons, connection designs (cuffs), and cuff strapping pressures. The results show that the linear spring-damper model is the best compromise, but its parameters must be adjusted for each individual and different conditions separately.

Real-time forecasting of exercise-induced fatigue from wearable sensors.

Although a number of studies attempt to classify human fatigue, most models can only identify fatigue after fatigue has already occurred. In this paper, we propose a novel time series approach to forecasting wearable sensor data and associated fatigue progression during exercise. The proposed framework consists of spatio-temporal attention-based Transformer with an auxiliary critic and a fatigue classifier. The Transformer network is used to analyze the person-independent pattern underlying the past kinematic sequence obtained from wearable sensors and generate short term predictions of the human motion. Adversarial training is employed to regularize the Transformer and improve the time series forecasting performance. A fatigue classifier is used to estimate person-independent fatigue levels based on the forecasted wearable sensor data from the Transformer model. The proposed approach is validated with simulated and real squat datasets which were collected from young healthy participants. The proposed network can accurately forecast a time horizon of up to 80 timesteps for motion signal forecasting and fatigue classification. In terms of fatigue prediction, an accuracy of 83% and a Pearson correlation coefficient of 0.92 were achieved on forecasted motion data with unseen participant data. The experimental results show that our model can predict fatigue progression and outperforms other state-of-the-art techniques, achieving 95% correlation compared to 83% for the best performing baseline method. Successfully predicting fatigue progression can help a patient or athlete monitor and adjust their exercise session to prevent overexertion and fatigue-induced injury.

Varying Joint Patterns and Compensatory Strategies Can Lead to the Same Functional Gait Outcomes: A Case Study.

This paper analyses joint-space walking mechanisms and redundancies in delivering functional gait outcomes. Multiple biomechanical measures are analysed for two healthy male adults who participated in a multi-factorial study and walked during three sessions. Both participants employed varying intra- and inter-personal compensatory strategies (e.g., vaulting, hip hiking) across walking conditions and exhibited notable gait pattern alterations while keeping task-space (functional) gait parameters invariant. They also preferred various levels of asymmetric step length but kept their symmetric step time consistent and cadence-invariant during free walking. The results demonstrate the importance of an individualised approach and the need for a paradigm shift from functional (task-space) to joint-space gait analysis in attending to (a)typical gaits and delivering human-centred human-robot interaction.

Model-based data augmentation for user-independent fatigue estimation.

OBJECTIVE: User-independent recognition of exercise-induced fatigue from wearable motion data is challenging, due to inter-participant variability. This study aims to develop algorithms that can accurately estimate fatigue during exercise. METHODS: A novel approach for wearable sensor data augmentation was used to generate (via OpenSim) a large corpus of simulated wearable human motion data, based on a small corpus of human motion data measured using optical sensors. Simulated data is generated using detailed kinematic modelling with variations based on human anthropometry datasets. Using both the recorded and generated data, we trained three different neural networks (Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), DeepConvLSTM) to perform person-independent fatigue estimation from wearable motion data. RESULTS: The estimation performance increased with the amount of simulated training data. Accuracy and correlation values were higher with the proposed data augmentation method as compared to other general time series augmentation methods (e.g, rotation, jettering, magnitude wrapping) with the same amount of training data. An accuracy of 87% and a Pearson correlation coefficient of 90% were achieved on unseen data when the DeepConvLSTM model was trained with the proposed augmented dataset. CONCLUSION: The enlarged dataset significantly improves the prediction of inter-individual fatigue. SIGNIFICANCE: Appropriate augmentation techniques for biomechanical data can improve model accuracy and reduce the need for expensive data collection.

Lower body kinematics estimation from wearable sensors for walking and running: A deep learning approach.

BACKGROUND: Inertial measurement units (IMUs) are promising tools for collecting human movement data. Model-based filtering approaches (e.g. Extended Kalman Filter) have been proposed to estimate joint angles from IMUs data but little is known about the potential of data-driven approaches. RESEARCH QUESTION: Can deep learning models accurately predict lower limb joint angles from IMU data during gait? METHODS: Lower-limb kinematic data were simultaneously measured with a marker-based motion capture system and running leggings with 5 integrated IMUs measuring acceleration and angular velocity at the pelvis, thighs and tibias. Data acquisition was performed on 27 participants (26.5 (3.9) years, 1.75 (0.07) m, 68.3 (10.0) kg) while walking at 4 and 6 km/h and running at 8, 10, 12 and 14 km/h on a treadmill. The model input consists of raw IMU data, while the output estimates the joint angles of the lower body. The model was trained with a nested k-fold cross-validation and tested considering a user-independent approach. Mean error (ME), mean absolute error (MAE) and Pearson correlation coefficient (r) were computed between the ground truth and predicted joint angles. RESULTS: MAE for the DOFs ranged from 2.2(0.9) to 5.1(2.7)° with an average of 3.6(2.1)°. r ranged from 0.67(0.23) to 0.99(0.01) with moderate correlation (0.4≤r<0.7) was found for the hip right rotation and lumbar extension, strong correlation (0.7≤r<0.9) was found for the hip left rotation and ankle right/left inversion while all other DOFs showed very strong correlation (r≥0.9). SIGNIFICANCE: The proposed model can reliably predict joint kinematics for walking, running and gait transitions without specific knowledge about the body characteristics of the wearer, or the position and orientation of the IMU relative to the attached segment. These results have been validated with treadmill gait, and have not yet been confirmed for gait in other settings.

A Novel Perception Framework for Automatic Laparoscope Zoom Factor Control Using Tool Geometry.

In conventional Minimally Invasive Surgery, the surgeon conducts the operation while a human or robot holds the laparoscope. Laparoscope control is returned to the surgeon in teleoperated camera holding robots, but simultaneously controlling the laparoscope and surgical tools might be cognitively demanding. On the other hand, fully automated camera holders are still limited in their performance. To help the surgeon to better focus on the main operation while maintaining their control authority, we propose an automatic laparoscope zoom factor control framework for Robot-Assisted Minimally Invasive Surgery. In this paper, we present the perception section of the framework. It extracts and uses the surgical tool's geometric characteristics to adjust the laparoscope's zoom factor, without any artificial markers. The acceptable range and tooltip's position frequency during operations are analysed based on the gallbladder removal surgery dataset (Cholec80). The common range and tooltip's heatmap are identified and presented quantitatively.

Haptics in Teleoperated Medical Interventions: Force Measurement, Haptic Interfaces and Their Influence on User's Performance.

OBJECTIVES: Haptics in teleoperated medical interventions enables measurement and transfer of force information to the operator during robot-environment interaction. This paper provides an overview of the current research in this domain and guidelines for future investigations. METHODS: We review current technologies in force measurement and haptic devices as well as their experimental evaluation and influence on user's performance. RESULTS: Force sensing is moving away from the conventional proximal measurement methods to distal sensing and contact-less methods. Wearable devices that deliver haptic feedback on different body parts are increasingly playing an important role. Performance and accuracy improvement are the widely reported benefits of haptic feedback, while there is a debate on its effect on task completion time and exerted force. CONCLUSION: With the surge of new ideas, there is a need for better and more systematic validation of the new sensing and feedback technology, through better user studies and novel methods like validated benchmarks and new taxonomies. SIGNIFICANCE: This review investigates haptics from sensing to interfaces within the context of user's performance and the validation procedures to highlight salient advances. It provides guidelines to future developments and highlights the shortcomings in the field.

Adversarial autoencoder for visualization and classification of human activity: Application to a low-cost commercial force plate.

The ability to visualize and interpret high dimensional time-series data will be critical as wearable and other sensors are adopted in rehabilitation protocols. This study proposes a latent space representation of high dimensional time-series data for data visualization. For that purpose, a deep learning model called Adversarial AutoEncoder (AAE) is proposed to perform efficient data dimensionality reduction by considering unsupervised and semi-supervised adversarial training. Eighteen subjects were recruited for the experiment and performed two sets of exercises (upper and lower body) on the Wii Balance Board. Then, the accuracy of the latent space representation is evaluated on both sets of exercises separately. Data dimensionality reduction with conventional Machine Learning (ML) and supervised Deep Learning (DL) classification are also performed to compare the efficiency of AAE approaches. The results showed that AAE can outperform conventional ML approaches while providing close results to DL supervised classification. AAE approaches for data visualization are a promising approach to monitor the subject's movements and detect adverse events or similarity with previous data, providing an intuitive way to monitor the patient's progress and provide potential information for rehabilitation tracking.

Inverse optimal control with time-varying objectives: application to human jumping movement analysis.

Analysis of complex human movements can provide valuable insights for movement rehabilitation, sports training, humanoid robot design and control, and human-robot interaction. To accomplish complex movement, the central nervous system must coordinate the musculo-skeletal system to achieve task and internal (e.g., effort minimisation) objectives. This paper proposes an inverse optimal control approach for analysing complex human movement that does not assume that the control objective(s) remains constant throughout the movement. The movement trajectory is assumed to be optimal with respect to a cost function composed of the sum of weighted basis cost functions, which may be time varying. The weights of the cost function are recovered using a sliding window. To illustrate the proposed approach, a dataset consisting of standing broad jump to targets at three different distances is collected. The method can be used to extract control objectives that influence task success, identify different motion strategies/styles, as well as to observe how control strategy changes during the motor learning process. Kinematic analysis confirms that the identified control objectives, including centre-of-mass takeoff vector and foot placement upon landing are important to ensure that a given participant lands on the target. The dataset, including nearly 800 jump trajectories from 22 participants is also provided.

Estimation and Observability Analysis of Human Motion on Lie Groups.

This article proposes a framework for human-pose estimation from the wearable sensors that rely on a Lie group representation to model the geometry of the human movement. Human body joints are modeled by matrix Lie groups, using special orthogonal groups SO(2) and SO(3) for joint pose and special Euclidean group SE(3) for base-link pose representation. To estimate the human joint pose, velocity, and acceleration, we develop the equations for employing the extended Kalman filter on Lie groups (LG-EKF) to explicitly account for the non-Euclidean geometry of the state space. We present the observability analysis of an arbitrarily long kinematic chain of SO(3) elements based on a differential geometric approach, representing a generalization of kinematic chains of a human body. The observability is investigated for the system using marker position measurements. The proposed algorithm is compared with two competing approaches: 1) the extended Kalman filter (EKF) and 2) unscented KF (UKF) based on the Euler angle parametrization, in both simulations and extensive real-world experiments. The results show that the proposed approach achieves significant improvements over the Euler angle-based filters. It provides more accurate pose estimates, is not sensitive to gimbal lock, and more consistently estimates the covariances.

High-Resolution Motor State Detection in Parkinson's Disease Using Convolutional Neural Networks.

Patients with advanced Parkinson's disease regularly experience unstable motor states. Objective and reliable monitoring of these fluctuations is an unmet need. We used deep learning to classify motion data from a single wrist-worn IMU sensor recording in unscripted environments. For validation purposes, patients were accompanied by a movement disorder expert, and their motor state was passively evaluated every minute. We acquired a dataset of 8,661 minutes of IMU data from 30 patients, with annotations about the motor state (OFF,ON, DYSKINETIC) based on MDS-UPDRS global bradykinesia item and the AIMS upper limb dyskinesia item. Using a 1-minute window size as an input for a convolutional neural network trained on data from a subset of patients, we achieved a three-class balanced accuracy of 0.654 on data from previously unseen subjects. This corresponds to detecting the OFF, ON, or DYSKINETIC motor state at a sensitivity/specificity of 0.64/0.89, 0.67/0.67 and 0.64/0.89, respectively. On average, the model outputs were highly correlated with the annotation on a per subject scale (r = 0.83/0.84; p < 0.0001), and sustained so for the highly resolved time windows of 1 minute (r = 0.64/0.70; p < 0.0001). Thus, we demonstrate the feasibility of long-term motor-state detection in a free-living setting with deep learning using motion data from a single IMU.

Inertial measurement unit-based pose estimation: Analyzing and reducing sensitivity to sensor placement and body measures.

INTRODUCTION: Inertial measurement units have been proposed for automated pose estimation and exercise monitoring in clinical settings. However, many existing methods assume an extensive calibration procedure, which may not be realizable in clinical practice. In this study, an inertial measurement unit-based pose estimation method using extended Kalman filter and kinematic chain modeling is adapted for lower body pose estimation during clinical mobility tests such as the single leg squat, and the sensitivity to parameter calibration is investigated. METHODS: The sensitivity of pose estimation accuracy to each of the kinematic model and sensor placement parameters was analyzed. Sensitivity analysis results suggested that accurate extraction of inertial measurement unit orientation on the body is a key factor in improving the accuracy. Hence, a simple calibration protocol was proposed to reach a better approximation for inertial measurement unit orientation. RESULTS: After applying the protocol, the ankle, knee, and hip joint angle errors improved to 4 . 2 ∘ , 6 . 3 ∘ , and 8 . 3 ∘ , without the need for any other calibration. CONCLUSIONS: Only a small subset of kinematic and sensor parameters contribute significantly to pose estimation accuracy when using body worn inertial sensors. A simple calibration procedure identifying the inertial measurement unit orientation on the body can provide good pose estimation performance.

A Multi-Layer Gaussian Process for Motor Symptom Estimation in People With Parkinson's Disease.

The assessment of Parkinson's disease (PD) poses a significant challenge, as it is influenced by various factors that lead to a complex and fluctuating symptom manifestation. Thus, a frequent and objective PD assessment is highly valuable for effective health management of people with Parkinson's disease (PwP). Here, we propose a method for monitoring PwP by stochastically modeling the relationships between wrist movements during unscripted daily activities and corresponding annotations about clinical displays of movement abnormalities. We approach the estimation of PD motor signs by independently modeling and hierarchically stacking Gaussian process models for three classes of commonly observed movement abnormalities in PwP including tremor, (non-tremulous) bradykinesia, and (non-tremulous) dyskinesia. We use clinically adopted severity measures as annotations for training the models, thus allowing our multi-layer Gaussian process prediction models to estimate not only their presence but also their severities. The experimental validation of our approach demonstrates strong agreement of the model predictions with these PD annotations. Our results show that the proposed method produces promising results in objective monitoring of movement abnormalities of PD in the presence of arbitrary and unknown voluntary motions, and makes an important step toward continuous monitoring of PD in the home environment.