View-independent gait events detection using CNN-transformer hybrid network.

Accurate gait detection is crucial in utilizing the ample health information embedded in it. Vision-based approaches for gait detection have emerged as an alternative to the exacting sensor-based approaches, but their application has been rather limited due to complicated feature engineering processes and heavy reliance on lateral views. Thus, this study aimed to find a simple vision-based approach that is view-independent and accurate. A total of 22 participants performed six different actions representing standard and peculiar gaits, and the videos acquired from these actions were used as the input of the deep learning networks. Four networks, including a 2D convolutional neural network and an attention-based deep learning network, were trained with standard gaits, and their detection performance for both standard and peculiar gaits was assessed using measures including F1-scores. While all networks achieved remarkable detection performance, the CNN-Transformer network achieved the best performance for both standard and peculiar gaits. Little deviation by the speed of actions or view angles was found. The study is expected to contribute to the wider application of vision-based approaches in gait detection and gait-based health monitoring both at home and in clinical settings.

sEMG-based Sarcopenia risk classification using empirical mode decomposition and machine learning algorithms.

Early detection of the risk of sarcopenia at younger ages is crucial for implementing preventive strategies, fostering healthy muscle development, and minimizing the negative impact of sarcopenia on health and aging. In this study, we propose a novel sarcopenia risk detection technique that combines surface electromyography (sEMG) signals and empirical mode decomposition (EMD) with machine learning algorithms. First, we recorded and preprocessed sEMG data from both healthy and at-risk individuals during various physical activities, including normal walking, fast walking, performing a standard squat, and performing a wide squat. Next, electromyography (EMG) features were extracted from a normalized EMG and its intrinsic mode functions (IMFs) were obtained through EMD. Subsequently, a minimum redundancy maximum relevance (mRMR) feature selection method was employed to identify the most influential subset of features. Finally, the performances of state-of-the-art machine learning (ML) classifiers were evaluated using a leave-one-subject-out cross-validation technique, and the effectiveness of the classifiers for sarcopenia risk classification was assessed through various performance metrics. The proposed method shows a high accuracy, with accuracy rates of 0.88 for normal walking, 0.89 for fast walking, 0.81 for a standard squat, and 0.80 for a wide squat, providing reliable identification of sarcopenia risk during physical activities. Beyond early sarcopenia risk detection, this sEMG-EMD-ML system offers practical values for assessing muscle function, muscle health monitoring, and managing muscle quality for an improved daily life and well-being.

Prediction of lower limb kinematics from vision-based system using deep learning approaches.

The joint angular velocity during daily life exercises is an important clinical outcome for injury risk index, rehabilitation progress monitoring and athlete's performance evaluation. Recently, wearable sensors have been widely used to monitor lower limb kinematics. However, these sensors are difficult and inconvenient to use in daily life. To mitigate these limitations, this study proposes a vision-based system for estimating lower limb joint kinematics using a deep convolution neural network with bi-directional long-short term memory and gated recurrent unit network. The normalized correlation coefficient, and the mean absolute error were computed between the ground truth obtained from the optical motion capture system and estimated joint angular velocities using proposed models. The estimated results show a highest correlation 0.93 in squat and 0.92 in walking on treadmill action. Furthermore, independent model for each joint angular velocity at the hip, knee, and ankle were analyzed and compared. Among the three joint angular velocities, knee joint has a best estimated accuracy (0.96 in squat and 0.96 in walking on the treadmill). The proposed models show higher estimation accuracy under both the lateral and the frontal view regardless of the camera positions and angles. This study proves the applicability of using sensor free vision-based system to monitor the lower limb kinematics during home workouts for healthcare and rehabilitation.

Deep Learning Networks for View-independent Knee and Elbow Joint Angle Estimation.

Vision-based human joint angle estimation is essential for remote and continuous health monitoring. Most vision-based angle estimation methods use the locations of human joints extracted using optical motion cameras, depth cameras, or human pose estimation models. This study aimed to propose a reliable and straightforward approach with deep learning networks for knee and elbow flexion/extension angle estimation from the RGB video. Fifteen healthy participants performed four daily activities in this study. The experiments were conducted with four different deep learning networks, and the networks took nine subsequent frames as input while output was knee and elbow joint angles extracted from an optical motion capture system for each frame. The BiLSTM network-based joint angles estimator can estimate both joint angles with a correlation of 0.955 for knee and 0.917 for elbow joints regardless of the camera view angles.

Vision-Based Gait Events Detection Using Deep Convolutional Neural Networks.

Accurate gait events detection from the video would be a challenging problem. However, most vision-based methods for gait event detection highly rely on gait features that are estimated using gait silhouettes and human pose information for accurate gait data acquisition. This paper presented an accurate, multi-view approach with deep convolutional neural networks for efficient and practical gait event detection without requiring additional gait feature engineering. Especially, we aimed to detect gait events from frontal views as well as lateral views. We conducted the experiments with four different deep CNN models on our own dataset that includes three different walking actions from 11 healthy participants. Models took 9 subsequence frames stacking together as inputs, while outputs of models were probability vectors of gait events: toe-off and heel-strike for each frame. The deep CNN models trained only with video frames enabled to detect gait events with 93% or higher accuracy while the user is walking straight and walking around on both frontal and lateral views.

Vision-based human joint angular velocity estimation during squat and walking on a treadmill actions.

Elderly health monitoring, rehabilitation training, and sport supervision could benefit from continuous assessment of joint angle, and angular velocity to identify the joint movement patterns. However, most of the measurement systems are designed based on special kinematic sensors to estimate angular velocities. The study aims to measure the lower limb joint angular velocity based on a 2D vision camera system during squat and walking on treadmill action using deep convolution neural network (CNN) architecture. Experiments were conducted on 12 healthy adults, and six digital cameras were used to capture the videos of the participant actions in lateral and frontal view. The normalized cross-correlation (Ccnorm) analysis was performed to obtain a degree of symmetry of the ground truth and estimated angular velocity waveform patterns. Mean Ccnorm for angular velocity estimation by deep CNN model has higher than 0.90 in walking on the treadmill and 0.89 in squat action. Furthermore, joint-wise angular velocities at the hip, knee, and ankle joints were observed and compared. The proposed system gets higher estimation performance under the lateral view and the frontal view of the camera. This study potentially eliminates the requirement of wearable sensors and proves the applicability of using video-based system to measure joint angular velocities during squat and walking on a treadmill actions.