View-independent gait events detection using CNN-transformer hybrid network.

Accurate gait detection is crucial in utilizing the ample health information embedded in it. Vision-based approaches for gait detection have emerged as an alternative to the exacting sensor-based approaches, but their application has been rather limited due to complicated feature engineering processes and heavy reliance on lateral views. Thus, this study aimed to find a simple vision-based approach that is view-independent and accurate. A total of 22 participants performed six different actions representing standard and peculiar gaits, and the videos acquired from these actions were used as the input of the deep learning networks. Four networks, including a 2D convolutional neural network and an attention-based deep learning network, were trained with standard gaits, and their detection performance for both standard and peculiar gaits was assessed using measures including F1-scores. While all networks achieved remarkable detection performance, the CNN-Transformer network achieved the best performance for both standard and peculiar gaits. Little deviation by the speed of actions or view angles was found. The study is expected to contribute to the wider application of vision-based approaches in gait detection and gait-based health monitoring both at home and in clinical settings.

Gait Events Prediction Using Hybrid CNN-RNN-Based Deep Learning Models through a Single Waist-Worn Wearable Sensor.

Elderly gait is a source of rich information about their physical and mental health condition. As an alternative to the multiple sensors on the lower body parts, a single sensor on the pelvis has a positional advantage and an abundance of information acquirable. This study aimed to improve the accuracy of gait event detection in the elderly using a single sensor on the waist and deep learning models. Data were gathered from elderly subjects equipped with three IMU sensors while they walked. The input taken only from the waist sensor was used to train 16 deep-learning models including a CNN, RNN, and CNN-RNN hybrid with or without the Bidirectional and Attention mechanism. The groundtruth was extracted from foot IMU sensors. A fairly high accuracy of 99.73% and 93.89% was achieved by the CNN-BiGRU-Att model at the tolerance window of ±6 TS (±6 ms) and ±1 TS (±1 ms), respectively. Advancing from the previous studies exploring gait event detection, the model demonstrated a great improvement in terms of its prediction error having an MAE of 6.239 ms and 5.24 ms for HS and TO events, respectively, at the tolerance window of ±1 TS. The results demonstrated that the use of CNN-RNN hybrid models with Attention and Bidirectional mechanisms is promising for accurate gait event detection using a single waist sensor. The study can contribute to reducing the burden of gait detection and increase its applicability in future wearable devices that can be used for remote health monitoring (RHM) or diagnosis based thereon.

The Relationship Between Foot Feature Parameters and Postural Stability in Healthy Subjects.

OBJECTIVE: Little has been explored about the disparate contribution of medial longitudinal arch (MLA) and lateral longitudinal arch (LLA) to human gait and postural stability. This study aims to investigate the correlation of foot feature parameters including both MLA and LLA with postural stability. METHOD: Thirteen young and healthy subjects participated in this study. The newly developed FFMS extracted foot feature parameters in nonweight-bearing (NWB) and weight-bearing (WB) conditions along with postural stability parameters in single-leg-standing (SLS) condition. A bivariate correlation analysis was carried out to investigate the correlation between the foot characteristics and the postural stability parameters. RESULTS: The foot length and width showed negative correlation with center of pressure (CoP) distance in medio-lateral (ML) and total direction, whereas the foot length in NWB and WB conditions, and the foot width in WB condition showed positive correlation with CoP distance in anterior-posterior (AP) direction. The height of the LLA curve and the area of the MLA were correlated with the postural stability parameters in AP direction. The ratios of the LLA height and area showed moderate correlation with the CoP distance in ML direction and total direction. CONCLUSION: The size of a foot, such as the length and width, is correlated with postural stability. Whereas the MLA features are associated with postural stability in AP direction, the LLA features are associated with that in ML and total direction. APPLICATION: The findings suggest that the roles and contributions of the MLA and LLA features in and to the postural control are different.

A Foot-Arch Parameter Measurement System Using a RGB-D Camera.

The conventional method of measuring foot-arch parameters is highly dependent on the measurer's skill level, so accurate measurements are difficult to obtain. To solve this problem, we propose an autonomous geometric foot-arch analysis platform that is capable of capturing the sole of the foot and yields three foot-arch parameters: arch index (AI), arch width (AW) and arch height (AH). The proposed system captures 3D geometric and color data on the plantar surface of the foot in a static standing pose using a commercial RGB-D camera. It detects the region of the foot surface in contact with the footplate by applying the clustering and Markov random field (MRF)-based image segmentation methods. The system computes the foot-arch parameters by analyzing the 2/3D shape of the contact region. Validation experiments were carried out to assess the accuracy and repeatability of the system. The average errors for AI, AW, and AH estimation on 99 data collected from 11 subjects during 3 days were -0.17%, 0.95 mm, and 0.52 mm, respectively. Reliability and statistical analysis on the estimated foot-arch parameters, the robustness to the change of weights used in the MRF, the processing time were also performed to show the feasibility of the system.

Primary and secondary gait deviations of stroke survivors and their association with gait performance.

[Purpose] Stroke survivors exhibit abnormal pelvic motion and significantly deteriorated gait performance. Although the gait of stroke survivors has been evaluated at the primary level pertaining to ankle, knee, and hip motions, secondary deviations involving the pelvic motions are strongly related to the primary level. Therefore, the aim of this study was to identify the kinematic differences of the primary and secondary joints and to identify mechanism differences that alter the gait performance of stroke survivors. [Subjects and Methods] Five healthy subjects and five stroke survivors were recruited. All the subjects were instructed to walk at a self-selected speed. The joint kinematics and gait parameters were calculated. [Results] For the stroke survivors, the range of motion of the primary-joint motions were significantly reduced, and the secondary-joint motions were significantly increased. Additionally, for the healthy subjects, the primary joint kinematics were the main factors ensuring gait performance, whereas for the stoke survivors, the secondary-joint motions were the main factors. [Conclusion] The results indicate that while increasing the range of motion of primary-joint movements is the main target to achieve, there is a strong need to constrain and support pelvic motions in order to improve the outcome of gait rehabilitation.

sEMG-based Sarcopenia risk classification using empirical mode decomposition and machine learning algorithms.

Early detection of the risk of sarcopenia at younger ages is crucial for implementing preventive strategies, fostering healthy muscle development, and minimizing the negative impact of sarcopenia on health and aging. In this study, we propose a novel sarcopenia risk detection technique that combines surface electromyography (sEMG) signals and empirical mode decomposition (EMD) with machine learning algorithms. First, we recorded and preprocessed sEMG data from both healthy and at-risk individuals during various physical activities, including normal walking, fast walking, performing a standard squat, and performing a wide squat. Next, electromyography (EMG) features were extracted from a normalized EMG and its intrinsic mode functions (IMFs) were obtained through EMD. Subsequently, a minimum redundancy maximum relevance (mRMR) feature selection method was employed to identify the most influential subset of features. Finally, the performances of state-of-the-art machine learning (ML) classifiers were evaluated using a leave-one-subject-out cross-validation technique, and the effectiveness of the classifiers for sarcopenia risk classification was assessed through various performance metrics. The proposed method shows a high accuracy, with accuracy rates of 0.88 for normal walking, 0.89 for fast walking, 0.81 for a standard squat, and 0.80 for a wide squat, providing reliable identification of sarcopenia risk during physical activities. Beyond early sarcopenia risk detection, this sEMG-EMD-ML system offers practical values for assessing muscle function, muscle health monitoring, and managing muscle quality for an improved daily life and well-being.

Machine learning-based classification and risk factor analysis of frailty in Korean community-dwelling older adults.

Frailty is a dynamic reversible state, characterized by frequent transitions between frailty status over time. The timely and effective detection of frailty is important to prevent adverse health outcomes. This study aims to develop machine learning-based classification models for frailty assessment and to investigate its risk factors. A total of 1,482 subjects, 1,266 robust and 216 frail older adults, were analyzed. Sixteen frail risk factors were selected from a random forest-based feature selection method, then used for the inputs of five ML models: logistic regression, K-nearest neighbor, support vector machine, gaussian naïve bayes, and random forest. Data resampling, stratified 10-fold cross-validation, and grid search were applied to improve the classification performance. The logistic regression model using the selected features showed the best performance with an accuracy of 0.93 and an F1-score of 0.92. The results suggest that machine learning techniques are an effective method for classifying frailty status and exploring frailty-related factors.Clinical Relevance- Our approach can predict frailty using data collectable in clinical setting and can help prevent and improve by identifying variables that change frailty status.

Image-based Gait Spatiotemporal Parameters Estimation using a Single Camera and CNN-Transformer Hybrid Network.

Vision-based gait analysis can play an important role in the remote and continuous monitoring of the elderly's health conditions. However, most vision-based approaches compute gait spatiotemporal parameters using human pose information and provide average parameters. This study aimed to propose a straightforward method for stride-by-stride gait spatiotemporal parameters estimation. A total of 160 elderly individuals participated in this study. Data were gathered with a GAITRite system and a mobile camera simultaneously. Three deep learning networks were trained with a few RGB frames as input and a continuous 1D signal containing both spatial and temporal gait parameters as output. The trained networks estimated the stride lengths with correlations of 0.938 and more and detected gait events with F1-scores of 0.914 and more.Clinical relevance- The proposed method showed excellent agreements with the GAITRite system in analyzing spatiotemporal gait parameters. Our approach can be applied to monitor the elderly's health conditions based on their gait parameters for early diagnosis of diseases, proper treatment, and timely intervention.

Identifying Depression in the Elderly Using Gait Accelerometry.

As the number of elderly people suffering from depression increases today, new techniques for active monitoring of depression are in need than ever. Hence this study aimed to propose an approach of identifying depression in the elderly using gait accelerometry and a machine learning algorithm. A total of 45 community-dwelling elderly individuals participated in the study. Twenty-two out of 45 participants were patients with depression and the remaining 23 participants were individuals without depression. The participants completed a 7-meter walking twice at their preferred speeds with an accelerometer on their lower back. The anterior-posterior acceleration signals measured at the lower back while walking were segmented into acceleration falling and rising phases. Then eight descriptive statistical and six morphological parameters were extracted from each phase. The extracted parameters were ordered chronologically and used as a gait sequence feature. The 4-fold cross-validation of the bidirectional long short-term memory network-based classifiers that used the gait sequence feature as input showed an average accuracy of 0.956 in classifying the elderly with depression and those without depression. The study is expected to serve as a milestone exploring the use of gait accelerometry in assessing various health conditions in the future. Clinical Relevance- The findings of this study will pave a new way for self-monitoring of health conditions in the daily life of individuals, which can open the door for earlier recognition of health risks and more timely treatment.

Prediction of lower limb kinematics from vision-based system using deep learning approaches.

The joint angular velocity during daily life exercises is an important clinical outcome for injury risk index, rehabilitation progress monitoring and athlete's performance evaluation. Recently, wearable sensors have been widely used to monitor lower limb kinematics. However, these sensors are difficult and inconvenient to use in daily life. To mitigate these limitations, this study proposes a vision-based system for estimating lower limb joint kinematics using a deep convolution neural network with bi-directional long-short term memory and gated recurrent unit network. The normalized correlation coefficient, and the mean absolute error were computed between the ground truth obtained from the optical motion capture system and estimated joint angular velocities using proposed models. The estimated results show a highest correlation 0.93 in squat and 0.92 in walking on treadmill action. Furthermore, independent model for each joint angular velocity at the hip, knee, and ankle were analyzed and compared. Among the three joint angular velocities, knee joint has a best estimated accuracy (0.96 in squat and 0.96 in walking on the treadmill). The proposed models show higher estimation accuracy under both the lateral and the frontal view regardless of the camera positions and angles. This study proves the applicability of using sensor free vision-based system to monitor the lower limb kinematics during home workouts for healthcare and rehabilitation.

Deep Learning Networks for View-independent Knee and Elbow Joint Angle Estimation.

Vision-based human joint angle estimation is essential for remote and continuous health monitoring. Most vision-based angle estimation methods use the locations of human joints extracted using optical motion cameras, depth cameras, or human pose estimation models. This study aimed to propose a reliable and straightforward approach with deep learning networks for knee and elbow flexion/extension angle estimation from the RGB video. Fifteen healthy participants performed four daily activities in this study. The experiments were conducted with four different deep learning networks, and the networks took nine subsequent frames as input while output was knee and elbow joint angles extracted from an optical motion capture system for each frame. The BiLSTM network-based joint angles estimator can estimate both joint angles with a correlation of 0.955 for knee and 0.917 for elbow joints regardless of the camera view angles.

A Study on the Contribution of Medial and Lateral Longitudinal Foot Arch to Human Gait.

This study aimed to investigate the contribution of medial longitudinal arch and lateral longitudinal arch in human gait and to study the correlation between foot features and gait characteristics. The foot arch plays a significant role in human movements, and understanding its contribution to spatiotemporal gait parameters is vital in predicting and rectifying gait patterns. To serve the objectives, the study developed a new foot feature measurement system and measured the foot features and spatiotemporal gait parameters of 17 young healthy subjects without any foot structure abnormality. The foot-feature parameters were measured under three movement conditions which were sitting, standing, and one-leg standing conditions. The spatiotemporal gait parameters were measured at three speeds which were fast, preferred, and slow speeds. The correlation study showed that medial longitudinal arch characteristics were found to be associated with temporal gait parameters while lateral longitudinal arch characteristics were found to be associated with spatial gait parameters. The developed system not only eases the burden of manual measuring but also secures accuracy of the collected data. Inviting variety of subjects including athletes and people with abnormal foot structures would extend the scope of this study in the future. The findings of this study break new ground in the field of the foot- and gait-related research work.Clinical Relevance-This study demonstrated that the medial longitudinal arch and lateral longitudinal arch characteristics were related to the temporal and spatial gait parameters, respectively. These underlying findings can be applied to investigate relationships between foot abnormality and gait characteristics.

Classifying the Risk of Cognitive Impairment Using Sequential Gait Characteristics and Long Short-Term Memory Networks.

Cognitive impairment in the elderly causes a significant decline in the quality of life and a substantial economic burden on society. Yet, diagnosing cognitive impairment is apt to be missed or delayed due to its assessment being laborious. This study aimed to propose a new approach of classifying the risk of cognitive impairment in the elderly using sequential gait characteristics and machine learning techniques. A total of 108 community-dwelling elderly individuals participated in this study. The participants were categorized into three groups based on their scores of the mini-mental state examination. Each participant completed both the usual- and fast-paced walking on the straight path with two gyroscopes on each foot. By analyzing the foot sagittal angular velocity signals, the temporal gait parameters were extracted from each gait cycle. Five classical machine learning models and a long short-term memory network were respectively employed to produce the classifiers that used the time-consecutive temporal gait parameters as predictors of cognitive impairment. Five-fold cross-validation was applied to 70% of the data in each group, and the remaining 30% were used as test data. An F1-score of 0.974 was achieved in classifying the three groups by the long short-term memory network-based classifier that used the double-limb support, stance, step, and stride times at usual-paced walking and the double- and single-limb support, stance, and stride times at fast-paced walking as inputs. The proposed approach would pave the way for earlier diagnosis of cognitive impairment in non-clinical settings without professional help, which can facilitate more timely intervention.

Frailty Assessment Using Temporal Gait Characteristics and a Long Short-Term Memory Network.

Faced with the rapidly aging world population, frailty has emerged as a major health burden among the elderly. This study aimed to investigate the feasibility of using temporal gait characteristics and a long short-term memory network for assessing frailty. Seventy-four community-dwelling elderly individuals participated in this study. The participants were categorized into three groups by their FRAIL scale: robust, pre-frail, and frail groups. The participants completed a 7-meter walking at the self-selected pace with a gyroscope on each foot. Analyzing the gyroscopic data produced seven temporal gait parameters per each gait cycle. Enumerating six consecutive values of each gait parameter produced the gait sequence features which were used as frailty predictors along with the demographic features. Five-fold cross-validation was applied to 70% of the data, and the remaining 30% were used as test data. An F1-score of 0.931 was achieved in classifying the robust, pre-frail, and frail groups by the random forest model trained with age, sex, and the outputs of the long short-term memory network-based classifier that used the initial and terminal double-limb support, step, and stride times as inputs. The proposed approach of assessing frailty using the arrhythmic gait pattern of the elderly and machine learning technique is novel and promising. Pioneering a way that self-monitor frailty at home without any help from experts, the study can contribute toearly diagnosis of frailty and make timely medical intervention possible.

Personal Identification Using Gait Spectrograms and Deep Convolutional Neural Networks.

Human gait can serve as a useful behavioral trait for biometrics. Compared to fingerprint, face, and iris, the most commonly used physiological identifiers, gait can be unobtrusively monitored from a distance without requiring explicit involvement and physical restraint from people. Advances in wearable technology facilitate direct and faithful measurement of gait motions with easy-to-use and low-cost inertial sensors. This study aimed to propose an approach to using kinematic gait data collected with wearable inertial sensors for reliable personal identification. Sixty-nine individuals ranged in age from 24 to 62 years old participated in this study. The 3-axis acceleration and the 3-axis angular velocity signals were measured using the inertial measurement units attached to the feet, shanks, thighs, and posterior pelvis while walking. The gait spectrograms were acquired by applying time-frequency analyses to the lower body movement signals measured in one stride. Among each participant's 15 strides, 12 strides were used in the 4-fold cross validation of the deep convolutional neural network-based classifiers, and the remaining three strides were used to evaluate the classifiers. An accuracy of 99.69% was achieved by using the foot, shank, thigh, and pelvic spectrograms, and the accuracy was 96.89% using only the foot spectrograms. This study has the potential to be applied in behavior-based biometric technologies by confirming the feasibility of the proposed kinematic and spectrographic approaches in identifying personal behavioral characteristics.

Gait-based Human Identification through Minimum Gait-phases and Sensors.

The incredible pace at which the world's elderly population is growing will put severe burdens on current healthcare systems and resources. To alleviate this concern the health care systems must rely on the transformation of eldercare and old homes to use Ambient Assisted Living (AAL). Human identification is one of the most common and critical tasks for condition monitoring, human-machine interaction, and providing assistive services in such environments. Recently, human gait has gained new attention as a biometric for identification to achieve contactless identification from a distance robust to physical appearances. However, an important aspect of gait identification through wearables and image-based systems alike is accurate identification when limited information is available for example, when only a fraction of the whole gait cycle or only a part of the subject's body is visible. In this paper, we present a gait identification technique based on temporal and descriptive statistic parameters of different gait phases as the features and we investigate the performance of using only single gait phases for the identification task using a minimum number of sensors. Gait data were collected from 60 individuals through pelvis and foot sensors. Six different machine learning algorithms were used for identification. It was shown that it is possible to achieve high accuracy of over 95.5% by monitoring a single phase of the whole gait cycle through only a single sensor. It was also shown that the proposed methodology could be used to achieve 100% identification accuracy when the whole gait cycle was monitored through pelvis and foot sensors combined. The ANN was found to be more robust to less number of data features compared to SVM and was concluded as the best machine algorithm for the purpose.

Gait-based Frailty Assessment using Image Representation of IMU Signals and Deep CNN.

Frailty is a common and critical condition in elderly adults, which may lead to further deterioration of health. However, difficulties and complexities exist in traditional frailty assessments based on activity-related questionnaires. These can be overcome by monitoring the effects of frailty on the gait. In this paper, it is shown that by encoding gait signals as images, deep learning-based models can be utilized for the classification of gait type. Two deep learning models (a) SS-CNN, based on single stride input images, and (b) MS-CNN, based on 3 consecutive strides were proposed. It was shown that MS-CNN performs best with an accuracy of 85.1%, while SS-CNN achieved an accuracy of 77.3%. This is because MS-CNN can observe more features corresponding to stride-to-stride variations which is one of the key symptoms of frailty. Gait signals were encoded as images using STFT, CWT, and GAF. While the MS-CNN model using GAF images achieved the best overall accuracy and precision, CWT has a slightly better recall. This study demonstrates how image encoded gait data can be used to exploit the full potential of deep learning CNN models for the assessment of frailty.

Vision-Based Gait Events Detection Using Deep Convolutional Neural Networks.

Accurate gait events detection from the video would be a challenging problem. However, most vision-based methods for gait event detection highly rely on gait features that are estimated using gait silhouettes and human pose information for accurate gait data acquisition. This paper presented an accurate, multi-view approach with deep convolutional neural networks for efficient and practical gait event detection without requiring additional gait feature engineering. Especially, we aimed to detect gait events from frontal views as well as lateral views. We conducted the experiments with four different deep CNN models on our own dataset that includes three different walking actions from 11 healthy participants. Models took 9 subsequence frames stacking together as inputs, while outputs of models were probability vectors of gait events: toe-off and heel-strike for each frame. The deep CNN models trained only with video frames enabled to detect gait events with 93% or higher accuracy while the user is walking straight and walking around on both frontal and lateral views.

Vision-based human joint angular velocity estimation during squat and walking on a treadmill actions.

Elderly health monitoring, rehabilitation training, and sport supervision could benefit from continuous assessment of joint angle, and angular velocity to identify the joint movement patterns. However, most of the measurement systems are designed based on special kinematic sensors to estimate angular velocities. The study aims to measure the lower limb joint angular velocity based on a 2D vision camera system during squat and walking on treadmill action using deep convolution neural network (CNN) architecture. Experiments were conducted on 12 healthy adults, and six digital cameras were used to capture the videos of the participant actions in lateral and frontal view. The normalized cross-correlation (Ccnorm) analysis was performed to obtain a degree of symmetry of the ground truth and estimated angular velocity waveform patterns. Mean Ccnorm for angular velocity estimation by deep CNN model has higher than 0.90 in walking on the treadmill and 0.89 in squat action. Furthermore, joint-wise angular velocities at the hip, knee, and ankle joints were observed and compared. The proposed system gets higher estimation performance under the lateral view and the frontal view of the camera. This study potentially eliminates the requirement of wearable sensors and proves the applicability of using video-based system to measure joint angular velocities during squat and walking on a treadmill actions.

Development of an fMRI-compatible driving simulator with simultaneous measurement of physiological and kinematic signals: The multi-biosignal measurement system for driving (MMSD).

BACKGROUND: A system that comprehensively analyzes a complex perceptual-motor behavior such as driving, by measuring changes in the central and autonomic nervous systems integrated with measurement of changes in vehicle operation, is lacking. OBJECTIVE: We aimed to develop a functional magnetic resonance imaging (fMRI)-compatible driving simulator to enable simultaneous measurement of physiological, kinematic, and brain activations. METHODS: The system mainly comprises a driving simulator and physiological/kinematic measurement. The driving simulator comprises a steering wheel, an accelerator, a brake pedal, and a virtual-reality optical system. The physiological system comprises a skin-conductance-level and a photoplethysmographic meter. The kinematic system comprises a 3-axis accelerometer and a 2-axis gyroscope attached to the accelerator foot. To evaluate the influence of the MR system on the MMSD, physiological and kinematic signals were measured. RESULTS: The system did not blur or deform the MR image. Moreover, the main magnetic field, the gradient magnetic field, and the RF pulse of the MR system did not introduce noise into the physiological or kinematic signals. CONCLUSION: This system can enable a comprehensive evaluation of cognitively complex behaviors such as driving, by quantitatively measuring and analyzing concurrent brain activity, autonomic nervous system activity, and human movement during simulated driving.