Enhancing automated lower limb rehabilitation exercise task recognition through multi-sensor data fusion in tele-rehabilitation.

BACKGROUND: Tele-rehabilitation is the provision of physiotherapy services to individuals in their own homes. Activity recognition plays a crucial role in the realm of automatic tele-rehabilitation. By assessing patient movements, identifying exercises, and providing feedback, these platforms can offer insightful information to clinicians, thereby facilitating an improved plan of care. This study introduces a novel deep learning approach aimed at identifying lower limb rehabilitation exercises. This is achieved through the integration of depth data and pressure heatmaps. We hypothesized that combining pressure heatmaps and depth data could improve the model's overall performance. METHODS: In this study, depth videos and body pressure data from an accessible online dataset were used. This dataset comprises data from 30 healthy individuals performing 7 lower limb rehabilitation exercises. To accomplish the classification task, three deep learning models were developed, all based on an established 3D-CNN architecture. The models were designed to classify the depth videos, sequences of pressure data frames, and combination of depth videos and pressure frames. The models' performance was assessed through leave-one-subject-out and leave-multiple-subjects-out cross-validation methods. Performance metrics, including accuracy, precision, recall, and F1 score, were reported for each model. RESULTS: Our findings indicated that the model trained on the fusion of depth and pressure data showed the highest and most stable performance when compared with models using individual modality inputs. This model could effectively identify the exercises with an accuracy of 95.71%, precision of 95.83%, recall of 95.71%, and an F1 score of 95.74%. CONCLUSION: Our results highlight the impact of data fusion for accurately classifying lower limb rehabilitation exercises. We showed that our model could capture different aspects of exercise movements using the visual and weight distribution data from the depth camera and pressure mat, respectively. This integration of data provides a better representation of exercise patterns, leading to higher classification performance. Notably, our results indicate the potential application of this model in automatic tele-rehabilitation platforms.

Technological advances in lower-limb tele-rehabilitation: A review of literature.

Tele-rehabilitation is a healthcare practice that leverages technology to provide rehabilitation services remotely to individuals in their own homes or other locations. With advancements in remote monitoring and Artificial Intelligence, automatic tele-rehabilitation systems that can measure joint angles, recognize exercises, and provide feedback based on movement analysis are being developed. Such platforms can offer valuable information to clinicians for improved care planning. However, with various methods and sensors being used, understanding their pros, cons, and performance is important. This paper reviews and compares the performance of recent vision-based, wearable, and pressure-sensing technologies used in lower limb tele-rehabilitation systems over the past 10 years (from 2014 to 2023). We selected studies that were published in English and focused on joint angle estimation, activity recognition, and exercise assessment. Vision-based approaches were the most common, accounting for 42% of studies. Wearable technology followed at approximately 37%, and pressure-sensing technology appeared in 21% of studies. Identified gaps include a lack of uniformity in reported performance metrics and evaluation methods, a need for cross-subject validation, inadequate testing with patients and older adults, restricted sets of exercises evaluated, and a scarcity of comprehensive datasets on lower limb exercises, especially those involving movements while lying down.