A Hierarchical Bayesian Model for Cyber-Human Assessment of Movement in Upper Extremity Stroke Rehabilitation.

The evidence-based quantification of the relation between changes in movement quality and functionality can assist clinicians in achieving more effective structuring or adapting of therapy. In this paper, clinicians rated task, segment, and composite movement feature performance for 478 videos of stroke survivors executing upper extremity therapy tasks. We used the clinician ratings to develop a Hierarchical Bayesian Model (HBM) with task, segment, and composite layers for computing the statistical relation of movement quality changes to function. The model was enhanced through a detailed correlation graph ( ∆HBM ) that links computationally extracted kinematics with clinician-rated composite features for different task-segment combinations. Utilizing the weights and correlation graphs, we finally derive reverse cascading probabilities of the proposed HBM from kinematics to composite features, segments, and tasks. In a test involving 98 cases where clinician ratings differed, the HBM resolved 95% of these discrepancies. The model effectively aligned kinematic data with specific task-segment combinations in over 90% of cases. Once the HBM is expanded and refined through additional data it can be used for the automated calculation of statistical relations between changes in kinematics and performance of functional tasks and the generation of therapy assessment recommendations for clinicians. While our work primarily focuses on the upper extremities of stroke survivors, the HBM can be adapted to many other neurorehabilitation contexts.

Inclusion of Clinicians in the Development and Evaluation of Clinical Artificial Intelligence Tools: A Systematic Literature Review.

The application of machine learning (ML) and artificial intelligence (AI) in healthcare domains has received much attention in recent years, yet significant questions remain about how these new tools integrate into frontline user workflow, and how their design will impact implementation. Lack of acceptance among clinicians is a major barrier to the translation of healthcare innovations into clinical practice. In this systematic review, we examine when and how clinicians are consulted about their needs and desires for clinical AI tools. Forty-five articles met criteria for inclusion, of which 24 were considered design studies. The design studies used a variety of methods to solicit and gather user feedback, with interviews, surveys, and user evaluations. Our findings show that tool designers consult clinicians at various but inconsistent points during the design process, and most typically at later stages in the design cycle (82%, 19/24 design studies). We also observed a smaller amount of studies adopting a human-centered approach and where clinician input was solicited throughout the design process (22%, 5/24). A third (15/45) of all studies reported on clinician trust in clinical AI algorithms and tools. The surveyed articles did not universally report validation against the "gold standard" of clinical expertise or provide detailed descriptions of the algorithms or computational methods used in their work. To realize the full potential of AI tools within healthcare settings, our review suggests there are opportunities to more thoroughly integrate frontline users' needs and feedback in the design process.

Capturing Upper Body Kinematics and Localization with Low-Cost Sensors for Rehabilitation Applications.

For upper extremity rehabilitation, quantitative measurements of a person's capabilities during activities of daily living could provide useful information for therapists, including in telemedicine scenarios. Specifically, measurements of a person's upper body kinematics could give information about which arm motions or movement features are in need of additional therapy, and their location within the home could give context to these motions. To that end, we present a new algorithm for identifying a person's location in a region of interest based on a Bluetooth received signal strength (RSS) and present an experimental evaluation of this and a different Bluetooth RSS-based localization algorithm via fingerprinting. We further present algorithms for and experimental results of inferring the complete upper body kinematics based on three standalone inertial measurement unit (IMU) sensors mounted on the wrists and pelvis. Our experimental results for localization find the target location with a mean square error of 1.78 m. Our kinematics reconstruction algorithms gave lower errors with the pelvis sensor mounted on the person's back and with individual calibrations for each test. With three standalone IMUs, the mean angular error for all of the upper body segment orientations was close to 21 degrees, and the estimated elbow and shoulder angles had mean errors of less than 4 degrees.

Smart Skin: Vision-Based Soft Pressure Sensing System for In-Home Hand Rehabilitation.

We introduce a novel in-home hand rehabilitation system for monitoring hand motions and assessing grip forces of stroke patients. The overall system is composed of a sensing device and a computer vision system. The sensing device is a lightweight cylindrical object for easy grip and manipulation, which is covered by a passive sensing layer called "Smart Skin." The Smart Skin is fabricated using soft silicone elastomer, which contains embedded microchannels partially filled with colored fluid. When the Smart Skin is compressed by grip forces, the colored fluid rises and fills in the top surface display area. Then, the computer vision system captures the image of the display area through a red-green-blue camera, detects the length change of the liquid through image processing, and eventually maps the liquid length to the calibrated force for estimating the gripping force. The passive sensing mechanism of the proposed Smart Skin device works in conjunction with a single camera setup, making the system simple and easy to use, while also requiring minimum maintenance effort. Our system, on one hand, aims to support home-based rehabilitation therapy with minimal or no supervision by recording the training process and the force data, which can be automatically conveyed to physical therapists. In contrast, the therapists can also remotely instruct the patients with their training prescriptions through online videos. This study first describes the design, fabrication, and calibration of the Smart Skin, and the algorithm for image processing, and then presents experimental results from the integrated system. The Smart Skin prototype shows a relatively linear relationship between the applied force and the length change of the liquid in the range of 0-35 N. The computer vision system shows the estimation error <4% and a relatively high stability in estimation under different hand motions.

Automated Movement Assessment in Stroke Rehabilitation.

We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).