Improving the recognition of grips and movements of the hand using myoelectric signals.

BACKGROUND: People want to live independently, but too often disabilities or advanced age robs them of the ability to do the necessary activities of daily living (ADLs). Finding relationships between electromyograms measured in the arm and movements of the hand and wrist needed to perform ADLs can help address performance deficits and be exploited in designing myoelectrical control systems for prosthetics and computer interfaces. METHODS: This paper reports on several machine learning techniques employed to discover the electromyogram patterns present when performing 24 typical fine motor functional activities of the hand and the rest position used to accomplish ADLs. Accelerometer data is collected from the hand as an aid in identifying the start and end of movements and to help in labeling the signal data. Techniques employed include classification of 100 ms individual signal instances, using a symbolic representation to approximate signal streams, and the use of nearest neighbor in two specific situations: creation of an affinity matrix to model learning instances and classify based on multiple adjacent signal values, and using Dynamic Time Warping (DTW) as a distance measure to classify entire activity segments. RESULTS: Results show the patterns can be learned to an accuracy of 76.64 % for a 25 class problem when classifying 100 ms instances, 83.63 % with the affinity matrix approach with symbolic representation, and 85.22 % with Dynamic Time Warping. Classification errors are, with a few exceptions, concentrated within particular grip action groups. CONCLUSION: The findings reported here support the view that grips and movements of the hand can be distinguished by combining electrical and mechanical properties of the task to an accuracy of 85.22 % for a 25 class problem. Converting the signals to a symbolic representation and classifying based on larger portions of the signal stream improve classification accuracy. This is both clinically useful and opens the way for an approach to help simulate hand functional activities. With improvements it may also prove useful in real time control applications.

The feasibility of using haptic devices to engage people with chronic traumatic brain injury in virtual 3D functional tasks.

BACKGROUND: The primary aim of this study was to assess the level of engagement in computer-based simulations of functional tasks, using a haptic device for people with chronic traumatic brain injury. The objectives were to design functional tasks using force feedback device and determine if it could measure motor performance improvement. METHODS: A prospective crosssectional study was performed in a biomedical research facility. The testing environment consisted of a single, interactive, stylus-driven computer session navigating virtual scenes in 3D space. Subjects had a haptic training session (TRAIN) and then had three chances to perform each virtual task: (i) remove tools from a workbench (TOOL), (ii) compose 3 letter words (SPELL), (iii) manipulate utensils to prepare a sandwich (SAND), and (iv) tool use (TUSE). Main Outcome Measures included self-report of engagement in the activities, improved performance on simulated tasks and observer estimate as measured by time to completion or number of words completed from baseline, correlations among performance measures and self-reports of boredom, neuropsychological symptom inventory (NSI), and The Purdue Peg Motor Test (PPT). RESULTS: Participants were 19 adults from the community with a 1 year history of non-penetrating traumatic brain injury (TBI) and were able to use computers. Seven had mild, 3 moderate and 9 severe TBIs. Mean score on the Boredom Proneness Scale (BPS): 107 (normal range 81-117); mean NSI:32; mean PPT 54 (normal range for assembly line workers >67). Responses to intervention: 3 (15%)subjects did not repeat all three trials of the tasks; 100% reported they were highly engaged in the interactions; 6 (30%) reported they had a high level of frustration with the tasks, but completed them with short breaks. Performance measures: Comparison of baseline to post training: TOOL time decreased by (mean) 60 sec; SPELL increased by 2.7 words; TUSE time decreased by (mean) 68 sec; and SAND time decreased by (mean) 72 sec. PPT correlated with TOOL (r=-0.65, p=0.016) and TUSE time (r=-0.6, p=0.014). SPELL correlated with Boredom score (r=0.41, p=0.08) and NSI (r=-.49, p=0.05). CONCLUSION: People with chronic TBI of various ages and severity report being engaged in using haptic devices that interact with 3D virtual environments. Haptic devices are able to capture objective data that provide useful information about fine motor and cognitive performance.

A Comparison of Approaches for Segmenting the Reaching and Targeting Motion Primitives in Functional Upper Extremity Reaching Tasks.

There is growing interest in the kinematic analysis of human functional upper extremity movement (FUEM) for applications such as health monitoring and rehabilitation. Deconstructing functional movements into activities, actions, and primitives is a necessary procedure for many of these kinematic analyses. Advances in machine learning have led to progress in human activity and action recognition. However, their utility for analyzing the FUEM primitives of reaching and targeting during reach-to-grasp and reach-to-point tasks remains limited. Domain experts use a variety of methods for segmenting the reaching and targeting motion primitives, such as kinematic thresholds, with no consensus on what methods are best to use. Additionally, current studies are small enough that segmentation results can be manually inspected for correctness. As interest in FUEM kinematic analysis expands, such as in the clinic, the amount of data needing segmentation will likely exceed the capacity of existing segmentation workflows used in research laboratories, requiring new methods and workflows for making segmentation less cumbersome. This paper investigates five reaching and targeting motion primitive segmentation methods in two different domains (haptics simulation and real world) and how to evaluate these methods. This work finds that most of the segmentation methods evaluated perform reasonably well given current limitations in our ability to evaluate segmentation results. Furthermore, we propose a method to automatically identify potentially incorrect segmentation results for further review by the human evaluator. Clinical impact: This work supports efforts to automate aspects of processing upper extremity kinematic data used to evaluate reaching and grasping, which will be necessary for more widespread usage in clinical settings.

Computer-assisted approaches for measuring, segmenting, and analyzing functional upper extremity movement: a narrative review of the current state, limitations, and future directions.

The analysis of functional upper extremity (UE) movement kinematics has implications across domains such as rehabilitation and evaluating job-related skills. Using movement kinematics to quantify movement quality and skill is a promising area of research but is currently not being used widely due to issues associated with cost and the need for further methodological validation. Recent developments by computationally-oriented research communities have resulted in potentially useful methods for evaluating UE function that may make kinematic analyses easier to perform, generally more accessible, and provide more objective information about movement quality, the importance of which has been highlighted during the COVID-19 pandemic. This narrative review provides an interdisciplinary perspective on the current state of computer-assisted methods for analyzing UE kinematics with a specific focus on how to make kinematic analyses more accessible to domain experts. We find that a variety of methods exist to more easily measure and segment functional UE movement, with a subset of those methods being validated for specific applications. Future directions include developing more robust methods for measurement and segmentation, validating these methods in conjunction with proposed kinematic outcome measures, and studying how to integrate kinematic analyses into domain expert workflows in a way that improves outcomes.

A Novel Computer Vision Approach to Kinematic Analysis of Handwriting with Implications for Assessing Neurodegenerative Diseases.

Fine motor movement is a demonstrated biomarker for many health conditions that are especially difficult to diagnose early and require sensitivity to change in order to monitor over time. This is particularly relevant for neurodegenerative diseases (NDs), including Parkinson's Disease (PD) and Alzheimer's Disease (AD), which are associated with early changes in handwriting and fine motor skills. Kinematic analysis of handwriting is an emerging method for assessing fine motor movement ability, with data typically collected by digitizing tablets; however, these are often expensive, unfamiliar to patients, and are limited in the scope of collectible data. In this paper, we present a vision-based system for the capture and analysis of handwriting kinematics using a commodity camera and RGB video. We achieve writing position estimation within 0.5 mm and speed and acceleration errors of less than 1.1%. We further demonstrate that this data collection process can be part of an ND screening system with a developed ensemble classifier achieving 74% classification accuracy of Parkinson's Disease patients with vision-based data. Overall, we demonstrate that this approach is an accurate, accessible, and informative alternative to digitizing tablets and with further validation has potential uses in early disease screening and long-term monitoring.Clinical relevance- This work establishes a more accessible alternative to digitizing tablets for extracting handwriting kinematic data through processing of RGB video data captured by commodity cameras, such as those in smartphones, with computer vision and machine learning. The collected data has potential for use in analysis to objectively and quantitatively differentiate between healthy individuals and patients with NDs, including AD and PD, as well as other diseases with biomarkers displayed in fine motor movement. The developed system has potential applications including providing widespread screening systems for NDs in low-income areas and resource-poor health systems, as well as an accessible form of disease long-term monitoring through telemedicine.

Characterizing Functional Upper Extremity Movement in Haptic Virtual Environments.

Haptic virtual environments have been used to assess cognitive and fine motor function. For tasks performed in physical environments, upper extremity movement is usually separated into reaching and object manipulation phases using fixed velocity thresholds. However, these thresholds can result in premature segmentation due to additional trajectory adjustments common in virtual environments. In this work, we address the issues of premature segmentation and the lack of a measure to characterize the spatial distribution of a trajectory while targeting an object. We propose a combined relative distance and velocity segmentation procedure and use principal component analysis (PCA) to capture the spatial distribution of the participant's targeting phase. Synthetic data and 3D motion data from twenty healthy adults were used to evaluate the methods with positive results. We found that these methods quantify motor skill improvement of healthy participants performing repeated trials of a haptic virtual environment task.

Motion estimation using Statistical Learning Theory.

This paper describes a novel application of Statistical Learning Theory (SLT) to single motion estimation and tracking. The problem of motion estimation can be related to statistical model selection, where the goal is to select one (correct) motion model from several possible motion models, given finite noisy samples. SLT, also known as Vapnik-Chervonenkis (VC), theory provides analytic generalization bounds for model selection, which have been used successfully for practical model selection. This paper describes a successful application of an SLT-based model selection approach to the challenging problem of estimating optimal motion models from small data sets of image measurements (flow). We present results of experiments on both synthetic and real image sequences for motion interpolation and extrapolation; these results demonstrate the feasibility and strength of our approach. Our experimental results show that for motion estimation applications, SLT-based model selection compares favorably against alternative model selection methods, such as the Akaike's fpe, Schwartz' criterion (sc), Generalized Cross-Validation (gcv), and Shibata's Model Selector (sms). The paper also shows how to address the aperture problem using SLT-based model selection for penalized linear (ridge regression) formulation.

Motion based markerless gait analysis using standard events of gait and ensemble Kalman filtering.

We present a novel approach to gait analysis using ensemble Kalman filtering which permits markerless determination of segmental movement. We use image flow analysis to reliably compute temporal and kinematic measures including the translational velocity of the torso and rotational velocities of the lower leg segments. Detecting the instances where velocity changes direction also determines the standard events of a gait cycle (double-support, toe-off, mid-swing and heel-strike). In order to determine the kinematics of lower limbs, we model the synergies between the lower limb motions (thigh-shank, shank-foot) by building a nonlinear dynamical system using CMUs 3D motion capture database. This information is fed into the ensemble Kalman Filter framework to estimate the unobserved limb (upper leg and foot) motion from the measured lower leg rotational velocity. Our approach does not require calibrated cameras or special markers to capture movement. We have tested our method on different gait sequences collected from the sagttal plane and presented the estimated kinematics overlaid on the original image frames. We have also validated our approach by manually labeling the videos and comparing our results against them.

Markerless identification of key events in gait cycle using image flow.

Gait analysis has been an interesting area of research for several decades. In this paper, we propose image-flow-based methods to compute the motion and velocities of different body segments automatically, using a single inexpensive video camera. We then identify and extract different events of the gait cycle (double-support, mid-swing, toe-off and heel-strike) from video images. Experiments were conducted in which four walking subjects were captured from the sagittal plane. Automatic segmentation was performed to isolate the moving body from the background. The head excursion and the shank motion were then computed to identify the key frames corresponding to different events in the gait cycle. Our approach does not require calibrated cameras or special markers to capture movement. We have also compared our method with the Optotrak 3D motion capture system and found our results in good agreement with the Optotrak results. The development of our method has potential use in the markerless and unencumbered video capture of human locomotion. Monitoring gait in homes and communities provides a useful application for the aged and the disabled. Our method could potentially be used as an assessment tool to determine gait symmetry or to establish the normal gait pattern of an individual.