Hand Grasp Classification in Egocentric Video After Cervical Spinal Cord Injury.

OBJECTIVE: The hand function of individuals with spinal cord injury (SCI) plays a crucial role in their independence and quality of life. Wearable cameras provide an opportunity to analyze hand function in non-clinical environments. Summarizing the video data and documenting dominant hand grasps and their usage frequency would allow clinicians to quickly and precisely analyze hand function. METHOD: We introduce a new hierarchical model to summarize the grasping strategies of individuals with SCI at home. The first level classifies hand-object interaction using hand-object contact estimation. We developed a new deep model in the second level by incorporating hand postures and hand-object contact points using contextual information. RESULTS: In the first hierarchical level, a mean of 86% ±1.0% was achieved among 17 participants. At the grasp classification level, the mean average accuracy was 66.2 ±12.9%. The grasp classifier's performance was highly dependent on the participants, with accuracy varying from 41% to 78%. The highest grasp classification accuracy was obtained for the model with smoothed grasp classification, using a ResNet50 backbone architecture for the contextual head and a temporal pose head. DISCUSSION: We introduce a novel algorithm that, for the first time, enables clinicians to analyze the quantity and type of hand movements in individuals with spinal cord injury at home. The algorithm can find applications in other research fields, including robotics, and most neurological diseases that affect hand function, notably, stroke and Parkinson's.

Grasp Analysis in the Home Environment as a Measure of Hand Function After Cervical Spinal Cord Injury.

BACKGROUND: Following a spinal cord injury, regaining hand function is a top priority. Current hand assessments are conducted in clinics, which may not fully represent real-world hand function. Grasp strategies used in the home environment are an important consideration when examining the impact of rehabilitation interventions. OBJECTIVE: The main objective of this study is to investigate the relationship between grasp use at home and clinical scores. METHOD: We used a previously collected dataset in which 21 individuals with spinal cord injuries (SCI) recorded egocentric video while performing activities of daily living in their homes. We manually annotated 4432 hand-object interactions into power, precision, intermediate, and non-prehensile grasps. We examined the distributions of grasp types used and their relationships with clinical assessments. RESULTS: Moderate to strong correlations were obtained between reliance on power grasp and the Spinal Cord Independence Measure III (SCIM; P < .05), the upper extremity motor score (UEMS; P < .01), and the Graded Redefined Assessment of Strength Sensibility and Prehension (GRASSP) Prehension (P < .01) and Strength (P < .01). Negative correlations were observed between the proportion of non-prehensile grasping and SCIM (P < .05), UEMS (P < .05), and GRASSP Prehension (P < .01) and Strength (P < .01). CONCLUSION: The types of grasp types used in naturalistic activities at home are related to upper limb impairment after cervical SCI. This study provides the first direct demonstration of the importance of hand grasp analysis in the home environment.

Measuring Hand Use in the Home after Cervical Spinal Cord Injury Using Egocentric Video.

Egocentric video has recently emerged as a potential solution for monitoring hand function in individuals living with tetraplegia in the community, especially for its ability to detect functional use in the home environment. The aim of this study was to develop and validate a wearable vision-based system for measuring hand use in the home among individuals living with tetraplegia. Several deep learning algorithms for detecting functional hand-object interactions were developed and compared. The most accurate algorithm was used to extract measures of hand function from 65 h of unscripted video recorded at home by 20 participants with tetraplegia. These measures were: the percentage of interaction time over total recording time (Perc); the average duration of individual interactions (Dur); and the number of interactions per hour (Num). To demonstrate the clinical validity of the technology, egocentric measures were correlated with validated clinical assessments of hand function and independence (Graded Redefined Assessment of Strength, Sensibility and Prehension [GRASSP], Upper Extremity Motor Score [UEMS], and Spinal Cord Independent Measure [SCIM]). Hand-object interactions were automatically detected with a median F1-score of 0.80 (0.67-0.87). Our results demonstrated that higher UEMS and better prehension were related to greater time spent interacting, whereas higher SCIM and better hand sensation resulted in a higher number of interactions performed during the egocentric video recordings. For the first time, measures of hand function automatically estimated in an unconstrained environment in individuals with tetraplegia have been validated against internationally accepted measures of hand function. Future work will necessitate a formal evaluation of the reliability and responsiveness of the egocentric-based performance measures for hand use.

Tenodesis Grasp Detection in Egocentric Video.

OBJECTIVE: Cervical spinal cord injury (cSCI) can impair motor function in the upper limbs. Video from wearable cameras (egocentric video) has the potential to provide monitoring of rehabilitation outcomes at home, but methods for automated analysis of this data are needed. Wrist flexion and extension are essential elements to track grasping strategies after cSCI, as they may reflect the use of the tenodesis grasp, a common compensatory strategy. However, there is no established method to evaluate wrist flexion and extension from egocentric video. METHODS: We propose a machine-learning-based approach comprising three steps-hand detection, pose estimation, and arm orientation estimation-to estimate wrist angle data, leading to the detection of tenodesis grasp. RESULTS: The hand detection in conjunction with the pose estimation algorithm correctly located wrist and index finger metacarpophalangeal coordinates in 63% and 76% of 15,319 annotated frames, respectively, extracted from egocentric videos of individuals with cSCI performing activities of daily living in a home simulation laboratory. The arm orientation algorithm had a mean absolute error of 2.76 +/- 0.39 degrees in 12,863 labeled frames. Using these estimates, the presence of a tenodesis grasp was correctly detected in 72% +/- 11% of frames in videos of 6 activities. CONCLUSION: The results provided a clear indication of which participants relied on tenodesis grasp and which did not. SIGNIFICANCE: This paradigm provides the first method that can enable clinicians and researchers to monitor the use of the tenodesis grasp by individuals with cSCI at home, with implications for remote therapeutic guidance.

Towards Clustering Hand Grasps of Individuals with Spinal Cord Injury in Egocentric Video.

Cervical spinal cord injury (cSCI) can cause paralysis and impair hand function. Existing assessments in clinical settings do not reflect an individual's performance in their daily environment. Videos from wearable cameras (egocentric video) provide a novel avenue to analyze hand function in non-clinical settings. Due to the large amounts of video data generated by this approach, automated analysis methods are necessary. We propose to employ an unsupervised learning process to produce a summary of the grasping strategies used in an egocentric video. To this end, an approach was developed consisting of hand detection, pose estimation, and clustering algorithms. The performance of the method was examined with external evaluation indicators and internal evaluation indicators for an uninjured and injured participant, respectively. The results demonstrated that a Gaussian mixture model obtained the highest accuracy in terms of the maximum match, 0.63, and the Rand index, 0.26, for the uninjured participant, and a silhouette score of 0.13 for the injured participant.

A wearable vision-based system for detecting hand-object interactions in individuals with cervical spinal cord injury: First results in the home environment.

Cervical spinal cord injury (cSCI) causes the paralysis of upper and lower limbs and trunk, significantly reducing quality of life and community participation of the affected individuals. The functional use of the upper limbs is the top recovery priority of people with cSCI and wearable vision-based systems have recently been proposed to extract objective outcome measures that reflect hand function in a natural context. However, previous studies were conducted in a controlled environment and may not be indicative of the actual hand use of people with cSCI living in the community. Thus, we propose a deep learning algorithm for automatically detecting hand-object interactions in egocentric videos recorded by participants with cSCI during their daily activities at home. The proposed approach is able to detect hand-object interactions with good accuracy (F1-score up to 0.82), demonstrating the feasibility of this system in uncontrolled situations (e.g., unscripted activities and variable illumination). This result paves the way for the development of an automated tool for measuring hand function in people with cSCI living in the community.

Estimation of global and local complexities of brain networks: A random walks approach.

The complexity of brain activity has been observed at many spatial scales and has been proposed to differentiate between mental states and disorders. Here we introduced a new measure of (global) network complexity, constructed as the sum of the complexities of its nodes (i.e., local complexity). The complexity of each node is obtained by comparing the sample entropy of the time series generated by the movement of a random walker on the network resulting from removing the node and its connections, with the sample entropy of the time series obtained from a regular lattice (ordered state) and a random network (disordered state). We studied the complexity of fMRI-based resting-state networks. We found that positively correlated (pos) networks comprising only the positive functional connections have higher complexity than anticorrelation (neg) networks (comprising the negative connections) and the network consisting of the absolute value of all connections (abs). We also observed a significant correlation between complexity and the strength of functional connectivity in the pos network. Our results suggest that the pos network is related to the information processing in the brain and that functional connectivity studies should analyze pos and neg networks separately instead of the abs network, as is commonly done.

Spatiotemporal Empirical Mode Decomposition of Resting-State fMRI Signals: Application to Global Signal Regression.

Resting-state functional connectivity MRI (rs-fcMRI) is a common method for mapping functional brain networks. However, estimation of these networks is affected by the presence of a common global systemic noise, or global signal (GS). Previous studies have shown that the common preprocessing steps of removing the GS may create spurious correlations between brain regions. In this paper, we decompose fMRI signals into 5 spatial and 3 temporal intrinsic mode functions (SIMF and TIMF, respectively) by means of the empirical mode decomposition (EMD), which is an adaptive data-driven method widely used to analyze non-linear and non-stationary phenomena. For each SIMF, functional connectivity matrices were computed by means of Pearson correlation between TIMFs of different brain areas. Thus, instead of a single connectivity matrix, we obtained 5 × 3 = 15 functional connectivity matrices. Given the high correlation and global efficiency values of the connectivity matrices related to the low spatial maps (SIMF3, SIMF4, and SIMF5), our results suggest that these maps can be considered as spatial global signal masks. Thus, by summing up the first two SIMFs extracted from the fMRI signals, we have automatically excluded the GS which is now voxel-specific. We compared the performance of our method with the conventional GS regression and to the results when the GS was not removed. While the correlation pattern identified by the other methods suffers from a low level of precision in identifying the correct brain network connectivity, our approach demonstrated expected connectivity patterns for the default mode network and task-positive network.

Design of optimal nonlinear network controllers for Alzheimer's disease.

Brain stimulation can modulate the activity of neural circuits impaired by Alzheimer's disease (AD), having promising clinical benefit. However, all individuals with the same condition currently receive identical brain stimulation, with limited theoretical basis for this generic approach. In this study, we introduce a control theory framework for obtaining exogenous signals that revert pathological electroencephalographic activity in AD at a minimal energetic cost, while reflecting patients' biological variability. We used anatomical networks obtained from diffusion magnetic resonance images acquired by the Alzheimer's Disease Neuroimaging Initiative (ADNI) as mediators for the interaction between Duffing oscillators. The nonlinear nature of the brain dynamics is preserved, given that we extend the so-called state-dependent Riccati equation control to reflect the stimulation objective in the high-dimensional neural system. By considering nonlinearities in our model, we identified regions for which control inputs fail to correct abnormal activity. There are changes to the way stimulated regions are ranked in terms of the energetic cost of controlling the entire network, from a linear to a nonlinear approach. We also found that limbic system and basal ganglia structures constitute the top target locations for stimulation in AD. Patients with highly integrated anatomical networks-namely, networks having low average shortest path length, high global efficiency-are the most suitable candidates for the propagation of stimuli and consequent success on the control task. Other diseases associated with alterations in brain dynamics and the self-control mechanisms of the brain can be addressed through our framework.

The effects of sedative music, arousal music, and silence on electrocardiography signals.

INTRODUCTION: Research indicates that music can affect heart rate, blood pressure, and skin conductance. Music can stimulate central emotions in the brain and release biochemical materials that change the physiologic state. We sought to compare changes in the electrical function of the heart in response to music. METHOD: Subjects were asked to listen to 2 types of music, namely, sedative and arousal music, in conjunction with two 30-second periods of complete silence. The experiment was conducted in 4 segments: the first and third parts were silence, and the second and fourth parts were music. First, the response to each type of music was compared with that to the preceding period of silence. Next, the responses to both types of music were compared. Finally, the response to music regardless of the type was compared with that to silence. RESULTS: The amplitude of polarization and depolarization changed in response to different kinds of music. The electrical function of the heart in response to music, irrespective of the music type, differed from that in response to silence. The 2 types of music impacted the electrical function of the heart in different ways: the arousal music influenced T-wave maximum amplitude, whereas no such change was recorded in response to the sedative music. CONCLUSIONS: The bandwidth of the polarization and depolarization of the heart rate and R-wave amplitude increased in response to music by comparison with silence. In addition, the heart did not seem to try to synchronize with music. The mean R-wave amplitude in sedative music is higher than the arousal music, so our heart works differently when different types of music are heard.