Automated Hand Prehension Assessment From Egocentric Video After Spinal Cord Injury.

Hand function assessments in a clinical setting are critical for upper limb rehabilitation after spinal cord injury (SCI) but may not accurately reflect performance in an individual's home environment. When paired with computer vision models, egocentric videos from wearable cameras provide an opportunity for remote hand function assessment during real activities of daily living (ADLs). This study demonstrates the use of computer vision models to predict clinical hand function assessment scores from egocentric video. SlowFast, MViT, and MaskFeat models were trained and validated on a custom SCI dataset, which contained a variety of ADLs carried out in a simulated home environment. The dataset was annotated with clinical hand function assessment scores using an adapted scale applicable to a wide range of object interactions. An accuracy of 0.551±0.139, mean absolute error (MAE) of 0.517±0.184, and F1 score of 0.547±0.151 was achieved on the 5-class classification task. An accuracy of 0.724±0.135, MAE of 0.290±0.140, and F1 score of 0.733±0.144 was achieved on a consolidated 3-class classification task. This novel approach, for the first time, demonstrates the prediction of hand function assessment scores from egocentric video after SCI.

Visual background information modulates motor contagions in humans.

Motor contagions refer to implicit effects induced by the observation of actions made by others on one's own actions. A plethora of studies conducted over the last two decades have demonstrated that both observed and predicted actions can induce various kinds of motor contagions in a human observer. However, motor contagions have always been investigated with regard to different features of an observed action, and it remains unclear whether the background environment in which an observed action takes place modulates motor contagions as well. Here, we investigated participant movements in an empirical hand steering task during which the participants were required to move a cursor through a visual channel after being presented with videos of an actor performing the same task. We manipulated the congruency between the actions shown in the video and the background channels and examined whether and how they affected the participants' own movements. We observed a clear interaction between the observed action and its background. The movement time of the participants' actions tended to increase or decrease depending on whether they observed a faster or slower movement, respectively, and these changes were amplified if the background was not congruent with the action contained within it. These results suggest that background information can modulate motor contagions in humans.

How bipedalism shapes humans' actions with hand tools.

The task for an embodied cognitive understanding of humans' actions with tools is to elucidate how the human body, as a whole, supports the perception of affordances and dexterous action with objects in relation to other objects. Here, we focus on the relationship between humans' actions with handheld tools and bipedal posture. Posture plays a pivotal role in shaping animals' perception and action dynamics. While humans stand and locomote bipedally, other primates predominantly employ quadrupedal postures and locomotion, relying on both hands and feet to support the body. Drawing upon evidence from evolutionary biology, developmental psychology and performance studies, we elucidate the influence of bipedalism on our actions with objects and on our proficiency in using tools. We use the metaphor of cascades to capture the dynamic, nonlinear transformations in morphology and behaviour associated with posture and the use of tools across evolutionary and developmental timescales. Recent work illustrates the promise of multifractal cascade analysis to reveal nonlinear, cross-scale interactions across the entire body in real-time, supporting the perception of affordances for actions with tools. Cascade analysis enriches our comprehension of real-time performance and facilitates exploration of the relationships among whole-body coordination, individual development, and evolutionary processes.This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

A visual representation of the hand in the resting somatomotor regions of the human brain.

Hand visibility affects motor control, perception, and attention, as visual information is integrated into an internal model of somatomotor control. Spontaneous brain activity, i.e., at rest, in the absence of an active task, is correlated among somatomotor regions that are jointly activated during motor tasks. Recent studies suggest that spontaneous activity patterns not only replay task activation patterns but also maintain a model of the body's and environment's statistical regularities (priors), which may be used to predict upcoming behavior. Here, we test whether spontaneous activity in the human somatomotor cortex as measured using fMRI is modulated by visual stimuli that display hands vs. non-hand stimuli and by the use/action they represent. A multivariate pattern analysis was performed to examine the similarity between spontaneous activity patterns and task-evoked patterns to the presentation of natural hands, robot hands, gloves, or control stimuli (food). In the left somatomotor cortex, we observed a stronger (multivoxel) spatial correlation between resting state activity and natural hand picture patterns compared to other stimuli. No task-rest similarity was found in the visual cortex. Spontaneous activity patterns in somatomotor brain regions code for the visual representation of human hands and their use.

Wearable high-density EMG sleeve for complex hand gesture classification and continuous joint angle estimation.

High-density electromyography (HD-EMG) can provide a natural interface to enhance human-computer interaction (HCI). This study aims to demonstrate the capability of a novel HD-EMG forearm sleeve equipped with up to 150 electrodes to capture high-resolution muscle activity, decode complex hand gestures, and estimate continuous hand position via joint angle predictions. Ten able-bodied participants performed 37 hand movements and grasps while EMG was recorded using the HD-EMG sleeve. Simultaneously, an 18-sensor motion capture glove calculated 23 joint angles from the hand and fingers across all movements for training regression models. For classifying across the 37 gestures, our decoding algorithm was able to differentiate between sequential movements with 97.3 ± 0.3 % accuracy calculated on a 100 ms bin-by-bin basis. In a separate mixed dataset consisting of 19 movements randomly interspersed, decoding performance achieved an average bin-wise accuracy of 92.8 ± 0.8 % . When evaluating decoders for use in real-time scenarios, we found that decoders can reliably decode both movements and movement transitions, achieving an average accuracy of 93.3 ± 0.9 % on the sequential set and 88.5 ± 0.9 % on the mixed set. Furthermore, we estimated continuous joint angles from the EMG sleeve data, achieving a R 2 of 0.884 ± 0.003 in the sequential set and 0.750 ± 0.008 in the mixed set. Median absolute error (MAE) was kept below 10° across all joints, with a grand average MAE of 1.8 ± 0 . 04 ∘ and 3.4 ± 0 . 07 ∘ for the sequential and mixed datasets, respectively. We also assessed two algorithm modifications to address specific challenges for EMG-driven HCI applications. To minimize decoder latency, we used a method that accounts for reaction time by dynamically shifting cue labels in time. To reduce training requirements, we show that pretraining models with historical data provided an increase in decoding performance compared with models that were not pretrained when reducing the in-session training data to only one attempt of each movement. The HD-EMG sleeve, combined with sophisticated machine learning algorithms, can be a powerful tool for hand gesture recognition and joint angle estimation. This technology holds significant promise for applications in HCI, such as prosthetics, assistive technology, rehabilitation, and human-robot collaboration.

Effectiveness of mixed reality-based rehabilitation on hands and fingers by individual finger-movement tracking in patients with stroke.

BACKGROUND: Mixed reality (MR) is helpful in hand training for patients with stroke, allowing them to fully submerge in a virtual space while interacting with real objects. The recognition of individual finger movements is required for MR rehabilitation. This study aimed to assess the effectiveness of updated MR-board 2, adding finger training for patients with stroke. METHODS: Twenty-one participants with hemiplegic stroke (10 with left hemiplegia and 11 with right hemiplegia; nine female patients; 56.7 ± 14.2 years of age; and onset of stroke 32.7 ± 34.8 months) participated in this study. MR-board 2 comprised a board plate, a depth camera, plastic-shaped objects, a monitor, a palm-worn camera, and seven gamified training programs. All participants performed 20 self-training sessions involving 30-min training using MR-board 2. The outcome measurements for upper extremity function were the Fugl-Meyer assessment (FMA) upper extremity score, repeated number of finger flexion and extension (Repeat-FE), the thumb opposition test (TOT), Box and Block Test score (BBT), Wolf Motor Function Test score (WMFT), and Stroke Impact Scale (SIS). One-way repeated measures analysis of variance and the post hoc test were applied for the measurements. MR-board 2 recorded the fingers' active range of motion (AROM) and Dunnett's test was used for pairwise comparisons. RESULTS: Except for the FMA-proximal score (p = 0.617) and TOT (p = 0.005), other FMA scores, BBT score, Repeat-FE, WMFT score, and SIS stroke recovery improved significantly (p < 0.001) during MR-board 2 training and were maintained until follow-up. All AROM values of the finger joints changed significantly during training (p < 0.001). CONCLUSIONS: MR-board 2 self-training, which includes natural interactions between humans and computers using a tangible user interface and real-time tracking of the fingers, improved upper limb function across impairment, activity, and participation. MR-board 2 could be used as a self-training tool for patients with stroke, improving their quality of life. TRIAL REGISTRATION NUMBER: This study was registered with the Clinical Research Information Service (CRIS: KCT0004167).

Tremor Estimation and Removal in Robot-Assisted Surgery Using Improved Enhanced Band-Limited Multiple Fourier Linear Combiner.

BACKGROUND: During a robot-assisted minimally invasive surgery, hand tremors in a surgeon's manipulation of the master manipulator can cause vibrations of the slave surgical instruments. METHODS: This letter addresses this problem by proposing an improved Enhanced Band-Limited Multiple Linear Fourier Combiner (E-BMFLC) algorithm for filtering the physiological tremor signals of a surgeon's hand. The proposed method uses the amplitude of the input signal to adapt the learning rate and a dense division of the combiner bands for the higher amplitude bands of the tremor signals. RESULTS: By using the proposed improved E-BMFLC algorithm, the compensation accuracy can be improved by 4.5%-8.9%, as well as a spatial position error of less than 1 mm. CONCLUSION: The results show that among all filtering methods, the improved E-BMFLC filtering method has the highest number of successful experiments and the lowest experimental time.

Human tactile sensing and sensorimotor mechanism: from afferent tactile signals to efferent motor control.

In tactile sensing, decoding the journey from afferent tactile signals to efferent motor commands is a significant challenge primarily due to the difficulty in capturing population-level afferent nerve signals during active touch. This study integrates a finite element hand model with a neural dynamic model by using microneurography data to predict neural responses based on contact biomechanics and membrane transduction dynamics. This research focuses specifically on tactile sensation and its direct translation into motor actions. Evaluations of muscle synergy during in -vivo experiments revealed transduction functions linking tactile signals and muscle activation. These functions suggest similar sensorimotor strategies for grasping influenced by object size and weight. The decoded transduction mechanism was validated by restoring human-like sensorimotor performance on a tendon-driven biomimetic hand. This research advances our understanding of translating tactile sensation into motor actions, offering valuable insights into prosthetic design, robotics, and the development of next-generation prosthetics with neuromorphic tactile feedback.

The implications of healthcare professionals wearing jewelry on patient care biosafety: observational insights and experimental approaches.

The use of jewelry among healthcare professionals poses a risk of cross contamination due to potential bacterial accumulation and spread. Through a mixed-method design, this study first analyzed the implications of healthcare professionals wearing jewelry on patient care biosafety as well as on the residual bacterial load of hands and rings after hand hygiene. Firstly, an observational prevalence study to verify whether nursing professionals wear personal accessories during healthcare assistance was carried out. Second, an experimental design involving intentional contamination and hygiene of the hands, with and without a ring, was conducted. The bacterial load of both hands and rings was measured by counting colony forming units. The observational study showed that nursing workers frequently wear jewelry during healthcare assistance. Nonetheless, the experimental study did not indicate differences in bacterial contamination between hands with and without a ring, despite the hand hygiene procedure applied. In conclusion, many nursing workers wear jewelry in the workplace. Although hands with and without a ring exhibited similar microbial load, rings appeared as a potential source of bacterial contamination, reinforcing the need to remove jewelry during working hours. Hand hygiene using alcohol, or soap and water significantly decreased the bacterial load on the participants' hands, with handwashing proving to be the most efficient method for removing intentional contamination.

Validation of the Kinematic Assessment Protocol Used in the Technology-Supported Neurorehabilitation System, Rehabilitation Technologies for Hand and Arm (R3THA™), in Children and Teenagers with Cerebral Palsy.

This study evaluates the R3THA™ assessment protocol (R3THA-AP™), a technology-supported testing module for personalized rehabilitation in children with cerebral palsy (CP). It focuses on the reliability and validity of the R3THA-AP in assessing hand and arm function, by comparing kinematic assessments with standard clinical assessments. Conducted during a 4-week summer camp, the study assessed the functional and impairment levels of children with CP aged 3-18. The findings suggest that R3THA is more reliable for children aged 8 and older, indicating that age significantly influences the protocol's effectiveness. The results also showed that the R3THA-AP's kinematic measurements of hand and wrist movements are positively correlated with the Box and Blocks Test Index (BBTI), reflecting hand function and dexterity. Additionally, the R3THA-AP's accuracy metrics for hand and wrist activities align with the Melbourne Assessment 2's Range of Motion (MA2-ROM) scores, suggesting a meaningful relationship between R3THA-AP data and clinical assessments of motor skills. However, no significant correlations were observed between the R3THA-AP and MA2's accuracy and dexterity measurements, indicating areas for further research. These findings validate the R3THA-AP's utility in assessing motor abilities in CP patients, supporting its integration into clinical practice.

Tool-sensed object information effectively supports vision for multisensory grasping.

Tools enable humans to extend their sensing abilities beyond the natural limits of their hands, allowing them to sense objects as if they were using their hands directly. The similarities between direct hand interactions with objects (hand-based sensing) and the ability to extend sensory information processing beyond the hand (tool-mediated sensing) entail the existence of comparable processes for integrating tool- and hand-sensed information with vision, raising the question of whether tools support vision in bimanual object manipulations. Here, we investigated participants' performance while grasping objects either held with a tool or with their hand and compared these conditions with visually guided grasping (Experiment 1). By measuring reaction time, peak velocity, and peak of grip aperture, we found that actions were initiated earlier and performed with a smaller peak grip aperture when the object was seen and held with the tool or the contralateral hand compared to when it was only seen. Thus, tool-mediated sensing effectively supports vision in multisensory grasping and, even more intriguingly, resembles hand-based sensing. We excluded that results were due to the force exerted on the tool's handle (Experiment 2). Additionally, as for hand-based sensing, we found evidence that the tool supports vision by mainly providing object positional information (Experiment 3). Thus, integrating the tool-sensed position of the object with vision is sufficient to promote a multisensory advantage in grasping. Our findings indicate that multisensory integration mechanisms significantly improve grasping actions and fine-tune contralateral hand movements even when object information is only indirectly sensed through a tool. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

An unconstrained palmprint region of interest extraction method based on lightweight networks.

Accurately extracting the Region of Interest (ROI) of a palm print was crucial for subsequent palm print recognition. However, under unconstrained environmental conditions, the user's palm posture and angle, as well as the background and lighting of the environment, were not controlled, making the extraction of the ROI of palm print a major challenge. In existing research methods, traditional ROI extraction methods relied on image segmentation and were difficult to apply to multiple datasets simultaneously under the aforementioned interference. However, deep learning-based methods typically did not consider the computational cost of the model and were difficult to apply to embedded devices. This article proposed a palm print ROI extraction method based on lightweight networks. Firstly, the YOLOv5-lite network was used to detect and preliminarily locate the palm, in order to eliminate most of the interference from complex backgrounds. Then, an improved UNet was used for keypoints detection. This network model reduced the number of parameters compared to the original UNet model, improved network performance, and accelerated network convergence. The output of this model combined Gaussian heatmap regression and direct regression and proposed a joint loss function based on JS loss and L2 loss for supervision. During the experiment, a mixed database consisting of 5 databases was used to meet the needs of practical applications. The results showed that the proposed method achieved an accuracy of 98.3% on the database, with an average detection time of only 28ms on the GPU, which was superior to other mainstream lightweight networks, and the model size was only 831k. In the open-set test, with a success rate of 93.4%, an average detection time of 5.95ms on the GPU, it was far ahead of the latest palm print ROI extraction algorithm and could be applied in practice.

A compact solution for vibrotactile proprioceptive feedback of wrist rotation and hand aperture.

BACKGROUND: Closing the control loop between users and their prostheses by providing artificial sensory feedback is a fundamental step toward the full restoration of lost sensory-motor functions. METHODS: We propose a novel approach to provide artificial proprioceptive feedback about two degrees of freedom using a single array of 8 vibration motors (compact solution). The performance afforded by the novel method during an online closed-loop control task was compared to that achieved using the conventional approach, in which the same information was conveyed using two arrays of 8 and 4 vibromotors (one array per degree of freedom), respectively. The new method employed Gaussian interpolation to modulate the intensity profile across a single array of vibration motors (compact feedback) to convey wrist rotation and hand aperture by adjusting the mean and standard deviation of the Gaussian, respectively. Ten able-bodied participants and four transradial amputees performed a target achievement control test by utilizing pattern recognition with compact and conventional vibrotactile feedback to control the Hannes prosthetic hand (test conditions). A second group of ten able-bodied participants performed the same experiment in control conditions with visual and auditory feedback as well as no-feedback. RESULTS: Conventional and compact approaches resulted in similar positioning accuracy, time and path efficiency, and total trial time. The comparison with control condition revealed that vibrational feedback was intuitive and useful, but also underlined the power of incidental feedback sources. Notably, amputee participants achieved similar performance to that of able-bodied participants. CONCLUSIONS: The study therefore shows that the novel feedback strategy conveys useful information about prosthesis movements while reducing the number of motors without compromising performance. This is an important step toward the full integration of such an interface into a prosthesis socket for clinical use.

Closed-Loop Force Control by Biorealistic Hand Prosthesis With Visual and Tactile Sensory Feedback.

The ability of a novel biorealistic hand prosthesis for grasp force control reveals improved neural compatibility between the human-prosthetic interaction. The primary purpose here was to validate a virtual training platform for amputee subjects and evaluate the respective roles of visual and tactile information in fundamental force control tasks. We developed a digital twin of tendon-driven prosthetic hand in the MuJoCo environment. Biorealistic controllers emulated a pair of antagonistic muscles controlling the index finger of the virtual hand by surface electromyographic (sEMG) signals from amputees' residual forearm muscles. Grasp force information was transmitted to amputees through evoked tactile sensation (ETS) feedback. Six forearm amputees participated in force tracking and holding tasks under different feedback conditions or using their intact hands. Test results showed that visual feedback played a predominant role than ETS feedback in force tracking and holding tasks. However, in the absence of visual feedback during the force holding task, ETS feedback significantly enhanced motor performance compared to feedforward control alone. Thus, ETS feedback still supplied reliable sensory information to facilitate amputee's ability of stable grasp force control. The effects of tactile and visual feedback on force control were subject-specific when both types of feedback were provided simultaneously. Amputees were able to integrate visual and tactile information to the biorealistic controllers and achieve a good sensorimotor performance in grasp force regulation. The virtual platform may provide a training paradigm for amputees to adapt the biorealistic hand controller and ETS feedback optimally.

Characterization of bilateral reaching development using augmented reality games.

Bilateral coordination is commonly impaired in neurodevelopmental conditions including cerebral palsy, developmental coordination disorder, and autism spectrum disorder. However, we lack objective clinical assessments that can quantify bilateral coordination in a clinically feasible manner and determine age-based norms to identify impairments. The objective of this study was to use augmented reality and computer vision to characterize bilateral reaching abilities in typically developing children. Typically developing children (n = 133) ages 6-17 years completed symmetric and asymmetric bilateral reaching tasks in an augmented reality game environment. We analyzed the number of target pairs they could reach in 50 s as well as the time lag between their hands reaching the targets. We found that performance on both tasks developed in parallel, with development slowing but not plateauing after age 12. Children performed better on the symmetric task than asymmetric, both in targets reached and with shorter hand lags. Variability between children in hand lag decreased with age. We also found gender differences with females outperforming males, which were most pronounced in the 10-11 year olds. Overall, this study demonstrates parallel development through childhood and adolescence of symmetric and asymmetric reaching abilities. Furthermore, it demonstrates the ability to quantify bilateral coordination using computer vision and augmented reality, which can be applied to assess clinical populations.

Scalp acupuncture guidance for identifying the optimal site for transcranial electrical stimulation of the hand.

Transcranial electrical stimulation (tES) often targets the EEG-guided C3/C4 area that may not accurately represent M1 for hand muscles. This study aimed to determine if the neuroanatomy-based scalp acupuncture-guided site (AC) was a more effective spot than the C3 site for neuromodulation. Fifteen healthy subjects received one 20-minute session of high-definition transcranial alternating current stimulation (HD-tACS) intervention (20 Hz at 2 mA) at the AC or C3 sites randomly with a 1-week washout period. Subjects performed ball-squeezing exercises with the dominant hand during the HD-tACS intervention. The AC site was indiscernible from the finger flexor hotspot detected by TMS. At the baseline, the MEP amplitude from finger flexors was greater with less variability at the AC site than at the C3 site. HD-tACS intervention at the AC site significantly increased the MEP amplitude. However, no significant changes were observed after tACS was applied to the C3 site. Our results provide evidence that HD-tACS at the AC site produces better neuromodulation effects on the flexor digitorum superficialis (FDS) muscle compared to the C3 site. The AC localization approach can be used for future tES studies.

Ontogeny and sexual dimorphism in the human hands through a 2D geometric morphometrics approach.

OBJECTIVES: This study aims to conduct a thorough characterization of hand morphology. Employing a 2D geometric morphometric approach, we scrutinize individual fingers and the palm, delineating the ontogenetic trajectories for each biological sex and investigating the alterations that take place at various stages of human development. MATERIALS AND METHODS: A set of thirty-two 2D anatomical landmarks were assessed in a sex-balanced sample of human hands (F = 275, M = 250 males), spanning all stages of human development. Following Procrustes registration, the data on size and shape for individual fingers and the palm were examined for each biological sex and age group. Regression analysis was utilized to quantify ontogenetic trajectories for each biological sex. RESULTS: The findings suggest a gradual escalation in sexual dimorphism throughout human development, with statistically noteworthy distinctions becoming apparent in size starting at the age of 3, and in shape from the age of 7 onwards. Additionally, our analyses uncover a distinctive sigmoid pattern between sexes, indicating that biological male hands exhibit a sturdier build compared to biological female hands from early childhood onward. CONCLUSIONS: In conclusion, this study enriches our insights into sexual dimorphism in human hands, stressing the importance of considering both size and shape across different ontogenetic stages. These findings not only expand our understanding of human biological variation but also lay the foundation for future interdisciplinary research in diverse scientific domains.