Cortical activations associated with spatial remapping of finger touch using EEG.

The spatial coding of tactile information is functionally essential for touch-based shape perception and motor control. However, the spatiotemporal dynamics of how tactile information is remapped from the somatotopic reference frame in the primary somatosensory cortex to the spatiotopic reference frame remains unclear. This study investigated how hand position in space or posture influences cortical somatosensory processing. Twenty-two healthy subjects received electrical stimulation to the right thumb (D1) or little finger (D5) in three position conditions: palm down on right side of the body (baseline), hand crossing the body midline (effect of position), and palm up (effect of posture). Somatosensory-evoked potentials (SEPs) were recorded using electroencephalography. One early-, two mid-, and two late-latency neurophysiological components were identified for both fingers: P50, P1, N125, P200, and N250. D1 and D5 showed different cortical activation patterns: compared with baseline, the crossing condition showed significant clustering at P1 for D1, and at P50 and N125 for D5; the change in posture showed a significant cluster at N125 for D5. Clusters predominated at centro-parietal electrodes. These results suggest that tactile remapping of fingers after electrical stimulation occurs around 100-125 ms in the parietal cortex.

Study on the Design and Performance of a Glove Based on the FBG Array for Hand Posture Sensing.

This study introduces a new wearable fiber-optic sensor glove. The glove utilizes a flexible material, polydimethylsiloxane (PDMS), and a silicone tube to encapsulate fiber Bragg gratings (FBGs). It is employed to enable the self-perception of hand posture, gesture recognition, and the prediction of grasping objects. The investigation employs the Support Vector Machine (SVM) approach for predicting grasping objects. The proposed fiber-optic sensor glove can concurrently monitor the motion of 14 hand joints comprising 5 metacarpophalangeal joints (MCP), 5 proximal interphalangeal joints (PIP), and 4 distal interphalangeal joints (DIP). To expand the measurement range of the sensors, a sinusoidal layout incorporates the FBG array into the glove. The experimental results indicate that the wearable sensing glove can track finger flexion within a range of 0° to 100°, with a modest minimum measurement error (Error) of 0.176° and a minimum standard deviation (SD) of 0.685°. Notably, the glove accurately detects hand gestures in real-time and even forecasts grasping actions. The fiber-optic smart glove technology proposed herein holds promising potential for industrial applications, including object grasping, 3D displays via virtual reality, and human-computer interaction.

Hand Posture and Force Estimation Using Surface Electromyography and an Artificial Neural Network.

OBJECTIVE: The purpose of this study was to develop an approach to predict hand posture (pinch versus grip) and grasp force using forearm surface electromyography (sEMG) and artificial neural networks (ANNs) during tasks that varied repetition rate and duty cycle. BACKGROUND: Prior studies have used electromyography with machine learning models to predict grip force but relatively few studies have assessed whether both hand posture and force can be predicted, particularly at varying levels of duty cycle and repetition rate. METHOD: Fourteen individuals participated in this experiment. sEMG data for five forearm muscles and force output data were collected. Calibration data (25, 50, 75, 100% of maximum voluntary contraction (MVC)) were used to train ANN models to predict hand posture (pinch versus grip) and force magnitude while performing tasks that varied load, repetition rate, and duty cycle. RESULTS: Across all participants, overall hand posture prediction accuracy was 79% (0.79 ± .08), whereas overall hand force prediction accuracy was 73% (0.73 ± .09). Accuracy ranged between 0.65 and 0.93 based on varying repetition rate and duty cycle. CONCLUSION: Hand posture and force prediction were possible using sEMG and ANNs, though there were important differences in the accuracy of predictions based on task characteristics including duty cycle and repetition rate. APPLICATION: The results of this study could be applied to the development of a dosimeter used for distal upper extremity biomechanical exposure measurement, risk assessment, job (re)design, and return to work programs.

Behavioral and Physiological Evidence of a favored Hand Posture in the Body Representation for Action.

Motor planning and execution require a representational map of our body. Since the body can assume different postures, it is not known how it is represented in this map. Moreover, is the generation of the motor command favored by some body configurations? We investigated the existence of a centrally favored posture of the hand for action, in search of physiological and behavioral advantages due to central motor processing. We tested two opposite hand pinch grips, equally difficult and commonly used: forearm pronated, thumb-down, index-up pinch against the same grip performed with thumb-up. The former revealed faster movement onset, sign of faster neural computation, and faster target reaching. It induced increased corticospinal excitability, independently on pre-stimulus tonic muscle contraction. Remarkably, motor excitability also increased when thumb-down pinch was only observed, imagined, or prepared, actually keeping the hand at rest. Motor advantages were independent of any concurrent modulation due to somatosensory input, as shown by testing afferent inhibition. Results provide strong behavioral and physiological evidence for a preferred hand posture favoring brain motor control, independently by somatosensory processing. This suggests the existence of a baseline postural representation that may serve as an a priori spatial reference for body-space interaction.

sEMG-Based Hand Posture Recognition and Visual Feedback Training for the Forearm Amputee.

sEMG-based gesture recognition is useful for human-computer interactions, especially for technology supporting rehabilitation training and the control of electric prostheses. However, high variability in the sEMG signals of untrained users degrades the performance of gesture recognition algorithms. In this study, the hand posture recognition algorithm and radar plot-based visual feedback training were developed using multichannel sEMG sensors. Ten healthy adults and one bilateral forearm amputee participated by repeating twelve hand postures ten times. The visual feedback training was performed for two days and five days in healthy adults and a forearm amputee, respectively. Artificial neural network classifiers were trained with two types of feature vectors: a single feature vector and a combination of feature vectors. The classification accuracy of the forearm amputee increased significantly after three days of hand posture training. These results indicate that the visual feedback training efficiently improved the performance of sEMG-based hand posture recognition by reducing variability in the sEMG signal. Furthermore, a bilateral forearm amputee was able to participate in the rehabilitation training by using a radar plot, and the radar plot-based visual feedback training would help the amputees to control various electric prostheses.

Hand Pose Recognition Using Parallel Multi Stream CNN.

Recently, several computer applications provided operating mode through pointing fingers, waving hands, and with body movement instead of a mouse, keyboard, audio, or touch input such as sign language recognition, robot control, games, appliances control, and smart surveillance. With the increase of hand-pose-based applications, new challenges in this domain have also emerged. Support vector machines and neural networks have been extensively used in this domain using conventional RGB data, which are not very effective for adequate performance. Recently, depth data have become popular due to better understating of posture attributes. In this study, a multiple parallel stream 2D CNN (two-dimensional convolution neural network) model is proposed to recognize the hand postures. The proposed model comprises multiple steps and layers to detect hand poses from image maps obtained from depth data. The hyper parameters of the proposed model are tuned through experimental analysis. Three publicly available benchmark datasets: Kaggle, First Person, and Dexter, are used independently to train and test the proposed approach. The accuracy of the proposed method is 99.99%, 99.48%, and 98% using the Kaggle hand posture dataset, First Person hand posture dataset, and Dexter dataset, respectively. Further, the results obtained for F1 and AUC scores are also near-optimal. Comparative analysis with state-of-the-art shows that the proposed model outperforms the previous methods.

sEMG-Based Hand Posture Recognition Considering Electrode Shift, Feature Vectors, and Posture Groups.

Surface electromyography (sEMG)-based gesture recognition systems provide the intuitive and accurate recognition of various gestures in human-computer interaction. In this study, an sEMG-based hand posture recognition algorithm was developed, considering three main problems: electrode shift, feature vectors, and posture groups. The sEMG signal was measured using an armband sensor with the electrode shift. An artificial neural network classifier was trained using 21 feature vectors for seven different posture groups. The inter-session and inter-feature Pearson correlation coefficients (PCCs) were calculated. The results indicate that the classification performance improved with the number of training sessions of the electrode shift. The number of sessions necessary for efficient training was four, and the feature vectors with a high inter-session PCC (r > 0.7) exhibited high classification accuracy. Similarities between postures in a posture group decreased the classification accuracy. Our results indicate that the classification accuracy could be improved with the addition of more electrode shift training sessions and that the PCC is useful for selecting the feature vector. Furthermore, hand posture selection was as important as feature vector selection. These findings will help in optimizing the sEMG-based pattern recognition algorithm more easily and quickly.

Recognition of Fingerspelling Sequences in Polish Sign Language Using Point Clouds Obtained from Depth Images.

The paper presents a method for recognizing sequences of static letters of the Polish finger alphabet using the point cloud descriptors: viewpoint feature histogram, eigenvalues-based descriptors, ensemble of shape functions, and global radius-based surface descriptor. Each sequence is understood as quick highly coarticulated motions, and the classification is performed by networks of hidden Markov models trained by transitions between postures corresponding to particular letters. Three kinds of the left-to-right Markov models of the transitions, two networks of the transition models-independent and dependent on a dictionary-as well as various combinations of point cloud descriptors are examined on a publicly available dataset of 4200 executions (registered as depth map sequences) prepared by the authors. The hand shape representation proposed in our method can also be applied for recognition of hand postures in single frames. We confirmed this using a known, challenging American finger alphabet dataset with about 60,000 depth images.

Visual and proprioceptive feedback differently modulate the spatial representation of number and time in children.

There has been compelling evidence favoring the idea that human adults similarly represent number and time along a horizontal mental number line (MNL) and mental time line (MTL), respectively. Yet, analogies drawn between the MNL and MTL have been challenged by recent studies suggesting that adults' representations of number and time arise from different spatial frames of reference; whereas the MNL relies on both hand-centered and object-centered coordinates, the MTL appears to be exclusively anchored on object-centered coordinates. To directly test this possibility, here we explored the extent to which visual feedback and proprioceptive feedback affect children's performance in a Number Comparison task (Experiment 1) and a Time Comparison task (Experiment 2), in which participants needed to associate a lateralized key with numerical and temporal words, respectively. Children (5- and 6-year-olds) performed the task with their hands either uncrossed or crossed over the body midline (i.e., manipulation of proprioceptive feedback) and with either visual control over their hands allowed or precluded under blindfolds (i.e., manipulation of visual feedback). Results showed that children were facilitated in associating smaller/larger numbers with the left/right side of the external space, but only when hands were uncrossed and visual feedback was available. On the contrary, blindfolding and crossing their hands over the midline did not affect spatial time mapping, with 6-year-olds showing facilitation in associating words referring to the past/future with the left/right side of the external space irrespective of visual and proprioceptive feedback. This same effect was also present in 5-year-olds despite their difficulty in performing the Time Comparison task. Together, these findings show, for the first time, that-just like adults-young children (a) map temporal events onto space in a rightward direction as they do for numbers and (b) anchor their spatial representation of time and numbers to different spatial frames of reference.

American Sign Language Alphabet Recognition Using a Neuromorphic Sensor and an Artificial Neural Network.

This paper reports the design and analysis of an American Sign Language (ASL) alphabet translation system implemented in hardware using a Field-Programmable Gate Array. The system process consists of three stages, the first being the communication with the neuromorphic camera (also called Dynamic Vision Sensor, DVS) sensor using the Universal Serial Bus protocol. The feature extraction of the events generated by the DVS is the second part of the process, consisting of a presentation of the digital image processing algorithms developed in software, which aim to reduce redundant information and prepare the data for the third stage. The last stage of the system process is the classification of the ASL alphabet, achieved with a single artificial neural network implemented in digital hardware for higher speed. The overall result is the development of a classification system using the ASL signs contour, fully implemented in a reconfigurable device. The experimental results consist of a comparative analysis of the recognition rate among the alphabet signs using the neuromorphic camera in order to prove the proper operation of the digital image processing algorithms. In the experiments performed with 720 samples of 24 signs, a recognition accuracy of 79.58% was obtained.

A bi-articular model for scapular-humeral rhythm reconstruction through data from wearable sensors.

BACKGROUND: Patient-specific performance assessment of arm movements in daily life activities is fundamental for neurological rehabilitation therapy. In most applications, the shoulder movement is simplified through a socket-ball joint, neglecting the movement of the scapular-thoracic complex. This may lead to significant errors. We propose an innovative bi-articular model of the human shoulder for estimating the position of the hand in relation to the sternum. The model takes into account both the scapular-toracic and gleno-humeral movements and their ratio governed by the scapular-humeral rhythm, fusing the information of inertial and textile-based strain sensors. METHOD: To feed the reconstruction algorithm based on the bi-articular model, an ad-hoc sensing shirt was developed. The shirt was equipped with two inertial measurement units (IMUs) and an integrated textile strain sensor. We built the bi-articular model starting from the data obtained in two planar movements (arm abduction and flexion in the sagittal plane) and analysing the error between the reference data - measured through an optical reference system - and the socket-ball approximation of the shoulder. The 3D model was developed by extending the behaviour of the kinematic chain revealed in the planar trajectories through a parameter identification that takes into account the body structure of the subject. RESULT: The bi-articular model was evaluated in five subjects in comparison with the optical reference system. The errors were computed in terms of distance between the reference position of the trochlea (end-effector) and the correspondent model estimation. The introduced method remarkably improved the estimation of the position of the trochlea (and consequently the estimation of the hand position during reaching activities) reducing position errors from 11.5 cm to 1.8 cm. CONCLUSION: Thanks to the developed bi-articular model, we demonstrated a reliable estimation of the upper arm kinematics with a minimal sensing system suitable for daily life monitoring of recovery.

Real-Time Hand Posture Recognition for Human-Robot Interaction Tasks.

In this work, we present a multiclass hand posture classifier useful for human-robot interaction tasks. The proposed system is based exclusively on visual sensors, and it achieves a real-time performance, whilst detecting and recognizing an alphabet of four hand postures. The proposed approach is based on the real-time deformable detector, a boosting trained classifier. We describe a methodology to design the ensemble of real-time deformable detectors (one for each hand posture that can be classified). Given the lack of standard procedures for performance evaluation, we also propose the use of full image evaluation for this purpose. Such an evaluation methodology provides us with a more realistic estimation of the performance of the method. We have measured the performance of the proposed system and compared it to the one obtained by using only the sampled window approach. We present detailed results of such tests using a benchmark dataset. Our results show that the system can operate in real time at about a 10-fps frame rate.

Functional connectivity associated with hand shape generation: Imitating novel hand postures and pantomiming tool grips challenge different nodes of a shared neural network.

Clinical research suggests that imitating meaningless hand postures and pantomiming tool-related hand shapes rely on different neuroanatomical substrates. We investigated the BOLD responses to different tasks of hand posture generation in 14 right handed volunteers. Conjunction and contrast analyses were applied to select regions that were either common or sensitive to imitation and/or pantomime tasks. The selection included bilateral areas of medial and lateral extrastriate cortex, superior and inferior regions of the lateral and medial parietal lobe, primary motor and somatosensory cortex, and left dorsolateral prefrontal, and ventral and dorsal premotor cortices. Functional connectivity analysis revealed that during hand shape generation the BOLD-response of every region correlated significantly with every other area regardless of the hand posture task performed, although some regions were more involved in some hand postures tasks than others. Based on between-task differences in functional connectivity we predict that imitation of novel hand postures would suffer most from left superior parietal disruption and that pantomiming hand postures for tools would be impaired following left frontal damage, whereas both tasks would be sensitive to inferior parietal dysfunction. We also unveiled that posterior temporal cortex is committed to pantomiming tool grips, but that the involvement of this region to the execution of hand postures in general appears limited. We conclude that the generation of hand postures is subserved by a highly interconnected task-general neural network. Depending on task requirements some nodes/connections will be more engaged than others and these task-sensitive findings are in general agreement with recent lesion studies.

Opposing influences of global and local stimulus-hand proximity on crosstalk interference in dual tasks.

In contrast to traditional dualistic views of cognition, visual stimulus processing appears not independent of bodily factors such as hand positioning. For example, reduced crosstalk between two temporally overlapping tasks has been observed when the hands are moved into the attentional window alongside their respective stimuli (i.e., establishing global stimulus-hand proximity). This result indicates that hand-specific attentional processing enhancements support a more serial rather than parallel processing of the two tasks. To further elucidate the nature of these processing modulations and their effect on multitasking performance, the present study consisted of three interrelated crosstalk experiments. Experiment 1 manipulated global stimulus-hand proximity and stimulus-effect proximity orthogonally, with results demonstrating that hand proximity rather than effect proximity drives the crosstalk reduction. Experiment 2 manipulated the physical distance between both hands (i.e., varying local stimulus-hand proximity), with results showing weak evidence of increased crosstalk when both hands are close to each other. Experiment 3 tested opposing influences of global and local stimulus-hand proximity as observed in Experiment 1 and 2 rigorously within one experiment, by employing an orthogonal manipulation of these two proximity measures. Again, we observed slightly increased crosstalk for hands close to each other (replicating Experiment 2); however, in contrast to Experiment 1, the effect of global stimulus-hand proximity on the observed crosstalk was not significant this time. Taken together, the experiments support the notion of hand-specific modulations of perception-action coupling, which can either lead to more or less interference in multitasking, depending on the exact arrangement of hands and stimuli.

Mechanisms and neuroanatomy of response selection in tool and non-tool action tasks: Evidence from left-hemisphere stroke.

The ability to select between potential actions is central to the complex process of tool use. After left hemisphere stroke, individuals with limb apraxia make more hand action errors when gesturing the use of tools with conflicting hand actions for grasping-to-move and use (e.g., screwdriver) relative to tools that are grasped-to-move and used with the same hand action (e.g., hammer). Prior research indicates that this grasp-use interference effect is driven by abnormalities in the competitive action selection process. The goal of this project was to determine whether common mechanisms and neural substrates support the competitive selection of task-appropriate responses in both tool and non-tool domains. If so, the grasp-use interference effect in a tool use gesturing task should be correlated with response interference effects in the classic Eriksen flanker and Simon tasks, and at least partly overlapping neural regions should subserve the 3 tasks. Sixty-four left hemisphere stroke survivors (33 with apraxia) participated in the tool- and non-tool interference tasks and underwent T1 anatomical MRI. There were robust grasp-use interference effects (grasp-use conflict test) and response interference effects (Eriksen flanker and Simon tasks), but these effects were not correlated. Lesion-symptom mapping analyses showed that lesions to the left inferior parietal lobule, ventral premotor cortex, and insula were associated with grasp-use interference. Lesions to the left inferior parietal lobule, postcentral gyrus, insula, caudate, and putamen were associated with response interference in the Eriksen flanker task. Lesions to the left caudate and putamen were also associated with response interference in the Simon task. Our results suggest that the selection of hand posture for tool use is mediated by distinct cognitive mechanisms and partly distinct neuroanatomic substrates from those mapping a stimulus to an appropriate motor response in non-tool domains.

Analysis of length of finger segments with different hand postures to enhance glove design.

It is important to understand how the hand and fingers elongate and contract with hand posture for optimally fitting and comfortable gloves. Nevertheless, the measurement and analysis of the finger segments for glove designs remain largely neglected. Here, the length and proportion of the finger segments when splayed and during gripping, and between the dorsal and palm sides of 30 participants are 3D scanned and analysed. The full digit lengths change by 7.6-11.9% with hand posture, but the finger segment changes are not proportional. The ratios of the fingertip to distal interphalangeal joint/full digit, and fingertip to the proximal interphalangeal joint/full digit, are important variables. The results are validated with 10 more subjects based on ratings of a ready-to-wear sports glove. Inaccurate proportioning of the finger regions causes shifting which results in displacement and discomfort. This research contributes to glove pattern engineering, with a focus on the finger segments.

Hand Gesture Recognition Based on Computer Vision: A Review of Techniques.

Hand gestures are a form of nonverbal communication that can be used in several fields such as communication between deaf-mute people, robot control, human-computer interaction (HCI), home automation and medical applications. Research papers based on hand gestures have adopted many different techniques, including those based on instrumented sensor technology and computer vision. In other words, the hand sign can be classified under many headings, such as posture and gesture, as well as dynamic and static, or a hybrid of the two. This paper focuses on a review of the literature on hand gesture techniques and introduces their merits and limitations under different circumstances. In addition, it tabulates the performance of these methods, focusing on computer vision techniques that deal with the similarity and difference points, technique of hand segmentation used, classification algorithms and drawbacks, number and types of gestures, dataset used, detection range (distance) and type of camera used. This paper is a thorough general overview of hand gesture methods with a brief discussion of some possible applications.

Using tools effectively despite defective hand posture: A single-case study.

Apraxia, a cognitive disorder of motor control, can severely impair transitive actions (object-related) and may lead to action errors (e.g., rubbing a hammer on a nail instead of pounding it) and hand posture errors (e.g., grasping a tool in a wrong way). Here, we report a rare observation of a left-handed patient, left-lateralized for language, who developed a severe apraxia following a right brain lesion. Interestingly the patient showed a significant number of hand posture errors, while she perfectly demonstrated the actual use of tools. This case stressed the predictions made by the current theories of tool use. According to the manipulation-based approach, the hand posture errors should be associated with an impaired manipulation knowledge. According to the reasoning-based approach, the absence of action errors should be associated with spared mechanical knowledge. Moreover, to better understand the neurocognitive origins of the deficit observed in VF, we examined several potential brain lateralization of praxis functions. We initiated a systematic examination of VF's performance in several contexts of tool use allowing us to investigate which kinds of tool-use representations were potentially impaired in VF. Our investigation did not reveal any major deficit of manipulation knowledge. This can hardly account for the high frequency of hand posture errors, contrary to the predictions of the manipulation-based approach. In contrast, in line with the reasoning-based approach, mechanical knowledge was spared and can explain the absence of action errors. We also found that VF probably had a bilateral organization of praxis functions, but irrespective of the possible brain lateralization considered, none of which established a satisfactory association between manipulation knowledge and hand posture errors. Taken together, the manipulation-based approach fails to explain fully the deficit presented by this patient and should lead us to consider alternative explanations.

Effect on hand kinematics when using assistive devices during activities of daily living.

Assistive devices (ADs) are products intended to overcome the difficulties produced by the reduction in mobility and grip strength entailed by ageing and different pathologies. Nevertheless, there is little information about the effect that the use of these devices produces on hand kinematics. Thus, the aim of this work is to quantify this effect through the comparison of kinematic parameters (mean posture, ROM, median velocity and peak velocity) while performing activities of daily living (ADL) using normal products and ADs. Twelve healthy right-handed subjects performed 11 ADL with normal products and with 17 ADs wearing an instrumented glove on their right hand, 16 joint angles being recorded. ADs significantly affected hand kinematics, although the joints affected differed according to the AD. Furthermore, some pattern effects were identified depending on the characteristics of the handle of the ADs, namely, handle thickening, addition of a handle to products that initially did not have one, extension of existing handles or addition of handles to apply higher torques. An overview of the effects of these design characteristics on hand kinematics is presented as a basis for the selection of the most suitable AD depending on the patient's impairments.

A letter on a hand: Rotating together or separately?

The motor system plays a role in some object mental rotation tasks, and researchers have reported that people may use a strategy of motor simulation to mentally rotate objects. In this study, we used images of a hand with a letter printed on the palm to directly determine whether a hand image can be automatically rotated during the deliberate mental rotation of an object and whether the hand and object are rotated in the same trajectory. A total of 41 participants were shown the stimuli and asked to decide whether the letters, which were upright or tilted at specific degrees, were normal or mirrored. The hand images in the background showed either a left or a right hand in the palm view, with fingers pointing upwards, medial, downwards, or lateral. Reaction times and error rates were measured to determine the speed and accuracy of mental rotation. A complex interaction between the hand posture and letter orientation revealed that the hand image was mentally rotated automatically, together with the deliberate mental rotation of the letter. The biomechanical constraints of the hand also influenced reaction times, suggesting the involvement of the motor system in the concomitant mental rotation of the hand image. Consistent with the motor simulation theory, the participants seemed to imagine the hand carrying the object in its movement. These behavioural data support the motor simulation theory and elucidate specific processes of mental rotation that have not been addressed by neuroimaging studies.