An Instrumented Glove for Restoring Sensorimotor Function of the Hand Through Augmented Sensory Feedback.

The loss of sensitivity of the upper limb due to neurological injuries severely limits the ability to manipulate objects, hindering personal independence. Non-invasive augmented sensory feedback techniques are used to promote neural plasticity hence to restore the grasping function. This work presents a wearable device for restoring sensorimotor hand functions based on Discrete Event-driven Sensory Control policy. It consists of an instrumented glove that, relying on piezoelectric sensors, delivers short-lasting vibrotactile stimuli synchronously with the relevant mechanical events (i.e., contact and release) of the manipulation. We first performed a feasibility study on healthy participants (20) that showed overall good performances of the device, with touch-event detection accuracy of 96.2% and a response delay of 22 ms. Later, we pilot tested it on two participants with limited sensorimotor functions. When using the device, they improved their hand motor coordination while performing tests for hand motor coordination assessment (i.e., pick and place test, pick and lift test). In particular, they exhibited more coordinated temporal correlations between grip force and load force profiles and enhanced performances when transferring objects, quantitatively proving the effectiveness of the device.

A highly integrated bionic hand with neural control and feedback for use in daily life.

Restoration of sensorimotor function after amputation has remained challenging because of the lack of human-machine interfaces that provide reliable control, feedback, and attachment. Here, we present the clinical implementation of a transradial neuromusculoskeletal prosthesis-a bionic hand connected directly to the user's nervous and skeletal systems. In one person with unilateral below-elbow amputation, titanium implants were placed intramedullary in the radius and ulna bones, and electromuscular constructs were created surgically by transferring the severed nerves to free muscle grafts. The native muscles, free muscle grafts, and ulnar nerve were implanted with electrodes. Percutaneous extensions from the titanium implants provided direct skeletal attachment and bidirectional communication between the implanted electrodes and a prosthetic hand. Operation of the bionic hand in daily life resulted in improved prosthetic function, reduced postamputation, and increased quality of life. Sensations elicited via direct neural stimulation were consistently perceived on the phantom hand throughout the study. To date, the patient continues using the prosthesis in daily life. The functionality of conventional artificial limbs is hindered by discomfort and limited and unreliable control. Neuromusculoskeletal interfaces can overcome these hurdles and provide the means for the everyday use of a prosthesis with reliable neural control fixated into the skeleton.

Marine Robotics for Deep-Sea Specimen Collection: A Taxonomy of Underwater Manipulative Actions.

In order to develop a gripping system or control strategy that improves scientific sampling procedures, knowledge of the process and the consequent definition of requirements is fundamental. Nevertheless, factors influencing sampling procedures have not been extensively described, and selected strategies mostly depend on pilots' and researchers' experience. We interviewed 17 researchers and remotely operated vehicle (ROV) technical operators, through a formal questionnaire or in-person interviews, to collect evidence of sampling procedures based on their direct field experience. We methodologically analyzed sampling procedures to extract single basic actions (called atomic manipulations). Available equipment, environment and species-specific features strongly influenced the manipulative choices. We identified a list of functional and technical requirements for the development of novel end-effectors for marine sampling. Our results indicate that the unstructured and highly variable deep-sea environment requires a versatile system, capable of robust interactions with hard surfaces such as pushing or scraping, precise tuning of gripping force for tasks such as pulling delicate organisms away from hard and soft substrates, and rigid holding, as well as a mechanism for rapidly switching among external tools.

Marine Robotics for Deep-Sea Specimen Collection: A Systematic Review of Underwater Grippers.

The collection of delicate deep-sea specimens of biological interest with remotely operated vehicle (ROV) industrial grippers and tools is a long and expensive procedure. Industrial grippers were originally designed for heavy manipulation tasks, while sampling specimens requires dexterity and precision. We describe the grippers and tools commonly used in underwater sampling for scientific purposes, systematically review the state of the art of research in underwater gripping technologies, and identify design trends. We discuss the possibility of executing typical manipulations of sampling procedures with commonly used grippers and research prototypes. Our results indicate that commonly used grippers ensure that the basic actions either of gripping or caging are possible, and their functionality is extended by holding proper tools. Moreover, the approach of the research status seems to have changed its focus in recent years: from the demonstration of the validity of a specific technology (actuation, transmission, sensing) for marine applications, to the solution of specific needs of underwater manipulation. Finally, we summarize the environmental and operational requirements that should be considered in the design of an underwater gripper.

A database of high-density surface electromyogram signals comprising 65 isometric hand gestures.

Control of contemporary, multi-joint prosthetic hands is commonly realized by using electromyographic signals from the muscles remaining after amputation at the forearm level. Although this principle is trying to imitate the natural control structure where muscles control the joints of the hand, in practice, myoelectric control provides only basic hand functions to an amputee using a dexterous prosthesis. This study aims to provide an annotated database of high-density surface electromyographic signals to aid the efforts of designing robust and versatile electromyographic control interfaces for prosthetic hands. The electromyographic signals were recorded using 128 channels within two electrode grids positioned on the forearms of 20 able-bodied volunteers. The participants performed 65 different hand gestures in an isometric manner. The hand movements were strictly timed using an automated recording protocol which also synchronously recorded the electromyographic signals and hand joint forces. To assess the quality of the recorded signals several quantitative assessments were performed, such as frequency content analysis, channel crosstalk, and the detection of poor skin-electrode contacts.

Neural feedback strategies to improve grasping coordination in neuromusculoskeletal prostheses.

Conventional prosthetic arms suffer from poor controllability and lack of sensory feedback. Owing to the absence of tactile sensory information, prosthetic users must rely on incidental visual and auditory cues. In this study, we investigated the effect of providing tactile perception on motor coordination during routine grasping and grasping under uncertainty. Three transhumeral amputees were implanted with an osseointegrated percutaneous implant system for direct skeletal attachment and bidirectional communication with implanted neuromuscular electrodes. This neuromusculoskeletal prosthesis is a novel concept of artificial limb replacement that allows to extract control signals from electrodes implanted on viable muscle tissue, and to stimulate severed afferent nerve fibers to provide somatosensory feedback. Subjects received tactile feedback using three biologically inspired stimulation paradigms while performing a pick and lift test. The grasped object was instrumented to record grasping and lifting forces and its weight was either constant or unexpectedly changed in between trials. The results were also compared to the no-feedback control condition. Our findings confirm, in line with the neuroscientific literature, that somatosensory feedback is necessary for motor coordination during grasping. Our results also indicate that feedback is more relevant under uncertainty, and its effectiveness can be influenced by the selected neuromodulation paradigm and arguably also the prior experience of the prosthesis user.

Hand Control With Invasive Feedback Is Not Impaired by Increased Cognitive Load.

Recent experiments have shown that neural stimulation can successfully restore sensory feedback in upper-limb amputees improving their ability to control the prosthesis. However, the potential advantages of invasive sensory feedback with respect to non-invasive solutions have not been yet identified. Our hypothesis was that a difference would appear when the subject cannot focus all the attention to the use of the prosthesis, but some additional activities require his/her cognitive attention, which is a quite common situation in real-life conditions. To verify this hypothesis, we asked a trans-radial amputee, equipped with a bidirectional hand prosthesis, to perform motor tasks also in combination with a cognitive task. Sensory feedback was provided via intraneural (invasive) or electro-tactile (non-invasive) stimulation. We collected also data related to self-confidence. While both approaches were able to significantly improve the motor performance of the subject when no additional cognitive effort was asked, the manual accuracy was not affected by the cognitive task only when intraneural feedback was provided. The highest self-confidence was obtained when intraneural sensory feedback was provided. Our findings show that intraneural sensory feedback is more robust to dual tasks than non-invasive feedback. This is the first direct comparison between invasive and non-invasive approaches for restoring sensory feedback and it could suggest an advantage of using invasive solutions. Clinical Trial Registration: www.ClinicalTrials.gov, identifier NCT02848846.

The Grasp Strategy of a Robot Passer Influences Performance and Quality of the Robot-Human Object Handover.

Task-aware robotic grasping is critical if robots are to successfully cooperate with humans. The choice of a grasp is multi-faceted; however, the task to perform primes this choice in terms of hand shaping and placement on the object. This grasping strategy is particularly important for a robot companion, as it can potentially hinder the success of the collaboration with humans. In this work, we investigate how different grasping strategies of a robot passer influence the performance and the perceptions of the interaction of a human receiver. Our findings suggest that a grasping strategy that accounts for the subsequent task of the receiver improves substantially the performance of the human receiver in executing the subsequent task. The time to complete the task is reduced by eliminating the need of a post-handover re-adjustment of the object. Furthermore, the human perceptions of the interaction improve when a task-oriented grasping strategy is adopted. The influence of the robotic grasp strategy increases as the constraints induced by the object's affordances become more restrictive. The results of this work can benefit the wider robotics community, with application ranging from industrial to household human-robot interaction for cooperative and collaborative object manipulation.

Grip control and motor coordination with implanted and surface electrodes while grasping with an osseointegrated prosthetic hand.

BACKGROUND: Replacement of a lost limb by an artificial substitute is not yet ideal. Resolution and coordination of motor control approximating that of a biological limb could dramatically improve the functionality of prosthetic devices, and thus reduce the gap towards a suitable limb replacement. METHODS: In this study, we investigated the control resolution and coordination exhibited by subjects with transhumeral amputation who were implanted with epimysial electrodes and an osseointegrated interface that provides bidirectional communication in addition to skeletal attachment (e-OPRA Implant System). We assessed control resolution and coordination in the context of routine and delicate grasping using the Pick and Lift and the Virtual Eggs Tests. Performance when utilizing implanted electrodes was compared with the standard-of-care technology for myoelectric prostheses, namely surface electrodes. RESULTS: Results showed that implanted electrodes provide superior controllability over the prosthetic terminal device compared to conventional surface electrodes. Significant improvements were found in the control of the grip force and its reliability during object transfer. However, these improvements failed to increase motor coordination, and surprisingly decreased the temporal correlation between grip and load forces observed with surface electrodes. We found that despite being more functional and reliable, prosthetic control via implanted electrodes still depended highly on visual feedback. CONCLUSIONS: Our findings indicate that incidental sensory feedback (visual, auditory, and osseoperceptive in this case) is insufficient for restoring natural grasp behavior in amputees, and support the idea that supplemental tactile sensory feedback is needed to learn and maintain the motor tasks internal model, which could ultimately restore natural grasp behavior in subjects using prosthetic hands.

Intraneural sensory feedback restores grip force control and motor coordination while using a prosthetic hand.

OBJECTIVE: Tactile afferents in the human hand provide fundamental information about hand-environment interactions, which is used by the brain to adapt the motor output to the physical properties of the object being manipulated. A hand amputation disrupts both afferent and efferent pathways from/to the hand, completely invalidating the individual's motor repertoire. Although motor functions may be partially recovered by using a myoelectric prosthesis, providing functionally effective sensory feedback to users of prosthetics is a largely unsolved challenge. While past studies using invasive stimulation suggested that sensory feedback may help in handling fragile objects, none explored the underpinning, relearned, motor coordination during grasping. In this study, we aimed at showing for the first time that intraneural sensory feedback of the grip force (GF) improves the sensorimotor control of a transradial amputee controlling a myoelectric prosthesis. APPROACH: We performed a longitudinal study testing a single subject (clinical trial registration number NCT02848846). A stacking cups test (CUP) performed over two weeks aimed at measuring the subject's ability to finely regulate the GF applied with the prosthesis. A pick and lift test (PLT), performed at the end of the study, measured the level of motor coordination, and whether the subject transferred the motor skills learned in the CUP to an alien task. MAIN RESULTS: The results show that intraneural sensory feedback increases the subject's ability in regulating the GF and allows for improved performance over time. Additionally, the PLT demonstrated that the subject was able to generalize and transfer her manipulation skills to an unknown task and to improve her motor coordination. SIGNIFICANCE: Our findings suggest that intraneural sensory feedback holds the potential of restoring functionally effective tactile feedback. This opens up new possibilities to improve the quality of life of amputees using a neural prosthesis.

A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback.

Current myoelectric prostheses allow transradial amputees to regain voluntary motor control of their artificial limb by exploiting residual muscle function in the forearm. However, the overreliance on visual cues resulting from a lack of sensory feedback is a common complaint. Recently, several groups have provided tactile feedback in upper limb amputees using implanted electrodes, surface nerve stimulation, or sensory substitution. These approaches have led to improved function and prosthesis embodiment. Nevertheless, the provided information remains limited to a subset of the rich sensory cues available to healthy individuals. More specifically, proprioception, the sense of limb position and movement, is predominantly absent from current systems. Here, we show that sensory substitution based on intraneural stimulation can deliver position feedback in real time and in conjunction with somatotopic tactile feedback. This approach allowed two transradial amputees to regain high and close-to-natural remapped proprioceptive acuity, with a median joint angle reproduction precision of 9.1° and a median threshold to detection of passive movements of 9.5°, which was comparable with results obtained in healthy participants. The simultaneous delivery of position information and somatotopic tactile feedback allowed both amputees to discriminate the size and compliance of four objects with high levels of performance (75.5%). These results demonstrate that tactile information delivered via somatotopic neural stimulation and position information delivered via sensory substitution can be exploited simultaneously and efficiently by transradial amputees. This study paves a way to more sophisticated bidirectional bionic limbs conveying richer, multimodal sensations.

Six-Month Assessment of a Hand Prosthesis with Intraneural Tactile Feedback.

OBJECTIVE: Hand amputation is a highly disabling event, which significantly affects quality of life. An effective hand replacement can be achieved if the user, in addition to motor functions, is provided with the sensations that are naturally perceived while grasping and moving. Intraneural peripheral electrodes have shown promising results toward the restoration of the sense of touch. However, the long-term usability and clinical relevance of intraneural sensory feedback have not yet been clearly demonstrated. METHODS: To this aim, we performed a 6-month clinical study with 3 transradial amputees who received implants of transverse intrafascicular multichannel electrodes (TIMEs) in their median and ulnar nerves. After calibration, electrical stimulation was delivered through the TIMEs connected to artificial sensors in the digits of a prosthesis to generate sensory feedback, which was then used by the subjects while performing different grasping tasks. RESULTS: All subjects, notwithstanding their important clinical differences, reported stimulation-induced sensations from the phantom hand for the whole duration of the trial. They also successfully integrated the sensory feedback into their motor control strategies while performing experimental tests simulating tasks of real life (with and without the support of vision). Finally, they reported a decrement of their phantom limb pain and a general improvement in mood state. INTERPRETATION: The promising results achieved with all subjects show the feasibility of the use of intraneural stimulation in clinical settings. ANN NEUROL 2019;85:137-154.

Restoring Natural Forearm Rotation in Transradial Osseointegrated Amputees.

Osseointegrated transradial prostheses have the potential to preserve the natural range of wrist rotation, which improves the performance of activities of daily living and reduces compensatory movements that potentially lead to secondary health problems over time. This is possible by enabling the radius and the ulna bone to move with respect to each other, restoring the functionality of the original distal-radioulnar joint. In this paper, we report on psychophysics tests performed on an osseointegrated transradial amputee with the aim to understand the extent of mobility of the implants that is required to preserve the natural forearm rotation. Based on these experiments, we designed and developed an attachment device between the implants and the hand prosthesis that serves as an artificial distal radio-ulnar joint. This device was fitted on an osseointegrated transradial amputee and its functionality assessed by means of the Southampton Hand Assessment Procedure (SHAP) and the Minnesota Manual Dexterity test (MMDT). We found that the axial rotation of the implants is required to preserve forearm rotation, to distribute loads equally over the two implants (60% radius - 40% ulna), and to enable loading of the implants without unpleasant feelings for the patient. Higher function was recorded when our attachment device enabled forearm rotation: SHAP from 61 to 71, MMDT from 258s to 231s. Natural forearm rotation can be successfully restored in transradial amputees by using osseointegration and our novel mechanical attachment to the hand prosthesis.

Biomimetic Intraneural Sensory Feedback Enhances Sensation Naturalness, Tactile Sensitivity, and Manual Dexterity in a Bidirectional Prosthesis.

Peripheral intraneural stimulation can provide tactile information to amputees. However, efforts are still necessary to identify encoding strategy eliciting percepts that are felt as both natural and effective for prosthesis control. Here we compared the naturalness and efficacy of different encoding strategies to deliver neural stimulation to trans-radial amputees implanted with intraneural electrodes. Biomimetic frequency modulation was perceived as more natural, while amplitude modulation enabled better performance in tasks requiring fine identification of the applied force. Notably, the optimal combination of naturalness and sensitivity of the tactile feedback can be achieved with "hybrid" encoding strategies based on simultaneous biomimetic frequency and amplitude neuromodulation. These strategies improved the gross manual dexterity of the subjects during functional task while maintaining high levels of manual accuracy. They also improved prosthesis embodiment, reducing abnormal phantom limb perceptions ("telescoping effect"). Hybrid strategies are able to provide highly sensitive and natural percepts and should be preferred. VIDEO ABSTRACT.

Humans adjust their grip force when passing an object according to the observed speed of the partner's reaching out movement.

The way an object is released by the passer to a partner is fundamental for the success of the handover and for the experienced fluency and quality of the interaction. Nonetheless, although its apparent simplicity, object handover involves a complex combination of predictive and reactive control mechanisms that were not fully investigated so far. Here, we show that passers use visual-feedback based anticipatory control to trigger the beginning of the release, to launch the appropriate motor program, and adapt such predictions to different speeds of the receiver's reaching out movements. In particular, the passer starts releasing the object in synchrony with the collision with the receiver, regardless of the receiver's speed, but the passer's speed of grip force release is correlated with receiver speed. When visual feedback is removed, the beginning of the passer's release is delayed proportionally with the receiver's reaching out speed; however, the correlation between the passer's peak rate of change of grip force is maintained. In a second study with 11 participants receiving an object from a robotic hand programmed to release following stereotypical biomimetic profiles, we found that handovers are experienced as more fluent when they exhibit more reactive release behaviours, shorter release durations, and shorter handover durations. The outcomes from the two studies contribute understanding of the roles of sensory input in the strategy that empower humans to perform smooth and safe handovers, and they suggest methods for programming controllers that would enable artificial hands to hand over objects with humans in an easy, natural and efficient way.

Improving internal model strength and performance of prosthetic hands using augmented feedback.

BACKGROUND: The loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays. METHODS: Our recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance. RESULTS: Results obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback. CONCLUSIONS: We extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.

Corrections to "Treatment of the Partial Hand Amputation: An Engineering Perspective".

In the above paper, by Imbinto et al. (IEEE Rev. Biomed. Eng., vol. 9, pp. 32-48, 2016), typographical errors appear in (1) (2) (3), whereas (4) is missing of a factor; all the equations are reported within Section III-C of the paper. The modifications are provided in this paper.

Compliant Prosthetic Wrists Entail More Natural Use Than Stiff Wrists During Reaching, Not (Necessarily) During Manipulation.

Developing an artificial arm with functions equivalent to those of the human arm is one of the challenging goals of bioengineering. State-of-the-artprostheses lack several degrees of freedom and force the individuals to compensate for them by means of compensatory movements, which often result in residual limb pain and overuse syndromes. Passive wristsmay reduce such compensatory actions, nonethelessto date their actual efficacy, associated to conventional myoelectric hands is a matter of debate. We hypothesized that a transradial prosthesiswould allow a simpler operation if its wrist behaved compliant during the reaching and grasping phase, and stiff during the holding andmanipulation phase. To assess this, we compared a stiff and a compliant wrist and evaluating the extent of compensatory movements in the trunk and shoulder, with unimpaired subjects wearing orthoses, while performing nine activities of daily living taken from the southampton hand assessment procedure. Our findings show indeed that the optimal compliance for a prosthetic wrist is specific to the phase of the motor task: the compliant wrist outperforms the stiff wrist during the reaching phase, whereas the stiff wrist exhibits more natural movements during the manipulation phase of heavy objects. Hence, this paper invites rehabilitation engineers to develop wrists with switchable compliance.

Gesture Decoding Using ECoG Signals from Human Sensorimotor Cortex: A Pilot Study.

Electrocorticography (ECoG) has been demonstrated as a promising neural signal source for developing brain-machine interfaces (BMIs). However, many concerns about the disadvantages brought by large craniotomy for implanting the ECoG grid limit the clinical translation of ECoG-based BMIs. In this study, we collected clinical ECoG signals from the sensorimotor cortex of three epileptic participants when they performed hand gestures. The ECoG power spectrum in hybrid frequency bands was extracted to build a synchronous real-time BMI system. High decoding accuracy of the three gestures was achieved in both offline analysis (85.7%, 84.5%, and 69.7%) and online tests (80% and 82%, tested on two participants only). We found that the decoding performance was maintained even with a subset of channels selected by a greedy algorithm. More importantly, these selected channels were mostly distributed along the central sulcus and clustered in the area of 3 interelectrode squares. Our findings of the reduced and clustered distribution of ECoG channels further supported the feasibility of clinically implementing the ECoG-based BMI system for the control of hand gestures.

A cosmetic prosthetic digit with bioinspired embedded touch feedback.

Partial hand amputation is the most frequent amputation level worldwide, accounting for approximately 90% of all upper limb amputations. Passive cosmetic prostheses represent one of the possible choices for its treatment, probably the most affordable one. However, these devices restore very limited motor function and subtle sensory feedback. The latter is an important component for restoring the body schema. In this work we present a simple yet potentially effective and low cost cosmetic digital prosthesis that embeds touch feedback; we dubbed this DESC-finger. It delivers short-lasting vibrotactile bursts when it makes and breaks contact with the environment, based on the Discrete Event-driven Sensory feedback Control (DESC) policy. One prototype was developed and used by one amputee at home, for two months. The effectiveness of the device was experimentally assessed by means of an interview and a virtual eggs test, which showed, albeit preliminarily, that time discrete feedback can improve the motor control of a partial hand prosthesis in daily life conditions. Besides targeting people that already use cosmetic digits, the DESC-finger targets those that do not use them complaining for loss of sensibility. The production costs and manufacturing process makes the DESC-finger suitable for exploitation in high- and low-income countries.