Marine Robotics for Deep-Sea Specimen Collection: A Systematic Review of Underwater Grippers.

The collection of delicate deep-sea specimens of biological interest with remotely operated vehicle (ROV) industrial grippers and tools is a long and expensive procedure. Industrial grippers were originally designed for heavy manipulation tasks, while sampling specimens requires dexterity and precision. We describe the grippers and tools commonly used in underwater sampling for scientific purposes, systematically review the state of the art of research in underwater gripping technologies, and identify design trends. We discuss the possibility of executing typical manipulations of sampling procedures with commonly used grippers and research prototypes. Our results indicate that commonly used grippers ensure that the basic actions either of gripping or caging are possible, and their functionality is extended by holding proper tools. Moreover, the approach of the research status seems to have changed its focus in recent years: from the demonstration of the validity of a specific technology (actuation, transmission, sensing) for marine applications, to the solution of specific needs of underwater manipulation. Finally, we summarize the environmental and operational requirements that should be considered in the design of an underwater gripper.

Marine Robotics for Deep-Sea Specimen Collection: A Taxonomy of Underwater Manipulative Actions.

In order to develop a gripping system or control strategy that improves scientific sampling procedures, knowledge of the process and the consequent definition of requirements is fundamental. Nevertheless, factors influencing sampling procedures have not been extensively described, and selected strategies mostly depend on pilots' and researchers' experience. We interviewed 17 researchers and remotely operated vehicle (ROV) technical operators, through a formal questionnaire or in-person interviews, to collect evidence of sampling procedures based on their direct field experience. We methodologically analyzed sampling procedures to extract single basic actions (called atomic manipulations). Available equipment, environment and species-specific features strongly influenced the manipulative choices. We identified a list of functional and technical requirements for the development of novel end-effectors for marine sampling. Our results indicate that the unstructured and highly variable deep-sea environment requires a versatile system, capable of robust interactions with hard surfaces such as pushing or scraping, precise tuning of gripping force for tasks such as pulling delicate organisms away from hard and soft substrates, and rigid holding, as well as a mechanism for rapidly switching among external tools.

Six-Month Assessment of a Hand Prosthesis with Intraneural Tactile Feedback.

OBJECTIVE: Hand amputation is a highly disabling event, which significantly affects quality of life. An effective hand replacement can be achieved if the user, in addition to motor functions, is provided with the sensations that are naturally perceived while grasping and moving. Intraneural peripheral electrodes have shown promising results toward the restoration of the sense of touch. However, the long-term usability and clinical relevance of intraneural sensory feedback have not yet been clearly demonstrated. METHODS: To this aim, we performed a 6-month clinical study with 3 transradial amputees who received implants of transverse intrafascicular multichannel electrodes (TIMEs) in their median and ulnar nerves. After calibration, electrical stimulation was delivered through the TIMEs connected to artificial sensors in the digits of a prosthesis to generate sensory feedback, which was then used by the subjects while performing different grasping tasks. RESULTS: All subjects, notwithstanding their important clinical differences, reported stimulation-induced sensations from the phantom hand for the whole duration of the trial. They also successfully integrated the sensory feedback into their motor control strategies while performing experimental tests simulating tasks of real life (with and without the support of vision). Finally, they reported a decrement of their phantom limb pain and a general improvement in mood state. INTERPRETATION: The promising results achieved with all subjects show the feasibility of the use of intraneural stimulation in clinical settings. ANN NEUROL 2019;85:137-154.

Grip control and motor coordination with implanted and surface electrodes while grasping with an osseointegrated prosthetic hand.

BACKGROUND: Replacement of a lost limb by an artificial substitute is not yet ideal. Resolution and coordination of motor control approximating that of a biological limb could dramatically improve the functionality of prosthetic devices, and thus reduce the gap towards a suitable limb replacement. METHODS: In this study, we investigated the control resolution and coordination exhibited by subjects with transhumeral amputation who were implanted with epimysial electrodes and an osseointegrated interface that provides bidirectional communication in addition to skeletal attachment (e-OPRA Implant System). We assessed control resolution and coordination in the context of routine and delicate grasping using the Pick and Lift and the Virtual Eggs Tests. Performance when utilizing implanted electrodes was compared with the standard-of-care technology for myoelectric prostheses, namely surface electrodes. RESULTS: Results showed that implanted electrodes provide superior controllability over the prosthetic terminal device compared to conventional surface electrodes. Significant improvements were found in the control of the grip force and its reliability during object transfer. However, these improvements failed to increase motor coordination, and surprisingly decreased the temporal correlation between grip and load forces observed with surface electrodes. We found that despite being more functional and reliable, prosthetic control via implanted electrodes still depended highly on visual feedback. CONCLUSIONS: Our findings indicate that incidental sensory feedback (visual, auditory, and osseoperceptive in this case) is insufficient for restoring natural grasp behavior in amputees, and support the idea that supplemental tactile sensory feedback is needed to learn and maintain the motor tasks internal model, which could ultimately restore natural grasp behavior in subjects using prosthetic hands.

Humans adjust their grip force when passing an object according to the observed speed of the partner's reaching out movement.

The way an object is released by the passer to a partner is fundamental for the success of the handover and for the experienced fluency and quality of the interaction. Nonetheless, although its apparent simplicity, object handover involves a complex combination of predictive and reactive control mechanisms that were not fully investigated so far. Here, we show that passers use visual-feedback based anticipatory control to trigger the beginning of the release, to launch the appropriate motor program, and adapt such predictions to different speeds of the receiver's reaching out movements. In particular, the passer starts releasing the object in synchrony with the collision with the receiver, regardless of the receiver's speed, but the passer's speed of grip force release is correlated with receiver speed. When visual feedback is removed, the beginning of the passer's release is delayed proportionally with the receiver's reaching out speed; however, the correlation between the passer's peak rate of change of grip force is maintained. In a second study with 11 participants receiving an object from a robotic hand programmed to release following stereotypical biomimetic profiles, we found that handovers are experienced as more fluent when they exhibit more reactive release behaviours, shorter release durations, and shorter handover durations. The outcomes from the two studies contribute understanding of the roles of sensory input in the strategy that empower humans to perform smooth and safe handovers, and they suggest methods for programming controllers that would enable artificial hands to hand over objects with humans in an easy, natural and efficient way.

Improving internal model strength and performance of prosthetic hands using augmented feedback.

BACKGROUND: The loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays. METHODS: Our recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance. RESULTS: Results obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback. CONCLUSIONS: We extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.

An Instrumented Glove for Restoring Sensorimotor Function of the Hand Through Augmented Sensory Feedback.

The loss of sensitivity of the upper limb due to neurological injuries severely limits the ability to manipulate objects, hindering personal independence. Non-invasive augmented sensory feedback techniques are used to promote neural plasticity hence to restore the grasping function. This work presents a wearable device for restoring sensorimotor hand functions based on Discrete Event-driven Sensory Control policy. It consists of an instrumented glove that, relying on piezoelectric sensors, delivers short-lasting vibrotactile stimuli synchronously with the relevant mechanical events (i.e., contact and release) of the manipulation. We first performed a feasibility study on healthy participants (20) that showed overall good performances of the device, with touch-event detection accuracy of 96.2% and a response delay of 22 ms. Later, we pilot tested it on two participants with limited sensorimotor functions. When using the device, they improved their hand motor coordination while performing tests for hand motor coordination assessment (i.e., pick and place test, pick and lift test). In particular, they exhibited more coordinated temporal correlations between grip force and load force profiles and enhanced performances when transferring objects, quantitatively proving the effectiveness of the device.

A highly integrated bionic hand with neural control and feedback for use in daily life.

Restoration of sensorimotor function after amputation has remained challenging because of the lack of human-machine interfaces that provide reliable control, feedback, and attachment. Here, we present the clinical implementation of a transradial neuromusculoskeletal prosthesis-a bionic hand connected directly to the user's nervous and skeletal systems. In one person with unilateral below-elbow amputation, titanium implants were placed intramedullary in the radius and ulna bones, and electromuscular constructs were created surgically by transferring the severed nerves to free muscle grafts. The native muscles, free muscle grafts, and ulnar nerve were implanted with electrodes. Percutaneous extensions from the titanium implants provided direct skeletal attachment and bidirectional communication between the implanted electrodes and a prosthetic hand. Operation of the bionic hand in daily life resulted in improved prosthetic function, reduced postamputation, and increased quality of life. Sensations elicited via direct neural stimulation were consistently perceived on the phantom hand throughout the study. To date, the patient continues using the prosthesis in daily life. The functionality of conventional artificial limbs is hindered by discomfort and limited and unreliable control. Neuromusculoskeletal interfaces can overcome these hurdles and provide the means for the everyday use of a prosthesis with reliable neural control fixated into the skeleton.

The SmartHand transradial prosthesis.

BACKGROUND: Prosthetic components and control interfaces for upper limb amputees have barely changed in the past 40 years. Many transradial prostheses have been developed in the past, nonetheless most of them would be inappropriate if/when a large bandwidth human-machine interface for control and perception would be available, due to either their limited (or inexistent) sensorization or limited dexterity. SmartHand tackles this issue as is meant to be clinically experimented in amputees employing different neuro-interfaces, in order to investigate their effectiveness. This paper presents the design and on bench evaluation of the SmartHand. METHODS: SmartHand design was bio-inspired in terms of its physical appearance, kinematics, sensorization, and its multilevel control system. Underactuated fingers and differential mechanisms were designed and exploited in order to fit all mechatronic components in the size and weight of a natural human hand. Its sensory system was designed with the aim of delivering significant afferent information to the user through adequate interfaces. RESULTS: SmartHand is a five fingered self-contained robotic hand, with 16 degrees of freedom, actuated by 4 motors. It integrates a bio-inspired sensory system composed of 40 proprioceptive and exteroceptive sensors and a customized embedded controller both employed for implementing automatic grasp control and for potentially delivering sensory feedback to the amputee. It is able to perform everyday grasps, count and independently point the index. The weight (530 g) and speed (closing time: 1.5 seconds) are comparable to actual commercial prostheses. It is able to lift a 10 kg suitcase; slippage tests showed that within particular friction and geometric conditions the hand is able to stably grasp up to 3.6 kg cylindrical objects. CONCLUSIONS: Due to its unique embedded features and human-size, the SmartHand holds the promise to be experimentally fitted on transradial amputees and employed as a bi-directional instrument for investigating -during realistic experiments- different interfaces, control and feedback strategies in neuro-engineering studies.

A database of high-density surface electromyogram signals comprising 65 isometric hand gestures.

Control of contemporary, multi-joint prosthetic hands is commonly realized by using electromyographic signals from the muscles remaining after amputation at the forearm level. Although this principle is trying to imitate the natural control structure where muscles control the joints of the hand, in practice, myoelectric control provides only basic hand functions to an amputee using a dexterous prosthesis. This study aims to provide an annotated database of high-density surface electromyographic signals to aid the efforts of designing robust and versatile electromyographic control interfaces for prosthetic hands. The electromyographic signals were recorded using 128 channels within two electrode grids positioned on the forearms of 20 able-bodied volunteers. The participants performed 65 different hand gestures in an isometric manner. The hand movements were strictly timed using an automated recording protocol which also synchronously recorded the electromyographic signals and hand joint forces. To assess the quality of the recorded signals several quantitative assessments were performed, such as frequency content analysis, channel crosstalk, and the detection of poor skin-electrode contacts.

Cognitive vision system for control of dexterous prosthetic hands: experimental evaluation.

BACKGROUND: Dexterous prosthetic hands that were developed recently, such as SmartHand and i-LIMB, are highly sophisticated; they have individually controllable fingers and the thumb that is able to abduct/adduct. This flexibility allows implementation of many different grasping strategies, but also requires new control algorithms that can exploit the many degrees of freedom available. The current study presents and tests the operation of a new control method for dexterous prosthetic hands. METHODS: The central component of the proposed method is an autonomous controller comprising a vision system with rule-based reasoning mounted on a dexterous hand (CyberHand). The controller, termed cognitive vision system (CVS), mimics biological control and generates commands for prehension. The CVS was integrated into a hierarchical control structure: 1) the user triggers the system and controls the orientation of the hand; 2) a high-level controller automatically selects the grasp type and size; and 3) an embedded hand controller implements the selected grasp using closed-loop position/force control. The operation of the control system was tested in 13 healthy subjects who used Cyberhand, attached to the forearm, to grasp and transport 18 objects placed at two different distances. RESULTS: The system correctly estimated grasp type and size (nine commands in total) in about 84% of the trials. In an additional 6% of the trials, the grasp type and/or size were different from the optimal ones, but they were still good enough for the grasp to be successful. If the control task was simplified by decreasing the number of possible commands, the classification accuracy increased (e.g., 93% for guessing the grasp type only). CONCLUSIONS: The original outcome of this research is a novel controller empowered by vision and reasoning and capable of high-level analysis (i.e., determining object properties) and autonomous decision making (i.e., selecting the grasp type and size). The automatic control eases the burden from the user and, as a result, the user can concentrate on what he/she does, not on how he/she should do it. The tests showed that the performance of the controller was satisfactory and that the users were able to operate the system with minimal prior training.

The Grasp Strategy of a Robot Passer Influences Performance and Quality of the Robot-Human Object Handover.

Task-aware robotic grasping is critical if robots are to successfully cooperate with humans. The choice of a grasp is multi-faceted; however, the task to perform primes this choice in terms of hand shaping and placement on the object. This grasping strategy is particularly important for a robot companion, as it can potentially hinder the success of the collaboration with humans. In this work, we investigate how different grasping strategies of a robot passer influence the performance and the perceptions of the interaction of a human receiver. Our findings suggest that a grasping strategy that accounts for the subsequent task of the receiver improves substantially the performance of the human receiver in executing the subsequent task. The time to complete the task is reduced by eliminating the need of a post-handover re-adjustment of the object. Furthermore, the human perceptions of the interaction improve when a task-oriented grasping strategy is adopted. The influence of the robotic grasp strategy increases as the constraints induced by the object's affordances become more restrictive. The results of this work can benefit the wider robotics community, with application ranging from industrial to household human-robot interaction for cooperative and collaborative object manipulation.

Neural feedback strategies to improve grasping coordination in neuromusculoskeletal prostheses.

Conventional prosthetic arms suffer from poor controllability and lack of sensory feedback. Owing to the absence of tactile sensory information, prosthetic users must rely on incidental visual and auditory cues. In this study, we investigated the effect of providing tactile perception on motor coordination during routine grasping and grasping under uncertainty. Three transhumeral amputees were implanted with an osseointegrated percutaneous implant system for direct skeletal attachment and bidirectional communication with implanted neuromuscular electrodes. This neuromusculoskeletal prosthesis is a novel concept of artificial limb replacement that allows to extract control signals from electrodes implanted on viable muscle tissue, and to stimulate severed afferent nerve fibers to provide somatosensory feedback. Subjects received tactile feedback using three biologically inspired stimulation paradigms while performing a pick and lift test. The grasped object was instrumented to record grasping and lifting forces and its weight was either constant or unexpectedly changed in between trials. The results were also compared to the no-feedback control condition. Our findings confirm, in line with the neuroscientific literature, that somatosensory feedback is necessary for motor coordination during grasping. Our results also indicate that feedback is more relevant under uncertainty, and its effectiveness can be influenced by the selected neuromodulation paradigm and arguably also the prior experience of the prosthesis user.

Hand Control With Invasive Feedback Is Not Impaired by Increased Cognitive Load.

Recent experiments have shown that neural stimulation can successfully restore sensory feedback in upper-limb amputees improving their ability to control the prosthesis. However, the potential advantages of invasive sensory feedback with respect to non-invasive solutions have not been yet identified. Our hypothesis was that a difference would appear when the subject cannot focus all the attention to the use of the prosthesis, but some additional activities require his/her cognitive attention, which is a quite common situation in real-life conditions. To verify this hypothesis, we asked a trans-radial amputee, equipped with a bidirectional hand prosthesis, to perform motor tasks also in combination with a cognitive task. Sensory feedback was provided via intraneural (invasive) or electro-tactile (non-invasive) stimulation. We collected also data related to self-confidence. While both approaches were able to significantly improve the motor performance of the subject when no additional cognitive effort was asked, the manual accuracy was not affected by the cognitive task only when intraneural feedback was provided. The highest self-confidence was obtained when intraneural sensory feedback was provided. Our findings show that intraneural sensory feedback is more robust to dual tasks than non-invasive feedback. This is the first direct comparison between invasive and non-invasive approaches for restoring sensory feedback and it could suggest an advantage of using invasive solutions. Clinical Trial Registration: www.ClinicalTrials.gov, identifier NCT02848846.

Intraneural sensory feedback restores grip force control and motor coordination while using a prosthetic hand.

OBJECTIVE: Tactile afferents in the human hand provide fundamental information about hand-environment interactions, which is used by the brain to adapt the motor output to the physical properties of the object being manipulated. A hand amputation disrupts both afferent and efferent pathways from/to the hand, completely invalidating the individual's motor repertoire. Although motor functions may be partially recovered by using a myoelectric prosthesis, providing functionally effective sensory feedback to users of prosthetics is a largely unsolved challenge. While past studies using invasive stimulation suggested that sensory feedback may help in handling fragile objects, none explored the underpinning, relearned, motor coordination during grasping. In this study, we aimed at showing for the first time that intraneural sensory feedback of the grip force (GF) improves the sensorimotor control of a transradial amputee controlling a myoelectric prosthesis. APPROACH: We performed a longitudinal study testing a single subject (clinical trial registration number NCT02848846). A stacking cups test (CUP) performed over two weeks aimed at measuring the subject's ability to finely regulate the GF applied with the prosthesis. A pick and lift test (PLT), performed at the end of the study, measured the level of motor coordination, and whether the subject transferred the motor skills learned in the CUP to an alien task. MAIN RESULTS: The results show that intraneural sensory feedback increases the subject's ability in regulating the GF and allows for improved performance over time. Additionally, the PLT demonstrated that the subject was able to generalize and transfer her manipulation skills to an unknown task and to improve her motor coordination. SIGNIFICANCE: Our findings suggest that intraneural sensory feedback holds the potential of restoring functionally effective tactile feedback. This opens up new possibilities to improve the quality of life of amputees using a neural prosthesis.

A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback.

Current myoelectric prostheses allow transradial amputees to regain voluntary motor control of their artificial limb by exploiting residual muscle function in the forearm. However, the overreliance on visual cues resulting from a lack of sensory feedback is a common complaint. Recently, several groups have provided tactile feedback in upper limb amputees using implanted electrodes, surface nerve stimulation, or sensory substitution. These approaches have led to improved function and prosthesis embodiment. Nevertheless, the provided information remains limited to a subset of the rich sensory cues available to healthy individuals. More specifically, proprioception, the sense of limb position and movement, is predominantly absent from current systems. Here, we show that sensory substitution based on intraneural stimulation can deliver position feedback in real time and in conjunction with somatotopic tactile feedback. This approach allowed two transradial amputees to regain high and close-to-natural remapped proprioceptive acuity, with a median joint angle reproduction precision of 9.1° and a median threshold to detection of passive movements of 9.5°, which was comparable with results obtained in healthy participants. The simultaneous delivery of position information and somatotopic tactile feedback allowed both amputees to discriminate the size and compliance of four objects with high levels of performance (75.5%). These results demonstrate that tactile information delivered via somatotopic neural stimulation and position information delivered via sensory substitution can be exploited simultaneously and efficiently by transradial amputees. This study paves a way to more sophisticated bidirectional bionic limbs conveying richer, multimodal sensations.

Compliant Prosthetic Wrists Entail More Natural Use Than Stiff Wrists During Reaching, Not (Necessarily) During Manipulation.

Developing an artificial arm with functions equivalent to those of the human arm is one of the challenging goals of bioengineering. State-of-the-artprostheses lack several degrees of freedom and force the individuals to compensate for them by means of compensatory movements, which often result in residual limb pain and overuse syndromes. Passive wristsmay reduce such compensatory actions, nonethelessto date their actual efficacy, associated to conventional myoelectric hands is a matter of debate. We hypothesized that a transradial prosthesiswould allow a simpler operation if its wrist behaved compliant during the reaching and grasping phase, and stiff during the holding andmanipulation phase. To assess this, we compared a stiff and a compliant wrist and evaluating the extent of compensatory movements in the trunk and shoulder, with unimpaired subjects wearing orthoses, while performing nine activities of daily living taken from the southampton hand assessment procedure. Our findings show indeed that the optimal compliance for a prosthetic wrist is specific to the phase of the motor task: the compliant wrist outperforms the stiff wrist during the reaching phase, whereas the stiff wrist exhibits more natural movements during the manipulation phase of heavy objects. Hence, this paper invites rehabilitation engineers to develop wrists with switchable compliance.

Restoring Natural Forearm Rotation in Transradial Osseointegrated Amputees.

Osseointegrated transradial prostheses have the potential to preserve the natural range of wrist rotation, which improves the performance of activities of daily living and reduces compensatory movements that potentially lead to secondary health problems over time. This is possible by enabling the radius and the ulna bone to move with respect to each other, restoring the functionality of the original distal-radioulnar joint. In this paper, we report on psychophysics tests performed on an osseointegrated transradial amputee with the aim to understand the extent of mobility of the implants that is required to preserve the natural forearm rotation. Based on these experiments, we designed and developed an attachment device between the implants and the hand prosthesis that serves as an artificial distal radio-ulnar joint. This device was fitted on an osseointegrated transradial amputee and its functionality assessed by means of the Southampton Hand Assessment Procedure (SHAP) and the Minnesota Manual Dexterity test (MMDT). We found that the axial rotation of the implants is required to preserve forearm rotation, to distribute loads equally over the two implants (60% radius - 40% ulna), and to enable loading of the implants without unpleasant feelings for the patient. Higher function was recorded when our attachment device enabled forearm rotation: SHAP from 61 to 71, MMDT from 258s to 231s. Natural forearm rotation can be successfully restored in transradial amputees by using osseointegration and our novel mechanical attachment to the hand prosthesis.

Corrections to "Treatment of the Partial Hand Amputation: An Engineering Perspective".

In the above paper, by Imbinto et al. (IEEE Rev. Biomed. Eng., vol. 9, pp. 32-48, 2016), typographical errors appear in (1) (2) (3), whereas (4) is missing of a factor; all the equations are reported within Section III-C of the paper. The modifications are provided in this paper.

Biomimetic Intraneural Sensory Feedback Enhances Sensation Naturalness, Tactile Sensitivity, and Manual Dexterity in a Bidirectional Prosthesis.

Peripheral intraneural stimulation can provide tactile information to amputees. However, efforts are still necessary to identify encoding strategy eliciting percepts that are felt as both natural and effective for prosthesis control. Here we compared the naturalness and efficacy of different encoding strategies to deliver neural stimulation to trans-radial amputees implanted with intraneural electrodes. Biomimetic frequency modulation was perceived as more natural, while amplitude modulation enabled better performance in tasks requiring fine identification of the applied force. Notably, the optimal combination of naturalness and sensitivity of the tactile feedback can be achieved with "hybrid" encoding strategies based on simultaneous biomimetic frequency and amplitude neuromodulation. These strategies improved the gross manual dexterity of the subjects during functional task while maintaining high levels of manual accuracy. They also improved prosthesis embodiment, reducing abnormal phantom limb perceptions ("telescoping effect"). Hybrid strategies are able to provide highly sensitive and natural percepts and should be preferred. VIDEO ABSTRACT.