How Positioning Wearable Haptic Interfaces on Limbs Influences Virtual Embodiment.

With increasing use of computer applications and robotic devices in our everyday life, and with the advent of metaverse, there is an urgent need of developing new types of interfaces that facilitate a more intuitive interaction in physical and virtual space. In this work, we investigate the influence of the location of haptic feedback devices on embodiment of virtual hands and user load during an interactive pick-and-place task. To do this, we conducted a user study with a 3x2 repeated measure experiment design: feedback position is varied between the distal phalanx of the index finger and the thumb, the proximal phalanx of the index finger and the thumb, and the wrist. These conditions of feedback are tested with the stimuli applied synchronously to the participant in one case, and with an additional delay of 350 ms in the second case. The results show that the location of the haptic feedback device does not affect embodiment, whereas the delay, i.e., whether the feedback is applied synchronously or asynchronously, affects embodiment. This suggests that for pick-and-place tasks, haptic feedback devices can be placed on the user's wrist without compromising performance making the hands to remain free, allowing unobstructed hand visibility for precise motion tracking, thereby improving accuracy.

Detachable Robotic Grippers for Human-Robot Collaboration.

Collaborative robots promise to add flexibility to production cells thanks to the fact that they can work not only close to humans but also with humans. The possibility of a direct physical interaction between humans and robots allows to perform operations that were inconceivable with industrial robots. Collaborative soft grippers have been recently introduced to extend this possibility beyond the robot end-effector, making humans able to directly act on robotic hands. In this work, we propose to exploit collaborative grippers in a novel paradigm in which these devices can be easily attached and detached from the robot arm and used also independently from it. This is possible only with self-powered hands, that are still quite uncommon in the market. In the presented paradigm not only hands can be attached/detached to/from the robot end-effector as if they were simple tools, but they can also remain active and fully functional after detachment. This ensures all the advantages brought in by tool changers, that allow for quick and possibly automatic tool exchange at the robot end-effector, but also gives the possibility of using the hand capabilities and degrees of freedom without the need of an arm or of external power supplies. In this paper, the concept of detachable robotic grippers is introduced and demonstrated through two illustrative tasks conducted with a new tool changer designed for collaborative grippers. The novel tool changer embeds electromagnets that are used to add safety during attach/detach operations. The activation of the electromagnets is controlled through a wearable interface capable of providing tactile feedback. The usability of the system is confirmed by the evaluations of 12 users.

Emerging of new bioartificial corticospinal motor synergies using a robotic additional thumb.

It is likely that when using an artificially augmented hand with six fingers, the natural five plus a robotic one, corticospinal motor synergies controlling grasping actions might be different. However, no direct neurophysiological evidence for this reasonable assumption is available yet. We used transcranial magnetic stimulation of the primary motor cortex to directly address this issue during motor imagery of objects' grasping actions performed with or without the Soft Sixth Finger (SSF). The SSF is a wearable robotic additional thumb patented for helping patients with hand paresis and inherent loss of thumb opposition abilities. To this aim, we capitalized from the solid notion that neural circuits and mechanisms underlying motor imagery overlap those of physiological voluntary actions. After a few minutes of training, healthy humans wearing the SSF rapidly reshaped the pattern of corticospinal outputs towards forearm and hand muscles governing imagined grasping actions of different objects, suggesting the possibility that the extra finger might rapidly be encoded into the user's body schema, which is integral part of the frontal-parietal grasping network. Such neural signatures might explain how the motor system of human beings is open to very quickly welcoming emerging augmentative bioartificial corticospinal grasping strategies. Such an ability might represent the functional substrate of a final common pathway the brain might count on towards new interactions with the surrounding objects within the peripersonal space. Findings provide a neurophysiological framework for implementing augmentative robotic tools in humans and for the exploitation of the SSF in conceptually new rehabilitation settings.

Integration of a Passive Exoskeleton and a Robotic Supernumerary Finger for Grasping Compensation in Chronic Stroke Patients: The SoftPro Wearable System.

Upper-limb impairments are all-pervasive in Activities of Daily Living (ADLs). As a consequence, people affected by a loss of arm function must endure severe limitations. To compensate for the lack of a functional arm and hand, we developed a wearable system that combines different assistive technologies including sensing, haptics, orthotics and robotics. The result is a device that helps lifting the forearm by means of a passive exoskeleton and improves the grasping ability of the impaired hand by employing a wearable robotic supernumerary finger. A pilot study involving 3 patients, which was conducted to test the capability of the device to assist in performing ADLs, confirmed its usefulness and serves as a first step in the investigation of novel paradigms for robotic assistance.

Can Wearable Haptic Devices Foster the Embodiment of Virtual Limbs?

Increasing presence is one of the primary goals of virtual reality research. A crucial aspect is that users are capable of distinguishing their self from the external virtual world. The hypothesis we investigate is that wearable haptics play an important role in the body experience and could thereby contribute to the immersion of the user in the virtual environment. A within-subject study (n=32) comparing the embodiment of a virtual hand with different implementations of haptic feedback (force feedback, vibrotactile feedback, and no haptic feedback) is presented. Participants wore a glove with haptic feedback devices at thumb and index finger. They were asked to put virtual cubes on a moving virtual target. Touching a virtual object caused vibrotactile-feedback, force-feedback or no feedback depending on the condition. These conditions were provided both synchronously and asynchronously. Embodiment was assessed quantitatively with the proprioceptive drift and subjectively via a questionnaire. Results show that haptic feedback significantly improves the subjective embodiment of a virtual hand and that force feedback leads to stronger responses to certain subscales of subjective embodiment. These outcomes are useful guidelines for wearable haptic designer and represent a basis for further research concerning human body experience, in reality, and in virtual environments.

Human-Robot Team Interaction Through Wearable Haptics for Cooperative Manipulation.

The interaction of robot teams and single human in teleoperation scenarios is beneficial in cooperative tasks, for example, the manipulation of heavy and large objects in remote or dangerous environments. The main control challenge of the interaction is its asymmetry, arising because robot teams have a relatively high number of controllable degrees of freedom compared to the human operator. Therefore, we propose a control scheme that establishes the interaction on spaces of reduced dimensionality taking into account the low number of human command and feedback signals imposed by haptic devices. We evaluate the suitability of wearable haptic fingertip devices for multi-contact teleoperation in a user study. The results show that the proposed control approach is appropriate for human-robot team interaction and that the wearable haptic fingertip devices provide suitable assistance in cooperative manipulation tasks.

Exploiting Robot Hand Compliance and Environmental Constraints for Edge Grasps.

This paper presents a method to grasp objects that cannot be picked directly from a table, using a soft, underactuated hand. These grasps are achieved by dragging the object to the edge of a table, and grasping it from the protruding part, performing so-called slide-to-edge grasps. This type of approach, which uses the environment to facilitate the grasp, is named Environmental Constraint Exploitation (ECE), and has been shown to improve the robustness of grasps while reducing the planning effort. The paper proposes two strategies, namely Continuous Slide and Grasp and Pivot and Re-Grasp, that are designed to deal with different objects. In the first strategy, the hand is positioned over the object and assumed to stick to it during the sliding until the edge, where the fingers wrap around the object and pick it up. In the second strategy, instead, the sliding motion is performed using pivoting, and thus the object is allowed to rotate with respect to the hand that drags it toward the edge. Then, as soon as the object reaches the desired position, the hand detaches from the object and moves to grasp the object from the side. In both strategies, the hand positioning for grasping the object is implemented using a recently proposed functional model for soft hands, the closure signature, whereas the sliding motion on the table is executed by using a hybrid force-velocity controller. We conducted 320 grasping trials with 16 different objects using a soft hand attached to a collaborative robot arm. Experiments showed that the Continuous Slide and Grasp is more suitable for small objects (e.g., a credit card), whereas the Pivot and Re-Grasp performs better with larger objects (e.g., a big book). The gathered data were used to train a classifier that selects the most suitable strategy to use, according to the object size and weight. Implementing ECE strategies with soft hands is a first step toward their use in real-world scenarios, where the environment should be seen more as a help than as a hindrance.

Replicating Human Hand Synergies Onto Robotic Hands: A Review on Software and Hardware Strategies.

This review reports the principal solutions proposed in the literature to reduce the complexity of the control and of the design of robotic hands taking inspiration from the organization of the human brain. Several studies in neuroscience concerning the sensorimotor organization of the human hand proved that, despite the complexity of the hand, a few parameters can describe most of the variance in the patterns of configurations and movements. In other words, humans exploit a reduced set of parameters, known in the literature as synergies, to control their hands. In robotics, this dimensionality reduction can be achieved by coupling some of the degrees of freedom (DoFs) of the robotic hand, that results in a reduction of the needed inputs. Such coupling can be obtained at the software level, exploiting mapping algorithm to reproduce human hand organization, and at the hardware level, through either rigid or compliant physical couplings between the joints of the robotic hand. This paper reviews the main solutions proposed for both the approaches.

Compensating Hand Function in Chronic Stroke Patients Through the Robotic Sixth Finger.

A novel solution to compensate hand grasping abilities is proposed for chronic stroke patients. The goal is to provide the patients with a wearable robotic extra-finger that can be worn on the paretic forearm by means of an elastic band. The proposed prototype, the Robotic Sixth Finger, is a modular articulated device that can adapt its structure to the grasped object shape. The extra-finger and the paretic hand act like the two parts of a gripper cooperatively holding an object. We evaluated the feasibility of the approach with four chronic stroke patients performing a qualitative test, the Frenchay Arm Test. In this proof of concept study, the use of the Robotic Sixth Finger has increased the total score of the patients by two points in a five points scale. The subjects were able to perform the two grasping tasks included in the test that were not possible without the robotic extra-finger. Adding a robotic opposing finger is a very promising approach that can significantly improve the functional compensation of the chronic stroke patient during everyday life activities.

Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications.

Although Augmented Reality (AR) has been around for almost five decades, only recently we have witnessed AR systems and applications entering in our everyday life. Representative examples of this technological revolution are the smartphone games "Pokémon GO" and "Ingress" or the Google Translate real-time sign interpretation app. Even if AR applications are already quite compelling and widespread, users are still not able to physically interact with the computer-generated reality. In this respect, wearable haptics can provide the compelling illusion of touching the superimposed virtual objects without constraining the motion or the workspace of the user. In this paper, we present the experimental evaluation of two wearable haptic interfaces for the fingers in three AR scenarios, enrolling 38 participants. In the first experiment, subjects were requested to write on a virtual board using a real chalk. The haptic devices provided the interaction forces between the chalk and the board. In the second experiment, subjects were asked to pick and place virtual and real objects. The haptic devices provided the interaction forces due to the weight of the virtual objects. In the third experiment, subjects were asked to balance a virtual sphere on a real cardboard. The haptic devices provided the interaction forces due to the weight of the virtual sphere rolling on the cardboard. Providing haptic feedback through the considered wearable device significantly improved the performance of all the considered tasks. Moreover, subjects significantly preferred conditions providing wearable haptic feedback.

A Human-Robot Interaction Perspective on Assistive and Rehabilitation Robotics.

Assistive and rehabilitation devices are a promising and challenging field of recent robotics research. Motivated by societal needs such as aging populations, such devices can support motor functionality and subject training. The design, control, sensing, and assessment of the devices become more sophisticated due to a human in the loop. This paper gives a human-robot interaction perspective on current issues and opportunities in the field. On the topic of control and machine learning, approaches that support but do not distract subjects are reviewed. Options to provide sensory user feedback that are currently missing from robotic devices are outlined. Parallels between device acceptance and affective computing are made. Furthermore, requirements for functional assessment protocols that relate to real-world tasks are discussed. In all topic areas, the design of human-oriented frameworks and methods is dominated by challenges related to the close interaction between the human and robotic device. This paper discusses the aforementioned aspects in order to open up new perspectives for future robotic solutions.

An EMG Interface for the Control of Motion and Compliance of a Supernumerary Robotic Finger.

In this paper, we propose a novel electromyographic (EMG) control interface to control motion and joints compliance of a supernumerary robotic finger. The supernumerary robotic fingers are a recently introduced class of wearable robotics that provides users additional robotic limbs in order to compensate or augment the existing abilities of natural limbs without substituting them. Since supernumerary robotic fingers are supposed to closely interact and perform actions in synergy with the human limbs, the control principles of extra finger should have similar behavior as human's ones including the ability of regulating the compliance. So that, it is important to propose a control interface and to consider the actuators and sensing capabilities of the robotic extra finger compatible to implement stiffness regulation control techniques. We propose EMG interface and a control approach to regulate the compliance of the device through servo actuators. In particular, we use a commercial EMG armband for gesture recognition to be associated with the motion control of the robotic device and surface one channel EMG electrodes interface to regulate the compliance of the robotic device. We also present an updated version of a robotic extra finger where the adduction/abduction motion is realized through ball bearing and spur gears mechanism. We have validated the proposed interface with two sets of experiments related to compensation and augmentation. In the first set of experiments, different bimanual tasks have been performed with the help of the robotic device and simulating a paretic hand since this novel wearable system can be used to compensate the missing grasping abilities in chronic stroke patients. In the second set, the robotic extra finger is used to enlarge the workspace and manipulation capability of healthy hands. In both sets, the same EMG control interface has been used. The obtained results demonstrate that the proposed control interface is intuitive and can successfully be used, not only to control the motion of a supernumerary robotic finger but also to regulate its compliance. The proposed approach can be exploited also for the control of different wearable devices that has to actively cooperate with the human limbs.

Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands.

The term 'synergy' - from the Greek synergia - means 'working together'. The concept of multiple elements working together towards a common goal has been extensively used in neuroscience to develop theoretical frameworks, experimental approaches, and analytical techniques to understand neural control of movement, and for applications for neuro-rehabilitation. In the past decade, roboticists have successfully applied the framework of synergies to create novel design and control concepts for artificial hands, i.e., robotic hands and prostheses. At the same time, robotic research on the sensorimotor integration underlying the control and sensing of artificial hands has inspired new research approaches in neuroscience, and has provided useful instruments for novel experiments. The ambitious goal of integrating expertise and research approaches in robotics and neuroscience to study the properties and applications of the concept of synergies is generating a number of multidisciplinary cooperative projects, among which the recently finished 4-year European project "The Hand Embodied" (THE). This paper reviews the main insights provided by this framework. Specifically, we provide an overview of neuroscientific bases of hand synergies and introduce how robotics has leveraged the insights from neuroscience for innovative design in hardware and controllers for biomedical engineering applications, including myoelectric hand prostheses, devices for haptics research, and wearable sensing of human hand kinematics. The review also emphasizes how this multidisciplinary collaboration has generated new ways to conceptualize a synergy-based approach for robotics, and provides guidelines and principles for analyzing human behavior and synthesizing artificial robotic systems based on a theory of synergies.