A Novel Master-Slave Interventional Surgery Robot with Force Feedback and Collaborative Operation.

In recent years, master-slave vascular robots have been developed to address the problem of radiation exposure during vascular interventions for surgeons. However, the single visual feedback reduces surgeon immersion and transparency of the system. In this work, we have developed a haptic interface based on the magnetorheological fluid (MRF) on the master side. The haptic interface can provide passive feedback force with high force fidelity and low inertia. Additionally, the manipulation of the master device does not change the operating posture of traditional surgery, which allows the surgeon to better adapt to the robotic system. For the slave robot, the catheter and guidewire can be navigated simultaneously which allows the two degrees of action on the catheter and axial action of a guidewire. The resistance force of the catheter navigation is measured and reflected to the user through the master haptic interface. To verify the proposed master-slave robotic system, the evaluation experiments are carried out in vitro, and the effectiveness of the system was demonstrated experimentally.

Bimanual Intravenous Needle Insertion Simulation Using Nonhomogeneous Haptic Device Integrated into Mixed Reality.

In this study, we developed a new haptic-mixed reality intravenous (HMR-IV) needle insertion simulation system, providing a bimanual haptic interface integrated into a mixed reality system with programmable variabilities considering real clinical environments. The system was designed for nursing students or healthcare professionals to practice IV needle insertion into a virtual arm with unlimited attempts under various changing insertion conditions (e.g., skin: color, texture, stiffness, friction; vein: size, shape, location depth, stiffness, friction). To achieve accurate hand-eye coordination under dynamic mixed reality scenarios, two different haptic devices (Dexmo and Geomagic Touch) and a standalone mixed reality system (HoloLens 2) were integrated and synchronized through multistep calibration for different coordinate systems (real world, virtual world, mixed reality world, haptic interface world, HoloLens camera). In addition, force-profile-based haptic rendering proposed in this study was able to successfully mimic the real tactile feeling of IV needle insertion. Further, a global hand-tracking method, combining two depth sensors (HoloLens and Leap Motion), was developed to accurately track a haptic glove and simulate grasping a virtual hand with force feedback. We conducted an evaluation study with 20 participants (9 experts and 11 novices) to measure the usability of the HMR-IV simulation system with user performance under various insertion conditions. The quantitative results from our own metric and qualitative results from the NASA Task Load Index demonstrate the usability of our system.

TactCube: An Intelligent Device to 'converse' with Smart Environments.

Ambient Intelligence is a vision of daily life in which intelligent devices interact with humans to make their lives easier, and the technology is invisible. Artificial Intelligence (AI) governs this smart environment and must interact with humans to best meet their needs and demands. Although voice assistants are very popular and efficient as conversational AI, under some conditions they cannot be used. Therefore, this work proposed a complementary tactile and tangible interface to converse with AI, creating new Tactile Signs. A prototype of TactCube, a wireless cube-shaped device that can interact with AI using only the tactile sense, is presented. The hypothesis is that TactCube can be manipulated with one hand and generate a sequence of numbers that can be interpreted as a new tactile language by a neural network solution. The paper describes the initial research made to define how these sequences can be generated and demonstrates how TactCube is able to do it.

Evaluation of Physical Interaction during Walker-Assisted Gait with the AGoRA Walker: Strategies Based on Virtual Mechanical Stiffness.

Smart walkers are commonly used as potential gait assistance devices, to provide physical and cognitive assistance within rehabilitation and clinical scenarios. To understand such rehabilitation processes, several biomechanical studies have been conducted to assess human gait with passive and active walkers. Several sessions were conducted with 11 healthy volunteers to assess three interaction strategies based on passive, low and high mechanical stiffness values on the AGoRA Smart Walker. The trials were carried out in a motion analysis laboratory. Kinematic data were also collected from the smart walker sensory interface. The interaction force between users and the device was recorded. The force required under passive and low stiffness modes was 56.66% and 67.48% smaller than the high stiffness mode, respectively. An increase of 17.03% for the hip range of motion, as well as the highest trunk's inclination, were obtained under the resistive mode, suggesting a compensating motion to exert a higher impulse force on the device. Kinematic and physical interaction data suggested that the high stiffness mode significantly affected the users' gait pattern. Results suggested that users compensated their kinematics, tilting their trunk and lower limbs to exert higher impulse forces on the device.

Periodic Kinesthetic Guidance Cannot Expedite Learning Surgical Skills.

Introduction. Connecting multiple haptic devices in a master-slave fashion enables us to deliver kinesthetic (haptic) feedback from 1 person to another. This study examined whether inter-user feedback delivered from an expert to a novice would facilitate skill acquisition of the novice in learning laparoscopic surgery and expedite it compared to traditional methods. Methods. We recruited fourteen novices and divided them into 1 of 2 training groups with 6 half-hour training sessions. The task was precision cutting adopted from one of the tasks listed in Fundamentals of Laparoscopic Surgery using laparoscopic instruments. In the haptic feedback group (haptic), 8 subjects had the chance to passively feel an expert's performance before they started to practice in each training session. In the self-learning group (control), 6 subjects watched a video before practicing. Each session was video recorded, and task performance was measured by task completion time, number of grasper adjustments, and instrument crossings. Cutting accuracy, defined as the percentage of deviation of the cutting line from the predefined line, was analyzed via computer analysis. Results. Results show no significant difference among performance measures between the 2 groups. Participants performed similarly when practicing alone or with periodic haptic feedback. Discussion. Further research will be needed for improving our way of integrating between-person haptic feedback with skills training protocol.

Effects of Asymmetric Vibration Frequency on Pulling Illusions.

It is known that humans experience a haptic illusion, such as the sensation of being pulled in a particular direction, when asymmetric vibrations are presented. A pulling illusion has been used to provide a force feedback for a virtual reality (VR) system and a pedestrian navigation system, and the asymmetric vibrations can be implemented in any small non-grounded device. However, the design methodology of asymmetric vibration stimuli to induce the pulling illusion has not been fully demonstrated. Although the frequency of the asymmetric vibration is important, findings on the frequency have not been reported. In this study, we clarified the influences of the effects on the pulling illusion based on the investigation of asymmetric vibration frequency differences. Two psychophysical experiments that related to the frequency of asymmetric vibration were performed. Experiment I showed that the illusion occurs for specific vibration waveforms at 40 Hz and 75 Hz. As a result of Experiment II, the threshold was the lowest when the frequency was 40 Hz, and highest when the frequency was 110 Hz. This result supports the previous hypothesis that the Meissner corpuscles and the Ruffini endings contribute to the illusion, while the Pacinian corpuscles do not.

Bio-inspired Design and Additive Manufacturing of Soft Materials, Machines, Robots, and Haptic Interfaces.

Soft materials possess several distinctive characteristics, such as controllable deformation, infinite degrees of freedom, and self-assembly, which make them promising candidates for building soft machines, robots, and haptic interfaces. In this Review, we give an overview of recent advances in these areas, with an emphasis on two specific topics: bio-inspired design and additive manufacturing. Biology is an abundant source of inspiration for functional materials and systems that mimic the function or mechanism of biological tissues, agents, and behaviors. Additive manufacturing has enabled the fabrication of materials and structures prevalent in biology, thereby leading to more-capable soft robots and machines. We believe that bio-inspired design and additive manufacturing have been, and will continue to be, important tools for the design of soft robots.

An Electroactive and Transparent Haptic Interface Utilizing Soft Elastomer Actuators with Silver Nanowire Electrodes.

An electroactive and transparent haptic interface having a rectangular void pattern creates tunable surface textures by controlling the wavelength and amplitude of independent void-lines. To make an active tactile surface, the transparent haptic interface employs a silver nanowire (AgNW) electrode to be compliant with the deformed elastomer surface. Here, the dielectric elastomer is newly blended with polydimethylsiloxane and Ecoflex prepolymer to simultaneously control the mechanical stiffness and transparency. The relative resistance of the AgNW electrode on a single void line is nearly unchanged under bending test, confirming the high stretchability and conductivity of the nanowire-networked electrode. The optical transparencies are 92-85%, depending on the ratio of the Ecoflex solution. Transparency values decreas by 7 and 16% after coating with AgNWs at densities of 30 and 140 mg m-2 , respectively. Using EP31, the void line is deformed by 90 µm under a field intensity of 13.0 V µm-1 . The haptic surface is successfully controlled by applying voltage, which produces four different surface textures, from relatively smooth to rough feeling, depending on the distance between deformed void lines. This haptic interface can be applied to diverse display systems as an external add-on screen and will help to realize programmable surface textures in the future.

Performance evaluation of a robot-assisted catheter operating system with haptic feedback.

In this paper, a novel robot-assisted catheter operating system (RCOS) has been proposed as a method to reduce physical stress and X-ray exposure time to physicians during endovascular procedures. The unique design of this system allows the physician to apply conventional bedside catheterization skills (advance, retreat and rotate) to an input catheter, which is placed at the master side to control another patient catheter placed at the slave side. For this purpose, a magnetorheological (MR) fluids-based master haptic interface has been developed to measure the axial and radial motions of an input catheter, as well as to provide the haptic feedback to the physician during the operation. In order to achieve a quick response of the haptic force in the master haptic interface, a hall sensor-based closed-loop control strategy is employed. In slave side, a catheter manipulator is presented to deliver the patient catheter, according to position commands received from the master haptic interface. The contact forces between the patient catheter and blood vessel system can be measured by designed force sensor unit of catheter manipulator. Four levels of haptic force are provided to make the operator aware of the resistance encountered by the patient catheter during the insertion procedure. The catheter manipulator was evaluated for precision positioning. The time lag from the sensed motion to replicated motion is tested. To verify the efficacy of the proposed haptic feedback method, the evaluation experiments in vitro are carried out. The results demonstrate that the proposed system has the ability to enable decreasing the contact forces between the catheter and vasculature.

Graphical message transmission using the monotonic vibration function of a smart phone.

This study aimed at evaluating four vibrotactile capabilities for perceiving graphical information presented on a smart phone. Thirty-two blindfolded college students participated in four experiments to test their capabilities of two-point discrimination, relative and absolute judgments of line thickness, and recognition of basic shapes. All the information was received through the default vibration function of the phone, sensed by their scanning fingers. The results showed a good two-point discrimination accuracy rate, reaching 98.8% when the distance between two points was set at 3.2 mm; the relative judgment of line thickness reached the level of 78.3% accuracy when the two-line width difference ratio was set at 3%; the absolute judgment reached the level of 78.8% when the participants recognised line thickness from one of two. Overall, especially for the shapes judgment, the information transmitted by the various codes may be quite low. These findings should inspire advanced investigations and provide design guidelines. PRACTITIONER SUMMARY: This study tested four vibrotactile capabilities for perception of graphical information when solely using the monotonic vibration function of a smart phone. The results show low information transmission. These findings encourage advanced investigations of new coding systems so that relevant mobile applications could be developed to help the visually impaired.

MARSBot: A Bristle-Bot Microrobot with Augmented Reality Steering Control for Wireless Structural Health Monitoring.

Microrobots are effective for monitoring infrastructure in narrow spaces. However, they have limited computing power, and most of them are not wireless and stable enough for accessing infrastructure in difficult-to-reach areas. In this paper, we describe the fabrication of a microrobot with bristle-bot locomotion using a novel centrifugal yaw-steering control scheme. The microrobot operates in a network consisting of an augmented reality headset and an access point to monitor infrastructures using augmented reality (AR) haptic controllers for human-robot collaboration. For the development of the microrobot, the dynamics of bristle-bots in several conditions were studied, and multiple additive manufacturing processes were investigated to develop the most suitable prototype for structural health monitoring. Using the proposed network, visual data are sent in real time to a hub connected to an AR headset upon request, which can be utilized by the operator to monitor and make decisions in the field. This allows the operators wearing an AR headset to inspect the exterior of a structure with their eyes, while controlling the surveying robot to monitor the interior side of the structure.

Potential Haptic Perceptual Dimensionality of Rendered Compliance.

Studies have proven that humans perceive haptic textures through different perceptual dimensions, such as rough/smooth and soft/hard, which provide useful guidance in the design of haptic devices. However, few of these have focused on the perception of compliance, which is another important perceptual property in haptic interfaces. This research was conducted to investigate the potential basic perceptual dimensions of the rendered compliance and quantify the effects of the simulation parameters. Two perceptual experiments were designed based on 27 stimuli samples generated by a 3-DOF haptic feedback device. Subjects were asked to describe these stimuli using adjectives, classify the samples, and rate them according to corresponding adjective labels. Multi-dimensional scaling (MDS) methods were then used to project adjective ratings into 2D and 3D perception spaces. According to the results, hardness and viscosity are considered two basic perceptual dimensions of the rendered compliance, while crispness can be regarded as a subsidiary perceptual dimension. Then, the relations between simulation parameters and perceptual feelings were analyzed by the regression analysis. This paper may provide a better understanding of the compliance perception mechanism and useful guidance for the improvement of rendering algorithms and devices for haptic human-computer interaction.

Enhancing skill learning with dual-user haptic feedback: insights from a task-specific approach.

Introduction: This study was to examine whether inter-user haptic feedback would have a differential impact on skill acquisition based on the nature of the surgical task involved. Specifically, we hypothesized that haptic feedback would facilitate target orientation more than cutting tasks in the context of laparoscopic surgery. Methods: Ten novice participants were recruited and assigned to one of two training groups. Each group underwent six half-hour training sessions dedicated to laparoscopic pattern-cutting tasks. In the haptic group, five participants received expert guidance during the training sessions, whereas the remaining five participants in the control group engaged in self-practice. All trials were recorded on video, enabling a comparative analysis of task performance between the participants' left hand (target manipulation) and right hand (cutting task). Additionally, the number of haptic feedback instances provided to the trainees in the haptic group was recorded. Results: Practice led to a reduction in total task time, grasping time, and cutting errors. However, no significant differences were observed between the two training groups, except for the grasping time, where haptic feedback significantly reduced the grasping time compared to the control group. Moreover, the frequency of haptic feedback instances provided to the trainees was notably higher for the grasping than for the cutting task. Discussion: Our study suggests that haptic feedback has a more substantial impact on orientation tasks than on cutting tasks in laparoscopic surgery training. However, we acknowledge that a larger sample size would provide a more robust evaluation of this effect.

PREDICTOR: A Physical emulatoR enabling safEty anD ergonomICs evaluation and Training of physical human-rObot collaboRation.

Safety and ergonomics of Physical Human-Robot Collaboration (PHRC) are crucial to make human-robot collaborative systems trustworthy and make a significant impact in real-world applications. One big obstacle to the development of relevant research is the lack of a general platform for evaluating the safety and ergonomics of proposed PHRC systems. This paper aims to create a Physical emulatoR enabling safEty anD ergonomICs evaluation and Training of physical human-rObot collaboRation (PREDICTOR). PREDICTOR consists of a dual-arm robot system and a VR headset as its hardware and contains physical simulation, haptic rendering and visual rendering modules as its software. The dual-arm robot system is used as an integrated admittance-type haptic device, which senses the force/torque applied by a human operator as an input to drive the simulation of a PHRC system and constrains the handles' motion to match their virtual counterparts in the simulation. The motion of the PHRC system in the simulation is fed back to the operator through the VR headset. PREDICTOR combines haptics and VR to emulate PHRC tasks in a safe environment since the interactive forces are monitored to avoid any risky events. PREDICTOR also brings flexibility as different PHRC tasks can be easily set up by changing the PHRC system model and the robot controller in the simulation. The effectiveness and performance of PREDICTOR were evaluated by experiments.

Design, characterisation and validation of a haptic interface based on twisted string actuation.

This paper presents the design and experimental characterisation of a wrist haptic interface based on a twisted string actuator. The interface is designed for controlled actuation of wrist flexion/extension and is capable of rendering torque feedback through a rotary handle driven by the twisted string actuator and spring-loaded cable mechanisms. The interface was characterised to obtain its static and dynamic haptic feedback rendering capabilities. Compliance in the spring and actuation mechanism makes the interface suitable for smooth rendering of haptic feedback of large magnitudes due to the high motion transmission ratio of the twisted strings. Haptic virtual wall rendering capabilities are demonstrated.

Perception by Palpation: Development and Testing of a Haptic Ferrogranular Jamming Surface.

Tactile hands-only training is particularly important for medical palpation. Generally, equipment for palpation training is expensive, static, or provides too few study cases to practice on. We have therefore developed a novel haptic surface concept for palpation training, using ferrogranular jamming. The concept's design consists of a tactile field spanning 260 x 160 mm, and uses ferromagnetic granules to alter shape, position, and hardness of palpable irregularities. Granules are enclosed in a compliant vacuum-sealed chamber connected to a pneumatic system. A variety of geometric shapes (output) can be obtained by manipulating and arranging granules with permanent magnets. The tactile hardness of the palpable output can be controlled by adjusting the chamber's vacuum level. A psychophysical experiment (N = 28) investigated how people interact with the palpable surface and evaluated the proposed concept. Untrained participants characterized irregularities with different position, form, and hardness through palpation, and their performance was evaluated. A baseline (no irregularity) was compared to three irregularity conditions: two circular shapes with different hardness (Hard Lump and Soft Lump), and an Annulus shape. 100% of participants correctly identified an irregularity in the three irregularity conditions, whereas 78.6% correctly identified baseline. Overall agreement between participants was high (κ= 0.723). The Intersection over Union (IoU) for participants sketched outline over the actual shape was IoU Mdn = 79.3% for Soft Lump, IoU Mdn = 68.8% for Annulus, and IoU Mdn = 76.7% for Hard Lump. The distance from actual to drawn center was Mdn = 6.4 mm (Soft Lump), Mdn = 5.3 mm (Annulus), and Mdn = 7.4 mm (Hard Lump), which are small distances compared to the size of the field. The participants subjectively evaluated Soft Lump to be significantly softer than Hard Lump and Annulus. Moreover, 71% of participants thought they improved their palpation skills throughout the experiment. Together, these results show that the concept can render irregularities with different position, form, and hardness, and that users are able to locate and characterize these through palpation. Participants experienced an improvement in palpation skills throughout the experiment, which indicates the concepts feasibility as a palpation training device.

Haptic Glove Using Tendon-Driven Soft Robotic Mechanism.

Recent advancements in virtual reality and augmented reality call for light-weight and compliant haptic interfaces to maximize the task-performance interactivity with the virtual or extended environment. Noting this, we propose a haptic glove using a tendon-driven compliant robotic mechanism. Our proposed interface can provide haptic feedback to two fingers of a user, an index finger and a thumb. It can provide both cutaneous and kinesthetic feedback to the fingers by using the tendon-driven system. Each actuator is paired with a force sensor to exert the desired tension accurately. In order to optimize the perception of the kinesthetic feedback, we propose a perception-based kinesthetic feedback distribution strategy. We experimentally measured the force perception weight for peripheral interphalangeal (PIP) and metacarpophalangeal (MCP) joints. We observed no significant difference in the force perception between the two joints. Then, based on the obtained weights, our proposed force distribution method calculates the force for each joint. We also evaluated the effect of additional cutaneous feedback to the kinesthetic feedback, on the force perception at the fingertip. The experimental result has shown that additional cutaneous feedback has significantly increased the sensitivity of the human perception. Finally, we evaluated our proposed system and force distribution algorithm by conducting a human subject test. The experimental result indicates that the availability of the cutaneous feedback significantly improved the perceived realism and acuity of the contact force.

Corrigendum: Haptic Glove Using Tendon-Driven Soft Robotic Mechanism.

[This corrects the article DOI: 10.3389/fbioe.2020.541105.].

Evaluation of haptic devices and end-users: Novel performance metrics in tele-robotic microsurgery.

BACKGROUND: Here, we present performance evaluation methodology that distinguishes the performance of a haptic device from end-user skill level in a tele-robotic system. METHODS: A pick-&-place experiment was designed and eight participants micromanipulated cotton strips, similar to maneuvers performed during microsurgery. Using three nonredundant haptic devices: neuroArmPLUS HD , a custom developed master manipulator, and two commercially available products, sigma.7 and HD2 , several features including the speed, effort, consistency, hand/gimbal agility, and force characteristics were measured and recorded for each participant and device. RESULTS: The participants showed variable skill level. For consistency, hand/gimbal agility and force characteristics, they performed significantly better when using neuroArmPLUS HD prototype. Based on the experimental data, performance metrics for both the device and the end-users were established. CONCLUSIONS: The integrated performance metrics allows independent evaluation of both the user and haptic device, thereby quantifying human-machine interactions.

Load cell torques and force data collection during tele-operated robotic gas tungsten arc welding in presence of collisions.

Torque and force signals data were acquired from a load-cell sensor during a robotic welding process, in presence of collisions between the tool and the workpiece edges outlined in part in "Haptic-based touch detection for collaborative robots in welding applications" [1]. The dataset is composed from 15 tests captured during a tele-operated welding robot performing a 1G ASME/AWS (i.e., PA ISO) welding process. The raw data files have been provided. These data can be used to correlate torque signal features with collision events, to improve algorithms of collision detection/avoidance and to develop reliable real-time haptic feedback to the welder. This dataset can also be used to study the torque signal variation in different welding positions (e.g., 2G, 3G, 2F, etc.). Dataset is provided as raw data and in MATLAB files.