Preoperative interactive virtual simulation applying three-dimensional multifusion images using a haptic device for lumbosacral lipoma.

PURPOSE: Untethering surgery for lumbosacral lipoma is a preventive procedure, and avoidance of complications and good long-term outcomes are required. We introduced presurgical interactive virtual simulation (IVS) applying three-dimensional multifusion images using a haptic device aimed at improving operative outcomes. METHODS: Fourteen patients with newly diagnosed lumbosacral lipoma were recruited and underwent preoperative IVS. The median age at surgery was 8 months. A three-dimensional image analysis system was used to extract and fuse structures necessary for surgery, such as the lipoma, spinal cord and skin, from CT and MRI, and create three-dimensional multifusion images. The created images were individually converted to standard triangulated language format and loaded onto a workstation (Geomagic freeform™) that could be freely transformed, and the laminectomy range and lipoma extraction procedure were examined. Presurgical IVS was performed, and the actual surgery was performed. RESULTS: The disease types were dorsal, caudal, lipomyelomeningocele, transitional, and filum in 5, 5, 2, 1, and 1 patients, respectively. The surgical procedure and extent of the laminectomy were as planned for all patients. Resection of the lipomas tended to be less than expected preoperatively because of positive reactions on intraoperative monitoring. No postoperative complications were observed. The median postoperative follow-up period was 29 months, and there were no reoperations during the observation period. CONCLUSIONS: Although there are various types of lumbosacral lipoma, surgery can be safely performed by performing presurgical IVS. The short-term course is good; however, long-term follow-up is necessary for the appearance of neurological symptoms associated with growth and re-tethering.

Intuitive Cell Manipulation Microscope System with Haptic Device for Intracytoplasmic Sperm Injection Simplification.

In recent years, the demand for effective intracytoplasmic sperm injection (ICSI) for the treatment of male infertility has increased. The ICSI operation is complicated as it involves delicate organs and requires a high level of skill. Several cell manipulation systems that do not require such skills have been proposed; notably, several automated methods are available for cell rotation. However, these methods are unfeasible for the delicate ICSI medical procedure because of safety issues. Thus, this study proposes a microscopic system that enables intuitive micropipette manipulation using a haptic device that safely and efficiently performs the entire ICSI procedure. The proposed system switches between field-of-view expansion and three-dimensional image presentation to present images according to the operational stage. In addition, the system enables intuitive pipette manipulation using a haptic device. Experiments were conducted on microbeads instead of oocytes. The results confirmed that the time required for the experimental task was improved by 52.6%, and the injection error was improved by 75.3% compared to those observed in the conventional system.

Recent Developments in Haptic Devices Designed for Hearing-Impaired People: A Literature Review.

Haptic devices transmit information to the user, using tactile stimuli to augment or replace sensory input. People with limited sensory abilities, such as vision or hearing can receive supplementary information by relying on them. This review analyses recent developments in haptic devices for deaf and hard-of-hearing individuals by extracting the most relevant information from each of the selected papers. The process of finding relevant literature is detailed using the PRISMA guidelines for literature reviews. In this review, the devices are categorized to better understand the review topic. The categorization results have highlighted several areas of future research into haptic devices for hearing-impaired users. We believe this review may be useful to researchers interested in haptic devices, assistive technologies, and human-computer interaction.

Bimanual Intravenous Needle Insertion Simulation Using Nonhomogeneous Haptic Device Integrated into Mixed Reality.

In this study, we developed a new haptic-mixed reality intravenous (HMR-IV) needle insertion simulation system, providing a bimanual haptic interface integrated into a mixed reality system with programmable variabilities considering real clinical environments. The system was designed for nursing students or healthcare professionals to practice IV needle insertion into a virtual arm with unlimited attempts under various changing insertion conditions (e.g., skin: color, texture, stiffness, friction; vein: size, shape, location depth, stiffness, friction). To achieve accurate hand-eye coordination under dynamic mixed reality scenarios, two different haptic devices (Dexmo and Geomagic Touch) and a standalone mixed reality system (HoloLens 2) were integrated and synchronized through multistep calibration for different coordinate systems (real world, virtual world, mixed reality world, haptic interface world, HoloLens camera). In addition, force-profile-based haptic rendering proposed in this study was able to successfully mimic the real tactile feeling of IV needle insertion. Further, a global hand-tracking method, combining two depth sensors (HoloLens and Leap Motion), was developed to accurately track a haptic glove and simulate grasping a virtual hand with force feedback. We conducted an evaluation study with 20 participants (9 experts and 11 novices) to measure the usability of the HMR-IV simulation system with user performance under various insertion conditions. The quantitative results from our own metric and qualitative results from the NASA Task Load Index demonstrate the usability of our system.

An Overview of Wearable Haptic Technologies and Their Performance in Virtual Object Exploration.

We often interact with our environment through manual handling of objects and exploration of their properties. Object properties (OP), such as texture, stiffness, size, shape, temperature, weight, and orientation provide necessary information to successfully perform interactions. The human haptic perception system plays a key role in this. As virtual reality (VR) has been a growing field of interest with many applications, adding haptic feedback to virtual experiences is another step towards more realistic virtual interactions. However, integrating haptics in a realistic manner, requires complex technological solutions and actual user-testing in virtual environments (VEs) for verification. This review provides a comprehensive overview of recent wearable haptic devices (HDs) categorized by the OP exploration for which they have been verified in a VE. We found 13 studies which specifically addressed user-testing of wearable HDs in healthy subjects. We map and discuss the different technological solutions for different OP exploration which are useful for the design of future haptic object interactions in VR, and provide future recommendations.

Arthrodesis Versus Arthroplasty in Thumb Carpometacarpal Osteoarthritis: Impact on Maximal Voluntary Force, Endurance, and Accuracy of Pinch.

PURPOSE: To investigate differences in pinch strength recovery among patients with first carpometacarpal joint osteoarthritis treated with either arthrodesis or suspension arthroplasty. METHODS: Thirty-seven subjects who underwent arthrodesis or suspension arthroplasty for carpometacarpal osteoarthritis were included. Force exerted during maximal voluntary contraction (MVC) in a pinch task was measured. Maximal voluntary contraction was recorded using a haptic device equipped with a load cell from which an analog signal was acquired and digitized for visual feedback. Dynamic force was assessed by a task consisting of 10 repetitions, with a target of 70% of MVC. Endurance was assessed by the length of sustained pinch task at 30% of MVC. Task performance was quantified by mean distance and offset error from the target force as error indices, and standard deviation of force was used as index of force steadiness. RESULTS: The arthrodesis group obtained considerably higher MVC values than the arthroplasty group. No notable differences were found for pinch endurance. The standard deviation for dynamic force was lower for arthroplasty, indicating greater force steadiness. CONCLUSIONS: Arthrodesis is associated with greater pinch strength. Arthroplasty is associated with better pinch precision. TYPE OF STUDY/LEVEL OF EVIDENCE: Therapeutic IV.

A Cost-Effective, Integrated Haptic Device for an Exoskeletal System.

The paper presents an innovative integrated sensor-effector designed for use in exoskeletal haptic devices. The research efforts aimed to achieve high cost-effectiveness for a design assuring proper monitoring of joint rotations and providing passive force feedback. A review of market products revealed that there is space for new designs of haptic devices with such features. To determine the feasibility of the proposed solution, a series of simulations and experiments were conducted to verify the adopted design concept. The focus was set on an investigation of the force of attraction between one and two magnets interacting with a steel plate. Further, a physical model of an integrated joint was fabricated, and its performance was evaluated and compared to a similar commercially available device. The proposed solution is cost-effective due to the use of standard parts and inexpensive components. However, it is light and assures a 19 Nm braking torque adequate for the intended use as a haptic device for upper limbs.

Development of a Wearable Haptic Glove Presenting Haptic Sensation by Electrical Stimulation.

Most haptic devices generate haptic sensation using mechanical actuators. However, the workload and limited workspace handicap the operator from operating freely. Electrical stimulation is an alternative approach to generate haptic sensations without using mechanical actuators. The light weight of the electrodes adhering to the body brings no limitations to free motion. Because a real haptic sensation consists of feelings from several areas, mounting the electrodes to several different body areas can make the sensations more realistic. However, simultaneously stimulating multiple electrodes may result in "noise" sensations. Moreover, the operators may feel tingling because of unstable stimulus signals when using the dry electrodes to help develop an easily mounted haptic device using electrical stimulation. In this study, we first determine the appropriate stimulation areas and stimulus signals to generate a real touch sensation on the forearm. Then, we propose a circuit design guideline for generating stable electrical stimulus signals using a voltage divider resistor. Finally, based on the aforementioned results, we develop a wearable haptic glove prototype. This haptic glove allows the user to experience the haptic sensations of touching objects with five different degrees of stiffness.

Individuals with and without Visual Impairments Use a Force Feedback Device to Identify the Friction and Hardness of Haptic Surfaces.

The general purpose of this study is to promote access to haptic virtual environments. Using a haptic device, people with and without visual impairments (VI) are able to feel different textures and compare these textures based on different surface properties, i.e., friction and hardness. The objectives of this study were to examine the following: (a) whether the variables of friction and hardness were identifiable through the Touch device (Phantom Omni) and could therefore function as 3D haptic variables; (b) if there were differences between people with VI and sighted individuals in terms of their performance; (c) the differences that should exist between the values of each variable so that the virtual surfaces could be identified as different to each other; and (d) if the individual characteristics of participants have an impact on their performance. The results showed that it is necessary to use surfaces which are differentiated based on the degree of friction and hardness because the haptic properties of a virtual object are then better perceived. Individuals with VI need more time and more effort to understand friction and hardness, respectively. With the motivation of increasing access to object perception for people with VI in a virtual environment, accessibility advisors and experts can extract useful information for the development of functional and efficient 3D objects for haptic perception.

Differences in adults' spatial scaling based on visual or haptic information.

The present study examined differences in adults' spatial-scaling abilities across three perceptual conditions: (1) visual, (2) haptic, and (3) visual and haptic. Participants were instructed to encode the position of a convex target presented in a simple map without a time limit. Immediately after encoding the map, participants were presented with a referent space and asked to place a disc at the same location from memory. All spaces were designed as tactile graphics. Positions of targets varied along the horizontal dimension. The referent space was constant in size while sizes of maps were systematically varied, resulting in three scaling factor conditions: 1:4, 1:2, 1:1. Sixty adults participated in the study (M = 21.18; SD = 1.05). One-third of them was blindfolded throughout the entire experiment (haptic condition). The second group of participants was allowed to see the graphics (visual condition); the third group were instructed to see and touch the graphics (bimodal condition). An analysis of participants' absolute errors showed that participants produced larger errors in the haptic condition as opposed to the visual and bimodal conditions. There was also a significant interaction effect between scaling factor and perceptual condition. In the visual and bimodal conditions, results showed a linear increase in errors with higher scaling factors (which may suggest that adults adopted mental transformation strategies during the spatial scaling process), whereas, in the haptic condition, this relation was quadratic. Findings imply that adults' spatial-scaling performance decreases when visual information is not available.

iCub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots.

We present an avatar system designed to facilitate the embodiment of humanoid robots by human operators, validated through iCub3, a humanoid developed at the Istituto Italiano di Tecnologia. More precisely, the paper makes two contributions: First, we present the humanoid iCub3 as a robotic avatar that integrates the latest significant improvements after about 15 years of development of the iCub series. Second, we present a versatile avatar system enabling humans to embody humanoid robots encompassing aspects such as locomotion, manipulation, voice, and facial expressions with comprehensive sensory feedback including visual, auditory, haptic, weight, and touch modalities. We validated the system by implementing several avatar architecture instances, each tailored to specific requirements. First, we evaluated the optimized architecture for verbal, nonverbal, and physical interactions with a remote recipient. This testing involved the operator in Genoa and the avatar in the Biennale di Venezia, Venice-about 290 kilometers away-thus allowing the operator to visit the Italian art exhibition remotely. Second, we evaluated the optimized architecture for recipient physical collaboration and public engagement on stage, live, at the We Make Future show, a prominent world digital innovation festival. In this instance, the operator was situated in Genoa while the avatar operated in Rimini-about 300 kilometers away-interacting with a recipient who entrusted the avatar with a payload to carry on stage before an audience of approximately 2000 spectators. Third, we present the architecture implemented by the iCub Team for the All Nippon Airways (ANA) Avatar XPrize competition.

Multimodal Haptic Feedback for Virtual Collisions Combining Vibrotactile and Electrical Muscle Stimulation.

In this paper, we explore the effects of multimodal haptic feedback combining vibrotactile and electrical muscle stimulation (EMS) on expressing virtual collisions. We first present a wearable multimodal haptic device capable of generating both mechanical vibration and EMS stimuli. The two types of haptic stimulus are combined into a haptic rendering method that conveys improved virtual collision sensations. This multimodal rendering method highlights the strengths of each modality while compensating for mutual weaknesses. The multimodal rendering method is compared in subjective quality with two unimodal methods (vibration only and EMS only) by a user study. Experimental results demonstrate that our multimodal feedback method can elicit more realistic, enjoyable, expressive, and preferable user experiences.

Haptic Interactions Subject to Variable Latency.

Model-Mediated Teleoperation (MMT) between a haptic device and a remote or virtual environment uses a local model of the environment to compensate for latency of communication. MMT is often case-specific, and requires underlying latency distributions to be known. We propose a novel approach - which we refer to as the DelayRIM - which uses the time-stepping aspect of a Reduced Interface Model for the environment to render an up-to-date force to the haptic device from the delayed information. RIM is applicable to any physical or virtual system, and the DelayRIM itself makes no underlying assumption about the latency distribution. We show that for realistic variable delays, the DelayRIM improves transparency compared to other methods for a virtual drone bilateral teleoperation scenario.

Cable-Driven Haptic Interface With Movable Bases Achieving Maximum Workspace and Isotropic Force Exertion.

Haptic interactions play an essential role in education to enhance learning efficiency; however, haptic information for virtual educational content remains lacking. This article proposes a planar cable-driven haptic interface with movable bases that can display isotropic force feedback with maximum workspace extension on a commercial screen display. A generalized kinematic and static analysis of the cable-driven mechanism is derived by considering movable pulleys. Based on the analyses, a system including movable bases is designed and controlled to maximize the workspace subject to isotropic force exertion for the target screen area. The proposed system is evaluated experimentally as a haptic interface represented by the workspace, isotropic force-feedback range, bandwidth, Z-width, and user experiment. The results indicate that the proposed system can maximize workspace to the target rectangular area and exert isotropic force up to 94.0% of the theoretical computed one within the workspace.

Design and Assessment of a Bimanual Haptic Epidural Needle Insertion Simulator.

The case experience of anesthesiologists is one of the leading causes of accidental dural punctures and failed epidurals-the most common complications of epidural analgesia used for pain relief during delivery. We designed a bimanual haptic simulator to train anesthesiologists and optimize epidural analgesia skill acquisition. We present an assessment study conducted with 22 anesthesiologists of different competency levels from several Israeli hospitals. Our simulator emulates the forces applied to the epidural (Touhy) needle, held by one hand, and those applied to the Loss of Resistance (LOR) syringe, held by the other one. The resistance is calculated based on a model of the epidural region layers parameterized by the weight of the patient. We measured the movements of both haptic devices and quantified the results' rate (success, failed epidurals, and dural punctures), insertion strategies, and the participants' answers to questionnaires about their perception of the simulation realism. We demonstrated good construct validity by showing that the simulator can distinguish between real-life novices and experts. Face and content validity were examined by studying users' impressions regarding the simulator's realism and fulfillment of purpose. We found differences in strategies between different level anesthesiologists, and suggest trainee-based instruction in advanced training stages.

Macro-Fiber Composite-Based Tactors for Haptic Applications.

Haptic technology is a critical component of human-computer interfaces. Traditional haptic actuators are often unable to provide the broad frequency range and latency that is required in many advanced applications. To address this problem, we propose a new type of tactor based on macro-fiber composites (MFCs), composites of piezoelectric fibers. We propose a physics-based model for the actuation of the tactors, calibrated and validated through experiments. As our tactors are intended for haptic applications, we consider the role of skin on their response, an aspect seldom analyzed in the literature. In our experiments, we simulate the presence of the skin with a rubber membrane in contact with the tactor, with varying pre-stretch, mimicking different indentations of the tactor on the skin. The MFC-based tactor can always generate vibration amplitudes higher than skin discrimination thresholds, over the range of frequencies of interest for haptics, with a latency much smaller than traditional actuators. We theoretically investigate the effect of the skin on tactor vibrations, highlighting the individual roles of skin stiffness and damping and their combined effect across a series of pre-stretches. Our tactor shows promise in haptic applications, including assistive technologies and real-time feedback systems for training, safety, and monitoring.

Easy-To-Wear Auxetic SMA Knot-Architecture for Spatiotemporal and Multimodal Haptic Feedbacks.

Wearable haptic interfaces prioritize user comfort, but also value the ability to provide diverse feedback patterns for immersive interactions with the virtual or augmented reality. Here, to provide both comfort and diverse tactile feedback, an easy-to-wear and multimodal wearable haptic auxetic fabric (WHAF) is prepared by knotting shape-memory alloy wires into an auxetic-structured fabric. This unique meta-design allows the WHAF to completely expand and contract in 3D, providing superior size-fitting and shape-fitting capabilities. Additionally, a microscale thin layer of Parylene is coated on the surface to create electrically separated zones within the WHAF, featuring zone-specified actuation for conveying diverse spatiotemporal information to users with using the WHAF alone. Depending on the body part it is worn on, the WHAF conveys either cutaneous or kinesthetic feedback, thus, working as a multimodal wearable haptic interface. As a result, when worn on the forearm, the WHAF intuitively provides spatiotemporal information to users during hands-free navigation and teleoperation in virtual reality, and when worn on the elbow, the WHAF guides users to reach the desired elbow flexion, like a personal exercise advisor.

Human Factors in Sensory Feedback for Haptic Gloves.

Manual therapy training requires close proximity between the clinical teacher and students, which limits the training of people in remote and rural regions. Video-based online training can provide visual but not tactile information, which is also essential for manual therapies. This project describes the development and testing of an inexpensive sensor glove developed using commercially available sensors, suitable for monitoring the shape and force applied by the hand of a person delivering a spinal manipulation. Its focus was the development of software to provide the human user with tactile information that is usually acquired intuitively in face-to-face teaching. Though rigorous assessment of the glove's application showed errors at low levels of force in actual force measurement and interpretation by users, these errors were reduced at higher levels of force. Trainers of spinal manipulation reported the device to be very useful and suitable for the purpose. We conclude that this glove has the potential for being used for online training of students.Clinical Impact: The outcome of this study shows the feasibility of developing an inexpensive haptic glove using proprietary software for online training of students of manual therapy.

Distally-referred surface electrical nerve stimulation (DR-SENS) for haptic feedback.

Objective.This study's objective is to understand distally-referred surface electrical nerve stimulation (DR-SENS) and evaluates the effects of electrode placement, polarity, and stimulation intensity on the location of elicited sensations in non-disabled individuals.Approach.A two-phased human experiment was used to characterize DR-SENS. In Experiment One, we explored 182 electrode combinations to identify a subset of electrode position combinations that would be most likely to elicit distally-referred sensations isolated to the index finger without discomfort. In Experiment Two, we further examined this subset of electrode combinations to determine the effect of stimulation intensity and electrode position on perceived sensation location. Stimulation thresholds were evaluated using parameter estimation by sequential testing and sensation locations were characterized using psychometric intensity tests.Main Results.We found that electrode positions distal to the wrist can consistently evoke distally referred sensations with no significant polarity dependency. The finger-palm combination had the most occurrences of distal sensations, and the different variations of this combination did not have a significant effect on sensation location. Increasing stimulation intensity significantly expanded the area of the sensation, moved the most distal sensation distally, and moved the vertical centroid proximally. Also, a large anodic-leading electrode at the elbow mitigated all sensation at the anodic-leading electrode site while using symmetric stimulation waveforms. Furthermore, this study showed that the most intense sensation for a given percept can be distally referred. Lastly, for each participant, at least one of the finger-palm combinations evaluated in this study worked at both perception threshold and maximum comfortable stimulation intensities.Significance.These findings show that a non-invasive surface electrical stimulation charge modulated haptic interface can be used to elicit distally-referred sensations on non-disabled users. Furthermore, these results inform the design of novel haptic interfaces and other applications of surface electrical stimulation based haptic feedback on electrodes positioned distally from the wrist.

Human Stabilization of Delay-Induced Instability of Haptic Rendering in a Stiffness Discrimination Task.

Towards developing a coupled stability theory for haptic systems, we study the interaction of operators with time-delayed force feedback. In this work, we analyzed and validated experimentally the stability boundaries of an uncoupled system - without considering the human. We then designed an experiment in which the participants used a haptic device to interact with virtual elastic force fields in a stiffness discrimination task. We compared the performance and kinematics of users in uncoupled-unstable and uncoupled-stable conditions and characterized the stabilizing contribution of the users. We found that the users were able to perform the task regardless of the uncoupled-stability conditions. In addition, in uncoupled-unstable conditions, users maintained movement characteristics that were important for exploratory mediation, such as depth and duration of the movement, whereas other characteristics were not preserved. The results were reproduced in a simulation of the human controller that combined an inverse model and an optimal feedback controller. Adequate performance under the uncoupled-unstable yet coupled-stable conditions supports the potential benefit of designing for coupled stability of haptic systems. This could lead to the use of less conservative controllers than state-of-the-art solutions in haptic and teleoperation systems, and advance the fidelity of haptic feedback.