SORI: A softness-rendering interface to unravel the nature of softness perception.

Tactile perception of softness serves a critical role in the survival, well-being, and social interaction among various species, including humans. This perception informs activities from food selection in animals to medical palpation for disease detection in humans. Despite its fundamental importance, a comprehensive understanding of how softness is neurologically and cognitively processed remains elusive. Previous research has demonstrated that the somatosensory system leverages both cutaneous and kinesthetic cues for the sensation of softness. Factors such as contact area, depth, and force play a particularly critical role in sensations experienced at the fingertips. Yet, existing haptic technologies designed to explore this phenomenon are limited, as they often couple force and contact area, failing to provide a real-world experience of softness perception. Our research introduces the softness-rendering interface (SORI), a haptic softness display designed to bridge this knowledge gap. Unlike its predecessors, SORI has the unique ability to decouple contact area and force, thereby allowing for a quantitative representation of softness sensations at the fingertips. Furthermore, SORI incorporates individual physical fingertip properties and model-based softness cue estimation and mapping to provide a highly personalized experience. Utilizing this method, SORI quantitatively replicates the sensation of softness on stationary, dynamic, homogeneous, and heterogeneous surfaces. We demonstrate that SORI accurately renders the surfaces of both virtual and daily objects, thereby presenting opportunities across a range of fields, from teleoperation to medical technology. Finally, our proposed method and SORI will expedite psychological and neuroscience research to unlock the nature of softness perception.

Improving electrotactile communication with a multi-pad electrode under cognitive load.

BACKGROUND: Electrotactile systems are compact interfaces that can be used to convey information through the skin by producing a range of haptic sensations. In many applications, however, the user needs to perceive and interpret haptic stimulation while being engaged in parallel activities. Developing methods that ensure reliable recognition of electrotactile messages despite additional cognitive load is, therefore, an important step for the practical application of electrotactile displays. METHODS: This study investigated if a simple strategy of repeating electrotactile messages can improve message identification during multitasking. Ten participants identified 36 spatiotemporal electrotactile messages delivered through a 3 × 2 pad-matrix electrode placed on the torso while performing a concomitant cognitive task in three conditions: the messages were presented once (No-REP), and each message was repeated three (REP3) and five (REP5) times. The main outcome measure was the success rate (SR) of message identification. RESULTS: During multitasking, in the No-REP condition, the SR (median (IQR)) dropped to 56.25% (22.62%), demonstrating that the cognitive task decreased performance. However, the SR significantly improved with message repetitions, reaching 72.92% (21.87%) and 81.25% (18.66%) in REP3 and REP5 conditions respectively, without a statistically significant difference between REP3 and REP5. CONCLUSIONS: Multitasking affected the efficacy of haptic communication, but message repetition was shown to be an effective strategy for improving performance. Additionally, only three repetitions were enough, as an additional increase in the duration of message transmission (5 repetitions) did not lead to further improvement. This study is an important step toward delivering electrotactile communication that can cope with the demands of real-world applications.

Telehealth and Virtual Reality Technologies in Chronic Pain Management: A Narrative Review.

PURPOSE OF REVIEW: This review provides medical practitioners with an overview of the present and emergent roles of telehealth and associated virtual reality (VR) applications in chronic pain (CP) management, particularly in the post-COVID-19 healthcare landscape. RECENT FINDINGS: Accumulated evidence points to the efficacy of now well-established telehealth modalities, such as videoconferencing, short messaging service (SMS), and mobile health (mHealth) applications in complementing remote CP care. More recently, and although still in early phases of clinical implementation, a wide range of VR-based interventions have demonstrated potential for improving the asynchronous remote management of CP. Additionally, VR-associated technologies at the leading edge of science and engineering, such as VR-assisted biofeedback, haptic technology, high-definition three-dimensional (HD3D) conferencing, VR-enabled interactions in a Metaverse, and the use of wearable monitoring devices, herald a new era for remote, synchronous patient-physician interactions. These advancements hold the potential to facilitate remote physical examinations, personalized remote care, and innovative interventions such as ultra-realistic biofeedback. Despite the promise of VR-associated technologies, several limitations remain, including the paucity of robust long-term effectiveness data, heterogeneity of reported pain-related outcomes, challenges with scalability and insurance coverage, and demographic-specific barriers to patient acceptability. Future research efforts should be directed toward mitigating these limitations to facilitate the integration of telehealth-associated VR into the conventional management of CP. Despite ongoing barriers to widespread adoption, recent evidence suggests that VR-based interventions hold an increasing potential to complement and enhance the remote delivery of CP care.

Developmental changes in the visual, haptic, and bimodal perception of geometric angles.

Geometrical knowledge is typically taught to children through a combination of vision and repetitive drawing (i.e. haptics), yet our understanding of how different spatial senses contribute to geometric perception during childhood is poor. Studies of line orientation suggest a dominant role of vision affecting the calibration of haptics during development; however, the associated multisensory interactions underpinning angle perception are unknown. Here we examined visual, haptic, and bimodal perception of angles across three age groups of children: 6 to 8 years, 8 to 10 years, and 10 to 12 years, with age categories also representing their class (grade) in primary school. All participants first learned an angular shape, presented dynamically, in one of three sensory tracing conditions: visual only, haptic only, or bimodal exploration. At test, which was visual only, participants selected a target angle from four possible alternatives with distractor angle sizes varying relative to the target angle size. We found a clear improvement in accuracy of angle perception with development for all learning modalities. Angle perception in the youngest group was equally poor (but above chance) for all modalities; however, for the two older child groups, visual learning was better than haptics. Haptic perception did not improve to the level of vision with age (even in a comparison adult group), and we found no specific benefit for bimodal learning over visual learning in any age group, including adults. Our results support a developmental increment in both spatial accuracy and precision in all modalities, which was greater in vision than in haptics, and are consistent with previous accounts of cross-sensory calibration in the perception of geometric forms.

Child development and the role of visual experience in the use of spatial and non-spatial features in haptic object perception.

Previous work has suggested a different developmental timeline and role of visual experience for the use of spatial and non-spatial features in haptic object recognition. To investigate this conjecture, we used a haptic ambiguous odd-one-out task in which one object needed to be selected as being different from two other objects. The odd-one-out could be selected based on four characteristics: size, shape (spatial), texture, and weight (non-spatial). We tested sighted children from 4 to 12 years of age; congenitally blind, late blind, and adult participants with low vision; and normally sighted adults. Given the protracted developmental time course for spatial perception, we expected a shift from a preference for non-spatial features toward spatial features during typical development. Due to the dominant influence of vision for spatial perception, we expected congenitally blind adults to show a similar preference for non-spatial features as the youngest children. The results confirmed our first hypothesis; the 4-year-olds demonstrated a lower dominance for spatial features for object classification compared with older children and sighted adults. In contrast, our second hypothesis was not confirmed; congenitally blind adults' preferred categorization criteria were indistinguishable from those of sighted controls. These findings suggest an early development, but late maturation, of spatial processing in haptic object recognition independent of visual experience.

Humans can sense large numbers of objects in a box by touch alone.

Everyday experiences suggest that a container, such as a box of cereal, can convey pertinent information about the nature and quantity of its content. This study investigated how well people can judge large quantities of objects in a container through haptic perception. Stimuli consisted of plastic drinking straws cut to "small" (1.5 cm) or "big" (4.5 cm) pieces contained in plastic food containers. Participants performed both a magnitude estimation of the number of objects and a direct estimation of the proportion of the container perceived to be filled with objects. Overall, participants demonstrated considerable accuracy for both tasks and irrespective of the size of the content. Post-experiment interviews revealed three potential strategies. Participants either focused on the container's contents, the excess space in the container, or the perceived weight of the container (content).

Visual softness perception can be manipulated through exploratory procedures.

Both visual and haptic softness perception have recently been shown to have multiple dimensions, such as deformability, granularity, fluidity, surface softness, and roughness. During haptic exploration, people adjust their hand motions (exploratory procedures, EPs) based on the material qualities of the object and the particular information they intend to acquire. Some of these EPs are also shown to be associated with perceived softness dimensions, for example, stroking a silk blouse or applying pressure to a pillow. Here, we aimed to investigate whether we can manipulate observers' judgments about softness attributes through exposure to videos of others performing various EPs on everyday soft materials. In two experiments, participants watched two videos of the same material: one with a corresponding EP and the other without correspondence; then, they judged these materials based on 12 softness-related adjectives (semantic differentiation method). The results of the second experiment suggested that when the EP is congruent with the dimension from which the material is chosen, the ratings for the adjectives from the same dimension are higher than the incongruent EP. This study provides evidence that participants can assess material properties from optic and mechanical cues without needing haptic signals. Additionally, our findings indicate that manipulating the hand motion can selectively facilitate material-related judgments.

Planning lane changes using advance visual and haptic information.

Taking a motor planning perspective, this study investigates whether haptic force cues displayed on the steering wheel are more effective than visual cues in signaling the direction of an upcoming lane change. Licensed drivers drove in a fixed-base driving simulator equipped with an active steering system for realistic force feedback. They were instructed to make lane changes upon registering a directional cue. Cues were delivered according to the movement precuing technique employing a pair of precues and imperative cues which could be either visual, haptic, or crossmodal (a visual precue with a haptic imperative cue, and vice versa). The main dependent variable was response time. Additional analyses were conducted on steering wheel angle profiles and the rate of initial steering errors. Conditions with a haptic imperative cue produced considerably faster responses than conditions with a visual imperative cue, irrespective of the precue modality. Valid and invalid precues produced the typical gains and costs, with one exception. There appeared to be little cost in response time or initial steering errors associated with invalid cueing when both cues were haptic. The results are consistent with the hypothesis that imperative haptic cues facilitate action selection while visual stimuli require additional time-consuming cognitive processing.

Initial contact shapes the perception of friction.

Humans efficiently estimate the grip force necessary to lift a variety of objects, including slippery ones. The regulation of grip force starts with the initial contact and takes into account the surface properties, such as friction. This estimation of the frictional strength has been shown to depend critically on cutaneous information. However, the physical and perceptual mechanism that provides such early tactile information remains elusive. In this study, we developed a friction-modulation apparatus to elucidate the effects of the frictional properties of objects during initial contact. We found a correlation between participants' conscious perception of friction and radial strain patterns of skin deformation. The results provide insights into the tactile cues made available by contact mechanics to the sensorimotor regulation of grip, as well as to the conscious perception of the frictional properties of an object.

Valveless microliter combustion for densely packed arrays of powerful soft actuators.

Existing tactile stimulation technologies powered by small actuators offer low-resolution stimuli compared to the enormous mechanoreceptor density of human skin. Arrays of soft pneumatic actuators initially show promise as small-resolution (1- to 3-mm diameter), highly conformable tactile display strategies yet ultimately fail because of their need for valves bulkier than the actuators themselves. In this paper, we demonstrate an array of individually addressable, soft fluidic actuators that operate without electromechanical valves. We achieve this by using microscale combustion and localized thermal flame quenching. Precisely, liquid metal electrodes produce sparks to ignite fuel lean methane-oxygen mixtures in a 5-mm diameter, 2-mm tall silicone cylinder. The exothermic reaction quickly pressurizes the cylinder, displacing a silicone membrane up to 6 mm in under 1 ms. This device has an estimated free-inflation instantaneous stroke power of 3 W. The maximum reported operational frequency of these cylinders is 1.2 kHz with average displacements of ∼100 µm. We demonstrate that, at these small scales, the wall-quenching flame behavior also allows operation of a 3 × 3 array of 3-mm diameter cylinders with 4-mm pitch. Though we primarily present our device as a tactile display technology, it is a platform microactuator technology with application beyond this one.

Static and Dynamic Hand Gestures: A Review of Techniques of Virtual Reality Manipulation.

This review explores the historical and current significance of gestures as a universal form of communication with a focus on hand gestures in virtual reality applications. It highlights the evolution of gesture detection systems from the 1990s, which used computer algorithms to find patterns in static images, to the present day where advances in sensor technology, artificial intelligence, and computing power have enabled real-time gesture recognition. The paper emphasizes the role of hand gestures in virtual reality (VR), a field that creates immersive digital experiences through the Ma blending of 3D modeling, sound effects, and sensing technology. This review presents state-of-the-art hardware and software techniques used in hand gesture detection, primarily for VR applications. It discusses the challenges in hand gesture detection, classifies gestures as static and dynamic, and grades their detection difficulty. This paper also reviews the haptic devices used in VR and their advantages and challenges. It provides an overview of the process used in hand gesture acquisition, from inputs and pre-processing to pose detection, for both static and dynamic gestures.

A horizon for haptic perception.

The spatial limits of sensory acquisition (its sensory horizon) are a fundamental property of any sensorimotor system. In the present study, we sought to determine whether there is a sensory horizon for the human haptic modality. At first blush, it seems obvious that the haptic system is bounded by the space where the body can interact with the environment (e.g., the arm span). However, the human somatosensory system is exquisitely tuned to sensing with tools-blind-cane navigation being a classic example of this. The horizon of haptic perception therefore extends beyond body space, but to what extent is unknown. We first used neuromechanical modeling to determine the theoretical horizon, which we pinpointed as 6 m. We then used a psychophysical localization paradigm to behaviorally confirm that humans can haptically localize objects using a 6-m rod. This finding underscores the incredible flexibility of the brain's sensorimotor representations, as they can be adapted to sense an object many times longer than the user's own body.NEW & NOTEWORTHY There are often spatial limits to where an active sensory system can sample information from the environment. Hand-held tools can extend human haptic perception beyond the body, but the limits of this extension are unknown. We used theoretical modeling and psychophysics to determine these spatial limits. We find that the ability to spatially localize objects through a tool extends at least 6 m beyond the user's body.

The distorted hand: systematic but 'independent' distortions in both explicit and implicit hand representations in young female adults.

It has long been assumed that an accurate representation of the size and shape of one's body is necessary to successfully interact with the environment. Previous research has shown accurate representations when healthy participants make overt judgments (i.e. explicit) about the size of their bodies. However, when body size is judged implicitly, studies have shown systematic distortions. One suggestion for these differences, is that explicit and implicit representations are informed by different sensory modalities. Explicit representations rely on vision whereas implicit representations are informed by haptics. We designed an experiment to investigate if explicit representations that are informed by haptics are more like implicit representation featuring systematic distortions. We asked female participants to estimate the size of their fingers and hands in three different tasks: an explicit-haptic, an implicit, and an explicit-vision task. The results showed that all three representations were distorted and furthermore, the distortions for each representation were different from one another. These results suggest that inaccurate finger and hand length are a stereotypical feature of body representation that is present in both visual and haptic domains. We discuss the results in relation to theories of body representation.

Haptic simulators accelerate laparoscopic simulator training, but skills are not transferable to a non-haptic simulator: a randomized trial.

BACKGROUND: Laparoscopy requires specific psychomotor skills and can be challenging to learn. Most proficiency-based laparoscopic training programs have used non-haptic virtual reality simulators; however, haptic simulators can provide the tactile sensations that the surgeon would experience in the operating room. The objective was to investigate the effect of adding haptic simulators to a proficiency-based laparoscopy training program. METHODS: A randomized controlled trial was designed where residents (n = 36) were randomized to proficiency-based laparoscopic simulator training using haptic or non-haptic simulators. Subsequently, participants from the haptic group completed a follow-up test, where they had to reach proficiency again using the non-haptic simulator. Participants from the non-haptic group returned to train until reaching proficiency again using the non-haptic simulator. RESULTS: Mean completion times during the intervention were 120 min (SD 38.7 min) and 183 min (SD 66.3 min) for the haptic group and the non-haptic group, respectively (p = 0.001). The mean times to proficiency during the follow-up test were 107 min (SD 41.0 min) and 58 min (SD 23.7 min) for the haptic and the non-haptic group, respectively (p < 0.001). The haptic group was not faster to reach proficiency in the follow-up test than during the intervention (p = 0.22). In contrast, the non-haptic group reached the required proficiency level significantly faster in the follow-up test (p < 0.001). CONCLUSION: Haptic virtual reality simulators reduce the time to reach proficiency compared to non-haptic simulators. However, the acquired skills are not transferable to the conventional non-haptic setting.

Surgical teaching of the retropubic midurethral sling: a virtual reality training system.

INTRODUCTION AND HYPOTHESIS: The objective is to develop a low-risk, cost-effective method to teach procedures that require learning by feel and high-volume pattern recognition, starting with the midurethral sling. METHODS: This video describes the creation of a virtual reality model utilizing de-identified patient data, artificial intelligence algorithms and haptics; and demonstrates the use of the training system for trocar passage of the retropubic midurethral sling procedure. RESULTS: This innovative system overcomes the lack of visualization and "blind" nature of sling surgery. Novel artificial intelligence provides high accuracy of anatomical landmarks and a realistic 3D environment. The trainee benefits from haptic and visual alerts for real-time feedback on the trocar insertion pathway and scoring to develop competency. CONCLUSION: This is one of the first noncadaveric, nonstatic models available in the field. It allows for multiple low-risk exercises and provides more surgeons with training outside the operating room, at their own institution, and avoids the need for patient subjects. Training can be disseminated at a significantly lower cost and greater convenience than remote cadaver laboratories or intraoperative observation and has a higher fidelity than available static models, particularly after multiple passes. This has implications not only for retropubic midurethral slings but also for urogynecological and "blind" surgery as a whole.

Gone Fishin': Perceiving the length of one object that is non-rigidly attached to a wielded object.

We performed four experiments to investigate whether people can perceive the length of a target object (a "fish") that is attached to a freely wielded object (the "fishing pole") by a length of string, and if so, whether this ability is grounded in the sensitivity of the touch system to invariant mechanical parameters that describe the forces and torques required to move the target object. In particular, we investigated sensitivity to mass, static moment, and rotational inertia-the forces required to keep an object from falling due to gravity, the torque required to keep an object from rotating due to gravity, and the torques required to actively rotate an object in different directions, respectively. We manipulated the length of the target object (Experiment 1), the mass of the target object (Experiment 2), and the mass distribution of the target object (Experiments 3 and 4). Overall, the results of the four experiments showed that participants can perform this task. Moreover, when the task is configured such that it more closely approximates a wielding at a distance task, the ability to do so is grounded in sensitivity to such forces and torques.

Humans can sense small numbers of objects in a box by touch alone.

Everyday experiences suggest that a container, such as a box of chocolate sprinkles, can convey pertinent information about the nature of its content. Despite the familiarity of the experience, we do not know whether people can perceive the number of objects in the container from touch alone and how accurately they can do so. In three experiments, participants handled containers holding between one and five objects and verbally estimated their number. Containers were small cardboard jewelry boxes, and objects were round beads of varying diameter and weight. Any useful visual and auditory cues were precluded. Experiment 1 demonstrated very accurate performance, provided the objects were of sufficient weight. Experiment 2 demonstrated that withholding information about the possible number of objects inside the container does not affect accuracy at a group level but does produce occasional overestimations at an individual level. Experiment 3 demonstrated that removing the weight cue leads to systematic underestimations but does not eliminate people's ability to distinguish between different numbers of objects in the container. This study contributes to a growing picture that container haptics is surprisingly capable.

Stimulus intensity modulates perceived tactile distance.

Several features of tactile stimuli modulate the perceived distance between touches. In particular, distances are perceived as farther apart when the time interval between them is longer, than when it is shorter. Such effects have been interpreted as a form of 'psychological relativity', analogous to Einstein's conception of a four-dimensional space-time. We investigated whether similar effects occur for stimulus features other than time, specifically stimulus intensity. We hypothesised that perceived distance would be increased when the two stimuli differed in intensity, since they would then be farther apart in a multi-dimensional feature space. Participants made verbal estimates of the perceived distance between two touches on their left hand. Intensity was manipulated such that both stimuli could be intense, both could be light, or one could be intense and the other light. We found no evidence for change in perceived tactile distance when stimuli intensity mis-matched. In contrast, there were clear effects of average stimulus intensity on perceived distance. Intense stimuli were judged as farther apart than light stimuli, and mixed stimuli were intermediate. These results are consistent with theories of general magnitude representation, which argue that multiple dimensions of magnitude are dependent on a shared underlying representation of domain-general magnitude.

Visual experience is not necessary for the development of face-selectivity in the lateral fusiform gyrus.

The fusiform face area responds selectively to faces and is causally involved in face perception. How does face-selectivity in the fusiform arise in development, and why does it develop so systematically in the same location across individuals? Preferential cortical responses to faces develop early in infancy, yet evidence is conflicting on the central question of whether visual experience with faces is necessary. Here, we revisit this question by scanning congenitally blind individuals with fMRI while they haptically explored 3D-printed faces and other stimuli. We found robust face-selective responses in the lateral fusiform gyrus of individual blind participants during haptic exploration of stimuli, indicating that neither visual experience with faces nor fovea-biased inputs is necessary for face-selectivity to arise in the lateral fusiform gyrus. Our results instead suggest a role for long-range connectivity in specifying the location of face-selectivity in the human brain.

Experimental Evaluation of a Hybrid Sensory Feedback System for Haptic and Kinaesthetic Perception in Hand Prostheses.

This study proposes a new hybrid multi-modal sensory feedback system for prosthetic hands that can provide not only haptic and proprioceptive feedback but also facilitate object recognition without the aid of vision. Modality-matched haptic perception was provided using a mechanotactile feedback system that can proportionally apply the gripping force through the use of a force controller. A vibrotactile feedback system was also employed to distinguish four discrete grip positions of the prosthetic hand. The system performance was evaluated with a total of 32 participants in three different experiments (i) haptic feedback, (ii) proprioceptive feedback and (iii) object recognition with hybrid haptic-proprioceptive feedback. The results from the haptic feedback experiment showed that the participants' ability to accurately perceive applied force depended on the amount of force applied. As the feedback force was increased, the participants tended to underestimate the force levels, with a decrease in the percentage of force estimation. Of the three arm locations (forearm volar, forearm ventral and bicep), and two muscle states (relaxed and tensed) tested, the highest accuracy was obtained for the bicep location in the relaxed state. The results from the proprioceptive feedback experiment showed that participants could very accurately identify four different grip positions of the hand prosthesis (i.e., open hand, wide grip, narrow grip, and closed hand) without a single case of misidentification. In experiment 3, participants could identify objects with different shapes and stiffness with an overall high success rate of 90.5% across all combinations of location and muscle state. The feedback location and muscle state did not have a significant effect on object recognition accuracy. Overall, our study results indicate that the hybrid feedback system may be a very effective way to enrich a prosthetic hand user's experience of the stiffness and shape of commonly manipulated objects.