Tactile Echoes: Multisensory Augmented Reality for the Hand.

Touch interactions are central to many human activities, but there are few technologies for computationally augmenting free-hand interactions with real environments. Here, we describe Tactile Echoes, a finger-wearable system for augmenting touch interactions with physical objects. This system captures and processes touch-elicited vibrations in real-time in order to enliven tactile experiences. In this article, we process these signals via a parametric signal processing network in order to generate responsive tactile and auditory feedback. Just as acoustic echoes are produced through the delayed replication and modification of sounds, so are Tactile Echoes produced through transformations of vibrotactile inputs in the skin. The echoes also reflect the contact interactions and touched objects involved. A transient tap produces discrete echoes, while a continuous slide yields sustained feedback. We also demonstrate computational and spatial tracking methods that allow these effects to be selectively assigned to different objects or actions. A large variety of distinct multisensory effects can be designed via ten processing parameters. We investigated how Tactile Echoes are perceived in several perceptual experiments using multidimensional scaling methods. This allowed us to deduce low-dimensional, semantically grounded perceptual descriptions. We present several virtual and augmented reality applications of Tactile Echoes. In a user study, we found that these effects made interactions more responsive and engaging. Our findings show how to endow a large variety of touch interactions with expressive multisensory effects.

Static Weight Perception Through Skin Stretch and Kinesthetic Information: Detection Thresholds, JNDs, and PSEs.

We examined the contributions of kinesthetic and skin stretch cues to static weight perception. In three psychophysical experiments, several aspects of static weight perception were assessed by asking participants either to detect on which hand a weight was presented or to compare between two weight cues. Two closed-loop controlled haptic devices were used to present cutaneous and kinesthetic weights, in isolation and together, with a precision of 0.05 g. Our results show that combining skin stretch and kinesthetic information leads to better weight detection thresholds than presenting uni-sensory cues does. For supra-threshold stimuli, Weber fractions were 22-44%. Kinesthetic information was less reliable for lighter weights, while both sources of information were equally reliable for weights up to 300 g. Weight was perceived as equally heavy regardless of whether skin stretch and kinesthetic cues were presented together or alone. Data for lighter weights complied with an Optimal Integration model, while for heavier weights, measurements were closer to predictions from a Sensory Capture model. The presence of correlated noise might explain this discrepancy, since that would shift predictions from the Optimal Integration model towards our measurements. Our experiments provide device-independent perceptual measures, and can be used to inform, for instance, skin stretch device design.