The Single-Pitch Texel: A flexible and practical texture-rendering algorithm.

As the number of applications for tactile feedback technology rapidly increases, so too does the need for efficient, flexible, and extensible representations of virtual textures. The previously introduced Single-Pitch Texel rendering algorithm offers designers the ability to produce textures with perceptually wide-band spectral characteristics while requiring very few input parameters. This paper expands on the capabilities of the rendering algorithm. Diverse families of fine textures, with widely varied spectral characteristics, were shown to be rendered reliably using the Texel algorithm. Furthermore, by leveraging an assistive algorithm, subjects were shown to consistently navigate the Texel parameter space in a matching task. Finally, a psychophysical study was conducted to demonstrate the rendering algorithm's resilience to spectral quantization, further reducing the data required to represent a virtual texture.

Realism of Tactile Texture Playback: A Combination of Stretch and Vibration.

This study investigates the effects of two stimulation modalities (stretch and vibration) on natural touch sensation on the volar forearm. The skin-textile interaction was implemented by scanning three textures across the left forearm. The resulting skin displacements were recorded by the digital image correlation technique to capture the information imparted by the textures. The texture recordings were used to create three playback modes (stretch, vibration, and both), which were reproduced on the right forearm. Two psychophysical experiments compared the texture scans to rendered texture playbacks. The first experiment used a matching task and found that to maximize perceptual realism, i.e., similarity to a physical reference, subjects preferred the rendered texture to have a playback intensity of 1X-2X higher on DC components (stretch), and 1X-3.5X higher on AC components (vibration), varying across textures. The second experiment elicited similarity ratings between the texture scans and playbacks and showed that a combination of stretch and vibration was required to create differentiated texture sensations. However, the intensity amplification and use of two stimuli were still insufficient to create fully realistic texture sensations. We conclude that mechanisms beyond single-site uniaxial stimuli are needed to reproduce realistic textural sensations.

The spatial profile of skin indentation shapes tactile perception across stimulus frequencies.

Multiple human sensory systems exhibit sensitivity to spatial and temporal variations of physical stimuli. Vision has evolved to offer high spatial acuity with limited temporal sensitivity, while audition has developed complementary characteristics. Neural coding in touch has been believed to transition from a spatial to a temporal domain in relation to surface scale, such that coarse features (e.g., a braille cell or corduroy texture) are coded as spatially distributed signals, while fine textures (e.g., fine-grit sandpaper) are encoded by temporal variation. However, the interplay between the two domains is not well understood. We studied tactile encoding with a custom-designed pin array apparatus capable of deforming the fingerpad at 5 to 80 Hz in each of 14 individual locations spaced 2.5 mm apart. Spatial variation of skin indentation was controlled by moving each of the pins at the same frequency and amplitude, but with phase delays distributed across the array. Results indicate that such stimuli enable rendering of shape features at actuation frequencies up to 20 Hz. Even at frequencies > 20 Hz, however, spatial variation of skin indentation continues to play a vital role. In particular, perceived roughness is affected by spatial variation within the fingerpad even at 80 Hz. We provide evidence that perceived roughness is encoded via a summary measure of skin displacement. Relative displacements in neighboring pins of less than 10 µm generate skin stretch, which regulates the roughness percept.

A Low-Parameter Rendering Algorithm for Fine Textures.

This paper introduces a novel rendering algorithm for virtual textures, specifically those with characteristic length scales below 1 mm. By leveraging the relatively lossy mode of human tactile perception at this length scale, a virtual texture with wide-band spectral characteristics can be reduced to a spatial sequence of single-frequency texels, where each frequency is pulled stochastically from a distribution. A psychophysical study was conducted to demonstrate that, below a limiting physical texel length, virtual textures defined by identical frequency distributions are perceptually indiscriminable. Additionally, an exploratory study mapped the distribution parameters of the texel-based rendering to spectral characteristics of perceptually similar multi-frequency virtual textures.

Data-Driven Playback of Natural Tactile Texture Via Broadband Friction Modulation.

We used broadband electroadhesion to reproduce the friction force profile measured as a finger slid across a textured surface. In doing so, we were also able to reproduce with high fidelity the skin vibrations characteristic of that texture; however, we found that this did not reproduce the original perception. To begin, the reproduction felt weak. In order to maximize perceptual similarity between a real texture and its friction force playback, the vibratory magnitude of the latter must be scaled up on average ≈ 3X for fine texture and ≈ 5X for coarse texture samples. This additional gain appears to correlate with perceived texture roughness. Additionally, even with optimal scaling and high fidelity playback, subjects could identify which of two reproductions corresponds to a real texture with only 71 % accuracy, as compared to 95 % accuracy when using real texture alternatives. We conclude that while tribometry and vibrometry data can be useful for texture classification, they appear to contribute only partially to texture perception. We propose that spatially distributed excitation of skin within the fingerpad may play an additional key role, and may thus be able to contribute to high fidelity texture reproduction.

Building a Navigable Fine Texture Design Space.

Friction modulation technology enables the creation of textural effects on flat haptic displays. However, an intuitive and manageably small design space for construction of such haptic textures remains an unfulfilled goal for user interface designers. In this paper, we explore perceptually relevant features of fine texture for use in texture construction and modification. Beginning with simple sinusoidal patterns of friction force that vary in frequency and amplitude, we define irregularity, essentially a variable amount of introduced noise, as a third building block of a texture pattern. We demonstrate using multidimensional scaling that all three parameters are scalable features perceptually distinct from each other. Additionally, participants' verbal descriptions of this 3-dimensional design space provide insight into their intuitive interpretation of the physical parameter changes.

Comparison of Wide-Band Vibrotactile and Friction Modulation Surface Gratings.

This article seeks to understand conditions under which virtual gratings produced via vibrotaction and friction modulation are perceived as similar and to find physical origins in the results. To accomplish this, we developed two single-axis devices, one based on electroadhesion and one based on out-of-plane vibration. The two devices had identical touch surfaces, and the vibrotactile device used a novel closed-loop controller to achieve precise control of out-of-plane plate displacement under varying load conditions across a wide ranget of frequencies. A first study measured the perceptual intensity equivalence curve of gratings generated under electroadhesion and vibrotaction across the 20-400 Hz frequency range. A second study assessed the perceptual similarity between two forms of skin excitation given the same driving frequency and same perceived intensity. Our results indicate that it is largely the out-of-plane velocity that predicts vibrotactile intensity relative to shear forces generated by friction modulation. A high degree of perceptual similarity between gratings generated through friction modulation and through vibrotaction is apparent and tends to scale with actuation frequency suggesting perceptual indifference to the manner of fingerpad actuation in the upper frequency range.

How the Mechanical Properties and Thickness of Glass Affect TPaD Performance.

One well-known class of surface haptic devices that we have called Tactile Pattern Displays (TPaDs) uses ultrasonic transverse vibrations of a touch surface to modulate fingertip friction. This article addresses the power consumption of glass TPaDs, which is an important consideration in the context of mobile touchscreens. In particular, based on existing ultrasonic friction reduction models, we consider how the mechanical properties (density and Young's modulus) and thickness of commonly-used glass formulations affect TPaD performance, namely the relation between its friction reduction ability and its real power consumption. Experiments performed with eight types of TPaDs and an electromechanical model for the fingertip-TPaD system indicate: 1) TPaD performance decreases as glass thickness increases; 2) TPaD performance increases as the Young's modulus and density of glass decrease; and 3) real power consumption of a TPaD decreases as the contact force increases. Proper applications of these results can lead to significant increases in TPaD performance.

Localizable Button Click Rendering via Active Lateral Force Feedback.

In this article, we have developed a novel button click rendering mechanism based on active lateral force feedback. The effect can be localized because electroadhesion between a finger and a surface can be localized. Psychophysical experiments were conducted to evaluate the quality of a rendered button click, which subjects judged to be acceptable. Both the experiment results and the subjects' comments confirm that this button click rendering mechanism has the ability to generate a range of realistic button click sensations that could match subjects' different preferences. We can, thus, generate a button click on a flat surface without macroscopic motion of the surface in the lateral or normal direction, and can localize this haptic effect to an individual finger.

Electrowetting: A Consideration in Electroadhesion.

With the commercialization of haptic devices, understanding behavior under various environmental conditions is crucial for product optimization and cost reduction. Specifically, for surface haptic devices, the dependence of the friction force and the electroadhesion effect on the environmental relative humidity and the finger hydration level can directly impact their design and performance. This article presents the influence of relative humidity on the finger-surface friction force and the electroadhesion performance. Mechanisms including changes to Young's modulus of skin, contact angle change and capillary force were analyzed separately with experimental and numerical methods. Through comparison of the calculated capillary force in this paper and the electroadhesion force calculated in published papers, it was found that electrowetting at high voltage could contribute up to 60% of the total friction force increase in electroadhesion. Therefore, in future design of surface haptic devices, the effect of electrowetting should be considered carefully.

Closed Loop Application of Electroadhesion for Increased Precision in Texture Rendering.

Tactile displays based on friction modulation offer wide-bandwidth forces rendered directly on the fingertip. However, due to a number of touch conditions (e.g., normal force, skin hydration) that result in variations in the friction force and the strength of modulation effect, the precision of the force rendering remains limited. In this paper we demonstrate a closed-loop electroadhesion method for precise playback of friction force profiles on a human finger and we apply this method to the tactile rendering of several textiles encountered in everyday life.

UltraShiver: Lateral Force Feedback on a Bare Fingertip via Ultrasonic Oscillation and Electroadhesion.

We propose a new lateral force feedback device, the UltraShiver, which employs a combination of in-plane ultrasonic oscillation (around 30 kHz) and out-of-plane electroadhesion. It can achieve a strong active lateral force (400 mN) on the bare fingertip while operating silently. The lateral force is a function of pressing force, lateral vibration velocity, and electroadhesive voltage, as well as the relative phase between the velocity and voltage. In this paper, we perform experiments to investigate characteristics of the UltraShiver and their influence on lateral force. A lumped-parameter model is developed to understand the physical underpinnings of these influences. The model with frequency-weighted electroadhesion forces shows good agreement with experimental results. In addition, a Gaussian-like potential well is rendered as an application of the UltraShiver.

The Application of Tactile, Audible, and Ultrasonic Forces to Human Fingertips Using Broadband Electroadhesion.

We report an electroadhesive approach to controlling friction forces on sliding fingertips which is capable of producing vibrations across an exceedingly broad range of tactile, audible, and ultrasonic frequencies. Vibrations on the skin can be felt directly, and vibrations in the air can be heard emanating from the finger. Additionally, we report evidence from an investigation of the electrical dynamics of the system suggesting that an air gap at the skin/surface interface is primarily responsible for the induced electrostatic attraction underlying the electroadhesion effect. We developed an experimental apparatus capable of recording friction forces up to a frequency of 6 kHz, and used it to characterize two different electroadhesive systems, both of which exhibit flat force magnitude responses throughout the measurement range. These systems use custom electrical hardware to modulate a high frequency current and apply surprisingly low distortion, broadband forces to the skin. Recordings of skin vibrations with a laser Doppler vibrometer demonstrate the tactile capabilities of the system, while recordings of vibrations in the air with a MEMS microphone quantify the audible response and reveal the existence of ultrasonic forces applied to the skin via electronic friction modulation. Implications for surface haptic and audio-haptic displays are briefly discussed.

eShiver: Lateral Force Feedback on Fingertips through Oscillatory Motion of an Electroadhesive Surface.

We describe a new haptic force feedback device capable of creating lateral shear force on a bare fingertip-the eShiver. The eShiver creates a net lateral force from in-plane oscillatory motion of a surface synchronized with a "friction switch" based on Johnsen-Rahbek electroadhesion. Using an artificial finger, a maximum net lateral force of ±300 mN is achieved at 55 Hz lateral oscillation frequency, and net force is shown to be a function of velocity and applied voltage, as well as the phase between them. A second set of experiments is carried out on a human finger, and a lateral force of up to ±450 mN is achieved at a lateral oscillation frequency of 1,000 Hz. This force is reached at a peak lateral surface velocity of 400 mm/s and a peak applied voltage of 400 V. We develop a simple lumped parameter model of the eShiver, and a time domain simulation of the artificial finger is shown to agree with the experimental results. Three distinct zones of operation are found, which predict the limitations of force generation and which may be used for optimization. The human finger is found to be similar to the artificial finger in its dependence on actuation parameters, suggesting that the same lumped parameter model may be applied, albeit with different parameters. Curiously, the friction force due to Johnsen-Rahbek electroadhesion is found to increase substantially over time as the finger remains in contact with the surface. Considerations for optimizing the performance of the eShiver are discussed.

Partial squeeze film levitation modulates fingertip friction.

When touched, a glass plate excited with ultrasonic transverse waves feels notably more slippery than it does at rest. To study this phenomenon, we use frustrated total internal reflection to image the asperities of the skin that are in intimate contact with a glass plate. We observed that the load at the interface is shared between the elastic compression of the asperities of the skin and a squeeze film of air. Stroboscopic investigation reveals that the time evolution of the interfacial gap is partially out of phase with the plate vibration. Taken together, these results suggest that the skin bounces against the vibrating plate but that the bounces are cushioned by a squeeze film of air that does not have time to escape the interfacial separation. This behavior results in dynamic levitation, in which the average number of asperities in intimate contact is reduced, thereby reducing friction. This improved understanding of the physics of friction reduction provides key guidelines for designing interfaces that can dynamically modulate friction with soft materials and biological tissues, such as human fingertips.

Multiple Fingers - One Gestalt.

The Gestalt theory of perception offered principles by which distributed visual sensations are combined into a structured experience ("Gestalt"). We demonstrate conditions whereby haptic sensations at two fingertips are integrated in the perception of a single object. When virtual bumps were presented simultaneously to the right hand's thumb and index finger during lateral arm movements, participants reported perceiving a single bump. A discrimination task measured the bump's perceived location and perceptual reliability (assessed by differential thresholds) for four finger configurations, which varied in their adherence to the Gestalt principles of proximity (small versus large finger separation) and synchrony (virtual spring to link movements of the two fingers versus no spring). According to models of integration, reliability should increase with the degree to which multi-finger cues integrate into a unified percept. Differential thresholds were smaller in the virtual-spring condition (synchrony) than when fingers were unlinked. Additionally, in the condition with reduced synchrony, greater proximity led to lower differential thresholds. Thus, with greater adherence to Gestalt principles, thresholds approached values predicted for optimal integration. We conclude that the Gestalt principles of synchrony and proximity apply to haptic perception of surface properties and that these principles can interact to promote multi-finger integration.

Power optimization of ultrasonic friction-modulation tactile interfaces.

Ultrasonic friction-modulation devices provide rich tactile sensation on flat surfaces and have the potential to restore tangibility to touchscreens. To date, their adoption into consumer electronics has been in part limited by relatively high power consumption, incompatible with the requirements of battery-powered devices. This paper introduces a method that optimizes the energy efficiency and performance of this class of devices. It considers optimal energy transfer to the impedance provided by the finger interacting with the surface. Constitutive equations are determined from the mode shape of the interface and the piezoelectric coupling of the actuator. The optimization procedure employs a lumped parameter model to simplify the treatment of the problem. Examples and an experimental study show the evolution of the optimal design as a function of the impedance of the finger.

Search efficiency for tactile features rendered by surface haptic displays.

Haptic interfaces controlled by a single fingertip or hand-held probe tend to display surface features individually, requiring serial search for multiple features. Novel surface haptic devices, however, have the potential to provide displays to multiple fingertips simultaneously, affording the possibility of parallel search. Using variable-friction surface haptic devices, we investigated the ability of participants to detect a target feature among a set of distractors in parallel across the fingers. We found that searches for a material property (slipperiness) and an illusory shape (virtual hole) were significantly impaired by distractors, while search for an abrupt discontinuity (virtual edge) was not. The efficiency of search for edges rendered by surface haptics suggests that they engage primitive detectors in the haptic perceptual system.