The influence of visual and haptic material information on early grasping force.

In this paper, we assess the importance of visual and haptic information about materials for scaling the grasping force when picking up an object. We asked 12 participants to pick up and lift objects with six different textures, either blindfolded or with visual information present. We measured the grip force and estimated the load force from the object's weight and vertical acceleration. The coefficient of friction of the materials was measured separately. Already at an early phase in the grasp (before lift-off), the grip force correlated highly with the textures' static coefficient of friction. However, no strong influence on the presence of visual information was found. We conclude that the main mechanism for modulation of grip force in the early phase of grasping is the real-time sensation of the texture's friction.

A mass-density model can account for the size-weight illusion.

When judging the heaviness of two objects with equal mass, people perceive the smaller and denser of the two as being heavier. Despite the large number of theories, covering bottom-up and top-down approaches, none of them can fully account for all aspects of this size-weight illusion and thus for human heaviness perception. Here we propose a new maximum-likelihood estimation model which describes the illusion as the weighted average of two heaviness estimates with correlated noise: One estimate derived from the object's mass, and the other from the object's density, with estimates' weights based on their relative reliabilities. While information about mass can directly be perceived, information about density will in some cases first have to be derived from mass and volume. However, according to our model at the crucial perceptual level, heaviness judgments will be biased by the objects' density, not by its size. In two magnitude estimation experiments, we tested model predictions for the visual and the haptic size-weight illusion. Participants lifted objects which varied in mass and density. We additionally varied the reliability of the density estimate by varying the quality of either visual (Experiment 1) or haptic (Experiment 2) volume information. As predicted, with increasing quality of volume information, heaviness judgments were increasingly biased towards the object's density: Objects of the same density were perceived as more similar and big objects were perceived as increasingly lighter than small (denser) objects of the same mass. This perceived difference increased with an increasing difference in density. In an additional two-alternative forced choice heaviness experiment, we replicated that the illusion strength increased with the quality of volume information (Experiment 3). Overall, the results highly corroborate our model, which seems promising as a starting point for a unifying framework for the size-weight illusion and human heaviness perception.

Correcting for Visuo-Haptic Biases in 3D Haptic Guidance.

Visuo-haptic biases are observed when bringing your unseen hand to a visual target. The biases are different between, but consistent within participants. We investigated the usefulness of adjusting haptic guidance to these user-specific biases in aligning haptic and visual perception. By adjusting haptic guidance according to the biases, we aimed to reduce the conflict between the modalities. We first measured the biases using an adaptive procedure. Next, we measured performance in a pointing task using three conditions: 1) visual images that were adjusted to user-specific biases, without haptic guidance, 2) veridical visual images combined with haptic guidance, and 3) shifted visual images combined with haptic guidance. Adding haptic guidance increased precision. Combining haptic guidance with user-specific visual information yielded the highest accuracy and the lowest level of conflict with the guidance at the end point. These results show the potential of correcting for user-specific perceptual biases when designing haptic guidance.

Integrating force and position: testing model predictions.

In this study, we investigated the integration of force and position information in a task in which participants were asked to estimate the center of a weak force field. Two hypotheses, describing how participants solved this task, were tested: (1) by only using the position(s) where the force reaches the detection threshold, and (2) by extrapolating the force field based on perceived stiffness. Both hypotheses were also described formally, assuming a psychophysical function obeying a power law with an exponent smaller than one. The hypotheses were tested in two psychophysical experiments, in which 12 participants took part. In Experiment 1, an asymmetric force field was used and the presence of visual feedback about hand position was varied. In Experiment 2, a unilateral force field was used. For both experiments, hypothesis 1 predicts biases between (Experiment 1) or at (Experiment 2) the position(s) of the force detection threshold, while hypothesis 2 predicts smaller biases. The measured data show significant biases in both experiments that coincide with the biases predicted by using force detection thresholds from the literature. The average measured responses and their variabilities also fitted very well with the mathematical model of hypothesis 1. These results underline the validity of hypothesis 1. So, participants did not use a percept of the stiffness of the force field, but based their estimation of the center of the force field on the position(s) where the force reached the detection threshold. This shows that force and position information were not integrated in this task.

The Effect of Global and Local Damping on the Perception of Hardness.

In tele-operation systems, damping is often injected to guarantee system stability during contact with hard objects. In this study, we used psychophysical experiments to assess the effect of adding damping on the user's perception of object hardness. In Experiments 1 and 2, combinations of stiffness and damping were tested to assess their effect on perceived hardness. In both experiments, two tasks were used: an in-contact task, starting at the object's surface, and a contact-transition task, including a free-air movement. In Experiment 3, the difference between inserting damping globally (equally throughout the workspace) and locally (inside the object only) was tested. In all experiments, the correlation between the participant's perceptual decision and force and position data was also investigated. Experiments 1 and 2 show that when injecting damping globally, perceived hardness slightly increased for an in-contact task, while it decreased considerably for a contact-transition task. Experiment 3 shows that this effect was mainly due to inserting damping globally, since there was a large perceptual difference between inserting damping globally and locally. The force and position parameters suggest that participants used the same force profile during the two movements of one trial and assessed the system's reaction to this force to perceive hardness.

Haptic perception of force magnitude and its relation to postural arm dynamics in 3D.

In a previous study, we found the perception of force magnitude to be anisotropic in the horizontal plane. In the current study, we investigated this anisotropy in three dimensional space. In addition, we tested our previous hypothesis that the perceptual anisotropy was directly related to anisotropies in arm dynamics. In experiment 1, static force magnitude perception was studied using a free magnitude estimation paradigm. This experiment revealed a significant and consistent anisotropy in force magnitude perception, with forces exerted perpendicular to the line between hand and shoulder being perceived as 50% larger than forces exerted along this line. In experiment 2, postural arm dynamics were measured using stochastic position perturbations exerted by a haptic device and quantified through system identification. By fitting a mass-damper-spring model to the data, the stiffness, damping and inertia parameters could be characterized in all the directions in which perception was also measured. These results show that none of the arm dynamics parameters were oriented either exactly perpendicular or parallel to the perceptual anisotropy. This means that endpoint stiffness, damping or inertia alone cannot explain the consistent anisotropy in force magnitude perception.

Expertise effects in cutaneous wind perception.

We examined whether expertise effects are present in cutaneous wind perception. To this end, we presented wind stimuli consisting of different wind directions and speeds in a wind simulator. The wind simulator generated wind stimuli from 16 directions and with three speeds by means of eight automotive wind fans. Participants were asked to judge cutaneously perceived wind directions and speeds without having access to any visual or auditory information. Expert sailors (n = 6), trained to make the most effective use of wind characteristics, were compared to less-skilled sailors (n = 6) and to a group of nonsailors (n = 6). The results indicated that expert sailors outperformed nonsailors in perceiving wind direction (i.e., smaller mean signed errors) when presented with low wind speeds. This suggests that expert sailors are more sensitive in picking up differences in wind direction, particularly when confronted with low wind speeds that demand higher sensitivity.

Haptic exploratory behavior during object discrimination: a novel automatic annotation method.

In order to acquire information concerning the geometry and material of handheld objects, people tend to execute stereotypical hand movement patterns called haptic Exploratory Procedures (EPs). Manual annotation of haptic exploration trials with these EPs is a laborious task that is affected by subjectivity, attentional lapses, and viewing angle limitations. In this paper we propose an automatic EP annotation method based on position and orientation data from motion tracking sensors placed on both hands and inside a stimulus. A set of kinematic variables is computed from these data and compared to sets of predefined criteria for each of four EPs. Whenever all criteria for a specific EP are met, it is assumed that that particular hand movement pattern was performed. This method is applied to data from an experiment where blindfolded participants haptically discriminated between objects differing in hardness, roughness, volume, and weight. In order to validate the method, its output is compared to manual annotation based on video recordings of the same trials. Although mean pairwise agreement is less between human-automatic pairs than between human-human pairs (55.7% vs 74.5%), the proposed method performs much better than random annotation (2.4%). Furthermore, each EP is linked to a specific object property for which it is optimal (e.g., Lateral Motion for roughness). We found that the percentage of trials where the expected EP was found does not differ between manual and automatic annotation. For now, this method cannot yet completely replace a manual annotation procedure. However, it could be used as a starting point that can be supplemented by manual annotation.

Tactual perception of liquid material properties.

In this paper, studies into the tactual perception of two liquid material properties, viscosity and wetness, are reviewed. These properties are very relevant in the context of interaction with liquids, both real, such as cosmetics or food products, and simulated, as in virtual reality or teleoperation. Both properties have been the subject of psychophysical characterisation in terms of magnitude estimation experiments and discrimination experiments, which are discussed. For viscosity, both oral and manual perception is discussed, as well as the perception of the viscosity of a mechanical system. For wetness, the relevant cues are identified and factors affecting perception are discussed. Finally, some conclusions are drawn pertaining to both properties.

Target contact and exploration strategies in haptic search.

In a haptic search task, one has to detect the presence of a target among distractors using the sense of touch. A salient target can be detected faster than a non-salient target. However, little is known about the exploration strategies that are used, especially in 3D search tasks where items are held in the hand. In this study, we investigated which parts of the hand were used to contact the target and which strategies were performed. Blindfolded participants performed search tasks in four conditions, where the targets differed in relevant property and saliency. The positions of the target and the hand were tracked during exploration. It was found that target saliency had a large effect on the use of the hand parts and the strategies. In the non-salient target conditions, the fingers, especially the thumb, contacted the target more often than in the salient target conditions. This could also be seen in the strategies, where the thumb was used to explore the items in a serial way by moving them in the hand or touching them individually. In the salient target conditions, more parallel strategies like grasping or shuffling of the items in the hand were used.

Haptic discrimination of distance.

While quite some research has focussed on the accuracy of haptic perception of distance, information on the precision of haptic perception of distance is still scarce, particularly regarding distances perceived by making arm movements. In this study, eight conditions were measured to answer four main questions, which are: what is the influence of reference distance, movement axis, perceptual mode (active or passive) and stimulus type on the precision of this kind of distance perception? A discrimination experiment was performed with twelve participants. The participants were presented with two distances, using either a haptic device or a real stimulus. Participants compared the distances by moving their hand from a start to an end position. They were then asked to judge which of the distances was the longer, from which the discrimination threshold was determined for each participant and condition. The precision was influenced by reference distance. No effect of movement axis was found. The precision was higher for active than for passive movements and it was a bit lower for real stimuli than for rendered stimuli, but it was not affected by adding cutaneous information. Overall, the Weber fraction for the active perception of a distance of 25 or 35 cm was about 11% for all cardinal axes. The recorded position data suggest that participants, in order to be able to judge which distance was the longer, tried to produce similar speed profiles in both movements. This knowledge could be useful in the design of haptic devices.

Parallel processing of shape and texture in haptic search.

In a haptic search task, one has to determine the presence of a target among distractors. It has been shown that if the target differs from the distractors in two properties, shape and texture, performance is better than in both single-property conditions (Van Polanen, Bergmann Tiest, & Kappers, 2013). The search for a smooth sphere among rough cubical distractors was faster than both the searches for a rough sphere (shape information only) and for a smooth cube (texture information only). This effect was replicated in this study as a baseline. The main focus here was to further investigate the nature of this integration. It was shown that performance is better when the two properties are combined in a single target (smooth sphere), than when located in two separate targets (rough sphere and smooth cube) that are simultaneously present. A race model that assumes independent parallel processing of the two properties could explain the enhanced performance with two properties, but this could only take place effectively when the two properties were located in a single target.

Bimanual and unimanual length perception.

From previous studies, it is unclear how bimanual length discrimination differs from unimanual length discrimination. To investigate the difference, we designed an experiment with four conditions. In the first two conditions, unimanual and bimanual discrimination thresholds are determined. In the third and fourth conditions, length is explored with the two index fingers like in the bimanual condition, but the reference is either internal, by clasping the hands together, or external, by grasping handles connected to the table. We find that thresholds for the unimanual condition (7.0%) and the clasping condition (9.2%) are both lower than for the bimanual condition (16%) and the grasping handles condition (15%). We conclude that when discriminating length unimanually and bimanually while clasping the hands together, the internal reference within the hand can be used and that explains the lower discrimination thresholds.

Influence of shape on the haptic size aftereffect.

Recently, we showed a strong haptic size aftereffect by means of a size bisection task: after adaptation to a large sphere, subsequently grasped smaller test spheres felt even smaller, and vice versa. In the current study, we questioned whether the strength of this aftereffect depends on shape. In four experimental conditions, we determined the aftereffect after adaptation to spheres and tetrahedra and subsequent testing also with spheres and tetrahedra. The results showed a clear influence of shape: the haptic aftereffect was much stronger if adaptation and test stimuli were identical in shape than if their shapes were different. Therefore, it would be more appropriate to term such aftereffects haptic shape-size aftereffects, as size alone could not be the determining factor. This influence of shape suggests that higher cortical areas are involved in this aftereffect and that it cannot be due to adaptation of peripheral receptors. An additional finding is that the geometric property or combination of properties participants use in the haptic size bisection task varies widely over participants, although participants themselves are quite consistent.

Contact force and scanning velocity during active roughness perception.

Haptic perception is bidirectionally related to exploratory movements, which means that exploration influences perception, but perception also influences exploration. We can optimize or change exploratory movements according to the perception and/or the task, consciously or unconsciously. This paper presents a psychophysical experiment on active roughness perception to investigate movement changes as the haptic task changes. Exerted normal force and scanning velocity are measured in different perceptual tasks (discrimination or identification) using rough and smooth stimuli. The results show that humans use a greater variation in contact force for the smooth stimuli than for the rough stimuli. Moreover, they use higher scanning velocities and shorter break times between stimuli in the discrimination task than in the identification task. Thus, in roughness perception humans spontaneously use different strategies that seem effective for the perceptual task and the stimuli. A control task, in which the participants just explore the stimuli without any perceptual objective, shows that humans use a smaller contact force and a lower scanning velocity for the rough stimuli than for the smooth stimuli. Possibly, these strategies are related to aversiveness while exploring stimuli.

Illusory rotation in the haptic perception of a moving bar.

Haptic matching of the orientation of bars separated by a horizontal distance leads to large systematic deviations. This finding leads to the following intriguing question which we investigated in this study: How will a bar moving from left to right in a fixed orientation be perceived by blindfolded observers? Interestingly, this previous finding predicts that the translating bar will cause the illusory perception of a rotation. In our experiment, we used psychophysical methods to determine the rotation needed to perceive the bar as non-rotating, for both rightward and leftward translations. From our results, it can be estimated that, on average, a bar translating in parallel over 60 cm is perceived as rotating 18°, so we established that the predicted illusory rotation indeed exists. This implies that static and dynamic signals are processed in a similar way.

Integration and disruption effects of shape and texture in haptic search.

In a search task, where one has to search for the presence of a target among distractors, the target is sometimes easily found, whereas in other searches it is much harder to find. The performance in a search task is influenced by the identity of the target, the identity of the distractors and the differences between the two. In this study, these factors were manipulated by varying the target and distractors in shape (cube or sphere) and roughness (rough or smooth) in a haptic search task. Participants had to grasp a bundle of items and determine as fast as possible whether a predefined target was present or not. It was found that roughness and edges were relatively salient features and the search for the presence of these features was faster than for their absence. If the task was easy, the addition of these features could also disrupt performance, even if they were irrelevant for the search task. Another important finding was that the search for a target that differed in two properties from the distractors was faster than a task with only a single property difference, although this was only found if the two target properties were non-salient. This means that shape and texture can be effectively integrated. Finally, it was found that edges are more beneficial to a search task than disrupting, whereas for roughness this was the other way round.

Binding in haptics: integration of "what" and "where" information in working memory for active touch.

Information about the identity and the location of perceptual objects can be automatically integrated in perception and working memory (WM). Contrasting results in visual and auditory WM studies indicate that the characteristics of feature-to-location binding can vary according to the sensory modality of the input. The present study provides first evidence of binding between "what" and "where" information in WM for haptic stimuli. In an old-new recognition task, blindfolded participants were presented in their peripersonal space with sequences of three haptic stimuli varying in texture and location. They were then required to judge if a single probe stimulus was previously included in the sequence. Recall was measured both in a condition in which both texture and location were relevant for the task (Experiment 1) and in two conditions where only one feature had to be recalled (Experiment 2). Results showed that when both features were task-relevant, even if the association of location and texture was neither necessary nor required to perform the task, participants exhibited a recall advantage in conditions in which the location and the texture of the target probe was kept unaltered between encoding and recall. By contrast, when only one feature was task-relevant, the concurrent feature did not influence the recall of the target feature. We conclude that attention to feature binding is not necessary for the emergence of feature integration in haptic WM. For binding to take place, however, it is necessary to encode and maintain in memory both the identity and the location of items.

Haptic perception.

Fueled by novel applications, interest in haptic perception is growing. This paper provides an overview of the state of the art of a number of important aspects of haptic perception. By means of touch we can not only perceive quite different material properties, such as roughness, compliance, friction, coldness and slipperiness, but we can also perceive spatial properties, such as shape, curvature, size and orientation. Moreover, the number of objects we have in our hand can be determined, either by counting or subitizing. All these aspects will be presented and discussed in this paper. Although our intuition tells us that touch provides us with veridical information about our environment, the existence of prominent haptic illusions will show otherwise. Knowledge about haptic perception is interesting from a fundamental viewpoint, but it also is of eminent importance in the technological development of haptic devices. At the end of this paper, a few recent applications will be presented. WIREs Cogn Sci 2013, 4:357-374. DOI: 10.1002/wcs.1238 CONFLICT OF INTEREST: The authors have declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website.

Haptic discrimination and matching of viscosity.

In three experiments, viscosity perception of liquids using the sense of touch was studied. The first two were discrimination experiments in which Weber fractions were determined for a number of viscosities spanning the range of what is encountered in daily life, and for two ways of perceiving viscosity (stirring with a spatula or with the index finger). For high viscosities, Weber fractions were around 0.3, whereas they increased for lower viscosities. For low viscosities, discrimination performance was much worse with the finger than with the spatula. In the third experiment, subjects matched liquids perceived with these two methods, which resulted in biases of around 80. Control experiments and force measurements were performed to find an explanation for these results. It was concluded that the relationship between perceived and physical viscosity is steeper for stirring liquids with a spatula than stirring with the finger.