Organic Haptics: Intersection of Materials Chemistry and Tactile Perception.

The goal of the field of haptics is to create technologies that manipulate the sense of touch. In virtual and augmented reality, haptic devices are for touch what loudspeakers and RGB displays are for hearing and vision. Haptic systems that utilize micromotors or other miniaturized mechanical devices (e.g., for vibration and pneumatic actuation) produce interesting effects, but are quite far from reproducing the feeling of real materials. They are especially deficient in recapitulating surface properties: fine texture, friction, viscoelasticity, tack, and softness. The central argument of this Progress Report is that to reproduce the feel of everyday objects requires chemistry: molecular control over the properties of materials and ultimately design of materials which can change these properties in real time. Stimuli-responsive organic materials, such as polymers and composites, are a class of materials which can change their oxidation state, conductivity, shape, and rheological properties, and thus might be useful in future haptic technologies. Moreover, the use of such materials in research on tactile perception could help elucidate the limits of human tactile sensitivity. The work described represents the beginnings of this new area of inquiry, in which the defining approach is the marriage of materials science and psychology.

External self-representations improve self-awareness in a child with autism.

We have previously suggested that the social symptoms of autism spectrum disorder (ASD) could be caused in part by a dysfunctional mirror neuron system (MNS). Since the recursive activity of a functioning MNS might enable the brain to integrate visual and motor sensations into a coherent body schema, the deficits in self-awareness often seen in ASD might be caused by the same mirror neuron dysfunction. CL is an autistic adolescent who is profoundly fascinated with his reflection, looking in mirrors at every opportunity. We demonstrate that CL's abnormal gait improves significantly when using a mirror for visual feedback. We also show that both the fascination and the happiness that CL derives from looking at a computer-generated reflection diminish when a delay is introduced between the camera input and screen output. We believe that immediate, real-time visual feedback allows CL to integrate motor sensations with external visual ones into a coherent body schema that he cannot internally generate, perhaps due to a dysfunctional MNS.

Electropneumotactile Stimulation: Multimodal Haptic Actuators Enabled by a Stretchable Conductive Polymer on Inflatable Pockets.

This paper describes a type of haptic device that delivers two modes of stimulation simultaneously and at the same locations on the skin. The two modes of stimulation are mechanical (delivered pneumatically by inflatable air pockets embedded within a silicone elastomer) and electrical (delivered by a conductive polymer). The key enabling aspect of this work is the use of a highly plasticized conductive polymer based on poly(3,4-ethylenedioxythiphene) (PEDOT) blended with elastomeric polyurethane (PU). To fabricate the "electropneumotactile" device, the polymeric electrodes are overlaid directly on top of the elastomeric pneumatic actuator pockets. Co-placement of the pneumatic actuators and the electrotactile electrodes is enabled by the stretchability of the PEDOT:OTs/PU blend, allowing the electrotactiles to conform to underlying pneumatic pockets under deformation. The blend of PEDOT and PU has a Young's modulus of ~150 MPa with little degradation in conductivity following repeated inflation of the air pockets. The ability to perceive simultaneous delivery of two sensations to the same location on the skin are supported by experiments using human subjects. These results show that participants can successfully detect the location of pneumatic stimulation and whether electrotactile stimulation is delivered (yes/no) at a rate significantly above chance (mean accuracy = 94%).

Virtual Texture Generated using Elastomeric Conductive Block Copolymer in Wireless Multimodal Haptic Glove.

Haptic devices are in general more adept at mimicking the bulk properties of materials than they are at mimicking the surface properties. This paper describes a haptic glove capable of producing sensations reminiscent of three types of near-surface properties: hardness, temperature, and roughness. To accomplish this mixed mode of stimulation, three types of haptic actuators were combined: vibrotactile motors, thermoelectric devices, and electrotactile electrodes made from a stretchable conductive polymer synthesized in our laboratory. This polymer consisted of a stretchable polyanion which served as a scaffold for the polymerization of poly(3,4-ethylenedioxythiophene) (PEDOT). The scaffold was synthesized using controlled radical polymerization to afford material of low dispersity, relatively high conductivity (0.1 S cm-1), and low impedance relative to metals. The glove was equipped with flex sensors to make it possible to control a robotic hand and a hand in virtual reality (VR). In psychophysical experiments, human participants were able to discern combinations of electrotactile, vibrotactile, and thermal stimulation in VR. Participants trained to associate these sensations with roughness, hardness, and temperature had an overall accuracy of 98%, while untrained participants had an accuracy of 85%. Sensations could similarly be conveyed using a robotic hand equipped with sensors for pressure and temperature.

Distinct colours in the 'synaesthetic colour palette'.

In grapheme-colour synaesthesia, particular linguistic elements evoke particular colour sensations. Interestingly, when asked, non-synaesthetes can also associate colours to letters, and previous studies show that specific letter-to-colour associations have similar biases to those of synaesthetes. However, it is an open question whether the colours reported by synaesthetes and non-synaesthetes differ overall: is there a 'synaesthetic colour palette'? In this study, we visualize the overall distribution in colour space of colour concurrents in grapheme-colour synaesthetes, and colour associations in non-synaesthetic controls. We confirm the existence of a synaesthetic colour palette: colour concurrents in synaesthetes are different from colour associations in non-synaesthetes. We quantify three factors that distinguish the colour palette of synaesthetes and non-synaesthetes: synaesthetes have an increased over-representation of 'pure' (unmixed) hues, an increased presence of 'warm' (yellow, orange, brown) colours, and an increased presence of achromatic (grey, white, black) colours. Furthermore, we demonstrate that differences in the synaesthetic colour palette can be used to train a machine learning algorithm to reliably classify single subjects as synaesthetes versus non-synaesthetes without using test-retest consistency data. As far as we know, this is the first time an individual could be 'diagnosed' as a synaesthete, based only on his or her colours evoked by letters. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Echoes from the past: synaesthetic colour associations reflect childhood gender stereotypes.

Grapheme-colour synaesthesia is a neurological phenomenon in which linguistic symbols evoke consistent colour sensations. Synaesthesia is believed to be influenced by both genetic and environmental factors, but how these factors interact to create specific associations in specific individuals is poorly understood. In this paper, we show that a grapheme-colour association in adult synaesthetes can be traced to a particular environmental effect at a particular moment in childhood. We propose a model in which specific grapheme-colour associations are 'locked in' during development in children predisposed to become synaesthetes, whereas grapheme-colour associations remain flexible in non-synaesthetes. We exploit Western gender-colour stereotypes to test our model: we found that young girls in general tend to associate their first initial with the colour pink. Consistent with our model, adult female synaesthetes are influenced by their childhood environment: they associate their first initial with pink. Adult female non-synaesthetes do not show this bias. Instead, in our study, non-synaesthetes tended to associate their first initial with their current favourite colour. The results thus support the 'locking in' model of synaesthesia, suggesting that synaesthetic associations can be used as a 'time capsule', revealing childhood influences on adult linguistic associations. Grapheme-colour synaesthesia may thus offer an extraordinary opportunity to study linguistic development. This article is part of a discussion meeting issue 'Bridging senses: novel insights from synaesthesia'.

Role of indentation depth and contact area on human perception of softness for haptic interfaces.

In engineering, the "softness" of an object, as measured by an indenter, manifests as two measurable parameters: (i) indentation depth and (ii) contact area. For humans, softness is not well defined, although it is believed that perception depends on the same two parameters. Decoupling their relative contributions, however, has not been straightforward because most bulk-"off-the-shelf"-materials exhibit the same ratio between the indentation depth and contact area. Here, we decoupled indentation depth and contact area by fabricating elastomeric slabs with precise thicknesses and microstructured surfaces. Human subject experiments using two-alternative forced-choice and magnitude estimation tests showed that the indentation depth and contact area contributed independently to perceived softness. We found an explicit relationship between the perceived softness of an object and its geometric properties. Using this approach, it is possible to design objects for human interaction with a desired level of perceived softness.

Why is the synesthete's "A" red? Using a five-language dataset to disentangle the effects of shape, sound, semantics, and ordinality on inducer-concurrent relationships in grapheme-color synesthesia.

Grapheme-color synesthesia is a neurological phenomenon in which viewing a grapheme elicits an additional, automatic, and consistent sensation of color. Color-to-letter associations in synesthesia are interesting in their own right, but also offer an opportunity to examine relationships between visual, acoustic, and semantic aspects of language. Research using large populations of synesthetes has indeed found that grapheme-color pairings can be influenced by numerous properties of graphemes, but the contributions made by each of these explanatory factors are often confounded in a monolingual dataset (i.e., only English-speaking synesthetes). Here, we report the first demonstration of how a multilingual dataset can reveal potentially-universal influences on synesthetic associations, and disentangle previously-confounded hypotheses about the relationship between properties of synesthetic color and properties of the grapheme that induces it. Numerous studies have reported that for English-speaking synesthetes, "A" tends to be colored red more often than predicted by chance, and several explanatory factors have been proposed that could explain this association. Using a five-language dataset (native English, Dutch, Spanish, Japanese, and Korean speakers), we compare the predictions made by each explanatory factor, and show that only an ordinal explanation makes consistent predictions across all five languages, suggesting that the English "A" is red because the first grapheme of a synesthete's alphabet or syllabary tends to be associated with red. We propose that the relationship between the first grapheme and the color red is an association between an unusually-distinct ordinal position ("first") and an unusually-distinct color (red). We test the predictions made by this theory, and demonstrate that the first grapheme is unusually distinct (has a color that is distant in color space from the other letters' colors). Our results demonstrate the importance of considering cross-linguistic similarities and differences in synesthesia, and suggest that some influences on grapheme-color associations in synesthesia might be universal.