Sensory representation of surface reflectances: assessments with hyperspectral images.

Specifying surface reflectances in a simple and perceptually informative way would be beneficial for many areas of research and application. We assessed whether a 3×3 matrix may be used to approximate how a surface reflectance modulates the sensory color signal across illuminants. We tested whether observers could discriminate between the model's approximate and accurate spectral renderings of hyperspectral images under narrowband and naturalistic, broadband illuminants for eight hue directions. Discriminating the approximate from the spectral rendering was possible with narrowband, but almost never with broadband illuminants. These results suggest that our model specifies the sensory information of reflectances across naturalistic illuminants with high fidelity, and with lower computational cost than spectral rendering.

Tactile training facilitates infants' ability to reach to targets on the body.

This longitudinal study investigated the effect of experience with tactile stimulation on infants' ability to reach to targets on the body, an important adaptive skill. Infants were provided weekly tactile stimulation on eight body locations from 4 to 8 months of age (N = 11), comparing their ability to reach to the body to infants in a control group who did not receive stimulation (N = 10). Infants who received stimulation were more likely to successfully reach targets on the body than controls by 7 months of age. These findings indicate that tactile stimulation facilitates the development of reaching to the body by allowing infants to explore the sensorimotor correlations emerging from the stimulation.

The most reasonable explanation of "the dress": Implicit assumptions about illumination.

Millions of Internet users around the world challenged science by asking why a certain photo of a dress led different observers to have surprisingly different judgments about the color of the dress. The reason this particular photo produces so diverse a variety of judgments presumably is that the photo allows a variety of interpretations about the illumination of the dress. The most obvious explanation from color science should be that observers have different implicit assumptions about the illumination in the photo. We show that the perceived color of the dress is negatively correlated with the assumed illumination along the daylight locus. Moreover, by manipulating the observers' assumptions prior to seeing the photo, we can steer how observers will see the colors of the dress. These findings confirm the idea that the perceived colors of the dress depend on the assumptions about the illumination. The phenomenon illustrates the power of unconscious inferences and implicit assumptions in perception.

Uncertainty of sensory signal explains variation of color constancy.

Color constancy is the ability to recognize the color of an object (or more generally of a surface) under different illuminations. Without color constancy, surface color as a perceptual attribute would not be meaningful in the visual environment, where illumination changes all the time. Nevertheless, it is not obvious how color constancy is possible in the light of metamer mismatching. Surfaces that produce exactly the same sensory color signal under one illumination (metamerism) may produce utterly different sensory signals under another illumination (metamer mismatching). Here we show that this phenomenon explains to a large extent the variation of color constancy across different colors. For this purpose, color constancy was measured for different colors in an asymmetric matching task with photorealistic images. Color constancy performance was strongly correlated to the size of metamer mismatch volumes, which describe the uncertainty of the sensory signal due to metamer mismatching for a given color. The higher the uncertainty of the sensory signal, the lower the observers' color constancy. At the same time, sensory singularities, color categories, and cone ratios did not affect color constancy. The present findings do not only provide considerable insight into the determinants of color constancy, they also show that metamer mismatch volumes must be taken into account when investigating color as a perceptual property of objects and surfaces.

What determines the relationship between color naming, unique hues, and sensory singularities: Illuminations, surfaces, or photoreceptors?

The relationship between the sensory signal of the photoreceptors on one hand and color appearance and language on the other hand is completely unclear. A recent finding established a surprisingly accurate correlation between focal colors, unique hues, and so-called singularities in the laws governing how sensory signals for different surfaces change across illuminations. This article examines how this correlation with singularities depends on reflectances, illuminants, and cone sensitivities. Results show that this correlation holds for a large range of illuminants and for a large range of sensors, including sensors that are fundamentally different from human photoreceptors. In contrast, the spectral characteristics of the reflectance spectra turned out to be the key factor that determines the correlation between focal colors, unique hues, and sensory singularities. These findings suggest that the origins of color appearance and color language may be found in particular characteristics of the reflectance spectra that correspond to focal colors and unique hues.

Handedness in infants' tool use.

In this study, we investigated whether hand preference influences infants' choice of what hand to use in grasping a new tool presented at the midline, and whether this will change in the course of learning the functionality of a tool. The tool was a rake within reach placed beside an out-of-reach toy presented either to its right or to its left. Forty-eight infants from 16 to 22 months of age were tested. The results show that use of the right-preferred hand to grasp the rake is strong as of 16 months of age and does not change significantly with age in the condition where using the right hand leads to a better outcome than using the left hand. In the condition where using the left-non-preferred hand makes toy retrieval easier, infants increasingly use the left hand with age. Thus, when grasping the tool, younger infants are more influenced by their hand preference than older infants, who are better at anticipating the most successful strategies.

The emergence of tool use during the second year of life.

Despite a growing interest in the question of tool-use development in infants, no study so far has systematically investigated how learning to use a tool to retrieve an out-of-reach object progresses with age. This was the first aim of this study, in which 60 infants, aged 14, 16, 18, 20, and 22 months, were presented with an attractive toy and a rake-like tool. There were five conditions of spatial relationships between the toy and the tool, going from the toy and tool being connected to there being a large spatial gap between them. A second aim of the study was to evaluate at what age infants who spontaneously fail the task can learn this complex skill by being given a demonstration from an adult. Results show that even some of the youngest infants could spontaneously retrieve the toy when it was presented inside and touching the top part of the tool. In contrast, in conditions with a spatial gap, the first spontaneous successes were observed at 18 months, suggesting that a true understanding of the use of the tool has not been fully acquired before that age. Interestingly, it is also at 18 months that infants began to benefit from the demonstration in the conditions with a spatial gap. The developmental steps for tool use observed here are discussed in terms of changes in infants' ability to attend to more than one item in the environment. The work provides insight into the progressive understanding of tool use during infancy and into how observational learning improves with age.

A new spectrally sharpened sensor basis to predict color naming, unique hues, and hue cancellation.

When light is reflected off a surface, there is a linear relation between the three human photoreceptor responses to the incoming light and the three photoreceptor responses to the reflected light. Different colored surfaces have different linear relations. Recently, Philipona and O'Regan (2006) showed that when this relation is singular in a mathematical sense, then the surface is perceived as having a highly nameable color. Furthermore, white light reflected by that surface is perceived as corresponding precisely to one of the four psychophysically measured unique hues. However, Philipona and O'Regan's approach seems unrelated to classical psychophysical models of color constancy. In this paper we make this link. We begin by transforming cone sensors to spectrally sharpened counterparts. In sharp color space, illumination change can be modeled by simple von Kries type scalings of response values within each of the spectrally sharpened response channels. In this space, Philipona and O'Regan's linear relation is captured by a simple Land-type color designator defined by dividing reflected light by incident light. This link between Philipona and O'Regan's theory and Land's notion of color designator gives the model biological plausibility. We then show that Philipona and O'Regan's singular surfaces are surfaces which are very close to activating only one or only two of such newly defined spectrally sharpened sensors, instead of the usual three. Closeness to zero is quantified in a new simplified measure of singularity which is also shown to relate to the chromaticness of colors. As in Philipona and O'Regan's original work, our new theory accounts for a large variety of psychophysical color data.

How voluntary control over information and body movements determines "what it's like" to have perceptual, bodily, emotional and mental experiences.

Two very fundamental aspects of phenomenal experiences underline the fact that they seem to have "something it's like." One aspect is the fact that experiences have a locus: they Can seem "external" (perceptual), "internal" (interoceptive, bodily or emotional) or "mental." A second fundamental aspect is the imposingness of experiences. They can seem "present" to us in different ways, sometimes seeming displayed before us with "spatio-temporal presence." Both these aspects of "what it's like" can be identified with the degree to which we can voluntarily control what we are doing when we engage in an experience. The external/internal/mental dimension is determined by how our voluntary bodily actions can influence the sensorimotor flow of information. The degree of imposingness of experiences and their "spatio-temporal presence" Is determined by how our voluntary actions are impeded or assisted by innate, attention-grabbing mechanisms. By elucidating these two most fundamental aspects of "what it's like," and taken together with prior work on inter- and intra-modal differences in experiences, this article suggests a path toward a scientific theory of the "hard problem" of phenomenal consciousness, explaining why experiences feel like something rather than feeling like nothing.

Missing: Empirical theories of phenomenal consciousness.

Doerig et al. evaluate how current empirical theories approach access consciousness, but they neglect how they approach phenomenal consciousness - probably because most theories don't deal with phenomenal consciousness at all. One exception is the sensorimotor theory, but Doerig et al. did not evaluate it as being directed to phenomenal consciousness.

6-Month-Old Infants' Sensitivity to Contingency in a Variant of the Mobile Paradigm With Proximal Stimulation Studied at Fine Temporal Resolution in the Laboratory.

Infants' ability to monitor "sensorimotor contingencies," i.e., the sensory effects of their own actions, is an important mechanism underlying learning. One method that has been used to investigate this is the "mobile paradigm," in which a mobile above an infant's crib is activated by motion of one of the infant's limbs. Although successfully used in numerous experiments performed in infants' homes to investigate memory and other types of learning, the paradigm seems less robust for demonstrating sensitivity to sensorimotor contingencies when used in the laboratory. One purpose of the present work was to show that certain changes to the mobile paradigm would make it easier for infants to show their sensitivity to the contingency in the lab. In particular, we used proximal stimulation on infants' wrists instead of the usual mobile, and our stimulation was coincident with the limbs that caused it. Our stimulation was either on or off, i.e., not modulated by the amount the infant moved. Finally, we used a "shaping" procedure to help the infant discover the contingency. In addition to these changes in the paradigm, by analyzing infants' limb activity at 10-s resolution instead of the usual 1-min resolution, we were able to show that infants' sensitivity to the contingency became apparent already within the first minute of establishment of the contingency. Finally, we showed how two alternate measures of sensitivity to contingency based on probability of repeated movements and on "stop and go" motion strategies may be of interest for future work.

Detection of sensorimotor contingencies in infants before the age of 1 year: A comprehensive review.

To benefit from the exploration of their bodies and their physical and social environments, infants need to detect sensorimotor contingencies linking their actions to sensory feedback. This ability, which seems to be present in babies from birth and even in utero, has been widely used by researchers in their study of early development. However, a careful review of the literature, particularly the recent literature, suggests that babies may not be uniformly sensitive to all sensorimotor contingencies. This literature review examines in detail sensorimotor contingency detection in infants before the age of 1 year. Four aspects of sensorimotor contingency detection are considered: characteristics of action and feedback, contingency parameters, exposure conditions, and interindividual differences. For each topic, we highlight what favors and what hinders the detection of sensorimotor contingencies in infants. Our review also demonstrates the limitations of our knowledge about sensorimotor contingency detection. We emphasize the importance of making progress in this field at a time when sensorimotor contingency detection is of major interest in developmental robotics and artificial intelligence. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Sensorimotor Contingencies as a Key Drive of Development: From Babies to Robots.

Much current work in robotics focuses on the development of robots capable of autonomous unsupervised learning. An essential prerequisite for such learning to be possible is that the agent should be sensitive to the link between its actions and the consequences of its actions, called sensorimotor contingencies. This sensitivity, and more particularly its role as a key drive of development, has been widely studied by developmental psychologists. However, the results of these studies may not necessarily be accessible or intelligible to roboticians. In this paper, we review the main experimental data demonstrating the role of sensitivity to sensorimotor contingencies in infants' acquisition of four fundamental motor and cognitive abilities: body knowledge, memory, generalization, and goal-directedness. We relate this data from developmental psychology to work in robotics, highlighting the links between these two domains of research. In the last part of the article we present a blueprint architecture demonstrating how exploitation of sensitivity to sensorimotor contingencies, combined with the notion of "goal," allows an agent to develop new sensorimotor skills. This architecture can be used to guide the design of specific computational models, and also to possibly envisage new empirical experiments.

Convergent flash localization near saccades without equivalent "compression" of perceived separation.

Visual space is sometimes said to be "compressed" before saccadic eye movements. The most central evidence for this hypothesis is a converging pattern of localization errors on single flashes presented close to saccade time under certain conditions. An intuitive version of the compression hypothesis predicts that the reported distance between simultaneous, spatially separated presaccadic flashes should contract in the same way as their individual locations. In our experiment we tested this prediction by having subjects perform one of two tasks on stimuli made up of two bars simultaneously flashed near saccade time: either localizing one of the bars or judging the separation between the two. Localization judgments showed the previously observed converging pattern over the 50-100 ms before saccades. Contractions in perceived separation between the two bars were not accurately predicted by this pattern: they occurred mainly during saccades and were much weaker than convergence in localization. Different forms of spatial information about flashed stimuli can be differentially modulated before, during, and after saccades. Structural alterations in the perceptual field around saccades may explain these different effects, but alternative hypotheses based on decision making under uncertainty and on the influence of other perisaccadic mechanisms are also consistent with this and other evidence.

Erratum: Sensory augmentation: integration of an auditory compass signal into human perception of space.

This corrects the article DOI: 10.1038/srep42197.

Discovering space - Grounding spatial topology and metric regularity in a naive agent's sensorimotor experience.

In line with the sensorimotor contingency theory, we investigate the problem of the perception of space from a fundamental sensorimotor perspective. Despite its pervasive nature in our perception of the world, the origin of the concept of space remains largely mysterious. For example in the context of artificial perception, this issue is usually circumvented by having engineers pre-define the spatial structure of the problem the agent has to face. We here show that the structure of space can be autonomously discovered by a naive agent in the form of sensorimotor regularities, that correspond to so called compensable sensory experiences: these are experiences that can be generated either by the agent or its environment. By detecting such compensable experiences the agent can infer the topological and metric structure of the external space in which its body is moving. We propose a theoretical description of the nature of these regularities and illustrate the approach on a simulated robotic arm equipped with an eye-like sensor, and which interacts with an object. Finally we show how these regularities can be used to build an internal representation of the sensor's external spatial configuration.

Which limb is it? Responses to vibrotactile stimulation in early infancy.

This study focuses on how the body schema develops during the first months of life, by investigating infants' motor responses to localized vibrotactile stimulation on their limbs. Vibrotactile stimulation was provided by small buzzers that were attached to the infants' four limbs one at a time. Four age groups were compared cross-sectionally (3-, 4-, 5-, and 6-month-olds). We show that before they actually reach for the buzzer, which, according to previous studies, occurs around 7-8 months of age, infants demonstrate emerging knowledge about their body's configuration by producing specific movement patterns associated with the stimulated body area. At 3 months, infants responded with an increase in general activity when the buzzer was placed on the body, independently of the vibrator's location. Differentiated topographical awareness of the body seemed to appear around 5 months, with specific responses resulting from stimulation of the hands emerging first, followed by the differentiation of movement patterns associated with the stimulation of the feet. Qualitative analyses revealed specific movement types reliably associated with each stimulated location by 6 months of age, possibly preparing infants' ability to actually reach for the vibrating target. We discuss this result in relation to newborns' ability to learn specific movement patterns through intersensory contingency. Statement of contribution what is already known on infants' sensorimotor knowledge about their own bodies 3-month-olds readily learn to produce specific limb movements to obtain a desired effect (movement of a mobile). infants detect temporal and spatial correspondences between events involving their own body and visual events. what the present study adds until 4-5 months of age, infants mostly produce general motor responses to localized touch. this is because in the present study, infants could not rely on immediate contingent feedback. we propose a cephalocaudal developmental trend of topographic differentiation of body areas.

Why Early Tactile Speech Aids May Have Failed: No Perceptual Integration of Tactile and Auditory Signals.

Tactile speech aids, though extensively studied in the 1980's and 1990's, never became a commercial success. A hypothesis to explain this failure might be that it is difficult to obtain true perceptual integration of a tactile signal with information from auditory speech: exploitation of tactile cues from a tactile aid might require cognitive effort and so prevent speech understanding at the high rates typical of everyday speech. To test this hypothesis, we attempted to create true perceptual integration of tactile with auditory information in what might be considered the simplest situation encountered by a hearing-impaired listener. We created an auditory continuum between the syllables /BA/ and /VA/, and trained participants to associate /BA/ to one tactile stimulus and /VA/ to another tactile stimulus. After training, we tested if auditory discrimination along the continuum between the two syllables could be biased by incongruent tactile stimulation. We found that such a bias occurred only when the tactile stimulus was above, but not when it was below its previously measured tactile discrimination threshold. Such a pattern is compatible with the idea that the effect is due to a cognitive or decisional strategy, rather than to truly perceptual integration. We therefore ran a further study (Experiment 2), where we created a tactile version of the McGurk effect. We extensively trained two Subjects over 6 days to associate four recorded auditory syllables with four corresponding apparent motion tactile patterns. In a subsequent test, we presented stimulation that was either congruent or incongruent with the learnt association, and asked Subjects to report the syllable they perceived. We found no analog to the McGurk effect, suggesting that the tactile stimulation was not being perceptually integrated with the auditory syllable. These findings strengthen our hypothesis according to which tactile aids failed because integration of tactile cues with auditory speech occurred at a cognitive or decisional level, rather than truly at a perceptual level.

Sensory augmentation: integration of an auditory compass signal into human perception of space.

Bio-mimetic approaches to restoring sensory function show great promise in that they rapidly produce perceptual experience, but have the disadvantage of being invasive. In contrast, sensory substitution approaches are non-invasive, but may lead to cognitive rather than perceptual experience. Here we introduce a new non-invasive approach that leads to fast and truly perceptual experience like bio-mimetic techniques. Instead of building on existing circuits at the neural level as done in bio-mimetics, we piggy-back on sensorimotor contingencies at the stimulus level. We convey head orientation to geomagnetic North, a reliable spatial relation not normally sensed by humans, by mimicking sensorimotor contingencies of distal sounds via head-related transfer functions. We demonstrate rapid and long-lasting integration into the perception of self-rotation. Short training with amplified or reduced rotation gain in the magnetic signal can expand or compress the perceived extent of vestibular self-rotation, even with the magnetic signal absent in the test. We argue that it is the reliability of the magnetic signal that allows vestibular spatial recalibration, and the coding scheme mimicking sensorimotor contingencies of distal sounds that permits fast integration. Hence we propose that contingency-mimetic feedback has great potential for creating sensory augmentation devices that achieve fast and genuinely perceptual experiences.

The dress and individual differences in the perception of surface properties.

This study investigates systematic individual differences in the way observers perceive different kinds of surface properties and their relationship to the dress, which shows striking individual differences in colour perception. We tested whether these individual differences have a common source, namely differences in perceptual strategies according to which observers attribute features in two-dimensional images to surfaces or to their illumination. First, we reanalysed data from two previous experiments on the dress and colour constancy. The comparison of the two experiments revealed that the colour perception of the dress is strongly related to individual differences in colour constancy. Second, two online surveys measured individual differences in the perception of colour-ambiguous images including the dress, in colour constancy, in gloss perception, in the subjective grey-point, in colour naming, and in the perception of an image with ambiguous shading. The results of the surveys replicated and extended previous findings according to which individual differences in the colour perception of the dress are due to implicit assumptions about the illumination. However, results also showed that the individual differences for other phenomena were independent of the dress and of each other. Overall, these results suggest that the striking individual differences in dress colour perception are due to individual differences in the interpretation of illumination cues to achieve colour constancy. At the same time, they undermine the idea of an overall perceptual strategy that encompasses other phenomena more generally related to the interpretation of illumination and surface properties.