Sensory representation of surface reflectances: assessments with hyperspectral images.

Specifying surface reflectances in a simple and perceptually informative way would be beneficial for many areas of research and application. We assessed whether a 3×3 matrix may be used to approximate how a surface reflectance modulates the sensory color signal across illuminants. We tested whether observers could discriminate between the model's approximate and accurate spectral renderings of hyperspectral images under narrowband and naturalistic, broadband illuminants for eight hue directions. Discriminating the approximate from the spectral rendering was possible with narrowband, but almost never with broadband illuminants. These results suggest that our model specifies the sensory information of reflectances across naturalistic illuminants with high fidelity, and with lower computational cost than spectral rendering.

Tactile training facilitates infants' ability to reach to targets on the body.

This longitudinal study investigated the effect of experience with tactile stimulation on infants' ability to reach to targets on the body, an important adaptive skill. Infants were provided weekly tactile stimulation on eight body locations from 4 to 8 months of age (N = 11), comparing their ability to reach to the body to infants in a control group who did not receive stimulation (N = 10). Infants who received stimulation were more likely to successfully reach targets on the body than controls by 7 months of age. These findings indicate that tactile stimulation facilitates the development of reaching to the body by allowing infants to explore the sensorimotor correlations emerging from the stimulation.

How voluntary control over information and body movements determines "what it's like" to have perceptual, bodily, emotional and mental experiences.

Two very fundamental aspects of phenomenal experiences underline the fact that they seem to have "something it's like." One aspect is the fact that experiences have a locus: they Can seem "external" (perceptual), "internal" (interoceptive, bodily or emotional) or "mental." A second fundamental aspect is the imposingness of experiences. They can seem "present" to us in different ways, sometimes seeming displayed before us with "spatio-temporal presence." Both these aspects of "what it's like" can be identified with the degree to which we can voluntarily control what we are doing when we engage in an experience. The external/internal/mental dimension is determined by how our voluntary bodily actions can influence the sensorimotor flow of information. The degree of imposingness of experiences and their "spatio-temporal presence" Is determined by how our voluntary actions are impeded or assisted by innate, attention-grabbing mechanisms. By elucidating these two most fundamental aspects of "what it's like," and taken together with prior work on inter- and intra-modal differences in experiences, this article suggests a path toward a scientific theory of the "hard problem" of phenomenal consciousness, explaining why experiences feel like something rather than feeling like nothing.

6-Month-Old Infants' Sensitivity to Contingency in a Variant of the Mobile Paradigm With Proximal Stimulation Studied at Fine Temporal Resolution in the Laboratory.

Infants' ability to monitor "sensorimotor contingencies," i.e., the sensory effects of their own actions, is an important mechanism underlying learning. One method that has been used to investigate this is the "mobile paradigm," in which a mobile above an infant's crib is activated by motion of one of the infant's limbs. Although successfully used in numerous experiments performed in infants' homes to investigate memory and other types of learning, the paradigm seems less robust for demonstrating sensitivity to sensorimotor contingencies when used in the laboratory. One purpose of the present work was to show that certain changes to the mobile paradigm would make it easier for infants to show their sensitivity to the contingency in the lab. In particular, we used proximal stimulation on infants' wrists instead of the usual mobile, and our stimulation was coincident with the limbs that caused it. Our stimulation was either on or off, i.e., not modulated by the amount the infant moved. Finally, we used a "shaping" procedure to help the infant discover the contingency. In addition to these changes in the paradigm, by analyzing infants' limb activity at 10-s resolution instead of the usual 1-min resolution, we were able to show that infants' sensitivity to the contingency became apparent already within the first minute of establishment of the contingency. Finally, we showed how two alternate measures of sensitivity to contingency based on probability of repeated movements and on "stop and go" motion strategies may be of interest for future work.

Missing: Empirical theories of phenomenal consciousness.

Doerig et al. evaluate how current empirical theories approach access consciousness, but they neglect how they approach phenomenal consciousness - probably because most theories don't deal with phenomenal consciousness at all. One exception is the sensorimotor theory, but Doerig et al. did not evaluate it as being directed to phenomenal consciousness.

Detection of sensorimotor contingencies in infants before the age of 1 year: A comprehensive review.

To benefit from the exploration of their bodies and their physical and social environments, infants need to detect sensorimotor contingencies linking their actions to sensory feedback. This ability, which seems to be present in babies from birth and even in utero, has been widely used by researchers in their study of early development. However, a careful review of the literature, particularly the recent literature, suggests that babies may not be uniformly sensitive to all sensorimotor contingencies. This literature review examines in detail sensorimotor contingency detection in infants before the age of 1 year. Four aspects of sensorimotor contingency detection are considered: characteristics of action and feedback, contingency parameters, exposure conditions, and interindividual differences. For each topic, we highlight what favors and what hinders the detection of sensorimotor contingencies in infants. Our review also demonstrates the limitations of our knowledge about sensorimotor contingency detection. We emphasize the importance of making progress in this field at a time when sensorimotor contingency detection is of major interest in developmental robotics and artificial intelligence. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Sensorimotor Contingencies as a Key Drive of Development: From Babies to Robots.

Much current work in robotics focuses on the development of robots capable of autonomous unsupervised learning. An essential prerequisite for such learning to be possible is that the agent should be sensitive to the link between its actions and the consequences of its actions, called sensorimotor contingencies. This sensitivity, and more particularly its role as a key drive of development, has been widely studied by developmental psychologists. However, the results of these studies may not necessarily be accessible or intelligible to roboticians. In this paper, we review the main experimental data demonstrating the role of sensitivity to sensorimotor contingencies in infants' acquisition of four fundamental motor and cognitive abilities: body knowledge, memory, generalization, and goal-directedness. We relate this data from developmental psychology to work in robotics, highlighting the links between these two domains of research. In the last part of the article we present a blueprint architecture demonstrating how exploitation of sensitivity to sensorimotor contingencies, combined with the notion of "goal," allows an agent to develop new sensorimotor skills. This architecture can be used to guide the design of specific computational models, and also to possibly envisage new empirical experiments.

Discovering space - Grounding spatial topology and metric regularity in a naive agent's sensorimotor experience.

In line with the sensorimotor contingency theory, we investigate the problem of the perception of space from a fundamental sensorimotor perspective. Despite its pervasive nature in our perception of the world, the origin of the concept of space remains largely mysterious. For example in the context of artificial perception, this issue is usually circumvented by having engineers pre-define the spatial structure of the problem the agent has to face. We here show that the structure of space can be autonomously discovered by a naive agent in the form of sensorimotor regularities, that correspond to so called compensable sensory experiences: these are experiences that can be generated either by the agent or its environment. By detecting such compensable experiences the agent can infer the topological and metric structure of the external space in which its body is moving. We propose a theoretical description of the nature of these regularities and illustrate the approach on a simulated robotic arm equipped with an eye-like sensor, and which interacts with an object. Finally we show how these regularities can be used to build an internal representation of the sensor's external spatial configuration.

Why Early Tactile Speech Aids May Have Failed: No Perceptual Integration of Tactile and Auditory Signals.

Tactile speech aids, though extensively studied in the 1980's and 1990's, never became a commercial success. A hypothesis to explain this failure might be that it is difficult to obtain true perceptual integration of a tactile signal with information from auditory speech: exploitation of tactile cues from a tactile aid might require cognitive effort and so prevent speech understanding at the high rates typical of everyday speech. To test this hypothesis, we attempted to create true perceptual integration of tactile with auditory information in what might be considered the simplest situation encountered by a hearing-impaired listener. We created an auditory continuum between the syllables /BA/ and /VA/, and trained participants to associate /BA/ to one tactile stimulus and /VA/ to another tactile stimulus. After training, we tested if auditory discrimination along the continuum between the two syllables could be biased by incongruent tactile stimulation. We found that such a bias occurred only when the tactile stimulus was above, but not when it was below its previously measured tactile discrimination threshold. Such a pattern is compatible with the idea that the effect is due to a cognitive or decisional strategy, rather than to truly perceptual integration. We therefore ran a further study (Experiment 2), where we created a tactile version of the McGurk effect. We extensively trained two Subjects over 6 days to associate four recorded auditory syllables with four corresponding apparent motion tactile patterns. In a subsequent test, we presented stimulation that was either congruent or incongruent with the learnt association, and asked Subjects to report the syllable they perceived. We found no analog to the McGurk effect, suggesting that the tactile stimulation was not being perceptually integrated with the auditory syllable. These findings strengthen our hypothesis according to which tactile aids failed because integration of tactile cues with auditory speech occurred at a cognitive or decisional level, rather than truly at a perceptual level.

Erratum: Sensory augmentation: integration of an auditory compass signal into human perception of space.

This corrects the article DOI: 10.1038/srep42197.

Which limb is it? Responses to vibrotactile stimulation in early infancy.

This study focuses on how the body schema develops during the first months of life, by investigating infants' motor responses to localized vibrotactile stimulation on their limbs. Vibrotactile stimulation was provided by small buzzers that were attached to the infants' four limbs one at a time. Four age groups were compared cross-sectionally (3-, 4-, 5-, and 6-month-olds). We show that before they actually reach for the buzzer, which, according to previous studies, occurs around 7-8 months of age, infants demonstrate emerging knowledge about their body's configuration by producing specific movement patterns associated with the stimulated body area. At 3 months, infants responded with an increase in general activity when the buzzer was placed on the body, independently of the vibrator's location. Differentiated topographical awareness of the body seemed to appear around 5 months, with specific responses resulting from stimulation of the hands emerging first, followed by the differentiation of movement patterns associated with the stimulation of the feet. Qualitative analyses revealed specific movement types reliably associated with each stimulated location by 6 months of age, possibly preparing infants' ability to actually reach for the vibrating target. We discuss this result in relation to newborns' ability to learn specific movement patterns through intersensory contingency. Statement of contribution what is already known on infants' sensorimotor knowledge about their own bodies 3-month-olds readily learn to produce specific limb movements to obtain a desired effect (movement of a mobile). infants detect temporal and spatial correspondences between events involving their own body and visual events. what the present study adds until 4-5 months of age, infants mostly produce general motor responses to localized touch. this is because in the present study, infants could not rely on immediate contingent feedback. we propose a cephalocaudal developmental trend of topographic differentiation of body areas.

The dress and individual differences in the perception of surface properties.

This study investigates systematic individual differences in the way observers perceive different kinds of surface properties and their relationship to the dress, which shows striking individual differences in colour perception. We tested whether these individual differences have a common source, namely differences in perceptual strategies according to which observers attribute features in two-dimensional images to surfaces or to their illumination. First, we reanalysed data from two previous experiments on the dress and colour constancy. The comparison of the two experiments revealed that the colour perception of the dress is strongly related to individual differences in colour constancy. Second, two online surveys measured individual differences in the perception of colour-ambiguous images including the dress, in colour constancy, in gloss perception, in the subjective grey-point, in colour naming, and in the perception of an image with ambiguous shading. The results of the surveys replicated and extended previous findings according to which individual differences in the colour perception of the dress are due to implicit assumptions about the illumination. However, results also showed that the individual differences for other phenomena were independent of the dress and of each other. Overall, these results suggest that the striking individual differences in dress colour perception are due to individual differences in the interpretation of illumination cues to achieve colour constancy. At the same time, they undermine the idea of an overall perceptual strategy that encompasses other phenomena more generally related to the interpretation of illumination and surface properties.

Sensory augmentation: integration of an auditory compass signal into human perception of space.

Bio-mimetic approaches to restoring sensory function show great promise in that they rapidly produce perceptual experience, but have the disadvantage of being invasive. In contrast, sensory substitution approaches are non-invasive, but may lead to cognitive rather than perceptual experience. Here we introduce a new non-invasive approach that leads to fast and truly perceptual experience like bio-mimetic techniques. Instead of building on existing circuits at the neural level as done in bio-mimetics, we piggy-back on sensorimotor contingencies at the stimulus level. We convey head orientation to geomagnetic North, a reliable spatial relation not normally sensed by humans, by mimicking sensorimotor contingencies of distal sounds via head-related transfer functions. We demonstrate rapid and long-lasting integration into the perception of self-rotation. Short training with amplified or reduced rotation gain in the magnetic signal can expand or compress the perceived extent of vestibular self-rotation, even with the magnetic signal absent in the test. We argue that it is the reliability of the magnetic signal that allows vestibular spatial recalibration, and the coding scheme mimicking sensorimotor contingencies of distal sounds that permits fast integration. Hence we propose that contingency-mimetic feedback has great potential for creating sensory augmentation devices that achieve fast and genuinely perceptual experiences.

The most reasonable explanation of "the dress": Implicit assumptions about illumination.

Millions of Internet users around the world challenged science by asking why a certain photo of a dress led different observers to have surprisingly different judgments about the color of the dress. The reason this particular photo produces so diverse a variety of judgments presumably is that the photo allows a variety of interpretations about the illumination of the dress. The most obvious explanation from color science should be that observers have different implicit assumptions about the illumination in the photo. We show that the perceived color of the dress is negatively correlated with the assumed illumination along the daylight locus. Moreover, by manipulating the observers' assumptions prior to seeing the photo, we can steer how observers will see the colors of the dress. These findings confirm the idea that the perceived colors of the dress depend on the assumptions about the illumination. The phenomenon illustrates the power of unconscious inferences and implicit assumptions in perception.

Uncertainty of sensory signal explains variation of color constancy.

Color constancy is the ability to recognize the color of an object (or more generally of a surface) under different illuminations. Without color constancy, surface color as a perceptual attribute would not be meaningful in the visual environment, where illumination changes all the time. Nevertheless, it is not obvious how color constancy is possible in the light of metamer mismatching. Surfaces that produce exactly the same sensory color signal under one illumination (metamerism) may produce utterly different sensory signals under another illumination (metamer mismatching). Here we show that this phenomenon explains to a large extent the variation of color constancy across different colors. For this purpose, color constancy was measured for different colors in an asymmetric matching task with photorealistic images. Color constancy performance was strongly correlated to the size of metamer mismatch volumes, which describe the uncertainty of the sensory signal due to metamer mismatching for a given color. The higher the uncertainty of the sensory signal, the lower the observers' color constancy. At the same time, sensory singularities, color categories, and cone ratios did not affect color constancy. The present findings do not only provide considerable insight into the determinants of color constancy, they also show that metamer mismatch volumes must be taken into account when investigating color as a perceptual property of objects and surfaces.

When do infants understand that they can obtain a desired part of a composite object by grasping another part?

When do infants start to understand that they can grasp an object by its handle when the interesting part is out of reach? Whereas it is known from preferential looking tasks that already at three months of age infants show surprise when all parts of an object do not move together, little is known about when infants are able to use such knowledge in an active grasp situation. To answer this question we presented six, eight, and 10 month-old infants in a cross-sectional and a longitudinal study with a white cardboard handle within reach and a bright ball at the end of the handle and out of reach. A trick condition, where the handle and the ball seem attached but were not, was added to get an indication of the infant's expectation by observing a possible surprise reaction. Results show that 6-month-olds' most frequent first behaviors consisted in pointing toward the ball without grasping the handle, or grasping the handle without looking at the ball until it moved. In addition, they often did not look surprised in the trick condition. Eight- and 10-month-olds most often grasped the handle while looking at the ball, and showed clear surprise in the trick condition. This was interpreted as showing that around eight or 10 months, infants take a significant step in understanding the cohesiveness of composite objects during grasping.

What determines the relationship between color naming, unique hues, and sensory singularities: Illuminations, surfaces, or photoreceptors?

The relationship between the sensory signal of the photoreceptors on one hand and color appearance and language on the other hand is completely unclear. A recent finding established a surprisingly accurate correlation between focal colors, unique hues, and so-called singularities in the laws governing how sensory signals for different surfaces change across illuminations. This article examines how this correlation with singularities depends on reflectances, illuminants, and cone sensitivities. Results show that this correlation holds for a large range of illuminants and for a large range of sensors, including sensors that are fundamentally different from human photoreceptors. In contrast, the spectral characteristics of the reflectance spectra turned out to be the key factor that determines the correlation between focal colors, unique hues, and sensory singularities. These findings suggest that the origins of color appearance and color language may be found in particular characteristics of the reflectance spectra that correspond to focal colors and unique hues.

Comparison of active and purely visual performance in a multiple-string means-end task in infants.

The aim of the present study was to understand what factors influence infants' problem-solving behaviours on the multiple-string task. The main question focused on why infants usually solve the single string-pulling task at 12months at the latest, whereas most 16-month-old infants still cannot solve the task when several strings are presented, only one of which is attached to the desired object. We investigated whether this difficulty is related to infants' ability to inhibit their spontaneous immediate actions by comparing active and purely visual performance in this task. During the first part of the experiment, we assessed the ability of infants aged 16-20months to solve the multiple-string task. The infants were then divided into three groups based on performance (a "failure" group, an "intermediate" group, and a "success" group). The results of this action task suggest that there were differences in infants' performance according to their level of inhibitory control of their preferred hand. In the second part of the experiment, the three groups' predictive looking strategies were compared when seeing an adult performing the task. We found that only infants who successfully performed the action task also visually anticipated which string the adult had to pull in the visual task. Our results suggests that inhibitory control was not the only factor influencing infants' performance on the task. Furthermore, the data support the direct matching hypothesis (Rizzolatti and Fadiga, 2005), according to which infants need to be able to perform actions themselves before being able to anticipate similar actions performed by others.

The emergence of use of a rake-like tool: a longitudinal study in human infants.

We describe the results of a longitudinal study on five infants from age 12 to 20 months, presented with an out of reach toy and a rake-like tool within reach. Five conditions of spatial relationship between toy and rake were tested. Outcomes and types of behavior were analyzed. There were successes observed around 12 months in the condition of spatial contiguity between rake and toy, but these could not be interpreted as corresponding to full understanding of the use of the rake. At this age and for the following months, in the conditions involving spatial separation between rake and toy, infants' strategies fluctuated between paying attention to the toy only, exploring the rake for its own sake, and connecting rake and toy but with no apparent attempt to bring the toy closer. Only between 16 and 20 months did infants fairly suddenly start to intentionally try to bring the toy closer with the tool: at this stage the infants also became able to learn from their failures and to correct their actions, as well as to benefit from demonstration from an adult. We examine the individual differences in the pattern of change in behaviors leading to tool use in the five infants, and find no increase in any one type of behavior that systematically precedes success. We conclude that sudden success at 18 months probably corresponds to the coming together of a variety of capacities.