Visual form predictions facilitate auditory processing at the N1.

Auditory-visual (AV) events often involve a leading visual cue (e.g. auditory-visual speech) that allows the perceiver to generate predictions about the upcoming auditory event. Electrophysiological evidence suggests that when an auditory event is predicted, processing is sped up, i.e., the N1 component of the ERP occurs earlier (N1 facilitation). However, it is not clear (1) whether N1 facilitation is based specifically on predictive rather than multisensory integration and (2) which particular properties of the visual cue it is based on. The current experiment used artificial AV stimuli in which visual cues predicted but did not co-occur with auditory cues. Visual form cues (high and low salience) and the auditory-visual pairing were manipulated so that auditory predictions could be based on form and timing or on timing only. The results showed that N1 facilitation occurred only for combined form and temporal predictions. These results suggest that faster auditory processing (as indicated by N1 facilitation) is based on predictive processing generated by a visual cue that clearly predicts both what and when the auditory stimulus will occur.

[Delayed Copying of Unfamiliar Contour Shapes: Analysis of Potentials Related to Stimuli].

The high-density EEG was recorded and ERPs related to showing unfamiliar contour shapes and delivering imperative signal (a short sound) were estimated in the task requiring delayed motor reproduction of these shapes. A total of 22 right-handed adults participated in the experiment. They performed 5-blocks of trials corresponding-to 5 different delays (T= 0; 500, 1000, 2000 4000 ms) between the imperative signal relative to the end of the contour trajectory presentation. An ERP analysis showed that, unlike ERPs related to the contour shape presentation, those related to the imperative signal delivery do change with growing delay T. A subsequent analysis of cortical sources of the ERPs related to the imperative stimulus showed corresponding pronounced grows of reactivity of orbito-frontal cortex of the right hemisphere and a symmetrical bilateral grows of reactivity of dorsal parts of the sensorimotor cortex. The reported findings are discussed in the framework of the proposal according to which the internal representation of a trajectory undergoes a transi- tion from a sensory-specific format towards more abstract neither sensory- nor motor-specific format.

A neurocomputational account of the face configural effect.

A striking phenomenon in face perception is the configural effect in which a difference in a single part appears more distinct in the context of a face than it does by itself. The face context would be expected to increase search complexity, rendering discrimination more--not less--difficult. Remarkably, there has never been a biologically plausible explanation of this fundamental signature of face recognition.We show that the configural effect can be simply derived from a model composed of overlapping receptive fields (RFs) characteristic of early cortical simple-cell tuning but also present in face-selective areas. Because of the overlap in RFs, the difference in a single part is not only represented in the RFs centered on it but also propagated to larger RFs centered on distant parts of the face. Dissimilarity values computed from the model between pairs of faces and pairs of face parts closely matched the recognition accuracy of human observers who had learned a set of faces composed of composite parts and were tested on wholes (Which is Larry?) and parts (Which is Larry's nose?). When stimuli were high versus low passed the contributions of different spatial frequency (SF) bands to the configural effect were largely comparable. Therefore, it was the larger RFs rather than the low SFs that accounted for most of the configural effect. The representation explains why, relative to objects, face recognition is so adversely affected by inversion and contrast reversal and why distinctions between similar faces are ineffable.

Dissociation between small and large numerosities in newborn infants.

In the first year of life, infants possess two cognitive systems encoding numerical information: one for processing the numerosity of sets of 4 or more items, and the second for tracking up to 3 objects in parallel. While a previous study showed the former system to be already present a few hours after birth, it is unknown whether the latter system is functional at this age. Here, we adapt the auditory-visual matching paradigm that previously revealed sensitivity to large numerosities to test sensitivity to numerosities spanning the range from 2 to 12. Across studies, newborns discriminated pairs of large numerosities in a 3:1 ratio, even when the smaller numerosity was 3 (3 vs. 9). In contrast, newborn infants failed to discriminate pairs including the numerosity 2, even at the same ratio (2 vs. 6). These findings mirror the dissociation that has been reported with older infants, albeit with a discontinuity situated between numerosities 2 and 3. Two alternative explanations are compatible with our results: either newborn infants have a separate system for processing small sets, and the capacity of this system is limited to 2 objects; or newborn infants possess only one system to represent numerosities, and this system either is not functional or is extremely imprecise when it is applied to small numerosities.

The contribution of amplitude and phase spectra-defined scene statistics to the masking of rapid scene categorization.

Viewers can recognize the gist of a scene (i.e., its holistic semantic representation, such as its category) in less time than a single fixation, and backward masking has traditionally been employed as a means to determine that time course. The masks used in those paradigms are often characterized by either specific amplitude spectra only, or amplitude and phase spectra-defined structural properties. However, it remains unclear whether there would be a differential contribution of amplitude only or amplitude + phase defined image statistics to the effective backward masking of rapid scene categorization. The current study addresses this issue. Experiments 1-3 explored amplitude spectra defined contributions to category masking and revealed that the slope of the amplitude spectrum was more important for modulating scene category masking strength than amplitude orientation. Further, the masking effects followed an "amplitude spectrum slope similarity principle" whereby the more similar the amplitude spectrum slope of the mask was to the target's amplitude spectrum slope, the stronger the masking. Experiment 5 showed that, when holding mask amplitude spectrum slope approximately constant, both categorically specific unrecognizable amplitude only and amplitude + phase statistical regularities disrupted rapid scene categorization. Specifically, the masking effects observed in Experiment 5 followed a target-mask categorical dissimilarity principle whereby the more dissimilar the mask category is to the target image category, the stronger the masking. Overall, the results support the notion that amplitude only or amplitude + phase-defined image statistics differentially contribute to the effective backward masking of rapid scene gist recognition.

Object shape and orientation do not routinely influence performance during language processing.

The role of visual representations during language processing remains unclear: They could be activated as a necessary part of the comprehension process, or they could be less crucial and influence performance in a task-dependent manner. In the present experiments, participants read sentences about an object. The sentences implied that the object had a specific shape or orientation. They then either named a picture of that object (Experiments 1 and 3) or decided whether the object had been mentioned in the sentence (Experiment 2). Orientation information did not reliably influence performance in any of the experiments. Shape representations influenced performance most strongly when participants were asked to compare a sentence with a picture or when they were explicitly asked to use mental imagery while reading the sentences. Thus, in contrast to previous claims, implied visual information often does not contribute substantially to the comprehension process during normal reading.

A new illusion of height and width: taller people are perceived as thinner.

It is commonly said that tall people look thinner. Here, we asked whether an illusion exists such that the taller of two equally wide stimuli looks thinner, and conversely whether the thinner of two equally tall stimuli looks taller. In five experiments, participants judged the horizontal or vertical extents of two identical bodies, rectangles, or cylinders that differed only in their vertical or horizontal extents. Our results confirmed the folk wisdom that being tall makes you look thinner. We similarly found that being thin makes you look taller, although this effect was less pronounced. The same illusion was present for filled rectangles and cylinders, but it was consistently stronger for both photographs and silhouettes of the human body, raising the question of why the human form should be more prone to this illusion.

Complex visual hallucinations and attentional performance in eye disease and dementia: a test of the Perception and Attention Deficit model.

OBJECTIVE: This study aimed to test the prediction from the Perception and Attention Deficit model of complex visual hallucinations (CVH) that impairments in visual attention and perception are key risk factors for complex hallucinations in eye disease and dementia. METHODS: Two studies ran concurrently to investigate the relationship between CVH and impairments in perception (picture naming using the Graded Naming Test) and attention (Stroop task plus a novel Imagery task). The studies were in two populations-older patients with dementia (n = 28) and older people with eye disease (n = 50) with a shared control group (n = 37). The same methodology was used in both studies, and the North East Visual Hallucinations Inventory was used to identify CVH. RESULTS: A reliable relationship was found for older patients with dementia between impaired perceptual and attentional performance and CVH. A reliable relationship was not found in the population of people with eye disease. CONCLUSIONS: The results add to previous research that object perception and attentional deficits are associated with CVH in dementia, but that risk factors for CVH in eye disease are inconsistent, suggesting that dynamic rather than static impairments in attentional processes may be key in this population.

Kinematics fingerprints of leader and follower role-taking during cooperative joint actions.

Performing online complementary motor adjustments is quintessential to joint actions since it allows interacting people to coordinate efficiently and achieve a common goal. We sought to determine whether, during dyadic interactions, signaling strategies and simulative processes are differentially implemented on the basis of the interactional role played by each partner. To this aim, we recorded the kinematics of the right hand of pairs of individuals who were asked to grasp as synchronously as possible a bottle-shaped object according to an imitative or complementary action schedule. Task requirements implied an asymmetric role assignment so that participants performed the task acting either as (1) Leader (i.e., receiving auditory information regarding the goal of the task with indications about where to grasp the object) or (2) Follower (i.e., receiving instructions to coordinate their movements with their partner's by performing imitative or complementary actions). Results showed that, when acting as Leader, participants used signaling strategies to enhance the predictability of their movements. In particular, they selectively emphasized kinematic parameters and reduced movement variability to provide the partner with implicit cues regarding the action to be jointly performed. Thus, Leaders make their movements more "communicative" even when not explicitly instructed to do so. Moreover, only when acting in the role of Follower did participants tend to imitate the Leader, even in complementary actions where imitation is detrimental to joint performance. Our results show that mimicking and signaling are implemented in joint actions according to the interactional role of the agent, which in turn is reflected in the kinematics of each partner.

The Vanderbilt Expertise Test reveals domain-general and domain-specific sex effects in object recognition.

Individual differences in face recognition are often contrasted with differences in object recognition using a single object category. Likewise, individual differences in perceptual expertise for a given object domain have typically been measured relative to only a single category baseline. In Experiment 1, we present a new test of object recognition, the Vanderbilt Expertise Test (VET), which is comparable in methods to the Cambridge Face Memory Task (CFMT) but uses eight different object categories. Principal component analysis reveals that the underlying structure of the VET can be largely explained by two independent factors, which demonstrate good reliability and capture interesting sex differences inherent in the VET structure. In Experiment 2, we show how the VET can be used to separate domain-specific from domain-general contributions to a standard measure of perceptual expertise. While domain-specific contributions are found for car matching for both men and women and for plane matching in men, women in this sample appear to use more domain-general strategies to match planes. In Experiment 3, we use the VET to demonstrate that holistic processing of faces predicts face recognition independently of general object recognition ability, which has a sex-specific contribution to face recognition. Overall, the results suggest that the VET is a reliable and valid measure of object recognition abilities and can measure both domain-general skills and domain-specific expertise, which were both found to depend on the sex of observers.

The experience of force: the role of haptic experience of forces in visual perception of object motion and interactions, mental simulation, and motion-related judgments.

Forces are experienced in actions on objects. The mechanoreceptor system is stimulated by proximal forces in interactions with objects, and experiences of force occur in a context of information yielded by other sensory modalities, principally vision. These experiences are registered and stored as episodic traces in the brain. These stored representations are involved in generating visual impressions of forces and causality in object motion and interactions. Kinematic information provided by vision is matched to kinematic features of stored representations, and the information about forces and causality in those representations then forms part of the perceptual interpretation. I apply this account to the perception of interactions between objects and to motions of objects that do not have perceived external causes, in which motion tends to be perceptually interpreted as biological or internally caused. I also apply it to internal simulations of events involving mental imagery, such as mental rotation, trajectory extrapolation and judgment, visual memory for the location of moving objects, and the learning of perceptual judgments and motor skills. Simulations support more accurate judgments when they represent the underlying dynamics of the event simulated. Mechanoreception gives us whatever limited ability we have to perceive interactions and object motions in terms of forces and resistances; it supports our practical interventions on objects by enabling us to generate simulations that are guided by inferences about forces and resistances, and it helps us learn novel, visually based judgments about object behavior.

Cerebral correlates and statistical criteria of cross-modal face and voice integration.

Perception of faces and voices plays a prominent role in human social interaction, making multisensory integration of cross-modal speech a topic of great interest in cognitive neuroscience. How to define potential sites of multisensory integration using functional magnetic resonance imaging (fMRI) is currently under debate, with three statistical criteria frequently used (e.g., super-additive, max and mean criteria). In the present fMRI study, 20 participants were scanned in a block design under three stimulus conditions: dynamic unimodal face, unimodal voice and bimodal face-voice. Using this single dataset, we examine all these statistical criteria in an attempt to define loci of face-voice integration. While the super-additive and mean criteria essentially revealed regions in which one of the unimodal responses was a deactivation, the max criterion appeared stringent and only highlighted the left hippocampus as a potential site of face- voice integration. Psychophysiological interaction analysis showed that connectivity between occipital and temporal cortices increased during bimodal compared to unimodal conditions. We concluded that, when investigating multisensory integration with fMRI, all these criteria should be used in conjunction with manipulation of stimulus signal-to-noise ratio and/or cross-modal congruency.

Brain region white matter associations with visual selective attention.

To understand how normal variations in white matter relate to cognition, magnetization transfer imaging ratios (MTR) of a hypothesized neural network were associated with a test of visual selective attention (VST). Healthy adults (N = 16) without abnormal signal on brain scans viewed a version of DeSchepper and Treisman's test of VST (1996) with two levels of processing (novel shape matching with and without distractors, contingency feedback). A hypothesized neural network and component regions was significantly associated with accuracy and response times when distractors were present, with betas predicting 55% of variance in accuracy, and 59% of response times. MTR for anterior and posterior cingulate, prefrontal region, and thalami comprised a model predicting 55% of accuracy when distractors were present, and the anterior cingulate accounted for the majority of this effect. Prefrontal MTR predicted longer response times which was associated with increased accuracy. Distal neural areas involved in complex, processing-driven tasks (error processing, response selection, and variable response competition and processing load) may be dependent on white matter fibers to connect distal brain regions/nuclei of a macronetwork, including prefrontal executive functions.

Multisensory object representation: insights from studies of vision and touch.

Behavioral studies show that the unisensory representations underlying within-modal visual and haptic object recognition are strikingly similar in terms of view- and size-sensitivity, and integration of structural and surface properties. However, the basis for these attributes differs in each modality, indicating that while these representations are functionally similar, they are not identical. Imaging studies reveal bisensory, visuo-haptic object selectivity, notably in the lateral occipital complex and the intraparietal sulcus, that suggests a shared representation of objects. Such a multisensory representation could underlie visuo-haptic cross-modal object recognition. In this chapter, we compare visual and haptic within-modal object recognition and trace a progression from functionally similar but separate unisensory representations to a shared multisensory representation underlying cross-modal object recognition as well as view-independence, regardless of modality. We outline, and provide evidence for, a model of multisensory object recognition in which representations are flexibly accessible via top-down or bottom-up processing, the choice of route being influenced by object familiarity and individual preference along the object-spatial continuum of mental imagery.

Shape from sound: evidence for a shape operator in the lateral occipital cortex.

A recent view of cortical functional specialization suggests that the primary organizing principle of the cortex is based on task requirements, rather than sensory modality. Consistent with this view, recent evidence suggests that a region of the lateral occipitotemporal cortex (LO) may process object shape information regardless of the modality of sensory input. There is considerable evidence that area LO is involved in processing visual and haptic shape information. However, sound can also carry acoustic cues to an object's shape, for example, when a sound is produced by an object's impact with a surface. Thus, the current study used auditory stimuli that were created from recordings of objects impacting a hard surface to test the hypothesis that area LO is also involved in auditory shape processing. The objects were of two shapes, rods and balls, and of two materials, metal and wood. Subjects were required to categorize the impact sounds in one of three tasks, (1) by the shape of the object while ignoring material, (2) by the material of the object while ignoring shape, or (3) by using all the information available. Area LO was more strongly recruited when subjects discriminated impact sounds based on the shape of the object that made them, compared to when subjects discriminated those same sounds based on material. The current findings suggest that activation in area LO is shape selective regardless of sensory input modality, and are consistent with an emerging theory of perceptual functional specialization of the brain that is task-based rather than sensory modality-based.

Electrophysiological chronometry of semantic context effects in language production.

In this study, we investigated semantic context effects in language production with event-related brain potentials, extracted from the ongoing EEG recorded during overt speech production. We combined the picture-word interference paradigm and the semantic blocking paradigm to investigate the temporal dynamics and functional loci of semantic facilitation and interference effects. Objects were named in the context of semantically homogeneous blocks consisting of related objects and heterogeneous blocks consisting of unrelated objects. In each blocking condition, semantically related and unrelated distractor words were presented. Results show that classic patterns of semantically induced facilitation and interference effects in RTs can be directly related to ERP modulations located at temporal and frontal sites, starting at about 200 msec. Results also suggest that the processes associated with semantic facilitation and interference effects (i.e., conceptual and lexical processing) are highly interactive and coincide in time. Implications for the use of event-related brain potentials in speech production research and implications for current models of speech production are discussed.

The accentuation principle of visual organization and the illusion of musical suspension.

The aim of this work is to demonstrate a new principle of grouping and shape formation that we called the accentuation principle, stating that, all else being equal, the elements tend to group in the same oriented direction of the element discontinuity placed within a whole set of continuous/homogeneous components. The discontinuous element is like an accent, i.e., a visual emphasis within a whole. We showed that this principle is independent from other gestalt principles. In fact, it shows vectorial properties not present in the other principles. It can be pitted against them. Furthermore, it is not only a grouping principle but it also influences shape formation, by inducing effects like the square/diamond and the rectangle illusions. Finally, the accentuation operates under stroboscopic conditions and manifests filling-in properties and long range effects. Through experimental phenomenology, it was shown that the accentuation principle can influence grouping and shape formation not only in space but also in time and, therefore, not only in vision but also in music perception. This was suggested by phenomenally linking visual and musical accents and by demonstrating a new illusion of musical suspension, related with its opposite effect, the downbeat illusion. This kind of illusions can be appreciated in two solo piano compositions respectively by Debussy and Chopin-Rêverie and Nocturne, op. 27 no. 1. Variations in the note where the accent is placed and in the kind of accent demonstrated their basic role in inducing the illusion of musical suspension.

Pitch and loudness information encoded in auditory imagery as revealed by event-related potentials.

Two experiments using the ERP method and a task that involved comparing an imagined-S1 (the first stimulus) with a perceived-S2 (the second stimulus) were conducted to investigate whether imagined auditory representations encode pitch and loudness information. It was found that the amplitude of the imagery-related late positive complex (LPC) decreased with pitch but increased with loudness of the imagined sound, which was consistent with amplitude modulations of the auditory perception-related N1 component, thereby providing the first neural evidence that auditory imagery encodes perceptual attributes of auditory experiences.

Staying within the lines: the formation of visuospatial boundaries influences multisensory feature integration.

The brain processes multisensory features of an object (e.g., its sound and shape) in separate cortical regions. A key question is how representations of these features bind together to form a coherent percept (the 'binding problem'). Here we tested the hypothesis that the determination of an object's visuospatial boundaries is paramount to the linking of its multisensory features (i.e., that the refinement of attended space through the formation of visual boundaries establishes the boundaries for multisensory feature integration). We recorded both scalp and intracranial electrophysiological data in response to Kanizsa-type illusory contour stimuli (in which pacman-like elements give the impression of a single object), their non-illusory counterparts, and auditory stimuli. Participants performed a visual task and ignored sounds. Enhanced processing of task-irrelevant sounds when paired with attended visual stimuli served as our metric for multisensory feature integration [e.g., Busse et al. (2005) Proc. Natl Acad. Sci. USA 102: 18751-18756]. According to our hypothesis, task-irrelevant sounds paired with Kanizsa-type illusory contour stimuli (which have well-defined boundaries) should receive enhanced processing relative to task-irrelevant sounds paired with non-illusory contour stimuli (which have ambiguous boundaries). The scalp data clearly support this prediction and, combined with the intracranial data, advocate for an important extension of models for multisensory feature integration. We propose a model in which (i) the visual boundaries of an object are established through processing in occipitotemporal cortex, and (ii) attention then spreads to cortical regions that process features that fall within the object's established visual boundaries, including its task-irrelevant multisensory features.

Hypnosis and hemispheric asymmetry.

Participants of low and high hypnotic susceptibility were tested on a temporal order judgement task, both with and without hypnosis. Judgements were made of the order of presentation of light flashes appearing in first one hemi-field then the other. There were differences in the inter-stimulus intervals required accurately to report the order, depending upon which hemi-field led. This asymmetry was most marked in hypnotically susceptible participants and reversed when they were hypnotized. This implies not only that brain activity changes in hypnosis, but also that there is a difference in brain function between people of low and high hypnotic susceptibility. The latter exhibited a faster-acting left hemisphere in the waking state, but faster right when hypnotized.