Isolating Neural Signatures of Conscious Speech Perception with a No-Report Sine-Wave Speech Paradigm.

Identifying neural correlates of conscious perception is a fundamental endeavor of cognitive neuroscience. Most studies so far have focused on visual awareness along with trial-by-trial reports of task-relevant stimuli, which can confound neural measures of perceptual awareness with postperceptual processing. Here, we used a three-phase sine-wave speech paradigm that dissociated between conscious speech perception and task relevance while recording EEG in humans of both sexes. Compared with tokens perceived as noise, physically identical sine-wave speech tokens that were perceived as speech elicited a left-lateralized, near-vertex negativity, which we interpret as a phonological version of a perceptual awareness negativity. This response appeared between 200 and 300 ms after token onset and was not present for frequency-flipped control tokens that were never perceived as speech. In contrast, the P3b elicited by task-irrelevant tokens did not significantly differ when the tokens were perceived as speech versus noise and was only enhanced for tokens that were both perceived as speech and relevant to the task. Our results extend the findings from previous studies on visual awareness and speech perception and suggest that correlates of conscious perception, across types of conscious content, are most likely to be found in midlatency negative-going brain responses in content-specific sensory areas.

Hearing Shapes: Event-related Potentials Reveal the Time Course of Auditory-Visual Sensory Substitution.

In auditory-visual sensory substitution, visual information (e.g., shape) can be extracted through strictly auditory input (e.g., soundscapes). Previous studies have shown that image-to-sound conversions that follow simple rules [such as the Meijer algorithm; Meijer, P. B. L. An experimental system for auditory image representation. Transactions on Biomedical Engineering, 39, 111-121, 1992] are highly intuitive and rapidly learned by both blind and sighted individuals. A number of recent fMRI studies have begun to explore the neuroplastic changes that result from sensory substitution training. However, the time course of cross-sensory information transfer in sensory substitution is largely unexplored and may offer insights into the underlying neural mechanisms. In this study, we recorded ERPs to soundscapes before and after sighted participants were trained with the Meijer algorithm. We compared these posttraining versus pretraining ERP differences with those of a control group who received the same set of 80 auditory/visual stimuli but with arbitrary pairings during training. Our behavioral results confirmed the rapid acquisition of cross-sensory mappings, and the group trained with the Meijer algorithm was able to generalize their learning to novel soundscapes at impressive levels of accuracy. The ERP results revealed an early cross-sensory learning effect (150-210 msec) that was significantly enhanced in the algorithm-trained group compared with the control group as well as a later difference (420-480 msec) that was unique to the algorithm-trained group. These ERP modulations are consistent with previous fMRI results and provide additional insight into the time course of cross-sensory information transfer in sensory substitution.

Perception of ensemble statistics requires attention.

To overcome inherent limitations in perceptual bandwidth, many aspects of the visual world are represented as summary statistics (e.g., average size, orientation, or density of objects). Here, we investigated the relationship between summary (ensemble) statistics and visual attention. Recently, it was claimed that one ensemble statistic in particular, color diversity, can be perceived without focal attention. However, a broader debate exists over the attentional requirements of conscious perception, and it is possible that some form of attention is necessary for ensemble perception. To test this idea, we employed a modified inattentional blindness paradigm and found that multiple types of summary statistics (color and size) often go unnoticed without attention. In addition, we found attentional costs in dual-task situations, further implicating a role for attention in statistical perception. Overall, we conclude that while visual ensembles may be processed efficiently, some amount of attention is necessary for conscious perception of ensemble statistics.

ERP signatures of conscious and unconscious word and letter perception in an inattentional blindness paradigm.

A three-phase inattentional blindness paradigm was combined with ERPs. While participants performed a distracter task, line segments in the background formed words or consonant-strings. Nearly half of the participants failed to notice these word-forms and were deemed inattentionally blind. All participants noticed the word-forms in phase 2 of the experiment while they performed the same distracter task. In the final phase, participants performed a task on the word-forms. In all phases, including during inattentional blindness, word-forms elicited distinct ERPs during early latencies (∼200-280ms) suggesting unconscious orthographic processing. A subsequent ERP (∼320-380ms) similar to the visual awareness negativity appeared only when subjects were aware of the word-forms, regardless of the task. Finally, word-forms elicited a P3b (∼400-550ms) only when these stimuli were task-relevant. These results are consistent with previous inattentional blindness studies and help distinguish brain activity associated with pre- and post-perceptual processing from correlates of conscious perception.

Neural Signatures of Conscious Face Perception in an Inattentional Blindness Paradigm.

Previous studies suggest that early stages of face-specific processing are performed preattentively and unconsciously, whereas conscious perception emerges with late-stage (>300 ms) neuronal activity. A conflicting view, however, posits that attention is necessary for face-specific processing and that early-to-mid latency neural responses (∼ 100-300 ms) correspond more closely with perceptual awareness. The current study capitalized on a recently developed method for manipulating attention and conscious perception during EEG recording (modified inattentional blindness paradigm) and used face stimuli that elicit a well known marker of early face processing, the N170 event-related potential (ERP). In Phase 1 of the experiment, subjects performed a demanding distracter task while line drawings of faces and matched control stimuli were presented in the center of their view. When queried, half of the subjects reported no awareness of the faces and were deemed inattentionally blind. In Phase 2, subjects performed the same distracter task, but now consciously perceived the face stimuli due to the intervening questioning. In Phase 3, subjects performed a discrimination task on the faces. Two primary contrasts were made: aware versus unaware (equally task irrelevant) and task-relevant versus task-irrelevant (equally aware). The N170 and a subsequent ERP component, the visual awareness negativity (∼ 260-300 ms), were absent during inattentional blindness and present in the aware conditions. The P3b (> 300 ms) was absent for task-irrelevant faces, even when consciously perceived, and present only when the faces were task relevant. These results inform contemporary theories of conscious face perception in particular and visual attention and perceptual awareness in general.

Gamma band activity and the P3 reflect post-perceptual processes, not visual awareness.

A primary goal in cognitive neuroscience is to identify neural correlates of conscious perception (NCC). By contrasting conditions in which subjects are aware versus unaware of identical visual stimuli, a number of candidate NCCs have emerged; among them are induced gamma band activity in the EEG and the P3 event-related potential. In most previous studies, however, the critical stimuli were always directly relevant to the subjects' task, such that aware versus unaware contrasts may well have included differences in post-perceptual processing in addition to differences in conscious perception per se. Here, in a series of EEG experiments, visual awareness and task relevance were manipulated independently. Induced gamma activity and the P3 were absent for task-irrelevant stimuli regardless of whether subjects were aware of such stimuli. For task-relevant stimuli, gamma and the P3 were robust and dissociable, indicating that each reflects distinct post-perceptual processes necessary for carrying-out the task but not for consciously perceiving the stimuli. Overall, this pattern of results challenges a number of previous proposals linking gamma band activity and the P3 to conscious perception.

Visual processing of contour patterns under conditions of inattentional blindness.

An inattentional blindness paradigm was adapted to measure ERPs elicited by visual contour patterns that were or were not consciously perceived. In the first phase of the experiment, subjects performed an attentionally demanding task while task-irrelevant line segments formed square-shaped patterns or random configurations. After the square patterns had been presented 240 times, subjects' awareness of these patterns was assessed. More than half of all subjects, when queried, failed to notice the square patterns and were thus considered inattentionally blind during this first phase. In the second phase of the experiment, the task and stimuli were the same, but following this phase, all of the subjects reported having seen the patterns. ERPs recorded over the occipital pole differed in amplitude from 220 to 260 msec for the pattern stimuli compared with the random arrays regardless of whether subjects were aware of the patterns. At subsequent latencies (300-340 msec) however, ERPs over bilateral occipital-parietal areas differed between patterns and random arrays only when subjects were aware of the patterns. Finally, in a third phase of the experiment, subjects viewed the same stimuli, but the task was altered so that the patterns became task relevant. Here, the same two difference components were evident but were followed by a series of additional components that were absent in the first two phases of the experiment. We hypothesize that the ERP difference at 220-260 msec reflects neural activity associated with automatic contour integration whereas the difference at 300-340 msec reflects visual awareness, both of which are dissociable from task-related postperceptual processing.

Early stages of figure-ground segregation during perception of the face-vase.

The temporal sequence of neural processes supporting figure-ground perception was investigated by recording ERPs associated with subjects' perceptions of the face-vase figure. In Experiment 1, subjects continuously reported whether they perceived the face or the vase as the foreground figure by pressing one of two buttons. Each button press triggered a probe flash to the face region, the vase region, or the borders between the two. The N170/vertex positive potential (VPP) component of the ERP elicited by probes to the face region was larger when subjects perceived the faces as figure. Preceding the N170/VPP, two additional components were identified. First, when the borders were probed, ERPs differed in amplitude as early as 110 msec after probe onset depending on subjects' figure-ground perceptions. Second, when the face or vase regions were probed, ERPs were more positive (at ∼ 150-200 msec) when that region was perceived as figure versus background. These components likely reflect an early "border ownership" stage, and a subsequent "figure-ground segregation" stage of processing. To explore the influence of attention on these stages of processing, two additional experiments were conducted. In Experiment 2, subjects selectively attended to the face or vase region, and the same early ERP components were again produced. In Experiment 3, subjects performed an identical selective attention task, but on a display lacking distinctive figure-ground borders, and neither of the early components were produced. Results from these experiments suggest sequential stages of processing underlying figure-ground perception, each which are subject to modifications by selective attention.

Right parietal brain activity precedes perceptual alternation during binocular rivalry.

We investigated perceptual reversals for intermittently presented stimuli during binocular rivalry and physical alternation while the ongoing EEG was recorded from 64 channels. EEG topographies immediately preceding stimulus-onset were analyzed and two topographies doubly dissociated perceptual reversals from non-reversals. The estimated intracranial generators associated with these topographies were stronger in right inferior parietal cortex and weaker bilaterally in the ventral stream before perceptual reversals. No such differences were found for physical alternation of the same stimuli. These results replicate and extend findings from a previous study with the Necker cube and suggest common neural mechanisms associated with perceptual reversals during binocular rivalry and ambiguous figure perception. For both types of multi-stable stimuli, the dorsal stream is more active preceding perceptual reversals. Activity in the ventral stream, however, differed for binocular rivalry compared to ambiguous figures. The results from the two studies suggest a causal role for the right inferior parietal cortex in generating perceptual reversals regardless of the type of multi-stable stimulus, while activity in the ventral stream appears to depend on the particular type of stimulus.

When and where is binocular rivalry resolved in the visual cortex?

Anatomically "early" visual areas including striate cortex (V1) are known to play a role in the resolution of binocular rivalry. However, the time course of such activity and its relationship with subjective perception are unclear. The present study used an intermittent stimulation design to measure event-related potentials (ERPs) associated with perception during binocular rivalry and during physical alternations of the same stimuli. When gratings of high or low spatial frequencies were presented in physical alternation, the amplitude of the earliest cortical ERP component (the C1, at 60-100 ms) was larger for high spatial frequencies. When the same two stimuli competed during rivalry, however, C1 amplitudes were equivalent for perceptions of high versus low spatial frequency. These findings suggest that rivalry is not resolved before or during the initial cortical response at 60-100 ms. At longer latencies (approximately 130-160 ms), however, occipital ERPs with similar topographies and estimated sources as the C1 differed according to perception in both the rivalry and physical alternation conditions. These results suggest that during rivalry, neural activity coupled with the dominant percept first emerges in anatomically early visual cortex, but at delayed latencies.

Early top-down influences on bistable perception revealed by event-related potentials.

A longstanding debate exists in the literature concerning bottom-up vs. top-down influences on bistable perception. Recently, a technique has been developed to measure early changes in brain activity (via ERPs) related to perceptual reversals (Kornmeier & Bach, 2004). An ERP component, the reversal negativity (RN) has been identified, and is characterized as an increase in negative potential over the posterior scalp from 150 to 350 ms for perceptual reversals compared to perceptual stability. This finding, although interesting, has not helped resolve issues related to the bottom-up vs. top-down debate because top-down influences have not been directly manipulated. The current study focused on resolving some of these issues by measuring the RN while observers maintained one of three 'intentional approaches', (1) try to reverse perception as often as possible, (2) try to stabilize perception for as long as possible, and (3) maintain a passive approach. Enhancements in RN amplitude were found for the intention-to-reverse condition compared to the passive condition. This finding suggests an early influence (150 ms) of top-down control on perceptual reversals of bistable figures. Results are discussed in terms of competing attention shifting vs. fatigue-based theories of bistable perception.

Still wanted: a reproducible demonstration of a genuine C1 attention effect.

Slotnick (this issue) has specified a number of experimental parameters that appear critical for enabling an attention-related modulation of the C1 component. These include stimulus presentation in the upper visual field, the presence of distractors, a high perceptual or attentional load, and measurements at midline occipito-parietal sites. While we agree with many of these recommendations, we would modify others and even dispute a few. Despite the employment of these parameters in a few existing studies, there has not yet been a convincing, reproducible demonstration of a modulation of the C1 component by spatial attention that can be localized to primary visual cortex.

The relationship between attention and consciousness: an expanded taxonomy and implications for 'no-report' paradigms.

Tensions between global neuronal workspace theory and recurrent processing theory have sparked much debate in the field of consciousness research. Here, we focus on one of the key distinctions between these theories: the proposed relationship between attention and consciousness. By reviewing recent empirical evidence, we argue that both theories contain key insights and that certain aspects of each theory can be reconciled into a novel framework that may help guide future research. Alternative theories are also considered, including attended intermediate-level representations theory, integrated information theory and higher order thought theory. With the aim of offering a fresh and nuanced perspective to current theoretical debates, an updated taxonomy of conscious and non-conscious states is proposed. This framework maps a wider spectrum of conscious states by incorporating contemporary views from cognitive neuroscience regarding the variety of attentional mechanisms that are known to interact with sensory processing. Whether certain types of attention are necessary for phenomenal and access consciousness is considered and incorporated into this extended taxonomy. To navigate this expanded space, we review recent 'no-report' paradigms and address several methodological misunderstandings in order to pave a clear path forward for identifying the neural basis of perceptual awareness.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Does spatial attention modulate the C1 component? The jury continues to deliberate.

The thoughful comments on our study (Baumgartner et al., this issue) that failed to replicate the C1 attention effect reported by a previous study roughly fall into three broad categories. First, the commentators identified specific differences between the two studies that may have contributed to the discrepant results. Second, they highlighted some of the theoretical and methodological problems that are encountered when trying to demonstrate attention effects on the initial evoked response in primary visual cortex. Third, they offered a number of proposals for optimizing experimental designs and analysis methods that may increase the likelihood of observing attention-related modulations of the C1. We consider each of these topics in turn.

Does spatial attention modulate the earliest component of the visual evoked potential?

Whether visual spatial attention can modulate feedforward input to human primary visual cortex (V1) is debated. A prominent and long-standing hypothesis is that visual spatial attention can influence processing in V1, but only at delayed latencies suggesting a feedback-mediated mechanism and a lack of modulation during the initial afferent volley. The most promising challenge to this hypothesis comes from an event-related potential (ERP) study that showed an amplitude enhancement of the earliest visual ERP component, called the 'C1', in response to spatially attended relative to spatially unattended stimuli. In the Kelly et al. study, several important experimental design modifications were introduced, including tailoring the stimulus locations and recording electrodes to each individual subject. In the current study, we employed the same methodological procedures and tested for attentional enhancements of the C1 component in each quadrant of the visual field. Using the same analysis strategies as Kelly et al., we found no evidence for an attention-based modulation of the C1 (measured from 50-80 ms). Attention-based amplitude enhancements were clear and robust for the subsequent P1 component (90-140 ms). Thus, despite using methods specifically designed to reveal C1 attention effects, the current study provided no confirmatory evidence for such effects.

Electrophysiological correlates of perceptual reversals for three different types of multistable images.

Electrophysiological recordings were made in 21 observers to investigate whether differences in signature components (P1, N1, selection negativity [SN]) would be revealed during perceptual reversals of three different multistable figures. Using a lattice of Necker cubes as a stimulus, J. Kornmeier and M. Bach (2004, 2005) reported differences in P1 amplitudes as well a broad reversal-related negativity occurring 200-400 ms poststimulus. The current study investigated whether these event-related potentials of Necker cube reversals represent general "perceptual switching" mechanisms and would, therefore, be common to other types of multistable figures. Three different types of multistable stimuli were utilized: a modified Rubin's face/vase, a modified Schröder's staircase, and a novel natural stimulus, Lemmo's cheetahs. Results revealed the broad reversal-related negativity for the face/vase and the reversible staircase but not for the cheetahs. This component is comparable to the SN in polarity, latency, and scalp topography. An effect of early visual spatial attention on figure reversals was suggested by an analysis of the occipital P1 and N1 components. The P1, N1, or both were enhanced for trials in which the observer reported perceptual reversals compared with trials in which no reversals were reported for the face/vase and reversible staircase stimuli. These results support a model of multistable perception in which changes in early spatial attention (indicated by P1 and N1 enhancement) modulate perceptual reversals (indicated by the reversal negativity or SN).

Chromatic perceptive field sizes change with retinal illuminance.

The effect of retinal illuminance (0.3-3.3 log td) on chromatic perceptive field size was investigated at 10 degrees eccentricity along the horizontal meridian of the temporal retina. Using the 4+1 color-naming procedure, observers described the hue and saturation of a series of monochromatic stimuli (440-660 nm, in 10-nm steps) of various test sizes (.098-5 degrees) after 30-min dark adaptation. Perceptive field sizes of the four elemental hues and the saturation component were estimated for each wavelength at each retinal illuminance. Results indicate that perceptive field sizes for blue, green, yellow, and saturation all decrease with increasing retinal illuminance; the perceptive field size for red is the smallest and invariant with intensity. The influence of rods on perceptive field size may account for some of the results; other factors are also considered.

Testing domain-general theories of perceptual awareness with auditory brain responses.

Past research has identified several candidate neural correlates of consciousness (NCCs) during visual perception. Recent research on auditory perception shows promise for establishing the generality of various NCCs across sensory modalities, as well as for revealing differences in how conscious processing unfolds in different sensory systems.

Electrophysiological assessment of the time course of bilingual visual word recognition: Early access to language membership.

Previous research examining the time course of lexical access during word recognition suggests that phonological processing precedes access to semantic information, which in turn precedes access to syntactic information. Bilingual word recognition likely requires an additional level: knowledge of which language a specific word belongs to. Using the recording of event-related potentials, we investigated the time course of access to language membership information relative to semantic (Experiment 1) and syntactic (Experiment 2) encoding during visual word recognition. In Experiment 1, Spanish-English bilinguals viewed a series of printed words while making dual-choice go/nogo and left/right hand decisions based on semantic (whether the word referred to an animal or an object) and language membership information (whether the word was in English or in Spanish). Experiment 2 used a similar paradigm but with syntactic information (whether the word was a noun or a verb) as one of the response contingencies. The onset and peak latency of the N200, a component related to response inhibition, indicated that language information is accessed earlier than semantic information. Similarly, language information was also accessed earlier than syntactic information (but only based on peak latency). We discuss these findings with respect to models of bilingual word recognition and language comprehension in general.

Auditory event-related potentials associated with perceptual reversals of bistable pitch motion.

Previous event-related potential (ERP) experiments have consistently identified two components associated with perceptual transitions of bistable visual stimuli, the "reversal negativity" (RN) and the "late positive complex" (LPC). The RN (~200 ms post-stimulus, bilateral occipital-parietal distribution) is thought to reflect transitions between neural representations that form the moment-to-moment contents of conscious perception, while the LPC (~400 ms, central-parietal) is considered an index of post-perceptual processing related to accessing and reporting one's percept. To explore the generality of these components across sensory modalities, the present experiment utilized a novel bistable auditory stimulus. Pairs of complex tones with ambiguous pitch relationships were presented sequentially while subjects reported whether they perceived the tone pairs as ascending or descending in pitch. ERPs elicited by the tones were compared according to whether perceived pitch motion changed direction or remained the same across successive trials. An auditory reversal negativity (aRN) component was evident at ~170 ms post-stimulus over bilateral fronto-central scalp locations. An auditory LPC component (aLPC) was evident at subsequent latencies (~350 ms, fronto-central distribution). These two components may be auditory analogs of the visual RN and LPC, suggesting functionally equivalent but anatomically distinct processes in auditory vs. visual bistable perception.