Neuroscientific Evidence for Processing Without Awareness.

The extent to which we are affected by perceptual input of which we are unaware is widely debated. By measuring neural responses to sensory stimulation, neuroscientific data could complement behavioral results with valuable evidence. Here we review neuroscientific findings of processing of high-level information, as well as interactions with attention and memory. Although the results are mixed, we find initial support for processing object categories and words, possibly to the semantic level, as well as emotional expressions. Robust neural evidence for face individuation and integration of sentences or scenes is lacking. Attention affects the processing of stimuli that are not consciously perceived, and such stimuli may exogenously but not endogenously capture attention when relevant, and be maintained in memory over time. Sources of inconsistency in the literature include variability in control for awareness as well as individual differences, calling for future studies that adopt stricter measures of awareness and probe multiple processes within subjects.

Context Sensitivity across Multiple Time scales with a Flexible Frequency Bandwidth.

Everyday auditory streams are complex, including spectro-temporal content that varies at multiple timescales. Using EEG, we investigated the sensitivity of human auditory cortex to the content of past stimulation in unattended sequences of equiprobable tones. In 3 experiments including 82 participants overall, we found that neural responses measured at different latencies after stimulus onset were sensitive to frequency intervals computed over distinct timescales. Importantly, early responses were sensitive to a longer history of stimulation than later responses. To account for these results, we tested a model consisting of neural populations with frequency-specific but broad tuning that undergo adaptation with exponential recovery. We found that the coexistence of neural populations with distinct recovery rates can explain our results. Furthermore, the adaptation bandwidth of these populations depended on spectral context-it was wider when the stimulation sequence had a wider frequency range. Our results provide electrophysiological evidence as well as a possible mechanistic explanation for dynamic and multiscale context-dependent auditory processing in the human cortex.

Neural mechanisms of rhythm-based temporal prediction: Delta phase-locking reflects temporal predictability but not rhythmic entrainment.

Predicting the timing of upcoming events enables efficient resource allocation and action preparation. Rhythmic streams, such as music, speech, and biological motion, constitute a pervasive source for temporal predictions. Widely accepted entrainment theories postulate that rhythm-based predictions are mediated by synchronizing low-frequency neural oscillations to the rhythm, as indicated by increased phase concentration (PC) of low-frequency neural activity for rhythmic compared to random streams. However, we show here that PC enhancement in scalp recordings is not specific to rhythms but is observed to the same extent in less periodic streams if they enable memory-based prediction. This is inconsistent with the predictions of a computational entrainment model of stronger PC for rhythmic streams. Anticipatory change in alpha activity and facilitation of electroencephalogram (EEG) manifestations of response selection are also comparable between rhythm- and memory-based predictions. However, rhythmic sequences uniquely result in obligatory depression of preparation-related premotor brain activity when an on-beat event is omitted, even when it is strategically beneficial to maintain preparation, leading to larger behavioral costs for violation of prediction. Thus, while our findings undermine the validity of PC as a sign of rhythmic entrainment, they constitute the first electrophysiological dissociation, to our knowledge, between mechanisms of rhythmic predictions and of memory-based predictions: the former obligatorily lead to resonance-like preparation patterns (that are in line with entrainment), while the latter allow flexible resource allocation in time regardless of periodicity in the input. Taken together, they delineate the neural mechanisms of three distinct modes of preparation: continuous vigilance, interval-timing-based prediction and rhythm-based prediction.

Face Selective Neural Activity: Comparisons Between Fixed and Free Viewing.

Event Related Potentials (ERPs) are widely used to study category-selective EEG responses to visual stimuli, such as the face-selective N170 component. Typically, this is done by flashing stimuli at the point of static gaze fixation. While allowing for good experimental control, these paradigms ignore the dynamic role of eye-movements in natural vision. Fixation-related potentials (FRPs), obtained using simultaneous EEG and eye-tracking, overcome this limitation. Various studies have used FRPs to study processes such as lexical processing, target detection and attention allocation. The goal of this study was to carefully compare face-sensitive activity time-locked to an abrupt stimulus onset at fixation, with that time-locked to a self-generated fixation on a stimulus. Twelve participants participated in three experimental conditions: Free-viewing (FRPs), Cued-viewing (FRPs) and Control (ERPs). We used a multiple regression approach to disentangle overlapping activity components. Our results show that the N170 face-effect is evident for the first fixation on a stimulus, whether it follows a self-generated saccade or stimulus appearance at fixation point. The N170 face-effect has similar topography across viewing conditions, but there were major differences within each stimulus category. We ascribe these differences to an overlap of the fixation-related lambda response and the N170. We tested the plausibility of this account using dipole simulations. Finally, the N170 exhibits category-specific adaptation in free viewing. This study establishes the comparability of the free-viewing N170 face-effect with the classic event-related effect, while highlighting the importance of accounting for eye-movement related effects.

Direct Evidence for Prediction Signals in Frontal Cortex Independent of Prediction Error.

Predictive coding (PC) has been suggested as one of the main mechanisms used by brains to interact with complex environments. PC theories posit top-down prediction signals, which are compared with actual outcomes, yielding in turn prediction error (PE) signals, which are used, bottom-up, to modify the ensuing predictions. However, disentangling prediction from PE signals has been challenging. Critically, while many studies found indirect evidence for PC in the form of PE signals, direct evidence for the prediction signal is mostly lacking. Here, we provide clear evidence, obtained from intracranial cortical recordings in human surgical patients, that the human lateral prefrontal cortex evinces prediction signals while anticipating an event. Patients listened to task-irrelevant sequences of repetitive tones including infrequent predictable or unpredictable pitch deviants. The broadband high-frequency amplitude (HFA) was decreased prior to the onset of expected relative to unexpected deviants in the frontal cortex only, and its amplitude was sensitive to the increasing likelihood of deviants following longer trains of standards in the unpredictable condition. Single-trial HFA predicted deviations and correlated with poststimulus response to deviations. These results provide direct evidence for frontal cortex prediction signals independent of PE signals.

Evidence for Linear but Not Helical Automatic Representation of Pitch in the Human Auditory System.

The perceptual organization of pitch is frequently described as helical, with a monotonic dimension of pitch height and a circular dimension of pitch chroma, accounting for the repeating structure of the octave. Although the neural representation of pitch height is widely studied, the way in which pitch chroma representation is manifested in neural activity is currently debated. We tested the automaticity of pitch chroma processing using the MMN-an ERP component indexing automatic detection of deviations from auditory regularity. Musicians trained to classify pure or complex tones across four octaves, based on chroma-C versus G (21 participants, Experiment 1) or C versus F# (27, Experiment 2). Next, they were passively exposed to MMN protocols designed to test automatic detection of height and chroma deviations. Finally, in an "attend chroma" block, participants had to detect the chroma deviants in a sequence similar to the passive MMN sequence. The chroma deviant tones were accurately detected in the training and the attend chroma parts both for pure and complex tones, with a slightly better performance for complex tones. However, in the passive blocks, a significant MMN was found only to height deviations and complex tone chroma deviations, but not to pure tone chroma deviations, even for perfect performers in the active tasks. These results indicate that, although height is represented preattentively, chroma is not. Processing the musical dimension of chroma may require higher cognitive processes, such as attention and working memory.

Systematic comparison between a wireless EEG system with dry electrodes and a wired EEG system with wet electrodes.

Recent advances in dry electrodes technology have facilitated the recording of EEG in situations not previously possible, thanks to the relatively swift electrode preparation and avoidance of applying gel to subject's hair. However, to become a true alternative, these systems should be compared to state-of-the-art wet EEG systems commonly used in clinical or research applications. In our study, we conducted a systematic comparison of electrodes application speed, subject comfort, and most critically electrophysiological signal quality between the conventional and wired Biosemi EEG system using wet active electrodes and the compact and wireless F1 EEG system consisting of dry passive electrodes. All subjects (n = 27) participated in two recording sessions on separate days, one with the wet EEG system and one with the dry EEG system, in which the session order was counterbalanced across subjects. In each session, we recorded their EEG during separate rest periods with eyes open and closed followed by two versions of a serial visual presentation target detection task. Each task component allows for a neural measure reflecting different characteristics of the data, including spectral power in canonical low frequency bands, event-related potential components (specifically, the P3b), and single trial classification based on machine learning. The performance across the two systems was similar in most measures, including the P3b amplitude and topography, as well as low frequency (theta, alpha, and beta) spectral power at rest. Both EEG systems performed well above chance in the classification analysis, with a marginal advantage of the wet system over the dry. Critically, all aforementioned electrophysiological metrics showed significant positive correlations (r = 0.54-0.89) between the two EEG systems. This multitude of measures provides a comprehensive comparison that captures different aspects of EEG data, including temporal precision, frequency domain as well as multivariate patterns of activity. Taken together, our results indicate that the dry EEG system used in this experiment can effectively record electrophysiological measures commonly used across the research and clinical contexts with comparable quality to the conventional wet EEG system.

Hierarchy of prediction errors for auditory events in human temporal and frontal cortex.

Predictive coding theories posit that neural networks learn statistical regularities in the environment for comparison with actual outcomes, signaling a prediction error (PE) when sensory deviation occurs. PE studies in audition have capitalized on low-frequency event-related potentials (LF-ERPs), such as the mismatch negativity. However, local cortical activity is well-indexed by higher-frequency bands [high-γ band (Hγ): 80-150 Hz]. We compared patterns of human Hγ and LF-ERPs in deviance detection using electrocorticographic recordings from subdural electrodes over frontal and temporal cortices. Patients listened to trains of task-irrelevant tones in two conditions differing in the predictability of a deviation from repetitive background stimuli (fully predictable vs. unpredictable deviants). We found deviance-related responses in both frequency bands over lateral temporal and inferior frontal cortex, with an earlier latency for Hγ than for LF-ERPs. Critically, frontal Hγ activity but not LF-ERPs discriminated between fully predictable and unpredictable changes, with frontal cortex sensitive to unpredictable events. The results highlight the role of frontal cortex and Hγ activity in deviance detection and PE generation.

Human posterior parietal cortex responds to visual stimuli as early as peristriate occipital cortex.

Much of what is known about the timing of visual processing in the brain is inferred from intracranial studies in monkeys, with human data limited to mainly noninvasive methods with lower spatial resolution. Here, we estimated visual onset latencies from electrocorticographic (ECoG) recordings in a patient who was implanted with 112 subdural electrodes, distributed across the posterior cortex of the right hemisphere, for presurgical evaluation of intractable epilepsy. Functional MRI prior to surgery was used to determine boundaries of visual areas. The patient was presented with images of objects from several categories. Event-related potentials (ERPs) were calculated across all categories excluding targets, and statistically reliable onset latencies were determined, using a bootstrapping procedure over the single trial baseline activity in individual electrodes. The distribution of onset latencies broadly reflected the known hierarchy of visual areas, with the earliest cortical responses in primary visual cortex, and higher areas showing later responses. A clear exception to this pattern was a robust, statistically reliable and spatially localized, very early response, on the bank of the posterior intraparietal sulcus (IPS). The response in the IPS started nearly simultaneously with responses detected in peristriate visual areas, around 60 ms poststimulus onset. Our results support the notion of early visual processing in the posterior parietal lobe, not respecting traditional hierarchies, and give direct evidence for onset times of visual responses across the human cortex.

When Synchronizing to Rhythms Is Not a Good Thing: Modulations of Preparatory and Post-Target Neural Activity When Shifting Attention Away from On-Beat Times of a Distracting Rhythm.

UNLABELLED: Environmental rhythms potently drive predictive resource allocation in time, typically leading to perceptual and motor benefits for on-beat, relative to off-beat, times, even if the rhythmic stream is not intentionally used. In two human EEG experiments, we investigated the behavioral and electrophysiological expressions of using rhythms to direct resources away from on-beat times. This allowed us to distinguish goal-directed attention from the automatic capture of attention by rhythms. The following three conditions were compared: (1) a rhythmic stream with targets appearing frequently at a fixed off-beat position; (2) a rhythmic stream with targets appearing frequently at on-beat times; and (3) a nonrhythmic stream with matched target intervals. Shifting resources away from on-beat times was expressed in the slowing of responses to on-beat targets, but not in the facilitation of off-beat targets. The shifting of resources was accompanied by anticipatory adjustment of the contingent negative variation (CNV) buildup toward the expected off-beat time. In the second experiment, off-beat times were jittered, resulting in a similar CNV adjustment and also in preparatory amplitude reduction of beta-band activity. Thus, the CNV and beta activity track the relevance of time points and not the rhythm, given sufficient incentive. Furthermore, the effects of task relevance (appearing in a task-relevant vs irrelevant time) and rhythm (appearing on beat vs off beat) had additive behavioral effects and also dissociable neural manifestations in target-evoked activity: rhythm affected the target response as early as the P1 component, while relevance affected only the later N2 and P3. Thus, these two factors operate by distinct mechanisms. SIGNIFICANCE STATEMENT: Rhythmic streams are widespread in our environment, and are typically conceptualized as automatic, bottom-up resource attractors to on-beat times-preparatory neural activity peaks at rhythm-on-beat times and behavioral benefits are seen to on-beat compared with off-beat targets. We show that this behavioral benefit is reversed when targets are more frequent at off-beat compared with on-beat times, and that preparatory neural activity, previously thought to be driven by the rhythm to on-beat times, is adjusted toward off-beat times. Furthermore, the effect of this relevance-based shifting on target-evoked brain activity was dissociable from the automatic effect of rhythms. Thus, rhythms can act as cues for flexible resource allocation according to the goal relevance of each time point, instead of being obligatory resource attractors.

Hierarchies of Attention and Experimental Designs: Effects of Spatial and Intermodal Attention Revisited.

When attention is directed to stimuli in a given modality and location, information processing in other irrelevant modalities at this location is affected too. This spread of attention to irrelevant stimuli is often interpreted as superiority of location selection over modality selection. However, this conclusion is based on experimental paradigms in which spatial attention was transient whereas intermodal attention was sustained. Furthermore, whether modality selection affects processing in the task-relevant modality at irrelevant locations remains an open question. Here, we addressed effects of simultaneous spatial and intermodal attention in an EEG study using a balanced design where spatial attention was transient and intermodal attention sustained or vice versa. Effects of spatial attention were not affected by which modality was attended and effects of intermodal attention were not affected by whether the stimuli were at the attended location or not. This suggests not only spread of spatial attention to task-irrelevant modalities but also spread of intermodal attention to task-irrelevant locations. Whether spatial attention was transient or sustained did not alter the effect of spatial attention on visual N1 and Nd1 responses. Prestimulus preparatory occipital alpha band responses were affected by both transient and sustained spatial cueing, whereas late post-stimulus responses were more strongly affected by sustained than by transient spatial attention. Sustained but not transient intermodal attention affected late responses (>200 msec) to visual stimuli. Together, the results undermine the universal superiority of spatial attention and suggest that the mode of attention manipulation is an important factor determining attention effects.

Cortical representation of persistent visual stimuli.

Research into visual neural activity has focused almost exclusively on onset- or change-driven responses and little is known about how information is encoded in the brain during sustained periods of visual perception. We used intracranial recordings in humans to determine the degree to which the presence of a visual stimulus is persistently encoded by neural activity. The correspondence between stimulus duration and neural response duration was strongest in early visual cortex and gradually diminished along the visual hierarchy, such that is was weakest in inferior-temporal category-selective regions. A similar posterior-anterior gradient was found within inferior temporal face-selective regions, with posterior but not anterior sites showing persistent face-selective activity. The results suggest that regions that appear uniform in terms of their category selectivity are dissociated by how they temporally represent a stimulus in support of ongoing visual perception, and delineate a large-scale organizing principle of the ventral visual stream.

Evidence of Change of Intention in Picking Situations.

Intending to perform an action and then immediately executing it is a mundane process. The cognitive and neural mechanisms involved in this process of "proximal" intention formation and execution, in the face of multiple options to choose from, are not clear, however. Especially, it is not clear how intentions are formed when the choice makes no difference. Here we used behavioral and electrophysiological measures to investigate the temporal dynamics of proximal intention formation and "change of intention" in a free picking scenario, in which the alternatives are on a par for the participant. Participants pressed a right or left button following either an instructive visible arrow cue or a visible neutral "free-choice" cue, both preceded by a masked arrow prime. The goal of the prime was to induce a bias toward pressing the left or right button. Presumably, when the choice is arbitrary, such bias should determine the decision. EEG lateralized readiness potentials and EMG measurements revealed that the prime indeed induced an intention to move in one direction. However, we discovered a signature of "change of intention" in both the Instructed and Free-choice decisions. These results suggest that, even in arbitrary choices, biases present in the neural system for choosing one or another option may be overruled and point to a curious "picking deliberation" phenomenon. We discuss a possible neural scenario that could explain this phenomenon.

Automatic bias of temporal expectations following temporally regular input independently of high-level temporal expectation.

Exposure to rhythmic stimulation results in facilitated responses to events that appear in-phase with the rhythm and modulation of anticipatory and target-evoked brain activity, presumably reflecting "exogenous," unintentional temporal expectations. However, the extent to which this effect is independent from intentional processes is not clear. In two EEG experiments, we isolated the unintentional component of this effect from high-level, intentional factors. Visual targets were presented either in-phase or out-of-phase with regularly flickering colored stimuli. In different blocks, the rhythm could be predictive (i.e., high probability for in-phase target) or not, and the color could be predictive (i.e., validly cue the interval to the target) or not. Exposure to nonpredictive rhythms resulted in faster responses for in-phase targets, even when the color predicted specific out-of-phase target times. Also, the contingent negative variation, an EEG component reflecting temporal anticipation, followed the interval of the nonpredictive rhythm and not that of the predictive color. Thus, rhythmic stimulation unintentionally induced expectations, even when this was detrimental. Intentional usage of predictive rhythms to form expectations resulted in a stronger behavioral effect, and only predictive cues modulated the latency of the target-evoked P3, presumably reflecting stimulus evaluation. These findings establish the existence of unintentional temporal expectations in rhythmic contexts, dissociate them from intentional expectations, and highlight the need to distinguish between the source of expectation (exogenous-endogenous) and the level of voluntary control involved in it (unintentional-intentional).

Spatial localization of auditory stimuli in human auditory cortex is based on both head-independent and head-centered coordinate systems.

In humans, whose ears are fixed on the head, auditory stimuli are initially registered in space relative to the head. Eventually, locations of sound sources need to be encoded also relative to the body, or in absolute allocentric space, to allow orientation toward the sounds sources and consequent action. We can therefore distinguish between two spatial representation systems: a basic head-centered coordinate system and a more complex head-independent system. In an ERP experiment, we attempted to reveal which of these two coordinate systems is represented in the human auditory cortex. We dissociated the two systems using the mismatch negativity (MMN), a well studied EEG effect evoked by acoustic deviations. Contrary to previous findings suggesting that only primary head-related information is present at this early stage of processing, we observed significant MMN effects for both head-independent and head-centered deviant stimuli. Our findings thus reveal that both primary head-related and secondary body- or world-related reference frames are represented at this stage of auditory processing.

Error monitoring when no errors are possible: Arbitrary free-choice decisions invoke error monitoring processes.

Some decisions make a difference, but most are arbitrary and inconsequential, like which of several identical new pairs of socks should I wear? Healthy people swiftly make such decisions even with no rational reasons to rely on. In fact, arbitrary decisions have been suggested as demonstrating "free will". However, several clinical populations and some healthy individuals have significant difficulties in making such arbitrary decisions. Here, we investigate the mechanisms involved in arbitrary picking decisions. We show that these decisions, arguably based on a whim, are subject to similar control mechanisms as reasoned decisions. Specifically, error-related negativity (ERN) brain response is elicited in the EEG following change of intention, without an external definition of error, and motor activity in the non-responding hand resembles actual errors both by its muscle EMG temporal dynamics and by the lateralized readiness potential (LRP) pattern. This provides new directions in understanding decision-making and its deficits.

Distinct ventral stream and prefrontal cortex representational dynamics during sustained conscious visual perception.

Instances of sustained stationary sensory input are ubiquitous. However, previous work focused almost exclusively on transient onset responses. This presents a critical challenge for neural theories of consciousness, which should account for the full temporal extent of experience. To address this question, we use intracranial recordings from ten human patients with epilepsy to view diverse images of multiple durations. We reveal that, in sensory regions, despite dramatic changes in activation magnitude, the distributed representation of categories and exemplars remains sustained and stable. In contrast, in frontoparietal regions, we find transient content representation at stimulus onset. Our results highlight the connection between the anatomical and temporal correlates of experience. To the extent perception is sustained, it may rely on sensory representations and to the extent perception is discrete, centered on perceptual updating, it may rely on frontoparietal representations.

Hearing while blinking: multisensory attentional blink revisited.

It is well established that cognitive system overload is reflected in the attentional blink (AB), the failure to report a second target when it closely follows detection of a first target within a rapid series of stimuli. However, there is intense controversy concerning the effect of first-target detection in one modality on subsequent dynamics of attentional resources in other modalities. Mixed results were found using an audiovisual AB paradigm: depletion of resources in one modality either impaired performance in the other modality or had no effect. Here, we circumvent the need for task switching by measuring an event-related potential, the mismatch negativity, which reflects implicit auditory change detection without requiring task engagement and is present even for background sounds that participants ignore. Surprisingly, we find that during the visual AB, auditory processing is enhanced rather than inhibited, as would be expected by system overload. We suggest that multimodal attentional resources may be freed rather than engaged during the visual AB. Suppression of irrelevant input may require active control by a central executive, which is preoccupied during the visual AB, and/or there may be no reason to suppress other-modal input since the visual system will miss its second target anyway.

Distinct interacting cortical networks for stimulus-response and repetition-suppression.

Non-invasive studies consider the initial neural stimulus response (SR) and repetition suppression (RS) - the decreased response to repeated sensory stimuli - as engaging the same neurons. That is, RS is a suppression of the SR. We challenge this conjecture using electrocorticographic (ECoG) recordings with high spatial resolution in ten patients listening to task-irrelevant trains of auditory stimuli. SR and RS were indexed by high-frequency activity (HFA) across temporal, parietal, and frontal cortices. HFASR and HFARS were temporally and spatially distinct, with HFARS emerging later than HFASR and showing only a limited spatial intersection with HFASR: most HFASR sites did not demonstrate HFARS, and HFARS was found where no HFASR could be recorded. ß activity was enhanced in HFARS compared to HFASR cortical sites. θ activity was enhanced in HFASR compared to HFARS sites. Furthermore, HFASR sites propagated information to HFARS sites via transient θ:ß phase-phase coupling. In contrast to predictive coding (PC) accounts our results indicate that HFASR and HFARS are functionally linked but have minimal spatial overlap. HFASR might enable stable and rapid perception of environmental stimuli across extended temporal intervals. In contrast HFARS might support efficient generation of an internal model based on stimulus history.