Conditional deviant repetition in the oddball paradigm modulates processing at the level of P3a but not MMN.

The auditory system has an amazing ability to rapidly encode auditory regularities. Evidence comes from the popular oddball paradigm, in which frequent (standard) sounds are occasionally exchanged for rare deviant sounds, which then elicit signs of prediction error based on their unexpectedness (e.g., MMN and P3a). Here, we examine the widely neglected characteristics of deviants being bearers of predictive information themselves; naive participants listened to sound sequences constructed according to a new, modified version of the oddball paradigm including two types of deviants that followed diametrically opposed rules: one deviant sound occurred mostly in pairs (repetition rule), the other deviant sound occurred mostly in isolation (non-repetition rule). Due to this manipulation, the sound following a first deviant (either the same deviant or a standard) was either predictable or unpredictable based on its conditional probability associated with the preceding deviant sound. Our behavioral results from an active deviant detection task replicate previous findings that deviant repetition rules (based on conditional probability) can be extracted when behaviorally relevant. Our electrophysiological findings obtained in a passive listening setting indicate that conditional probability also translates into differential processing at the P3a level. However, MMN was confined to global deviants and was not sensitive to conditional probability. This suggests that higher-level processing concerned with stimulus selection and/or evaluation (reflected in P3a) but not lower-level sensory processing (reflected in MMN) considers rarely encountered rules.

That sounds awful! Does sound unpleasantness modulate the mismatch negativity and its habituation?

There are sounds that most people perceive as highly unpleasant, for instance, the sound of rubbing pieces of polystyrene together. Previous research showed larger physiological and neural responses for such aversive compared to neutral sounds. Hitherto, it remains unclear whether habituation, i.e., diminished responses to repeated stimulus presentation, which is typically reported for neutral sounds, occurs to the same extent for aversive stimuli. We measured the mismatch negativity (MMN) in response to rare occurrences of aversive or neutral deviant sounds within an auditory oddball sequence in 24 healthy participants, while they performed a demanding visual distractor task. Deviants occurred as single events (i.e., between two standards) or as double deviants (i.e., repeating the identical deviant sound in two consecutive trials). All deviants elicited a clear MMN, and amplitudes were larger for aversive than for neutral deviants (irrespective of their position within a deviant pair). This supports the claim of preattentive emotion evaluation during early auditory processing. In contrast to our expectations, MMN amplitudes did not show habituation, but increased in response to deviant repetition-similarly for aversive and neutral deviants. A more fine-grained analysis of individual MMN amplitudes in relation to individual arousal and valence ratings of each sound item revealed that stimulus-specific MMN amplitudes were best predicted by the interaction of deviant position and perceived arousal, but not by valence. Deviants with perceived higher arousal elicited larger MMN amplitudes only at the first deviant position, indicating that the MMN reflects preattentive processing of the emotional content of sounds.

Markov chains as a proxy for the predictive memory representations underlying mismatch negativity.

Events not conforming to a regularity inherent to a sequence of events elicit prediction error signals of the brain such as the Mismatch Negativity (MMN) and impair behavioral task performance. Events conforming to a regularity lead to attenuation of brain activity such as stimulus-specific adaptation (SSA) and behavioral benefits. Such findings are usually explained by theories stating that the information processing system predicts the forthcoming event of the sequence via detected sequential regularities. A mathematical model that is widely used to describe, to analyze and to generate event sequences are Markov chains: They contain a set of possible events and a set of probabilities for transitions between these events (transition matrix) that allow to predict the next event on the basis of the current event and the transition probabilities. The accuracy of such a prediction depends on the distribution of the transition probabilities. We argue that Markov chains also have useful applications when studying cognitive brain functions. The transition matrix can be regarded as a proxy for generative memory representations that the brain uses to predict the next event. We assume that detected regularities in a sequence of events correspond to (a subset of) the entries in the transition matrix. We apply this idea to the Mismatch Negativity (MMN) research and examine three types of MMN paradigms: classical oddball paradigms emphasizing sound probabilities, between-sound regularity paradigms manipulating transition probabilities between adjacent sounds, and action-sound coupling paradigms in which sounds are associated with actions and their intended effects. We show that the Markovian view on MMN yields theoretically relevant insights into the brain processes underlying MMN and stimulates experimental designs to study the brain's processing of event sequences.

Deviants violating higher-order auditory regularities can become predictive and facilitate behaviour.

The human auditory system is believed to represent regularities inherent in auditory information in internal models. Sounds not matching the standard regularity (deviants) elicit prediction error, alerting the system to information not explainable within currently active models. Here, we examine the widely neglected characteristic of deviants bearing predictive information themselves. In a modified version of the oddball paradigm, using higher-order regularities, we set up different expectations regarding the sound following a deviant. Higher-order regularities were defined by the relation of pitch within tone pairs (rather than absolute pitch of individual tones). In a deviant detection task participants listened to oddball sequences including two deviant types following diametrically opposed rules: one occurred mostly in succession (high repetition probability) and the other mostly in isolation (low repetition probability). Participants in Experiment 1 were not informed (naïve), whereas in Experiment 2 they were made aware of the repetition rules. Response times significantly decreased from first to second deviant when repetition probability was high-albeit more in the presence of explicit rule knowledge. There was no evidence of a facilitation effect when repetition probability was low. Significantly more false alarms occurred in response to standards following high compared with low repetition probability deviants, but only in participants aware of the repetition rules. These findings provide evidence that not only deviants violating lower- but also higher-order regularities can inform predictions about auditory events. More generally, they confirm the utility of this new paradigm to gather further insights into the predictive properties of the human brain.

Encoding of deterministic and stochastic auditory rules in the human brain: The mismatch negativity mechanism does not reflect basic probability.

Regularities in a sequence of sounds can be automatically encoded in a predictive model by the auditory system. When a sound deviates from the one predicted by the model, a mismatch negativity (MMN) is elicited, which is taken to reflect a prediction error at a particular level of the model hierarchy. Although there are many studies on deterministic regularities, only a few have investigated the brain's ability to encode non-deterministic regularities. We studied a simple stochastic regularity: two tone pitches (standards, each occurring on 45% of trials); this regularity was occasionally violated by another tone pitch (deviant, occurring on 10% of trials). We found MMN when the deviant's pitch was outside those of the standards, but not when it was between them. Importantly, when we alternated the occurrence of the same two standards, making them deterministic, the deviant elicited MMN, even when its pitch was between those of the standards. Thus, although the MMN system is extremely powerful in establishing even quite complex deterministic regularities, it fails with a simple stochastic regularity. We argue that the MMN system does not know basic probability.

The quest for the genuine visual mismatch negativity (vMMN): Event-related potential indications of deviance detection for low-level visual features.

Research shows that the visual system monitors the environment for changes. For example, a left-tilted bar, a deviant, that appears after several presentations of a right-tilted bar, standards, elicits a classic visual mismatch negativity (vMMN): greater negativity for deviants than standards in event-related potentials (ERPs) between 100 and 300 ms after onset of the deviant. The classic vMMN is contributed to by adaptation; it can be distinguished from the genuine vMMN that, through use of control conditions, compares standards and deviants that are equally adapted and physically identical. To determine whether the vMMN follows similar principles to the auditory mismatch negativity (MMN), in two experiments we searched for a genuine vMMN from simple, physiologically plausible stimuli that change in fundamental dimensions: orientation, contrast, phase, and spatial frequency. We carefully controlled for attention and eye movements. We found no evidence for the genuine vMMN, despite adequate statistical power. We conclude that either the genuine vMMN is a rather unstable phenomenon that depends on still-to-be-identified experimental parameters, or it is confined to visual stimuli for which monitoring across time is more natural than monitoring over space, such as for high-level features. We also observed an early deviant-related positivity that we propose might reflect earlier predictive processing.

Do early neural correlates of visual consciousness show the oblique effect? A binocular rivalry and event-related potential study.

When dissimilar images are presented one to each eye, we do not see both images; rather, we see one at a time, alternating unpredictably. This is called binocular rivalry, and it has recently been used to study brain processes that correlate with visual consciousness, because perception changes without any change in the sensory input. Such studies have used various types of images, but the most popular have been gratings: sets of bright and dark lines of orthogonal orientations presented one to each eye. We studied whether using cardinal rival gratings (vertical, 0°, and horizontal, 90°) versus oblique rival gratings (left-oblique, -45°, and right-oblique, 45°) influences early neural correlates of visual consciousness, because of the oblique effect: the tendency for visual performance to be greater for cardinal gratings than for oblique gratings. Participants viewed rival gratings and pressed keys indicating which of the two gratings they perceived, was dominant. Next, we changed one of the gratings to match the grating shown to the other eye, yielding binocular fusion. Participants perceived the rivalry-to-fusion change to the dominant grating and not to the other, suppressed grating. Using event-related potentials (ERPs), we found neural correlates of visual consciousness at the P1 for both sets of gratings, as well as at the P1-N1 for oblique gratings, and we found a neural correlate of the oblique effect at the N1, but only for perceived changes. These results show that the P1 is the earliest neural activity associated with visual consciousness and that visual consciousness might be necessary to elicit the oblique effect.

On the Discovery of Monocular Rivalry by Tscherning in 1898: Translation and Review.

Monocular rivalry was named by Breese in 1899. He made prolonged observation of superimposed orthogonal gratings; they fluctuated in clarity with either one or the other grating occasionally being visible alone. A year earlier, Tscherning observed similar fluctuations with a grid of vertical and horizontal lines and with other stimuli; we draw attention to his prior account. Monocular rivalry has since been shown to occur with a wide variety of superimposed patterns with several independent rediscoveries of it. We also argue that Helmholtz described some phenomenon other than monocular rivalry in 1867.

Gottfried Wilhelm Osann (1833, 1836) on Simultaneous Color Contrast: Translation and Commentary.

Gottfried Wilhelm Osann (1796-1866) was a German scientist most renowned for his work in chemistry and physics. However, inspired by Goethe's work on color, he published a paper on simultaneous color contrast in 1833 using a method that is similar to that of later authors: reflection of an achromatic spot from an angled piece of glass. He wrote at least four more papers on color contrasts, in 1836 using essentially the same method as credited to others. We provide a description and translation of Osann's 1833 paper and the relevant part of his 1836 paper, say why these papers are interesting and important, give some biographical information about Osann, comment on the fate of Osann's papers, and describe Osann's other papers on color.

Brain activity from stimuli that are not perceived: Visual mismatch negativity during binocular rivalry suppression.

Predictive coding explains visual perception as the result of an interaction between bottom-up sensory input and top-down generative models at each level of the visual hierarchy. Evidence for this comes from the visual mismatch negativity (vMMN): a more negative ERP for rare, unpredictable visual stimuli-deviants, than for frequent, predictable visual stimuli-standards. Here, we show that the vMMN does not require conscious experience. We measured the vMMN from monocular luminance-decrement deviants that were either perceived or not during binocular rivalry dominance or suppression, respectively. We found that both sorts of deviants elicited the vMMN at about 250 ms after stimulus onset, with perceived deviants eliciting a bigger vMMN than not-perceived deviants. These results show that vMMN occurs in the absence of consciousness, and that consciousness enhances the processing underlying vMMN. We conclude that generative models of visual perception are tested, even when sensory input for those models is not perceived.

We make predictions about eye of origin of visual input: Visual mismatch negativity from binocular rivalry.

The visual mismatch negativity (vMMN) is a negative component of event-related potentials (ERPs). It occurs when an infrequent visual stimulus, a deviant, is randomly and unpredictably presented in a sequence of frequent visual stimuli, the standards, and is thought to reflect prediction and prediction error of visual input. We investigated the sensitivity of vMMN to eye of origin (utrocular) information as well as to orientation information. We presented 80% of binocular rivalry standards (one grating to one eye and an identical, orthogonally oriented grating to the other eye), and 20% of deviants, either by swapping the gratings between the eyes to change the eye of origin of the gratings (an eye-swap deviant) or by rotating the gratings by 45° to change the orientation of the gratings (an orientation deviant). We found an orientation vMMN that was maximal at about 250 ms and an eye-swap vMMN that was maximal at about 380 ms. We also found deviance-related activity to both sorts of stimuli earlier than is traditionally defined as a vMMN. We used standardized low-resolution brain electromagnetic tomography (sLORETA) to localize each vMMN component and found similar sources for both vMMNs in occipital and frontal areas of the brain but differences in parietal and temporal areas. We conclude that eye of origin information can be used to elicit vMMN, that eye-swap vMMN is different to orientation vMMN, and that vMMN can be generated from information of which observers are unaware.

Predicting visual consciousness electrophysiologically from intermittent binocular rivalry.

PURPOSE: We sought brain activity that predicts visual consciousness. METHODS: We used electroencephalography (EEG) to measure brain activity to a 1000-ms display of sine-wave gratings, oriented vertically in one eye and horizontally in the other. This display yields binocular rivalry: irregular alternations in visual consciousness between the images viewed by the eyes. We replaced both gratings with 200 ms of darkness, the gap, before showing a second display of the same rival gratings for another 1000 ms. We followed this by a 1000-ms mask then a 2000-ms inter-trial interval (ITI). Eleven participants pressed keys after the second display in numerous trials to say whether the orientation of the visible grating changed from before to after the gap or not. Each participant also responded to numerous non-rivalry trials in which the gratings had identical orientations for the two eyes and for which the orientation of both either changed physically after the gap or did not. RESULTS: We found that greater activity from lateral occipital-parietal-temporal areas about 180 ms after initial onset of rival stimuli predicted a change in visual consciousness more than 1000 ms later, on re-presentation of the rival stimuli. We also found that less activity from parietal, central, and frontal electrodes about 400 ms after initial onset of rival stimuli predicted a change in visual consciousness about 800 ms later, on re-presentation of the rival stimuli. There was no such predictive activity when the change in visual consciousness occurred because the stimuli changed physically. CONCLUSION: We found early EEG activity that predicted later visual consciousness. Predictive activity 180 ms after onset of the first display may reflect adaption of the neurons mediating visual consciousness in our displays. Predictive activity 400 ms after onset of the first display may reflect a less-reliable brain state mediating visual consciousness.

Can eye of origin serve as a deviant? Visual mismatch negativity from binocular rivalry.

The visual mismatch negativity (vMMN) is a negative deflection in an event-related potential (ERP) between 200 and 400 ms after onset of an infrequent stimulus in a sequence of frequent stimuli. Binocular rivalry occurs when one image is presented to one eye and a different image is presented to the other. Although the images in the two eyes are unchanging, perception alternates unpredictably between the two images for as long as one cares to look. Binocular rivalry, therefore, provides a useful test of whether the vMMN is produced by low levels of the visual system at which the images are processed, or by higher levels at which perception is mediated. To investigate whether a vMMN can be evoked during binocular rivalry, we showed 80% standards comprising a vertical grating to one eye and a horizontal grating to the other and 20% deviants, in which the gratings either swapped between the eyes (eye-swap deviants) or changed their orientations by 45° (oblique deviants). Fourteen participants observed the stimuli in 16, 4-min blocks. In eight consecutive blocks, participants recorded their experiences of rivalry by pressing keys-we call this the attend-to-rivalry condition. In the remaining eight consecutive blocks, participants performed a demanding task at fixation (a 2-back task), also by pressing keys-we call this the reduced-attention condition. We found deviance-related negativity from about 140 ms to about 220 ms after onset of a deviant. There were two noticeable troughs that we call an early vMMN (140-160 ms) and a late vMMN (200-220 ms). These were essentially similar for oblique deviants and eye-swap deviants. They were also essentially similar in the attend-to-rivalry conditions and the reduced-attention conditions. We also found a late, deviance-related negativity from about 270 to about 290 ms in the attend-to-rivalry conditions. We conclude that the vMMN can be evoked during the ever-changing perceptual changes of binocular rivalry and that it is sensitive to the eye of origin of binocular-rivalry stimuli. This is consistent with the vMMN's being produced by low levels of the visual system.

Encoding into Visual Working Memory: Event-Related Brain Potentials Reflect Automatic Processing of Seemingly Redundant Information.

Encoding and maintenance of information in visual working memory in an S1-S2 task with a 1500 ms retention phase were investigated by means of event-related brain potentials (ERPs). Participants were asked to decide whether two visual stimuli were physically identical (identity comparison (IC) task) or belonged to the same set or category of equivalent patterns (category comparison (CC) task). The stimuli differ with regard to two features. (1) Each pattern can belong to a set of either four (ESS 4) or eight (ESS 8) equivalent patterns, mirroring differences in the complexity with regard to the representational structure of each pattern (i.e., equivalence set size (ESS)). (2) The set of patterns differ with regard to the rated complexity. Memory performance obtained the effects of the task instructions (IC versus CC) and the ESS (ESS 4 versus ESS 8) but not of the rated complexity. ERPs in the retention interval reveal that the stimulus-related factors (subjective complexity and ESS) affect the encoding of the stimuli as mirrored by the pronounced P3b amplitude in ESS 8 compared to ESS 4 patterns. Importantly, these effects are independent of task instructions. The pattern of results suggests an automatic processing of the ESS in the encoding phase.

Neural processing of orientation differences between the eyes' images.

The aim of this study was to explore the neural mechanisms underlying visual processing of brief stimuli that were either the same in the two eyes or differed in orientation between the two eyes. To examine the neural mechanisms, I measured event-related potentials (ERPs) to 200-ms sine-wave gratings differing in orientation between the eyes from 0° to 90°. The gratings were either both of high contrast or both of low contrast. They elicited typical ERPs at occipital electrodes, with a first major component (P100) 100 ms after stimulus onset and a second major component (N170) 170 ms after stimulus onset. Global electrical field strength and focal amplitudes of both components were affected by grating contrast: High-contrast gratings elicited larger amplitudes than low-contrast gratings, confirming that neural responses depend on stimulus salience. P100 amplitude followed a U-shaped function: It was larger when the orientations were the same in the two eyes (yielding binocular fusion), intermediate when the orientations were maximally different between the eyes (leading to binocular rivalry), and smallest for in-between orientation differences. N170 amplitude followed a linear function: It was smallest when the orientations were the same and increased with orientation difference between the eyes. These results suggest that the P100 reflects processes in which the binocular input are offset against each other, and that the N170 reflects binocular rivalry. I argue that the N170 shows the effects of reciprocal inhibition and adaptation--both critical factors in theories of binocular rivalry.

Impact of lower- vs. upper-hemifield presentation on automatic colour-deviance detection: a visual mismatch negativity study.

The automatic processing of deviances from the temporal context of the visual environment has become an important topic in visual cognitive sciences, which is often investigated using the visual mismatch negativity (vMMN). This event-related potential (ERP) component is elicited by an irregular stimulus (e.g., a red disc) presented in a series of stimuli (e.g., green discs) comprising a temporal regularity (e.g., colour repetition). We determined the influence of lower- vs. upper-hemifield presentation of the irregular stimulus on the vMMN while using whole-field stimulus displays controlling for sustained shifts in spatial attention. Deviances presented in the lower hemifield elicited a larger vMMN than the ones presented in the upper hemifield at a latency of 200-280ms. However, this asymmetry was preceded by deviance-related hemifield effects already emerging at an earlier latency (110-150ms), where upper-hemifield deviances elicited a positive potential but lower-hemifield deviances did not. With variable resolution electromagnetic tomography (VARETA) early deviance-related activity was localised to retinotopically organised regions of the visual cortex (BA 17/18) and vMMN-sources were localised to the middle/superior occipital gyrus, to higher areas along the temporal visual stream, but also to BA 17/18. We argue that the upper/lower-hemifield vMMN asymmetry relies at least partially on the hemifield-dependent differential sensitivity of early deviance-related activity generated in retinotopically organised regions of the visual cortex. However, a superior automatic processing of deviances presented in the lower visual hemifield may also contribute to the effect.

On the role of attention in binocular rivalry: electrophysiological evidence.

During binocular rivalry visual consciousness fluctuates between two dissimilar monocular images. We investigated the role of attention in this phenomenon by comparing event-related potentials (ERPs) when binocular-rivalry stimuli were attended with when they were unattended. Stimuli were dichoptic, orthogonal gratings that yielded binocular rivalry and dioptic, identically oriented gratings that yielded binocular fusion. Events were all possible orthogonal changes in orientation of one or both gratings. We had two attention conditions: In the attend-to-grating condition, participants had to report changes in perceived orientation, focussing their attention on the gratings. In the attend-to-fixation condition participants had to report changes in a central fixation target, taking attention away from the gratings. We found, surprisingly, that attending to rival gratings yielded a smaller ERP component (the N1, from 160-210 ms) than attending to the fixation target. To explain this paradoxical effect of attention, we propose that rivalry occurs in the attend-to-fixation condition (we found an ERP signature of rivalry in the form of a sustained negativity from 210-300 ms) but that the mechanism processing the stimulus changes is more adapted in the attend-to-grating condition than in the attend-to-fixation condition. This is consistent with the theory that adaptation gives rise to changes of visual consciousness during binocular rivalry.

The effects of response sharing and stimulus presentation frequency on event-related potentials in an auditory oddball paradigm.

An experimental model for investigating the processes involved in reacting to unpredictable events is the oddball paradigm. We investigated how the commonality or independence of response options (i.e., many-to-one vs. one-to-one stimulus-response mappings) influences processing in an auditory oddball paradigm. Participants performed a discrimination task with two one-to-one and one two-to-one mappings. The pattern of conflict- and oddball-related N2 event-related potentials suggest that information that would allow correct responding is represented at the latency of the N2. Integration of this information takes place only by the latency of P3b, and longer reaction times to rare stimuli are probably due to processes preventing the utilization of this information. We also suggest that, in the given task context, conflict-related N2 may reflect the number of alternative stimuli leading to alternative response options.

Lengthy suppression from similar stimuli during rapid serial visual presentation.

The stimulus at any point in the visual field is rarely static during normal viewing: observer and object movement conspire to produce a continually changing series of stimuli. Our aim was to study both the short- and long-term interactions between responses to a series of stimuli presented at a single visual location. We used rapid serial visual presentation (RSVP) in which the stimuli were randomly oriented gratings delivered at the rate of 30 per second. Human subjects pressed a key whenever they saw a target orientation, for example horizontal. The results were analyzed by finding two orientations before each key-press. The first preceded the key-press by the reaction time, and the second preceded the first by an interval of variable duration. There were two main findings. First, the subject was more likely to press the key when the target was immediately preceded by a grating of similar orientation. This facilitation presumably results from the summation of sub-threshold inputs. Second, a key-press was reduced in probability when a target orientation was preceded by a similar orientation with an interstimulus interval of 100-400 ms. The time course of this suppression is similar to that seen in attentional blink experiments.

Visual object representations can be formed outside the focus of voluntary attention: evidence from event-related brain potentials.

There is an ongoing debate whether visual object representations can be formed outside the focus of voluntary attention. Recently, implicit behavioral measures suggested that grouping processes can occur for task-irrelevant visual stimuli, thus supporting theories of preattentive object formation (e.g., Lamy, D., Segal, H., & Ruderman, L. Grouping does not require attention. Perception and Psychophysics, 68, 17-31, 2006; Russell, C., & Driver, J. New indirect measures of "inattentive" visual grouping in a change-detection task. Perception and Psychophysics, 67, 606-623, 2005). We developed an ERP paradigm that allows testing for visual grouping when neither the objects nor its constituents are related to the participant's task. Our paradigm is based on the visual mismatch negativity ERP component, which is elicited by stimuli deviating from a regular stimulus sequence even when the stimuli are ignored. Our stimuli consisted of four pairs of colored discs that served as objects. These objects were presented isochronously while participants were engaged in a task related to the continuously presented fixation cross. Occasionally, two color deviances occurred simultaneously either within the same object or across two different objects. We found significant ERP differences for same- versus different-object deviances, supporting the notion that forming visual object representations by grouping can occur outside the focus of voluntary attention. Also our behavioral experiment, in which participants responded to color deviances--thus, this time the discs but, again, not the objects were task relevant--showed that the object status matters. Our results stress the importance of early grouping processes for structuring the perceptual world.