Suppression and omission effects in auditory predictive processing-Two of the same?

Recent theories describe perception as an inferential process based on internal predictive models that are adjusted by prediction violations (prediction error). Two different modulations of the auditory N1 event-related brain potential component are often discussed as an expression of auditory predictive processing. The sound-related N1 component is attenuated for self-generated sounds compared to the N1 elicited by externally generated sounds (N1 suppression). An omission-related component in the N1 time-range is elicited when the self-generated sounds are occasionally omitted (omission N1). Both phenomena were explained by action-related forward modelling, which takes place when the sensory input is predictable: prediction error signals are reduced when predicted sensory input is presented (N1 suppression) and elicited when predicted sensory input is omitted (omission N1). This common theoretical account is appealing but has not yet been directly tested. We manipulated the predictability of a sound in a self-generation paradigm in which, in two conditions, either 80% or 50% of the button presses did generate a sound, inducing a strong or a weak expectation for the occurrence of the sound. Consistent with the forward modelling account, an omission N1 was observed in the 80% but not in the 50% condition. However, N1 suppression was highly similar in both conditions. Thus, our results demonstrate a clear effect of predictability for the omission N1 but not for the N1 suppression. These results imply that the two phenomena rely (at least in part) on different mechanisms and challenge prediction related accounts of N1 suppression.

Somatosensory omissions reveal action-related predictive processing.

The intricate relation between action and somatosensory perception has been studied extensively in the past decades. Generally, a forward model is thought to predict the somatosensory consequences of an action. These models propose that when an action is reliably coupled to a tactile stimulus, unexpected absence of the stimulus should elicit prediction error. Although such omission responses have been demonstrated in the auditory modality, it remains unknown whether this mechanism generalizes across modalities. This study therefore aimed to record action-induced somatosensory omission responses using EEG in humans. Self-paced button presses were coupled to somatosensory stimuli in 88% of trials, allowing a prediction, or in 50% of trials, not allowing a prediction. In the 88% condition, stimulus omission resulted in a neural response consisting of multiple components, as revealed by temporal principal component analysis. The oN1 response suggests similar sensory sources as stimulus-evoked activity, but an origin outside primary cortex. Subsequent oN2 and oP3 responses, as previously observed in the auditory domain, likely reflect modality-unspecific higher order processes. Together, findings straightforwardly demonstrate somatosensory predictions during action and provide evidence for a partially amodal mechanism of prediction error generation.

Hearing "Birch" Hampers Saying "Duck"-An Event-Related Potential Study on Phonological Interference in Immediate and Delayed Word Production.

When speakers name a picture (e.g., "duck"), a distractor word phonologically related to an alternative name (e.g., "birch" related to "bird") slows down naming responses compared with an unrelated distractor word. This interference effect obtained with the picture-word interference task is assumed to reflect the phonological coactivation of close semantic competitors and is critical for evaluating contemporary models of word production. In this study, we determined the ERP signature of this effect in immediate and delayed versions of the picture-word interference task. ERPs revealed a differential processing of related and unrelated distractors: an early (305-436 msec) and a late (537-713 msec) negativity for related as compared with unrelated distractors. In the behavioral data, the interference effect was only found in immediate naming, whereas its ERP signature was also present in delayed naming. The time window of the earlier ERP effect suggests that the behavioral interference effect indeed emerges at a phonological processing level, whereas the functional significance of the later ERP effect is as yet not clear. The finding of a robust ERP correlate of phonological coactivation might facilitate future research on lexical processing in word production.

Attentional control in middle childhood is highly dynamic-Strong initial distraction is followed by advanced attention control.

The ability to shield against distraction while focusing on a task requires the operation of executive functions and is essential for successful learning. We investigated the short-term dynamics of distraction control in a data set of 269 children aged 4-10 years and 51 adults pooled from three studies using multilevel models. Participants performed a visual categorization task while a task-irrelevant sequence of sounds was presented which consisted of frequently repeated standard sounds and rarely interspersed novel sounds. On average, participants responded slower in the categorization task after novel sounds. This distraction effect was more pronounced in children. Throughout the experiment, the initially strong distraction effects declined to the level of adults in the groups of 6- to 10-year-olds. Such a decline was neither observed in the groups of the 4- and 5-year-olds, who consistently showed a high level of distraction, nor in adults, who showed a constantly low level of distraction throughout the experimental session. Results indicate that distraction control is a highly dynamic process that qualitatively and quantitatively differs between age groups. We conclude that the analysis of short-term dynamics provides valuable insights into the development of attention control and might explain inconsistent findings regarding distraction control in middle childhood. In addition, models of attention control need to be refined to account for age-dependent rapid learning mechanisms. Our findings have implications for the design of learning situations and provide an additional source of information for the diagnosis and treatment of children with attention deficit disorders.

Omission related brain responses reflect specific and unspecific action-effect couplings.

When an auditory stimulus is predicted but unexpectedly omitted, an omission response can be observed in the EEG. This endogenous response to the absence of a stimulus demonstrates the important role of prediction in perception. SanMiguel et al. (2013a) showed that in order to observe an omission response, a specific prediction concerning the identity of an upcoming stimulus is necessary. They used button presses coupled to either a single sound (predictable identity), or a random sound (unpredictable identity). In the event-related potentials, a sequence of omission responses consisting of oN1, oN2, and oP3 was observed in the single condition but not in the random condition. Given the importance of omission studies to understand the role of prediction in perception, we replicated this study. We enhanced statistical power by doubling the sample size and adjusting data pre-processing, and applied temporal principal component analysis and replication Bayes statistics. Results in the single sound condition were successfully replicated. Principal component analysis additionally revealed attenuated oN1 and oP3 omission responses in the random sound condition. These results suggest the existence of both specific and unspecific predictions along the sound processing hierarchy, where precision weighting possibly influences the strength of prediction error. Results are discussed in the framework of predictive coding and are congruent with everyday life, where uncertainty often requires broader or more general predictions.

What exactly is missing here? The sensory processing of unpredictable omissions is modulated by the specificity of expected action-effects.

We select our actions according to the desired outcomes; for instance, piano players press certain keys to generate specific musical notes. It is well-described that the omission of a predicted action-effect may elicit prediction error signals in the brain, but what happens in the case of simultaneous effector-specific (by contrast to effector-unspecific) predictions? To answer this question, we asked participants to press left and right keys to generate tones A and B; based on the action-effect association, the tones' identity was either predictable or unpredictable, while rarely, the expected input was omitted. Crucially, the data show that omissions following hand-specific associations reliably elicited a late omission N1 (oN1) component, by contrast to the hand-unspecific associations, where the late oN1 was rather weak. An additional condition where both key-presses generated a unique tone was implemented. Here, rare omissions of the expected tone generated both early and late oN1 responses, by contrast to the condition in which two simultaneous action-effect representations had to be maintained, where only late oN1 responses were elicited. Finally, omission P3 (oP3) responses were strongly elicited for all omission types without differences, indicating that a general expectation based on a tone presentation (rather than which tone), is likely indexed at this stage. The present results emphasize the top-down effects of action intention on the sensory processing of omissions, where unspecific (vs. specific) and multiple (vs. single) action-effect representations are associated with processing costs at the early sensory levels.

Picture-evoked changes in pupil size predict learning success in children.

Episodic memory, the ability to remember past events in time and place, develops during childhood. Much knowledge about the underlying neuronal mechanisms has been gained from methods not suitable for children. We applied pupillometry to study memory encoding and recognition mechanisms. Children aged 8 and 9 years (n = 24) and adults (n = 24) studied a set of visual scenes to later distinguish them from new pictures. Children performed worse than adults, demonstrating immature episodic memory. During memorization, picture-related changes in pupil diameter predicted later successful recognition. This prediction effect was also observed on a single-trial level. During retrieval, novel pictures showed stronger pupil constriction than familiar pictures in both age groups. The statistically independent effects of objective familiarity (previously presented pictures) versus subjective familiarity (pictures evaluated as familiar independent of the prior presentation) suggest dissociable underlying brain mechanisms. In addition, we isolated principal components of the picture-related pupil response that were differently affected by the memorization and retrieval effects. Results are discussed in the context of the maturation of the medial temporal lobe and prefrontal networks. Our results demonstrate the dissociation of distinct contributions to episodic memory with a psychophysiological method that is suitable for a wide age spectrum.

Action Intention-based and Stimulus Regularity-based Predictions: Same or Different?

We act on the environment to produce desired effects, but we also adapt to the environmental demands by learning what to expect next, based on experience: How do action-based predictions and sensory predictions relate to each other? We explore this by implementing a self-generation oddball paradigm, where participants performed random sequences of left and right button presses to produce frequent standard and rare deviant tones. By manipulating the action-tone association as well as the likelihood of a button press over the other one, we compare ERP effects evoked by the intention to produce a specific tone, tone regularity, and both intention and regularity. We show that the N1b and Tb components of the N1 response are modulated by violations of tone regularity only. However, violations of action intention as well as of regularity elicit MMN responses, which occur similarly in all three conditions. Regardless of whether the predictions at sensory levels were based on either intention, regularity, or both, the tone deviance was further and equally well detected at hierarchically higher processing level, as reflected in similar P3a effects between conditions. We did not observe additive prediction errors when intention and regularity were violated concurrently, suggesting the two integrate despite presumably having independent generators. Even though they are often discussed as individual prediction sources in the literature, this study represents to our knowledge the first to directly compare them. Finally, these results show how, in the context of action, our brain can easily switch between top-down intention-based expectations and bottom-up regularity cues to efficiently predict future events.

Presentation Probability of Visual-Auditory Pairs Modulates Visually Induced Auditory Predictions.

Predictions about forthcoming auditory events can be established on the basis of preceding visual information. Sounds being incongruent to predictive visual information have been found to elicit an enhanced negative ERP in the latency range of the auditory N1 compared with physically identical sounds being preceded by congruent visual information. This so-called incongruency response (IR) is interpreted as reduced prediction error for predicted sounds at a sensory level. The main purpose of this study was to examine the impact of probability manipulations on the IR. We manipulated the probability with which particular congruent visual-auditory pairs were presented (83/17 vs. 50/50 condition). This manipulation led to two conditions with different strengths of the association of visual with auditory information. A visual cue was presented either above or below a fixation cross and was followed by either a high- or low-pitched sound. In 90% of trials, the visual cue correctly predicted the subsequent sound. In one condition, one of the sounds was presented more frequently (83% of trials), whereas in the other condition both sounds were presented with equal probability (50% of trials). Therefore, in the 83/17 condition, one congruent combination of visual cue and corresponding sound was presented more frequently than the other combinations presumably leading to a stronger visual-auditory association. A significant IR for unpredicted compared with predicted but otherwise identical sounds was observed only in the 83/17 condition, but not in the 50/50 condition, where both congruent visual cue-sound combinations were presented with equal probability. We also tested whether the processing of the prediction violation is dependent on the task relevance of the visual information. Therefore, we contrasted a visual-auditory matching task with a pitch discrimination task. It turned out that the task only had an impact on the behavioral performance but not on the prediction error signals. Results suggest that the generation of visual-to-auditory sensory predictions is facilitated by a strong association between the visual cue and the predicted sound (83/17 condition) but is not influenced by the task relevance of the visual information.

Can't Ignore-Distraction by Task-Irrelevant Sounds in Early and Middle Childhood.

Attention control abilities are relevant for learning success. Little is known about the development of audio-visual attention in early childhood. Four groups of children between the ages of 4 and 10 years and adults performed an audio-visual distraction paradigm (N = 106). Multilevel analyses revealed increased reaction times in a visual categorization task when task-irrelevant novel sounds were presented, demonstrating involuntary distraction of attention. This distraction effect decreased with age and significantly differed between age groups. In addition, the two youngest age groups responded with a delay in trials following a distractor trial, indicating delayed reallocation of attention to the task at hand. Results indicate a significant maturation of audio-visual attention control within a few years during early childhood that continues throughout middle childhood.

Infant and adult pupil dilation in response to unexpected sounds.

Surprisingly occurring sounds outside the focus of attention can involuntarily capture attention. This study focuses on the impact of deviant sounds on the pupil size as a marker of auditory involuntary attention in infants. We presented an oddball paradigm including four types of deviant sounds within a sequence of repeated standard sounds to 14-month-old infants and to adults. Environmental and noise deviant sounds elicited a strong pupil dilation response (PDR) in both age groups. In contrast, moderate frequency deviants elicited a significant PDR in adults only. Moreover, a principal component analysis revealed two components underlying the PDR. Component scores differ, depending on deviant types, between age groups. Results indicate age effects of parasympathetic inhibition and sympathetic activation of the pupil size caused by deviant sounds with a high arousing potential. Results demonstrate that the PDR is a sensitive tool for the investigation of involuntary attention to sounds in preverbal children.

Microsaccadic responses indicate fast categorization of sounds: a novel approach to study auditory cognition.

The mental chronometry of the human brain's processing of sounds to be categorized as targets has intensively been studied in cognitive neuroscience. According to current theories, a series of successive stages consisting of the registration, identification, and categorization of the sound has to be completed before participants are able to report the sound as a target by button press after ∼300-500 ms. Here we use miniature eye movements as a tool to study the categorization of a sound as a target or nontarget, indicating that an initial categorization is present already after 80-100 ms. During visual fixation, the rate of microsaccades, the fastest components of miniature eye movements, is transiently modulated after auditory stimulation. In two experiments, we measured microsaccade rates in human participants in an auditory three-tone oddball paradigm (including rare nontarget sounds) and observed a difference in the microsaccade rates between targets and nontargets as early as 142 ms after sound onset. This finding was replicated in a third experiment with directed saccades measured in a paradigm in which tones had to be matched to score-like visual symbols. Considering the delays introduced by (motor) signal transmission and data analysis constraints, the brain must have differentiated target from nontarget sounds as fast as 80-100 ms after sound onset in both paradigms. We suggest that predictive information processing for expected input makes higher cognitive attributes, such as a sound's identity and category, available already during early sensory processing. The measurement of eye movements is thus a promising approach to investigate hearing.

Hearing silences: human auditory processing relies on preactivation of sound-specific brain activity patterns.

The remarkable capabilities displayed by humans in making sense of an overwhelming amount of sensory information cannot be explained easily if perception is viewed as a passive process. Current theoretical and computational models assume that to achieve meaningful and coherent perception, the human brain must anticipate upcoming stimulation. But how are upcoming stimuli predicted in the brain? We unmasked the neural representation of a prediction by omitting the predicted sensory input. Electrophysiological brain signals showed that when a clear prediction can be formulated, the brain activates a template of its response to the predicted stimulus before it arrives to our senses.

Temporal regularity facilitates higher-order sensory predictions in fast auditory sequences.

Does temporal regularity facilitate prediction in audition? To test this, we recorded human event-related potentials to frequent standard tones and infrequent pitch deviant tones, pre-attentively delivered within isochronous and anisochronous (20% onset jitter) rapid sequences. Deviant tones were repeated, either with high or low probability. Standard tone repetition sets a first-order prediction, which is violated by deviant tone onset, leading to a first-order prediction error response (Mismatch Negativity). The response to highly probable deviant repetitions is, however, attenuated relative to less probable repetitions, reflecting the formation of higher-order sensory predictions. Results show that temporal regularity is required for higher-order predictions, but does not modulate first-order prediction error responses. Inverse solution analyses (Variable Resolution Electrical Tomography; VARETA) localized the error response attenuation to posterior regions of the left superior temporal gyrus. In a control experiment with a slower stimulus rate, we found no evidence for higher-order predictions, and again no effect of temporal information on first-order prediction error. We conclude that: (i) temporal regularity facilitates the establishing of higher-order sensory predictions, i.e. 'knowing what next', in fast auditory sequences; (ii) first-order prediction error relies predominantly on stimulus feature mismatch, reflecting the adaptive fit of fast deviance detection processes.

Salient omissions-pupil dilation in response to unexpected omissions of sound and touch.

Introduction: Recent theories describe perception as an inferential process based on internal predictive models adjusted by means of prediction violations (prediction error). To study and demonstrate predictive processing in the brain the use of unexpected stimulus omissions has been suggested as a promising approach as the evoked brain responses are uncontaminated by responses to stimuli. Here, we aimed to investigate the pupil's response to unexpected stimulus omissions in order to better understand surprise and orienting of attention resulting from prediction violation. So far only few studies have used omission in pupillometry research and results have been inconsistent. Methods: This study adapted an EEG paradigm that has been shown to elicit omission responses in auditory and somatosensory modalities. Healthy adults pressed a button at their own pace, which resulted in the presentation of sounds or tactile stimuli in either 88%, 50% or 0% (motor-control) of cases. Pupil size was recorded continuously and averaged to analyze the pupil dilation response associated with each condition. Results: Results revealed that omission responses were observed in both modalities in the 88%-condition compared to motor-control. Similar pupil omission responses were observed between modalities, suggesting modality-unspecific activation of the underlying brain circuits. Discussion: In combination with previous omission studies using EEG, the findings demonstrate predictive models in brain processing and point to the involvement of subcortical structures in the omission response. Our pupillometry approach is especially suitable to study sensory prediction in vulnerable populations within the psychiatric field.

On the relationship of arousal and attentional distraction by emotional novel sounds.

Unexpected and task-irrelevant sounds can impair performance in a task. It has been shown that highly arousing emotional distractor sounds impaired performance less compared to moderately arousing neutral distractor sounds. The present study tests whether these differential emotion-related distraction effects are directly related to an enhancement of arousal evoked by processing of emotional distractor sounds. We disentangled costs of orienting of attention and benefits of increased arousal levels during the presentation of highly arousing emotional and moderately arousing neutral novel sounds that were embedded in a sequence of repeated standard sounds. We used sound-related pupil dilation responses as a marker of arousal and RTs as a marker of distraction in a visual categorization task in 57 healthy young adults. Multilevel analyses revealed increased RT and increased pupil dilation in response to novel vs. standard sounds. Emotional novel sounds reduced distraction effects on the behavioral level and increased pupil dilation responses compared to neutral novel sounds. Bayes Factors revealed strong evidence against an inverse proportional relationship between behavioral distraction effects and sound-related pupil dilation responses for emotional sounds. Given that the activity of the locus coeruleus has been linked to both changes in pupil diameter and arousal, it may embody an indirect relationship as a common antecedent by the release of norepinephrine into brain networks involved in attention control and control of the pupil. The present study provides new insights into the relation of changes in arousal and attentional distraction during the processing of emotional task-irrelevant novel sounds.

Intention-based predictive information modulates auditory deviance processing.

The human brain is highly responsive to (deviant) sounds violating an auditory regularity. Respective brain responses are usually investigated in situations when the sounds were produced by the experimenter. Acknowledging that humans also actively produce sounds, the present event-related potential study tested for differences in the brain responses to deviants that were produced by the listeners by pressing one of two buttons. In one condition, deviants were unpredictable with respect to the button-sound association. In another condition, deviants were predictable with high validity yielding correctly predicted deviants and incorrectly predicted (mispredicted) deviants. Temporal principal component analysis revealed deviant-specific N1 enhancement, mismatch negativity (MMN) and P3a. N1 enhancements were highly similar for each deviant type, indicating that the underlying neural mechanism is not affected by intention-based expectation about the self-produced forthcoming sound. The MMN was abolished for predictable deviants, suggesting that the intention-based prediction for a deviant can overwrite the prediction derived from the auditory regularity (predicting a standard). The P3a was present for each deviant type but was largest for mispredicted deviants. It is argued that the processes underlying P3a not only evaluate the deviant with respect to the fact that it violates an auditory regularity but also with respect to the intended sensorial effect of an action. Overall, our results specify current theories of auditory predictive processing, as they reveal that intention-based predictions exert different effects on different deviance-specific brain responses.

The effect of background speech on attentive sound processing: A pupil dilation study.

Listening to task-irrelevant speech while performing a cognitive task can involuntarily deviate our attention and lead to decreases in performance. One explanation for the impairing effect of irrelevant speech is that semantic processing can consume attentional resources. In the present study, we tested this assumption by measuring performance in a non-linguistic attentional task while participants were exposed to meaningful (native) and non-meaningful (foreign) speech. Moreover, based on the tight relation between pupillometry and attentional processes, we also registered changes in pupil diameter size to quantify the effect of meaningfulness upon attentional allocation. To these aims, we recruited 41 native German speakers who had neither received formal instruction in French nor had extensive informal contact with this language. The focal task consisted of an auditory oddball task. Participants performed a duration discrimination task containing frequently repeated standard sounds and rarely presented deviant sounds while a story was read in German or (non-meaningful) French in the background. Our results revealed that, whereas effects of language meaningfulness on attention were not detectable at the behavioural level, participants' pupil dilated more in response to the sounds of the auditory task when background speech was played in non-meaningful French compared to German, independent of sound type. In line with the initial hypothesis, this suggested that semantic processing of the native language required attentional resources, which lead to fewer resources devoted to the processing of the sounds of the focal task. Our results highlight the potential of the pupil dilation response for the investigation of subtle cognitive processes that might not surface when only behaviour is measured.

A tutorial on the use of temporal principal component analysis in developmental ERP research - Opportunities and challenges.

Developmental researchers are often interested in event-related potentials (ERPs). Data-analytic approaches based on the observed ERP suffer from major problems such as arbitrary definition of analysis time windows and regions of interest and the observed ERP being a mixture of latent underlying components. Temporal principal component analysis (PCA) can reduce these problems. However, its application in developmental research comes with the unique challenge that the component structure differs between age groups (so-called measurement non-invariance). Separate PCAs for the groups can cope with this challenge. We demonstrate how to make results from separate PCAs accessible for inferential statistics by re-scaling to original units. This tutorial enables readers with a focus on developmental research to conduct a PCA-based ERP analysis of amplitude differences. We explain the benefits of a PCA-based approach, introduce the PCA model and demonstrate its application to a developmental research question using real-data from a child and an adult group (code and data openly available). Finally, we discuss how to cope with typical challenges during the analysis and name potential limitations such as suboptimal decomposition results, data-driven analysis decisions and latency shifts.

Sound omission related brain responses in children.

Action is an important way for children to learn about the world. Recent theories suggest that action is inherently accompanied by the sensory prediction of its effects. Such predictions can be revealed by rarely omitting the expected sensory consequence of the action, resulting in an omission response that is observable in the EEG. Although prediction errors play an important role in models of learning and development, little is known about omission-related brain responses in children. This study used a motor-auditory omission paradigm, testing a group of 6-8-year-old children and an adult group (N = 31 each). In an identity-specific condition, the sound coupled to the motor action was predictable, while in an identity unspecific condition the sound was unpredictable. Results of a temporal principal component analysis revealed that sound-related brain responses underlying the N1-complex differed considerably between age groups. Despite these developmental differences, omission responses (oN1) were similar between age groups. Two subcomponents of the oN1 were differently affected by specific and unspecific predictions. Results demonstrate that children, independent from the maturation of sound processing mechanisms, can implement specific and unspecific predictions as flexibly as adults. This supports theories that regard action and prediction error as important drivers of cognitive development.