Concurrent Encoding of Sequence Predictability and Event-Evoked Prediction Error in Unfolding Auditory Patterns.

Human listeners possess an innate capacity to discern patterns within rapidly unfolding sensory input. Core questions, guiding ongoing research, focus on the mechanisms through which these representations are acquired and whether the brain prioritizes or suppresses predictable sensory signals. Previous work, using fast auditory sequences (tone-pips presented at a rate of 20 Hz), revealed sustained response effects that appear to track the dynamic predictability of the sequence. Here, we extend the investigation to slower sequences (4 Hz), permitting the isolation of responses to individual tones. Stimuli were 50 ms tone-pips, ordered into random (RND) and regular (REG; a repeating pattern of 10 frequencies) sequences; Two timing profiles were created: in "fast" sequences, tone-pips were presented in direct succession (20 Hz); in "slow" sequences, tone-pips were separated by a 200 ms silent gap (4 Hz). Naive participants (N = 22; both sexes) passively listened to these sequences, while brain responses were recorded using magnetoencephalography (MEG). Results unveiled a heightened magnitude of sustained brain responses in REG when compared to RND patterns. This manifested from three tones after the onset of the pattern repetition, even in the context of slower sequences characterized by extended pattern durations (2,500 ms). This observation underscores the remarkable implicit sensitivity of the auditory brain to acoustic regularities. Importantly, brain responses evoked by single tones exhibited the opposite pattern-stronger responses to tones in RND than REG sequences. The demonstration of simultaneous but opposing sustained and evoked response effects reveals concurrent processes that shape the representation of unfolding auditory patterns.

Conceptual Associations Generate Sensory Predictions.

A crucial ability of the human brain is to learn and exploit probabilistic associations between stimuli to facilitate perception and behavior by predicting future events. Although studies have shown how perceptual relationships are used to predict sensory inputs, relational knowledge is often between concepts rather than percepts (e.g., we learned to associate cats with dogs, rather than specific images of cats and dogs). Here, we asked if and how sensory responses to visual input may be modulated by predictions derived from conceptual associations. To this end we exposed participants of both sexes to arbitrary word-word pairs (e.g., car-dog) repeatedly, creating an expectation of the second word, conditional on the occurrence of the first. In a subsequent session, we exposed participants to novel word-picture pairs, while measuring fMRI BOLD responses. All word-picture pairs were equally likely, but half of the pairs conformed to the previously formed conceptual (word-word) associations, whereas the other half violated this association. Results showed suppressed sensory responses throughout the ventral visual stream, including early visual cortex, to pictures that corresponded to the previously expected words compared with unexpected words. This suggests that the learned conceptual associations were used to generate sensory predictions that modulated processing of the picture stimuli. Moreover, these modulations were tuning specific, selectively suppressing neural populations tuned toward the expected input. Combined, our results suggest that recently acquired conceptual priors are generalized across domains and used by the sensory brain to generate category-specific predictions, facilitating processing of expected visual input.SIGNIFICANCE STATEMENT Perceptual predictions play a crucial role in facilitating perception and the integration of sensory information. However, little is known about whether and how the brain uses more abstract, conceptual priors to form sensory predictions. In our preregistered study, we show that priors derived from recently acquired arbitrary conceptual associations result in category-specific predictions that modulate perceptual processing throughout the ventral visual hierarchy, including early visual cortex. These results suggest that the predictive brain uses prior knowledge across various domains to modulate perception, thereby extending our understanding of the extensive role predictions play in perception.

Humans predict the forest, not the trees: statistical learning of spatiotemporal structure in visual scenes.

The human brain is capable of using statistical regularities to predict future inputs. In the real world, such inputs typically comprise a collection of objects (e.g. a forest constitutes numerous trees). The present study aimed to investigate whether perceptual anticipation relies on lower-level or higher-level information. Specifically, we examined whether the human brain anticipates each object in a scene individually or anticipates the scene as a whole. To explore this issue, we first trained participants to associate co-occurring objects within fixed spatial arrangements. Meanwhile, participants implicitly learned temporal regularities between these displays. We then tested how spatial and temporal violations of the structure modulated behavior and neural activity in the visual system using fMRI. We found that participants only showed a behavioral advantage of temporal regularities when the displays conformed to their previously learned spatial structure, demonstrating that humans form configuration-specific temporal expectations instead of predicting individual objects. Similarly, we found suppression of neural responses for temporally expected compared with temporally unexpected objects in lateral occipital cortex only when the objects were embedded within expected configurations. Overall, our findings indicate that humans form expectations about object configurations, demonstrating the prioritization of higher-level over lower-level information in temporal expectation.

Inferior temporal cortex leads prefrontal cortex in response to a violation of a learned sequence.

Primates learn statistical regularities that are embedded in visual sequences, a form of statistical learning. Single-unit recordings in macaques showed that inferior temporal (IT) neurons are sensitive to statistical regularities in visual sequences. Here, we asked whether ventrolateral prefrontal cortex (VLPFC), which is connected to IT, is also sensitive to the transition probabilities in visual sequences and whether the statistical learning signal in IT originates in VLPFC. We recorded simultaneously multiunit activity (MUA) and local field potentials (LFPs) in IT and VLPFC after monkeys were exposed to triplets of images with a fixed presentation order. In both areas, the MUA was stronger to images that violated the learned sequence (deviants) compared to the same images presented in the learned triplets. The high-gamma and beta LFP power showed an enhanced and suppressed response, respectively, to the deviants in both areas. The enhanced response was present also for the image following the deviant, suggesting a sensitivity for temporal adjacent dependencies in IT and VLPFC. The increased response to the deviant occurred later in VLPFC than in IT, suggesting that the deviant response in IT was not inherited from VLPFC. These data support predictive coding theories that propose a feedforward flow of prediction errors.

The effect of short-term training on repetition probability effects for non-face objects.

The magnitude of repetition suppression (RS) is modulated by the probability of stimulus repetitions when measured by fMRI. This repetition probability effect (P(rep)) is usually expressed in a stronger RS when the probability of repetition trials is higher when compared to blocks with less frequent repetitions. Previous studies have suggested that the P(rep) effect depends on the extensive long-term experience of participants with the stimuli. At the same time, the how short-term learning affects RS and its modulation by P(rep) remains largely unknown. To address this issue, we used fMRI and measured the RS and its modulation by P(rep) for non-face objects (cars) before and after a 10-day long perceptual learning (PL) period and for unfamiliar faces as control. The results showed a significant P(rep) effect for faces within the Fusiform Face Area (FFA) and for cars within the Lateral Occipital Complex (LO) in the pre-training fMRI measurement session. Following the PL period, participants exhibited strong improvements in the subordinate categorization of the trained stimuli. Surprisingly, the magnitude of RS did not change as a function of training, but the P(rep) effect was absent in the post-training fMRI sessions for both stimulus categories. These results suggest that the predictive processes, measured by P(rep) modulation of RS, may be modulated by the short-term perceptual learning experience.

Dampened sensory representations for expected input across the ventral visual stream.

Expectations, derived from previous experience, can help in making perception faster, more reliable and informative. A key neural signature of perceptual expectations is expectation suppression, an attenuated neural response to expected compared with unexpected stimuli. While expectation suppression has been reported using a variety of paradigms and recording methods, it remains unclear what neural modulation underlies this response attenuation. Sharpening models propose that neural populations tuned away from an expected stimulus are particularly suppressed by expectations, thereby resulting in an attenuated, but sharper population response. In contrast, dampening models suggest that neural populations tuned toward the expected stimulus are most suppressed, thus resulting in a dampened, less redundant population response. Empirical support is divided, with some studies favoring sharpening, while others support dampening. A key limitation of previous neuroimaging studies is the ability to draw inferences about neural-level modulations based on population (e.g. voxel) level signals. Indeed, recent simulations of repetition suppression showed that opposite neural modulations can lead to comparable population-level modulations. Forward models provide one solution to this inference limitation. Here, we used forward models to implement sharpening and dampening models, mapping neural modulations to voxel-level data. We show that a feature-specific gain modulation, suppressing neurons tuned toward the expected stimulus, best explains the empirical fMRI data. Thus, our results support the dampening account of expectation suppression, suggesting that expectations reduce redundancy in sensory cortex, and thereby promote updating of internal models on the basis of surprising information.

Evaluating the evidence for expectation suppression in the visual system.

Reports of expectation suppression have shaped the development of influential predictive coding-based theories of visual perception. However recent work has highlighted confounding factors that may mimic or inflate expectation suppression effects. In this review, we describe four confounds that are prevalent across experiments that tested for expectation suppression: effects of surprise, attention, stimulus repetition and adaptation, and stimulus novelty. With these confounds in mind we then critically review the evidence for expectation suppression across probabilistic cueing, statistical learning, oddball, action-outcome learning and apparent motion designs. We found evidence for expectation suppression within a specific subset of statistical learning designs that involved weeks of sequence learning prior to neural activity measurement. Across other experimental contexts, whereby stimulus appearance probabilities were learned within one or two testing sessions, there was inconsistent evidence for genuine expectation suppression. We discuss how an absence of expectation suppression could inform models of predictive processing, repetition suppression and perceptual decision-making. We also provide suggestions for designing experiments that may better test for expectation suppression in future work.

Can expectation suppression be explained by reduced attention to predictable stimuli?

The expectation-suppression effect - reduced stimulus-evoked responses to expected stimuli - is widely considered to be an empirical hallmark of reduced prediction errors in the framework of predictive coding. Here we challenge this notion by proposing that that expectation suppression could be explained by a reduced attention effect. Specifically, we argue that reduced responses to predictable stimuli can also be explained by a reduced saliency-driven allocation of attention. We base our discussion mainly on findings in the visual cortex and propose that resolving this controversy requires the assessment of qualitative differences between the ways in which attention and surprise enhance brain responses.

Dynamic Brain Responses Modulated by Precise Timing Prediction in an Opposing Process.

The brain function of prediction is fundamental for human beings to shape perceptions efficiently and successively. Through decades of effort, a valuable brain activation map has been obtained for prediction. However, much less is known about how the brain manages the prediction process over time using traditional neuropsychological paradigms. Here, we implemented an innovative paradigm for timing prediction to precisely study the temporal dynamics of neural oscillations. In the experiment recruiting 45 participants, expectation suppression was found for the overall electroencephalographic activity, consistent with previous hemodynamic studies. Notably, we found that N1 was positively associated with predictability while N2 showed a reversed relation to predictability. Furthermore, the matching prediction had a similar profile with no timing prediction, both showing an almost saturated N1 and an absence of N2. The results indicate that the N1 process showed a 'sharpening' effect for predictable inputs, while the N2 process showed a 'dampening' effect. Therefore, these two paradoxical neural effects of prediction, which have provoked wide confusion in accounting for expectation suppression, actually co-exist in the procedure of timing prediction but work in separate time windows. These findings strongly support a recently-proposed opposing process theory.

Encoding of Stimulus Probability in Macaque Inferior Temporal Cortex.

Optimal perceptual decisions require sensory signals to be combined with prior information about stimulus probability. Although several theories propose that probabilistic information about stimulus occurrence is encoded in sensory cortex, evidence from neuronal recordings has not yet fully supported this view. We recorded activity from single neurons in inferior temporal cortex (IT) while monkeys performed a task that involved discriminating degraded images of faces and fruit. The relative probability of the cue being a face versus a fruit was manipulated by a latent variable that was not revealed to the monkeys and that changed unpredictably over the course of each recording session. In addition to responding to stimulus identity (face or fruit), population responses in IT encoded the long-term stimulus probability of whether a face or a fruit stimulus was more likely to occur. Face-responsive neurons showed reduced firing rates to expected faces, an effect consistent with "expectation suppression," but expected stimuli were decoded from multivariate population signals with greater accuracy. These findings support "predictive coding" theories, whereby neural signals in the mammalian visual system actively encode and update predictions about the local sensory environment.

Distinct Mechanisms for Distractor Suppression and Target Facilitation.

It is well established that preparatory attention improves processing of task-relevant stimuli. Although it is often more important to ignore task-irrelevant stimuli, comparatively little is known about preparatory attentional mechanisms for inhibiting expected distractions. Here, we establish that distractor inhibition is not under the same top-down control as target facilitation. Using a variant of the Posner paradigm, participants were cued to either the location of a target stimulus, the location of a distractor, or were provided no predictive information. In Experiment 1, we found that participants were able to use target-relevant cues to facilitate target processing in both blocked and flexible conditions, but distractor cueing was only effective in the blocked version of the task. In Experiment 2, we replicate these findings in a larger sample and leveraged the additional statistical power to perform individual differences analyses to tease apart potential underlying mechanisms. We found no evidence for a correlation between these two types of benefit, suggesting that flexible target cueing and distractor suppression depend on distinct cognitive mechanisms. In Experiment 3, we use EEG to show that preparatory distractor suppression is associated with a diminished P1, but we found no evidence to suggest that this effect was mediated by top-down control of oscillatory activity in the alpha band (8-12 Hz). We conclude that flexible top-down mechanisms of cognitive control are specialized for target-related attention, whereas distractor suppression only emerges when the predictive information can be derived directly from experience. This is consistent with a predictive coding model of expectation suppression. SIGNIFICANCE STATEMENT: If you were told to ignore a white bear, you might find it quite difficult. Holding something in working memory is thought to automatically facilitate feature processing, even if doing so is detrimental to the current task. Despite this paradox, it is often assumed that distractor suppression is controlled via similar top-down mechanisms of attention that prepare brain areas for target enhancement. In particular, low-frequency oscillations in visual cortex appear especially well suited for gating task-irrelevant information. We describe the results of a series of studies exploring distractor suppression and challenge this popular notion. We draw on behavioral and EEG evidence to show that selective distractor suppression operates via an alternative mechanism, such as expectation suppression within a predictive coding framework.