Eye movements explain decodability during perception and cued attention in MEG.

Eye movements are an integral part of human perception, but can induce artifacts in many magneto-encephalography (MEG) and electroencephalography (EEG) studies. For this reason, investigators try to minimize eye movements and remove these artifacts from their data using different techniques. When these artifacts are not purely random, but consistent regarding certain stimuli or conditions, the possibility arises that eye movements are actually inducing effects in the MEG signal. It remains unclear how much of an influence eye movements can have on observed effects in MEG, since most MEG studies lack a control analysis to verify whether an effect found in the MEG signal is induced by eye movements. Here, we find that we can decode stimulus location from eye movements in two different stages of a working memory match-to-sample task that encompass different areas of research typically done with MEG. This means that the observed MEG effect might be (partly) due to eye movements instead of any true neural correlate. We suggest how to check for eye movement effects in the data and make suggestions on how to minimize eye movement artifacts from occurring in the first place.

Vividness of Visual Imagery Depends on the Neural Overlap with Perception in Visual Areas.

Research into the neural correlates of individual differences in imagery vividness point to an important role of the early visual cortex. However, there is also great fluctuation of vividness within individuals, such that only looking at differences between people necessarily obscures the picture. In this study, we show that variation in moment-to-moment experienced vividness of visual imagery, within human subjects, depends on the activity of a large network of brain areas, including frontal, parietal, and visual areas. Furthermore, using a novel multivariate analysis technique, we show that the neural overlap between imagery and perception in the entire visual system correlates with experienced imagery vividness. This shows that the neural basis of imagery vividness is much more complicated than studies of individual differences seemed to suggest. SIGNIFICANCE STATEMENT: Visual imagery is the ability to visualize objects that are not in our direct line of sight: something that is important for memory, spatial reasoning, and many other tasks. It is known that the better people are at visual imagery, the better they can perform these tasks. However, the neural correlates of moment-to-moment variation in visual imagery remain unclear. In this study, we show that the more the neural response during imagery is similar to the neural response during perception, the more vivid or perception-like the imagery experience is.

Reinstatement of associative memories in early visual cortex is signaled by the hippocampus.

The cortical reinstatement hypothesis of memory retrieval posits that content-specific cortical activity at encoding is reinstated at retrieval. Evidence for cortical reinstatement was found in higher-order sensory regions, reflecting reactivation of complex object-based information. However, it remains unclear whether the same detailed sensory, feature-based information perceived during encoding is subsequently reinstated in early sensory cortex and what the role of the hippocampus is in this process. In this study, we used a combination of visual psychophysics, functional neuroimaging, multivoxel pattern analysis, and a well controlled cued recall paradigm to address this issue. We found that the visual information human participants were retrieving could be predicted by the activation patterns in early visual cortex. Importantly, this reinstatement resembled the neural pattern elicited when participants viewed the visual stimuli passively, indicating shared representations between stimulus-driven activity and memory. Furthermore, hippocampal activity covaried with the strength of stimulus-specific cortical reinstatement on a trial-by-trial level during cued recall. These findings provide evidence for reinstatement of unique associative memories in early visual cortex and suggest that the hippocampus modulates the mnemonic strength of this reinstatement.

No Evidence for Neural Overlap between Unconsciously Processed and Imagined Stimuli.

Visual representations can be generated via feedforward or feedback processes. The extent to which these processes result in overlapping representations remains unclear. Previous work has shown that imagined stimuli elicit similar representations as perceived stimuli throughout the visual cortex. However, while representations during imagery are indeed only caused by feedback processing, neural processing during perception is an interplay of both feedforward and feedback processing. This means that any representational overlap could be because of overlap in feedback processes. In the current study, we aimed to investigate this issue by characterizing the overlap between feedforward- and feedback-initiated category representations during imagined stimuli, conscious perception, and unconscious processing using fMRI in humans of either sex. While all three conditions elicited stimulus representations in left lateral occipital cortex (LOC), significant similarities were observed only between imagery and conscious perception in this area. Furthermore, connectivity analyses revealed stronger connectivity between frontal areas and left LOC during conscious perception and in imagery compared with unconscious processing. Together, these findings can be explained by the idea that long-range feedback modifies visual representations, thereby reducing representational overlap between purely feedforward- and feedback-initiated stimulus representations measured by fMRI. Neural representations influenced by feedback, either stimulus driven (perception) or purely internally driven (imagery), are, however, relatively similar.

Population codes of prior knowledge learned through environmental regularities.

How the brain makes correct inferences about its environment based on noisy and ambiguous observations is one of the fundamental questions in Neuroscience. Prior knowledge about the probability with which certain events occur in the environment plays an important role in this process. Humans are able to incorporate such prior knowledge in an efficient, Bayes optimal, way in many situations, but it remains an open question how the brain acquires and represents this prior knowledge. The long time spans over which prior knowledge is acquired make it a challenging question to investigate experimentally. In order to guide future experiments with clear empirical predictions, we used a neural network model to learn two commonly used tasks in the experimental literature (i.e. orientation classification and orientation estimation) where the prior probability of observing a certain stimulus is manipulated. We show that a population of neurons learns to correctly represent and incorporate prior knowledge, by only receiving feedback about the accuracy of their inference from trial-to-trial and without any probabilistic feedback. We identify different factors that can influence the neural responses to unexpected or expected stimuli, and find a novel mechanism that changes the activation threshold of neurons, depending on the prior probability of the encoded stimulus. In a task where estimating the exact stimulus value is important, more likely stimuli also led to denser tuning curve distributions and narrower tuning curves, allocating computational resources such that information processing is enhanced for more likely stimuli. These results can explain several different experimental findings, clarify why some contradicting observations concerning the neural responses to expected versus unexpected stimuli have been reported and pose some clear and testable predictions about the neural representation of prior knowledge that can guide future experiments.

Shared Neural Mechanisms of Visual Perception and Imagery.

For decades, the extent to which visual imagery relies on the same neural mechanisms as visual perception has been a topic of debate. Here, we review recent neuroimaging studies comparing these two forms of visual experience. Their results suggest that there is a large overlap in neural processing during perception and imagery: neural representations of imagined and perceived stimuli are similar in the visual, parietal, and frontal cortex. Furthermore, perception and imagery seem to rely on similar top-down connectivity. The most prominent difference is the absence of bottom-up processing during imagery. These findings fit well with the idea that imagery and perception rely on similar emulation or prediction processes.

Neuroimaging Findings on Amodal Completion: A Review.

Amodal completion is the phenomenon of perceiving completed objects even though physically they are partially occluded. In this review, we provide an extensive overview of the results obtained from a variety of neuroimaging studies on the neural correlates of amodal completion. We discuss whether low-level and high-level cortical areas are implicated in amodal completion; provide an overview of how amodal completion unfolds over time while dissociating feedforward, recurrent, and feedback processes; and discuss how amodal completion is represented at the neuronal level. The involvement of low-level visual areas such as V1 and V2 is not yet clear, while several high-level structures such as the lateral occipital complex and fusiform face area seem invariant to occlusion of objects and faces, respectively, and several motor areas seem to code for object permanence. The variety of results on the timing of amodal completion hints to a mixture of feedforward, recurrent, and feedback processes. We discuss whether the invisible parts of the occluded object are represented as if they were visible, contrary to a high-level representation. While plenty of questions on amodal completion remain, this review presents an overview of the neuroimaging findings reported to date, summarizes several insights from computational models, and connects research of other perceptual completion processes such as modal completion. In all, it is suggested that amodal completion is the solution to deal with various types of incomplete retinal information, and highly depends on stimulus complexity and saliency, and therefore also give rise to a variety of observed neural patterns.

Evidence for confounding eye movements under attempted fixation and active viewing in cognitive neuroscience.

Eye movements can have serious confounding effects in cognitive neuroscience experiments. Therefore, participants are commonly asked to fixate. Regardless, participants will make so-called fixational eye movements under attempted fixation, which are thought to be necessary to prevent perceptual fading. Neural changes related to these eye movements could potentially explain previously reported neural decoding and neuroimaging results under attempted fixation. In previous work, under attempted fixation and passive viewing, we found no evidence for systematic eye movements. Here, however, we show that participants' eye movements are systematic under attempted fixation when active viewing is demanded by the task. Since eye movements directly affect early visual cortex activity, commonly used for neural decoding, our findings imply alternative explanations for previously reported results in neural decoding.

Mnemonic convergence in the human hippocampus.

The ability to form associations between a multitude of events is the hallmark of episodic memory. Computational models have espoused the importance of the hippocampus as convergence zone, binding different aspects of an episode into a coherent representation, by integrating information from multiple brain regions. However, evidence for this long-held hypothesis is limited, since previous work has largely focused on representational and network properties of the hippocampus in isolation. Here we identify the hippocampus as mnemonic convergence zone, using a combination of multivariate pattern and graph-theoretical network analyses of functional magnetic resonance imaging data from humans performing an associative memory task. We observe overlap of conjunctive coding and hub-like network attributes in the hippocampus. These results provide evidence for mnemonic convergence in the hippocampus, underlying the integration of distributed information into episodic memory representations.

Spatial and temporal aspects of navigation in two neurological patients.

We present two cases (A.C. and W.J.) with navigation problems resulting from parieto-occipital right hemisphere damage. For both the cases, performance on the neuropsychological tests did not indicate specific impairments in spatial processing, despite severe subjective complaints of spatial disorientation. Various aspects of navigation were tested in a new virtual reality task, the Virtual Tübingen task. A double dissociation between spatial and temporal deficits was found; A.C. was impaired in route ordering, a temporal test, whereas W.J. was impaired in scene recognition and route continuation, which are spatial in nature. These findings offer important insights in the functional and neural architecture of navigation.