Movement trajectories as a window into the dynamics of emerging neural representations.

The rapid transformation of sensory inputs into meaningful neural representations is critical to adaptive human behaviour. While non-invasive neuroimaging methods are the de-facto method for investigating neural representations, they remain expensive, not widely available, time-consuming, and restrictive. Here we show that movement trajectories can be used to measure emerging neural representations with fine temporal resolution. By combining online computer mouse-tracking and publicly available neuroimaging data via representational similarity analysis (RSA), we show that movement trajectories track the unfolding of stimulus- and category-wise neural representations along key dimensions of the human visual system. We demonstrate that time-resolved representational structures derived from movement trajectories overlap with those derived from M/EEG (albeit delayed) and those derived from fMRI in functionally-relevant brain areas. Our findings highlight the richness of movement trajectories and the power of the RSA framework to reveal and compare their information content, opening new avenues to better understand human perception.

Different Mechanisms for Supporting Mental Imagery and Perceptual Representations: Modulation Versus Excitation.

Recent research suggests imagery is functionally equivalent to a weak form of visual perception. Here we report evidence across five independent experiments on adults that perception and imagery are supported by fundamentally different mechanisms: Whereas perceptual representations are largely formed via increases in excitatory activity, imagery representations are largely supported by modulating nonimagined content. We developed two behavioral techniques that allowed us to first put the visual system into a state of adaptation and then probe the additivity of perception and imagery. If imagery drives similar excitatory visual activity to perception, pairing imagery with perceptual adapters should increase the state of adaptation. Whereas pairing weak perception with adapters increased measures of adaptation, pairing imagery reversed their effects. Further experiments demonstrated that these nonadditive effects were due to imagery weakening representations of nonimagined content. Together these data provide empirical evidence that the brain uses categorically different mechanisms to represent imagery and perception.

Implicit bias training can remove bias from subliminal stimuli, restoring choice divergence: A proof-of-concept study.

Subliminal information can influence our conscious life. Subliminal stimuli can influence cognitive tasks, while endogenous subliminal neural information can sway decisions before volition. Are decisions inextricably biased towards subliminal information? Or can they diverge away from subliminal biases via training? We report that implicit bias training can remove biases from subliminal sensory primes. We first show that subliminal stimuli biased an imagery-content decision task. Participants (n = 17) had to choose one of two different patterns to subsequently imagine. Subliminal primes significantly biased decisions towards imagining the primed option. Then, we trained participants (n = 7) to choose the non-primed option, via post choice feedback. This training was successful despite participants being unaware of the purpose or structure of the reward schedule. This implicit bias training persisted up to one week later. Our proof-of-concept study indicates that decisions might not always have to be biased towards non-conscious information, but instead can diverge from subliminal primes through training.

Why do imagery and perception look and feel so different?

Despite the past few decades of research providing convincing evidence of the similarities in function and neural mechanisms between imagery and perception, for most of us, the experience of the two are undeniably different, why? Here, we review and discuss the differences between imagery and perception and the possible underlying causes of these differences, from function to neural mechanisms. Specifically, we discuss the directional flow of information (top-down versus bottom-up), the differences in targeted cortical layers in primary visual cortex and possible different neural mechanisms of modulation versus excitation. For the first time in history, neuroscience is beginning to shed light on this long-held mystery of why imagery and perception look and feel so different. This article is part of the theme issue 'Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation'.

Decoding Nonconscious Thought Representations during Successful Thought Suppression.

Controlling our thoughts is central to mental well-being, and its failure is at the crux of a number of mental disorders. Paradoxically, behavioral evidence shows that thought suppression often fails. Despite the broad importance of understanding the mechanisms of thought control, little is known about the fate of neural representations of suppressed thoughts. Using fMRI, we investigated the brain areas involved in controlling visual thoughts and tracked suppressed thought representations using multivoxel pattern analysis. Participants were asked to either visualize a vegetable/fruit or suppress any visual thoughts about those objects. Surprisingly, the content (object identity) of successfully suppressed thoughts was still decodable in visual areas with algorithms trained on imagery. This suggests that visual representations of suppressed thoughts are still present despite reports that they are not. Thought generation was associated with the left hemisphere, and thought suppression was associated with right hemisphere engagement. Furthermore, general linear model analyses showed that subjective success in thought suppression was correlated with engagement of executive areas, whereas thought-suppression failure was associated with engagement of visual and memory-related areas. These results suggest that the content of suppressed thoughts exists hidden from awareness, seemingly without an individual's knowledge, providing a compelling reason why thought suppression is so ineffective. These data inform models of unconscious thought production and could be used to develop new treatment approaches to disorders involving maladaptive thoughts.

Expectation and attention increase the integration of top-down and bottom-up signals in perception through different pathways.

Perception likely results from the interplay between sensory information and top-down signals. In this electroencephalography (EEG) study, we utilised the hierarchical frequency tagging (HFT) method to examine how such integration is modulated by expectation and attention. Using intermodulation (IM) components as a measure of nonlinear signal integration, we show in three different experiments that both expectation and attention enhance integration between top-down and bottom-up signals. Based on a multispectral phase coherence (MSPC) measure, we present two direct physiological measures to demonstrate the distinct yet related mechanisms of expectation and attention, which would not have been possible using other amplitude-based measures. Our results link expectation to the modulation of descending signals and to the integration of top-down and bottom-up information at lower levels of the visual hierarchy. Meanwhile, the results link attention to the modulation of ascending signals and to the integration of information at higher levels of the visual hierarchy. These results are consistent with the predictive coding account of perception.

Measuring Thought-Control Failure: Sensory Mechanisms and Individual Differences.

The ability to control one's thoughts is crucial for attention, focus, ideation, and mental well-being. Although there is a long history of research into thought control, the inherent subjectivity of thoughts has made objective examination, and thus mechanistic understanding, difficult. Here, we report a novel method to objectively investigate thought-control success and failure by measuring the sensory strength of visual thoughts using binocular rivalry, a perceptual illusion. Across five experiments (N = 67), we found that thought-control failure may occur because of the involuntary and antithetical formation of nonreportable sensory representations during attempts at thought suppression but not during thought substitution. Notably, thought control was worse in individuals with high levels of anxiety and schizotypy but more successful in mindful individuals. Overall, our study offers insight into the underlying mechanisms of thought control and suggests that individual differences play an important role in the ability to control thoughts.

Decoding the contents and strength of imagery before volitional engagement.

Is it possible to predict the freely chosen content of voluntary imagery from prior neural signals? Here we show that the content and strength of future voluntary imagery can be decoded from activity patterns in visual and frontal areas well before participants engage in voluntary imagery. Participants freely chose which of two images to imagine. Using functional magnetic resonance (fMRI) and multi-voxel pattern analysis, we decoded imagery content as far as 11 seconds before the voluntary decision, in visual, frontal and subcortical areas. Decoding in visual areas in addition to perception-imagery generalization suggested that predictive patterns correspond to visual representations. Importantly, activity patterns in the primary visual cortex (V1) from before the decision, predicted future imagery vividness. Our results suggest that the contents and strength of mental imagery are influenced by sensory-like neural representations that emerge spontaneously before volition.

Neural markers of predictive coding under perceptual uncertainty revealed with Hierarchical Frequency Tagging.

There is a growing understanding that both top-down and bottom-up signals underlie perception. But it is not known how these signals integrate with each other and how this depends on the perceived stimuli's predictability. 'Predictive coding' theories describe this integration in terms of how well top-down predictions fit with bottom-up sensory input. Identifying neural markers for such signal integration is therefore essential for the study of perception and predictive coding theories. To achieve this, we combined EEG methods that preferentially tag different levels in the visual hierarchy. Importantly, we examined intermodulation components as a measure of integration between these signals. Our results link the different signals to core aspects of predictive coding, and suggest that top-down predictions indeed integrate with bottom-up signals in a manner that is modulated by the predictability of the sensory input, providing evidence for predictive coding and opening new avenues to studying such interactions in perception.

Semantic Wavelet-Induced Frequency-Tagging (SWIFT) Periodically Activates Category Selective Areas While Steadily Activating Early Visual Areas.

Primate visual systems process natural images in a hierarchical manner: at the early stage, neurons are tuned to local image features, while neurons in high-level areas are tuned to abstract object categories. Standard models of visual processing assume that the transition of tuning from image features to object categories emerges gradually along the visual hierarchy. Direct tests of such models remain difficult due to confounding alteration in low-level image properties when contrasting distinct object categories. When such contrast is performed in a classic functional localizer method, the desired activation in high-level visual areas is typically accompanied with activation in early visual areas. Here we used a novel image-modulation method called SWIFT (semantic wavelet-induced frequency-tagging), a variant of frequency-tagging techniques. Natural images modulated by SWIFT reveal object semantics periodically while keeping low-level properties constant. Using functional magnetic resonance imaging (fMRI), we indeed found that faces and scenes modulated with SWIFT periodically activated the prototypical category-selective areas while they elicited sustained and constant responses in early visual areas. SWIFT and the localizer were selective and specific to a similar extent in activating category-selective areas. Only SWIFT progressively activated the visual pathway from low- to high-level areas, consistent with predictions from standard hierarchical models. We confirmed these results with criterion-free methods, generalizing the validity of our approach and show that it is possible to dissociate neural activation in early and category-selective areas. Our results provide direct evidence for the hierarchical nature of the representation of visual objects along the visual stream and open up future applications of frequency-tagging methods in fMRI.

SWIFT: a novel method to track the neural correlates of recognition.

Isolating the neural correlates of object recognition and studying their fine temporal dynamics have been a great challenge in neuroscience. A major obstacle has been the difficulty to dissociate low-level feature extraction from the actual object recognition activity. Here we present a new technique called semantic wavelet-induced frequency-tagging (SWIFT), where cyclic wavelet-scrambling allowed us to isolate neural correlates of object recognition from low-level feature extraction in humans using EEG. We show that SWIFT is insensitive to unrecognized visual objects in natural images, which were presented up to 30s, but is highly selective to the recognition of the same objects after their identity has been revealed. The enhancement of object representations by top-down attention was particularly strong with SWIFT due to its selectivity for high-level representations. Finally, we determined the temporal dynamics of object representations tracked by SWIFT and found that SWIFT can follow a maximum of between 4 and 7 different object representations per second. This result is consistent with a reduction in temporal capacity processing from low to high-level brain areas.

Spatiotemporal mapping of visual attention.

The spatial distribution and the temporal dynamics of attention are well understood in isolation, but their interaction remains an open question. How does the shape of the attentional focus evolve over time? To answer this question, we measured spatiotemporal maps of endogenous and exogenous attention in humans (more than 140,000 trials in 23 subjects). We tested the visibility of a low-contrast target presented (50 ms) at different spatial distances and temporal delays from a cue in a noisy background. The cue was a non-informative salient peripheral (5°) stimulus for exogenous attention and a central arrow cue (valid 66.6%) pointing left or right for endogenous attention. As a measure of attention, we determined, for each distance and delay, the background contrast compensation required to keep performance at 75%. The spatiotemporal mapping of exogenous attention revealed a significant enhancement zone from 150 to 430 ms, extending up to 6° from the cue. Endogenous attention maps showed a peak at the cued side at 400 ms and between 8 and 10° from the cue. Modeling suggests that the data are compatible with a constant spotlight shape across time. Our results represent the first detailed spatiotemporal maps of both endogenous and exogenous attention.

Ultra-rapid categorization of fourier-spectrum equalized natural images: macaques and humans perform similarly.

BACKGROUND: Comparative studies of cognitive processes find similarities between humans and apes but also monkeys. Even high-level processes, like the ability to categorize classes of object from any natural scene under ultra-rapid time constraints, seem to be present in rhesus macaque monkeys (despite a smaller brain and the lack of language and a cultural background). An interesting and still open question concerns the degree to which the same images are treated with the same efficacy by humans and monkeys when a low level cue, the spatial frequency content, is controlled. METHODOLOGY/PRINCIPAL FINDINGS: We used a set of natural images equalized in Fourier spectrum and asked whether it is still possible to categorize them as containing an animal and at what speed. One rhesus macaque monkey performed a forced-choice saccadic task with a good accuracy (67.5% and 76% for new and familiar images respectively) although performance was lower than with non-equalized images. Importantly, the minimum reaction time was still very fast (100 ms). We compared the performances of human subjects with the same setup and the same set of (new) images. Overall mean performance of humans was also lower than with original images (64% correct) but the minimum reaction time was still short (140 ms). CONCLUSION: Performances on individual images (% correct but not reaction times) for both humans and the monkey were significantly correlated suggesting that both species use similar features to perform the task. A similar advantage for full-face images was seen for both species. The results also suggest that local low spatial frequency information could be important, a finding that fits the theory that fast categorization relies on a rapid feedforward magnocellular signal.

Marlin-1 and conventional kinesin link GABAB receptors to the cytoskeleton and regulate receptor transport.

The cytoskeleton and cytoskeletal motors play a fundamental role in neurotransmitter receptor trafficking, but proteins that link GABA(B) receptors (GABA(B)Rs) to the cytoskeleton have not been described. We recently identified Marlin-1, a protein that interacts with GABA(B)R1. Here, we explore the association of GABA(B)Rs and Marlin-1 to the cytoskeleton using a combination of biochemistry, microscopy and live cell imaging. Our results indicate that Marlin-1 is associated to microtubules and the molecular motor kinesin-I. We demonstrate that a fraction of Marlin-1 is mobile in dendrites of cultured hippocampal neurons and that mobility is microtubule-dependent. We also show that GABA(B)Rs interact robustly with kinesin-I and that intracellular membranes containing GABA(B)Rs are sensitive to treatments that disrupt a protein complex containing Marlin-1, kinesin-I and tubulin. Finally, we report that a kinesin-I mutant severely impairs receptor transport. We conclude that Marlin-1 and kinesin-1 link GABA(B)Rs to the tubulin cytoskeleton in neurons.