Decoupling of BOLD amplitude and pattern classification of orientation-selective activity in human visual cortex.

Multivariate pattern analysis (MVPA) of fMRI data has allowed the investigation of neural representations of stimuli on the basis of distributed patterns of activity within a brain region, independently from overall brain activity. For instance, several studies on early visual cortex have reported reliable MVPA decoding of the identity of a stimulus representation that was kept in working memory or internally generated, despite the fact that the overall BOLD response was low or even at baseline levels. Here we ask how it is possible that reliable stimulus information can be decoded from early visual cortex even when the overall BOLD signal remains low. We reanalyzed a data set in which human participants (N = 24) imagined or kept in working memory an oriented visual grating. We divided voxels from V1, V2, and V3 into groups based on orientation preference, and compared the time course of mean BOLD responses to preferred and non-preferred orientations with the time course of the multivariate decoding performance. Decoding accuracy related to a numerically small, but reliable univariate difference in the mean BOLD response to preferred and non-preferred stimuli. The time course of the difference in BOLD responses to preferred and non-preferred orientations was highly similar to the time course of the multivariate pattern classification accuracy. The reliability of the classification strongly correlated with the magnitude of differences in BOLD signal between preferred and non-preferred stimuli. These activity differences were small compared to the large overall BOLD modulations. This suggests that a substantial part of the task-related BOLD response to visual stimulation might not be stimulus-specific. Rather, stimulus-evoked BOLD signals in early visual cortex during a task context may be an amalgam of small stimulus-specific responses and large task-related but non-stimulus-specific responses. The latter are not evident during the maintenance or internal generation of stimulus representations, but provide an explanation of how reliable stimulus information can be decoded from early visual cortex even though its overall BOLD signal remains low.

The what and how components of cognitive control.

In daily life, people show remarkable flexibility in adapting to novel circumstances. Although there is general agreement on which brain areas are involved in cognitive flexibility, little is known about the precise representational content of these cognitive control areas in different sub-processes involved in cognitive control. In the present study, we used an adaptation approach to differentiate the brain areas selectively representing the what and the how components of cognitive control in task preparation. When selectively repeating the task goal (the what component) without repeating the stimulus-response (S-R) mapping (the how component), task goal preferential adaptation was found in the left lateral prefrontal cortex, the medial prefrontal cortex and the left posterior parietal cortex. Within these areas, task goal specific adaptation was found in the left inferior frontal gyrus, the posterior part of the left inferior parietal lobule and the precuneus. Selectively repeating the S-R mapping, by contrast, resulted in S-R mapping preferential adaptation in the bilateral pre-central gyrus extending bilaterally to the intra-parietal lobule, indicating representation of the how component in these areas. Adaptation general to both task goal and S-R mapping was only found in Broca's area extending to the inferior frontal junction, suggesting that the what and the how components of cognitive control are similarly represented in this part of the brain.

An independent contribution of colour to the aesthetic preference for paintings.

Walking around an art museum we can see how colours influence our aesthetic preferences: many great works of art would not be as impressive in grey scales. Is the beauty of colours in abstract paintings anchored to the spatial composition of the paintings, or can it be preserved even with random spatial arrangements? To test whether colour can have an independent contribution to aesthetic appreciation, we asked participants to select the preferred image among pairs of colour-manipulated versions of the same painting. We changed hue, but preserved lightness and saturation, by rotating the colour volume around the L* axis in CIELAB space. To test the influence of the spatial structure, the images of the paintings were presented: (1) in their original format, (2) spatially scrambled but preserving the colour composition, and (3) in a control condition with both colour and spatial scrambling. Relative preference as a function of hue angle was obtained for the four paintings in their original and modified forms. For the original paintings, we found that participants generally preferred the colour angles that matched the original version of the paintings. Crucially, participants preferred the same colour distributions for spatially scrambled paintings as for the original paintings. For the control condition, there were no preferred colour configurations. This suggests that the aesthetic preference of colours in our abstract paintings is not anchored to particular spatial compositions, but is at least partly preserved even when the spatial composition is destroyed. Paintings thus can contain an aesthetic component that is exclusively related to colour.

Eye Movement-Related Confounds in Neural Decoding of Visual Working Memory Representations.

A relatively new analysis technique, known as neural decoding or multivariate pattern analysis (MVPA), has become increasingly popular for cognitive neuroimaging studies over recent years. These techniques promise to uncover the representational contents of neural signals, as well as the underlying code and the dynamic profile thereof. A field in which these techniques have led to novel insights in particular is that of visual working memory (VWM). In the present study, we subjected human volunteers to a combined VWM/imagery task while recording their neural signals using magnetoencephalography (MEG). We applied multivariate decoding analyses to uncover the temporal profile underlying the neural representations of the memorized item. Analysis of gaze position however revealed that our results were contaminated by systematic eye movements, suggesting that the MEG decoding results from our originally planned analyses were confounded. In addition to the eye movement analyses, we also present the original analyses to highlight how these might have readily led to invalid conclusions. Finally, we demonstrate a potential remedy, whereby we train the decoders on a functional localizer that was specifically designed to target bottom-up sensory signals and as such avoids eye movements. We conclude by arguing for more awareness of the potentially pervasive and ubiquitous effects of eye movement-related confounds.

Shared representations for working memory and mental imagery in early visual cortex.

Early visual areas contain specific information about visual items maintained in working memory, suggesting a role for early visual cortex in more complex cognitive functions [1-4]. It is an open question, however, whether these areas also underlie the ability to internally generate images de novo (i.e., mental imagery). Research on mental imagery has to this point focused mostly on whether mental images activate early sensory areas, with mixed results [5-7]. Recent studies suggest that multivariate pattern analysis of neural activity patterns in visual regions can reveal content-specific representations during cognitive processes, even though overall activation levels are low [1-4]. Here, we used this approach [8, 9] to study item-specific activity patterns in early visual areas (V1-V3) when these items are internally generated. We could reliably decode stimulus identity from neural activity patterns in early visual cortex during both working memory and mental imagery. Crucially, these activity patterns resembled those evoked by bottom-up visual stimulation, suggesting that mental images are indeed "perception-like" in nature. These findings suggest that the visual cortex serves as a dynamic "blackboard" [10, 11] that is used during both bottom-up stimulus processing and top-down internal generation of mental content.