Working memory signals in early visual cortex are present in weak and strong imagers.

It has been suggested that visual images are memorized across brief periods of time by vividly imagining them as if they were still there. In line with this, the contents of both working memory and visual imagery are known to be encoded already in early visual cortex. If these signals in early visual areas were indeed to reflect a combined imagery and memory code, one would predict them to be weaker for individuals with reduced visual imagery vividness. Here, we systematically investigated this question in two groups of participants. Strong and weak imagers were asked to remember images across brief delay periods. We were able to reliably reconstruct the memorized stimuli from early visual cortex during the delay. Importantly, in contrast to the prediction, the quality of reconstruction was equally accurate for both strong and weak imagers. The decodable information also closely reflected behavioral precision in both groups, suggesting it could contribute to behavioral performance, even in the extreme case of completely aphantasic individuals. Our data thus suggest that working memory signals in early visual cortex can be present even in the (near) absence of phenomenal imagery.

The effect of context and reason on the neural correlates of intentions.

Many studies have identified networks in parietal and prefrontal cortex that are involved in intentional action. Yet, our understanding of the way these networks are involved in intentions is still very limited. In this study, we investigate two characteristics of these processes: context- and reason-dependence of the neural states associated with intentions. We ask whether these states depend on the context a person is in and the reasons they have for choosing an action. We used a combination of functional magnetic resonance imaging (fMRI) and multivariate decoding to directly assess the context- and reason-dependency of the neural states underlying intentions. We show that action intentions can be decoded from fMRI data based on a classifier trained in the same context and with the same reason, in line with previous decoding studies. Furthermore, we found that intentions can be decoded across different reasons for choosing an action. However, decoding across different contexts was not successful. We found anecdotal to moderate evidence against context-invariant information in all regions of interest and for all conditions but one. These results suggest that the neural states associated with intentions are modulated by the context of the action.

Neural correlates of changing food choices while bypassing values.

Current theories suggest that altering choices requires value modification. To investigate this, normal-weight female participants' food choices and values were tested before and after an approach-avoidance training (AAT), while neural activity was recorded during the choice task using functional magnetic resonance imaging (fMRI). During AAT, participants consistently approached low- while avoiding high-calorie food cues. AAT facilitated low-calorie food choices, leaving food values unchanged. Instead, we observed a shift in indifference points, indicating the decreased contribution of food values in food choices. Training-induced choice shifts were associated with increased activity in the posterior cingulate cortex (PCC). In contrast, the medial PFC activity was not changed. Additionally, PCC gray matter density predicted individual differences in training-induced functional changes, suggesting anatomic predispositions to training impact. Our findings demonstrate neural mechanisms underlying choice modulation independent of valuation-related processes, which has substantial theoretical significance for decision-making frameworks and translational implications for health-related decisions resilient to value shifts.

A tightly controlled fMRI dataset for receptive field mapping in human visual cortex.

Four right-handed, healthy subjects participated in a visual stimulation experiment. Subjects were viewing a dartboard-shaped flickering checkerboard stimulus, divided into 4 rings and 12 segments, defining 48 sectors in the visual field. Local contrast in each sector was continuously varying across four levels and updated every 3 s. To maintain fixation, subjects had to respond to a stimulus at the center of the visual field. During the entire experiment, in which subjects performed 8 runs, each consisting of 100 trials, brain activity was measured with functional magnetic resonance imaging (MRI). Using a 3-T Siemens Trio MRI scanner, 220 echo-planar images were acquired in each run, with a repetition time of 1.5 s and voxel size of 3 x 3 x 3 mm. The dataset is publicly available from OpenNeuro and additionally includes region of interest maps for visual areas V1 to V4, left and right, obtained from another retinotopic mapping experiment. As such, the dataset allows for accurate mapping of receptive fields and their properties across several stages of human visual cortex.

Structural differences in adolescent brains can predict alcohol misuse.

Alcohol misuse during adolescence (AAM) has been associated with disruptive development of adolescent brains. In this longitudinal machine learning (ML) study, we could predict AAM significantly from brain structure (T1-weighted imaging and DTI) with accuracies of 73 -78% in the IMAGEN dataset (n∼1182). Our results not only show that structural differences in brain can predict AAM, but also suggests that such differences might precede AAM behavior in the data. We predicted 10 phenotypes of AAM at age 22 using brain MRI features at ages 14, 19, and 22. Binge drinking was found to be the most predictable phenotype. The most informative brain features were located in the ventricular CSF, and in white matter tracts of the corpus callosum, internal capsule, and brain stem. In the cortex, they were spread across the occipital, frontal, and temporal lobes and in the cingulate cortex. We also experimented with four different ML models and several confound control techniques. Support Vector Machine (SVM) with rbf kernel and Gradient Boosting consistently performed better than the linear models, linear SVM and Logistic Regression. Our study also demonstrates how the choice of the predicted phenotype, ML model, and confound correction technique are all crucial decisions in an explorative ML study analyzing psychiatric disorders with small effect sizes such as AAM.

The same analysis approach: Practical protection against the pitfalls of novel neuroimaging analysis methods.

Standard neuroimaging data analysis based on traditional principles of experimental design, modelling, and statistical inference is increasingly complemented by novel analysis methods, driven e.g. by machine learning methods. While these novel approaches provide new insights into neuroimaging data, they often have unexpected properties, generating a growing literature on possible pitfalls. We propose to meet this challenge by adopting a habit of systematic testing of experimental design, analysis procedures, and statistical inference. Specifically, we suggest to apply the analysis method used for experimental data also to aspects of the experimental design, simulated confounds, simulated null data, and control data. We stress the importance of keeping the analysis method the same in main and test analyses, because only this way possible confounds and unexpected properties can be reliably detected and avoided. We describe and discuss this Same Analysis Approach in detail, and demonstrate it in two worked examples using multivariate decoding. With these examples, we reveal two sources of error: A mismatch between counterbalancing (crossover designs) and cross-validation which leads to systematic below-chance accuracies, and linear decoding of a nonlinear effect, a difference in variance.

Neural Representations of Hierarchical Rule Sets: The Human Control System Represents Rules Irrespective of the Hierarchical Level to Which They Belong.

Humans use rules to organize their actions to achieve specific goals. Although simple rules that link a sensory stimulus to one response may suffice in some situations, often, the application of multiple, hierarchically organized rules is required. Recent theories suggest that progressively higher level rules are encoded along an anterior-to-posterior gradient within PFC. Although some evidence supports the existence of such a functional gradient, other studies argue for a lesser degree of specialization within PFC. We used fMRI to investigate whether rules at different hierarchical levels are represented at distinct locations in the brain or encoded by a single system. Thirty-seven male and female participants represented and applied hierarchical rule sets containing one lower-level stimulus-response rule and one higher-level selection rule. We used multivariate pattern analysis to investigate directly the representation of rules at each hierarchical level in absence of information about rules from other levels or other task-related information, thus providing a clear identification of low- and high-level rule representations. We could decode low- and high-level rules from local patterns of brain activity within a wide frontoparietal network. However, no significant difference existed between regions encoding representations of rules from both levels except for precentral gyrus, which represented only low-level rule information. Our findings show that the brain represents conditional rules regardless of their level in the explored hierarchy, so the human control system did not organize task representation according to this dimension. Our paradigm represents a promising approach to identifying critical principles that shape this control system.SIGNIFICANCE STATEMENT Several recent studies investigating the organization of the human control system propose that rules at different control levels are organized along an anterior-to-posterior gradient within PFC. In this study, we used multivariate pattern analysis to explore independently the representation of formally identical conditional rules belonging to different levels of a cognitive hierarchy and provide for the first time a clear identification of low- and high-level rule representations. We found no major spatial differences between regions encoding rules from different hierarchical levels. This suggests that the human brain does not use levels in the investigated hierarchy as a topographical organization principle to represent rules controlling our behavior. Our paradigm represents a promising approach to identifying which principles are critical.

Valid population inference for information-based imaging: From the second-level t-test to prevalence inference.

In multivariate pattern analysis of neuroimaging data, 'second-level' inference is often performed by entering classification accuracies into a t-test vs chance level across subjects. We argue that while the random-effects analysis implemented by the t-test does provide population inference if applied to activation differences, it fails to do so in the case of classification accuracy or other 'information-like' measures, because the true value of such measures can never be below chance level. This constraint changes the meaning of the population-level null hypothesis being tested, which becomes equivalent to the global null hypothesis that there is no effect in any subject in the population. Consequently, rejecting it only allows to infer that there are some subjects in which there is an information effect, but not that it generalizes, rendering it effectively equivalent to fixed-effects analysis. This statement is supported by theoretical arguments as well as simulations. We review possible alternative approaches to population inference for information-based imaging, converging on the idea that it should not target the mean, but the prevalence of the effect in the population. One method to do so, 'permutation-based information prevalence inference using the minimum statistic', is described in detail and applied to empirical data.

The point of no return in vetoing self-initiated movements.

In humans, spontaneous movements are often preceded by early brain signals. One such signal is the readiness potential (RP) that gradually arises within the last second preceding a movement. An important question is whether people are able to cancel movements after the elicitation of such RPs, and if so until which point in time. Here, subjects played a game where they tried to press a button to earn points in a challenge with a brain-computer interface (BCI) that had been trained to detect their RPs in real time and to emit stop signals. Our data suggest that subjects can still veto a movement even after the onset of the RP. Cancellation of movements was possible if stop signals occurred earlier than 200 ms before movement onset, thus constituting a point of no return.

On the interpretation of weight vectors of linear models in multivariate neuroimaging.

The increase in spatiotemporal resolution of neuroimaging devices is accompanied by a trend towards more powerful multivariate analysis methods. Often it is desired to interpret the outcome of these methods with respect to the cognitive processes under study. Here we discuss which methods allow for such interpretations, and provide guidelines for choosing an appropriate analysis for a given experimental goal: For a surgeon who needs to decide where to remove brain tissue it is most important to determine the origin of cognitive functions and associated neural processes. In contrast, when communicating with paralyzed or comatose patients via brain-computer interfaces, it is most important to accurately extract the neural processes specific to a certain mental state. These equally important but complementary objectives require different analysis methods. Determining the origin of neural processes in time or space from the parameters of a data-driven model requires what we call a forward model of the data; such a model explains how the measured data was generated from the neural sources. Examples are general linear models (GLMs). Methods for the extraction of neural information from data can be considered as backward models, as they attempt to reverse the data generating process. Examples are multivariate classifiers. Here we demonstrate that the parameters of forward models are neurophysiologically interpretable in the sense that significant nonzero weights are only observed at channels the activity of which is related to the brain process under study. In contrast, the interpretation of backward model parameters can lead to wrong conclusions regarding the spatial or temporal origin of the neural signals of interest, since significant nonzero weights may also be observed at channels the activity of which is statistically independent of the brain process under study. As a remedy for the linear case, we propose a procedure for transforming backward models into forward models. This procedure enables the neurophysiological interpretation of the parameters of linear backward models. We hope that this work raises awareness for an often encountered problem and provides a theoretical basis for conducting better interpretable multivariate neuroimaging analyses.

The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data.

The multivariate analysis of brain signals has recently sparked a great amount of interest, yet accessible and versatile tools to carry out decoding analyses are scarce. Here we introduce The Decoding Toolbox (TDT) which represents a user-friendly, powerful and flexible package for multivariate analysis of functional brain imaging data. TDT is written in Matlab and equipped with an interface to the widely used brain data analysis package SPM. The toolbox allows running fast whole-brain analyses, region-of-interest analyses and searchlight analyses, using machine learning classifiers, pattern correlation analysis, or representational similarity analysis. It offers automatic creation and visualization of diverse cross-validation schemes, feature scaling, nested parameter selection, a variety of feature selection methods, multiclass capabilities, and pattern reconstruction from classifier weights. While basic users can implement a generic analysis in one line of code, advanced users can extend the toolbox to their needs or exploit the structure to combine it with external high-performance classification toolboxes. The toolbox comes with an example data set which can be used to try out the various analysis methods. Taken together, TDT offers a promising option for researchers who want to employ multivariate analyses of brain activity patterns.

Distributed representations of rule identity and rule order in human frontal cortex and striatum.

Humans are able to flexibly devise and implement rules to reach their desired goals. For simple situations, we can use single rules, such as "if traffic light is green then cross the street." In most cases, however, more complex rule sets are required, involving the integration of multiple layers of control. Although it has been shown that prefrontal cortex is important for rule representation, it has remained unclear how the brain encodes more complex rule sets. Here, we investigate how the brain represents the order in which different parts of a rule set are evaluated. Participants had to follow compound rule sets that involved the concurrent application of two single rules in a specific order, where one of the rules always had to be evaluated first. The rules and their assigned order were independently manipulated. By applying multivariate decoding to fMRI data, we found that the identity of the current rule was encoded in a frontostriatal network involving right ventrolateral prefrontal cortex, right superior frontal gyrus, and dorsal striatum. In contrast, rule order could be decoded in the dorsal striatum and in the right premotor cortex. The nonhomogeneous distribution of information across brain areas was confirmed by follow-up analyses focused on relevant regions of interest. We argue that the brain encodes complex rule sets by "decomposing" them in their constituent features, which are represented in different brain areas, according to the aspect of information to be maintained.

Compositionality of rule representations in human prefrontal cortex.

Rules are widely used in everyday life to organize actions and thoughts in accordance with our internal goals. At the simplest level, single rules can be used to link individual sensory stimuli to their appropriate responses. However, most tasks are more complex and require the concurrent application of multiple rules. Experiments on humans and monkeys have shown the involvement of a frontoparietal network in rule representation. Yet, a fundamental issue still needs to be clarified: Is the neural representation of multiple rules compositional, that is, built on the neural representation of their simple constituent rules? Subjects were asked to remember and apply either simple or compound rules. Multivariate decoding analyses were applied to functional magnetic resonance imaging data. Both ventrolateral frontal and lateral parietal cortex were involved in compound representation. Most importantly, we were able to decode the compound rules by training classifiers only on the simple rules they were composed of. This shows that the code used to store rule information in prefrontal cortex is compositional. Compositional coding in rule representation suggests that it might be possible to decode other complex action plans by learning the neural patterns of the known composing elements.