Selective Attention and Decision-Making Have Separable Neural Bases in Space and Time.

Attention and decision-making processes are fundamental to cognition. However, they are usually experimentally confounded, making it difficult to link neural observations to specific processes. Here we separated the effects of selective attention from the effects of decision-making on brain activity obtained from human participants (both sexes), using a two-stage task where the attended stimulus and decision were orthogonal and separated in time. Multivariate pattern analyses of multimodal neuroimaging data revealed the dynamics of perceptual and decision-related information coding through time with magnetoencephalography (MEG), through space with functional magnetic resonance imaging (fMRI), and their combination (MEG-fMRI fusion). Our MEG results showed an effect of attention before decision-making could begin, and fMRI results showed an attention effect in early visual and frontoparietal regions. Model-based MEG-fMRI fusion suggested that attention boosted stimulus information in the frontoparietal and early visual regions before decision-making was possible. Together, our results suggest that attention affects neural stimulus representations in the frontoparietal regions independent of decision-making.

Auditory change detection and visual selective attention: association between MMN and N2pc.

While the auditory and visual systems each provide distinct information to our brain, they also work together to process and prioritize input to address ever-changing conditions. Previous studies highlighted the trade-off between auditory change detection and visual selective attention; however, the relationship between them is still unclear. Here, we recorded electroencephalography signals from 106 healthy adults in three experiments. Our findings revealed a positive correlation at the population level between the amplitudes of event-related potential indices associated with auditory change detection (mismatch negativity) and visual selective attention (posterior contralateral N2) when elicited in separate tasks. This correlation persisted even when participants performed a visual task while disregarding simultaneous auditory stimuli. Interestingly, as visual attention demand increased, participants whose posterior contralateral N2 amplitude increased the most exhibited the largest reduction in mismatch negativity, suggesting a within-subject trade-off between the two processes. Taken together, our results suggest an intimate relationship and potential shared mechanism between auditory change detection and visual selective attention. We liken this to a total capacity limit that varies between individuals, which could drive correlated individual differences in auditory change detection and visual selective attention, and also within-subject competition between the two, with task-based modulation of visual attention causing within-participant decrease in auditory change detection sensitivity.

Causal Evidence for the Multiple Demand Network in Change Detection: Auditory Mismatch Magnetoencephalography across Focal Neurodegenerative Diseases.

The multiple demand (MD) system is a network of fronto-parietal brain regions active during the organization and control of diverse cognitive operations. It has been argued that this activation may be a nonspecific signal of task difficulty. However, here we provide convergent evidence for a causal role for the MD network in the "simple task" of automatic auditory change detection, through the impairment of top-down control mechanisms. We employ independent structure-function mapping, dynamic causal modeling (DCM), and frequency-resolved functional connectivity analyses of MRI and magnetoencephalography (MEG) from 75 mixed-sex human patients across four neurodegenerative syndromes [behavioral variant fronto-temporal dementia (bvFTD), nonfluent variant primary progressive aphasia (nfvPPA), posterior cortical atrophy (PCA), and Alzheimer's disease mild cognitive impairment with positive amyloid imaging (ADMCI)] and 48 age-matched controls. We show that atrophy of any MD node is sufficient to impair auditory neurophysiological response to change in frequency, location, intensity, continuity, or duration. There was no similar association with atrophy of the cingulo-opercular, salience or language networks, or with global atrophy. MD regions displayed increased functional but decreased effective connectivity as a function of neurodegeneration, suggesting partially effective compensation. Overall, we show that damage to any of the nodes of the MD network is sufficient to impair top-down control of sensation, providing a common mechanism for impaired change detection across dementia syndromes.SIGNIFICANCE STATEMENT Previous evidence for fronto-parietal networks controlling perception is largely associative and may be confounded by task difficulty. Here, we use a preattentive measure of automatic auditory change detection [mismatch negativity (MMN) magnetoencephalography (MEG)] to show that neurodegeneration in any frontal or parietal multiple demand (MD) node impairs primary auditory cortex (A1) neurophysiological response to change through top-down mechanisms. This explains why the impaired ability to respond to change is a core feature across dementias, and other conditions driven by brain network dysfunction, such as schizophrenia. It validates theoretical frameworks in which neurodegenerating networks upregulate connectivity as partially effective compensation. The significance extends beyond network science and dementia, in its construct validation of dynamic causal modeling (DCM), and human confirmation of frequency-resolved analyses of animal neurodegeneration models.

Adaptive coding of stimulus information in human frontoparietal cortex during visual classification.

The neural mechanisms of how frontal and parietal brain regions support flexible adaptation of behavior remain poorly understood. Here, we used functional magnetic resonance imaging (fMRI) and representational similarity analysis (RSA) to investigate frontoparietal representations of stimulus information during visual classification under varying task demands. Based on prior research, we predicted that increasing perceptual task difficulty should lead to adaptive changes in stimulus coding: task-relevant category information should be stronger, while task-irrelevant exemplar-level stimulus information should become weaker, reflecting a focus on the behaviorally relevant category information. Counter to our expectations, however, we found no evidence for adaptive changes in category coding. We did find weakened coding at the exemplar-level within categories however, demonstrating that task-irrelevant information is de-emphasized in frontoparietal cortex. These findings reveal adaptive coding of stimulus information at the exemplar-level, highlighting how frontoparietal regions might support behavior even under challenging conditions.

Neural Coding of Visual Objects Rapidly Reconfigures to Reflect Subtrial Shifts in Attentional Focus.

Every day, we respond to the dynamic world around us by choosing actions to meet our goals. Flexible neural populations are thought to support this process by adapting to prioritize task-relevant information, driving coding in specialized brain regions toward stimuli and actions that are currently most important. Accordingly, human fMRI shows that activity patterns in frontoparietal cortex contain more information about visual features when they are task-relevant. However, if this preferential coding drives momentary focus, for example, to solve each part of a task in turn, it must reconfigure more quickly than we can observe with fMRI. Here, we used multivariate pattern analysis of magnetoencephalography data to test for rapid reconfiguration of stimulus information when a new feature becomes relevant within a trial. Participants saw two displays on each trial. They attended to the shape of a first target then the color of a second, or vice versa, and reported the attended features at a choice display. We found evidence of preferential coding for the relevant features in both trial phases, even as participants shifted attention mid-trial, commensurate with fast subtrial reconfiguration. However, we only found this pattern of results when the stimulus displays contained multiple objects and not in a simpler task with the same structure. The data suggest that adaptive coding in humans can operate on a fast, subtrial timescale, suitable for supporting periods of momentary focus when complex tasks are broken down into simpler ones, but may not always do so.

Spatial and Feature-selective Attention Have Distinct, Interacting Effects on Population-level Tuning.

Attention can be deployed in different ways: When searching for a taxi in New York City, we can decide where to attend (e.g., to the street) and what to attend to (e.g., yellow cars). Although we use the same word to describe both processes, nonhuman primate data suggest that these produce distinct effects on neural tuning. This has been challenging to assess in humans, but here we used an opportunity afforded by multivariate decoding of MEG data. We found that attending to an object at a particular location and attending to a particular object feature produced effects that interacted multiplicatively. The two types of attention induced distinct patterns of enhancement in occipital cortex, with feature-selective attention producing relatively more enhancement of small feature differences and spatial attention producing relatively larger effects for larger feature differences. An information flow analysis further showed that stimulus representations in occipital cortex were Granger-caused by coding in frontal cortices earlier in time and that the timing of this feedback matched the onset of attention effects. The data suggest that spatial and feature-selective attention rely on distinct neural mechanisms that arise from frontal-occipital information exchange, interacting multiplicatively to selectively enhance task-relevant information.

Linking the Brain with Behavior: The Neural Dynamics of Success and Failure in Goal-directed Behavior.

The human brain is extremely flexible and capable of rapidly selecting relevant information in accordance with task goals. Regions of frontoparietal cortex flexibly represent relevant task information such as task rules and stimulus features when participants perform tasks successfully, but less is known about how information processing breaks down when participants make mistakes. This is important for understanding whether and when information coding recorded with neuroimaging is directly meaningful for behavior. Here, we used magnetoencephalography to assess the temporal dynamics of information processing and linked neural responses with goal-directed behavior by analyzing how they changed on behavioral error. Participants performed a difficult stimulus-response task using two stimulus-response mapping rules. We used time-resolved multivariate pattern analysis to characterize the progression of information coding from perceptual information about the stimulus, cue and rule coding, and finally, motor response. Response-aligned analyses revealed a ramping up of perceptual information before a correct response, suggestive of internal evidence accumulation. Strikingly, when participants made a stimulus-related error, and not when they made other types of errors, patterns of activity initially reflected the stimulus presented, but later reversed, and accumulated toward a representation of the "incorrect" stimulus. This suggests that the patterns recorded at later time points reflect an internally generated stimulus representation that was used to make the (incorrect) decision. These results illustrate the orderly and overlapping temporal dynamics of information coding in perceptual decision-making and show a clear link between neural patterns in the late stages of processing and behavior.

Perceptual difficulty modulates the direction of information flow in familiar face recognition.

Humans are fast and accurate when they recognize familiar faces. Previous neurophysiological studies have shown enhanced representations for the dichotomy of familiar vs. unfamiliar faces. As familiarity is a spectrum, however, any neural correlate should reflect graded representations for more vs. less familiar faces along the spectrum. By systematically varying familiarity across stimuli, we show a neural familiarity spectrum using electroencephalography. We then evaluated the spatiotemporal dynamics of familiar face recognition across the brain. Specifically, we developed a novel informational connectivity method to test whether peri-frontal brain areas contribute to familiar face recognition. Results showed that feed-forward flow dominates for the most familiar faces and top-down flow was only dominant when sensory evidence was insufficient to support face recognition. These results demonstrate that perceptual difficulty and the level of familiarity influence the neural representation of familiar faces and the degree to which peri-frontal neural networks contribute to familiar face recognition.

Feature-selective Attention in Frontoparietal Cortex: Multivoxel Codes Adjust to Prioritize Task-relevant Information.

Human cognition is characterized by astounding flexibility, enabling us to select appropriate information according to the objectives of our current task. A circuit of frontal and parietal brain regions, often referred to as the frontoparietal attention network or multiple-demand (MD) regions, are believed to play a fundamental role in this flexibility. There is evidence that these regions dynamically adjust their responses to selectively process information that is currently relevant for behavior, as proposed by the "adaptive coding hypothesis" [Duncan, J. An adaptive coding model of neural function in prefrontal cortex. Nature Reviews Neuroscience, 2, 820-829, 2001]. Could this provide a neural mechanism for feature-selective attention, the process by which we preferentially process one feature of a stimulus over another? We used multivariate pattern analysis of fMRI data during a perceptually challenging categorization task to investigate whether the representation of visual object features in the MD regions flexibly adjusts according to task relevance. Participants were trained to categorize visually similar novel objects along two orthogonal stimulus dimensions (length/orientation) and performed short alternating blocks in which only one of these dimensions was relevant. We found that multivoxel patterns of activation in the MD regions encoded the task-relevant distinctions more strongly than the task-irrelevant distinctions: The MD regions discriminated between stimuli of different lengths when length was relevant and between the same objects according to orientation when orientation was relevant. The data suggest a flexible neural system that adjusts its representation of visual objects to preferentially encode stimulus features that are currently relevant for behavior, providing a neural mechanism for feature-selective attention.

Multisensory coding in the multiple-demand regions: vibrotactile task information is coded in frontoparietal cortex.

At any given moment, our brains receive input from multiple senses. Successful behavior depends on our ability to prioritize the most important information and ignore the rest. A multiple-demand (MD) network of frontal and parietal regions is thought to support this process by adjusting to code information that is currently relevant (Duncan 2010). Accordingly, the network is proposed to encode a range of different types of information, including perceptual stimuli, task rules, and responses, as needed for the current cognitive operation. However, most MD research has used visual tasks, leaving limited information about whether these regions encode other sensory domains. We used multivoxel pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data to test whether the MD regions code the details of somatosensory stimuli, in addition to tactile-motor response transformation rules and button-press responses. Participants performed a stimulus-response task in which they discriminated between two possible vibrotactile frequencies and applied a stimulus-response transformation rule to generate a button-press response. For MD regions, we found significant coding of tactile stimulus, rule, and response. Primary and secondary somatosensory regions encoded the tactile stimuli and the button-press responses but did not represent task rules. Our findings provide evidence that MD regions can code nonvisual somatosensory task information, commensurate with a domain-general role in cognitive control.NEW & NOTEWORTHY How does the brain encode the breadth of information from our senses and use this to produce goal-directed behavior? A network of frontoparietal multiple-demand (MD) regions is implicated but has been studied almost exclusively in the context of visual tasks. We used multivariate pattern analysis of fMRI data to show that these regions encode tactile stimulus information, rules, and responses. This provides evidence for a domain-general role of the MD network in cognitive control.

Coding of Visual, Auditory, Rule, and Response Information in the Brain: 10 Years of Multivoxel Pattern Analysis.

How is the processing of task information organized in the brain? Many views of brain function emphasize modularity, with different regions specialized for processing different types of information. However, recent accounts also highlight flexibility, pointing especially to the highly consistent pattern of frontoparietal activation across many tasks. Although early insights from functional imaging were based on overall activation levels during different cognitive operations, in the last decade many researchers have used multivoxel pattern analyses to interrogate the representational content of activations, mapping out the brain regions that make particular stimulus, rule, or response distinctions. Here, we drew on 100 searchlight decoding analyses from 57 published papers to characterize the information coded in different brain networks. The outcome was highly structured. Visual, auditory, and motor networks predominantly (but not exclusively) coded visual, auditory, and motor information, respectively. By contrast, the frontoparietal multiple-demand network was characterized by domain generality, coding visual, auditory, motor, and rule information. The contribution of the default mode network and voxels elsewhere was minor. The data suggest a balanced picture of brain organization in which sensory and motor networks are relatively specialized for information in their own domain, whereas a specific frontoparietal network acts as a domain-general "core" with the capacity to code many different aspects of a task.

Representational dynamics of object recognition: Feedforward and feedback information flows.

Object perception involves a range of visual and cognitive processes, and is known to include both a feedfoward flow of information from early visual cortical areas to higher cortical areas, along with feedback from areas such as prefrontal cortex. Previous studies have found that low and high spatial frequency information regarding object identity may be processed over different timescales. Here we used the high temporal resolution of magnetoencephalography (MEG) combined with multivariate pattern analysis to measure information specifically related to object identity in peri-frontal and peri-occipital areas. Using stimuli closely matched in their low-level visual content, we found that activity in peri-occipital cortex could be used to decode object identity from ~80ms post stimulus onset, and activity in peri-frontal cortex could also be used to decode object identity from a later time (~265ms post stimulus onset). Low spatial frequency information related to object identity was present in the MEG signal at an earlier time than high spatial frequency information for peri-occipital cortex, but not for peri-frontal cortex. We additionally used Granger causality analysis to compare feedforward and feedback influences on representational content, and found evidence of both an early feedfoward flow and later feedback flow of information related to object identity. We discuss our findings in relation to existing theories of object processing and propose how the methods we use here could be used to address further questions of the neural substrates underlying object perception.

Flexible Coding of Task Rules in Frontoparietal Cortex: An Adaptive System for Flexible Cognitive Control.

How do our brains achieve the cognitive control that is required for flexible behavior? Several models of cognitive control propose a role for frontoparietal cortex in the structure and representation of task sets or rules. For behavior to be flexible, however, the system must also rapidly reorganize as mental focus changes. Here we used multivoxel pattern analysis of fMRI data to demonstrate adaptive reorganization of frontoparietal activity patterns following a change in the complexity of the task rules. When task rules were relatively simple, frontoparietal cortex did not hold detectable information about these rules. In contrast, when the rules were more complex, frontoparietal cortex showed clear and decodable rule discrimination. Our data demonstrate that frontoparietal activity adjusts to task complexity, with better discrimination of rules that are behaviorally more confusable. The change in coding was specific to the rule element of the task and was not mirrored in more specialized cortex (early visual cortex) where coding was independent of difficulty. In line with an adaptive view of frontoparietal function, the data suggest a system that rapidly reconfigures in accordance with the difficulty of a behavioral task. This system may provide a neural basis for the flexible control of human behavior.

A frontotemporoparietal network common to initiating and responding to joint attention bids.

Joint attention is a fundamental cognitive ability that supports daily interpersonal relationships and communication. The Parallel Distributed Processing model (PDPM) postulates that responding to (RJA) and initiating (IJA) joint attention are predominantly supported by posterior-parietal and frontal regions respectively. It also argues that these neural networks integrate during development, supporting the parallel processes of self- and other-attention representation during interactions. However, direct evidence for the PDPM is limited due to a lack of ecologically valid experimental paradigms that can capture both RJA and IJA. Building on existing interactive approaches, we developed a virtual reality paradigm where participants engaged in an online interaction to complete a cooperative task. By including tightly controlled baseline conditions to remove activity associated with non-social task demands, we were able to directly contrast the neural correlates of RJA and IJA to determine whether these processes are supported by common brain regions. Both RJA and IJA activated broad frontotemporoparietal networks. Critically, a conjunction analysis identified that a subset of these regions were common to both RJA and IJA. This right-lateralised network included the dorsal portion of the middle frontal gyrus (MFG), inferior frontal gyrus (IFG), middle temporal gyrus (MTG), precentral gyrus, posterior superior temporal sulcus (pSTS), temporoparietal junction (TPJ) and precuneus. Additional activation was observed in this network for IJA relative to RJA at MFG, IFG, TPJ and precuneus. This is the first imaging study to directly investigate the neural correlates common to RJA and IJA engagement, and thus support the assumption that a broad integrated network underlies the parallel aspects of both initiating and responding to joint attention.

Attention enhances multi-voxel representation of novel objects in frontal, parietal and visual cortices.

Selective attention is fundamental for human activity, but the details of its neural implementation remain elusive. One influential theory, the adaptive coding hypothesis (Duncan, 2001, An adaptive coding model of neural function in prefrontal cortex, Nature Reviews Neuroscience 2:820-829), proposes that single neurons in certain frontal and parietal regions dynamically adjust their responses to selectively encode relevant information. This selective representation may in turn support selective processing in more specialized brain regions such as the visual cortices. Here, we use multi-voxel decoding of functional magnetic resonance images to demonstrate selective representation of attended--and not distractor--objects in frontal, parietal, and visual cortices. In addition, we highlight a critical role for task demands in determining which brain regions exhibit selective coding. Strikingly, representation of attended objects in frontoparietal cortex was highest under conditions of high perceptual demand, when stimuli were hard to perceive and coding in early visual cortex was weak. Coding in early visual cortex varied as a function of attention and perceptual demand, while coding in higher visual areas was sensitive to the allocation of attention but robust to changes in perceptual difficulty. Consistent with high-profile reports, peripherally presented objects could also be decoded from activity at the occipital pole, a region which corresponds to the fovea. Our results emphasize the flexibility of frontoparietal and visual systems. They support the hypothesis that attention enhances the multi-voxel representation of information in the brain, and suggest that the engagement of this attentional mechanism depends critically on current task demands.

Coping with confounds in multivoxel pattern analysis: what should we do about reaction time differences? A comment on Todd, Nystrom & Cohen 2013.

Multivoxel pattern analysis (MVPA) is a sensitive and increasingly popular method for examining differences between neural activation patterns that cannot be detected using classical mass-univariate analysis. Recently, Todd et al. ("Confounds in multivariate pattern analysis: Theory and rule representation case study", 2013, NeuroImage 77: 157-165) highlighted a potential problem for these methods: high sensitivity to confounds at the level of individual participants due to the use of directionless summary statistics. Unlike traditional mass-univariate analyses where confounding activation differences in opposite directions tend to approximately average out at group level, group level MVPA results may be driven by any activation differences that can be discriminated in individual participants. In Todd et al.'s empirical data, factoring out differences in reaction time (RT) reduced a classifier's ability to distinguish patterns of activation pertaining to two task rules. This raises two significant questions for the field: to what extent have previous multivoxel discriminations in the literature been driven by RT differences, and by what methods should future studies take RT and other confounds into account? We build on the work of Todd et al. and compare two different approaches to remove the effect of RT in MVPA. We show that in our empirical data, in contrast to that of Todd et al., the effect of RT on rule decoding is negligible, and results were not affected by the specific details of RT modelling. We discuss the meaning of and sensitivity for confounds in traditional and multivoxel approaches to fMRI analysis. We observe that the increased sensitivity of MVPA comes at a price of reduced specificity, meaning that these methods in particular call for careful consideration of what differs between our conditions of interest. We conclude that the additional complexity of the experimental design, analysis and interpretation needed for MVPA is still not a reason to favour a less sensitive approach.

Global increase in task-related fronto-parietal activity after focal frontal lobe lesion.

A critical question for neuropsychology is how complex brain networks react to damage. Here, we address this question for the well-known executive control or multiple-demand (MD) system, a fronto-parietal network showing increased activity with many different kinds of cognitive demand, including standard tests of fluid intelligence. Using fMRI, we ask how focal frontal lobe damage affects MD activity during a standard fluid intelligence task. Despite poor behavioral performance, frontal patients showed increased fronto-parietal activity relative to controls. The activation difference was not accounted for by difference in IQ. Moreover, rather than specific focus on perilesional or contralesional cortex, additional recruitment was distributed throughout the MD regions and surrounding cortex and included parietal MD regions distant from the injury. The data suggest that, following local frontal lobe damage, there is a global compensatory recruitment of an adaptive and integrated fronto-parietal network.

Fluid intelligence loss linked to restricted regions of damage within frontal and parietal cortex.

Tests of fluid intelligence predict success in a wide range of cognitive activities. Much uncertainty has surrounded brain lesions producing deficits in these tests, with standard group comparisons delivering no clear result. Based on findings from functional imaging, we propose that the uncertainty of lesion data may arise from the specificity and complexity of the relevant neural circuit. Fluid intelligence tests give a characteristic pattern of activity in posterolateral frontal, dorsomedial frontal, and midparietal cortex. To test the causal role of these regions, we examined fluid intelligence in 80 patients with focal cortical lesions. Damage to each of the proposed regions predicted fluid intelligence loss, whereas damage outside these regions was not predictive. The results suggest that coarse group comparisons (e.g., frontal vs. posterior) cannot show the neural underpinnings of fluid intelligence tests. Instead, deficits reflect the extent of damage to a restricted but complex brain circuit comprising specific regions within both frontal and posterior cortex.

Fractionating distraction: How past- and future-relevant distractors influence integrated decisions.

Many everyday tasks require us to integrate information from multiple steps to make a decision. Dominant accounts of flexible cognition suggest that we are able to navigate such complex tasks by attending to each step in turn, yet few studies measure how we direct our attention to immediate and future task steps. Here, we used a two-step task to test whether participants are sensitive to information that is currently irrelevant but will be relevant in a future task step. Participants viewed two displays in sequence, each containing two superimposed moving dot clouds of different colors. Participants attended to one cued target color in each display and reported the average direction of the two target dot clouds. In a subset of trials, we presented a "decoy" distractor: the second target color appeared as the distractor in the first display. We regressed behavioral responses on the dot clouds' motion directions to track how this future-relevant "decoy" distractor influenced participants' reporting of the average target direction. We compared the influence of decoy distractors to never-relevant, recently relevant, and globally relevant distractor baselines. Across four experiments, we found that responses reflected what was immediately relevant, as well as the broader historical relevance of the distractors. However, relevance for a future task step did not reliably influence attention. We propose that attention in multistep tasks is shaped by what has been relevant in the current setting, and by the immediate demands of each task step. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Adaptive coding of task-relevant information in human frontoparietal cortex.

Frontoparietal cortex is thought to be essential for flexible behavior, but the mechanism for control remains elusive. Here, we demonstrate a potentially critical property of this cortex: its dynamic configuration for coding of task-critical information. Using multivoxel pattern analysis of human functional imaging data, we demonstrate an adaptive change in the patterns of activation coding task-relevant stimulus distinctions. When task demands made perceptual information more difficult to discriminate, frontoparietal regions showed increased coding of this information. Visual cortices showed the opposite result: a weaker representation of perceptual information in line with the physical change in the stimulus. On a longer timescale, a rebalancing of coding was also seen after practice, with a diminished representation of task rules as they became familiar. The results suggest a flexible neural system, exerting cognitive control in a wide range of tasks by adaptively representing the task features most challenging for successful goal-directed behavior.