Healthy dietary choices involve prefrontal mechanisms associated with long-term reward maximization but not working memory.

Taste and health are critical factors to be considered when choosing foods. Prioritizing healthiness over tastiness requires self-control. It has also been suggested that self-control is guided by cognitive control. We then hypothesized that neural mechanisms underlying healthy food choice are associated with both self-control and cognitive control. Human participants performed a food choice task and a working memory task during functional MRI scanning. Their degree of self-control was assessed behaviorally by the value discount of delayed monetary rewards in intertemporal choice. Prioritizing healthiness in food choice was associated with greater activity in the superior, dorsolateral, and medial prefrontal cortices. Importantly, the prefrontal activity was greater in individuals with smaller delay discounting (i.e. high self-control) who preferred a delayed larger reward to an immediate smaller reward in intertemporal choice. On the other hand, working memory activity did not show a correlation with delay discounting or food choice activity, which was further supported by supplementary results that analyzed data from the Human Connectome Project. Our results suggest that the prefrontal cortex plays a critical role in healthy food choice, which requires self-control, but not working memory, for maximization of reward attainments in a remote future.

Continuous decision to wait for a future reward is guided by fronto-hippocampal anticipatory dynamics.

Deciding whether to wait for a future reward is crucial for surviving in an uncertain world. While seeking rewards, agents anticipate a reward in the present environment and constantly face a trade-off between staying in their environment or leaving it. It remains unclear, however, how humans make continuous decisions in such situations. Here, we show that anticipatory activity in the anterior prefrontal cortex, ventrolateral prefrontal cortex, and hippocampus underpins continuous stay-leave decision-making. Participants awaited real liquid rewards available after tens of seconds, and their continuous decision was tracked by dynamic brain activity associated with the anticipation of a reward. Participants stopped waiting more frequently and sooner after they experienced longer delays and received smaller rewards. When the dynamic anticipatory brain activity was enhanced in the anterior prefrontal cortex, participants remained in their current environment, but when this activity diminished, they left the environment. Moreover, while experiencing a delayed reward in a novel environment, the ventrolateral prefrontal cortex and hippocampus showed anticipatory activity. Finally, the activity in the anterior prefrontal cortex and ventrolateral prefrontal cortex was enhanced in participants adopting a leave strategy, whereas those remaining stationary showed enhanced hippocampal activity. Our results suggest that fronto-hippocampal anticipatory dynamics underlie continuous decision-making while anticipating a future reward.

Cognitive Dynamics Estimation: A whole-brain spatial regression paradigm for extracting the temporal dynamics of cognitive processes.

Functional MRI (fMRI) has been instrumental in understanding how cognitive processes are spatially mapped in the brain, yielding insights about brain regions and functions. However, in case the orthogonality of behavioral or stimulus timing is not guaranteed, the estimated brain maps fail to dissociate each cognitive process, and the resultant maps become unstable. Also, the brain mapping exercise can not provide temporal information on the cognitive process. Here we propose a qualitatively different approach to fMRI analysis, named Cognitive Dynamics Estimation (CDE), that estimates how multiple cognitive processes change over time even when behavior or stimulus logs are unavailable. This method transposes the conventional brain mapping; the brain activity pattern at each time point is subject to regression analysis with data-driven maps of cognitive processes as regressors, resulting in the time series of cognitive processes. The estimated time series captured the fluctuation of intensity and timing of cognitive processes on a trial-by-trial basis, which conventional analysis could not capture. Notably, the estimated time series predicted participants' cognitive ability to perform each psychological task. As an addition to our fMRI analytic toolkit, these results suggest the potential for CDE to elucidate underexplored cognitive phenomena, especially in the temporal domain.

Multimodal deep neural decoding reveals highly resolved spatiotemporal profile of visual object representation in humans.

Perception and categorization of objects in a visual scene are essential to grasp the surrounding situation. Recently, neural decoding schemes, such as machine learning in functional magnetic resonance imaging (fMRI), has been employed to elucidate the underlying neural mechanisms. However, it remains unclear as to how spatially distributed brain regions temporally represent visual object categories and sub-categories. One promising strategy to address this issue is neural decoding with concurrently obtained neural response data of high spatial and temporal resolution. In this study, we explored the spatial and temporal organization of visual object representations using concurrent fMRI and electroencephalography (EEG), combined with neural decoding using deep neural networks (DNNs). We hypothesized that neural decoding by multimodal neural data with DNN would show high classification performance in visual object categorization (faces or non-face objects) and sub-categorization within faces and objects. Visualization of the fMRI DNN was more sensitive than that in the univariate approach and revealed that visual categorization occurred in brain-wide regions. Interestingly, the EEG DNN valued the earlier phase of neural responses for categorization and the later phase of neural responses for sub-categorization. Combination of the two DNNs improved the classification performance for both categorization and sub-categorization compared with fMRI DNN or EEG DNN alone. These deep learning-based results demonstrate a categorization principle in which visual objects are represented in a spatially organized and coarse-to-fine manner, and provide strong evidence of the ability of multimodal deep learning to uncover spatiotemporal neural machinery in sensory processing.

The Stroop effect involves an excitatory-inhibitory fronto-cerebellar loop.

The Stroop effect is a classical, well-known behavioral phenomenon in humans that refers to robust interference between language and color information. It remains unclear, however, when the interference occurs and how it is resolved in the brain. Here we show that the Stroop effect occurs during perception of color-word stimuli and involves a cross-hemispheric, excitatory-inhibitory loop functionally connecting the lateral prefrontal cortex and cerebellum. Participants performed a Stroop task and a non-verbal control task (which we term the Swimmy task), and made a response vocally or manually. The Stroop effect involved the lateral prefrontal cortex in the left hemisphere and the cerebellum in the right hemisphere, independently of the response type; such lateralization was absent during the Swimmy task, however. Moreover, the prefrontal cortex amplified cerebellar activity, whereas the cerebellum suppressed prefrontal activity. This fronto-cerebellar loop may implement language and cognitive systems that enable goal-directed behavior during perceptual conflicts.

Perceptual Uncertainty Alternates Top-down and Bottom-up Fronto-Temporal Network Signaling during Response Inhibition.

Response inhibition is a primary executive control function that allows the withholding of inappropriate responses, and requires appropriate perception of the external environment to achieve a behavioral goal. It remains unclear, however, how response inhibition is achieved when goal-relevant information involves perceptual uncertainty. Twenty-six human participants of both sexes performed a go/no-go task where visually presented random-dot motion stimuli involved perceptual uncertainties. The right inferior frontal cortex (rIFC) was involved in response inhibition, and the middle temporal (MT) region showed greater activity when dot motions involved less uncertainty. A neocortical temporal region in the superior temporal sulcus (STS) specifically showed greater activity during response inhibition in more perceptually certain trials. In this STS region, activity was greater when response inhibition was successful than when it failed. Directional effective connectivity analysis revealed that, in more coherent trials, the MT and STS regions showed enhanced connectivity to the rIFC, whereas in less coherent trials, the signal direction was reversed. These results suggest that a reversible fronto-temporal functional network guides response inhibition and perceptual decision-making under perceptual uncertainty, and in this network, perceptual information in the MT is converted to control information in the rIFC via STS, enabling achievement of response inhibition.SIGNIFICANCE STATEMENT Response inhibition refers to withholding inappropriate behavior and is important for achieving goals. Often, however, decision must be made based on limited environmental evidence. We showed that successful response inhibition is guided by a neocortical temporal region that plays a hub role in converting perceived information coded in a posterior temporal region to control information coded in the PFC. Interestingly, when a perceived stimulus becomes more uncertain, the PFC supplements stimulus encoding in the temporal regions. Our results highlight fronto-temporal mechanisms of response inhibition in which conversion of stimulus-control information is regulated based on the uncertainty of environmental evidence.

On co-activation pattern analysis and non-stationarity of resting brain activity.

The non-stationarity of resting-state brain activity has received increasing attention in recent years. Functional connectivity (FC) analysis with short sliding windows and coactivation pattern (CAP) analysis are two widely used methods for assessing the dynamic characteristics of brain activity observed with functional magnetic resonance imaging (fMRI). However, the statistical nature of the dynamics captured by these techniques needs to be verified. In this study, we found that the results of CAP analysis were similar for real fMRI data and simulated stationary data with matching covariance structures and spectral contents. We also found that, for both the real and simulated data, CAPs were clustered into spatially heterogeneous modules. Moreover, for each of the modules in the real data, a spatially similar module was found in the simulated data. The present results suggest that care needs to be taken when interpreting observations drawn from CAP analysis as it does not necessarily reflect non-stationarity or a mixture of states in resting brain activity.

Executive control by fronto-parietal activity explains counterintuitive decision behavior in complex value-based decision-making.

In real life, humans make decisions by taking into account multiple independent factors, such as delay and probability. Cognitive psychology suggests that cognitive control mechanisms play a key role when facing such complex task conditions. However, in value-based decision-making, it still remains unclear to what extent cognitive control mechanisms become essential when the task condition is complex. In this study, we investigated decision-making behaviors and underlying neural mechanisms using a multifactor gambling task where participants simultaneously considered probability and delay. Decision-making behavior in the multifactor task was modulated by both probability and delay. The behavioral effect of probability was stronger than delay, consistent with previous studies. Furthermore, in a subset of conditions that recruited fronto-parietal activations, reaction times were paradoxically elongated despite lower probabilistic uncertainty. Notably, such a reaction time elongation did not occur in control tasks involving single factors. Meta-analysis of brain activations suggested an interpretation that the paradoxical increase of reaction time may be associated with strategy switching. Consistent with this interpretation, logistic regression analysis of the behavioral data suggested a presence of multiple decision strategies. Taken together, we found that a novel complex value-based decision-making task cause prominent activations in fronto-parietal cortex. Furthermore, we propose that these activations can be interpreted as recruitment of cognitive control system in complex situations.

A method for reconstruction of interpretable brain networks from transient synchronization in resting-state BOLD fluctuations.

Resting-state (rs) fMRI has been widely used to examine brain-wide large-scale spatiotemporal architectures, known as resting-state networks (RSNs). Recent studies have focused on the temporally evolving characteristics of RSNs, but it is unclear what temporal characteristics are reflected in the networks. To address this issue, we devised a novel method for voxel-based visualization of spatiotemporal characteristics of rs-fMRI with a time scale of tens of seconds. We first extracted clusters of dominant activity-patterns using a region-of-interest approach and then used these temporal patterns of the clusters to obtain voxel-based activation patterns related to the clusters. We found that activation patterns related to the clusters temporally evolved with a characteristic temporal structure and showed mutual temporal alternations over minutes. The voxel-based representation allowed the decoding of activation patterns of the clusters in rs-fMRI using a meta-analysis of functional activations. The activation patterns of the clusters were correlated with behavioral measures. Taken together, our analysis highlights a novel approach to examine brain activity dynamics during rest.

Reversible Fronto-occipitotemporal Signaling Complements Task Encoding and Switching under Ambiguous Cues.

Adaptation to changing environments involves the appropriate extraction of environmental information to achieve a behavioral goal. It remains unclear how behavioral flexibility is guided under situations where the relevant behavior is ambiguous. Using functional brain mapping of machine learning decoders and directional functional connectivity, we show that brain-wide reversible neural signaling underpins task encoding and behavioral flexibility in ambiguously changing environments. When relevant behavior is cued ambiguously during behavioral shifting, neural coding is attenuated in distributed cortical regions, but top-down signals from the prefrontal cortex complement the coding. When behavioral shifting is cued more explicitly, modality-specialized occipitotemporal regions implement distinct neural coding about relevant behavior, and bottom-up signals from the occipitotemporal region to the prefrontal cortex supplement the behavioral shift. These results suggest that our adaptation to an ever-changing world is orchestrated by the alternation of top-down and bottom-up signaling in the fronto-occipitotemporal circuit depending on the availability of environmental information.

Prefrontal-Striatal Mechanisms of Behavioral Impulsivity During Consumption of Delayed Real Liquid Rewards.

Intertemporal choice involves the evaluation of future rewards and reflects behavioral impulsivity. After choosing a delayed reward in an intertemporal choice, a behavioral agent waits for, receives, and then consumes the reward. The current study focused on the consumption of the delayed reward and examined the neural mechanisms of behavioral impulsivity. In humans consuming delayed real liquid rewards in an intertemporal choice, the ventral striatum (VS) showed differential activity between anterior (aVS) and posterior (pVS) regions depending on the degree of behavioral impulsivity. Additionally, impulsive individuals showed activity in the anterior prefrontal cortex (aPFC). An analysis of task-related effective connectivity based on psychophysiological interaction (PPI) revealed that PPI was robust from the aPFC to pVS, but not in the opposite direction. On the other hand, strong bidirectional PPIs were observed between the aVS and pVS, but PPIs from the pVS to aVS were enhanced in impulsive individuals. These results suggest that behavioral impulsivity is reflected in aPFC-VS mechanisms during the consumption of delayed real liquid rewards.

Cross-Hemispheric Complementary Prefrontal Mechanisms during Task Switching under Perceptual Uncertainty.

Flexible adaptation to changing environments is a representative executive control function implicated in the frontoparietal network that requires appropriate extraction of goal-relevant information through perception of the external environment. It remains unclear, however, how the flexibility is achieved under situations where goal-relevant information is uncertain. To address this issue, the current study examined neural mechanisms for task switching in which task-relevant information involved perceptual uncertainty. Twenty-eight human participants of both sexes alternated behavioral tasks in which they judged motion direction or color of visually presented colored dot stimuli that moved randomly. Task switching was associated with frontoparietal regions in the left hemisphere, and perception of ambiguous stimuli involved contralateral homologous frontoparietal regions. On the other hand, in stimulus-modality-dependent occipitotemporal regions, task coding information was increased during task switching. Effective connectivity analysis revealed that the frontal regions signaled toward the modality-dependent occipitotemporal regions when a relevant stimulus was more ambiguous, whereas the occipitotemporal regions signaled toward the frontal regions when the stimulus was more distinctive. These results suggest that complementary prefrontal mechanisms in the left and right hemispheres help to achieve a behavioral goal when the external environment involves perceptual uncertainty.SIGNIFICANCE STATEMENT In our daily life, environmental information to achieve a goal is not always certain, but we make judgments in such situations, and change our behavior accordingly. This study examined how the flexibility of behavior is achieved in a situation where goal-relevant information involves perceptual uncertainty. fMRI revealed that the lateral prefrontal cortex (PFC) in the left hemisphere is associated with behavioral flexibility, and the perception of ambiguous stimuli involves the PFC in the right hemisphere. These bilateral PFC signaled to stimulus-modality-dependent occipitotemporal regions, depending on perceptual uncertainty and the task to be performed. These top-down signals supplement task coding in the occipitotemporal regions, and highlight interhemispheric prefrontal mechanisms involved in executive control and perceptual decision-making.

Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network.

Deep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the non-linearity of DNNs, it is generally difficult to explain how and why they assign certain behavioral tasks to given brain activations, either correctly or incorrectly. One of the promising approaches for explaining such a black-box system is counterfactual explanation. In this framework, the behavior of a black-box system is explained by comparing real data and realistic synthetic data that are specifically generated such that the black-box system outputs an unreal outcome. The explanation of the system's decision can be explained by directly comparing the real and synthetic data. Recently, by taking advantage of advances in DNN-based image-to-image translation, several studies successfully applied counterfactual explanation to image domains. In principle, the same approach could be used in functional magnetic resonance imaging (fMRI) data. Because fMRI datasets often contain multiple classes (e.g., multiple behavioral tasks), the image-to-image transformation applicable to counterfactual explanation needs to learn mapping among multiple classes simultaneously. Recently, a new generative neural network (StarGAN) that enables image-to-image transformation among multiple classes has been developed. By adapting StarGAN with some modifications, here, we introduce a novel generative DNN (counterfactual activation generator, CAG) that can provide counterfactual explanations for DNN-based classifiers of brain activations. Importantly, CAG can simultaneously handle image transformation among all the seven classes in a publicly available fMRI dataset. Thus, CAG could provide a counterfactual explanation of DNN-based multiclass classifiers of brain activations. Furthermore, iterative applications of CAG were able to enhance and extract subtle spatial brain activity patterns that affected the classifier's decisions. Together, these results demonstrate that the counterfactual explanation based on image-to-image transformation would be a promising approach to understand and extend the current application of DNNs in fMRI analyses.

Self-Controlled Choice Arises from Dynamic Prefrontal Signals That Enable Future Anticipation.

Self-control allows humans the patience necessary to maximize reward attainment in the future. Yet it remains elusive when and how the preference to self-controlled choice is formed. We measured brain activity while female and male humans performed an intertemporal choice task in which they first received delayed real liquid rewards (forced-choice trial), and then made a choice between the reward options based on the experiences (free-choice trial). We found that, while subjects were awaiting an upcoming reward in the forced-choice trial, the anterior prefrontal cortex (aPFC) tracked a dynamic signal reflecting the pleasure of anticipating the future reward. Importantly, this prefrontal signal was specifically observed in self-controlled individuals, and moreover, interregional negative coupling between the prefrontal region and the ventral striatum (VS) became stronger in those individuals. During consumption of the liquid rewards, reduced ventral striatal activity predicted self-controlled choices in the subsequent free-choice trials. These results suggest that a well-coordinated prefrontal-striatal mechanism during the reward experience shapes preferences regarding the future self-controlled choice.SIGNIFICANCE STATEMENT Anticipating future desirable events is a critical mental function that guides self-controlled behavior in humans. When and how are the self-controlled choices formed in the brain? We monitored brain activity while humans awaited a real liquid reward that became available in tens of seconds. We found that the frontal polar cortex tracked temporally evolving signals reflecting the pleasure of anticipating the future reward, which was enhanced in self-controlled individuals. Our results highlight the contribution of the fronto-polar cortex to the formation of self-controlled preferences, and further suggest that future prospect in the prefrontal cortex (PFC) plays an important role in shaping future choice behavior.

Within-Subject Correlation Analysis to Detect Functional Areas Associated With Response Inhibition.

Functional areas in fMRI studies are often detected by brain-behavior correlation, calculating across-subject correlation between the behavioral index and the brain activity related to a function of interest. Within-subject correlation analysis is also employed in a single subject level, which utilizes cognitive fluctuations in a shorter time period by correlating the behavioral index with the brain activity across trials. In the present study, the within-subject analysis was applied to the stop-signal task, a standard task to probe response inhibition, where efficiency of response inhibition can be evaluated by the stop-signal reaction time (SSRT). Since the SSRT is estimated, by definition, not in a trial basis but from pooled trials, the correlation across runs was calculated between the SSRT and the brain activity related to response inhibition. The within-subject correlation revealed negative correlations in the anterior cingulate cortex and the cerebellum. Moreover, the dissociation pattern was observed in the within-subject analysis when earlier vs. later parts of the runs were analyzed: negative correlation was dominant in earlier runs, whereas positive correlation was dominant in later runs. Regions of interest analyses revealed that the negative correlation in the anterior cingulate cortex, but not in the cerebellum, was dominant in earlier runs, suggesting multiple mechanisms associated with inhibitory processes that fluctuate on a run-by-run basis. These results indicate that the within-subject analysis compliments the across-subject analysis by highlighting different aspects of cognitive/affective processes related to response inhibition.

Large-scale network integration in the human brain tracks temporal fluctuations in memory encoding performance.

Although activation/deactivation of specific brain regions has been shown to be predictive of successful memory encoding, the relationship between time-varying large-scale brain networks and fluctuations of memory encoding performance remains unclear. Here, we investigated time-varying functional connectivity patterns across the human brain in periods of 30-40 s, which have recently been implicated in various cognitive functions. During functional magnetic resonance imaging, participants performed a memory encoding task, and their performance was assessed with a subsequent surprise memory test. A graph analysis of functional connectivity patterns revealed that increased integration of the subcortical, default-mode, salience, and visual subnetworks with other subnetworks is a hallmark of successful memory encoding. Moreover, multivariate analysis using the graph metrics of integration reliably classified the brain network states into the period of high (vs. low) memory encoding performance. Our findings suggest that a diverse set of brain systems dynamically interact to support successful memory encoding.

Visual perceptual training reconfigures post-task resting-state functional connectivity with a feature-representation region.

The neural mechanisms underlying visual perceptual learning (VPL) have typically been studied by examining changes in task-related brain activation after training. However, the relationship between post-task "offline" processes and VPL remains unclear. The present study examined this question by obtaining resting-state functional magnetic resonance imaging (fMRI) scans of human brains before and after a task-fMRI session involving visual perceptual training. During the task-fMRI session, participants performed a motion coherence discrimination task in which they judged the direction of moving dots with a coherence level that varied between trials (20, 40, and 80%). We found that stimulus-induced activation increased with motion coherence in the middle temporal cortex (MT+), a feature-specific region representing visual motion. On the other hand, stimulus-induced activation decreased with motion coherence in the dorsal anterior cingulate cortex (dACC) and bilateral insula, regions involved in decision making under perceptual ambiguity. Moreover, by comparing pre-task and post-task rest periods, we revealed that resting-state functional connectivity (rs-FC) with the MT+ was significantly increased after training in widespread cortical regions including the bilateral sensorimotor and temporal cortices. In contrast, rs-FC with the MT+ was significantly decreased in subcortical regions including the thalamus and putamen. Importantly, the training-induced change in rs-FC was observed only with the MT+, but not with the dACC or insula. Thus, our findings suggest that perceptual training induces plastic changes in offline functional connectivity specifically in brain regions representing the trained visual feature, emphasising the distinct roles of feature-representation regions and decision-related regions in VPL.

Intertemporal Decision-Making Involves Prefrontal Control Mechanisms Associated with Working Memory.

Intertemporal decision-making involves simultaneous evaluation of both the magnitude and delay to reward, which may require the integrated representation and comparison of these dimensions within working memory (WM). In the current study, neural activation associated with intertemporal decision-making was directly compared with WM load-related activation. During functional magnetic resonance imaging, participants performed an intermixed series of WM trials and intertemporal decision-making trials both varying in load, with the latter in terms of choice difficulty, via options tailored to each participant's subjective value function for delayed rewards. The right anterior prefrontal cortex (aPFC) and dorsolateral prefrontal cortex (dlPFC) showed activity modulation by choice difficulty within WM-related brain regions. In aPFC, these 2 effects (WM, choice difficulty) correlated across individuals. In dlPFC, activation increased with choice difficulty primarily in patient (self-controlled) individuals, and moreover was strongest when the delayed reward was chosen on the most difficult trials. Finally, the choice-difficulty effects in dlPFC and aPFC were correlated across individuals, suggesting a functional relationship between the 2 regions. Together, these results suggest a more precise account of the relationship between WM and intertemporal decision-making that is specifically tied to choice difficulty, and involves the coordinated activation of a lateral PFC circuit supporting successful self-control.

Data for behavioral results and brain regions showing a time effect during pair-association retrieval.

The current data article provides behavioral and neuroimaging data for the research article "Relatedness-dependent rapid development of brain activity in anterior temporal cortex during pair-association retrieval" (Jimura et al., 2016) [1]. Behavioral performance is provided in a table. Fig. 2 of the article is based on this table. Brain regions showing time effect are provided in a table. A statistical activation map for the time effect is shown in Fig. 3C of the article.

Relatedness-dependent rapid development of brain activity in anterior temporal cortex during pair-association retrieval.

Functional MRI studies have revealed that the brain activity in the anterior temporal cortex during memory retrieval increases over months after memory encoding. Behavioral evidence has demonstrated that long-term memory can sometimes be consolidated more rapidly in one or two days. In the present functional MRI study, we manipulated the relatedness between paired faces to be retrieved in a pair-association task. The brain activity in the anterior temporal cortex during retrieval of paired associates increased rapidly in one day, as shown in previous studies. We found that the speed of the brain activity development was dependent on the level of semantic relatedness of paired faces. The results suggest that the semantic relatedness enhances the speed of formation of memory representation in the anterior temporal cortex.