Continuous decision to wait for a future reward is guided by fronto-hippocampal anticipatory dynamics.

Deciding whether to wait for a future reward is crucial for surviving in an uncertain world. While seeking rewards, agents anticipate a reward in the present environment and constantly face a trade-off between staying in their environment or leaving it. It remains unclear, however, how humans make continuous decisions in such situations. Here, we show that anticipatory activity in the anterior prefrontal cortex, ventrolateral prefrontal cortex, and hippocampus underpins continuous stay-leave decision-making. Participants awaited real liquid rewards available after tens of seconds, and their continuous decision was tracked by dynamic brain activity associated with the anticipation of a reward. Participants stopped waiting more frequently and sooner after they experienced longer delays and received smaller rewards. When the dynamic anticipatory brain activity was enhanced in the anterior prefrontal cortex, participants remained in their current environment, but when this activity diminished, they left the environment. Moreover, while experiencing a delayed reward in a novel environment, the ventrolateral prefrontal cortex and hippocampus showed anticipatory activity. Finally, the activity in the anterior prefrontal cortex and ventrolateral prefrontal cortex was enhanced in participants adopting a leave strategy, whereas those remaining stationary showed enhanced hippocampal activity. Our results suggest that fronto-hippocampal anticipatory dynamics underlie continuous decision-making while anticipating a future reward.

Evaluation of the Hierarchical Correspondence between the Human Brain and Artificial Neural Networks: A Review.

Artificial neural networks (ANNs) that are heavily inspired by the human brain now achieve human-level performance across multiple task domains. ANNs have thus drawn attention in neuroscience, raising the possibility of providing a framework for understanding the information encoded in the human brain. However, the correspondence between ANNs and the brain cannot be measured directly. They differ in outputs and substrates, neurons vastly outnumber their ANN analogs (i.e., nodes), and the key algorithm responsible for most of modern ANN training (i.e., backpropagation) is likely absent from the brain. Neuroscientists have thus taken a variety of approaches to examine the similarity between the brain and ANNs at multiple levels of their information hierarchy. This review provides an overview of the currently available approaches and their limitations for evaluating brain-ANN correspondence.

A clearing in the objectivity of aesthetics?

As subjective experiences go, beauty matters. Although aesthetics has long been a topic of study, research in this area has not resulted in a level of interest and progress commensurate with its import. Here, we briefly discuss two recent advances, one computational and one neuroscientific, and their pertinence to aesthetic processing. First, we hypothesize that deep neural networks provide the capacity to model representations essential to aesthetic experiences. Second, we highlight the principal gradient as an axis of information processing that is potentially key to examining where and how aesthetic processing takes place in the brain. In concert with established neuroimaging tools, we suggest that these advances may cultivate a new frontier in the understanding of our aesthetic experiences.

Cognitive Dynamics Estimation: A whole-brain spatial regression paradigm for extracting the temporal dynamics of cognitive processes.

Functional MRI (fMRI) has been instrumental in understanding how cognitive processes are spatially mapped in the brain, yielding insights about brain regions and functions. However, in case the orthogonality of behavioral or stimulus timing is not guaranteed, the estimated brain maps fail to dissociate each cognitive process, and the resultant maps become unstable. Also, the brain mapping exercise can not provide temporal information on the cognitive process. Here we propose a qualitatively different approach to fMRI analysis, named Cognitive Dynamics Estimation (CDE), that estimates how multiple cognitive processes change over time even when behavior or stimulus logs are unavailable. This method transposes the conventional brain mapping; the brain activity pattern at each time point is subject to regression analysis with data-driven maps of cognitive processes as regressors, resulting in the time series of cognitive processes. The estimated time series captured the fluctuation of intensity and timing of cognitive processes on a trial-by-trial basis, which conventional analysis could not capture. Notably, the estimated time series predicted participants' cognitive ability to perform each psychological task. As an addition to our fMRI analytic toolkit, these results suggest the potential for CDE to elucidate underexplored cognitive phenomena, especially in the temporal domain.

Perceptual Uncertainty Alternates Top-down and Bottom-up Fronto-Temporal Network Signaling during Response Inhibition.

Response inhibition is a primary executive control function that allows the withholding of inappropriate responses, and requires appropriate perception of the external environment to achieve a behavioral goal. It remains unclear, however, how response inhibition is achieved when goal-relevant information involves perceptual uncertainty. Twenty-six human participants of both sexes performed a go/no-go task where visually presented random-dot motion stimuli involved perceptual uncertainties. The right inferior frontal cortex (rIFC) was involved in response inhibition, and the middle temporal (MT) region showed greater activity when dot motions involved less uncertainty. A neocortical temporal region in the superior temporal sulcus (STS) specifically showed greater activity during response inhibition in more perceptually certain trials. In this STS region, activity was greater when response inhibition was successful than when it failed. Directional effective connectivity analysis revealed that, in more coherent trials, the MT and STS regions showed enhanced connectivity to the rIFC, whereas in less coherent trials, the signal direction was reversed. These results suggest that a reversible fronto-temporal functional network guides response inhibition and perceptual decision-making under perceptual uncertainty, and in this network, perceptual information in the MT is converted to control information in the rIFC via STS, enabling achievement of response inhibition.SIGNIFICANCE STATEMENT Response inhibition refers to withholding inappropriate behavior and is important for achieving goals. Often, however, decision must be made based on limited environmental evidence. We showed that successful response inhibition is guided by a neocortical temporal region that plays a hub role in converting perceived information coded in a posterior temporal region to control information coded in the PFC. Interestingly, when a perceived stimulus becomes more uncertain, the PFC supplements stimulus encoding in the temporal regions. Our results highlight fronto-temporal mechanisms of response inhibition in which conversion of stimulus-control information is regulated based on the uncertainty of environmental evidence.

Coexistence of sensory qualities and value representations in human orbitofrontal cortex.

Despite the multiple regions and neural networks associated with value-based decision-making, the orbitofrontal cortex (OFC) is possible a particularly important one. Although the role of the OFC in reinforcer devaluation tasks, which assess the ability to represent identity, sensory qualities, and subjective values of the expected outcomes, has been established, the specific aspect represented in this area remains unclear. In this study, using functional magnetic resonance imaging, wherein participants rated the palatability of 128 food items using photographs, we investigated whether the human OFC represents object identity, sensory qualities, or value. Employing many items helped us dissociate object identity from sensory qualities and values; the inferred sensory qualities of identical items were manipulated by a change in metabolic state. Moreover, value differences between items were analytically controlled by employing a technique similar to age adjustment. The palatability ratings for food items significantly decreased after a meal. Using representational similarity analysis, we confirmed that the OFC represents value. Moreover, identical items were represented similarly in the lateral OFC in a given metabolic state; however, these representations were altered post-feeding. Importantly, this change was not explained by subjective value, suggesting that the OFC represents sensory quality and value, but not object identity.

On co-activation pattern analysis and non-stationarity of resting brain activity.

The non-stationarity of resting-state brain activity has received increasing attention in recent years. Functional connectivity (FC) analysis with short sliding windows and coactivation pattern (CAP) analysis are two widely used methods for assessing the dynamic characteristics of brain activity observed with functional magnetic resonance imaging (fMRI). However, the statistical nature of the dynamics captured by these techniques needs to be verified. In this study, we found that the results of CAP analysis were similar for real fMRI data and simulated stationary data with matching covariance structures and spectral contents. We also found that, for both the real and simulated data, CAPs were clustered into spatially heterogeneous modules. Moreover, for each of the modules in the real data, a spatially similar module was found in the simulated data. The present results suggest that care needs to be taken when interpreting observations drawn from CAP analysis as it does not necessarily reflect non-stationarity or a mixture of states in resting brain activity.

Reversible Fronto-occipitotemporal Signaling Complements Task Encoding and Switching under Ambiguous Cues.

Adaptation to changing environments involves the appropriate extraction of environmental information to achieve a behavioral goal. It remains unclear how behavioral flexibility is guided under situations where the relevant behavior is ambiguous. Using functional brain mapping of machine learning decoders and directional functional connectivity, we show that brain-wide reversible neural signaling underpins task encoding and behavioral flexibility in ambiguously changing environments. When relevant behavior is cued ambiguously during behavioral shifting, neural coding is attenuated in distributed cortical regions, but top-down signals from the prefrontal cortex complement the coding. When behavioral shifting is cued more explicitly, modality-specialized occipitotemporal regions implement distinct neural coding about relevant behavior, and bottom-up signals from the occipitotemporal region to the prefrontal cortex supplement the behavioral shift. These results suggest that our adaptation to an ever-changing world is orchestrated by the alternation of top-down and bottom-up signaling in the fronto-occipitotemporal circuit depending on the availability of environmental information.

Distillation of Regional Activity Reveals Hidden Content of Neural Information in Visual Processing.

Multivoxel pattern analysis (MVPA) has become a standard tool for decoding mental states from brain activity patterns. Recent studies have demonstrated that MVPA can be applied to decode activity patterns of a certain region from those of the other regions. By applying a similar region-to-region decoding technique, we examined whether the information represented in the visual areas can be explained by those represented in the other visual areas. We first predicted the brain activity patterns of an area on the visual pathway from the others, then subtracted the predicted patterns from their originals. Subsequently, the visual features were derived from these residuals. During the visual perception task, the elimination of the top-down signals enhanced the simple visual features represented in the early visual cortices. By contrast, the elimination of the bottom-up signals enhanced the complex visual features represented in the higher visual cortices. The directions of such modulation effects varied across visual perception/imagery tasks, indicating that the information flow across the visual cortices is dynamically altered, reflecting the contents of visual processing. These results demonstrated that the distillation approach is a useful tool to estimate the hidden content of information conveyed across brain regions.

Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network.

Deep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the non-linearity of DNNs, it is generally difficult to explain how and why they assign certain behavioral tasks to given brain activations, either correctly or incorrectly. One of the promising approaches for explaining such a black-box system is counterfactual explanation. In this framework, the behavior of a black-box system is explained by comparing real data and realistic synthetic data that are specifically generated such that the black-box system outputs an unreal outcome. The explanation of the system's decision can be explained by directly comparing the real and synthetic data. Recently, by taking advantage of advances in DNN-based image-to-image translation, several studies successfully applied counterfactual explanation to image domains. In principle, the same approach could be used in functional magnetic resonance imaging (fMRI) data. Because fMRI datasets often contain multiple classes (e.g., multiple behavioral tasks), the image-to-image transformation applicable to counterfactual explanation needs to learn mapping among multiple classes simultaneously. Recently, a new generative neural network (StarGAN) that enables image-to-image transformation among multiple classes has been developed. By adapting StarGAN with some modifications, here, we introduce a novel generative DNN (counterfactual activation generator, CAG) that can provide counterfactual explanations for DNN-based classifiers of brain activations. Importantly, CAG can simultaneously handle image transformation among all the seven classes in a publicly available fMRI dataset. Thus, CAG could provide a counterfactual explanation of DNN-based multiclass classifiers of brain activations. Furthermore, iterative applications of CAG were able to enhance and extract subtle spatial brain activity patterns that affected the classifier's decisions. Together, these results demonstrate that the counterfactual explanation based on image-to-image transformation would be a promising approach to understand and extend the current application of DNNs in fMRI analyses.

Emotional Objectivity: Neural Representations of Emotions and Their Interaction with Cognition.

Recent advances in our understanding of information states in the human brain have opened a new window into the brain's representation of emotion. While emotion was once thought to constitute a separate domain from cognition, current evidence suggests that all events are filtered through the lens of whether they are good or bad for us. Focusing on new methods of decoding information states from brain activation, we review growing evidence that emotion is represented at multiple levels of our sensory systems and infuses perception, attention, learning, and memory. We provide evidence that the primary function of emotional representations is to produce unified emotion, perception, and thought (e.g., "That is a good thing") rather than discrete and isolated psychological events (e.g., "That is a thing. I feel good"). The emergent view suggests ways in which emotion operates as a fundamental feature of cognition, by design ensuring that emotional outcomes are the central object of perception, thought, and action.

The neural representation of facial-emotion categories reflects conceptual structure.

Humans reliably categorize configurations of facial actions into specific emotion categories, leading some to argue that this process is invariant between individuals and cultures. However, growing behavioral evidence suggests that factors such as emotion-concept knowledge may shape the way emotions are visually perceived, leading to variability-rather than universality-in facial-emotion perception. Understanding variability in emotion perception is only emerging, and the neural basis of any impact from the structure of emotion-concept knowledge remains unknown. In a neuroimaging study, we used a representational similarity analysis (RSA) approach to measure the correspondence between the conceptual, perceptual, and neural representational structures of the six emotion categories Anger, Disgust, Fear, Happiness, Sadness, and Surprise. We found that subjects exhibited individual differences in their conceptual structure of emotions, which predicted their own unique perceptual structure. When viewing faces, the representational structure of multivoxel patterns in the right fusiform gyrus was significantly predicted by a subject's unique conceptual structure, even when controlling for potential physical similarity in the faces themselves. Finally, cross-cultural differences in emotion perception were also observed, which could be explained by individual differences in conceptual structure. Our results suggest that the representational structure of emotion expressions in visual face-processing regions may be shaped by idiosyncratic conceptual understanding of emotion categories.

Distinct representations of basic taste qualities in human gustatory cortex.

The mammalian tongue contains gustatory receptors tuned to basic taste types, providing an evolutionarily old hedonic compass for what and what not to ingest. Although representation of these distinct taste types is a defining feature of primary gustatory cortex in other animals, their identification has remained elusive in humans, leaving the demarcation of human gustatory cortex unclear. Here we used distributed multivoxel activity patterns to identify regions with patterns of activity differentially sensitive to sweet, salty, bitter, and sour taste qualities. These were found in the insula and overlying operculum, with regions in the anterior and middle insula discriminating all tastes and representing their combinatorial coding. These findings replicated at super-high 7 T field strength using different compounds of sweet and bitter taste types, suggesting taste sensation specificity rather than chemical or receptor specificity. Our results provide evidence of the human gustatory cortex in the insula.

Representation of affect in sensory cortex.

Contemporary neuroscience suggests that perception is perhaps best understood as a dynamically iterative process that does not honor cleanly segregated "bottom-up" or "top-down" streams. We argue that there is substantial empirical support for the idea that affective influences infiltrate the earliest reaches of sensory processing and even that primitive internal affective dimensions (e.g., goodness-to-badness) are represented alongside physical dimensions of the external world.

Population coding of affect across stimuli, modalities and individuals.

It remains unclear how the brain represents external objective sensory events alongside our internal subjective impressions of them--affect. Representational mapping of population activity evoked by complex scenes and basic tastes in humans revealed a neural code supporting a continuous axis of pleasant-to-unpleasant valence. This valence code was distinct from low-level physical and high-level object properties. Although ventral temporal and anterior insular cortices supported valence codes specific to vision and taste, both the medial and lateral orbitofrontal cortices (OFC) maintained a valence code independent of sensory origin. Furthermore, only the OFC code could classify experienced affect across participants. The entire valence spectrum was represented as a collective pattern in regional neural activity as sensory-specific and abstract codes, whereby the subjective quality of affect can be objectively quantified across stimuli, modalities and people.

FMRI activity in the macaque cerebellum evoked by intracortical microstimulation of the primary somatosensory cortex: evidence for polysynaptic propagation.

Simultaneous electrical microstimulation (EM) and functional magnetic resonance imaging (fMRI) is a useful tool for probing connectivity across brain areas in vivo. However, it is not clear whether intracortical EM can evoke blood-oxygenation-level-dependent (BOLD) signal in areas connected polysynaptically to the stimulated site. To test for the presence of the BOLD activity evoked by polysynaptic propagation of the EM signal, we conducted simultaneous fMRI and EM in the primary somatosensory cortex (S1) of macaque monkeys. We in fact observed BOLD activations in the contralateral cerebellum which is connected to the stimulation site (i.e. S1) only through polysynaptic pathways. Furthermore, the magnitude of cerebellar activations was dependent on the current amplitude of the EM, confirming the EM is the cause of the cerebellar activations. These results suggest the importance of considering polysynaptic signal propagation, particularly via pathways including subcortical structures, for correctly interpreting 'functional connectivity' as assessed by simultaneous EM and fMRI.

Efficiency of go/no-go task performance implemented in the left hemisphere.

It is well known that the efficiency of response inhibition differs from person to person, but the neural mechanism that implements the efficiency is less understood. In the present fMRI study, we devised an index to evaluate the efficiency of response inhibition in the go/no-go task, and investigated the neural correlates of the efficiency of response inhibition. The human subjects who perform the go/no-go task with a shorter reaction time in go trials (Go-RT) and with a higher percentage of correct no-go trials (Nogo-PC) are thought to have the ability to conduct response inhibition more efficiently. To quantify the efficiency, we defined an efficiency index as the difference in the Nogo-PC between each subject and an ordinarily efficient subject, under the same Go-RT. An across-subject correlation analysis revealed that the brain activity in multiple regions in the left frontal and parietal cortex positively correlated with the efficiency index. Moreover, a test of hemispheric asymmetry with regard to the across-subject correlation revealed left-hemispheric dominance. The significant correlation in the left frontal and parietal regions complements the results of previous studies that used the stop-signal reaction time (SSRT), a well known index to evaluate the efficiency of response inhibition used in the stop-signal task. Our results also indicate that, although it is well known that the neural substrates for response inhibition common in a subject group exist dominantly in the right hemisphere, the neural substrates for efficiency exist dominantly in the left hemisphere.

Prediction of subsequent recognition performance using brain activity in the medial temporal lobe.

Application of multivoxel pattern analysis (MVPA) to functional magnetic resonance imaging (fMRI) data enables reconstruction and classification of cognitive status from brain activity. However, previous studies using MVPA have extracted information about cognitive status that is experienced simultaneously with fMRI scanning, but not one that will be observed after the scanning. In this study, by focusing on activity in the medial temporal lobe (MTL), we demonstrate that MVPA on fMRI data is capable of predicting subsequent recognition performance. In this experiment, six runs of fMRI signals were acquired during encoding of phonogram stimuli. In the analysis, using data acquired in runs 1-3, we first conducted MVPA-based voxel-wise search for the clusters in the MTL whose signals contained the most information about subsequent recognition performance. Next, using the fMRI signals acquired in runs 1-3 from the selected clusters, we trained a classifier function in MVPA. Finally, the trained classifier function was applied to fMRI signals acquired in runs 4-6. Consequently, we succeeded in predicting the subsequent recognition performance for stimuli studied in runs 4-6 with significant accuracy. This accurate prediction suggests that MVPA can extract information that is associated not only with concurrent cognitive status, but also with behavior in the near future.

Role for presupplementary motor area in inhibition of cognitive set interference.

Proactive interference (PI), which is formed through repetition of certain behavior and lasts for a while, needs to be inhibited in order for subsequent behavior to prevail over the antecedent one. Although the inhibitory mechanisms in the pFC have been reported that are recruited long after one behavior is updated to another, very little is known about the inhibitory mechanisms that are recruited immediately after the update. The WCST was modified in the present fMRI study such that inhibition of PI could be examined both immediately after and long after update of behavior. Use of "dual-match" stimuli allowed us to compare two types of trials where inhibition of PI was and was not required (control and release trials, respectively). Significant activation was observed in the left pre-SMA during control versus release trials. The pre-SMA activation was selective to PI inhibition required immediately after update of behavior, which exhibited marked contrast to the left anterior prefrontal activation selective to PI inhibition required long after the update. These results reveal dissociable inhibitory mechanisms in these two regions that are recruited in the different temporal contexts of the inhibitory demands imposed during performance of the task.