Determining the effects of training duration on the behavioral expression of habitual control in humans: a multilaboratory investigation.

It has been suggested that there are two distinct and parallel mechanisms for controlling instrumental behavior in mammals: goal-directed actions and habits. To gain an understanding of how these two systems interact to control behavior, it is essential to characterize the mechanisms by which the balance between these systems is influenced by experience. Studies in rodents have shown that the amount of training governs the relative expression of these two systems: Behavior is goal-directed following moderate training, but the more extensively an instrumental action is trained, the more it becomes habitual. It is less clear whether humans exhibit similar training effects on the expression of goal-directed and habitual behavior, as human studies have reported contradictory findings. To tackle these contradictory findings, we formed a consortium, where four laboratories undertook a preregistered experimental induction of habits by manipulating the amount of training. There was no statistical evidence for a main effect of the amount of training on the formation and expression of habits. However, exploratory analyses suggest a moderating effect of the affective component of stress on the impact of training over habit expression. Participants who were lower in affective stress appeared to be initially goal-directed, but became habitual with increased training, whereas participants who were high in affective stress were already habitual even after moderate training, thereby manifesting insensitivity to overtraining effects. Our findings highlight the importance of the role of moderating variables such as individual differences in stress and anxiety when studying the experimental induction of habits in humans.

A novel free-operant framework enables experimental habit induction in humans.

Habits are a prominent feature of both adaptive and maladaptive behavior. Yet, despite substantial research efforts, there are currently no well-established experimental procedures for habit induction in humans. It is likely that laboratory experimental settings, as well as the session-based structure typically used in controlled experiments (also outside the lab), impose serious constraints on studying habits and other effects that are sensitive to context, motivation, and training duration and frequency. To overcome these challenges, we devised a unique real-world free-operant task structure, implemented through a novel smartphone application, whereby participants could freely enter the app (24 hours a day, 7 days a week) to win rewards. This procedure is free of typical laboratory constraints, yet well controlled. Using the canonical sensitivity to outcome devaluation criterion, we successfully demonstrated habit formation as a function of training duration, a long-standing challenge in the field. Additionally, we show a positive relationship between multiple facets of engagement/motivation and goal-directedness. We suggest that our novel paradigm can be used to study the neurobehavioral and psychological mechanism underlying habits in humans. Moreover, the real-world free-operant framework can potentially be used to examine other instrumental behavior-related questions, with greater face validity in naturalistic conditions.

A Preferential Role for Ventromedial Prefrontal Cortex in Assessing "the Value of the Whole" in Multiattribute Object Evaluation.

Everyday decision-making commonly involves assigning values to complex objects with multiple value-relevant attributes. Drawing on object recognition theories, we hypothesized two routes to multiattribute evaluation: assessing the value of the whole object based on holistic attribute configuration or summing individual attribute values. In two samples of healthy human male and female participants undergoing eye tracking and functional magnetic resonance imaging (fMRI) while evaluating novel pseudo objects, we found evidence for both forms of evaluation. Fixations to and transitions between attributes differed systematically when the value of pseudo objects was associated with individual attributes or attribute configurations. Ventromedial prefrontal cortex (vmPFC) and perirhinal cortex were engaged when configural processing was required. These results converge with our recent findings that individuals with vmPFC lesions were impaired in decisions requiring configural evaluation but not when evaluating the sum of the parts. This suggests that multiattribute decision-making engages distinct evaluation mechanisms relying on partially dissociable neural substrates, depending on the relationship between attributes and value.SIGNIFICANCE STATEMENT Decision neuroscience has only recently begun to address how multiple choice-relevant attributes are brought together during evaluation and choice among complex options. Object recognition research makes a crucial distinction between individual attribute and holistic/configural object processing, but how the brain evaluates attributes and whole objects remains unclear. Using fMRI and eye tracking, we found that the vmPFC and the perirhinal cortex contribute to value estimation specifically when value was related to whole objects, that is, predicted by the unique configuration of attributes and not when value was predicted by the sum of individual attribute values. This perspective on the interactions between subjective value and object processing mechanisms provides a novel bridge between the study of object recognition and reward-guided decision-making.

Neural encoding of food and monetary reward delivery.

Different types of rewards such as food and money can similarly drive our behavior owing to shared brain processes encoding their subjective value. However, while the value of money is abstract and needs to be learned, the value of food is rooted in the innate processing of sensory properties and nutritional utilization. Yet, the actual consumption of food and the receipt of money have never been directly contrasted in the same experiment, questioning what unique neural processes differentiate those reward types. To fill this gap, we examined the distinct and common neural responses to the delivery of food and monetary rewards during fMRI. In a novel experimental approach, we parametrically manipulated the subjective value of food and monetary rewards by modulating the quantities of administered palatable milkshake and monetary gains. The receipt of increasing amounts of milkshake and money recruited the ventral striatum and the ventromedial prefrontal cortex, previously associated with value encoding. Notably, the consumption and the subsequent evaluation of increasing quantities of milkshake relative to money revealed an extended recruitment of brain regions related to taste, somatosensory processing, and salience. Moreover, we detected a decline of reward encoding in the ventral tegmental area, nucleus accumbens, and vmPFC, indicating that these regions may be susceptible to time-dependent effects upon accumulation of food and money rewards. Relative to monetary gains, the consumption and evaluation of palatable milkshakes engaged complex neural processing over and above value tracking, emphasizing the critical contribution of taste and other sensory properties to the processing of food rewards. Furthermore, our results highlight the need to closely monitor metabolic states and neural responses to the accumulation of rewards to pinpoint the mechanisms underlying time-dependent dynamics of reward-related processing.

Memory for individual items is related to nonreinforced preference change.

It is commonly assumed that memories contribute to value-based decisions. Nevertheless, most theories of value-based decision-making do not account for memory influences on choice. Recently, new interest has emerged in the interactions between these two fundamental processes, mainly using reinforcement-based paradigms. Here, we aimed to study the role memory processes play in preference change following the nonreinforced cue-approach training (CAT) paradigm. In CAT, the mere association of cued items with a speeded motor response influences choices. Previous studies with this paradigm showed that a single training session induces a long-lasting effect of enhanced preferences for high-value trained stimuli, that is maintained for several months. We hypothesized that CAT increases memory of trained items, leading to enhanced accessibility of their positive associative memories and in turn to preference changes. In two preregistered experiments, we found evidence that memory is enhanced for trained items and that better memory is correlated with enhanced preferences at the individual item level, both immediately and 1 mo following CAT. Our findings suggest that memory plays a central role in value-based decision-making following CAT, even in the absence of external reinforcements. These findings contribute to new theories relating memory and value-based decision-making and set the groundwork for the implementation of novel nonreinforced behavioral interventions that lead to long-lasting behavioral change.

Brain volumetric changes in the general population following the COVID-19 outbreak and lockdown.

The coronavirus disease 2019 (COVID-19) outbreak introduced unprecedented health-risks, as well as pressure on the economy, society, and psychological well-being due to the response to the outbreak. In a preregistered study, we hypothesized that the intense experience of the outbreak potentially induced stress-related brain modifications in the healthy population, not infected with the virus. We examined volumetric changes in 50 participants who underwent MRI scans before and after the COVID-19 outbreak and lockdown in Israel. Their scans were compared with those of 50 control participants who were scanned twice prior to the pandemic. Following COVID-19 outbreak and lockdown, the test group participants uniquely showed volumetric increases in bilateral amygdalae, putamen, and the anterior temporal cortices. Changes in the amygdalae diminished as time elapsed from lockdown relief, suggesting that the intense experience associated with the pandemic induced transient volumetric changes in brain regions commonly associated with stress and anxiety. The current work utilizes a rare opportunity for real-life natural experiment, showing evidence for brain plasticity following the COVID-19 global pandemic. These findings have broad implications, relevant both for the scientific community as well as the general public.

Enhanced Bottom-Up and Reduced Top-Down fMRI Activity Is Related to Long-Lasting Nonreinforced Behavioral Change.

Behavioral change studies and interventions focus on self-control and external reinforcements to influence preferences. Cue-approach training (CAT) has been shown to induce preference changes lasting months by merely associating items with neutral cues and speeded responses. We utilized this paradigm to study neural representation of preferences and their modification without external reinforcements. We scanned 36 participants with fMRI during a novel passive viewing task before, after and 30 days following CAT. We preregistered the predictions that activity in memory, top-down attention, and value-processing regions will underlie preference modification. While most theories associate preferences with prefrontal regions, we found that "bottom-up" perceptual mechanisms were associated with immediate change, whereas reduced "top-down" parietal activity was related to long-term change. Activity in value-related prefrontal regions was enhanced immediately after CAT for trained items and 1 month after for all items. Our findings suggest a novel neural mechanism of preference representation and modification. We suggest that nonreinforced change of preferences occurs initially in perceptual representation of items, putatively leading to long-term changes in "top-down" processes. These findings offer implementation of bottom-up instead of top-down targeted interventions for long-lasting behavioral change.

Enhanced striatal and prefrontal activity is associated with individual differences in nonreinforced preference change for faces.

Developing effective preference modification paradigms is crucial to improve the quality of life in a wide range of behaviors. The cue-approach training (CAT) paradigm has been introduced as an effective tool to modify preferences lasting months, without external reinforcements, using the mere association of images with a cue and a speeded button response. In the current work for the first time, we used fMRI with faces as stimuli in the CAT paradigm, focusing on face-selective brain regions. We found a behavioral change effect of CAT with faces immediately and 1-month after training, however face-selective regions were not indicative of behavioral change and thus preference change is less likely to rely on face processing brain regions. Nevertheless, we found that during training, fMRI activations in the ventral striatum were correlated with individual preference change. We also found a correlation between preference change and activations in the ventromedial prefrontal cortex during the binary choice phase. Functional connectivity among striatum, prefrontal regions, and high-level visual regions was also related to individual preference change. Our work sheds new light on the involvement of neural mechanisms in the process of valuation. This could lead to development of novel real-world interventions.

Neural correlates of effort-based valuation with prospective choices.

How is effort integrated in value-based decision-making? Animal models and human neuroimaging studies primarily linked the anterior cingulate cortex (ACC) and ventral striatum (VS) to the integration of effort in valuation. Other studies demonstrated the role of these regions in invigoration to effort demands, thus it is hard to separate the neural activity linked to anticipation and subjective valuation from actual performance. Here, we studied the neural basis of effort valuation separated from performance. We scanned forty participants with fMRI, while they were asked to accept or reject monetary gambles that could be resolved with future performance of a familiar grip force effort challenge or a fixed risk prospect. Participants' willingness to accept prospective gambles reflected discounting of values by physical effort and risk. Choice-locked neural activation in contralateral primary sensory cortex and ventromedial prefrontal cortex (vmPFC) tracked the magnitude of prospective effort the participants faced, independent of choice time and monetary stakes. Estimates of subjective value discounted by effort were found to be tracked by the activation of a network of regions common to valuation under risk and delay, including vmPFC, VS and sensorimotor cortex. Together, our findings show separate neural mechanisms underlying prospective effort and actual effort performance.

Neural mechanisms of cue-approach training.

Biasing choices may prove a useful way to implement behavior change. Previous work has shown that a simple training task (the cue-approach task), which does not rely on external reinforcement, can robustly influence choice behavior by biasing choice toward items that were targeted during training. In the current study, we replicate previous behavioral findings and explore the neural mechanisms underlying the shift in preferences following cue-approach training. Given recent successes in the development and application of machine learning techniques to task-based fMRI data, which have advanced understanding of the neural substrates of cognition, we sought to leverage the power of these techniques to better understand neural changes during cue-approach training that subsequently led to a shift in choice behavior. Contrary to our expectations, we found that machine learning techniques applied to fMRI data during non-reinforced training were unsuccessful in elucidating the neural mechanism underlying the behavioral effect. However, univariate analyses during training revealed that the relationship between BOLD and choices for Go items increases as training progresses compared to choices of NoGo items primarily in lateral prefrontal cortical areas. This new imaging finding suggests that preferences are shifted via differential engagement of task control networks that interact with value networks during cue-approach training.

Predicting risky choices from brain activity patterns.

Previous research has implicated a large network of brain regions in the processing of risk during decision making. However, it has not yet been determined if activity in these regions is predictive of choices on future risky decisions. Here, we examined functional MRI data from a large sample of healthy subjects performing a naturalistic risk-taking task and used a classification analysis approach to predict whether individuals would choose risky or safe options on upcoming trials. We were able to predict choice category successfully in 71.8% of cases. Searchlight analysis revealed a network of brain regions where activity patterns were reliably predictive of subsequent risk-taking behavior, including a number of regions known to play a role in control processes. Searchlights with significant predictive accuracy were primarily located in regions more active when preparing to avoid a risk than when preparing to engage in one, suggesting that risk taking may be due, in part, to a failure of the control systems necessary to initiate a safe choice. Additional analyses revealed that subject choice can be successfully predicted with minimal decrements in accuracy using highly condensed data, suggesting that information relevant for risky choice behavior is encoded in coarse global patterns of activation as well as within highly local activation within searchlights.

Influencing food choices by training: evidence for modulation of frontoparietal control signals.

To overcome unhealthy behaviors, one must be able to make better choices. Changing food preferences is an important strategy in addressing the obesity epidemic and its accompanying public health risks. However, little is known about how food preferences can be effectively affected and what neural systems support such changes. In this study, we investigated a novel extensive training paradigm where participants chose from specific pairs of palatable junk food items and were rewarded for choosing the items with lower subjective value over higher value ones. In a later probe phase, when choices were made for real consumption, participants chose the lower-valued item more often in the trained pairs compared with untrained pairs. We replicated the behavioral results in an independent sample of participants while they were scanned with fMRI. We found that, as training progressed, there was decreased recruitment of regions that have been previously associated with cognitive control, specifically the left dorsolateral pFC and bilateral parietal cortices. Furthermore, we found that connectivity of the left dorsolateral pFC was greater with primary motor regions by the end of training for choices of lower-valued items that required exertion of self-control, suggesting a formation of a stronger stimulus-response association. These findings demonstrate that it is possible to influence food choices through training and that this training is associated with a decreasing need for top-down frontoparietal control. The results suggest that training paradigms may be promising as the basis for interventions to influence real-world food preferences.

Reward-related regions play a role in natural story comprehension.

The reward system was shown to be involved in a wide array of processes. Nevertheless, the exploration of the involvement of the reward system during language processing has not yet been directly tested. We investigated the role of reward-processing regions while listening to a natural story. We utilized a published dataset in which half of the participants listened to a natural story and the others listened to a scrambled version of it to compare the functional MRI signals in the reward system between these conditions and discovered a distinct pattern between conditions. This suggests that the reward system is activated during the comprehension of natural stories. We also show evidence that the fMRI signals in reward-related areas might potentially correlate with the predictability level of processed sentences. Further research is needed to determine the nature of the involvement and the way the activity interacts with various aspects of the sentences.

Discovering relations between mind, brain, and mental disorders using topic mapping.

Neuroimaging research has largely focused on the identification of associations between brain activation and specific mental functions. Here we show that data mining techniques applied to a large database of neuroimaging results can be used to identify the conceptual structure of mental functions and their mapping to brain systems. This analysis confirms many current ideas regarding the neural organization of cognition, but also provides some new insights into the roles of particular brain systems in mental function. We further show that the same methods can be used to identify the relations between mental disorders. Finally, we show that these two approaches can be combined to empirically identify novel relations between mental disorders and mental functions via their common involvement of particular brain networks. This approach has the potential to discover novel endophenotypes for neuropsychiatric disorders and to better characterize the structure of these disorders and the relations between them.

Consensus-based guidance for conducting and reporting multi-analyst studies.

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study.

Animal studies have found that the phasic activity of dopamine neurons during reward-related learning resembles a "prediction error" (PE) signal derived from a class of computational models called reinforcement learning (RL). An apparently similar signal can be measured using fMRI in the human striatum, a primary dopaminergic target. However, the fMRI signal does not measure dopamine per se, and therefore further evidence is needed to determine if these signals are related to each other. Parkinson's disease (PD) involves the neurodegeneration of the dopamine system and is accompanied by deficits in reward-related decision-making tasks. In the current study we used a computational RL model to assess striatal error signals in PD patients performing an RL task during fMRI scanning. Results show that error signals were preserved in ventral striatum of PD patients, but impaired in dorsolateral striatum, relative to healthy controls, a pattern reflecting the known selective anatomical degeneration of dopamine nuclei in PD. These findings support the notion that PE signals measured in the human striatum by the BOLD signal may reflect phasic DA activity. These results also provide evidence for a deficiency in PE signaling in the dorsolateral striatum of PD patients that may offer an explanation for their deficits observed in other reward learning tasks.

A Neural Pathway for Nonreinforced Preference Change.

How is value processed in the brain to inform decision making? A plethora of studies describe how preferences are shaped by experience with external reinforcements. While research on this reinforced pathway is well established, far less research has explored the neural pathways promoting preference change in the absence of external reinforcements. Here, we review behavioral paradigms linking nonreinforced preference change with manipulations of stimulus exposure, response, and gaze position. Based on this work, we propose that several brain regions traditionally associated with selective attention constitute a pathway for nonreinforced preference change. Together, this nonreinforced pathway (termed here the dorsal value pathway; DVP) and the more famously studied reinforced pathway (ventral value pathway; VVP), interface with prefrontal regions of the primate brain to guide value-based decisions.

Item Features Interact With Item Category in Their Influence on Preferences.

Low-level visual features are known to play a role in value-based decision-making. However, most previous studies focused on the role of only a single low-level feature or only for one type of item. These studies also used only one method of measurement and provided a theory accounting for those specific findings. We aimed to utilize a different more robust approach. We tested the contribution of low-level visual features to value-based decision-making of three item types: fractal-art images, faces, and snack food items. We used two techniques to estimate values: subjective ratings and actual choices. We found that low-level visual features contribute to value-based decision-making even after controlling for higher level features relevant for each item category (for faces, features like eye distance and for food snacks, features like price and calories). Importantly, we show that, overall, while low-level visual features consistently contribute to value-based decision-making as was previously shown, different features distinctively contribute to preferences of specific item types, as was evident when we estimated values using both techniques. We claim that theories relying on the role of single features for individual item types do not capture the complexity of the contribution of low-level visual features to value-based decision-making. Our conclusions call for future studies using multiple item types and various measurement methods for estimating value in order to modify current theories and construct a unifying framework regarding the relationship between low-level visual features and choice.

Counterconditioning following memory retrieval diminishes the reinstatement of appetitive memories in humans.

Appetitive memories play a crucial role in learning and behavior, but under certain circumstances, such memories become maladaptive and play a vital role in addiction and other psychopathologies. Recent scientific research has demonstrated that memories can be modified following their reactivation through memory retrieval in a process termed memory reconsolidation. Several nonpharmacological behavioral manipulations yielded mixed results in their capacity to alter maladaptive memories in humans. Here, we aimed to translate the promising findings observed in rodents to humans. We constructed a novel three-day procedure using aversive counterconditioning to alter appetitive memories after short memory retrieval. On the first day, we used appetitive conditioning to form appetitive memories. On the second day, we retrieved these appetitive memories in one group (Retrieval group) but not in a second group. Subsequently, all participants underwent counterconditioning. On the third day, we attempted to reinstate the appetitive memories from day one. We observed a significant reduction in the reinstatement of the original appetitive memory when counterconditioning was induced following memory retrieval. Here, we provide a novel human paradigm that models several memory processes and demonstrate memory attenuation when counterconditioned after its retrieval. This paradigm can be used to study complex appetitive memory dynamics, e.g., memory reconsolidation and its underlying brain mechanisms.

Is ventromedial prefrontal cortex critical for behavior change without external reinforcement?

Cue-approach training (CAT) is a novel paradigm that has been shown to induce preference changes towards items without external reinforcements. In the task, the mere association of a neutral cue and a speeded button response has been shown to induce a behavioral choice preference change lasting for months. This paradigm includes several phases: after the training of individual items, behavior change is manifested in binary choices of items with similar initial values. Neuroimaging data have implicated the ventromedial prefrontal cortex (vmPFC) in the choice phase of this task. However, the neural mechanisms underlying the preference changes induced by training remain unclear. Here, we asked whether the ventromedial frontal lobe (VMF) is critical for the non-reinforced preference change induced by CAT. For this purpose, 11 participants with focal lesions involving the VMF and 30 healthy age-matched controls performed the CAT. The VMF group was similar to the healthy age-matched control group in the ranking and training phases. As a group, the healthy age-matched controls exhibited a training-induced behavior change, while the VMF group did not. However, on an individual level analysis we found that some of the VMF participants showed a significant preference shift. Thus, we find mixed evidence for the role of VMF in this paradigm. This is another step towards defining the mechanisms underlying the novel form of behavioral change that occurs with CAT.