The formation of preference in risky choice.

A key question in decision-making is how people integrate amounts and probabilities to form preferences between risky alternatives. Here we rely on the general principle of integration-to-boundary to develop several biologically plausible process models of risky-choice, which account for both choices and response-times. These models allowed us to contrast two influential competing theories: i) within-alternative evaluations, based on multiplicative interaction between amounts and probabilities, ii) within-attribute comparisons across alternatives. To constrain the preference formation process, we monitored eye-fixations during decisions between pairs of simple lotteries, designed to systematically span the decision-space. The behavioral results indicate that the participants' eye-scanning patterns were associated with risk-preferences and expected-value maximization. Crucially, model comparisons showed that within-alternative process models decisively outperformed within-attribute ones, in accounting for choices and response-times. These findings elucidate the psychological processes underlying preference formation when making risky-choices, and suggest that compensatory, within-alternative integration is an adaptive mechanism employed in human decision-making.

Supplementary Eye Field Encodes Confidence in Decisions Under Risk.

Choices are made with varying degrees of confidence, a cognitive signal representing the subjective belief in the optimality of the choice. Confidence has been mostly studied in the context of perceptual judgments, in which choice accuracy can be measured using objective criteria. Here, we study confidence in subjective value-based decisions. We recorded in the supplementary eye field (SEF) of monkeys performing a gambling task, where they had to use subjective criteria for placing bets. We found neural signals in the SEF that explicitly represent choice confidence independent from reward expectation. This confidence signal appeared after the choice and diminished before the choice outcome. Most of this neuronal activity was negatively correlated with confidence, and was strongest in trials on which the monkey spontaneously withdrew his choice. Such confidence-related activity indicates that the SEF not only guides saccade selection, but also evaluates the likelihood that the choice was optimal. This internal evaluation influences decisions concerning the willingness to bear later costs that follow from the choice or to avoid them. More generally, our findings indicate that choice confidence is an integral component of all forms of decision-making, whether they are based on perceptual evidence or on value estimations.

Dopamine regulates attitude toward risk.

Specific brain pathways can lower or raise the willingness of monkeys to take risks.

An Exploratory Study of Large-Scale Brain Networks during Gambling Using SEEG.

Decision-making is a cognitive process involving working memory, executive function, and attention. However, the connectivity of large-scale brain networks during decision-making is not well understood. This is because gaining access to large-scale brain networks in humans is still a novel process. Here, we used SEEG (stereoelectroencephalography) to record neural activity from the default mode network (DMN), dorsal attention network (DAN), and frontoparietal network (FN) in ten humans while they performed a gambling task in the form of the card game, "War". By observing these networks during a decision-making period, we related the activity of and connectivity between these networks. In particular, we found that gamma band activity was directly related to a participant's ability to bet logically, deciding what betting amount would result in the highest monetary gain or lowest monetary loss throughout a session of the game. We also found connectivity between the DAN and the relation to a participant's performance. Specifically, participants with higher connectivity between and within these networks had higher earnings. Our preliminary findings suggest that connectivity and activity between these networks are essential during decision-making.

Supplementary eye field encodes reward prediction error.

The outcomes of many decisions are uncertain and therefore need to be evaluated. We studied this evaluation process by recording neuronal activity in the supplementary eye field (SEF) during an oculomotor gambling task. While the monkeys awaited the outcome, SEF neurons represented attributes of the chosen option, namely, its expected value and the uncertainty of this value signal. After the gamble result was revealed, a number of neurons reflected the actual reward outcome. Other neurons evaluated the outcome by encoding the difference between the reward expectation represented during the delay period and the actual reward amount (i.e., the reward prediction error). Thus, SEF encodes not only reward prediction error but also all the components necessary for its computation: the expected and the actual outcome. This suggests that SEF might actively evaluate value-based decisions in the oculomotor domain, independent of other brain regions.

Performance monitoring by presupplementary and supplementary motor area during an arm movement countermanding task.

A key component of executive control and decision making is the ability to use the consequences of chosen actions to update and inform the process of future action selection. Evaluative signals, which monitor the outcomes of actions, are critical for this ability. Signals related to the evaluation of actions have been identified in eye movement-related areas of the medial frontal cortex. Here we examined whether such evaluative signals are also present in areas of the medial frontal cortex related to arm movements. To answer this question, we recorded from cells in the supplementary motor area (SMA) and pre-SMA, while monkeys performed an arm movement version of the countermanding paradigm. SMA and pre-SMA have been implicated in the higher-order control of movement selection and execution, although their precise role within the skeletomotor control circuit is unclear. We found evaluative signals that encode information about the expected outcome of the reward, the actual outcome, and the mismatch between actual and intended outcome. These findings suggest that signals that monitor and evaluate movement outcomes are represented throughout the medial frontal cortex, playing a general role across effector systems. These evaluation signals supervise the relationship between intentional motor behavior and reward expectation and could be used to adaptively shape future goal-directed behavior.

Mechanisms underlying the influence of saliency on value-based decisions.

Objects in the environment differ in their low-level perceptual properties (e.g., how easily a fruit can be recognized) as well as in their subjective value (how tasty it is). We studied the influence of visual salience on value-based decisions using a two alternative forced choice task, in which human subjects rapidly chose items from a visual display. All targets were equally easy to detect. Nevertheless, both value and salience strongly affected choices made and reaction times. We analyzed the neuronal mechanisms underlying these behavioral effects using stochastic accumulator models, allowing us to characterize not only the averages of reaction times but their full distributions. Independent models without interaction between the possible choices failed to reproduce the observed choice behavior, while models with mutual inhibition between alternative choices produced much better results. Mutual inhibition thus is an important feature of the decision mechanism. Value influenced the amount of accumulation in all models. In contrast, increased salience could either lead to an earlier start (onset model) or to a higher rate (speed model) of accumulation. Both models explained the data from the choice trials equally well. However, salience also affected reaction times in no-choice trials in which only one item was present, as well as error trials. Only the onset model could explain the observed reaction time distributions of error trials and no-choice trials. In contrast, the speed model could not, irrespective of whether the rate increase resulted from more frequent accumulated quanta or from larger quanta. Visual salience thus likely provides an advantage in the onset, not in the processing speed, of value-based decision making.

Quantifying Phase-Amplitude Modulation in Neural Data.

Phase-amplitude modulation (the modulation of the amplitude of higher frequency oscillations by the phase of lower frequency oscillations) is a specific type of cross-frequency coupling that has been observed in neural recordings from multiple species in a range of behavioral contexts. Given its potential importance, care must be taken with how it is measured and quantified. Previous studies have quantified phase-amplitude modulation by measuring the distance of the amplitude distribution from a uniform distribution. While this method is of general applicability, it is not targeted to the specific modulation pattern frequently observed with low-frequency oscillations. Here we develop a new method that has increased specificity to detect modulation in the sinusoidal shape commonly observed in neural data.

Primate anterior insular cortex represents economic decision variables proposed by prospect theory.

In humans, risk attitude is highly context-dependent, varying with wealth levels or for different potential outcomes, such as gains or losses. These behavioral effects have been modelled using prospect theory, with the key assumption that humans represent the value of each available option asymmetrically as a gain or loss relative to a reference point. It remains unknown how these computations are implemented at the neuronal level. Here we show that macaques, like humans, change their risk attitude across wealth levels and gain/loss contexts using a token gambling task. Neurons in the anterior insular cortex (AIC) encode the 'reference point' (i.e., the current wealth level of the monkey) and reflect 'loss aversion' (i.e., option value signals are more sensitive to change in the loss than in the gain context) as postulated by prospect theory. In addition, changes in the activity of a subgroup of AIC neurons correlate with the inter-trial fluctuations in choice and risk attitude. Taken together, we show that the primate AIC in risky decision-making may be involved in monitoring contextual information used to guide the animal's willingness to accept risk.

Decomposing Executive Function into Distinct Processes Underlying Human Decision Making.

Executive function (EF) consists of higher level cognitive processes including working memory, cognitive flexibility, and inhibition which together enable goal-directed behaviors. Many neurological disorders are associated with EF dysfunctions which can lead to suboptimal behavior. To assess the roles of these processes, we introduce a novel behavioral task and modeling approach. The gamble-like task, with sub-tasks targeting different EF capabilities, allows for quantitative assessment of the main components of EF. We demonstrate that human participants exhibit dissociable variability in the component processes of EF. These results will allow us to map behavioral outcomes to EEG recordings in future work in order to map brain networks associated with EF deficits. Clinical relevance- This work will allow us to quantify EF deficits and corresponding brain activity in patient populations in future work.

Medial frontal cortex motivates but does not control movement initiation in the countermanding task.

Voluntary control of behavior implies the ability to select what action is performed. The supplementary motor area (SMA) and pre-SMA are widely considered to be of central importance for this ability because of their role in movement initiation and inhibition. To test this hypothesis, we recorded from neurons in SMA and pre-SMA of monkeys performing an arm countermanding task. Temporal analysis of neural activity and behavior in this task allowed us to test whether neural activity is sufficient to control movement initiation or inhibition. Surprisingly, 99% (242 of 243) of movement-related neurons in SMA and pre-SMA failed to exhibit time-locked activity changes predictive of movement initiation in this task. We also found a second group of neurons that was more active during successful response cancelation. Of these putative inhibitory cells, 18% (7 of 40) responded early enough to be able to influence the cancelation of the movement. Thus, when tested with the countermanding task, the SMA/pre-SMA region may play a role in movement inhibition but does not appear to control movement initiation. However, the activity of 76% (202 of 267) of movement-related neurons was contingent on the expectation of reward and 42% of them reflected the amount of expected reward. These findings suggest that the movement-related activity in pre-SMA and SMA might represent the motivation for a specific action but does not determine whether or not that action is performed. This motivational signal in pre-SMA and SMA could provide an essential link between reward expectation and motor execution.

Supplementary motor area exerts proactive and reactive control of arm movements.

Adaptive behavior requires the ability to flexibly control actions. This can occur either proactively to anticipate task requirements, or reactively in response to sudden changes. Here we report neuronal activity in the supplementary motor area (SMA) that is correlated with both forms of behavioral control. Single-unit and multiunit activity and intracranial local field potentials (LFPs) were recorded in macaque monkeys during a stop-signal task, which elicits both proactive and reactive behavioral control. The LFP power in high- (60-150 Hz) and low- (25-40 Hz) frequency bands was significantly correlated with arm movement reaction time, starting before target onset. Multiunit and single-unit activity also showed a significant regression with reaction time. In addition, LFPs and multiunit and single-unit activity changed their activity level depending on the trial history, mirroring adjustments on the behavioral level. Together, these findings indicate that neuronal activity in the SMA exerts proactive control of arm movements by adjusting the level of motor readiness. On trials when the monkeys successfully canceled arm movements in response to an unforeseen stop signal, the LFP power, particularly in a low (10-50 Hz) frequency range, increased early enough to be causally related to the inhibition of the arm movement on those trials. This indicated that neuronal activity in the SMA is also involved in response inhibition in reaction to sudden task changes. Our findings indicate, therefore, that SMA plays a role in the proactive control of motor readiness and the reactive inhibition of unwanted movements.

Supplementary eye field encodes option and action value for saccades with variable reward.

We recorded neuronal activity in the supplementary eye field (SEF) while monkeys made saccades to targets that yielded rewards of variable amount and uncertainty of delivery. Some SEF cells (29%) represented the anticipated value of the saccade target. These neurons encoded the value of the reward option but did not reflect the action necessary to obtain the reward. A plurality of cells (45%) represented both saccade direction and value. These neurons reflect action value, i.e., the value that is expected to follow from a specific saccade. Other cells (13%) represented only saccade direction. The SEF neurons matched the monkey's risk-seeking behavior by responding more strongly to the uncertain reward options than would be expected based on their response to the sure options and the cued outcome probability. Thus SEF neurons represented subjective, not expected, value. Across the SEF population, option-value signals developed early, ∼120 ms prior to saccade execution. Action-value and saccade direction signals developed ∼60 ms later. These results suggest that the SEF is involved in transforming option-value signals into action-value signals. However, in contrast to other oculomotor neurons, SEF neurons did not reach a constant level of activity before saccade onset. Instead the activity level of many (52%) SEF neurons still reflected value at the time just before saccade initiation. This suggests that SEF neurons guide the selection of a saccade based on value information but do not participate in the initiation of that saccade.

Role of supplementary eye field in saccade initiation: executive, not direct, control.

The goal of this study was to determine whether the activity of neurons in the supplementary eye field (SEF) is sufficient to control saccade initiation in macaque monkeys performing a saccade countermanding (stop signal) task. As previously observed, many neurons in the SEF increase the discharge rate before saccade initiation. However, when saccades are canceled in response to a stop signal, effectively no neurons with presaccadic activity display discharge rate modulation early enough to contribute to saccade cancellation. Moreover, SEF neurons do not exhibit a specific threshold discharge rate that could trigger saccade initiation. Yet, we observed more subtle relations between SEF activation and saccade production. The activity of numerous SEF neurons was correlated with response time and varied with sequential adjustments in response latency. Trials in which monkeys canceled or produced a saccade in a stop signal trial were distinguished by a modest difference in discharge rate of these SEF neurons before stop signal or target presentation. These findings indicate that neurons in the SEF, in contrast to counterparts in the frontal eye field and superior colliculus, do not contribute directly and immediately to the initiation of visually guided saccades. However the SEF may proactively regulate saccade production by biasing the balance between gaze-holding and gaze-shifting based on prior performance and anticipated task requirements.

Executive control of countermanding saccades by the supplementary eye field.

The supplementary eye field registers the occurrence of conflict, errors and reward in macaque monkeys performing a saccade-countermanding task. Using intracortical microstimulation, we determined whether the supplementary eye field only monitors or can actually influence performance. Weak microstimulation of many sites in the supplementary eye field improved monkeys' performance on a 'stop signal' task by delaying saccade initiation. This effect depended on the context of the task because simple visually guided saccades were not delayed by the same stimulation. These results demonstrate that the supplementary eye field can exert contextual executive control over saccade generation.

Decision Making: How Is Information Represented in Orbitofrontal Cortex?

Classically, specific orbitofrontal cortex (OFC) neurons are thought to represent attributes of specific decision options. A new model proposes instead that OFC neurons represent whichever option is currently attended. A recent study, however, tests these two models and rules out the 'current-focus-of-attention' model.

Multi-View Broad Learning System for Primate Oculomotor Decision Decoding.

Multi-view learning improves the learning performance by utilizing multi-view data: data collected from multiple sources, or feature sets extracted from the same data source. This approach is suitable for primate brain state decoding using cortical neural signals. This is because the complementary components of simultaneously recorded neural signals, local field potentials (LFPs) and action potentials (spikes), can be treated as two views. In this paper, we extended broad learning system (BLS), a recently proposed wide neural network architecture, from single-view learning to multi-view learning, and validated its performance in decoding monkeys' oculomotor decision from medial frontal LFPs and spikes. We demonstrated that medial frontal LFPs and spikes in non-human primate do contain complementary information about the oculomotor decision, and that the proposed multi-view BLS is a more effective approach for decoding the oculomotor decision than several classical and state-of-the-art single-view and multi-view learning approaches.

Neuroeconomics: cardinal utility in the orbitofrontal cortex?

Modern economics no longer uses the concept of cardinal utility, which describes the value of a good independently of a comparison with another good. New electrophysiological recordings in primates performing economic choices suggest a neurological substrate for cardinal utility, a finding that economists should perhaps take note of.

Monitoring and control of action by the frontal lobes.

Success requires deciding among alternatives, controlling the initiation of movements, and judging the consequences of actions. When alternatives are difficult to distinguish, habitual responses must be overcome, or consequences are uncertain, deliberation is necessary and a supervisory system exerts control over the processes that produce sensory-guided movements. We have investigated these processes by recording neural activity in the frontal lobe of macaque monkeys performing a countermanding task. Distinct neurons in the frontal eye field respond to visual stimuli or control the production of the movements. In the supplementary eye field and anterior cingulate cortex, neurons appear not to control directly movement initiation but instead signal the production of errors, the anticipation and delivery of reinforcement, and the presence of processing conflict. These signals form the core of current models of supervisory control of sensorimotor processes.