Decision prioritization and causal reasoning in decision hierarchies.

From cooking a meal to finding a route to a destination, many real life decisions can be decomposed into a hierarchy of sub-decisions. In a hierarchy, choosing which decision to think about requires planning over a potentially vast space of possible decision sequences. To gain insight into how people decide what to decide on, we studied a novel task that combines perceptual decision making, active sensing and hierarchical and counterfactual reasoning. Human participants had to find a target hidden at the lowest level of a decision tree. They could solicit information from the different nodes of the decision tree to gather noisy evidence about the target's location. Feedback was given only after errors at the leaf nodes and provided ambiguous evidence about the cause of the error. Despite the complexity of task (with 107 latent states) participants were able to plan efficiently in the task. A computational model of this process identified a small number of heuristics of low computational complexity that accounted for human behavior. These heuristics include making categorical decisions at the branching points of the decision tree rather than carrying forward entire probability distributions, discarding sensory evidence deemed unreliable to make a choice, and using choice confidence to infer the cause of the error after an initial plan failed. Plans based on probabilistic inference or myopic sampling norms could not capture participants' behavior. Our results show that it is possible to identify hallmarks of heuristic planning with sensing in human behavior and that the use of tasks of intermediate complexity helps identify the rules underlying human ability to reason over decision hierarchies.

Variance misperception explains illusions of confidence in simple perceptual decisions.

Confidence in a perceptual decision is a judgment about the quality of the sensory evidence. The quality of the evidence depends not only on its strength ('signal') but critically on its reliability ('noise'), but the separate contribution of these quantities to the formation of confidence judgments has not been investigated before in the context of perceptual decisions. We studied subjective confidence reports in a multi-element perceptual task where evidence strength and reliability could be manipulated independently. Our results reveal a confidence paradox: confidence is higher for stimuli of lower reliability that are associated with a lower accuracy. We show that the subjects' overconfidence in trials with unreliable evidence is caused by a reduced sensitivity to stimulus variability. Our results bridge between the investigation of miss-attributions of confidence in behavioral economics and the domain of simple perceptual decisions amenable to neuroscience research.

Population Representation of the Confidence in a Decision in the Lateral Intraparietal Area of the Macaque.

Confidence in a decision is the belief, prior to feedback, that one's choice is correct. In the brain, many decisions are implemented as a race between competing evidence-accumulation processes. We ask whether the neurons that represent evidence accumulation also carry information about whether the choice is correct (i.e., confidence). Monkeys performed a reaction time version of the random dot motion task. Neuropixels probes were used to record from neurons in the lateral intraparietal (LIP) area. LIP neurons with response fields that overlap the choice-target contralateral to the recording site (Tin neurons) represent the accumulation of evidence in favor of contralateral target selection. We demonstrate that shortly before a contralateral choice is reported, the population of Tin neurons contains information about the accuracy of the choice (i.e., whether the choice is correct or incorrect). This finding is unexpected because, on average, Tin neurons exhibit a level of activity before the report that is independent of reaction time and evidence strength-both strong predictors of accuracy. This apparent contradiction is resolved by examining the variability in neuronal responses across the population of Tin neurons. While on average, Tin neurons exhibit a stereotyped level of activity before a contralateral choice, many neurons depart from this average in a consistent manner. From these neurons, the accuracy of the choice can be predicted using a simple logistic decoder. The accuracy of the choice predicted from neural activity reproduces the hallmarks of confidence identified in human behavioral experiments. Therefore, neurons that represent evidence accumulation can also inform the monkey's confidence.

Value construction through sequential sampling explains serial dependencies in decision making.

Many decisions are expressed as a preference for one item over another. When these items are familiar, it is often assumed that the decision maker assigns a value to each of the items and chooses the item with the highest value. These values may be imperfectly recalled, but are assumed to be stable over the course of an interview or psychological experiment. Choices that are inconsistent with a stated valuation are thought to occur because of unspecified noise that corrupts the neural representation of value. Assuming that the noise is uncorrelated over time, the pattern of choices and response times in value-based decisions are modeled within the framework of Bounded Evidence Accumulation (BEA), similar to that used in perceptual decision-making. In BEA, noisy evidence samples accumulate over time until the accumulated evidence for one of the options reaches a threshold. Here, we argue that the assumption of temporally uncorrelated noise, while reasonable for perceptual decisions, is not reasonable for value-based decisions. Subjective values depend on the internal state of the decision maker, including their desires, needs, priorities, attentional state, and goals. These internal states may change over time, or undergo revaluation, as will the subjective values. We reasoned that these hypothetical value changes should be detectable in the pattern of choices made over a sequence of decisions. We reanalyzed data from a well-studied task in which participants were presented with pairs of snacks and asked to choose the one they preferred. Using a novel algorithm (Reval), we show that the subjective value of the items changes significantly during a short experimental session (about 1 hour). Values derived with Reval explain choice and response time better than explicitly stated values. They also better explain the BOLD signal in the ventromedial prefrontal cortex, known to represent the value of decision alternatives. Revaluation is also observed in a BEA model in which successive evidence samples are not assumed to be independent. We argue that revaluation is a consequence of the process by which values are constructed during deliberation to resolve preference choices.

Incorporation of a cost of deliberation time in perceptual decision making.

Many decisions benefit from the accumulation of evidence obtained sequentially over time. In such circumstances, the decision maker must balance speed against accuracy, and the nature of this tradeoff mediates competing desiderata and costs, especially those associated with the passage of time. A neural mechanism to achieve this balance is to accumulate evidence in suitable units and to terminate the deliberation when enough evidence has accrued. To accommodate time costs, it has been hypothesized that the criterion to terminate a decision may become lax as a function of time. Here we tested this hypothesis by manipulating the cost of time in a perceptual choice-reaction time task. Participants discriminated the direction of motion in a dynamic random-dot display, which varied in difficulty across trials. After each trial, they received feedback in the form of points based on whether they made a correct or erroneous choice. They were instructed to maximize their points per unit of time. Unbeknownst to the participants, halfway through the experiment, we increased the time pressure by canceling a small fraction of trials if they had not made a decision by a provisional deadline. Although the manipulation canceled less than 5% of trials, it induced the participants to make faster decisions while lowering their decision accuracy. The pattern of choices and reaction times were explained by bounded drift-diffusion. In all phases of the experiment, stopping bounds were found to decline as a function of time, consistent with the optimal solution, and this decline was exaggerated in response to the time-cost manipulation.

Direct observation of the neural computations underlying a single decision.

Neurobiological investigations of perceptual decision-making have furnished the first glimpse of a flexible cognitive process at the level of single neurons. Neurons in the parietal and prefrontal cortex are thought to represent the accumulation of noisy evidence, acquired over time, leading to a decision. Neural recordings averaged over many decisions have provided support for the deterministic rise in activity to a termination bound. Critically, it is the unobserved stochastic component that is thought to confer variability in both choice and decision time. Here, we elucidate this drift-diffusion signal on individual decisions. We recorded simultaneously from hundreds of neurons in the lateral intraparietal cortex of monkeys while they made decisions about the direction of random dot motion. We show that a single scalar quantity, derived from the weighted sum of the population activity, represents a combination of deterministic drift and stochastic diffusion. Moreover, we provide direct support for the hypothesis that this drift-diffusion signal approximates the quantity responsible for the variability in choice and reaction times. The population-derived signals rely on a small subset of neurons with response fields that overlap the choice targets. These neurons represent the integral of noisy evidence. Another subset of direction-selective neurons with response fields that overlap the motion stimulus appear to represent the integrand. This parsimonious architecture would escape detection by state-space analyses, absent a clear hypothesis.

Judging the difficulty of perceptual decisions.

Deciding how difficult it is going to be to perform a task allows us to choose between tasks, allocate appropriate resources, and predict future performance. To be useful for planning, difficulty judgments should not require completion of the task. Here we examine the processes underlying difficulty judgments in a perceptual decision making task. Participants viewed two patches of dynamic random dots, which were colored blue or yellow stochastically on each appearance. Stimulus coherence (the probability, pblue, of a dot being blue) varied across trials and patches thus establishing difficulty, pblue-0.5. Participants were asked to indicate for which patch it would be easier to decide the dominant color. Accuracy in difficulty decisions improved with the difference in the stimulus difficulties, whereas the reaction times were not determined solely by this quantity. For example, when the patches shared the same difficulty, reaction times were shorter for easier stimuli. A comparison of several models of difficulty judgment suggested that participants compare the absolute accumulated evidence from each stimulus and terminate their decision when they differed by a set amount. The model predicts that when the dominant color of each stimulus is known, reaction times should depend only on the difference in difficulty, which we confirm empirically. We also show that this model is preferred to one that compares the confidence one would have in making each decision. The results extend evidence accumulation models, used to explain choice, reaction time and confidence to prospective judgments of difficulty.

Judging the difficulty of perceptual decisions.

Deciding how difficult it is going to be to perform a task allows us to choose between tasks, allocate appropriate resources, and predict future performance. To be useful for planning, difficulty judgments should not require completion of the task. Here, we examine the processes underlying difficulty judgments in a perceptual decision-making task. Participants viewed two patches of dynamic random dots, which were colored blue or yellow stochastically on each appearance. Stimulus coherence (the probability, pblue, of a dot being blue) varied across trials and patches thus establishing difficulty, |pblue -0.5|. Participants were asked to indicate for which patch it would be easier to decide the dominant color. Accuracy in difficulty decisions improved with the difference in the stimulus difficulties, whereas the reaction times were not determined solely by this quantity. For example, when the patches shared the same difficulty, reaction times were shorter for easier stimuli. A comparison of several models of difficulty judgment suggested that participants compare the absolute accumulated evidence from each stimulus and terminate their decision when they differed by a set amount. The model predicts that when the dominant color of each stimulus is known, reaction times should depend only on the difference in difficulty, which we confirm empirically. We also show that this model is preferred to one that compares the confidence one would have in making each decision. The results extend evidence accumulation models, used to explain choice, reaction time, and confidence to prospective judgments of difficulty.

When order matters: last-come first-served effect in sequential arithmetic operations.

Cognitive psychologists have relied on dual-task interference experiments to understand the low-capacity and serial nature of conscious mental operations. Two widely studied paradigms, the Attentional Blink (AB) and the Psychological Refractory Period (PRP) have demonstrated a first-come first-served policy; processing a stimulus either impedes conscious access (AB) or postpones treatment (PRP) of a concurrent stimulus. Here we explored the transition from dual-task paradigms to multi-step human cognition. We studied the relative weight of individual addends in a sequential arithmetic task, where number notation (symbolic/non-symbolic) and presentation speed were independently manipulated. For slow presentation and symbolic notation, the decision relied almost equally on all addends, whereas for fast or non-symbolic notation, the decision relied almost exclusively on the last item reflecting a last-come first-served policy. We suggest that streams of stimuli may be chunked in events in which the last stimuli may override previous items from sensory buffers.

Sequential sampling from memory underlies action selection during abstract decision-making.

The study of perceptual decision-making in monkeys has provided insights into the process by which sensory evidence is integrated toward a decision. When monkeys make decisions with the knowledge of the motor actions the decisions bear upon, the process of evidence integration is instantiated by neurons involved in the selection of said actions. It is less clear how monkeys make decisions when unaware of the actions required to communicate their choice-what we refer to as "abstract" decisions. We investigated this by training monkeys to associate the direction of motion of a noisy random-dot display with the color of two targets. Crucially, the targets were displayed at unpredictable locations after the motion stimulus was extinguished. We found that the monkeys postponed decision formation until the targets were revealed. Neurons in the parietal association area LIP represented the integration of evidence leading to a choice, but as the stimulus was no longer visible, the samples of evidence must have been retrieved from short-term memory. Our results imply that when decisions are temporally unyoked from the motor actions they bear upon, decision formation is protracted until they can be framed in terms of motor actions.

Consensus Goals in the Field of Visual Metacognition.

Despite the tangible progress in psychological and cognitive sciences over the last several years, these disciplines still trail other more mature sciences in identifying the most important questions that need to be solved. Reaching such consensus could lead to greater synergy across different laboratories, faster progress, and increased focus on solving important problems rather than pursuing isolated, niche efforts. Here, 26 researchers from the field of visual metacognition reached consensus on four long-term and two medium-term common goals. We describe the process that we followed, the goals themselves, and our plans for accomplishing these goals. If this effort proves successful within the next few years, such consensus building around common goals could be adopted more widely in psychological science.

The brain's router: a cortical network model of serial processing in the primate brain.

The human brain efficiently solves certain operations such as object recognition and categorization through a massively parallel network of dedicated processors. However, human cognition also relies on the ability to perform an arbitrarily large set of tasks by flexibly recombining different processors into a novel chain. This flexibility comes at the cost of a severe slowing down and a seriality of operations (100-500 ms per step). A limit on parallel processing is demonstrated in experimental setups such as the psychological refractory period (PRP) and the attentional blink (AB) in which the processing of an element either significantly delays (PRP) or impedes conscious access (AB) of a second, rapidly presented element. Here we present a spiking-neuron implementation of a cognitive architecture where a large number of local parallel processors assemble together to produce goal-driven behavior. The precise mapping of incoming sensory stimuli onto motor representations relies on a "router" network capable of flexibly interconnecting processors and rapidly changing its configuration from one task to another. Simulations show that, when presented with dual-task stimuli, the network exhibits parallel processing at peripheral sensory levels, a memory buffer capable of keeping the result of sensory processing on hold, and a slow serial performance at the router stage, resulting in a performance bottleneck. The network captures the detailed dynamics of human behavior during dual-task-performance, including both mean RTs and RT distributions, and establishes concrete predictions on neuronal dynamics during dual-task experiments in humans and non-human primates.

Multiple decisions about one object involve parallel sensory acquisition but time-multiplexed evidence incorporation.

The brain is capable of processing several streams of information that bear on different aspects of the same problem. Here, we address the problem of making two decisions about one object, by studying difficult perceptual decisions about the color and motion of a dynamic random dot display. We find that the accuracy of one decision is unaffected by the difficulty of the other decision. However, the response times reveal that the two decisions do not form simultaneously. We show that both stimulus dimensions are acquired in parallel for the initial ∼0.1 s but are then incorporated serially in time-multiplexed bouts. Thus, there is a bottleneck that precludes updating more than one decision at a time, and a buffer that stores samples of evidence while access to the decision is blocked. We suggest that this bottleneck is responsible for the long timescales of many cognitive operations framed as decisions.

Differentiating between integration and non-integration strategies in perceptual decision making.

Many tasks used to study decision-making encourage subjects to integrate evidence over time. Such tasks are useful to understand how the brain operates on multiple samples of information over prolonged timescales, but only if subjects actually integrate evidence to form their decisions. We explored the behavioral observations that corroborate evidence-integration in a number of task-designs. Several commonly accepted signs of integration were also predicted by non-integration strategies. Furthermore, an integration model could fit data generated by non-integration models. We identified the features of non-integration models that allowed them to mimic integration and used these insights to design a motion discrimination task that disentangled the models. In human subjects performing the task, we falsified a non-integration strategy in each and confirmed prolonged integration in all but one subject. The findings illustrate the difficulty of identifying a decision-maker's strategy and support solutions to achieve this goal.

The hippocampus supports deliberation during value-based decisions.

Choosing between two items involves deliberation and comparison of the features of each item and its value. Such decisions take more time when choosing between options of similar value, possibly because these decisions require more evidence, but the mechanisms involved are not clear. We propose that the hippocampus supports deliberation about value, given its well-known role in prospection and relational cognition. We assessed the role of the hippocampus in deliberation in two experiments. First, using fMRI in healthy participants, we found that BOLD activity in the hippocampus increased as a function of deliberation time. Second, we found that patients with hippocampal damage exhibited more stochastic choices and longer reaction times than controls, possibly due to their failure to construct value-based or internal evidence during deliberation. Both sets of results were stronger in value-based decisions compared to perceptual decisions.

Counterfactual Reasoning Underlies the Learning of Priors in Decision Making.

Accurate decisions require knowledge of prior probabilities (e.g., prevalence or base rate), but it is unclear how prior probabilities are learned in the absence of a teacher. We hypothesized that humans could learn base rates from experience making decisions, even without feedback. Participants made difficult decisions about the direction of dynamic random dot motion. Across blocks of 15-42 trials, the base rate favoring left or right varied. Participants were not informed of the base rate or choice accuracy, yet they gradually biased their choices and thereby increased accuracy and confidence in their decisions. They achieved this by updating knowledge of base rate after each decision, using a counterfactual representation of confidence that simulates a neutral prior. The strategy is consistent with Bayesian updating of belief and suggests that humans represent both true confidence, which incorporates the evolving belief of the prior, and counterfactual confidence, which discounts the prior.

Serial, parallel and hierarchical decision making in primates.

The study of decision-making has mainly focused on isolated decisions where choices are associated with motor actions. However, problem-solving often involves considering a hierarchy of sub-decisions. In a recent study (Lorteije et al. 2015), we reported behavioral and neuronal evidence for hierarchical decision making in a task with a small decision tree. We observed a first phase of parallel evidence integration for multiple sub-decisions, followed by a phase in which the overall strategy formed. It has been suggested that a 'flat' competition between the ultimate motor actions might also explain these results. A reanalysis of the data does not support the critical predictions of flat models. We also examined the time-course of decision making in other, related tasks and report conditions where evidence integration for successive decisions is decoupled, which excludes flat models. We conclude that the flexibility of decision-making implies that the strategies are genuinely hierarchical.

The influence of evidence volatility on choice, reaction time and confidence in a perceptual decision.

Many decisions are thought to arise via the accumulation of noisy evidence to a threshold or bound. In perception, the mechanism explains the effect of stimulus strength, characterized by signal-to-noise ratio, on decision speed, accuracy and confidence. It also makes intriguing predictions about the noise itself. An increase in noise should lead to faster decisions, reduced accuracy and, paradoxically, higher confidence. To test these predictions, we introduce a novel sensory manipulation that mimics the addition of unbiased noise to motion-selective regions of visual cortex, which we verified with neuronal recordings from macaque areas MT/MST. For both humans and monkeys, increasing the noise induced faster decisions and greater confidence over a range of stimuli for which accuracy was minimally impaired. The magnitude of the effects was in agreement with predictions of a bounded evidence accumulation model.

Individual consistency in the accuracy and distribution of confidence judgments.

We examine which aspects of the confidence distributions - its shape, its bias toward higher or lower values, and its ability to distinguish correct from erred trials - are idiosyncratic of the who (individual specificity), the when (variability across days) and the what (task specificity). Measuring confidence across different sessions of four different perceptual tasks we show that: (1) Confidence distributions are virtually identical when measured in different days for the same subject and the same task, constituting a subjective fingerprint, (2) The capacity of confidence reports to distinguish correct from incorrect responses is only modestly (but significantly) correlated when compared across tasks, (3) Confidence distributions are very similar for tasks that involve different sensory modalities but have similar structure, (4) Confidence accuracy is independent of the mean and width of the confidence distribution, (5) The mean of the confidence distribution (an individual's confidence bias) constitutes the most efficient indicator to infer a subject's identity from confidence reports and (6) Confidence bias measured in simple perceptual decisions correlates with an individual's optimism bias measured with standard questionnaire.

Confidence through consensus: a neural mechanism for uncertainty monitoring.

Models that integrate sensory evidence to a threshold can explain task accuracy, response times and confidence, yet it is still unclear how confidence is encoded in the brain. Classic models assume that confidence is encoded in some form of balance between the evidence integrated in favor and against the selected option. However, recent experiments that measure the sensory evidence's influence on choice and confidence contradict these classic models. We propose that the decision is taken by many loosely coupled modules each of which represent a stochastic sample of the sensory evidence integral. Confidence is then encoded in the dispersion between modules. We show that our proposal can account for the well established relations between confidence, and stimuli discriminability and reaction times, as well as the fluctuations influence on choice and confidence.