Curiosity: primate neural circuits for novelty and information seeking.

For many years, neuroscientists have investigated the behavioural, computational and neurobiological mechanisms that support value-based decisions, revealing how humans and animals make choices to obtain rewards. However, many decisions are influenced by factors other than the value of physical rewards or second-order reinforcers (such as money). For instance, animals (including humans) frequently explore novel objects that have no intrinsic value solely because they are novel and they exhibit the desire to gain information to reduce their uncertainties about the future, even if this information cannot lead to reward or assist them in accomplishing upcoming tasks. In this Review, I discuss how circuits in the primate brain responsible for detecting, predicting and assessing novelty and uncertainty regulate behaviour and give rise to these behavioural components of curiosity. I also briefly discuss how curiosity-related behaviours arise during postnatal development and point out some important reasons for the persistence of curiosity across generations.

How the value of the environment controls persistence in visual search.

Classic foraging theory predicts that humans and animals aim to gain maximum reward per unit time. However, in standard instrumental conditioning tasks individuals adopt an apparently suboptimal strategy: they respond slowly when the expected value is low. This reward-related bias is often explained as reduced motivation in response to low rewards. Here we present evidence this behavior is associated with a complementary increased motivation to search the environment for alternatives. We trained monkeys to search for reward-related visual targets in environments with different values. We found that the reward-related bias scaled with environment value, was consistent with persistent searching after the target was already found, and was associated with increased exploratory gaze to objects in the environment. A novel computational model of foraging suggests that this search strategy could be adaptive in naturalistic settings where both environments and the objects within them provide partial information about hidden, uncertain rewards.

Multiple Mechanisms for Processing Reward Uncertainty in the Primate Basal Forebrain.

UNLABELLED: The ability to use information about the uncertainty of future outcomes is critical for adaptive behavior in an uncertain world. We show that the basal forebrain (BF) contains at least two distinct neural-coding strategies to support this capacity. The dorsal-lateral BF, including the ventral pallidum (VP), contains reward-sensitive neurons, some of which are selectively suppressed by uncertain-reward predictions (U(-)). In contrast, the medial BF (mBF) contains reward-sensitive neurons, some of which are selectively enhanced (U(+)) by uncertain-reward predictions. In a two-alternative choice-task, U(-) neurons were selectively suppressed while monkeys chose uncertain options over certain options. During the same choice-epoch, U(+) neurons signaled the subjective reward value of the choice options. Additionally, after the choice was reported, U(+) neurons signaled reward uncertainty until the choice outcome. We suggest that uncertainty-related suppression of VP may participate in the mediation of uncertainty-seeking actions, whereas uncertainty-related enhancement of the mBF may direct cognitive resources to monitor and learn from uncertain-outcomes. SIGNIFICANCE STATEMENT: To survive in an uncertain world, we must approach uncertainty and learn from it. Here we provide evidence for two mostly distinct mechanisms for processing uncertainty about rewards within different subregions of the primate basal forebrain (BF). We found that uncertainty suppressed the representation of certain (or safe) reward values by some neurons in the dorsal-lateral BF, in regions occupied by the ventral pallidum. This uncertainty-related suppression was evident as monkeys made risky choices. We also found that uncertainty-enhanced the activity of many medial BF neurons, most prominently after the monkeys' choices were completed (as they awaited uncertain outcomes). Based on these findings, we propose that different subregions of the BF could support action and learning under uncertainty in distinct but complimentary manners.

Neurons in the Primate Medial Basal Forebrain Signal Combined Information about Reward Uncertainty, Value, and Punishment Anticipation.

It has been suggested that the basal forebrain (BF) exerts strong influences on the formation of memory and behavior. However, what information is used for the memory-behavior formation is unclear. We found that a population of neurons in the medial BF (medial septum and diagonal band of Broca) of macaque monkeys encodes a unique combination of information: reward uncertainty, expected reward value, anticipation of punishment, and unexpected reward and punishment. The results were obtained while the monkeys were expecting (often with uncertainty) a rewarding or punishing outcome during a Pavlovian procedure, or unexpectedly received an outcome outside the procedure. In vivo anterograde tracing using manganese-enhanced MRI suggested that the major recipient of these signals is the intermediate hippocampal formation. Based on these findings, we hypothesize that the medial BF identifies various contexts and outcomes that are critical for memory processing in the hippocampal formation.

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making.

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

A neural mechanism for conserved value computations integrating information and rewards.

Behavioral and economic theory dictate that we decide between options based on their values. However, humans and animals eagerly seek information about uncertain future rewards, even when this does not provide any objective value. This implies that decisions are made by endowing information with subjective value and integrating it with the value of extrinsic rewards, but the mechanism is unknown. Here, we show that human and monkey value judgements obey strikingly conserved computational principles during multi-attribute decisions trading off information and extrinsic reward. We then identify a neural substrate in a highly conserved ancient structure, the lateral habenula (LHb). LHb neurons signal subjective value, integrating information's value with extrinsic rewards, and the LHb predicts and causally influences ongoing decisions. Neurons in key input areas to the LHb largely signal components of these computations, not integrated value signals. Thus, our data uncover neural mechanisms of conserved computations underlying decisions to seek information about the future.

Ethological computational psychiatry: Challenges and opportunities.

Studying the intricacies of individual subjects' moods and cognitive processing over extended periods of time presents a formidable challenge in medicine. While much of systems neuroscience appropriately focuses on the link between neural circuit functions and well-constrained behaviors over short timescales (e.g., trials, hours), many mental health conditions involve complex interactions of mood and cognition that are non-stationary across behavioral contexts and evolve over extended timescales. Here, we discuss opportunities, challenges, and possible future directions in computational psychiatry to quantify non-stationary continuously monitored behaviors. We suggest that this exploratory effort may contribute to a more precision-based approach to treating mental disorders and facilitate a more robust reverse translation across animal species. We conclude with ethical considerations for any field that aims to bridge artificial intelligence and patient monitoring.

Regionally distinct processing of rewards and punishments by the primate ventromedial prefrontal cortex.

The ventromedial prefrontal cortex (vmPFC) is thought to be related to emotional experience and to the processing of stimulus and action values. However, little is known about how single vmPFC neurons process the prediction and reception of rewards and punishments. We recorded from monkey vmPFC neurons in an experimental situation with alternating blocks, one in which rewards were delivered and one in which punishments were delivered. Many vmPFC neurons changed their activity between blocks. Importantly, neurons in ventral vmPFC were persistently more active in the appetitive "reward" block, whereas neurons in dorsal vmPFC were persistently more active in the aversive "punishment" block. Furthermore, within ventral vmPFC, posterior neurons phasically encoded probability of reward, whereas anterior neurons tonically encoded possibility of reward. We found multiple distinct nonlinear valuation mechanisms within the primate prefrontal cortex. Our findings suggest that different subregions of vmPFC contribute differentially to the processing of valence. By conveying such multidimensional and nonlinear signals, the vmPFC may enable flexible control of decisions and emotions to adapt to complex environments.

What and where information in the caudate tail guides saccades to visual objects.

We understand the world by making saccadic eye movements to various objects. However, it is unclear how a saccade can be aimed at a particular object, because two kinds of visual information, what the object is and where it is, are processed separately in the dorsal and ventral visual cortical pathways. Here, we provide evidence suggesting that a basal ganglia circuit through the tail of the monkey caudate nucleus (CDt) guides such object-directed saccades. First, many CDt neurons responded to visual objects depending on where and what the objects were. Second, electrical stimulation in the CDt induced saccades whose directions matched the preferred directions of neurons at the stimulation site. Third, many CDt neurons increased their activity before saccades directed to the preferred objects and directions of the neurons in a free-viewing condition. Our results suggest that CDt neurons receive both "what" and "where" information and guide saccades to visual objects.

Paired neuron recordings in the prefrontal and inferotemporal cortices reveal that spatial selection precedes object identification during visual search.

We addressed the question of how we locate and identify objects in complex natural environments by simultaneously recording single neurons from two brain regions that play different roles in this familiar activity--the frontal eye field (FEF), an area in the prefrontal cortex that is involved in visual spatial selection, and the inferotemporal cortex (IT), which is involved in object recognition--in monkeys performing a covert visual search task. Although the monkeys reported object identity, not location, neural activity specifying target location was evident in FEF before neural activity specifying target identity in IT. These two distinct processes were temporally correlated implying a functional linkage between the end stages of "where" and "what" visual processing and indicating that spatial selection is necessary for the formation of complex object representations associated with visual perception.

Dorsal raphe neurons signal integrated value during multi-attribute decision-making.

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.

The effects of prefrontal cortex inactivation on object responses of single neurons in the inferotemporal cortex during visual search.

Inferotemporal cortex (IT) is believed to be directly involved in object processing and necessary for accurate and efficient object recognition. The frontal eye field (FEF) is an area in the primate prefrontal cortex that is involved in visual spatial selection and is thought to guide spatial attention and eye movements. We show that object-selective responses of IT neurons and behavioral performance are affected by changes in frontal eye field activity. This was found in monkeys performing a search classification task by temporarily inactivating subregions of FEF while simultaneously recording the activity from single neurons in IT. The effect on object selectivity and performance was specific, occurring in a predictable spatially dependent manner and was strongest when the IT neuron's preferred target was presented in the presence of distractors. FEF inactivation did not affect IT responses on trials in which the nonpreferred target was presented in the search array.

Dopamine in the rodent tail of striatum regulates behavioral variability in response to threatening novel objects.

Mice display variability in fear-like responses to many external salient events, such as encountering unexpected novel objects, but the neural basis of this variability has been unclear. Akiti et al. (2022) demonstrate that dopamine in the tail of the rodent striatum predicts and regulates salience-related variability in individuals' behavioral responses to unexpected novel objects.

Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change.

Hypotheses and beliefs guide credit assignment - the process of determining which previous events or actions caused an outcome. Adaptive hypothesis formation and testing are crucial in uncertain and changing environments in which associations and meanings are volatile. Despite primates' abilities to form and test hypotheses, establishing what is causally responsible for the occurrence of particular outcomes remains a fundamental challenge for credit assignment and learning. Hypotheses about what surprises are due to stochasticity inherent in an environment as opposed to real, systematic changes are necessary for identifying the environment's predictive features, but are often hard to test. We review evidence that two highly interconnected frontal cortical regions, anterior cingulate cortex and ventrolateral prefrontal area 47/12o, provide a biological substrate for linking two crucial components of hypothesis-formation and testing: the control of information seeking and credit assignment. Neuroimaging, targeted disruptions, and neurophysiological studies link an anterior cingulate - 47/12o circuit to generation of exploratory behaviour, non-instrumental information seeking, and interpretation of subsequent feedback in the service of credit assignment. Our observations support the idea that information seeking and credit assignment are linked at the level of neural circuits and explain why this circuit is important for ensuring behaviour is flexible and adaptive.

Surprise and recency in novelty detection in the primate brain.

Primates and other animals must detect novel objects. However, the neuronal mechanisms of novelty detection remain unclear. Prominent theories propose that visual object novelty is either derived from the computation of recency (how long ago a stimulus was experienced) or is a form of sensory surprise (stimulus unpredictability). Here, we use high-channel electrophysiology in primates to show that in many primate prefrontal, temporal, and subcortical brain areas, object novelty detection is intertwined with the computations of recency and sensory surprise. Also, distinct circuits could be engaged by expected versus unexpected sensory surprise. Finally, we studied neuronal novelty-to-familiarity transformations during learning across many days. We found a diversity of timescales in neurons' learning rates and between-session forgetting rates, both within and across brain areas, that are well suited to support flexible behavior and learning in response to novelty. Our findings show that novelty sensitivity arises on multiple timescales across single neurons due to diverse but related computations of sensory surprise and recency and shed light on the computational underpinnings of novelty detection in the primate brain.

The zona incerta in control of novelty seeking and investigation across species.

Many organisms rely on a capacity to rapidly replicate, disperse, and evolve when faced with uncertainty and novelty. But mammals do not evolve and replicate quickly. They rely on a sophisticated nervous system to generate predictions and select responses when confronted with these challenges. An important component of their behavioral repertoire is the adaptive context-dependent seeking or avoiding of perceptually novel objects, even when their values have not yet been learned. Here, we outline recent cross-species breakthroughs that shed light on how the zona incerta (ZI), a relatively evolutionarily conserved brain area, supports novelty-seeking and novelty-related investigations. We then conjecture how the architecture of the ZI's anatomical connectivity - the wide-ranging top-down cortical inputs to the ZI, and its specifically strong outputs to both the brainstem action controllers and to brain areas involved in action value learning - place the ZI in a unique role at the intersection of cognitive control and learning.

A primate temporal cortex-zona incerta pathway for novelty seeking.

Primates interact with the world by exploring visual objects; they seek opportunities to view novel objects even when these have no extrinsic reward value. How the brain controls this novelty seeking is unknown. Here we show that novelty seeking in monkeys is regulated by the zona incerta (ZI). As monkeys made eye movements to familiar objects to trigger an opportunity to view novel objects, many ZI neurons were preferentially activated by predictions of novel objects before the gaze shift. Low-intensity ZI stimulation facilitated gaze shifts, whereas ZI inactivation reduced novelty seeking. ZI-dependent novelty seeking was not regulated by neurons in the lateral habenula or by many dopamine neurons in the substantia nigra, traditionally associated with reward seeking. But the anterior ventral medial temporal cortex, an area important for object vision and memory, was a prominent source of novelty predictions. These data uncover a functional pathway in the primate brain that regulates novelty seeking.

Laser stimulation of the skin for quantitative study of decision-making and motivation.

Neuroeconomics studies how decision-making is guided by the value of rewards and punishments. But to date, little is known about how noxious experiences impact decisions. A challenge is the lack of an aversive stimulus that is dynamically adjustable in intensity and location, readily usable over many trials in a single experimental session, and compatible with multiple ways to measure neuronal activity. We show that skin laser stimulation used in human studies of aversion can be used for this purpose in several key animal models. We then use laser stimulation to study how neurons in the orbitofrontal cortex (OFC), an area whose many roles include guiding decisions among different rewards, encode the value of rewards and punishments. We show that some OFC neurons integrated the positive value of rewards with the negative value of aversive laser stimulation, suggesting that the OFC can play a role in more complex choices than previously appreciated.

A prefrontal network integrates preferences for advance information about uncertain rewards and punishments.

Humans and animals can be strongly motivated to seek information to resolve uncertainty about rewards and punishments. In particular, despite its clinical and societal relevance, very little is known about information seeking about punishments. We show that attitudes toward information about punishments and rewards are distinct and separable at both behavioral and neuronal levels. We demonstrate the existence of prefrontal neuronal populations that anticipate opportunities to gain information in a relatively valence-specific manner, separately anticipating information about either punishments or rewards. These neurons are located in anatomically interconnected subregions of anterior cingulate cortex (ACC) and ventrolateral prefrontal cortex (vlPFC) in area 12o/47. Unlike ACC, vlPFC also contains a population of neurons that integrate attitudes toward both reward and punishment information, to encode the overall preference for information in a bivalent manner. This cortical network is well suited to mediate information seeking by integrating the desire to resolve uncertainty about multiple, distinct motivational outcomes.