Post-injury pain and behaviour: a control theory perspective.

Injuries of various types occur commonly in the lives of humans and other animals and lead to a pattern of persistent pain and recuperative behaviour that allows safe and effective recovery. In this Perspective, we propose a control-theoretic framework to explain the adaptive processes in the brain that drive physiological post-injury behaviour. We set out an evolutionary and ethological view on how animals respond to injury, illustrating how the behavioural state associated with persistent pain and recuperation may be just as important as phasic pain in ensuring survival. Adopting a normative approach, we suggest that the brain implements a continuous optimal inference of the current state of injury from diverse sensory and physiological signals. This drives the various effector control mechanisms of behavioural homeostasis, which span the modulation of ongoing motivation and perception to drive rest and hyper-protective behaviours. However, an inherent problem with this is that these protective behaviours may partially obscure information about whether injury has resolved. Such information restriction may seed a tendency to aberrantly or persistently infer injury, and may thus promote the transition to pathological chronic pain states.

Confidence of probabilistic predictions modulates the cortical response to pain.

Pain typically evolves over time, and the brain needs to learn this temporal evolution to predict how pain is likely to change in the future and orient behavior. This process is termed temporal statistical learning (TSL). Recently, it has been shown that TSL for pain sequences can be achieved using optimal Bayesian inference, which is encoded in somatosensory processing regions. Here, we investigate whether the confidence of these probabilistic predictions modulates the EEG response to noxious stimuli, using a TSL task. Confidence measures the uncertainty about the probabilistic prediction, irrespective of its actual outcome. Bayesian models dictate that the confidence about probabilistic predictions should be integrated with incoming inputs and weight learning, such that it modulates the early components of the EEG responses to noxious stimuli, and this should be captured by a negative correlation: when confidence is higher, the early neural responses are smaller as the brain relies more on expectations/predictions and less on sensory inputs (and vice versa). We show that participants were able to predict the sequence transition probabilities using Bayesian inference, with some forgetting. Then, we find that the confidence of these probabilistic predictions was negatively associated with the amplitude of the N2 and P2 components of the vertex potential: the more confident were participants about their predictions, the smaller the vertex potential. These results confirm key predictions of a Bayesian learning model and clarify the functional significance of the early EEG responses to nociceptive stimuli, as being implicated in confidence-weighted statistical learning.

Hierarchical models of pain: Inference, information-seeking, and adaptive control.

Computational models of pain consider how the brain processes nociceptive information and allow mapping neural circuits and networks to cognition and behaviour. To date, they have generally have assumed two largely independent processes: perceptual inference, typically modelled as an approximate Bayesian process, and action control, typically modelled as a reinforcement learning process. However, inference and control are intertwined in complex ways, challenging the clarity of this distinction. Here, we consider how they may comprise a parallel hierarchical architecture that combines inference, information-seeking, and adaptive value-based control. This sheds light on the complex neural architecture of the pain system, and takes us closer to understanding from where pain 'arises' in the brain.

Reward Enhances Pain Discrimination in Humans.

The notion that reward inhibits pain is a well-supported observation in both humans and animals, allowing suppression of pain reflexes to acquired rewarding stimuli. However, a blanket inhibition of pain by reward would also impair pain discrimination. In contrast, early counterconditioning experiments implied that reward might actually spare pain discrimination. To test this hypothesis, we investigated whether discriminative performance was enhanced or inhibited by reward. We found in adult human volunteers (N = 25) that pain-based discriminative ability is actually enhanced by reward, especially when reward is directly contingent on discriminative performance. Drift-diffusion modeling shows that this relates to an augmentation of the underlying sensory signal strength and is not merely an effect of decision bias. This enhancement of sensory-discriminative pain-information processing suggests that whereas reward can promote reward-acquiring behavior by inhibition of pain in some circumstances, it can also facilitate important discriminative information of the sensory input when necessary.

Resting-state Amplitude of Low-frequency Fluctuation is a Potentially Useful Prognostic Functional Biomarker in Cervical Myelopathy.

BACKGROUND: Cervical MRI is the standard diagnostic imaging technique for patients with cervical myelopathy. However, the utility of conventional cervical MRI as a predictive biomarker for surgical recovery remains unclear, partly because of the limited information obtained from this anatomically small area. Brain resting-state functional MRI (rs-fMRI) may help identify candidate predictive biomarkers. Two analytical methods that assess local spontaneous brain activity are widely used for rs-fMRI: functional connectivity between two brain regions and amplitude of low-frequency fluctuation (ALFF). In our previous analysis of functional connectivity, we discovered that brain functional connectivity may be a predictive biomarker for neurologic recovery in patients with cervical myelopathy; however, the functional connectivity analysis identified a correlation with only one clinical outcome (the 10-second test). To establish a comprehensive prediction measure, we need to explore other brain biomarkers that can predict recovery of other clinical outcomes in patients with cervical myelopathy. QUESTIONS/PURPOSES: We aimed to (1) elucidate preoperative ALFF alterations in patients with cervical myelopathy and how ALFF changes after surgery, with a focus on postoperative normalization and (2) establish a predictive model using preoperative ALFF by investigating the correlation between preoperative ALFF and postoperative clinical recovery in patients with cervical myelopathy. METHODS: Between August 2015 and June 2017, we treated 40 patients with cervical myelopathy. Thirty patients met our prespecified inclusion criteria, all were invited to participate, and 28 patients opted to do so (93%; 14 men and 14 women; mean age: 67 years). The 28 patients and 28 age- and sex-matched controls underwent rs-fMRI (twice for patients with cervical myelopathy: before and 6 months after cervical decompression surgery). We analyzed the same study population that was used in our earlier study investigating functional connectivity. Controls had none of the following abnormalities: neck or arm pain, visual or auditory disorders, cognitive disorder, structural brain disorder, a history of brain surgery, mental and neurologic disorders, and medications for the central nervous system. We performed ALFF comparisons between preoperative patients with cervical myelopathy and controls, analyzed postoperative ALFF changes in patients with cervical myelopathy, and performed a correlation analysis between preoperative ALFF and clinical recovery in these patients. Clinical outcomes in the cervical myelopathy group were assessed using the 10-second test, the Japanese Orthopaedic Association upper-extremity motor (JOA-UEM) score, JOA upper-extremity sensory score (JOA-UES), and Japanese Orthopaedic Association Cervical Myelopathy Evaluation Questionnaire for upper-extremity function (JOACMEQ-UEF) score before and 6 months after surgery, which is when we believe these scores generally reach a plateau. A total of 93% of those enrolled (26 of 28 patients) were analyzed both preoperatively and postoperatively; the other two were lost to follow-up. RESULTS: The cervical myelopathy group had an increase in ALFF in the bilateral primary sensorimotor cortices (right, cluster size = 850 voxels, t-value = 6.10; left, cluster size = 370 voxels, t-value = 4.84) and left visual cortex (cluster size = 556 voxels, t-value = 4.21) compared with the control group. The cervical myelopathy group had a decrease in ALFF in the bilateral posterior supramarginal gyrus (right, cluster size = 222 voxels, t-value = 5.09; left, cluster size = 436 voxels, t-value = 5.28). After surgery, the bilateral sensorimotor cortices (right, cluster size = 468 voxels, t-value = 6.74; left, cluster size = 167 voxels, t-value = 5.40) and left visual cortex (cluster size = 3748 voxels, t-value = 6.66) showed decreased ALFF compared with preoperative ALFF, indicating postoperative normalization of spontaneous brain activities in these regions. However, the bilateral posterior supramarginal gyrus did not show an increase in ALFF postoperatively, although ALFF in this region decreased preoperatively. Greater levels of ALFF at the left and right frontal pole and left pars opercularis of the inferior frontal gyrus before surgery in the cervical myelopathy group were correlated with larger improvements in the JOACMEQ-UEF score 6 months after surgery (r = 0.784; p < 0.001, r = 0.734; p < 0.001 and r = 0.770, respectively; p < 0.001). The prediction formula, based on preoperative ALFF values in the left frontal pole, was as follows: the predicted postoperative improvement in the JOACMEQ-UEF score = 34.6 × preoperative ALFF value - 7.0 (r = 0.614; p < 0.001). CONCLUSIONS: Our findings suggest that preoperative ALFF may be a biomarker for postoperative recovery in that it predicted postoperative JOACMEQ-UEF scores. To establish a comprehensive prediction measure for neurologic recovery in patients with cervical myelopathy, a multicenter study is underway. LEVEL OF EVIDENCE: Level II, diagnostic study.

Thermosensory Perceptual Learning Is Associated with Structural Brain Changes in Parietal-Opercular (SII) Cortex.

The location of a sensory cortex for temperature perception remains a topic of substantial debate. Both the parietal-opercular (SII) and posterior insula have been consistently implicated in thermosensory processing, but neither region has yet been identified as the locus of fine temperature discrimination. Using a perceptual learning paradigm in male and female humans, we show improvement in discrimination accuracy for subdegree changes in both warmth and cool detection over 5 d of repetitive training. We found that increases in discriminative accuracy were specific to the temperature (cold or warm) being trained. Using structural imaging to look for plastic changes associated with perceptual learning, we identified symmetrical increases in gray matter volume in the SII cortex. Furthermore, we observed distinct, adjacent regions for cold and warm discrimination, with cold discrimination having a more anterior locus than warm. The results suggest that thermosensory discrimination is supported by functionally and anatomically distinct temperature-specific modules in the SII cortex.SIGNIFICANCE STATEMENT We provide behavioral and neuroanatomical evidence that perceptual learning is possible within the temperature system. We show that structural plasticity localizes to parietal-opercular (SII), and not posterior insula, providing the best evidence to date resolving a longstanding debate about the location of putative "temperature cortex." Furthermore, we show that cold and warm pathways are behaviorally and anatomically dissociable, suggesting that the temperature system has distinct temperature-dependent processing modules.

Pain: a distributed brain information network?

Understanding how pain is processed in the brain has been an enduring puzzle, because there doesn't appear to be a single "pain cortex" that directly codes the subjective perception of pain. An emerging concept is that, instead, pain might emerge from the coordinated activity of an integrated brain network. In support of this view, Woo and colleagues present evidence that distinct brain networks support the subjective changes in pain that result from nociceptive input and self-directed cognitive modulation. This evidence for the sensitivity of distinct neural subsystems to different aspects of pain opens up the way to more formal computational network theories of pain.

The habenula encodes negative motivational value associated with primary punishment in humans.

Learning what to approach, and what to avoid, involves assigning value to environmental cues that predict positive and negative events. Studies in animals indicate that the lateral habenula encodes the previously learned negative motivational value of stimuli. However, involvement of the habenula in dynamic trial-by-trial aversive learning has not been assessed, and the functional role of this structure in humans remains poorly characterized, in part, due to its small size. Using high-resolution functional neuroimaging and computational modeling of reinforcement learning, we demonstrate positive habenula responses to the dynamically changing values of cues signaling painful electric shocks, which predict behavioral suppression of responses to those cues across individuals. By contrast, negative habenula responses to monetary reward cue values predict behavioral invigoration. Our findings show that the habenula plays a key role in an online aversive learning system and in generating associated motivated behavior in humans.

Distinct Contributions of Ventromedial and Dorsolateral Subregions of the Human Substantia Nigra to Appetitive and Aversive Learning.

The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain in contributing to the elicitation of reward prediction errors during appetitive learning has been well established. Less is known about the differential contribution of these midbrain regions to appetitive versus aversive learning, especially in humans. Here we scanned human participants with high-resolution fMRI focused on the SN and VTA while they participated in a sequential Pavlovian conditioning paradigm involving an appetitive outcome (a pleasant juice), as well as an aversive outcome (an unpleasant bitter and salty flavor). We found a degree of regional specialization within the SN: Whereas a region of ventromedial SN correlated with a temporal difference reward prediction error during appetitive Pavlovian learning, a dorsolateral area correlated instead with an aversive expected value signal in response to the most distal cue, and to a reward prediction error in response to the most proximal cue to the aversive outcome. Furthermore, participants' affective reactions to both the appetitive and aversive conditioned stimuli more than 1 year after the fMRI experiment was conducted correlated with activation in the ventromedial and dorsolateral SN obtained during the experiment, respectively. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts. SIGNIFICANCE STATEMENT: The role of the substantia nigra (SN) and ventral tegmental area (VTA) in appetitive learning is well established, but less is known about their contribution to aversive compared with appetitive learning, especially in humans. We used high-resolution fMRI to measure activity in the SN and VTA while participants underwent higher-order Pavlovian learning. We found a regional specialization within the SN: a ventromedial area was selectively engaged during appetitive learning, and a dorsolateral area during aversive learning. Activity in these areas predicted affective reactions to appetitive and aversive conditioned stimuli over 1 year later. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts.

Anticipation and choice heuristics in the dynamic consumption of pain relief.

Humans frequently need to allocate resources across multiple time-steps. Economic theory proposes that subjects do so according to a stable set of intertemporal preferences, but the computational demands of such decisions encourage the use of formally less competent heuristics. Few empirical studies have examined dynamic resource allocation decisions systematically. Here we conducted an experiment involving the dynamic consumption over approximately 15 minutes of a limited budget of relief from moderately painful stimuli. We had previously elicited the participants' time preferences for the same painful stimuli in one-off choices, allowing us to assess self-consistency. Participants exhibited three characteristic behaviors: saving relief until the end, spreading relief across time, and early spending, of which the last was markedly less prominent. The likelihood that behavior was heuristic rather than normative is suggested by the weak correspondence between one-off and dynamic choices. We show that the consumption choices are consistent with a combination of simple heuristics involving early-spending, spreading or saving of relief until the end, with subjects predominantly exhibiting the last two.

Relative valuation of pain in human orbitofrontal cortex.

The valuation of health-related states, including pain, is a critical issue in clinical practice, health economics, and pain neuroscience. Surprisingly the monetary value people associate with pain is highly context-dependent, with participants willing to pay more to avoid medium-level pain when presented in a context of low-intensity, rather than high-intensity, pain. Here, we ask whether context impacts upon the neural representation of pain itself, or alternatively the transformation of pain into valuation-driven behavior. While undergoing fMRI, human participants declared how much money they would be willing to pay to avoid repeated instances of painful cutaneous electrical stimuli delivered to the foot. We also implemented a contextual manipulation that involved presenting medium-level painful stimuli in blocks with either low- or high-level stimuli. We found no evidence of context-dependent activity within a conventional "pain matrix," where pain-evoked activity reflected absolute stimulus intensity. By contrast, in right lateral orbitofrontal cortex, a strong contextual dependency was evident, and here activity tracked the contextual rank of the pain. The findings are in keeping with an architecture where an absolute pain valuation system and a rank-dependent system interact to influence willing to pay to avoid pain, with context impacting value-based behavior high in a processing hierarchy. This segregated processing hints that distinct neural representations reflect sensory aspects of pain and components that are less directly nociceptive whose integration also guides pain-related actions. A dominance of the latter might account for puzzling phenomena seen in somatization disorders where perceived pain is a dominant driver of behavior.

Uncertainty increases pain: evidence for a novel mechanism of pain modulation involving the periaqueductal gray.

Predictions about sensory input exert a dominant effect on what we perceive, and this is particularly true for the experience of pain. However, it remains unclear what component of prediction, from an information-theoretic perspective, controls this effect. We used a vicarious pain observation paradigm to study how the underlying statistics of predictive information modulate experience. Subjects observed judgments that a group of people made to a painful thermal stimulus, before receiving the same stimulus themselves. We show that the mean observed rating exerted a strong assimilative effect on subjective pain. In addition, we show that observed uncertainty had a specific and potent hyperalgesic effect. Using computational functional magnetic resonance imaging, we found that this effect correlated with activity in the periaqueductal gray. Our results provide evidence for a novel form of cognitive hyperalgesia relating to perceptual uncertainty, induced here by vicarious observation, with control mediated by the brainstem pain modulatory system.

Dread and the disvalue of future pain.

Standard theories of decision-making involving delayed outcomes predict that people should defer a punishment, whilst advancing a reward. In some cases, such as pain, people seem to prefer to expedite punishment, implying that its anticipation carries a cost, often conceptualized as 'dread'. Despite empirical support for the existence of dread, whether and how it depends on prospective delay is unknown. Furthermore, it is unclear whether dread represents a stable component of value, or is modulated by biases such as framing effects. Here, we examine choices made between different numbers of painful shocks to be delivered faithfully at different time points up to 15 minutes in the future, as well as choices between hypothetical painful dental appointments at time points of up to approximately eight months in the future, to test alternative models for how future pain is disvalued. We show that future pain initially becomes increasingly aversive with increasing delay, but does so at a decreasing rate. This is consistent with a value model in which moment-by-moment dread increases up to the time of expected pain, such that dread becomes equivalent to the discounted expectation of pain. For a minority of individuals pain has maximum negative value at intermediate delay, suggesting that the dread function may itself be prospectively discounted in time. Framing an outcome as relief reduces the overall preference to expedite pain, which can be parameterized by reducing the rate of the dread-discounting function. Our data support an account of disvaluation for primary punishments such as pain, which differs fundamentally from existing models applied to financial punishments, in which dread exerts a powerful but time-dependent influence over choice.

Statistical learning shapes pain perception and prediction independently of external cues.

The placebo and nocebo effects highlight the importance of expectations in modulating pain perception, but in everyday life we don't need an external source of information to form expectations about pain. The brain can learn to predict pain in a more fundamental way, simply by experiencing fluctuating, non-random streams of noxious inputs, and extracting their temporal regularities. This process is called statistical learning. Here, we address a key open question: does statistical learning modulate pain perception? We asked 27 participants to both rate and predict pain intensity levels in sequences of fluctuating heat pain. Using a computational approach, we show that probabilistic expectations and confidence were used to weigh pain perception and prediction. As such, this study goes beyond well-established conditioning paradigms associating non-pain cues with pain outcomes, and shows that statistical learning itself shapes pain experience. This finding opens a new path of research into the brain mechanisms of pain regulation, with relevance to chronic pain where it may be dysfunctional.

Serotonin selectively modulates reward value in human decision-making.

Establishing a function for the neuromodulator serotonin in human decision-making has proved remarkably difficult because if its complex role in reward and punishment processing. In a novel choice task where actions led concurrently and independently to the stochastic delivery of both money and pain, we studied the impact of decreased brain serotonin induced by acute dietary tryptophan depletion. Depletion selectively impaired both behavioral and neural representations of reward outcome value, and hence the effective exchange rate by which rewards and punishments were compared. This effect was computationally and anatomically distinct from a separate effect on increasing outcome-independent choice perseveration. Our results provide evidence for a surprising role for serotonin in reward processing, while illustrating its complex and multifarious effects.

Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease.

The role dopamine plays in decision-making has important theoretical, empirical and clinical implications. Here, we examined its precise contribution by exploiting the lesion deficit model afforded by Parkinson's disease. We studied patients in a two-stage reinforcement learning task, while they were ON and OFF dopamine replacement medication. Contrary to expectation, we found that dopaminergic drug state (ON or OFF) did not impact learning. Instead, the critical factor was drug state during the performance phase, with patients ON medication choosing correctly significantly more frequently than those OFF medication. This effect was independent of drug state during initial learning and appears to reflect a facilitation of generalization for learnt information. This inference is bolstered by our observation that neural activity in nucleus accumbens and ventromedial prefrontal cortex, measured during simultaneously acquired functional magnetic resonance imaging, represented learnt stimulus values during performance. This effect was expressed solely during the ON state with activity in these regions correlating with better performance. Our data indicate that dopamine modulation of nucleus accumbens and ventromedial prefrontal cortex exerts a specific effect on choice behaviour distinct from pure learning. The findings are in keeping with the substantial other evidence that certain aspects of learning are unaffected by dopamine lesions or depletion, and that dopamine plays a key role in performance that may be distinct from its role in learning.

Evidence for dopaminergic involvement in endogenous modulation of pain relief.

Relief of ongoing pain is a potent motivator of behavior, directing actions to escape from or reduce potentially harmful stimuli. Whereas endogenous modulation of pain events is well characterized, relatively little is known about the modulation of pain relief and its corresponding neurochemical basis. Here, we studied pain modulation during a probabilistic relief-seeking task (a 'wheel of fortune' gambling task), in which people actively or passively received reduction of a tonic thermal pain stimulus. We found that relief perception was enhanced by active decisions and unpredictability, and greater in high novelty-seeking trait individuals, consistent with a model in which relief is tuned by its informational content. We then probed the roles of dopaminergic and opioidergic signaling, both of which are implicated in relief processing, by embedding the task in a double-blinded cross-over design with administration of the dopamine precursor levodopa and the opioid receptor antagonist naltrexone. We found that levodopa enhanced each of these information-specific aspects of relief modulation but no significant effects of the opioidergic manipulation. These results show that dopaminergic signaling has a key role in modulating the perception of pain relief to optimize motivation and behavior.

Multimodal physiological sensing for the assessment of acute pain.

Pain assessment is a challenging task encountered by clinicians. In clinical settings, patients' self-report is considered the gold standard in pain assessment. However, patients who are unable to self-report pain are at a higher risk of undiagnosed pain. In the present study, we explore the use of multiple sensing technologies to monitor physiological changes that can be used as a proxy for objective measurement of acute pain. Electrodermal activity (EDA), photoplethysmography (PPG), and respiration (RESP) signals were collected from 22 participants under two pain intensities (low and high) and on two different anatomical locations (forearm and hand). Three machine learning models were implemented, including support vector machines (SVM), decision trees (DT), and linear discriminant analysis (LDA) for the identification of pain. Various pain scenarios were investigated, identification of pain (no pain, pain), multiclass (no pain, low pain, high pain), and identification of pain location (forearm, hand). Reference classification results from individual sensors and from all sensors together were obtained. After feature selection, results showed that EDA was the most informative sensor in the three pain conditions, 93.2±8% in identification of pain, 68.9±10% in the multiclass problem, and 56.0±8% for the identification of pain location. These results identify EDA as the superior sensor in our experimental conditions. Future work is required to validate the obtained features to improve its feasibility in more realistic scenarios. Finally, this study proposes EDA as a candidate to design a tool that can assist clinicians in the assessment of acute pain of nonverbal patients.

Improving sleep and learning in rehabilitation after stroke, part 2 (INSPIRES2): study protocol for a home-based randomised control trial of digital cognitive behavioural therapy (dCBT) for insomnia.

INTRODUCTION: Consolidation of motor skill learning, a key component of rehabilitation post-stroke, is known to be sleep dependent. However, disrupted sleep is highly prevalent after stroke and is often associated with poor motor recovery and quality of life. Previous research has shown that digital cognitive behavioural therapy (dCBT) for insomnia can be effective at improving sleep quality after stroke. Therefore, the aim of this trial is to evaluate the potential for sleep improvement using a dCBT programme, to improve rehabilitation outcomes after stroke. METHODS AND ANALYSIS: We will conduct a parallel-arm randomised controlled trial of dCBT (Sleepio) versus treatment as usual among individuals following stroke affecting the upper limb. Up to 100 participants will be randomly allocated (2:1) into either the intervention (6-8 week dCBT) or control (continued treatment as usual) group. The primary outcome of the study will be change in insomnia symptoms pre to post intervention compared with treatment as usual. Secondary outcomes include improvement in overnight motor memory consolidation and sleep measures between intervention groups, correlations between changes in sleep behaviour and overnight motor memory consolidation in the dCBT group and changes in symptoms of depression and fatigue between the dCBT and control groups. Analysis of covariance models and correlations will be used to analyse data from the primary and secondary outcomes. ETHICS AND DISSEMINATION: The study has received approval from the National Research Ethics Service (22/EM/0080), Health Research Authority (HRA) and Health and Care Research Wales (HCRW), IRAS ID: 306 291. The results of this trial will be disseminated via presentations at scientific conferences, peer-reviewed publication, public engagement events, stakeholder organisations and other forms of media where appropriate. TRIAL REGISTRATION NUMBER: NCT05511285.

Cortical substrates for exploratory decisions in humans.

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.