On the Role of Theory and Modeling in Neuroscience.

In recent years, the field of neuroscience has gone through rapid experimental advances and a significant increase in the use of quantitative and computational methods. This growth has created a need for clearer analyses of the theory and modeling approaches used in the field. This issue is particularly complex in neuroscience because the field studies phenomena that cross a wide range of scales and often require consideration at varying degrees of abstraction, from precise biophysical interactions to the computations they implement. We argue that a pragmatic perspective of science, in which descriptive, mechanistic, and normative models and theories each play a distinct role in defining and bridging levels of abstraction, will facilitate neuroscientific practice. This analysis leads to methodological suggestions, including selecting a level of abstraction that is appropriate for a given problem, identifying transfer functions to connect models and data, and the use of models themselves as a form of experiment.

Optogenetic disruption of the prelimbic cortex alters long-term decision strategy but not valuation on a spatial delay discounting task.

Rats demonstrate a preference for smaller, immediate rewards over larger, delayed ones, a phenomenon known as delay-discounting (DD). Behavior arises from the interaction of multiple decision-making systems, and the medial prefrontal cortex (mPFC) has been identified as a central component in the mediation between these decision systems. To investigate the role of the prelimbic (PL) subregion of mPFC on decision strategy interaction, we compared two cohorts of rats (ChR2-opsin-expressing 'Active' and opsin-absent 'Control') on a spatial delay-discounting task while delivering in-vivo light stimulation into PL at the choice point of select trials. By analyzing the overall delay-adjustment along with deliberative and procedural behavioral strategy markers, our study revealed differences in the decision strategies used between the active and control animals despite both groups showing similar valuations. Control animals developed the expected shift from deliberative to procedural decision strategy on this task (indicated by reaching delay-stability, particularly during late-session laps); however, active-virus animals repeatedly over-adjusted around their preferred delay throughout the entire session, suggesting a significant deficit in procedural decision-making on this task. Active animals showed a significant decrease in proportion of vicarious trial and error events (VTE, a behavior correlated with deliberative processes) on delay adjustment laps relative to control animals. This points to a more nuanced role for VTE, not just in executing deliberation, but in shifting from deliberative to procedural processes. This opto-induced change in VTE was especially pronounced for late-session adjustment laps. We found no other session-by-session or lap-by-lap effects, leaving a particular role for PL in the long-term development of procedural strategies on this task.

Dorsolateral Striatal Task-initiation Bursts Represent Past Experiences More than Future Action Plans.

The dorsolateral striatum (DLS) is involved in learning and executing procedural actions. Cell ensembles in the DLS, but not the dorsomedial striatum (DMS), exhibit a burst of firing at the start of a well-learned action sequence ("task-bracketing"). However, it is currently unclear what information is contained in these bursts. Some theories suggest that these bursts should represent the procedural action sequence itself (that they should be about future action chains), whereas others suggest that they should contain representations of the current state of the world, taking into account primarily past information. In addition, the DLS local field potential shows transient bursts of power in the 50 Hz range (γ50) around the time a learned action sequence is initiated. However, it is currently unknown how bursts of activity in DLS cell ensembles and bursts of γ50 power in the DLS local field potential are related to each other. We found that DLS bursts at lap initiation in rats represented recently experienced reward locations more than future procedural actions, indicating that task-initiation DLS bursts contain primarily retrospective, rather than prospective, information to guide procedural actions. Furthermore, representations of past reward locations increased during periods of increased γ50 power in the DLS. There was no evidence of task-initiation bursts, increased γ50 power, or retrospective reward location information in the neighboring dorsomedial striatum. These data support a role for the DLS in model-free theories of procedural decision-making over planned action-chain theories, suggesting that procedural actions derive from representations of the current and recent past.SIGNIFICANCE STATEMENT While it is well-established that the dorsolateral striatum (DLS) plays a critical role in procedural decision-making, open questions remain about the kinds of representations contained in DLS ensemble activity that guide procedural actions. We found that DLS, but not DMS, cell ensembles contained nonlocal representations of past reward locations that appear moments before task-initiation DLS bursts. These retrospective representations were temporally linked to a rise in γ50 power that also preceded the characteristic DLS burst at task-initiation. These results support models of procedural decision-making based on associations between available actions and the current state of the world over models based on planning over action-chains.

Global disruption in excitation-inhibition balance can cause localized network dysfunction and Schizophrenia-like context-integration deficits.

Poor context integration, the process of incorporating both previous and current information in decision making, is a cognitive symptom of schizophrenia. The maintenance of the contextual information has been shown to be sensitive to changes in excitation-inhibition (EI) balance. Many regions of the brain are sensitive to EI imbalances, however, so it is unknown how systemic manipulations affect the specific regions that are important to context integration. We constructed a multi-structure, biophysically-realistic agent that could perform context-integration as is assessed by the dot pattern expectancy task. The agent included a perceptual network, a memory network, and a decision making system and was capable of successfully performing the dot pattern expectancy task. Systemic manipulation of the agent's EI balance produced localized dysfunction of the memory structure, which resulted in schizophrenia-like deficits at context integration. When the agent's pyramidal cells were less excitatory, the agent fixated upon the cue and initiated responding later than the default agent, which were like the deficits one would predict that individuals on the autistic spectrum would make. This modelling suggests that it may be possible to parse between different types of context integration deficits by adding distractors to context integration tasks and by closely examining a participant's reaction times.

Beyond Description and Deficits: How Computational Psychiatry Can Enhance an Understanding of Decision-Making in Anorexia Nervosa.

PURPOSE OF REVIEW: Despite decades of research, knowledge of the mechanisms maintaining anorexia nervosa (AN) remains incomplete and clearly effective treatments elusive. Novel theoretical frameworks are needed to advance mechanistic and treatment research for this disorder. Here, we argue the utility of engaging a novel lens that differs from existing perspectives in psychiatry. Specifically, we argue the necessity of expanding beyond two historically common perspectives: (1) the descriptive perspective: the tendency to define mechanisms on the basis of surface characteristics and (2) the deficit perspective: the tendency to search for mechanisms associated with under-functioning of decision-making abilities and related circuity, rather than problems of over-functioning, in psychiatric disorders. RECENT FINDINGS: Computational psychiatry can provide a novel framework for understanding AN because this approach emphasizes the role of computational misalignments (rather than absolute deficits or excesses) between decision-making strategies and environmental demands as the key factors promoting psychiatric illnesses. Informed by this approach, we argue that AN can be understood as a disorder of excess goal pursuit, maintained by over-engagement, rather than disengagement, of executive functioning strategies and circuits. Emerging evidence suggests that this same computational imbalance may constitute an under-investigated phenotype presenting transdiagnostically across psychiatric disorders. A variety of computational models can be used to further elucidate excess goal pursuit in AN. Most traditional psychiatric treatments do not target excess goal pursuit or associated neurocognitive mechanisms. Thus, targeting at the level of computational dysfunction may provide a new avenue for enhancing treatment for AN and related disorders.

Disrupting the medial prefrontal cortex with designer receptors exclusively activated by designer drug alters hippocampal sharp-wave ripples and their associated cognitive processes.

The hippocampus and medial prefrontal cortex (mPFC) interact during a myriad of cognitive processes including decision-making and long-term memory consolidation. Exactly how the mPFC and hippocampus interact during goal-directed decision-making remains to be fully elucidated. During periods of rest, bursts of high-frequency oscillations, termed sharp-wave ripple (SWR), appear in the local field potential. Impairing SWRs on the maze or during post-learning rest can interfere with memory-guided decision-making and memory consolidation. We hypothesize that the hippocampus and mPFC bidirectionally interact during SWRs to support memory consolidation and decision-making. Rats were trained on the neuroeconomic spatial decision-making task, Restaurant Row, to make serial stay-skip decisions where the amount of effort (delay to reward) varied upon entry to each restaurant. Hippocampal cells and SWRs were recorded in rats with the mPFC transduced with inhibitory DREADDs. We found that disrupting the mPFC impaired consolidating SWRs in the hippocampus. Hippocampal SWR rates depended on the internalized value of the reward (derived from individual flavor preferences), a parameter important in decision-making, and disrupting the mPFC changed this relationship. Additionally, we found a dissociation between SWRs that occurred while rats were on the maze dependent upon whether those SWRs occurred while the rat was anticipating food reward or during post-reward consumption.

Vicarious trial and error.

When rats come to a decision point, they sometimes pause and look back and forth as if deliberating over the choice; at other times, they proceed as if they have already made their decision. In the 1930s, this pause-and-look behaviour was termed 'vicarious trial and error' (VTE), with the implication that the rat was 'thinking about the future'. The discovery in 2007 that the firing of hippocampal place cells gives rise to alternating representations of each of the potential path options in a serial manner during VTE suggested a possible neural mechanism that could underlie the representations of future outcomes. More-recent experiments examining VTE in rats suggest that there are direct parallels to human processes of deliberative decision making, working memory and mental time travel.

Mice learn to avoid regret.

Regret can be defined as the subjective experience of recognizing that one has made a mistake and that a better alternative could have been selected. The experience of regret is thought to carry negative utility. This typically takes two distinct forms: augmenting immediate postregret valuations to make up for losses, and augmenting long-term changes in decision-making strategies to avoid future instances of regret altogether. While the short-term changes in valuation have been studied in human psychology, economics, neuroscience, and even recently in nonhuman-primate and rodent neurophysiology, the latter long-term process has received far less attention, with no reports of regret avoidance in nonhuman decision-making paradigms. We trained 31 mice in a novel variant of the Restaurant Row economic decision-making task, in which mice make decisions of whether to spend time from a limited budget to achieve food rewards of varying costs (delays). Importantly, we tested mice longitudinally for 70 consecutive days, during which the task provided their only source of food. Thus, decision strategies were interdependent across both trials and days. We separated principal commitment decisions from secondary reevaluation decisions across space and time and found evidence for regret-like behaviors following change-of-mind decisions that corrected prior economically disadvantageous choices. Immediately following change-of-mind events, subsequent decisions appeared to make up for lost effort by altering willingness to wait, decision speed, and pellet consumption speed, consistent with past reports of regret in rodents. As mice were exposed to an increasingly reward-scarce environment, we found they adapted and refined distinct economic decision-making strategies over the course of weeks to maximize reinforcement rate. However, we also found that even without changes in reinforcement rate, mice transitioned from an early strategy rooted in foraging to a strategy rooted in deliberation and planning that prevented future regret-inducing change-of-mind episodes from occurring. These data suggest that mice are learning to avoid future regret, independent of and separate from reinforcement rate maximization.

Altering gain of the infralimbic-to-accumbens shell circuit alters economically dissociable decision-making algorithms.

The nucleus accumbens shell (NAcSh) is involved in reward valuation. Excitatory projections from infralimbic cortex (IL) to NAcSh undergo synaptic remodeling in rodent models of addiction and enable the extinction of disadvantageous behaviors. However, how the strength of synaptic transmission of the IL-NAcSh circuit affects decision-making information processing and reward valuation remains unknown, particularly because these processes can conflict within a given trial and particularly given recent data suggesting that decisions arise from separable information-processing algorithms. The approach of many neuromodulation studies is to disrupt information flow during on-going behaviors; however, this limits the interpretation of endogenous encoding of computational processes. Furthermore, many studies are limited by the use of simple behavioral tests of value which are unable to dissociate neurally distinct decision-making algorithms. We optogenetically altered the strength of synaptic transmission between glutamatergic IL-NAcSh projections in mice trained on a neuroeconomic task capable of separating multiple valuation processes. We found that induction of long-term depression in these synapses produced lasting changes in foraging processes without disrupting deliberative processes. Mice displayed inflated reevaluations to stay when deciding whether to abandon continued reward-seeking investments but displayed no changes during initial commitment decisions. We also developed an ensemble-level measure of circuit-specific plasticity that revealed individual differences in foraging valuation tendencies. Our results demonstrate that alterations in projection-specific synaptic strength between the IL and the NAcSh are capable of augmenting self-control economic valuations within a particular decision-making modality and suggest that the valuation mechanisms for these multiple decision-making modalities arise from different circuits.

Spatial encoding in dorsomedial prefrontal cortex and hippocampus is related during deliberation.

Deliberation is thought to involve the internal simulation of the outcomes of candidate actions, the valuation of those outcomes, and the selection of the actions with the highest expected value. While it is known that deliberation involves prefrontal cortical areas, specifically the dorsomedial prefrontal cortex (dmPFC), as well as the hippocampus (HPC) and other brain regions, how these areas process prospective information and select actions is not well understood. We recorded simultaneously from ensembles in dmPFC and CA1 of dorsal HPC in rats during performance of a spatial contingency switching task, and examined the relationships between spatial and reward encoding in these two areas during deliberation at the choice point. We found that CA1 and dmPFC represented either goal locations or the current position simultaneously, but that when goal locations were encoded, HPC and dmPFC did not always represent the same goal location. Ensemble activity in dmPFC predicted when HPC would represent goal locations, but on a broad timescale on the order of seconds. Also, reward encoding in dmPFC increased during hippocampal theta cycles where CA1 ensembles represented the goal location. These results suggest that dmPFC and HPC share prospective information during deliberation, that dmPFC may influence whether HPC represents prospective information, and that information recalled about goal locations by HPC may be integrated into dmPFC reward representations on fast timescales.

Dorsomedial prefrontal cortex and hippocampus represent strategic context even while simultaneously changing representation throughout a task session.

Dorsomedial prefrontal cortex (dmPFC) and hippocampus (HPC) are thought to play complementary roles in a spatial working memory and decision-making network, where spatial information from HPC informs representations in dmPFC, and contextual information from dmPFC biases how HPC recalls that information. We recorded simultaneously from neural ensembles in rodent dmPFC and HPC as rats performed a rule-switching task, and found that ensembles in dmPFC and HPC simultaneously encoded task contingencies and other time-varying information. While ensembles in HPC transitioned to represent new contingencies at the same time as rats updated their strategies to be consistent with the new contingency, dmPFC ensembles transitioned earlier. Neural representations of other time-varying information also changed faster in dmPFC than in HPC. Our results suggest that HPC and dmPFC represent contingencies while simultaneously representing other information which changes over time, and that this contextual information is integrated into hippocampal representations more slowly than in dmPFC.

Disrupting the medial prefrontal cortex alters hippocampal sequences during deliberative decision making.

Current theories of deliberative decision making suggest that deliberative decisions arise from imagined simulations that require interactions between the prefrontal cortex and hippocampus. In rodent navigation experiments, hippocampal theta sequences advance from the location of the rat ahead to the subsequent goal. To examine the role of the medial prefrontal cortex (mPFC) on the hippocampus, we disrupted the mPFC with DREADDs (designer receptors exclusively activated by designer drugs). Using the Restaurant Row foraging task, we found that mPFC disruption resulted in decreased vicarious trial and error behavior, reduced the number of theta sequences, and impaired theta sequences in hippocampus. mPFC disruption led to larger changes in the initiation of the hippocampal theta sequences that represent the current location of the rat rather than to the later portions that represent the future outcomes. These data suggest that the mPFC likely provides an important component to the initiation of deliberative sequences and provides support for an episodic-future thinking, working memory interpretation of deliberation. NEW & NOTEWORTHY The medial prefrontal cortex (mPFC) and hippocampus interact during deliberative decision making. Disruption of the mPFC impaired hippocampal processes, including the local and nonlocal representations of space along each theta cycle and the initiation of hippocampal theta sequences, while sparing place cell firing characteristics and phase precession. mPFC disruption reduced the deliberative behavioral process vicarious trial and error and improved economic behaviors on this task.

Neural signatures underlying deliberation in human foraging decisions.

Humans have a remarkable capacity to mentally project themselves far ahead in time. This ability, which entails the mental simulation of events, is thought to be fundamental to deliberative decision making, as it allows us to search through and evaluate possible choices. Many decisions that humans make are foraging decisions, in which one must decide whether an available offer is worth taking, when compared to unknown future possibilities (i.e., the background). Using a translational decision-making paradigm designed to reveal decision preferences in rats, we found that humans engaged in deliberation when making foraging decisions. A key feature of this task is that preferences (and thus, value) are revealed as a function of serial choices. Like rats, humans also took longer to respond when faced with difficult decisions near their preference boundary, which was associated with prefrontal and hippocampal activation, exemplifying cross-species parallels in deliberation. Furthermore, we found that voxels within the visual cortices encoded neural representations of the available possibilities specifically following regret-inducing experiences, in which the subject had previously rejected a good offer only to encounter a low-valued offer on the subsequent trial.

Taking an engineer's view: Implications of network analysis for computational psychiatry.

An engineer's viewpoint on psychiatry asks: What are the failure modes that underlie psychiatric dysfunction? And: How can we modify the system? Psychiatry has made great strides in understanding and treating disorders using biology; however, failure modes and modification access points can also exist extrinsically in environmental interactions. The network analysis suggested by Borsboom et al. in the target article provides a new viewpoint that should be incorporated into current theoretical constructs, not placed in opposition to them.

Beyond simple tests of value: measuring addiction as a heterogeneous disease of computation-specific valuation processes.

Addiction is considered to be a neurobiological disorder of learning and memory because addiction is capable of producing lasting changes in the brain. Recovering addicts chronically struggle with making poor decisions that ultimately lead to relapse, suggesting a view of addiction also as a neurobiological disorder of decision-making information processing. How the brain makes decisions depends on how decision-making processes access information stored as memories in the brain. Advancements in circuit-dissection tools and recent theories in neuroeconomics suggest that neurally dissociable valuation processes access distinct memories differently, and thus are uniquely susceptible as the brain changes during addiction. If addiction is to be considered a neurobiological disorder of memory, and thus decision-making, the heterogeneity with which information is both stored and processed must be taken into account in addiction studies. Addiction etiology can vary widely from person to person. We propose that addiction is not a single disease, nor simply a disorder of learning and memory, but rather a collection of symptoms of heterogeneous neurobiological diseases of distinct circuit-computation-specific decision-making processes.

Representations of Value in the Brain: An Embarrassment of Riches?

Over the past two decades, neuroscientists have increasingly turned their attention to the question of how the brain implements decisions between differently valued options. This emerging field, called neuroeconomics, has made quick progress in identifying a plethora of brain areas that track or are modulated by reward value. However, it is still unclear how and where in the brain value coding takes place. A primate study by Strait and colleagues in this issue of PLOS Biology finds overlapping signals of value coding in two brain regions central to the valuation process: the ventromedial prefrontal cortex and the ventral striatum. This finding reconciles the primate and rodent literatures, provides valuable insight into the complexity of value computation, and helps set the agenda for future work in this area.

Demystifying Graduate School: Navigating a PhD in Neuroscience and Beyond.

The decision to apply to a PhD-granting graduate program is both exciting and daunting. Understanding what graduate programs look for in an applicant will increase the chance of successful admission into a PhD program. It is also helpful for an applicant to understand what graduate training will look like once they matriculate into a PhD program to ensure they select programs that will help them reach their career objectives. This article focuses specifically on PhD programs in neuroscience, and while we use our program, the Graduate Program in Neuroscience at the University of Minnesota, as an example, most of what we describe is applicable to biomedical graduate programs generally. In order to ensure that our description of graduate programs is typical of neuroscience graduate programs generally, we surveyed the online websites of 52 neuroscience graduate programs around the U. S. and include our observations here. We will examine what graduate schools look for in an applicant, what to expect once admitted into a PhD graduate program, and the potential outcomes for those who successfully complete their PhD in neuroscience.

Manipulating Decisiveness in Decision Making: Effects of Clonidine on Hippocampal Search Strategies.

Decisiveness is the ability to commit to a decision quickly and efficiently; in contrast, indecision entails the repeated consideration of multiple alternative possibilities. In humans, the α2-adrenergic receptor agonist clonidine increases decisiveness in tasks that require planning through unknown neural mechanisms. In rats, indecision is manifested as reorienting behaviors at choice points (vicarious trial and error [VTE]), during which hippocampal representations alternate between prospective options. To determine whether the increase in decisiveness driven by clonidine also entails changes in hippocampal search processes, we compared the effect of clonidine on spatial representations in hippocampal neural ensembles as rats passed through a T-shaped decision point. Consistent with previous experiments, hippocampal representations reflected both chosen and unchosen paths during VTE events under saline control conditions. Also, consistent with previous experiments, hippocampal representations reflected the chosen path more than the unchosen path when the animal did not show VTE at the choice point. Injection of clonidine suppressed the spatial representation of the unchosen path at the choice point on VTE laps and hastened the differentiation of spatial representations of the chosen path from the unchosen path on non-VTE laps to appear before reaching the choice point. These results suggest that the decisiveness seen under clonidine is due to limited exploration of potential options in hippocampus, and suggest novel roles for noradrenaline as a modulator of the hippocampal search processes. Significance statement: Clonidine, an α2-adrenergic receptor agonist, which decreases the level of noradrenaline in vivo, has an interesting effect in humans and other animals: it makes them more decisive. However, the mechanisms by which clonidine makes them more decisive remain unknown. Researchers have speculated that clonidine limits the amount of mental search that subjects do when planning options. We test this hypothesis by measuring the mental search strategy in rats through hippocampal recordings. We find that clonidine limits the options searched by rats, suggesting that noradrenaline also plays a role in balancing exploration and exploitation in internally simulated behaviors, similar to its role in balancing exploration and exploitation in external behaviors.

The Web-Surf Task: A translational model of human decision-making.

Animal models of decision-making are some of the most highly regarded psychological process models; however, there remains a disconnection between how these models are used for pre-clinical applications and the resulting treatment outcomes. This may be due to untested assumptions that different species recruit the same neural or psychological mechanisms. We propose a novel human foraging paradigm (Web-Surf Task) that we translated from a rat foraging paradigm (Restaurant Row) to evaluate cross-species decision-making similarities. We examined behavioral parallels in human and non-human animals using the respective tasks. We also compared two variants of the human task, one using videos and the other using photos as rewards, by correlating revealed and stated preferences. We demonstrate similarities in choice behaviors and decision reaction times in human and rat subjects. Findings also indicate that videos yielded more reliable and valid results. The joint use of the Web-Surf Task and Restaurant Row is therefore a promising approach for functional translational research, aiming to bridge pre-clinical and clinical lines of research using analogous tasks.

Subjective costs drive overly patient foraging strategies in rats on an intertemporal foraging task.

Laboratory studies of decision making often take the form of two-alternative, forced-choice paradigms. In natural settings, however, many decision problems arise as stay/go choices. We designed a foraging task to test intertemporal decision making in rats via stay/go decisions. Subjects did not follow the rate-maximizing strategy of choosing only food items associated with short delays. Instead, rats were often willing to wait for surprisingly long periods, and consequently earned a lower rate of food intake than they might have by ignoring long-delay options. We tested whether foraging theory or delay discounting models predicted the behavior we observed but found that these models could not account for the strategies subjects selected. Subjects' behavior was well accounted for by a model that incorporated a cost for rejecting potential food items. Interestingly, subjects' cost sensitivity was proportional to environmental richness. These findings are at odds with traditional normative accounts of decision making but are consistent with retrospective considerations having a deleterious influence on decisions (as in the "sunk-cost" effect). More broadly, these findings highlight the utility of complementing existing assays of decision making with tasks that mimic more natural decision topologies.