Non-trivial relationship between behavioral avalanches and internal neuronal dynamics in a recurrent neural network.

Neuronal activity gives rise to behavior, and behavior influences neuronal dynamics, in a closed-loop control system. Is it possible then, to find a relationship between the statistical properties of behavior and neuronal dynamics? Measurements of neuronal activity and behavior have suggested a direct relationship between scale-free neuronal and behavioral dynamics. Yet, these studies captured only local dynamics in brain sub-networks. Here, we investigate the relationship between internal dynamics and output statistics in a mathematical model system where we have access to the dynamics of all network units. We train a recurrent neural network (RNN), initialized in a high-dimensional chaotic state, to sustain behavioral states for durations following a power-law distribution as observed experimentally. Changes in network connectivity due to training affect the internal dynamics of neuronal firings, leading to neuronal avalanche size distributions approximating power-laws over some ranges. Yet, randomizing the changes in network connectivity can leave these power-law features largely unaltered. Specifically, whereas neuronal avalanche duration distributions show some variations between RNNs with trained and randomized decoders, neuronal avalanche size distributions are invariant, in the total population and in output-correlated sub-populations. This is true independent of whether the randomized decoders preserve power-law distributed behavioral dynamics. This demonstrates that a one-to-one correspondence between the considered statistical features of behavior and neuronal dynamics cannot be established and their relationship is non-trivial. Our findings also indicate that statistical properties of the intrinsic dynamics may be preserved, even as the internal state responsible for generating the desired output dynamics is perturbed.

The Nonclassic Psychedelic Ibogaine Disrupts Cognitive Maps.

Background: The ability of psychedelic compounds to profoundly alter mental function has been long known, but the underlying changes in cellular-level information encoding remain poorly understood. Methods: We used two-photon microscopy to record from the retrosplenial cortex in head-fixed mice running on a treadmill before and after injection of the nonclassic psychedelic ibogaine (40 mg/kg intraperitoneally). Results: We found that the cognitive map, formed by the representation of position encoded by ensembles of individual neurons in the retrosplenial cortex, was destabilized by ibogaine when mice had to infer position between tactile landmarks. This corresponded with increased neural activity rates, loss of correlation structure, and increased responses to cues. Ibogaine had surprisingly little effect on the size-frequency distribution of network activity events, suggesting that signal propagation within the retrosplenial cortex was largely unaffected. Conclusions: Taken together, these data support proposals that compounds with psychedelic properties disrupt representations that are important for constraining neocortical activity, thereby increasing the entropy of neural signaling. Furthermore, the loss of expected position encoding between landmarks recapitulated effects of hippocampal impairment, suggesting that disruption of cognitive maps or other hippocampal processing may be a contributing mechanism of discoordinated neocortical activity in psychedelic states.

Changes in functional connectivity preserve scale-free neuronal and behavioral dynamics.

Does the brain optimize itself for storage and transmission of information, and if so, how? The critical brain hypothesis is based in statistical physics and posits that the brain self-tunes its dynamics to a critical point or regime to maximize the repertoire of neuronal responses. Yet, the robustness of this regime, especially with respect to changes in the functional connectivity, remains an unsolved fundamental challenge. Here, we show that both scale-free neuronal dynamics and self-similar features of behavioral dynamics persist following significant changes in functional connectivity. Specifically, we find that the psychedelic compound ibogaine that is associated with an altered state of consciousness fundamentally alters the functional connectivity in the retrosplenial cortex of mice. Yet, the scale-free statistics of movement and of neuronal avalanches among behaviorally related neurons remain largely unaltered. This indicates that the propagation of information within biological neural networks is robust to changes in functional organization of subpopulations of neurons, opening up a new perspective on how the adaptive nature of functional networks may lead to optimality of information transmission in the brain.

Acute gut inflammation reduces neural activity and spine maturity in hippocampus but not basolateral amygdala.

Gastrointestinal tract (gut) inflammation increases stress and threat-coping behaviors, which are associated with altered activity in fear-related neural circuits, such as the basolateral amygdala and hippocampus. It remains to be determined whether inflammation from the gut affects neural activity by altering dendritic spines. We hypothesized that acute inflammation alters dendritic spines in a brain region-specific manner. Here we show that acute gut inflammation (colitis) evoked by dextran sodium sulfate (DSS) did not affect the overall spine density in the CA1 region of hippocampus, but increased the relative proportion of immature spines to mature spines on basal dendrites of pyramidal neurons. In contrast, in animals with colitis, no changes in spine density or composition on dendrites of pyramidal cells was observed in the basolateral amygdala. Rather, we observed decreased spine density on dendrites of stellate neurons, but not the relative proportions of mature vs immature spines. We used cFos expression evoked by the forced swim task as a measure of neural activity during stress and found no effect of DSS on the density of cFos immunoreactive neurons in basolateral amygdala. In contrast, fewer CA1 neurons expressed cFos in mice with colitis, relative to controls. Furthermore, CA1 cFos expression negatively correlated with active stress-coping in the swim task and was negatively correlated with gut inflammation. These data reveal that the effects of acute gut inflammation on synaptic remodeling depend on brain region, neuronal phenotype, and dendrite location. In the hippocampus, a shift to immature spines and hypoactivity are more strongly related to colitis-evoked behavioral changes than is remodeling in basolateral amygdala.

Cannabis use is associated with sexually dimorphic changes in executive control of visuospatial decision-making.

When the outcome of a choice is less favorable than expected, humans and animals typically shift to an alternate choice option on subsequent trials. Several lines of evidence indicate that this "lose-shift" responding is an innate sensorimotor response strategy that is normally suppressed by executive function. Therefore, the lose-shift response provides a covert gauge of cognitive control over choice mechanisms. We report here that the spatial position, rather than visual features, of choice targets drives the lose-shift effect. Furthermore, the ability to inhibit lose-shift responding to gain reward is different among male and female habitual cannabis users. Increased self-reported cannabis use was concordant with suppressed response flexibility and an increased tendency to lose-shift in women, which reduced performance in a choice task in which random responding is the optimal strategy. On the other hand, increased cannabis use in men was concordant with reduced reliance on spatial cues during decision-making, and had no impact on the number of correct responses. These data (63,600 trials from 106 participants) provide strong evidence that spatial-motor processing is an important component of economic decision-making, and that its governance by executive systems is different in men and women who use cannabis frequently.

Amphetamine reduces reward encoding and stabilizes neural dynamics in rat anterior cingulate cortex.

Psychostimulants such as d-amphetamine (AMPH) often have behavioral effects that appear paradoxical within the framework of optimal choice theory. AMPH typically increases task engagement and the effort animals exert for reward, despite decreasing reward valuation. We investigated neural correlates of this phenomenon in the anterior cingulate cortex (ACC), a brain structure implicated in signaling cost-benefit utility. AMPH decreased signaling of reward, but not effort, in the ACC of freely-moving rats. Ensembles of simultaneously recorded neurons generated task-specific trajectories of neural activity encoding past, present, and future events. Low-dose AMPH contracted these trajectories and reduced their variance, whereas high-dose AMPH expanded both. We propose that under low-dose AMPH, increased network stability balances moderately increased excitability, which promotes accelerated unfolding of a neural 'script' for task execution, despite reduced reward valuation. Noise from excessive excitability at high doses overcomes stability enhancement to drive frequent deviation from the script, impairing task execution.

Behavioral adaptations in a relapsing mouse model of colitis.

Inflammatory bowel disease (IBD) is characterized by relapsing periods of gut inflammation, and is comorbid with depression, anxiety, and cognitive deficits. Animal models of IBD that explore the behavioral consequences almost exclusively use acute models of gut inflammation, which fails to recapitulate the cyclic, chronic nature of IBD. This study sought to identify behavioral differences in digging, memory, and stress-coping strategies in mice exposed to one (acute) or three (chronic) cycles of gut inflammation, using the dextran sodium sulfate (DSS) model of colitis. Similar levels of gut pathology were observed between acute and chronically exposed mice, although mice in the chronic treatment had significantly shorter colons, suggesting more severe disease. Behavioral measures revealed an unexpected pattern in which chronic treatment evoked fewer deficits than acute treatment. Specifically, acutely-treated mice showed alterations in measures of object burying, novel object recognition, object location memory, and stress-coping (forced swim task). Chronically-treated animals, however, showed similar alterations in object burying, but not the other measures. These data suggest an adaptive or tolerizing effect of repeated cycles of peripheral gut inflammation on mnemonic function and stress-coping, whereas some other behaviors continue to be affected by gut inflammation. We speculate that the normalization of some functions may involve the reversion to the baseline state of the hypothalamic-pituitary-adrenal axis and/or levels of neuroinflammation, which are both activated by the first exposure to the colitic agent.

Using Neuron Spiking Activity to Trigger Closed-Loop Stimuli in Neurophysiological Experiments.

Closed-loop neurophysiological systems use patterns of neuronal activity to trigger stimuli, which in turn affect brain activity. Such closed-loop systems are already found in clinical applications, and are important tools for basic brain research. A particularly interesting recent development is the integration of closed-loop approaches with optogenetics, such that specific patterns of neuronal activity can trigger optical stimulation of selected neuronal groups. However, setting up an electrophysiological system for closed-loop experiments can be difficult. Here, a ready-to-apply Matlab code is provided for triggering stimuli based on the activity of single or multiple neurons. This sample code can be easily modified based on individual needs. For instance, it shows how to trigger sound stimuli and how to change it to trigger an external device connected to a PC serial port. The presented protocol is designed to work with a popular neuronal recording system for animal studies (Neuralynx). The implementation of closed-loop stimulation is demonstrated in an awake rat.

Distributed Encoding of Reinforcement in Rat Cortico-Striatal-Limbic Networks.

Decision-making in the mammalian brain typically involves multiple brain structures within the midbrain, thalamus, striatum, limbic system, and cortex. Although task specific contributions of each brain region have been identified, neurons responding to reinforcement have been found throughout these structures. We sought to determine if any brain area, or cluster of areas, are the source of information, and if the fidelity of information varies among the areas. We recorded simultaneous field potentials (FPs) in rats from seven brain regions as they completed a binary choice task. The FPs of a 0.5 s window following reinforcement were given as input to a classifier that attempted to predict whether or not the rat received reward on each trial. The classifier correctly categorized reward on 77% of trials. Any region-specific signal could be omitted without lowering accuracy. Frequencies above 40 Hz and signals recorded later than 0.25 s following reinforcement were necessary to achieve this accuracy. Further, the classifier was able to predict reinforcement outcome above chance levels when using FPs from any single recorded brain region. Some combinations of structures, however, were more predictive than others. Analysis of FPs prior to reward revealed most regions reflected the prior probability of reward. Lastly, analyses of information flow suggested reinforcement information does not originate within a single structure of the network, within the resolution afforded by FP recordings. These data suggest reward delivery information is rapidly distributed non-uniformly across the network, and there is no canonical flow of information about reward events in the recorded structures.

Phase of electroencephalography theta oscillation during stimulus encoding affects accuracy of memory recall.

Oscillatory activity is a ubiquitous property of brain signals, and yet relatively few studies have investigated how the phase of such ongoing oscillations affects our cognition. One of the main findings in this field is that the phase of electroencephalography (EEG) in the alpha band can affect perception of milliseconds-long stimuli. However, the importance of the phase of EEG for processing more naturalistic stimuli, which have a much longer duration, is still not clear. To address this question here, we presented word-nonword pairs, each of which was visible for 5 s and measured the effect of EEG phase during stimulus onset on later memory recall. The task consisted of an encoding (learning) phase in which 20 novel word-nonword pairs were presented, followed by a test phase in which participants were shown one of the seen words with four target nonwords to choose from. We found that memory recall performance was higher when the words during encoding were presented at a descending phase of the theta oscillation. This effect was the strongest in the frontal cortex. These results suggest that the phase of ongoing cortical activity can affect memorization of seconds-long stimuli that are an integral part of many daily tasks.

Lesions of lateral habenula attenuate win-stay but not lose-shift responses in a competitive choice task.

Multiple neural systems contribute to choice adaptation following reinforcement. Recent evidence suggests that the lateral habenula (LHb) plays a key role in such adaptations, particularly when reinforcements are worse than expected. Here, we investigated the effects of bilateral LHb lesions on responding in a binary choice task with no discriminatory cues. LHb lesions in rats decreased win-stay responses but surprisingly left lose-shift responses intact. This same dissociated effect was also observed after systemic administration of d-amphetamine in a separate cohort of animals. These results suggest that at least some behavioural responses triggered by reward omission do not depend on an intact LHb.

Lesions of ventrolateral striatum eliminate lose-shift but not win-stay behaviour in rats.

Animals tend to repeat actions that are associated with reward delivery, whereas they tend to shift responses to alternate choices following reward omission. These so-called win-stay and lose-shift responses are employed by a wide range of animals in a variety of decision-making scenarios, and depend on dissociated regions of the striatum. Specifically, lose-shift responding is impaired by extensive excitotoxic lesions of the lateral striatum. Here we used focal lesions to assess whether dorsal and ventral regions of the lateral striatum contribute differently to this effect. We found that damage to ventrolateral striatum reduced lose-shift responding without impairing win-stay, motoric, or motivational aspects of behaviour in the task, whereas lesions confined to the dorsolateral striatum significantly impaired the ability of rats to complete trials of the task. Moreover, lesions to the dorsomedial striatum had no effect on either lose-shift or win-stay responding. Together, these data suggest a novel role of the ventral portion of the lateral striatum in driving lose-shift decisions.

Rat anterior cingulate cortex recalls features of remote reward locations after disfavoured reinforcements.

The anterior cingulate cortex (ACC) encodes information supporting mnemonic and cognitive processes. We show here that a rat's position can be decoded with high spatiotemporal resolution from ACC activity. ACC neurons encoded the current state of the animal and task, except for brief excursions that sometimes occurred at target feeders. During excursions, the decoded position became more similar to a remote target feeder than the rat's physical position. Excursions recruited activation of neurons encoding choice and reward, and the likelihood of excursions at a feeder was inversely correlated with feeder preference. These data suggest that the excursion phenomenon was related to evaluating real or fictive choice outcomes, particularly after disfavoured reinforcements. We propose that the multiplexing of position with choice-related information forms a mental model isomorphic with the task space, which can be mentally navigated via excursions to recall multimodal information about the utility of remote locations.

Lose-Shift Responding in Humans Is Promoted by Increased Cognitive Load.

The propensity of animals to shift choices immediately after unexpectedly poor reinforcement outcomes is a pervasive strategy across species and tasks. We report here on the memory supporting such lose-shift responding in humans, assessed using a binary choice task in which random responding is the optimal strategy. Participants exhibited little lose-shift responding when fully attending to the task, but this increased by 30%-40% in participants that performed with additional cognitive load that is known to tax executive systems. Lose-shift responding in the cognitively loaded adults persisted throughout the testing session, despite being a sub-optimal strategy, but was less likely as the time increased between reinforcement and the subsequent choice. Furthermore, children (5-9 years old) without load performed similarly to the cognitively loaded adults. This effect disappeared in older children aged 11-13 years old. These data provide evidence supporting our hypothesis that lose-shift responding is a default and reflexive strategy in the mammalian brain, likely mediated by a decaying memory trace, and is normally suppressed by executive systems. Reducing the efficacy of executive control by cognitive load (adults) or underdevelopment (children) increases its prevalence. It may therefore be an important component to consider when interpreting choice data, and may serve as an objective behavioral assay of executive function in humans that is easy to measure.

Sex differences in rat decision-making: The confounding role of extraneous feeder sampling between trials.

Although male and female rats appear to perform differently in some tasks, a clear picture of sex differences in decision-making has yet to develop. This is in part due to significant variability arising from differences in strains and tasks. The aim of this study was to characterize the effects of sex on specific response elements in a reinforcement learning task so as to help identify potential explanations for this variability. We found that the primary difference between sexes was the propensity to approach feeders out of the task context. This extraneous feeder sampling affects choice on subsequent trials in both sexes by promoting a lose-shift response away from the last feeder sampled. Female rats, however, were more likely to engage in this extraneous feeder sampling, and therefore exhibited a greater rate of this effect. Once trials following extraneous sampling were removed, there were no significant sex differences in any of the tested measures. These data suggest that feeder approach outside of the task context, which is often not recorded, could produce a confound in sex-based differences of reinforcement sensitivity in some tasks.

Implicit Valuation of the Near-Miss is Dependent on Outcome Context.

Gambling studies have described a "near-miss effect" wherein the experience of almost winning increases gambling persistence. The near-miss has been proposed to inflate the value of preceding actions through its perceptual similarity to wins. We demonstrate here, however, that it acts as a conditioned stimulus to positively or negatively influence valuation, dependent on reward expectation and cognitive engagement. When subjects are asked to choose between two simulated slot machines, near-misses increase valuation of machines with a low payout rate, whereas they decrease valuation of high payout machines. This contextual effect impairs decisions and persists regardless of manipulations to outcome feedback or financial incentive provided for good performance. It is consistent with proposals that near-misses cause frustration when wins are expected, and we propose that it increases choice stochasticity and overrides avoidance of low-valued options. Intriguingly, the near-miss effect disappears when subjects are required to explicitly value machines by placing bets, rather than choosing between them. We propose that this task increases cognitive engagement and recruits participation of brain regions involved in cognitive processing, causing inhibition of otherwise dominant systems of decision-making. Our results reveal that only implicit, rather than explicit strategies of decision-making are affected by near-misses, and that the brain can fluidly shift between these strategies according to task demands.

Acute Δ-9-tetrahydrocannabinol administration in female rats attenuates immediate responses following losses but not multi-trial reinforcement learning from wins.

Δ-9-Tetrahydrocannabinol (THC) is the main psychoactive component of marijuana and has potent effects on decision-making, including a proposed reduction in cognitive flexibility. We demonstrate here that acute THC administration differentially affects some of the processes that contribute to cognitive flexibility. Specifically, THC reduces lose-shift responding in which female rats tend to immediately shift choice responses away from options that result in reward omission on the previous trial. THC, however, did not impair the ability of rats to flexibly bias responses toward feeders with higher probability of reward in a reversal task. This response adaptation developed over several trials, suggesting that THC did not impair slower forms of reinforcement learning needed to choose among options with unequal utility. This dissociation of THC's effects on innate/rapid and learned/gradual decision-making processes was unexpected, but is supported by emerging evidence that lose-shift responding is mediated by neural mechanisms distinct from those involved in other forms of reinforcement learning. The present data suggest that, at least in some tasks, the apparent reductions in cognitive flexibility by THC may be explained by the immediate effects on loss sensitivity, rather than impairments of all processes used for choice adaptation.

Feeder Approach between Trials Is Increased by Uncertainty and Affects Subsequent Choices.

Animals quickly learn to approach sources of food. Here, we report on a form of approach in which rats made volitional orofacial contact with inactive feeders between trials of a self-paced operant task. This extraneous feeder sampling (EFS) was never reinforced and therefore imposed an opportunity and effort cost. EFS decreased during initial training but persisted thereafter. The relative rate of EFS to operant responding increased with novel changes to the operant chamber, reward devaluation by prefeeding, or lesions to the dorsolateral striatum. We speculate that this may function to increase exploration when the task is uncertain (early in learning or introduction of novel apparatus components), when the opportunity cost is low, or when the learned sensorimotor solution is compromised. Moreover, EFS strongly affected subsequent choices by triggering a lose-shift response away from the sampled feeder, even though it occurred outside of the trial context. This indicates that at least some behaviors occurring between trials impact future behaviors and should be considered in decision-making studies.

Opposing effects of acute and chronic d-amphetamine on decision-making in rats.

Amphetamine and other drugs of abuse have both short-term and long-lasting effects on brain function, and drug sensitization paradigms often result in chronic impairments in behavioral flexibility. Here we show that acute amphetamine administration temporarily renders rats less sensitive to reward omission, as revealed by a decrease in lose-shift responding during a binary choice task. Intracerebral infusions of amphetamine into the ventral striatum did not affect lose-shift responding but did increase impulsive behavior in which rats chose to check both reward feeders before beginning the next trial. In contrast to acute systemic and intracerebral infusions, sensitization through repeated exposure induced long-lasting increased sensitivity to reward omission. These treatments did not affect choices on trials following reward delivery (i.e. win-stay responding), and sensitization increased spine density in the sensorimotor striatum. The dichotomous effects of amphetamine on short-term and long-term loss sensitivity, and the null effect on win-stay responding, are consistent with a shift of behavioral control to the sensorimotor striatum after drug sensitization. These data provide a new demonstration of such a shift in a novel task unrelated to drug administration, and suggests that the dominance of sensorimotor control persists over many hundreds of trials after sensitization.

Computational Properties of the Hippocampus Increase the Efficiency of Goal-Directed Foraging through Hierarchical Reinforcement Learning.

The mammalian brain is thought to use a version of Model-based Reinforcement Learning (MBRL) to guide "goal-directed" behavior, wherein animals consider goals and make plans to acquire desired outcomes. However, conventional MBRL algorithms do not fully explain animals' ability to rapidly adapt to environmental changes, or learn multiple complex tasks. They also require extensive computation, suggesting that goal-directed behavior is cognitively expensive. We propose here that key features of processing in the hippocampus support a flexible MBRL mechanism for spatial navigation that is computationally efficient and can adapt quickly to change. We investigate this idea by implementing a computational MBRL framework that incorporates features inspired by computational properties of the hippocampus: a hierarchical representation of space, "forward sweeps" through future spatial trajectories, and context-driven remapping of place cells. We find that a hierarchical abstraction of space greatly reduces the computational load (mental effort) required for adaptation to changing environmental conditions, and allows efficient scaling to large problems. It also allows abstract knowledge gained at high levels to guide adaptation to new obstacles. Moreover, a context-driven remapping mechanism allows learning and memory of multiple tasks. Simulating dorsal or ventral hippocampal lesions in our computational framework qualitatively reproduces behavioral deficits observed in rodents with analogous lesions. The framework may thus embody key features of how the brain organizes model-based RL to efficiently solve navigation and other difficult tasks.