The effects of time horizon and guided choices on explore-exploit decisions in rodents.

Humans and animals have to balance the need for exploring new options with exploiting known options that yield good outcomes. This tradeoff is known as the explore-exploit dilemma. To better understand the neural mechanisms underlying how humans and animals address the explore-exploit dilemma, a good animal behavioral model is critical. Most previous rodents explore-exploit studies used ethologically unrealistic operant boxes and reversal learning paradigms in which the decision to abandon a bad option is confounded by the need for exploring a novel option for information collection, making it difficult to separate different drives and heuristics for exploration. In this study, we investigated how rodents make explore-exploit decisions using a spatial navigation horizon task (Wilson et al., 2014) adapted to rats to address the above limitations. We compared the rats' performance to that of humans using identical measures. We showed that rats use prior information to effectively guide exploration. In addition, rats use information-driven directed exploration like humans, but the extent to which they explore has the opposite dependance on time horizon than humans. Moreover, we found that free choices and guided choices have different influences on exploration in rodents, a finding that has not yet been tested in humans. This study reveals that the explore-exploit spatial behavior of rats is more complex than previously thought. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Adapting hippocampus multi-scale place field distributions in cluttered environments optimizes spatial navigation and learning.

Extensive studies in rodents show that place cells in the hippocampus have firing patterns that are highly correlated with the animal's location in the environment and are organized in layers of increasing field sizes or scales along its dorsoventral axis. In this study, we use a spatial cognition model to show that different field sizes could be exploited to adapt the place cell representation to different environments according to their size and complexity. Specifically, we provide an in-depth analysis of how to distribute place cell fields according to the obstacles in cluttered environments to optimize learning time and path optimality during goal-oriented spatial navigation tasks. The analysis uses a reinforcement learning (RL) model that assumes that place cells allow encoding the state. While previous studies have suggested exploiting different field sizes to represent areas requiring different spatial resolutions, our work analyzes specific distributions that adapt the representation to the environment, activating larger fields in open areas and smaller fields near goals and subgoals (e.g., obstacle corners). In addition to assessing how the multi-scale representation may be exploited in spatial navigation tasks, our analysis and results suggest place cell representations that can impact the robotics field by reducing the total number of cells for path planning without compromising the quality of the paths learned.

Integration of velocity-dependent spatio-temporal structure of place cell activation during navigation in a reservoir model of prefrontal cortex.

Sequential behavior unfolds both in space and in time. The same spatial trajectory can be realized in different manners in the same overall time by changing instantaneous speeds. The current research investigates how speed profiles might be given behavioral significance and how cortical networks might encode this information. We first demonstrate that rats can associate different speed patterns on the same trajectory with distinct behavioral choices. In this novel experimental paradigm, rats follow a small baited robot in a large megaspace environment where the rat's speed is precisely controlled by the robot's speed. Based on this proof of concept and research showing that recurrent reservoir networks are ideal for representing spatio-temporal structures, we then test reservoir networks in simulated navigation contexts and demonstrate they can discriminate between traversals of the same path with identical durations but different speed profiles. We then test the networks in an embodied robotic setup, where we use place cell representations from physically navigating robots as input and again successfully discriminate between traversals. To demonstrate that this capability is inherent to recurrent networks, we compared the model against simple linear integrators. Interestingly, although the linear integrators could also perform the speed profile discrimination, a clear difference emerged when examining information coding in both models. Reservoir neurons displayed a form of statistical mixed selectivity as a complex interaction between spatial location and speed that was not as abundant in the linear integrators. This mixed selectivity is characteristic of cortex and reservoirs and allows us to generate specific predictions about the neural activity that will be recorded in rat cortex in future experiments.

Dorsal CA1 hippocampal place cells form a multi-scale representation of megaspace.

Spatially firing "place cells" within the hippocampal CA1 region form internal maps of the environment necessary for navigation and memory. In rodents, these neurons have been almost exclusively studied in small environments (<4 m2). It remains unclear how place cells encode a very large open 2D environment that is commensurate with the natural environments experienced by rodents and other mammals. Such an ethologically realistic environment would require a complex spatial representation, capable of simultaneously representing space at multiple overlapping fine-to-coarse informational scales. Here, we show that in a "megaspace" (18.6 m2), the majority of dorsal CA1 place cells exhibited multiple place subfields of different sizes, akin to those observed along the septo-temporal axis. Furthermore, the total area covered by the subfields of each cell was not correlated with the number of subfields, and increased with the scale of the environment. The multiple different-sized subfields exhibited by place cells in the megaspace suggest that the ensemble population of subfields form a multi-scale representation of space within the dorsal hippocampus. Our findings point to a new dorsal hippocampus ensemble coding scheme that simultaneously supports navigational processes at both fine- and coarse-grained resolutions. VIDEO ABSTRACT.

Transcranial magnetic stimulation reveals diminished homoeostatic metaplasticity in cognitively impaired adults.

Homoeostatic metaplasticity is a neuroprotective physiological feature that counterbalances Hebbian forms of plasticity to prevent network destabilization and hyperexcitability. Recent animal models highlight dysfunctional homoeostatic metaplasticity in the pathogenesis of Alzheimer's disease. However, the association between homoeostatic metaplasticity and cognitive status has not been systematically characterized in either demented or non-demented human populations, and the potential value of homoeostatic metaplasticity as an early biomarker of cognitive impairment has not been explored in humans. Here, we report that, through pre-conditioning the synaptic activity prior to non-invasive brain stimulation, the association between homoeostatic metaplasticity and cognitive status could be established in a population of non-demented human subjects (older adults across cognitive spectrums; all within the non-demented range). All participants (n = 40; age range, 65-74, 47.5% female) underwent a standardized neuropsychological battery, magnetic resonance imaging and a transcranial magnetic stimulation protocol. Specifically, we sampled motor-evoked potentials with an input/output curve immediately before and after repetitive transcranial magnetic stimulation to assess neural plasticity with two experimental paradigms: one with voluntary muscle contraction (i.e. modulated synaptic activity history) to deliberately introduce homoeostatic interference, and one without to serve as a control condition. From comparing neuroplastic responses across these experimental paradigms and across cohorts grouped by cognitive status, we found that (i) homoeostatic metaplasticity is diminished in our cohort of cognitively impaired older adults and (ii) this neuroprotective feature remains intact in cognitively normal participants. This novel finding suggests that (i) future studies should expand their scope beyond just Hebbian forms of plasticity that are traditionally assessed when using non-invasive brain stimulation to investigate cognitive ageing and (ii) the potential value of homoeostatic metaplasticity in serving as a biomarker for cognitive impairment should be further explored.

Conjunctive reward-place coding properties of dorsal distal CA1 hippocampus cells.

Autonomous motivated spatial navigation in animals or robots requires the association between spatial location and value. Hippocampal place cells are involved in goal-directed spatial navigation and the consolidation of spatial memories. Recently, Gauthier and Tank (Neuron 99(1):179-193, 2018. https://doi.org/10.1016/j.neuron.2018.06.008) have identified a subpopulation of hippocampal cells selectively activated in relation to rewarded goals. However, the relationship between these cells' spiking activity and goal representation remains elusive. We analyzed data from experiments in which rats underwent five consecutive tasks in which reward locations and spatial context were manipulated. We found CA1 populations with properties continuously ranging from place cells to reward cells. Specifically, we found typical place cells insensitive to reward locations, reward cells that only fired at correct rewarded feeders in each task regardless of context, and "hybrid cells" that responded to spatial locations and change of reward locations. Reward cells responded mostly to the reward delivery rather than to its expectation. In addition, we found a small group of neurons that transitioned between place and reward cells properties within the 5-task session. We conclude that some pyramidal cells (if not all) integrate both spatial and reward inputs to various degrees. These results provide insights into the integrative coding properties of CA1 pyramidal cells, focusing on their abilities to carry both spatial and reward information in a mixed and plastic manner. This conjunctive coding property prompts a re-thinking of current computational models of spatial navigation in which hippocampal spatial and subcortical value representations are independent.

A computational model for spatial cognition combining dorsal and ventral hippocampal place field maps: multiscale navigation.

Classic studies have shown that place cells are organized along the dorsoventral axis of the hippocampus according to their field size, with dorsal hippocampal place cells having smaller field sizes than ventral place cells. Studies have also suggested that dorsal place cells are primarily involved in spatial navigation, while ventral place cells are primarily involved in context and emotional encoding. Additionally, recent work has shown that the entire longitudinal axis of the hippocampus may be involved in navigation. Based on the latter, in this paper we present a spatial cognition reinforcement learning model inspired by the multiscale organization of the dorsal-ventral axis of the hippocampus. The model analyzes possible benefits of a multiscale architecture in terms of the learning speed, the path optimality, and the number of cells in the context of spatial navigation. The model is evaluated in a goal-oriented task where simulated rats need to learn a path to the goal from multiple starting locations in various open-field maze configurations. The results show that smaller scales of representation are useful for improving path optimality, whereas larger scales are useful for reducing learning time and the number of cells required. The results also show that combining scales can enhance the performance of the multiscale model, with a trade-off between path optimality and learning time.

Reservoir computing model of prefrontal cortex creates novel combinations of previous navigation sequences from hippocampal place-cell replay with spatial reward propagation.

As rats learn to search for multiple sources of food or water in a complex environment, they generate increasingly efficient trajectories between reward sites. Such spatial navigation capacity involves the replay of hippocampal place-cells during awake states, generating small sequences of spatially related place-cell activity that we call "snippets". These snippets occur primarily during sharp-wave-ripples (SWRs). Here we focus on the role of such replay events, as the animal is learning a traveling salesperson task (TSP) across multiple trials. We hypothesize that snippet replay generates synthetic data that can substantially expand and restructure the experience available and make learning more optimal. We developed a model of snippet generation that is modulated by reward, propagated in the forward and reverse directions. This implements a form of spatial credit assignment for reinforcement learning. We use a biologically motivated computational framework known as 'reservoir computing' to model prefrontal cortex (PFC) in sequence learning, in which large pools of prewired neural elements process information dynamically through reverberations. This PFC model consolidates snippets into larger spatial sequences that may be later recalled by subsets of the original sequences. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to "learn" trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior.

Explainable Artificial Intelligence for Neuroscience: Behavioral Neurostimulation.

The use of Artificial Intelligence and machine learning in basic research and clinical neuroscience is increasing. AI methods enable the interpretation of large multimodal datasets that can provide unbiased insights into the fundamental principles of brain function, potentially paving the way for earlier and more accurate detection of brain disorders and better informed intervention protocols. Despite AI's ability to create accurate predictions and classifications, in most cases it lacks the ability to provide a mechanistic understanding of how inputs and outputs relate to each other. Explainable Artificial Intelligence (XAI) is a new set of techniques that attempts to provide such an understanding, here we report on some of these practical approaches. We discuss the potential value of XAI to the field of neurostimulation for both basic scientific inquiry and therapeutic purposes, as well as, outstanding questions and obstacles to the success of the XAI approach.

Ground truth construction and parameter tuning for the detection of sleep spindle timing in rodents.

BACKGROUND: The precise detection of cortical sleep spindles is critical to basic research on memory consolidation in rodents. Previous research using automatic spindle detection algorithms often lacks systematic parameter variations and validations. NEW METHOD: We present a method to systematically tune and validate algorithm parameters in automatic spindle detection algorithms using a moderate number of human raters. RESULTS: Comparing a Hilbert transform-based algorithm to a ground truth constructed by six human raters, this method produced a parameter set yielding an F1 score of 0.82 at 10 ms resolution. The algorithm performance fell within the range of human agreement with the ground truth. Both human and algorithm failures arose largely from disagreement in spindle boundaries rather than spindle occurrence. With no additional tuning, the algorithm performed similarly in recordings from different days or rats. COMPARISON WITH EXISTING METHODS: Most spindle detection algorithms do not perform systematic parameter variations and validation using a ground truth. To our knowledge, our study is the first in which rodent spindle data is scored by humans, and in which an automatic spindle detection algorithm is evaluated with respect to this ground truth. The rodent data from this study make it possible to compare our algorithm with others previously tested on human data. CONCLUSIONS: We present a general ground truth based approach for the tuning and validation of spindle extraction algorithms and suggest that algorithms aimed at extracting precise spindle timing in rats should use a systematic approach for parameter tuning.

The ventral hippocampus is involved in multi-goal obstacle-rich spatial navigation.

A large body of evidence shows that the hippocampus is necessary for successful spatial navigation. Various studies have shown anatomical and functional differences between the dorsal (DHC) and ventral (VHC) portions of this structure. The DHC is primarily involved in spatial navigation and contains cells with small place fields. The VHC is primarily involved in context and emotional encoding contains cells with large place fields and receives major projections from the medial prefrontal cortex. In the past, spatial navigation experiments have used relatively simple tasks that may not have required a strong coordination along the dorsoventral hippocampal axis. In this study, we tested the hypothesis that the DHC and VHC may be critical for goal-directed navigation in obstacle-rich environments. We used a learning task in which animals memorize the location of a set of rewarded feeders, and recall these locations in the presence of small or large obstacles. We report that bilateral DHC or VHC inactivation impaired spatial navigation in both large and small obstacle conditions. Importantly, this impairment did not result from a deficit in the spatial memory for the set of feeders (i.e., recognition of the goal locations) because DHC or VHC inactivation did not affect recall performance when there was no obstacle on the maze. We also show that the behavioral performance of the animals was correlated with several measures of maze complexity and that these correlations were significantly affected by inactivation only in the large object condition. These results suggest that as the complexity of the environment increases, both DHC and VHC are required for spatial navigation.

A new rat-compatible robotic framework for spatial navigation behavioral experiments.

BACKGROUND: Understanding the neural substrate of information encoding and processing requires a precise control of the animal's behavior. Most of what has been learned from the rodent navigational system results from relatively simple tasks in which the movements of the animal is controlled by corridors or walkways, passive movements, treadmills or virtual reality environments. While a lot has been and continues to be learned from these types of experiments, recent evidence has shown that such artificial constraints may have significant consequences on the functioning of the neural circuits of spatial navigation. NEW METHODS: We present a novel and alternative approach for effectively controlling the precise direction and speed of movement of the animal in an ethologically realistic environment, using a small robot (Sphero). RESULTS: We describe the robotic framework and demonstrate its use in replicating pre-programmed or rat-recorded paths. We show that the robot can control the movement of a rat in order to produce specific trajectories and speeds. We demonstrate that the robot can be used to aid the rat in learning a spatial memory task in a large and complex environment. We show that dorsal hippocampal CA1 place cells do not remap when the rat is following the robot. Comparison with existing method(s): Our framework only involves positive motivation and has been tested together with wireless electrophysiology in large and complex environments. CONCLUSIONS: Our robotic framework can be used to design novel tasks and experiments in which electrophysiological recordings would be largely devoid of maze or task-dependent artifacts.

Dorsoventral and Proximodistal Hippocampal Processing Account for the Influences of Sleep and Context on Memory (Re)consolidation: A Connectionist Model.

The context in which learning occurs is sufficient to reconsolidate stored memories and neuronal reactivation may be crucial to memory consolidation during sleep. The mechanisms of context-dependent and sleep-dependent memory (re)consolidation are unknown but involve the hippocampus. We simulated memory (re)consolidation using a connectionist model of the hippocampus that explicitly accounted for its dorsoventral organization and for CA1 proximodistal processing. Replicating human and rodent (re)consolidation studies yielded the following results. (1) Semantic overlap between memory items and extraneous learning was necessary to explain experimental data and depended crucially on the recurrent networks of dorsal but not ventral CA3. (2) Stimulus-free, sleep-induced internal reactivations of memory patterns produced heterogeneous recruitment of memory items and protected memories from subsequent interference. These simulations further suggested that the decrease in memory resilience when subjects were not allowed to sleep following learning was primarily due to extraneous learning. (3) Partial exposure to the learning context during simulated sleep (i.e., targeted memory reactivation) uniformly increased memory item reactivation and enhanced subsequent recall. Altogether, these results show that the dorsoventral and proximodistal organization of the hippocampus may be important components of the neural mechanisms for context-based and sleep-based memory (re)consolidations.

Beneficial effects of chronic oxytocin administration and social co-housing in a rodent model of post-traumatic stress disorder.

Post-traumatic stress disorder (PTSD) is in part due to a deficit in memory consolidation and extinction. Oxytocin (OXT) has anxiolytic effects and promotes prosocial behaviors in both rodents and humans, and evidence suggests that it plays a role in memory consolidation. We studied the effects of administered OXT and social co-housing in a rodent model of PTSD. Acute OXT yielded a short-term increase in the recall of the traumatic memory if administered immediately after trauma. Low doses of OXT delivered chronically had a cumulating anxiolytic effect that became apparent after 4 days and persisted. Repeated injections of OXT after short re-exposures to the trauma apparatus yielded a long-term reduction in anxiety. Co-housing with naive nonshocked animals decreased the memory of the traumatic context compared with single-housed animals. In the long term, these animals showed less thigmotaxis and increased interest in novel objects, and a low OXT plasma level. Co-housed PTSD animals showed an increase in risk-taking behavior. These results suggest beneficial effects of OXT if administered chronically through increases in memory consolidation after re-exposure to a safe trauma context. We also show differences between the benefits of social co-housing with naive rats and co-housing with other shocked animals on trauma-induced long-term anxiety.

Hippocampal CA1 Ripples as Inhibitory Transients.

Memories are stored and consolidated as a result of a dialogue between the hippocampus and cortex during sleep. Neurons active during behavior reactivate in both structures during sleep, in conjunction with characteristic brain oscillations that may form the neural substrate of memory consolidation. In the hippocampus, replay occurs within sharp wave-ripples: short bouts of high-frequency activity in area CA1 caused by excitatory activation from area CA3. In this work, we develop a computational model of ripple generation, motivated by in vivo rat data showing that ripples have a broad frequency distribution, exponential inter-arrival times and yet highly non-variable durations. Our study predicts that ripples are not persistent oscillations but result from a transient network behavior, induced by input from CA3, in which the high frequency synchronous firing of perisomatic interneurons does not depend on the time scale of synaptic inhibition. We found that noise-induced loss of synchrony among CA1 interneurons dynamically constrains individual ripple duration. Our study proposes a novel mechanism of hippocampal ripple generation consistent with a broad range of experimental data, and highlights the role of noise in regulating the duration of input-driven oscillatory spiking in an inhibitory network.