Computational models of Idling brain activity for memory processing.

Studying the underlying neural mechanisms of cognitive functions of the brain is one of the central questions in modern biology. Moreover, it has significantly impacted the development of novel technologies in artificial intelligence. Spontaneous activity is a unique feature of the brain and is currently lacking in many artificially constructed intelligent machines. Spontaneous activity may represent the brain's idling states, which are internally driven by neuronal networks and possibly participate in offline processing during awake, sleep, and resting states. Evidence is accumulating that the brain's spontaneous activity is not mere noise but part of the mechanisms to process information about previous experiences. A bunch of literature has shown how previous sensory and behavioral experiences influence the subsequent patterns of brain activity with various methods in various animals. It seems, however, that the patterns of neural activity and their computational roles differ significantly from area to area and from function to function. In this article, I review the various forms of the brain's spontaneous activity, especially those observed during memory processing, and some attempts to model the generation mechanisms and computational roles of such activities.

Internally Triggered Experiences of Hedonic Valence in Nonhuman Animals: Cognitive and Welfare Considerations.

Do any nonhuman animals have hedonically valenced experiences not directly caused by stimuli in their current environment? Do they, like us humans, experience anticipated or previously experienced pains and pleasures as respectively painful and pleasurable? We review evidence from comparative neuroscience about hippocampus-dependent simulation in relation to this question. Hippocampal sharp-wave ripples and theta oscillations have been found to instantiate previous and anticipated experiences. These hippocampal activations coordinate with neural reward and fear centers as well as sensory and cortical areas in ways that are associated with conscious episodic mental imagery in humans. Moreover, such hippocampal "re- and preplay" has been found to contribute to instrumental decision making, the learning of value representations, and the delay of rewards in rats. The functional and structural features of hippocampal simulation are highly conserved across mammals. This evidence makes it reasonable to assume that internally triggered experiences of hedonic valence (IHVs) are pervasive across (at least) all mammals. This conclusion has important welfare implications. Most prominently, IHVs act as a kind of "welfare multiplier" through which the welfare impacts of any given experience of pain or pleasure are increased through each future retrieval. However, IHVs also have practical implications for welfare assessment and cause prioritization.

Time as the fourth dimension in the hippocampus.

Experiences of animal and human beings are structured by the continuity of space and time coupled with the uni-directionality of time. In addition to its pivotal position in spatial processing and navigation, the hippocampal system also plays a central, multiform role in several types of temporal processing. These include timing and sequence learning, at scales ranging from meso-scales of seconds to macro-scales of minutes, hours, days and beyond, encompassing the classical functions of short term memory, working memory, long term memory, and episodic memories (comprised of information about when, what, and where). This review article highlights the principal findings and behavioral contexts of experiments in rats showing: 1) timing: tracking time during delays by hippocampal 'time cells' and during free behavior by hippocampal-afferent lateral entorhinal cortex ramping cells; 2) 'online' sequence processing: activity coding sequences of events during active behavior; 3) 'off-line' sequence replay: during quiescence or sleep, orderly reactivation of neuronal assemblies coding awake sequences. Studies in humans show neurophysiological correlates of episodic memory comparable to awake replay. Neural mechanisms are discussed, including ion channel properties, plateau and ramping potentials, oscillations of excitation and inhibition of population activity, bursts of high amplitude discharges (sharp wave ripples), as well as short and long term synaptic modifications among and within cell assemblies. Specifically conceived neural network models will suggest processes supporting the emergence of scalar properties (Weber's law), and include different classes of feedforward and recurrent network models, with intrinsic hippocampal coding for 'transitions' (sequencing of events or places).

Potential factors influencing replay across CA1 during sharp-wave ripples.

Sharp-wave ripples are complex neurophysiological events recorded along the trisynaptic hippocampal circuit (i.e. from CA3 to CA1 and the subiculum) during slow-wave sleep and awake states. They arise locally but scale brain-wide to the hippocampal target regions at cortical and subcortical structures. During these events, neuronal firing sequences are replayed retrospectively or prospectively and in the forward or reverse order as defined by experience. They could reflect either pre-configured firing sequences, learned sequences or an option space to inform subsequent decisions. How can different sequences arise during sharp-wave ripples? Emerging data suggest the hippocampal circuit is organized in different loops across the proximal (close to dentate gyrus) and distal (close to entorhinal cortex) axis. These data also disclose a so-far neglected laminar organization of the hippocampal output during sharp-wave events. Here, I discuss whether by incorporating cell-type-specific mechanisms converging on deep and superficial CA1 sublayers along the proximodistal axis, some novel factors influencing the organization of hippocampal sequences could be unveiled. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Human Replay Spontaneously Reorganizes Experience.

Knowledge abstracted from previous experiences can be transferred to aid new learning. Here, we asked whether such abstract knowledge immediately guides the replay of new experiences. We first trained participants on a rule defining an ordering of objects and then presented a novel set of objects in a scrambled order. Across two studies, we observed that representations of these novel objects were reactivated during a subsequent rest. As in rodents, human "replay" events occurred in sequences accelerated in time, compared to actual experience, and reversed their direction after a reward. Notably, replay did not simply recapitulate visual experience, but followed instead a sequence implied by learned abstract knowledge. Furthermore, each replay contained more than sensory representations of the relevant objects. A sensory code of object representations was preceded 50 ms by a code factorized into sequence position and sequence identity. We argue that this factorized representation facilitates the generalization of a previously learned structure to new objects.

Strengthened Temporal Coordination within Pre-existing Sequential Cell Assemblies Supports Trajectory Replay.

A central goal in learning and memory research is to reveal the neural substrates underlying episodic memory formation. The hallmark of sequential spatial trajectory learning, a model of episodic memory, has remained equivocal, with proposals ranging from de novo creation of compressed sequential replay from blank slate networks to selection of pre-existing compressed preplay sequences. Here, we show that increased millisecond-timescale activation of cell assemblies expressed during de novo sequential experience and increased neuronal firing rate correlations can explain the difference between post-experience trajectory replay and robust preplay. This increased activation results from an improved neuronal tuning to specific cell assemblies, higher recruitment of experience-tuned neurons into pre-existing cell assemblies, and increased recruitment of cell assemblies in replay. In contrast, changes in overall neuronal and cell assembly temporal order within extended sequences do not account for sequential trajectory learning. We propose the coordinated strengthening of cell assemblies played sequentially on robust pre-existing temporal frameworks could support rapid formation of episodic-like memory.

Rapid eye movement sleep mediates age-related decline in prospective memory consolidation.

STUDY OBJECTIVES: Prospective memory, or remembering to execute future intentions, accounts for half of everyday forgetting in older adults. Sleep intervals benefit prospective memory consolidation in young adults, but it is unknown whether age-related changes in slow wave activity, sleep spindles, and/or rapid eye movement (REM) sleep mediate hypothesized effects of aging on prospective memory consolidation. METHODS: After an adaptation night, 76 adults aged 18-84 completed two experimental nights of in-laboratory polysomnography recording. In the evening, participants encoded and practiced a prospective memory task and were tested the next morning. On a counterbalanced night, they encoded and practiced a control task, and were tested the following morning. RESULTS: Increasing age predicted worse prospective memory consolidation (r = -.34), even when controlling for encoding, speed, and control-task performance (all ps < .05). Frontal delta power, slow oscillations, and spindle density were not related to prospective memory consolidation. REM sleep duration, however, explained significant variance in prospective memory consolidation when controlling for age (∆R2 = .10). Bootstrapping mediation showed that less REM sleep significantly mediated the aging effect on prospective memory consolidation [b = -.0016, SE = 0.0009 (95% confidence interval [CI] = -0.0042 to -0.0004)]. REM sleep continued to mediate 24.29% of the total effect of age on prospective memory after controlling for numerous demographic, cognitive, mental health, and sleep variables. CONCLUSION: Age-related variance in REM sleep is informative to how prospective memory consolidation changes with increasing age. Future work should consider how both REM sleep and slow wave activity contribute, perhaps in a sequential or dynamic manner, to preserving cognitive functioning with increasing age.

Preconfigured patterns are the primary driver of offline multi-neuronal sequence replay.

Spontaneous neuronal ensemble activity in the hippocampus is believed to result from a combination of preconfigured internally generated dynamics and the unique patterns of activity driven by recent experience. Previous research has established that preconfigured sequential neuronal patterns (i.e., preplay) contribute to the expression of future place cell sequences, which in turn contribute to the sequential neuronal patterns expressed post-experience (i.e., replay). The relative contribution of preconfigured and of experience-related factors to replay and to overall sequential activity during post-run sleep is believed to be highly biased toward the recent run experience, despite never being tested directly. Here, we use multi-neuronal sequence analysis unbiased by firing rate to compute and directly compare the contributions of internally generated and of recent experience-driven factors to the sequential neuronal activity in post-run sleep in naïve adult rats. We find that multi-neuronal sequences during post-run sleep are dominantly contributed by the pre-run preconfigured patterns and to a much smaller extent by the place cell sequences and associated awake rest multi-neuronal sequences experienced during de novo run session, which are weakly and similarly correlated with pre- and post-run sleep multi-neuronal sequences. These findings indicate a robust default internal organization of the hippocampal network into sequential neuronal ensembles that withstands a de novo spatial experience and suggest that integration of novel information during de novo experience leading to lasting changes in sequential network patterns is much more subtle than previously assumed.

Generative Predictive Codes by Multiplexed Hippocampal Neuronal Tuplets.

Rapid internal representations are continuously formed based on single experiential episodes in space and time, but the neuronal ensemble mechanisms enabling rapid encoding without constraining the capacity for multiple distinct representations are unknown. We developed a probabilistic statistical model of hippocampal spontaneous sequential activity and revealed existence of an internal model of generative predictive codes for the regularities of multiple future novel spatial sequences. During navigation, the inferred difference between external stimuli and the internal model was encoded by emergence of intrinsic-unlikely, novel functional connections, which updated the model by preferentially potentiating post-experience. This internal model and these predictive codes depended on neuronal organization into inferred modules of short, high-repeat sequential neuronal "tuplets" operating as "neuro-codons." We propose that flexible multiplexing of neuronal tuplets into repertoires of extended sequences vastly expands the capacity of hippocampal predictive codes, which could initiate top-down hierarchical cortical loops for spatial and mental navigation and rapid learning.

A Knowledge-Based Arrangement of Prototypical Neural Representation Prior to Experience Contributes to Selectivity in Upcoming Knowledge Acquisition.

Knowledge acquisition is a process in which one actively selects a piece of information from the environment and assimilates it with prior knowledge. However, little is known about the neural mechanism underlying selectivity in knowledge acquisition. Here we executed a 2-day human experiment to investigate the involvement of characteristic spontaneous activity resembling a so-called "preplay" in selectivity in sentence comprehension, an instance of knowledge acquisition. On day 1, we presented 10 sentences (prior sentences) that were difficult to understand on their own. On the following day, we first measured the resting-state functional magnetic resonance imaging (fMRI). Then, we administered a sentence comprehension task using 20 new sentences (posterior sentences). The posterior sentences were also difficult to understand on their own, but some could be associated with prior sentences to facilitate their understanding. Next, we measured the posterior sentence-induced fMRI to identify the neural representation. From the resting-state fMRI, we extracted the appearances of activity patterns similar to the neural representations for posterior sentences. Importantly, the resting-state fMRI was measured before giving the posterior sentences, and thus such appearances could be considered as preplay-like or prototypical neural representations. We compared the intensities of such appearances with the understanding of posterior sentences. This gave a positive correlation between these two variables, but only if posterior sentences were associated with prior sentences. Additional analysis showed the contribution of the entorhinal cortex, rather than the hippocampus, to the correlation. The present study suggests that prior knowledge-based arrangement of neural activity before an experience contributes to the active selection of information to be learned. Such arrangement prior to an experience resembles preplay activity observed in the rodent brain. In terms of knowledge acquisition, the present study leads to a new view of the brain (or more precisely of the brain's knowledge) as an autopoietic system in which the brain (or knowledge) selects what it should learn by itself, arranges preplay-like activity as a position for the new information in advance, and actively reorganizes itself.

Hippocampal place cells construct reward related sequences through unexplored space.

Dominant theories of hippocampal function propose that place cell representations are formed during an animal's first encounter with a novel environment and are subsequently replayed during off-line states to support consolidation and future behaviour. Here we report that viewing the delivery of food to an unvisited portion of an environment leads to off-line pre-activation of place cells sequences corresponding to that space. Such 'preplay' was not observed for an unrewarded but otherwise similar portion of the environment. These results suggest that a hippocampal representation of a visible, yet unexplored environment can be formed if the environment is of motivational relevance to the animal. We hypothesise such goal-biased preplay may support preparation for future experiences in novel environments.

Not only but also: REM sleep creates and NREM Stage 2 instantiates landmark junctions in cortical memory networks.

This article argues both rapid eye movement (REM) and non-rapid eye movement (NREM) sleep contribute to overnight episodic memory processes but their roles differ. Episodic memory may have evolved from memory for spatial navigation in animals and humans. Equally, mnemonic navigation in world and mental space may rely on fundamentally equivalent processes. Consequently, the basic spatial network characteristics of pathways which meet at omnidirectional nodes or junctions may be conserved in episodic brain networks. A pathway is formally identified with the unidirectional, sequential phases of an episodic memory. In contrast, the function of omnidirectional junctions is not well understood. In evolutionary terms, both animals and early humans undertook tours to a series of landmark junctions, to take advantage of resources (food, water and shelter), whilst trying to avoid predators. Such tours required memory for emotionally significant landmark resource-place-danger associations and the spatial relationships amongst these landmarks. In consequence, these tours may have driven the evolution of both spatial and episodic memory. The environment is dynamic. Resource-place associations are liable to shift and new resource-rich landmarks may be discovered, these changes may require re-wiring in neural networks. To realise these changes, REM may perform an associative, emotional encoding function between memory networks, engendering an omnidirectional landmark junction which is instantiated in the cortex during NREM Stage 2. In sum, REM may preplay associated elements of past episodes (rather than replay individual episodes), to engender an unconscious representation which can be used by the animal on approach to a landmark junction in wake.

Selection of preconfigured cell assemblies for representation of novel spatial experiences.

Internal representations about the external world can be driven by the external stimuli or can be internally generated in their absence. It has been a matter of debate whether novel stimuli from the external world are instructive over the brain network to create de novo representations or, alternatively, are selecting from existing pre-representations hosted in preconfigured brain networks. The hippocampus is a brain area necessary for normal internally generated spatial-temporal representations and its dysfunctions have resulted in anterograde amnesia, impaired imagining of new experiences, and hallucinations. The compressed temporal sequence of place cell activity in the rodent hippocampus serves as an animal model of internal representation of the external space. Based on our recent results on the phenomenon of novel place cell sequence preplay, we submit that the place cell sequence of a novel spatial experience is determined, in part, by a selection of a set of cellular firing sequences from a repertoire of existing temporal firing sequences in the hippocampal network. Conceptually, this indicates that novel stimuli from the external world select from their pre-representations rather than create de novo our internal representations of the world.

A computational model for preplay in the hippocampus.

The hippocampal network produces sequences of neural activity even when there is no time-varying external drive. In offline states, the temporal sequence in which place cells fire spikes correlates with the sequence of their place fields. Recent experiments found this correlation even between offline sequential activity (OSA) recorded before the animal ran in a novel environment and the place fields in that environment. This preplay phenomenon suggests that OSA is generated intrinsically in the hippocampal network, and not established by external sensory inputs. Previous studies showed that continuous attractor networks with asymmetric patterns of connectivity, or with slow, local negative feedback, can generate sequential activity. This mechanism could account for preplay if the network only represented a single spatial map, or chart. However, global remapping in the hippocampus implies that multiple charts are represented simultaneously in the hippocampal network and it remains unknown whether the network with multiple charts can account for preplay. Here we show that it can. Driven with random inputs, the model generates sequences in every chart. Place fields in a given chart and OSA generated by the network are highly correlated. We also find significant correlations, albeit less frequently, even when the OSA is correlated with a new chart in which place fields are randomly scattered. These correlations arise from random correlations between the orderings of place fields in the new chart and those in a pre-existing chart. Our results suggest two different accounts for preplay. Either an existing chart is re-used to represent a novel environment or a new chart is formed.

Learning and prospective recall of noisy spike pattern episodes.

Spike patterns in vivo are often incomplete or corrupted with noise that makes inputs to neuronal networks appear to vary although they may, in fact, be samples of a single underlying pattern or repeated presentation. Here we present a recurrent spiking neural network (SNN) model that learns noisy pattern sequences through the use of homeostasis and spike-timing dependent plasticity (STDP). We find that the changes in the synaptic weight vector during learning of patterns of random ensembles are approximately orthogonal in a reduced dimension space when the patterns are constructed to minimize overlap in representations. Using this model, representations of sparse patterns maybe associated through co-activated firing and integrated into ensemble representations. While the model is tolerant to noise, prospective activity, and pattern completion differ in their ability to adapt in the presence of noise. One version of the model is able to demonstrate the recently discovered phenomena of preplay and replay reminiscent of hippocampal-like behaviors.