Accumulating evidence by sampling from temporally organized memory.

A recent study by Shushruth et al. (2022, Current Biology 32[9], 1949-1960) demonstrated that monkeys postpone evidence accumulation until the relevant motor actions are revealed and then sequentially sample the evidence from memory. Here, we reflect on the insights this work provides into reinforcement learning in evidence accumulation tasks and neural mechanisms for the temporal organization of memory.

A temporal record of the past with a spectrum of time constants in the monkey entorhinal cortex.

Episodic memory is believed to be intimately related to our experience of the passage of time. Indeed, neurons in the hippocampus and other brain regions critical to episodic memory code for the passage of time at a range of timescales. The origin of this temporal signal, however, remains unclear. Here, we examined temporal responses in the entorhinal cortex of macaque monkeys as they viewed complex images. Many neurons in the entorhinal cortex were responsive to image onset, showing large deviations from baseline firing shortly after image onset but relaxing back to baseline at different rates. This range of relaxation rates allowed for the time since image onset to be decoded on the scale of seconds. Further, these neurons carried information about image content, suggesting that neurons in the entorhinal cortex carry information about not only when an event took place but also, the identity of that event. Taken together, these findings suggest that the primate entorhinal cortex uses a spectrum of time constants to construct a temporal record of the past in support of episodic memory.

Conjunctive representation of what and when in monkey hippocampus and lateral prefrontal cortex during an associative memory task.

Adaptive memory requires the organism to form associations that bridge between events separated in time. Many studies show interactions between hippocampus (HPC) and prefrontal cortex (PFC) during formation of such associations. We analyze neural recording from monkey HPC and PFC during a memory task that requires the monkey to associate stimuli separated by about a second in time. After the first stimulus was presented, large numbers of units in both HPC and PFC fired in sequence. Many units fired only when a particular stimulus was presented at a particular time in the past. These results indicate that both HPC and PFC maintain a temporal record of events that could be used to form associations across time. This temporal record of the past is a key component of the temporal coding hypothesis, a hypothesis in psychology that memory not only encodes what happened, but when it happened.

Biological conservation law as an emerging functionality in dynamical neuronal networks.

Scientists strive to understand how functionalities, such as conservation laws, emerge in complex systems. Living complex systems in particular create high-ordered functionalities by pairing up low-ordered complementary processes, e.g., one process to build and the other to correct. We propose a network mechanism that demonstrates how collective statistical laws can emerge at a macro (i.e., whole-network) level even when they do not exist at a unit (i.e., network-node) level. Drawing inspiration from neuroscience, we model a highly stylized dynamical neuronal network in which neurons fire either randomly or in response to the firing of neighboring neurons. A synapse connecting two neighboring neurons strengthens when both of these neurons are excited and weakens otherwise. We demonstrate that during this interplay between the synaptic and neuronal dynamics, when the network is near a critical point, both recurrent spontaneous and stimulated phase transitions enable the phase-dependent processes to replace each other and spontaneously generate a statistical conservation law-the conservation of synaptic strength. This conservation law is an emerging functionality selected by evolution and is thus a form of biological self-organized criticality in which the key dynamical modes are collective.

A neural microcircuit model for a scalable scale-invariant representation of time.

Scale-invariant timing has been observed in a wide range of behavioral experiments. The firing properties of recently described time cells provide a possible neural substrate for scale-invariant behavior. Earlier neural circuit models do not produce scale-invariant neural sequences. In this article, we present a biologically detailed network model based on an earlier mathematical algorithm. The simulations incorporate exponentially decaying persistent firing maintained by the calcium-activated nonspecific (CAN) cationic current and a network structure given by the inverse Laplace transform to generate time cells with scale-invariant firing rates. This model provides the first biologically detailed neural circuit for generating scale-invariant time cells. The circuit that implements the inverse Laplace transform merely consists of off-center/on-surround receptive fields. Critically, rescaling temporal sequences can be accomplished simply via cortical gain control (changing the slope of the f-I curve).

Estimating Scale-Invariant Future in Continuous Time.

Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. An important drawback of model-free algorithms is the need to select a timescale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically compressed scale and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation that generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.

Compressed Timeline of Recent Experience in Monkey Lateral Prefrontal Cortex.

Cognitive theories suggest that working memory maintains not only the identity of recently presented stimuli but also a sense of the elapsed time since the stimuli were presented. Previous studies of the neural underpinnings of working memory have focused on sustained firing, which can account for maintenance of the stimulus identity, but not for representation of the elapsed time. We analyzed single-unit recordings from the lateral prefrontal cortex of macaque monkeys during performance of a delayed match-to-category task. Each sample stimulus triggered a consistent sequence of neurons, with each neuron in the sequence firing during a circumscribed period. These sequences of neurons encoded both stimulus identity and elapsed time. The encoding of elapsed time became less precise as the sample stimulus receded into the past. These findings suggest that working memory includes a compressed timeline of what happened when, consistent with long-standing cognitive theories of human memory.

Is working memory stored along a logarithmic timeline? Converging evidence from neuroscience, behavior and models.

A growing body of evidence suggests that short-term memory does not only store the identity of recently experienced stimuli, but also information about when they were presented. This representation of 'what' happened 'when' constitutes a neural timeline of recent past. Behavioral results suggest that people can sequentially access memories for the recent past, as if they were stored along a timeline to which attention is sequentially directed. In the short-term judgment of recency (JOR) task, the time to choose between two probe items depends on the recency of the more recent probe but not on the recency of the more remote probe. This pattern of results suggests a backward self-terminating search model. We review recent neural evidence from the macaque lateral prefrontal cortex (lPFC) (Tiganj, Cromer, Roy, Miller, & Howard, in press) and behavioral evidence from human JOR task (Singh & Howard, 2017) bearing on this question. Notably, both lines of evidence suggest that the timeline is logarithmically compressed as predicted by Weber-Fechner scaling. Taken together, these findings provide an integrative perspective on temporal organization and neural underpinnings of short-term memory.

Sequential Firing Codes for Time in Rodent Medial Prefrontal Cortex.

A subset of hippocampal neurons, known as "time cells" fire sequentially for circumscribed periods of time within a delay interval. We investigated whether medial prefrontal cortex (mPFC) also contains time cells and whether their qualitative properties differ from those in the hippocampus and striatum. We studied the firing correlates of neurons in the rodent mPFC during a temporal discrimination task. On each trial, the animals waited for a few seconds in the stem of a T-maze. A subpopulation of units fired in a sequence consistently across trials for a circumscribed period during the delay interval. These sequentially activated time cells showed temporal accuracy that decreased as time passed as measured by both the width of their firing fields and the number of cells that fired at a particular part of the interval. The firing dynamics of the time cells was significantly better explained with the elapse of time than with the animals' position and velocity. The findings observed here in the mPFC are consistent with those previously reported in the hippocampus and striatum, suggesting that the sequentially activated time cells are not specific to these areas, but are part of a common representational motif across regions.

Time Cells in Hippocampal Area CA3.

UNLABELLED: Studies on time cells in the hippocampus have so far focused on area CA1 in animals performing memory tasks. Some studies have suggested that temporal processing within the hippocampus may be exclusive to CA1 and CA2, but not CA3, and may occur only under strong demands for memory. Here we examined the temporal and spatial coding properties of CA3 and CA1 neurons in rats performing a maze task that demanded working memory and a control task with no explicit working memory demand. In the memory demanding task, CA3 cells exhibited robust temporal modulation similar to the pattern of time cell activity in CA1, and the same populations of cells also exhibited typical place coding patterns in the same task. Furthermore, the temporal and spatial coding patterns of CA1 and CA3 were equivalently robust when animals performed a simplified version of the task that made no demands on working memory. However, time and place coding did differ in that the resolution of temporal coding decreased over time within the delay interval, whereas the resolution of place coding was not systematically affected by distance along the track. These findings support the view that CA1 and CA3 both participate in encoding the temporal and spatial organization of ongoing experience. SIGNIFICANCE STATEMENT: Hippocampal "time cells" that fire at specific moments in a temporally structured memory task have so far been observed only in area CA1, and some studies have suggested that temporal coding within the hippocampus is exclusive to CA1. Here we describe time cells also in CA3, and time cells in both areas are observed even without working memory demands, similar to place cells in these areas. However, unlike equivalent spatial coding along a path, temporal coding is nonlinear, with greater temporal resolution earlier than later in temporally structured experiences. These observations reveal both similarities and differences in temporal and spatial coding within the hippocampus of importance to understanding how these features of memory are represented in the hippocampus.

A unified mathematical framework for coding time, space, and sequences in the hippocampal region.

The medial temporal lobe (MTL) is believed to support episodic memory, vivid recollection of a specific event situated in a particular place at a particular time. There is ample neurophysiological evidence that the MTL computes location in allocentric space and more recent evidence that the MTL also codes for time. Space and time represent a similar computational challenge; both are variables that cannot be simply calculated from the immediately available sensory information. We introduce a simple mathematical framework that computes functions of both spatial location and time as special cases of a more general computation. In this framework, experience unfolding in time is encoded via a set of leaky integrators. These leaky integrators encode the Laplace transform of their input. The information contained in the transform can be recovered using an approximation to the inverse Laplace transform. In the temporal domain, the resulting representation reconstructs the temporal history. By integrating movements, the equations give rise to a representation of the path taken to arrive at the present location. By modulating the transform with information about allocentric velocity, the equations code for position of a landmark. Simulated cells show a close correspondence to neurons observed in various regions for all three cases. In the temporal domain, novel secondary analyses of hippocampal time cells verified several qualitative predictions of the model. An integrated representation of spatiotemporal context can be computed by taking conjunctions of these elemental inputs, leading to a correspondence with conjunctive neural representations observed in dorsal CA1.

A simple biophysically plausible model for long time constants in single neurons.

Recent work in computational neuroscience and cognitive psychology suggests that a set of cells that decay exponentially could be used to support memory for the time at which events took place. Analytically and through simulations on a biophysical model of an individual neuron, we demonstrate that exponentially decaying firing with a range of time constants up to minutes could be implemented using a simple combination of well-known neural mechanisms. In particular, we consider firing supported by calcium-controlled cation current. When the amount of calcium leaving the cell during an interspike interval is larger than the calcium influx during a spike, the overall decay in calcium concentration can be exponential, resulting in exponential decay of the firing rate. The time constant of the decay can be several orders of magnitude larger than the time constant of calcium clearance, and it could be controlled externally via a variety of biologically plausible ways. The ability to flexibly and rapidly control time constants could enable working memory of temporal history to be generalized to other variables in computing spatial and ordinal representations.

Scanning a compressed ordered representation of the future.

Several authors have suggested a deep symmetry between the psychological processes that underlie our ability to remember the past and make predictions about the future. The judgment of recency (JOR) task measures temporal order judgments for the past by presenting pairs of probe stimuli; participants choose the probe that was presented more recently. We performed a short-term relative JOR task and introduced a novel judgment of imminence (JOI) task to study temporal order judgments for the future. In the JOR task, participants were presented with a sequence of stimuli and asked to choose which of two probe stimuli was presented closer to the present. In the JOI task, participants were trained on a probabilistic sequence. After training, the sequence was interrupted with probe stimuli. Participants were asked to choose which of two probe stimuli was expected to be presented closer to the present. Replicating prior work on JOR, we found that RT results supported a backward self-terminating search model operating on a temporally organized representation of the past. We also showed that RT distributions are consistent with this model and that the temporally organized representation is compressed. Critically, results for the JOI task probing expectations of the future suggest a forward self-terminating search model operating on a temporally organized representation of the future. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Evidence accumulation in a Laplace domain decision space.

Evidence accumulation models of simple decision-making have long assumed that the brain estimates a scalar decision variable corresponding to the log-likelihood ratio of the two alternatives. Typical neural implementations of this algorithmic cognitive model assume that large numbers of neurons are each noisy exemplars of the scalar decision variable. Here we propose a neural implementation of the diffusion model in which many neurons construct and maintain the Laplace transform of the distance to each of the decision bounds. As in classic findings from brain regions including LIP, the firing rate of neurons coding for the Laplace transform of net accumulated evidence grows to a bound during random dot motion tasks. However, rather than noisy exemplars of a single mean value, this approach makes the novel prediction that firing rates grow to the bound exponentially; across neurons there should be a distribution of different rates. A second set of neurons records an approximate inversion of the Laplace transform; these neurons directly estimate net accumulated evidence. In analogy to time cells and place cells observed in the hippocampus and other brain regions, the neurons in this second set have receptive fields along a "decision axis." This finding is consistent with recent findings from rodent recordings. This theoretical approach places simple evidence accumulation models in the same mathematical language as recent proposals for representing time and space in cognitive models for memory.

Influence of extracellular oscillations on neural communication: a computational perspective.

Neural communication generates oscillations of electric potential in the extracellular medium. In feedback, these oscillations affect the electrochemical processes within the neurons, influencing the timing and the number of action potentials. It is unclear whether this influence should be considered only as noise or it has some functional role in neural communication. Through computer simulations we investigated the effect of various sinusoidal extracellular oscillations on the timing and number of action potentials. Each simulation is based on a multicompartment model of a single neuron, which is stimulated through spatially distributed synaptic activations. A thorough analysis is conducted on a large number of simulations with different models of CA3 and CA1 pyramidal neurons which are modeled using realistic morphologies and active ion conductances. We demonstrated that the influence of the weak extracellular oscillations, which are commonly present in the brain, is rather stochastic and modest. We found that the stronger fields, which are spontaneously present in the brain only in some particular cases (e.g., during seizures) or that can be induced externally, could significantly modulate spike timings.