Striatal dopamine and the temporal control of behavior.

Striatal dopamine strongly regulates how individuals use time to guide behavior. Dopamine acts on D1- and D2- dopamine receptors in the striatum. However, the relative role of these receptors in the temporal control of behavior is unclear. To assess this, we trained rats on a task in which they decided to start and stop a series of responses based on the passage of time and evaluated how blocking D1 or D2-dopamine receptors in the dorsomedial or dorsolateral striatum impacted performance. D2 blockade delayed the decision to start and stop responding in both regions, and this effect was larger in the dorsomedial striatum. By contrast, dorsomedial D1 blockade delayed stop times, without significantly delaying start times, whereas dorsolateral D1 blockade produced no detectable effects. These findings suggest that striatal dopamine may tune decision thresholds during timing tasks. Furthermore, our data indicate that the dorsomedial striatum plays a key role in temporal control, which may be useful for localizing neural circuits that mediate the temporal control of action.

Recalibrating timing behavior via expected covariance between temporal cues.

Individuals must predict future events to proactively guide their behavior. Predicting when events will occur is a critical component of these expectations. Temporal expectations are often generated based on individual cue-duration relationships. However, the durations associated with different environmental cues will often co-vary due to a common cause. We show that timing behavior may be calibrated based on this expected covariance, which we refer to as the 'common cause hypothesis'. In five experiments using rats, we found that when the duration associated with one temporal cue changes, timed-responding to other cues shift in the same direction. Furthermore, training subjects that expecting covariance is not appropriate in a given situation blocks this effect. Finally, we confirmed that this transfer is context-dependent. These results reveal a novel principle that modulates timing behavior, which we predict will apply across a variety of magnitude-expectations.

5-HT1a Receptor Involvement in Temporal Memory and the Response to Temporal Ambiguity.

It has previously been demonstrated that rats trained on the peak-interval procedure to associate two different cues with two different fixed interval schedules will generate a scalar peak function at an intermediate time when presented with the compound cue. This response pattern has been interpreted as resulting from the simultaneous retrieval of different temporal memories, and a consequential averaging process to resolve the ambiguity. In the present set of studies, we investigated the role that serotonin 1a receptors play in this process. In Experiment 1, rats were trained on a peak-interval procedure to associate the interoceptive states induced by saline and the 5-HT1a agonist, 8-OH-DPAT, with a 5 s or 20 s fixed-interval schedule signaled by the same tone cue (counter-balanced). While peak functions following administration of saline were centered at the appropriate time (5 s or 20 s), peak functions following administration of the agonist were centered around 7 s, irrespective of the reinforced time during training, suggesting agonist-induced disruption in selective temporal memory retrieval, resulting in increased ambiguity regarding the appropriate time at which to respond. In Experiment 2, rats were trained in a peak-interval procedure to associate a tone cue with a 10 s fixed interval and a light cue with a 20 s fixed interval. Administration of the 5-HT1a antagonist, WAY-100635, had no impact on timing when single cues were presented, but altered the intermediate, scalar, response to the stimulus compound, suggesting antagonist-induced disruption in the processes used to deal with temporal memory ambiguity. Together, these data suggest that manipulations of 5HT transmission at the 5-HT1a receptor cause changes in the temporal pattern of responding that are consistent with alterations in temporal memory processes and responses to temporal ambiguity.

Temporal specificity in Pavlovian-to-instrumental transfer.

Presentation of a previously trained Pavlovian conditioned stimulus while an organism is engaged in operant responding can moderate the rate of responding, a phenomenon known as Pavlovian-to-instrumental transfer. Although it is well known that Pavlovian contingencies will generate conditioned behavior that is temporally organized with respect to the arrival of the predicted outcome, little work has examined the temporal dynamics of responding during Pavlovian-instrumental transfer. We trained rats using a fixed time 60-sec, fixed time 120-sec, or random time 60-sec schedule in an appetitive Pavlovian task, and found that presentation of the conditioned stimulus potentiated operant responding in a manner that reflected these previously established temporal expectancies. Further, this temporal specificity conformed to the scalar property as seen with other forms of interval timing behavior. Surprisingly, this effect was only seen when the conditioned stimulus was a visual cue, but not when it was an auditory cue. These data suggest that the motivational processes triggered by Pavlovian cues are not static, but fluctuate in strength as a function of temporally specific expectations of reward.

Rodent Medial Frontal Control of Temporal Processing in the Dorsomedial Striatum.

Although frontostriatal circuits are critical for the temporal control of action, how time is encoded in frontostriatal circuits is unknown. We recorded from frontal and striatal neurons while rats engaged in interval timing, an elementary cognitive function that engages both areas. We report four main results. First, "ramping" activity, a monotonic change in neuronal firing rate across time, is observed throughout frontostriatal ensembles. Second, frontostriatal activity scales across multiple intervals. Third, striatal ramping neurons are correlated with activity of the medial frontal cortex. Finally, interval timing and striatal ramping activity are disrupted when the medial frontal cortex is inactivated. Our results support the view that striatal neurons integrate medial frontal activity and are consistent with drift-diffusion models of interval timing. This principle elucidates temporal processing in frontostriatal circuits and provides insight into how the medial frontal cortex exerts top-down control of cognitive processing in the striatum.SIGNIFICANCE STATEMENT The ability to guide actions in time is essential to mammalian behavior from rodents to humans. The prefrontal cortex and striatum are critically involved in temporal processing and share extensive neuronal connections, yet it remains unclear how these structures represent time. We studied these two brain areas in rodents performing interval-timing tasks and found that time-dependent "ramping" activity, a monotonic increase or decrease in neuronal activity, was a key temporal signal. Furthermore, we found that striatal ramping activity was correlated with and dependent upon medial frontal activity. These results provide insight into information-processing principles in frontostriatal circuits.

Interval timing, temporal averaging, and cue integration.

In a series of recent experiments, we found that if rats are presented with two temporal cues, each signifying that reward will be delivered after a different duration elapses (e.g., tone-10 seconds / light-20 seconds), they will behave as if they have computed a weighted average of these respective durations. In the current article, we argue that this effect, referred to as "temporal averaging", can be understood within the context of Bayesian Decision Theory. Specifically, we propose and provide preliminary data showing that, when averaging, rats weight different durations based on the relative variability of the information their respective cues provide.

Temporal averaging across multiple response options: insight into the mechanisms underlying integration.

Rats trained on a dual-duration, dual-modality peak-interval procedure (e.g., tone = 10 s/light = 20 s) often show unimodal response distributions with peaks that fall in between the anchor durations when both cues are presented as a simultaneous compound. Two hypotheses can explain this finding. According to the averaging hypothesis, rats integrate the anchor durations into an average during compound trials, with each duration being weighted by its respective reinforcement probability. According to the simultaneous temporal processing hypothesis, rats time both durations veridically and simultaneously during compound trials and respond continuously across both durations, thereby producing a unimodal response distribution with a peak falling in between the anchor durations. In the present compounding experiment, rats were trained to associate a tone and light with two different durations (e.g., 5 and 20 s, respectively). However, in contrast to previous experiments, each cue was also associated with a distinct response requirement (e.g., left nosepoke for tone/right nosepoke for light). On the majority of compound trials, responding on a given nosepoke fell close to its respective duration, but was shifted in the direction of the other cue's duration, suggesting rats timed an average of the two durations. However, more weight appeared to be given to the duration associated with the manipulandum on which the rat responded, rather than the duration associated with a higher reinforcement probability as predicted by the averaging hypothesis. Group differences were also observed, with rats trained to associate the tone and light with the short and long durations, respectively, being more likely to show these shifts than the counterbalanced modality-duration group (i.e., light-short/tone-long). This parallels group differences observed in past studies and suggest that cue weighting in response to stimulus compounds is influenced by the modality-duration relationship of the anchor cues. The current results suggest that temporal averaging is a more flexible process than previously theorized and provide novel insight into the mechanisms that affect cue weighting.

The influence of multiple temporal memories in the peak-interval procedure.

Memories for when an event has occurred are used to anticipate future occurrences of the event, but what happens when the event is equally likely to occur at two different times? In this study, one group of rats was always reinforced at 21 s on the peak-interval procedure (21-only group), whereas another group of rats was reinforced at either 8 or 21 s, which varied daily (8-21 group). At the beginning of each session, the behavior of the 8-21 group largely lacked temporal control, but by the end of the session, temporal control was reestablished. When both groups were reinforced at 21 s, the patterns of responding were indistinguishable after subjects in the 8-21 group had experienced 13 reinforcement trials. Finally, the reinforcement times of previous sessions affected the 8-21 group, such that subjects were biased depending on the reinforcement time of the prior session. These results show that when the reinforcement time is initially ambiguous, rats respond in a way that combines their expectations of both possibilities; then they incrementally adjust their responding as they receive more information, but still information from prior sessions biases their initial expectation for the reinforcement time. Combined, these results imply that rats are sensitive to the age of encoded temporal memories in an environment in which the reinforcement time is variable. How these results inform the scalar expectancy theory, the currently accepted model of interval-timing behavior, is discussed.

Searching for the holy grail: temporally informative firing patterns in the rat.

This chapter reviews our work from the past decade investigating cortical and striatal firing patterns in rats while they time intervals in the multi-seconds range. We have found that both cortical and striatal firing rates contain information that the rat can use to identify how much time has elapsed both from trial onset and from the onset of an active response state. I describe findings showing that the striatal neurons that are modulated by time are also modulated by overt behaviors, suggesting that time modulates the strength of motor coding in the striatum, rather than being represented as an abstract quantity in isolation. I also describe work showing that there are a variety of temporally informative activity patterns in pre-motor cortex, and argue that the heterogeneity of these patterns can enhance an organism's temporal estimate. Finally, I describe recent behavioral work from my lab in which the simultaneous cueing of multiple durations leads to a scalar temporal expectation at an intermediate time, providing strong support for a monotonic representation of time.

Reinforcement probability modulates temporal memory selection and integration processes.

We have previously shown that rats trained in a mixed-interval peak procedure (tone=4s, light=12s) respond in a scalar manner at a time in between the trained peak times when presented with the stimulus compound (Swanton & Matell, 2011). In our previous work, the two component cues were reinforced with different probabilities (short=20%, long=80%) to equate response rates, and we found that the compound peak time was biased toward the cue with the higher reinforcement probability. Here, we examined the influence that different reinforcement probabilities have on the temporal location and shape of the compound response function. We found that the time of peak responding shifted as a function of the relative reinforcement probability of the component cues, becoming earlier as the relative likelihood of reinforcement associated with the short cue increased. However, as the relative probabilities of the component cues grew dissimilar, the compound peak became non-scalar, suggesting that the temporal control of behavior shifted from a process of integration to one of selection. As our previous work has utilized durations and reinforcement probabilities more discrepant than those used here, these data suggest that the processes underlying the integration/selection decision for time are based on cue value.

Timing in a variable interval procedure: evidence for a memory singularity.

Rats were trained in either a 30 s peak-interval procedure, or a 15-45 s variable interval peak procedure with a uniform distribution (Exp 1) or a ramping probability distribution (Exp 2). Rats in all groups showed peak shaped response functions centered around 30 s, with the uniform group having an earlier and broader peak response function and rats in the ramping group having a later peak function as compared to the single duration group. The changes in these mean functions, as well as the statistics from single trial analyses, can be better captured by a model of timing in which memory is represented by a single, average, delay to reinforcement compared to one in which all durations are stored as a distribution, such as the complete memory model of Scalar Expectancy Theory or a simple associative model. This article is part of a Special Issue entitled: Associative and Temporal Learning.

Hippocampus, time, and memory--a retrospective analysis.

In 1984, there was considerable evidence that the hippocampus was important for spatial learning and some evidence that it was also involved in duration discrimination. The article "Hippocampus, Time, and Memory" (Meck, Church, & Olton, 1984), however, was the first to isolate the effects of hippocampal damage on specific stages of temporal processing. In this review, to celebrate the 30th anniversary of Behavioral Neuroscience, we look back on factors that contributed to the long-lasting influence of this article. The major results were that a fimbria-fornix lesion (a) interferes with the ability to retain information in temporal working memory, and (b) distorts the content of temporal reference memory, but (c) did not decrease sensitivity to signal duration. This was the first lesion experiment in which the results were interpreted by a well-developed theory of behavior (scalar timing theory). It has led to extensive research on the role of the hippocampus in temporal processing by many investigators. The most important ones are the development of computational models with plausible neural mechanisms (such as the striatal beat-frequency model of interval timing), the use of multiple behavioral measures of timing, and empirical research on the neural mechanisms of timing and temporal memory using ensemble recording of neurons in prefrontal-striatal-hippocampal circuits.

Temporal memory averaging and post-encoding alterations in temporal expectation.

Recent work in our lab has demonstrated that rats trained to associate two different reinforcement delays with two different cues will generate a scalar temporal expectation at a time between these delays when presented with the cue compound. This work demonstrates that rats will integrate distinct temporal memories at retrieval, revealing that temporal expectation need not be a veridical representation of experience. Following from this recognition that processes occurring at or after memory retrieval may transform or bias temporal expectations, we suggest that previous pharmacological work that had been interpreted as resulting from sensorial, or clock-speed, changes, may be alternatively interpreted as resulting from mnemonic alterations. We end with a brief review of the impact of post-encoding alterations of memory on behavior other than timing.

Behavioral sensitivity of temporally modulated striatal neurons.

Recent investigations into the neural mechanisms that underlie temporal perception have revealed that the striatum is an important contributor to interval timing processes, and electrophysiological recording studies have shown that the firing rates of striatal neurons are modulated by the time in a trial at which an operant response is made. However, it remains unclear whether striatal firing rate modulations are related to the passage of time alone (i.e., whether temporal information is represented in an "abstract" manner independent of other attributes of biological importance), or whether this temporal information is embedded within striatal activity related to co-occurring contextual information, such as motor behaviors. This study evaluated these two hypotheses by recording from striatal neurons while rats performed a temporal production task. Rats were trained to respond at different nosepoke apertures for food reward under two simultaneously active reinforcement schedules: a variable-interval (VI-15 s) schedule and a fixed-interval (FI-15 s) schedule of reinforcement. Responding during a trial occurred in a sequential manner composing three phases; VI responding, FI responding, VI responding. The vast majority of task-sensitive striatal neurons (95%) varied their firing rates associated with equivalent behaviors (e.g., periods in which their snout was held within the nosepoke) across these behavioral phases, and 96% of cells varied their firing rates for the same behavior within a phase, thereby demonstrating their sensitivity to time. However, in a direct test of the abstract timing hypothesis, 91% of temporally modulated "hold" cells were further modulated by the overt motor behaviors associated with transitioning between nosepokes. As such, these data are inconsistent with the striatum representing time in an "abstract' manner, but support the hypothesis that temporal information is embedded within contextual and motor functions of the striatum.

Nucleus accumbens dopamine modulates response rate but not response timing in an interval timing task.

While previous work has demonstrated that systemic dopamine manipulations can modulate temporal perception by altering the speed of internal clock processes, the neural site of this modulation remains unclear. Based on recent research suggesting that changes in incentive salience can alter the perception of time, as well as work showing that nucleus accumbens (NAc) shell dopamine (DA) levels modulate the incentive salience of discriminative stimuli that predict instrumental outcomes, we assessed whether microinjections of DA agents into the NAc shell would impact temporal perception. Rats were trained on either a 10-s or 30-s temporal production procedure and received intra-NAc shell microinfusions of sulpiride, amphetamine, and saline. Results showed that NAc DA modulations had no effect on response timing, but intra-NAc shell sulpiride microinfusions significantly decreased response rates relative to saline and amphetamine. Our findings therefore suggest that neither NAc shell DA levels, nor the resultant changes in incentive salience signaled by this structure, impact temporal control.

A heterogeneous population code for elapsed time in rat medial agranular cortex.

The neural mechanisms underlying the temporal control of behavior are largely unknown. Here we recorded from medial agranular cortex neurons in rats while they freely behaved in a temporal production task, the peak-interval procedure. Due to variability in estimating the time of food availability, robust responding typically bracketed the expected duration, starting some time before and ending some time after the signaled delay. These response periods provided analytic "steady state" windows during which subjects actively indicated their temporal expectation of food availability. Remarkably, during these response periods, a variety of firing patterns were seen that could be broadly described as ramps, peaks, and dips, with different slopes, directions, and times at which maxima or minima occur. Regularized linear discriminant analysis indicated that these patterns provided sufficiently reliable information to discriminate the elapsed duration of responding within these response periods. Modeling this across neuron variability showed that the utilization of ramps, dips, and peaks, with different slopes and minimal/maximal rates at different times, led to a substantial improvement in temporal prediction errors, suggesting that heterogeneity in the neural representation of elapsed time may facilitate temporally controlled behavior.

Stimulus compounding in interval timing: the modality-duration relationship of the anchor durations results in qualitatively different response patterns to the compound cue.

We have previously demonstrated that rats trained on a two-duration peak procedure in which two modal signals (i.e., tone and houselight) predicted probabilistic reinforcement availability at two times (10 s and 20 s) would respond in a scalar manner at a time between the trained durations in response to the simultaneous compound cue (tone + houselight). In these experiments, we evaluated whether this scalar response pattern would remain with greater relative separation between the anchor durations. Results revealed an effect of the modality-duration relationship, such that scalar responding was seen on compound trials in rats trained that the auditory stimulus signaled the shorter duration, whereas the visual stimulus signaled the longer duration, but not in the reverse condition. In rats showing scalar responding on compound trials, post hoc analyses demonstrated that the peak time of compound responding was most accurately predicted by the reinforcement probability weighted average of anchor peak times. In contrast, rats trained that the visual stimulus signaled the shorter duration, whereas the auditory stimulus signaled the longer duration, responded in a highly rightward skewed manner. In these rats, initiation of responding to the compound stimulus appeared to be controlled by the visual stimulus only, whereas response terminations reflected control by both modal stimuli. These latter data provide evidence of separate determinants of response initiation and termination.

Fast forward: supramarginal gyrus stimulation alters time measurement.

The neural basis of temporal processing is unclear. We addressed this important issue by performing two experiments in which repetitive transcranial magnetic stimulation (rTMS) was administered in different sessions to the left or right supramarginal gyrus (SMG) or vertex; in both tasks, two visual stimuli were presented serially and subjects were asked to judge if the second stimulus was longer than the first (standard) stimulus. rTMS was presented on 50% of trials. Consistent with a previous literature demonstrating the effect of auditory clicks on temporal judgment, rTMS was associated with a tendency to perceive the paired visual stimulus as longer in all conditions. Crucially, rTMS to the right SMG was associated with a significantly greater subjective prolongation of the associated visual stimulus in both experiments. These findings demonstrate that the right SMG is an important element of the neural system underlying temporal processing and, as discussed, have implications for neural and cognitive models of temporal perception and attention.

Averaging of temporal memories by rats.

Rats were trained on a mixed fixed-interval schedule in which stimulus A (tone or light) indicated food availability after 10 s and stimulus B (the other stimulus) indicated food availability after 20 s. Testing consisted of nonreinforced probe trials in which the stimulus was A, B, or the compound AB. On single-stimulus trials, rats responded with a peak of activity around the programmed reinforced time. On compound-stimulus trials, rats showed a single scalar peak of responding at a time midway between those for stimulus A and B. These results suggest that when provided with discrepant information regarding the temporal predictability of reinforcement, rats compute an average of the scheduled reinforcement times for the A and B stimuli and use this average to generate an expectation of reward for the compound stimuli.