Memory traces of duration and location in the right intraparietal sulcus.

Time and space form an integral part of every human experience, and for the neuronal representation of these perceptual dimensions, previous studies point to the involvement of the right-hemispheric intraparietal sulcus and structures in the medial temporal lobe. Here we used multi-voxel pattern analysis (MVPA) to investigate long-term memory traces for temporal and spatial stimulus features in those areas. Participants were trained on four images associated with short versus long durations and with left versus right locations. Our results demonstrate stable representations of both temporal and spatial information in the right posterior intraparietal sulcus. Building upon previous findings of stable neuronal codes for directly perceived durations and locations, these results show that the reactivation of long-term memory traces for temporal and spatial features can be decoded from neuronal activation patterns in the right parietal cortex.

Adaptive Encoding Speed in Working Memory.

Humans can adapt when complex patterns unfold at a faster or slower pace, for instance when remembering a grocery list that is dictated at an increasingly fast rate. Integrating information over such timescales crucially depends on working memory, but although recent findings have shown that working memory capacity can be flexibly adapted, such adaptations have not yet been demonstrated for encoding speed. In a series of experiments, we found that young adults encoded at a faster rate when they were adapted to overall and recent stimulus duration. Interestingly, our participants were unable to use explicit cues to speed up encoding, even though these cues were objectively more informative than statistical information. Our findings suggest that adaptive tuning of encoding speed in working memory is a fundamental but largely implicit mechanism underlying our ability to keep up with the pace of our surroundings.

Temporal Context Actively Shapes EEG Signatures of Time Perception.

Our subjective perception of time is optimized to temporal regularities in the environment. This is illustrated by the central tendency effect: When estimating a range of intervals, short intervals are overestimated, whereas long intervals are underestimated to reduce the overall estimation error. Most models of interval timing ascribe this effect to the weighting of the current interval with previous memory traces after the interval has been perceived. Alternatively, the perception of the duration could already be flexibly tuned to its temporal context. We investigated this hypothesis using an interval reproduction task in which human participants (both sexes) reproduced a shorter and longer interval range. As expected, reproductions were biased toward the subjective mean of each presented range. EEG analyses showed that temporal context indeed affected neural dynamics during the perception phase. Specifically, longer previous durations decreased contingent negative variation and P2 amplitude and increased beta power. In addition, multivariate pattern analysis showed that it is possible to decode context from the transient EEG signal quickly after both onset and offset of the perception phase. Together, these results suggest that temporal context creates dynamic expectations which actively affect the perception of duration.SIGNIFICANCE STATEMENT The subjective sense of duration does not arise in isolation, but is informed by previous experiences. This is demonstrated by abundant evidence showing that the production of duration estimates is biased toward previously experienced time intervals. However, it is yet unknown whether this temporal context actively affects perception or only asserts its influence in later, postperceptual stages as proposed by most current formal models of this task. Using an interval reproduction task, we show that EEG signatures flexibly adapt to the temporal context during perceptual encoding. Furthermore, interval history can be decoded from the transient EEG signal even when the current duration was identical. Thus, our results demonstrate that context actively influences perception.

Temporal context effects are associated with cognitive status in advanced age.

The perception of temporal intervals changes during the life-span, and especially older adults demonstrate specific impairments of timing abilities. Recently, we demonstrated that timing performance and cognitive status are correlated in older adults, suggesting that timing tasks can serve as a behavioral marker for the development of dementia. Easy-to-administer and retest-capable timing tasks therefore have potential as diagnostic tools for tracking cognitive decline. However, before being tested in a clinical cohort study, a further validation and specification of the original findings is warranted. Here we introduce several modifications of the original task and investigated the effects of temporal context on time perception in older adults (> 65 years) with low versus high scores in the Montreal Cognitive Assessment survey (MoCA) and a test of memory functioning. In line with our previous work, we found that temporal context effects were more pronounced with increasing memory deficits, but also that these effects are stronger for realistic compared to abstract visual stimuli. Furthermore, we show that two distinct temporal contexts influence timing behavior in separate experimental blocks, as well as in a mixed block in which both contexts are presented together. These results replicate and extend our previous findings. They demonstrate the stability of the effect for different stimulus material and show that timing tasks can reveal valuable information about the cognitive status of older adults. In the future, these findings could serve as a basis for the development of a diagnostic tool for pathological cognitive decline at an early, pre-clinical stage.

Neural Repetition Suppression Modulates Time Perception: Evidence From Electrophysiology and Pupillometry.

Human time perception is malleable and subject to many biases. For example, it has repeatedly been shown that stimuli that are physically intense or that are unexpected seem to last longer. Two competing hypotheses have been proposed to account for such biases: One states that these temporal illusions are the result of increased levels of arousal that speeds up neural clock dynamics, whereas the alternative "magnitude coding" account states that the magnitude of sensory responses causally modulates perceived durations. Common experimental paradigms used to study temporal biases cannot dissociate between these accounts, as arousal and sensory magnitude covary and modulate each other. Here, we present two temporal discrimination experiments where two flashing stimuli demarcated the start and end of a to-be-timed interval. These stimuli could be either in the same or a different location, which led to different sensory responses because of neural repetition suppression. Crucially, changes and repetitions were fully predictable, which allowed us to explore effects of sensory response magnitude without changes in arousal or surprise. Intervals with changing markers were perceived as lasting longer than those with repeating markers. We measured EEG (Experiment 1) and pupil size (Experiment 2) and found that temporal perception was related to changes in ERPs (P2) and pupil constriction, both of which have been related to responses in the sensory cortex. Conversely, correlates of surprise and arousal (P3 amplitude and pupil dilation) were unaffected by stimulus repetitions and changes. These results demonstrate, for the first time, that sensory magnitude affects time perception even under constant levels of arousal.

Memory for Stimulus Duration Is Not Bound to Spatial Information.

Different theories have been proposed to explain how the human brain derives an accurate sense of time. One specific class of theories, intrinsic clock theories, postulate that temporal information of a stimulus is represented much like other features such as color and location, bound together to form a coherent percept. Here, we explored to what extent this holds for temporal information after it has been perceived and is held in working memory for subsequent comparison. We recorded EEG of participants who were asked to time stimuli at lateral positions of the screen followed by comparison stimuli presented in the center. Using well-established markers of working memory maintenance, we investigated whether the usage of temporal information evoked neural signatures that were indicative of the location where the stimuli had been presented, both during maintenance and during comparison. Behavior and neural measures including the contralateral delay activity, lateralized alpha suppression, and decoding analyses through time all supported the same conclusion: The representation of location was strongly involved during perception of temporal information, but when temporal information was to be used for comparison, it no longer showed a relation to spatial information. These results support a model where the initial perception of a stimulus involves intrinsic computations, but that this information is subsequently translated to a stimulus-independent format to be used to further guide behavior.

EEG-based Identification of Evidence Accumulation Stages in Decision-Making.

Dating back to the 19th century, the discovery of processing stages has been of great interest to researchers in cognitive science. The goal of this paper is to demonstrate the validity of a recently developed method, hidden semi-Markov model multivariate pattern analysis (HsMM-MVPA), for discovering stages directly from EEG data, in contrast to classical reaction-time-based methods. To test the validity of stages discovered with the HsMM-MVPA method, we applied it to two relatively simple tasks where the interpretation of processing stages is straightforward. In these visual discrimination EEG data experiments, perceptual processing and decision difficulty were manipulated. The HsMM-MVPA revealed that participants progressed through five cognitive processing stages while performing these tasks. The brain activation of one of those stages was dependent on perceptual processing, whereas the brain activation and the duration of two other stages were dependent on decision difficulty. In addition, evidence accumulation models (EAMs) were used to assess to what extent the results of HsMM-MVPA are comparable to standard reaction-time-based methods. Consistent with the HsMM-MVPA results, EAMs showed that nondecision time varied with perceptual difficulty and drift rate varied with decision difficulty. Moreover, nondecision and decision time of the EAMs correlated highly with the first two and last three stages of the HsMM-MVPA, respectively, indicating that the HsMM-MVPA gives a more detailed description of stages discovered with this more classical method. The results demonstrate that cognitive stages can be robustly inferred with the HsMM-MVPA.

Discovering the brain stages of lexical decision: Behavioral effects originate from a single neural decision process.

Lexical decision (LD) - judging whether a sequence of letters constitutes a word - has been widely investigated. In a typical lexical decision task (LDT), participants are asked to respond whether a sequence of letters is an actual word or a nonword. Although behavioral differences between types of words/nonwords have been robustly detected in LDT, there is an ongoing discussion about the exact cognitive processes that underlie the word identification process in this task. To obtain data-driven evidence on the underlying processes, we recorded electroencephalographic (EEG) data and applied a novel machine-learning method, hidden semi-Markov model multivariate pattern analysis (HsMM-MVPA). In the current study, participants performed an LDT in which we varied the frequency of words (high, low frequency) and "wordlikeness" of non-words (pseudowords, random non-words). The results revealed that models with six processing stages accounted best for the data in all conditions. While most stages were shared, Stage 5 differed between conditions. Together, these results indicate that the differences in word frequency and lexicality effects are driven by a single cognitive processing stage. Based on its latency and topology, we interpret this stage as a Decision process during which participants discriminate between words and nonwords using activated lexical information.

Precision Timing with α-ß Oscillatory Coupling: Stopwatch or Motor Control?

Precise timing is crucial for many behaviors ranging from conversational speech to athletic performance. The precision of motor timing has been suggested to result from the strength of phase-amplitude coupling (PAC) between the phase of alpha oscillations (α, 8-12 Hz) and the power of beta activity (ß, 14-30 Hz), herein referred to as α-ß PAC. The amplitude of ß oscillations has been proposed to code for temporally relevant information and the locking of ß power to the phase of α oscillations to maintain timing precision. Motor timing precision has at least two sources of variability: variability of timekeeping mechanism and variability of motor control. It is ambiguous to which of these two factors α-ß PAC should be ascribed: α-ß PAC could index precision of stopwatch-like internal timekeeping mechanisms, or α-ß PAC could index motor control precision. To disentangle these two hypotheses, we tested how oscillatory coupling at different stages of a time reproduction task related to temporal precision. Human participants encoded and subsequently reproduced a time interval while magnetoencephalography was recorded. The data show a robust α-ß PAC during both the encoding and reproduction of a temporal interval, a pattern that cannot be predicted by motor control accounts. Specifically, we found that timing precision resulted from the trade-off between the strength of α-ß PAC during the encoding and during the reproduction of intervals. These results support the hypothesis that α-ß PAC codes for the precision of temporal representations in the human brain.

Performance-informed EEG analysis reveals mixed evidence for EEG signatures unique to the processing of time.

Certain EEG components (e.g., the contingent negative variation, CNV, or beta oscillations) have been linked to the perception of temporal magnitudes specifically. However, it is as of yet unclear whether these EEG components are really unique to time perception or reflect the perception of magnitudes in general. In the current study we recorded EEG while participants had to make judgments about duration (time condition) or numerosity (number condition) in a comparison task. This design allowed us to directly compare EEG signals between the processing of time and number. Stimuli consisted of a series of blue dots appearing and disappearing dynamically on a black screen. Each stimulus was characterized by its duration and the total number of dots that it consisted of. Because it is known that tasks like these elicit perceptual interference effects that we used a maximum-likelihood estimation (MLE) procedure to determine, for each participant and dimension separately, to what extent time and numerosity information were taken into account when making a judgement in an extensive post hoc analysis. This approach enabled us to capture individual differences in behavioral performance and, based on the MLE estimates, to select a subset of participants who suppressed task-irrelevant information. Even for this subset of participants, who showed no or only small interference effects and thus were thought to truly process temporal information in the time condition and numerosity information in the number condition, we found CNV patterns in the time-domain EEG signals for both tasks that was more pronounced in the time-task. We found no substantial evidence for differences between the processing of temporal and numerical information in the time-frequency domain.

Thermosensory perception regulates speed of movement in response to temperature changes in Drosophila melanogaster.

Temperature influences the physiology and behavior of all organisms. For ectotherms, which lack central temperature regulation, temperature adaptation requires sheltering from or moving to a heat source. As temperature constrains the rate of metabolic reactions, it can directly affect ectotherm physiology and thus behavioral performance. This direct effect is particularly relevant for insects, as their small bodies readily equilibrate with ambient temperature. In fact, models of enzyme kinetics applied to insect behavior predict performance at different temperatures suggesting that thermal physiology governs behavior. However, insects also possess thermosensory neurons critical for locating preferred temperatures, showing cognitive control. This suggests that temperature-related behavior can emerge directly from a physiological effect, indirectly as a consequence of thermosensory processing, or through a combination of both. To separate the roles of thermal physiology and cognitive control, we developed an arena that allows fast temperature changes in time and space, and in which animals' movements are automatically quantified. We exposed wild-type Drosophila melanogaster and thermosensory receptor mutants to a dynamic temperature environment and tracked their movements. The locomotor speed of wild-type flies closely matched models of enzyme kinetics, but the behavior of thermosensory mutants did not. Mutations in thermosensory receptor gene dTrpA1 (Transient Receptor Potential A1) expressed in the brain resulted in a complete lack of response to temperature changes, while mutations in peripheral thermosensory receptor gene Gr28b(D) resulted in a diminished response. We conclude that flies react to temperature through cognitive control, informed by interactions between various thermosensory neurons, the behavioral output of which resembles models of enzyme kinetics.

What You See Is What You Remember: Visual Chunking by Temporal Integration Enhances Working Memory.

Human memory benefits from information clustering, which can be accomplished by chunking. Chunking typically relies on expertise and strategy, and it is unknown whether perceptual clustering over time, through temporal integration, can also enhance working memory. The current study examined the attentional and working memory costs of temporal integration of successive target stimulus pairs embedded in rapid serial visual presentation. ERPs were measured as a function of behavioral reports: One target, two separate targets, or two targets reported as a single integrated target. N2pc amplitude, reflecting attentional processing, depended on the actual number of successive targets. The memory-related CDA and P3 components instead depended on the perceived number of targets irrespective of their actual succession. The report of two separate targets was associated with elevated amplitude, whereas integrated as well as actual single targets exhibited lower amplitude. Temporal integration thus provided an efficient means of processing sensory input, offloading working memory so that the features of two targets were consolidated and maintained at a cost similar to that of a single target.

Pupillary response indexes the metrical hierarchy of unattended rhythmic violations.

The perception of music is a complex interaction between what we hear and our interpretation. This is reflected in beat perception, in which a listener infers a regular pulse from a musical rhythm. Although beat perception is a fundamental human ability, it is still unknown whether attention to the music is necessary to establish the perception of stronger and weaker beats, or meter. In addition, to what extent beat perception is dependent on musical expertise is still a matter of debate. Here, we address these questions by measuring the pupillary response to omissions at different metrical positions in drum rhythms, while participants attended to another task. We found that the omission of the salient first beat elicited a larger pupil dilation than the omission of the less-salient second beat. This result shows that participants not only detected the beat without explicit attention to the music, but also perceived a metrical hierarchy of stronger and weaker beats. This suggests that hierarchical beat perception is an automatic process that requires no or minimal attentional resources. In addition, we found no evidence for the hypothesis that hierarchical beat perception is affected by musical expertise, suggesting that elementary beat perception might be independent from musical expertise. Finally, our results show that pupil dilation reflects surprise without explicit attention, demonstrating that the pupil is an accessible index to signatures of unattentive processing.

Contrasting single and multi-component working-memory systems in dual tasking.

Working memory can be a major source of interference in dual tasking. However, there is no consensus on whether this interference is the result of a single working memory bottleneck, or of interactions between different working memory components that together form a complete working-memory system. We report a behavioral and an fMRI dataset in which working memory requirements are manipulated during multitasking. We show that a computational cognitive model that assumes a distributed version of working memory accounts for both behavioral and neuroimaging data better than a model that takes a more centralized approach. The model's working memory consists of an attentional focus, declarative memory, and a subvocalized rehearsal mechanism. Thus, the data and model favor an account where working memory interference in dual tasking is the result of interactions between different resources that together form a working-memory system.

Decoupling interval timing and climbing neural activity: a dissociation between CNV and N1P2 amplitudes.

It is often argued that climbing neural activity, as for example reflected by the contingent negative variation (CNV) in the electroencephalogram, is the signature of the subjective experience of time. According to this view, the resolution of the CNV coincides with termination of subjective timing processes. Paradoxically, behavioral data indicate that participants keep track of timing even after the standard interval (SI) has passed. This study addresses whether timing continues after CNV resolution. In Experiment 1, human participants were asked to discriminate time intervals while evoked potentials (EPs) elicited by the sound terminating a comparison interval (CI) were measured. As the amplitude of N1P2 components increases as a function of the temporal distance from the SI, and the latency of the P2 component followed the hazard rate of the CIs, timing processes continue after CNV resolution. Based on a novel experimental paradigm, statistical model comparisons and trial-by-trial analyses, Experiment 2 supports this finding as subjective time is more accurately indexed by the amplitude of early EPs than by CNV amplitude. These results provide the first direct evidence that subjective timing of multisecond intervals does not depend on climbing neural activity as indexed by the CNV and that the subjective experience of time is better reflected by distinct features of post-CI evoked potentials.

Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution.

The size of the human pupil increases as a function of mental effort. However, this response is slow, and therefore its use is thought to be limited to measurements of slow tasks or tasks in which meaningful events are temporally well separated. Here we show that high-temporal-resolution tracking of attention and cognitive processes can be obtained from the slow pupillary response. Using automated dilation deconvolution, we isolated and tracked the dynamics of attention in a fast-paced temporal attention task, allowing us to uncover the amount of mental activity that is critical for conscious perception of relevant stimuli. We thus found evidence for specific temporal expectancy effects in attention that have eluded detection using neuroimaging methods such as EEG. Combining this approach with other neuroimaging techniques can open many research opportunities to study the temporal dynamics of the mind's inner eye in great detail.

Validation of a simple response-time measure of listening effort.

This study compares two response-time measures of listening effort that can be combined with a clinical speech test for a more comprehensive evaluation of total listening experience; verbal response times to auditory stimuli (RT(aud)) and response times to a visual task (RTs(vis)) in a dual-task paradigm. The listening task was presented in five masker conditions; no noise, and two types of noise at two fixed intelligibility levels. Both the RTs(aud) and RTs(vis) showed effects of noise. However, only RTs(aud) showed an effect of intelligibility. Because of its simplicity in implementation, RTs(aud) may be a useful effort measure for clinical applications.

Single-task fMRI overlap predicts concurrent multitasking interference.

There is no consensus regarding the origin of behavioral interference that occurs during concurrent multitasking. Some evidence points toward a multitasking locus in the brain, while other results imply that interference is the consequence of task interactions in several brain regions. To investigate this issue, we conducted a functional MRI (fMRI) study consisting of three component tasks, which were performed both separately and in combination. The results indicated that no specific multitasking area exists. Instead, different patterns of activation across conditions could be explained by assuming that the interference is a result of task interactions. Additionally, similarity in single-task activation patterns correlated with a decrease in accuracy during dual-task conditions. Taken together, these results support the view that multitasking interference is not due to a bottleneck in a single "multitasking" brain region, but is a result of interactions between concurrently running processes.

Trial-by-trial fluctuations in CNV amplitude reflect anticipatory adjustment of response caution.

The contingent negative variation, a slow cortical potential, occurs when humans are warned by a stimulus about an upcoming task. The cognitive processes that give rise to this EEG potential are not yet well understood. To explain these processes, we adopt a recently developed theoretical framework from the area of perceptual decision-making. This framework assumes that the basal ganglia control the tradeoff between fast and accurate decision-making in the cortex. It suggests that an increase in cortical excitability serves to lower response caution, which results in faster but more error prone responding. We propose that the CNV reflects this increased cortical excitability. To test this hypothesis, we conducted an EEG experiment in which participants performed the random dot motion task either under speed or under accuracy stress. Our results show that trial-by-trial fluctuations in participants' response speed as well as model-based estimates of response caution correlated with single-trial CNV amplitude under conditions of speed but not accuracy stress. We conclude that the CNV might reflect adjustments of response caution, which serves to enhance quick decision-making.

Dissociable mechanisms underlying individual differences in visual working memory capacity.

Individuals scoring relatively high on measures of working memory tend to be more proficient at controlling attention to minimize the effect of distracting information. It is currently unknown whether such superior attention control abilities are mediated by stronger suppression of irrelevant information, enhancement of relevant information, or both. Here we used steady-state visual evoked potentials (SSVEPs) with the Eriksen flanker task to track simultaneously the attention to relevant and irrelevant information by tagging target and distractors with different frequencies. This design allowed us to dissociate attentional biasing of perceptual processing (via SSVEPs) and stimulus processing in the frontal cognitive control network (via time-frequency analyses of EEG data). We show that while preparing for the upcoming stimulus, high- and low-WMC individuals use different strategies: High-WMC individuals show attentional suppression of the irrelevant stimuli, whereas low-WMC individuals demonstrate attentional enhancement of the relevant stimuli. Moreover, behavioral performance was predicted by trial-to-trial fluctuations in strength of distractor-suppression for high-WMC participants. We found no evidence for WMC-related differences in cognitive control network functioning, as measured by midfrontal theta-band power. Taken together, these findings suggest that early suppression of irrelevant information is a key underlying neural mechanism by which superior attention control abilities are implemented.