Temporal context modulates cross-modality time discrimination: Electrophysiological evidence for supramodal temporal representation.

Although the peripheral nervous system lacks a dedicated receptor, the brain processes temporal information through different sensory channels. A critical question is whether temporal information from different sensory modalities at different times forms modality-specific representations or is integrated into a common representation in a supramodal manner. Behavioral studies on temporal memory mixing and the central tendency effect have provided evidence for supramodal temporal representations. We aimed to provide electrophysiological evidence for this proposal by employing a cross-modality time discrimination task combined with electroencephalogram (EEG) recordings. The task maintained a fixed auditory standard duration, whereas the visual comparison duration was randomly selected from the short and long ranges, creating two different audio-visual temporal contexts. The behavioral results showed that the point of subjective equality (PSE) in the short context was significantly lower than that in the long context. The EEG results revealed that the amplitude of the contingent negative variation (CNV) in the short context was significantly higher (more negative) than in the long context in the early stage, while it was lower (more positive) in the later stage. These results suggest that the audiovisual temporal context is integrated with the auditory standard duration to generate a subjective time criterion. Compared with the long context, the subjective time criterion in the short context was shorter, resulting in earlier decision-making and a preceding decrease in CNV. Our study provides electrophysiological evidence that temporal information from different modalities inputted into the brain at different times can form a supramodal temporal representation.

Time cells in the retrosplenial cortex.

The retrosplenial cortex (RSC) is a key component of the brain's memory systems, with anatomical connections to the hippocampus, anterior thalamus, and entorhinal cortex. This circuit has been implicated in episodic memory and many of these structures have been shown to encode temporal information, which is critical for episodic memory. For example, hippocampal time cells reliably fire during specific segments of time during a delay period. Although RSC lesions are known to disrupt temporal memory, time cells have not been observed there. In this study, we reanalyzed archival RSC neuronal firing data during the intertrial delay period from two previous experiments involving different behavioral tasks, a blocked alternation task and a cued T-maze task. For the blocked alternation task, rats were required to approach the east or west arm of a plus maze for reward during different blocks of trials. Because the reward locations were not cued, the rat had to remember the goal location for each trial. In the cued T-maze task, the reward location was explicitly cued with a light and the rats simply had to approach the light for reward, so there was no requirement to hold a memory during the intertrial delay. Time cells were prevalent in the blocked alternation task, and most time cells clearly differentiated the east and west trials. We also found that RSC neurons could exhibit off-response time fields, periods of reliably inhibited firing. Time cells were also observed in the cued T-maze, but they were less prevalent and they did not differentiate left and right trials as well as in the blocked alternation task, suggesting that RSC time cells are sensitive to the memory demands of the task. These results suggest that temporal coding is a prominent feature of RSC firing patterns, consistent with an RSC role in episodic memory.

The influence of emotion on temporal context models.

Temporal context models (TCMs) have been influential in understanding episodic memory and its neural underpinnings. Recently, TCMs have been extended to explain emotional memory effects, one of the most clinically important findings in the field of memory research. This review covers recent advances in hypotheses for the neural representation of spatiotemporal context through the lens of TCMs, including their ability to explain the influence of emotion on episodic and temporal memory. In recent years, simplifying assumptions of "classical" TCMs - with exponential trace decay and the mechanism by which temporal context is recovered - have become increasingly clear. The review also outlines how recent advances could be incorporated into a future TCM, beyond classical assumptions, to integrate emotional modulation.

Temporal context effects on suboptimal choice.

Choice can be driven both by rewards and stimuli that signal those rewards. Under certain conditions, pigeons will prefer options that lead to less probable reward when the reward is signaled. A recently quantified model, the Signal for Good News (SiGN) model, assumes that in the context of uncertainty, signals for a reduced delay to reward reinforce choice. The SiGN model provides an excellent fit to previous results from pigeons and the current studies are the first to test a priori quantitative predictions. Pigeons chose between a suboptimal alternative that led to signaled 20% food and an optimal alternative that led to 50% food. The duration of the choice period was manipulated across conditions in two experiments. Pigeons strongly preferred the suboptimal alternative at the shorter durations and strongly preferred the optimal alternative at the longer durations. The results from both experiments fit well with predictions from the SiGN model and show that altering the duration of the choice period has a dramatic effect in that it changes which of the two options pigeons prefer. More generally, these results suggest that the relative value of options is not fixed, but instead depends on the temporal context.

Time Cells in the Retrosplenial Cortex.

The retrosplenial cortex (RSC) is a key component of the brain's memory systems, with anatomical connections to the hippocampus, anterior thalamus, and entorhinal cortex. This circuit has been implicated in episodic memory and many of these structures have been shown to encode temporal information, which is critical for episodic memory. For example, hippocampal time cells reliably fire during specific segments of time during a delay period. Although RSC lesions are known to disrupt temporal memory, time cells have not been observed there. In the present study, we examined the firing patterns of RSC neurons during the intertrial delay period of two behavioral tasks, a blocked alternation task and a cued T-maze task. For the blocked alternation task, rats were required to approach the east or west arm of a plus maze for reward during different blocks of trials. Because the reward locations were not cued, the rat had to remember the goal location for each trial. In the cued T-maze task, the reward location was explicitly cued with a light and the rats simply had to approach the light for reward, so there was no requirement to hold a memory during the intertrial delay. Time cells were prevalent in the blocked alternation task, and most time cells clearly differentiated the east and west trials. We also found that RSC neurons could exhibit off-response time fields, periods of reliably inhibited firing. Time cells were also observed in the cued T-maze, but they were less prevalent and they did not differentiate left and right trials as well as in the blocked alternation task, suggesting that RSC time cells are sensitive to the memory demands of the task. These results suggest that temporal coding is a prominent feature of RSC firing patterns, consistent with an RSC role in episodic memory.

Is "memory-for-when" universal? Group and individual variability in temporal position memory for words, faces, and classrooms.

Many contemporary theories of memory assume that everyone automatically stores temporal contextual information about all types of encountered information, yet most studies on this topic have used words and ignored individual differences. Five experiments accumulated evidence that explicit storage of temporal context information does not appear to occur automatically for all people and types of memoranda. We collected judgments of temporal position (memory-for-when) for words (Experiments 1 & 3), faces (Experiments 2A, 3, 4, and 5), and classrooms (Experiments 2B & 3). At the group level, for each of these memoranda memory-for-when was sensitive to the original input position and showed a temporal primacy effect reflecting better memory for position for items near the beginning of the list, indicating some automatic storage of temporal context information. However, memory-for-when was significantly better for words than classrooms, with faces in the middle. Moreover, individuals varied dramatically in their ability to indicate memory-for-when, especially for classrooms where many people performed at or near chance. Taken together, the data suggest that explicit memory-for-when may be dissociable from the more implicit use of temporal contextual information that is theorised to occur during free recall.

Brain representations of space and time in episodic memory: A systematic review and meta-analysis.

All experiences preserved within episodic memory contain information on the space and time of events. The hippocampus is the main brain region involved in processing spatial and temporal information for incorporation within episodic memory representations. However, the other brain regions involved in the encoding and retrieval of spatial and temporal information within episodic memory are unclear, because a systematic review of related studies is lacking and the findings are scattered. The present study was designed to integrate the results of functional magnetic resonance imaging and positron emission tomography studies by means of a systematic review and meta-analysis to provide converging evidence. In particular, we focused on identifying the brain regions involved in the retrieval of spatial and temporal information. We identified a spatial retrieval network consisting of the inferior temporal gyrus, parahippocampal gyrus, superior parietal lobule, angular gyrus, and precuneus. Temporal context retrieval was supported by the dorsolateral prefrontal cortex. Thus, the retrieval of spatial and temporal information is supported by different brain regions, highlighting their different natures within episodic memory.

The attentional boost effect in free recall dynamics.

With the attentional boost effect (ABE), responding to a briefly presented target in a detection task enhances the encoding of other items presented at the same time. However, the effects of target detection on context memory for the event in which the stimulus appeared remain unclear. Here, we present findings from verbal free recall and recognition experiments that test the effects of target detection during encoding on temporal and relational aspects of context memory. Consistent with prior demonstrations of limited effects of target detection on context memory, in Experiment 1 there was no evidence that target detection influenced the likelihood of transitioning to items that were presented at similar times during encoding, or that were in the same encoding condition. These null effects were replicated in a second experiment, which added an old/new recognition and relational memory test. These results indicate that target detection during encoding has minimal effects on the formation of temporal associations between words in memory.

Rapid calibration to dynamic temporal contexts.

The prediction of future events and the preparation of appropriate behavioural reactions rely on an accurate perception of temporal regularities. In dynamic environments, temporal regularities are subject to slow and sudden changes, and adaptation to these changes is an important requirement for efficient behaviour. Bayesian models have proven a useful tool to understand the processing of temporal regularities in humans; yet an open question pertains to the degree of flexibility of the prior that is required for optimal modelling of behaviour. Here we directly compare dynamic models (with continuously changing prior expectations) and static models (a stable prior for each experimental session) with their ability to describe regression effects in interval timing. Our results show that dynamic Bayesian models are superior when describing the responses to slow, continuous environmental changes, whereas static models are more suitable to describe responses to sudden changes. In time perception research, these results will be informative for the choice of adequate computational models and enhance our understanding of the neuronal computations underlying human timing behaviour.

BTCRSleep: a boundary temporal context refinement-based fully convolutional network for sleep staging with single-channel EEG.

Objective. Sleep staging studies on single-channel EEG mainly exploit deep learning methods that combine convolutional neural networks (CNNs) and recurrent neural networks. However, when typical brain waves (such as K-complexes or sleep spindles) that identify sleep stages span two epochs, the abstract process of a CNN extracting features from each sleep stage may cause the loss of boundary context information. This study attempts to capture the boundary context, which contains the characteristics of brain waves during sleep stage transition, to improve the performance of sleep staging.Approach. In this paper we propose a fully convolutional network with boundary temporal context refinement, called BTCRSleep (Boundary Temporal Context Refinement Sleep). The boundary temporal context refinement module refines the boundary information on sleep stages on the basis of extracting multi-scale temporal dependences between epochs and enhances the abstract capability of the boundary temporal context. In addition, we design a class-aware data augmentation method to effectively learn the boundary temporal context between the minority class and other sleep stages.Main results. We evaluate the performance of our proposed network using four public datasets: the 2013 version of Sleep-EDF Expanded (SEDF), the 2018 version of Sleep-EDF Expanded (SEDFX), the Sleep Heart Health Study (SHHS) and CAP Sleep Database (CAP). The evaluation results on the four datasets showed that our model obtains the best total accuracy and kappa score compared with state-of-the-art methods. On average, accuracies of 84.9% in SEDF, 82.9% in SEDFX, 85.2% in SHHS and 76.9% in CAP are obtained under subject-independent cross-validation. We demonstrate that the boundary temporal context contributes to the improvement in capturing the temporal dependences across different epochs.

Attentional fluctuations and the temporal organization of memory.

Event boundaries and temporal context shape the organization of episodic memories. We hypothesized that attentional fluctuations during encoding serve as "events" that affect temporal context representations and recall organization. Individuals encoded trial-unique objects during a modified sustained attention task. Memory was tested with free recall. Response time variability during the encoding tasks was used to characterize "in the zone" and "out of the zone" attentional states. We predicted that: 1) "in the zone", vs. "out of the zone", attentional states should be more conducive to maintaining temporal context representations that can cue temporally organized recall; and 2) temporally distant "in the zone" states may enable more recall "leaps" across intervening items. We replicated several important findings in the sustained attention and memory fields, including more online errors during "out of the zone" vs. "in the zone" attentional states and recall that was temporally structured. Yet, across four studies, we found no evidence for either of our main hypotheses. Recall was robustly temporally organized, and there was no difference in recall organization for items encoded "in the zone" vs. "out of the zone". We conclude that temporal context serves as a strong scaffold for episodic memory, one that can support organized recall even for items encoded during relatively poor attentional states. We also highlight the numerous challenges in striking a balance between sustained attention tasks (long blocks of a repetitive task) and memory recall tasks (short lists of unique items) and describe strategies for researchers interested in uniting these two fields.

Temporal continuity shapes visual responses of macaque face patch neurons.

Macaque inferior temporal cortex neurons respond selectively to complex visual images, with recent work showing that they are also entrained reliably by the evolving content of natural movies. To what extent does temporal continuity itself shape the responses of high-level visual neurons? We addressed this question by measuring how cells in face-selective regions of the macaque visual cortex were affected by the manipulation of a movie's temporal structure. Sampling a 5-min movie at 1 s intervals, we measured neural responses to randomized, brief stimuli of different lengths, ranging from 800 ms dynamic movie snippets to 100 ms static frames. We found that the disruption of temporal continuity strongly altered neural response profiles, particularly in the early response period after stimulus onset. The results suggest that models of visual system function based on discrete and randomized visual presentations may not translate well to the brain's natural modes of operation.

Contextualising facial expressions: The effect of temporal context and individual differences on classification.

The influence of context on facial expression classification is most often investigated using simple cues in static faces portraying basic expressions with a fixed emotional intensity. We examined (1) whether a perceptually rich, dynamic audiovisual context, presented in the form of movie clips (to achieve closer resemblance to real life), affected the subsequent classification of dynamic basic (happy) and non-basic (sarcastic) facial expressions and (2) whether people's susceptibility to contextual cues was related to their ability to classify facial expressions viewed in isolation. Participants classified facial expressions-gradually progressing from neutral to happy/sarcastic in increasing intensity-that followed movie clips. Classification was relatively more accurate and faster when the preceding context predicted the upcoming expression, compared with when the context did not. Speeded classifications suggested that predictive contexts reduced the emotional intensity required to be accurately classified. More importantly, we show for the first time that participants' accuracy in classifying expressions without an informative context correlated with the magnitude of the contextual effects experienced by them-poor classifiers of isolated expressions were more susceptible to a predictive context. Our findings support the emerging view that contextual cues and individual differences must be considered when explaining mechanisms underlying facial expression classification.

Influence of Recent Trial History on Interval Timing.

Interval timing is involved in a variety of cognitive behaviors such as associative learning and decision-making. While it has been shown that time estimation is adaptive to the temporal context, it remains unclear how interval timing behavior is influenced by recent trial history. Here we found that, in mice trained to perform a licking-based interval timing task, a decrease of inter-reinforcement interval in the previous trial rapidly shifted the time of anticipatory licking earlier. Optogenetic inactivation of the anterior lateral motor cortex (ALM), but not the medial prefrontal cortex, for a short time before reward delivery caused a decrease in the peak time of anticipatory licking in the next trial. Electrophysiological recordings from the ALM showed that the response profiles preceded by short and long inter-reinforcement intervals exhibited task-engagement-dependent temporal scaling. Thus, interval timing is adaptive to recent experience of the temporal interval, and ALM activity during time estimation reflects recent experience of interval.

Perceptual bias contextualized in visually ambiguous stimuli.

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

The taxonomic attribution of African hominin postcrania from the Miocene through the Pleistocene: Associations and assumptions.

Postcranial bones may provide valuable information about fossil taxa relating to their locomotor habits, manipulative abilities and body sizes. Distinctive features of the postcranial skeleton are sometimes noted in species diagnoses. Although numerous isolated postcranial fossils have become accepted by many workers as belonging to a particular species, it is worthwhile revisiting the evidence for each attribution before including them in comparative samples in relation to the descriptions of new fossils, functional analyses in relation to particular taxa, or in evolutionary contexts. Although some workers eschew the taxonomic attribution of postcranial fossils as being less important (or interesting) than interpreting their functional morphology, it is impossible to consider the evolution of functional anatomy in a taxonomic and phylogenetic vacuum. There are 21 widely recognized hominin taxa that have been described from sites in Africa dated from the Late Miocene to the Middle Pleistocene; postcranial elements have been attributed to 17 of these. The bones that have been thus assigned range from many parts of a skeleton to isolated elements. However, the extent to which postcranial material can be reliably attributed to a specific taxon varies considerably from site to site and species to species, and is often the subject of considerable debate. Here, we review the postcranial remains attributed to African hominin taxa from the Late Miocene to the Middle and Late Pleistocene and place these assignations into categories of reliability. The catalog of attributions presented here may serve as a guide for making taxonomic decisions in the future.

A graph neural network framework based on preference-aware graph diffusion for recommendation.

Transforming user check-in data into graph structure data is a popular and powerful way to analyze users' behaviors in the field of recommendation. Graph-based deep learning methods such as graph embeddings and graph neural networks have shown promising performance on the task of point-of-interest recommendation in recent years. Despite effectiveness, existing methods fail to capture deep graph structural information, leading the suboptimal representations. In addition, they lack the ability of learning the influences of both global preference and user preference on the check-in behavior. To address the aforementioned issues, we propose a general framework based on preference-aware graph diffusion, named PGD. We first construct two types of graphs to represent the global preference and user preference. Then, we apply a graph diffusion process to capture the structural information of the generated graphs, resulting in weighted adjacency matrices. Finally, graph neural network-based backbones are introduced to learn the representations of users and POIs on weighted adjacency matrices. A learnable aggregation module is developed to learn the final representations from global preference and user preference adaptively. Extensive experiments on four real-world datasets demonstrate the superiority of PGD on POI recommendation, compared with the mainstream graph-based deep learning methods.

Serial and strategic memory processes in goal-directed selective remembering.

People often rely on habitual, serial processing when presented with to-be-learned information. We tested how strategic processing can override more bottom-up, serial processes when remembering information by having participants study a list of word triads (e.g., "dollar phone pizza"). Participants' goal was manipulated by maximizing either (i) their recall for each of the studied words or (ii) their total score associated with recalling certain words in each triad that were more valuable (worth more points) to engage either serial or strategic processing and retrieval mechanisms. Results revealed that when learners were told to maximize their total recall, they frequently engaged in serial remembering-remembering guided by an item's location within the study phase (i.e., words were retrieved according to a habitual reading bias). However, when words were paired with point values that counted towards participants' scores if recalled, participants were not only selective for high-value words but also attempted to overcome the tendency to engage in serial remembering; instead, they appeared to engage in strategic remembering whereby retrieval is guided by value. Thus, to maximize memory utility, it may be beneficial to override habitual processes and initiate retrieval with high-value words, and when making recall transitions, to recall high-value words together. Importantly, when certain to-be-remembered words were more valuable than their neighbors, participants still demonstrated some serial processing of the to-be-remembered words, indicating that even when engaging in strategic memory, some habitual processes can persist.

SleepContextNet: A temporal context network for automatic sleep staging based single-channel EEG.

BACKGROUND AND OBJECTIVE: Single-channel EEG is the most popular choice of sensing modality in sleep staging studies, because it widely conforms to the sleep staging guidelines. The current deep learning method using single-channel EEG signals for sleep staging mainly extracts the features of its surrounding epochs to obtain the short-term temporal context information of EEG epochs, and ignore the influence of the long-term temporal context information on sleep staging. However, the long-term context information includes sleep stage transition rules in a sleep cycle, which can further improve the performance of sleep staging. The aim of this research is to develop a temporal context network to capture the long-term context between EEG sleep stages. METHODS: In this paper, we design a sleep staging network named SleepContextNet for sleep stage sequence. SleepContextNet can extract and utilize the long-term temporal context between consecutive EEG epochs, and combine it with the short-term context. we utilize Convolutional Neural Network(CNN) layers for learning representative features from each sleep stage and the representation features sequence learned are fed into a Recurrent Neural Network(RNN) layer for learning long-term and short-term context information among sleep stage in chronological order. In addition, we design a data augmentation algorithm for EEG to retain the long-term context information without changing the number of samples. RESULTS: We evaluate the performance of our proposed network using four public datasets, the 2013 version of Sleep-EDF (SEDF), the 2018 version of Sleep-EDF Expanded (SEDFX), Sleep Heart Health Study (SHHS) and the CAP Sleep Database. The experimental results demonstrate that SleepContextNet outperforms state-of-the-art techniques in terms of different evaluation metrics by capturing long-term and short-term temporal context information. On average, accuracy of 84.8% in SEDF, 82.7% in SEDFX, 86.4% in SHHS and 78.8% in CAP are obtained under subject-independent cross validation. CONCLUSIONS: The network extracts the long-term and short-term temporal context information of sleep stages from the sequence features, which utilizes the temporal dependencies among the EEG epochs effectively and improves the accuracy of sleep stages. The sleep staging method based on forward temporal context information is suitable for real-time family sleep monitoring system.

Semi-supervised segmentation of echocardiography videos via noise-resilient spatiotemporal semantic calibration and fusion.

We present a novel model for left ventricle endocardium segmentation from echocardiography video, which is of great significance in clinical practice and yet a challenging task due to (1) the severe speckle noise in echocardiography videos, (2) the irregular motion of pathological heart, and (3) the limited training data caused by high annotation cost. The proposed model has three compelling characteristics. First, we propose a novel adaptive spatiotemporal semantic calibration method to align the feature maps of consecutive frames, where the spatiotemporal correspondences are figured out based on feature maps instead of pixels, thereby mitigating the adverse effects of speckle noise in the calibration. Second, we further learn the importance of each feature map of neighbouring frames to the current frame from the temporal perspective so as to distinctively rather than uniformly harness the temporal information to tackle the irregular and anisotropic motions. Third, we integrate these techniques into the mean teacher semi-supervised architecture to leverage a large amount of unlabeled data to improve the segmentation accuracy. We extensively evaluate the proposed method on two public echocardiography video datasets (EchoNet-Dynamic and CAMUS), where the average dice coefficient on the left ventricular endocardium segmentation achieves 92.87% and 93.79%, respectively. Comparisons with state-of-the-art methods also demonstrate the effectiveness of the proposed method by achieving a better segmentation performance with a faster speed.