The influence of emotion on temporal context models.

Temporal context models (TCMs) have been influential in understanding episodic memory and its neural underpinnings. Recently, TCMs have been extended to explain emotional memory effects, one of the most clinically important findings in the field of memory research. This review covers recent advances in hypotheses for the neural representation of spatiotemporal context through the lens of TCMs, including their ability to explain the influence of emotion on episodic and temporal memory. In recent years, simplifying assumptions of "classical" TCMs - with exponential trace decay and the mechanism by which temporal context is recovered - have become increasingly clear. The review also outlines how recent advances could be incorporated into a future TCM, beyond classical assumptions, to integrate emotional modulation.

Emotional state dynamics impacts temporal memory.

Emotional fluctuations are ubiquitous in everyday life, but precisely how they sculpt the temporal organisation of memories remains unclear. Here, we designed a novel task - the Emotion Boundary Task - wherein participants viewed sequences of negative and neutral images surrounded by a colour border. We manipulated perceptual context (border colour), emotional-picture valence, as well as the direction of emotional-valence shifts (i.e., shifts from neutral-to-negative and negative-to-neutral events) to create events with a shared perceptual and/or emotional context. We measured memory for temporal order and temporal distances for images processed within and across events. Negative images processed within events were remembered as closer in time compared to neutral ones. In contrast, temporal distances were remembered as longer for images spanning neutral-to-negative shifts - suggesting temporal dilation in memory with the onset of a negative event following a previously-neutral state. The extent of negative-picture induced temporal dilation in memory correlated with dispositional negativity across individuals. Lastly, temporal order memory was enhanced for recently-presented negative (versus neutral) images. These findings suggest that emotional-state dynamics matters when considering emotion-temporal memory interactions: While persistent negative events may compress subjectively remembered time, dynamic shifts from neutral-to-negative events produce temporal dilation in memory, with implications for adaptive emotional functioning.

Neural coding of space by time.

The intertwining of space and time poses a significant scientific challenge, transcending disciplines from philosophy and physics to neuroscience. Deciphering neural coding, marked by its inherent spatial and temporal dimensions, has proven to be a complex task. In this paper, we present insights into temporal and spatial modes of neural coding and their intricate interplay, drawn from neuroscientific findings. We illustrate the conversion of a purely spatial input into the temporal form of a singular spike train, demonstrating storage, transmission to remote locations, and recall through spike bursts corresponding to Sharp Wave Ripples. Moreover, the converted temporal representation can be transformed back into a spatiotemporal pattern. The principles of the transformation process are illustrated using a simple feed-forward spiking neural network. The frequencies and phases of Subthreshold Membrane potential Oscillations play a pivotal role in this framework. The model offers insights into information multiplexing and phenomena such as stretching or compressing time of spike patterns.

Event representation at the scale of ordinary experience.

Weeks are divided into weekdays and weekends; years into semesters and seasons; lives into stages like childhood, adulthood, and adolescence. How does the structure of experience shape memory? Though much work has examined event representation in human cognition, little work has explored event representation at the scale of ordinary experience. Here, we use shared experiences - in the form of popular television shows - to explore how memories are shaped by event structure at a large scale. We find that memories for events in these shows exhibit several hallmarks of event cognition. Namely, we find that memories are organized with respect to their event structure (boundaries), and that beginnings and endings are better remembered at multiple levels of the event hierarchy simultaneously. These patterns seem to be partially, but not fully, explained by the perceived story-relevance of events. Lastly, using a longitudinal design, we also show how event representations evolve over periods of several months. These results offer an understanding of event cognition at the scale of ordinary human lives.

Temporal memory for threatening events encoded in a haunted house.

Despite the salient experience of encoding threatening events, these memories are prone to distortions and often non-veridical from encoding to recall. Further, threat has been shown to preferentially disrupt the binding of event details and enhance goal-relevant information. While extensive work has characterised distinctive features of emotional memory, research has not fully explored the influence threat has on temporal memory, a process putatively supported by the binding of event details into a temporal context. Two primary competing hypotheses have been proposed; that threat can impair or enhance temporal memory. We analysed two datasets to assess temporal memory for an in-person haunted house experience. In study 1, we examined the temporal structure of memory by characterising memory contiguity in free recall as a function of individual levels of heart rate as a proxy of threat. In study 2, we replicated marginal findings of threat-related increases in memory contiguity found in study 1. We extended these findings by showing threat-related increases in recency discriminations, an explicit test of temporal memory. Together, these findings demonstrate that threat enhances temporal memory regarding free recall structure and during explicit memory judgments.

A Two-Layer Self-Organizing Map with Vector Symbolic Architecture for Spatiotemporal Sequence Learning and Prediction.

We propose a new nature- and neuro-science-inspired algorithm for spatiotemporal learning and prediction based on sequential recall and vector symbolic architecture. A key novelty is the learning of spatial and temporal patterns as decoupled concepts where the temporal pattern sequences are constructed using the learned spatial patterns as an alphabet of elements. The decoupling, motivated by cognitive neuroscience research, provides the flexibility for fast and adaptive learning with dynamic changes to data and concept drift and as such is better suited for real-time learning and prediction. The algorithm further addresses several key computational requirements for predicting the next occurrences based on real-life spatiotemporal data, which have been found to be challenging with current state-of-the-art algorithms. Firstly, spatial and temporal patterns are detected using unsupervised learning from unlabeled data streams in changing environments; secondly, vector symbolic architecture (VSA) is used to manage variable-length sequences; and thirdly, hyper dimensional (HD) computing-based associative memory is used to facilitate the continuous prediction of the next occurrences in sequential patterns. The algorithm has been empirically evaluated using two benchmark and three time-series datasets to demonstrate its advantages compared to the state-of-the-art in spatiotemporal unsupervised sequence learning where the proposed ST-SOM algorithm is able to achieve 45% error reduction compared to HTM algorithm.

NeoSLAM: Long-Term SLAM Using Computational Models of the Brain.

Simultaneous Localization and Mapping (SLAM) is a fundamental problem in the field of robotics, enabling autonomous robots to navigate and create maps of unknown environments. Nevertheless, the SLAM methods that use cameras face problems in maintaining accurate localization over extended periods across various challenging conditions and scenarios. Following advances in neuroscience, we propose NeoSLAM, a novel long-term visual SLAM, which uses computational models of the brain to deal with this problem. Inspired by the human neocortex, NeoSLAM is based on a hierarchical temporal memory model that has the potential to identify temporal sequences of spatial patterns using sparse distributed representations. Being known to have a high representational capacity and high tolerance to noise, sparse distributed representations have several properties, enabling the development of a novel neuroscience-based loop-closure detector that allows for real-time performance, especially in resource-constrained robotic systems. The proposed method has been thoroughly evaluated in terms of environmental complexity by using a wheeled robot deployed in the field and demonstrated that the accuracy of loop-closure detection was improved compared with the traditional RatSLAM system.

A Star Network of Bipolar Memristive Devices Enables Sensing and Temporal Computing.

Temporal (race) computing schemes rely on temporal memories, where information is represented with the timing of signal edges. Standard digital circuit techniques can be used to capture the relative timing characteristics of signal edges. However, the properties of emerging device technologies could be particularly exploited for more efficient circuit implementations. Specifically, the collective dynamics of networks of memristive devices could be leveraged to facilitate time-domain computations in emerging memristive memories. To this end, this work studies the star interconnect configuration of bipolar memristive devices. Through circuit simulations using a behavioral model of voltage-controlled bipolar memristive devices, we demonstrated the suitability of such circuits in two different contexts, namely sensing and "rank-order" coding. We particularly analyzed the conditions that the employed memristive devices should meet to guarantee the expected operation of the circuit and the possible effects of device variability in the storage and the reproduction of the information in arriving signal edges. The simulation results in LTSpice validate the correct operation and confirm the promising application prospects of such simple circuit structures, which, we show, natively exist in the crossbar geometry. Therefore, the star interconnect configuration could be considered for temporal computations inside resistive memory (ReRAM) arrays.

Are multiple types of associative memory differently impacted by emotion?

The effect of emotion on associative memory is still an open question. Our aim was to test whether discrepant findings are due to differential impact of emotion on different types of associative memory or to differences in the way participants encoded stimuli across studies. We examined the effect of negative content on multiple forms of associative memory, using the same encoding task. Two registered experiments were conducted in parallel with random allocation of participants to experiments. Each experiment included 4 encoding blocks, in which participants read a neutral text comprised of 6 paragraphs, which were interleaved with neutral or negative images. Images were controlled for visual properties and semantic similarity. Memory tests included recognition memory, Remember/Know, order memory, temporal source memory and contextual memory. Analyses showed that emotion decreased contextual memory but not order memory or temporal source memory. We also found that temporal source memory and contextual memory were correlated. Recognition accuracy and subjective recollection were not impacted by emotion. In agreement with previous work, participants self-reported a reduced ability to integrate blocks containing negative images with paragraphs. In contrast to our hypothesis, results suggest that emotion does not impact all types of associative memory when stimuli are controlled.

Collective events and individual affect shape autobiographical memory.

How do collective events shape how we remember our lives? We leveraged advances in natural language processing as well as a rich, longitudinal assessment of 1,000 Americans throughout 2020 to examine how memory is influenced by two prominent factors: surprise and emotion. Autobiographical memory for 2020 displayed a unique signature: There was a substantial bump in March, aligning with pandemic onset and lockdowns, consistent across three memory collections 1 y apart. We further investigated how emotion, using both immediate and retrieved measures, predicted the amount and content of autobiographical memory: Negative affect increased recall across all measures, whereas its more clinical indices, depression and posttraumatic stress disorder, selectively increased nonepisodic recall. Finally, in a separate cohort, we found pandemic news to be better remembered, surprising, and negative, while lockdowns compressed remembered time. Our work connects laboratory findings to the real world and delineates the effects of acute versus clinical signatures of negative emotion on memory.

Information-theoretic analysis of Hierarchical Temporal Memory-Spatial Pooler algorithm with a new upper bound for the standard information bottleneck method.

Hierarchical Temporal Memory (HTM) is an unsupervised algorithm in machine learning. It models several fundamental neocortical computational principles. Spatial Pooler (SP) is one of the main components of the HTM, which continuously encodes streams of binary input from various layers and regions into sparse distributed representations. In this paper, the goal is to evaluate the sparsification in the SP algorithm from the perspective of information theory by the information bottleneck (IB), Cramer-Rao lower bound, and Fisher information matrix. This paper makes two main contributions. First, we introduce a new upper bound for the standard information bottleneck relation, which we refer to as modified-IB in this paper. This measure is used to evaluate the performance of the SP algorithm in different sparsity levels and various amounts of noise. The MNIST, Fashion-MNIST and NYC-Taxi datasets were fed to the SP algorithm separately. The SP algorithm with learning was found to be resistant to noise. Adding up to 40% noise to the input resulted in no discernible change in the output. Using the probabilistic mapping method and Hidden Markov Model, the sparse SP output representation was reconstructed in the input space. In the modified-IB relation, it is numerically calculated that a lower noise level and a higher sparsity level in the SP algorithm lead to a more effective reconstruction and SP with 2% sparsity produces the best results. Our second contribution is to prove mathematically that more sparsity leads to better performance of the SP algorithm. The data distribution was considered the Cauchy distribution, and the Cramer-Rao lower bound was analyzed to estimate SP's output at different sparsity levels.

Multiple memory systems for efficient temporal order memory.

We report distinct contributions of multiple memory systems to the retrieval of the temporal order of events. The neural dynamics related to the retrieval of movie scenes revealed that recalling the temporal order of close events elevates hippocampal theta power, like that observed for recalling close spatial relationships. In contrast, recalling far events increases beta power in the orbitofrontal cortex, reflecting recall based on the overall movie structure.

Cross-Modal Facilitation of Episodic Memory by Sequential Action Execution.

Throughout our lives, the actions we produce are often highly familiar and repetitive (e.g., commuting to work). However, layered upon these routine actions are novel, episodic experiences. Substantial research has shown that prior knowledge can facilitate learning of conceptually related new information. But despite the central role our behavior plays in real-world experience, it remains unclear how engagement in a familiar sequence of actions influences memory for unrelated, nonmotor information coincident with those actions. To investigate this, we had healthy young adults encode novel items while simultaneously following a sequence of actions (key presses) that was either predictable and well-learned or random. Across three experiments (N = 80 each), we found that temporal order memory, but not item memory, was significantly enhanced for novel items encoded while participants executed predictable compared with random action sequences. These results suggest that engaging in familiar behaviors during novel learning scaffolds within-event temporal memory, an essential feature of episodic experiences.

A model of working memory for encoding multiple items and ordered sequences exploiting the theta-gamma code.

Recent experimental evidence suggests that oscillatory activity plays a pivotal role in the maintenance of information in working memory, both in rodents and humans. In particular, cross-frequency coupling between theta and gamma oscillations has been suggested as a core mechanism for multi-item memory. The aim of this work is to present an original neural network model, based on oscillating neural masses, to investigate mechanisms at the basis of working memory in different conditions. We show that this model, with different synapse values, can be used to address different problems, such as the reconstruction of an item from partial information, the maintenance of multiple items simultaneously in memory, without any sequential order, and the reconstruction of an ordered sequence starting from an initial cue. The model consists of four interconnected layers; synapses are trained using Hebbian and anti-Hebbian mechanisms, in order to synchronize features in the same items, and desynchronize features in different items. Simulations show that the trained network is able to desynchronize up to nine items without a fixed order using the gamma rhythm. Moreover, the network can replicate a sequence of items using a gamma rhythm nested inside a theta rhythm. The reduction in some parameters, mainly concerning the strength of GABAergic synapses, induce memory alterations which mimic neurological deficits. Finally, the network, isolated from the external environment ("imagination phase") and stimulated with high uniform noise, can randomly recover sequences previously learned, and link them together by exploiting the similarity among items.

Synapses, predictions, and prediction errors: A neocortical computational study of MDD using the temporal memory algorithm of HTM.

Introduction: Synapses and spines play a significant role in major depressive disorder (MDD) pathophysiology, recently highlighted by the rapid antidepressant effect of ketamine and psilocybin. According to the Bayesian brain and interoception perspectives, MDD is formalized as being stuck in affective states constantly predicting negative energy balance. To understand how spines and synapses relate to the predictive function of the neocortex and thus to symptoms, we used the temporal memory (TM), an unsupervised machine-learning algorithm. TM models a single neocortical layer, learns in real-time, and extracts and predicts temporal sequences. TM exhibits neocortical biological features such as sparse firing and continuous online learning using local Hebbian-learning rules. Methods: We trained a TM model on random sequences of upper-case alphabetical letters, representing sequences of affective states. To model depression, we progressively destroyed synapses in the TM model and examined how that affected the predictive capacity of the network. We found that the number of predictions decreased non-linearly. Results: Destroying 50% of the synapses slightly reduced the number of predictions, followed by a marked drop with further destruction. However, reducing the synapses by 25% distinctly dropped the confidence in the predictions. Therefore, even though the network was making accurate predictions, the network was no longer confident about these predictions. Discussion: These findings explain how interoceptive cortices could be stuck in limited affective states with high prediction error. Connecting ketamine and psilocybin's proposed mechanism of action to depression pathophysiology, the growth of new synapses would allow representing more futuristic predictions with higher confidence. To our knowledge, this is the first study to use the TM model to connect changes happening at synaptic levels to the Bayesian formulation of psychiatric symptomatology. Linking neurobiological abnormalities to symptoms will allow us to understand the mechanisms of treatments and possibly, develop new ones.

IoT and Deep Learning-Based Farmer Safety System.

Farming is a fundamental factor driving economic development in most regions of the world. As in agricultural activity, labor has always been hazardous and can result in injury or even death. This perception encourages farmers to use proper tools, receive training, and work in a safe environment. With the wearable device as an Internet of Things (IoT) subsystem, the device can read sensor data as well as compute and send information. We investigated the validation and simulation dataset to determine whether accidents occurred with farmers by applying the Hierarchical Temporal Memory (HTM) classifier with each dataset input from the quaternion feature that represents 3D rotation. The performance metrics analysis showed a significant 88.00% accuracy, precision of 0.99, recall of 0.04, F_Score of 0.09, average Mean Square Error (MSE) of 5.10, Mean Absolute Error (MAE) of 0.19, and a Root Mean Squared Error (RMSE) of 1.51 for the validation dataset, 54.00% accuracy, precision of 0.97, recall of 0.50, F_Score of 0.66, MSE = 0.06, MAE = 3.24, and = 1.51 for the Farming-Pack motion capture (mocap) dataset. The computational framework with wearable device technology connected to ubiquitous systems, as well as statistical results, demonstrate that our proposed method is feasible and effective in solving the problem's constraints in a time series dataset that is acceptable and usable in a real rural farming environment for optimal solutions.

Spatial and temporal massive memory in humans.

It is well known that humans have a massive memory for pictures and scenes.1,2,3,4 They show an ability to encode thousands of images with only a few seconds of exposure to each. In addition to this massive memory for "what" observers have seen, three experiments reported here show that observers have a "spatial massive memory" (SMM) for "where" stimuli have been seen and a "temporal massive memory" (TMM) for "when" stimuli have been seen. The positions in time and space for at least dozens of items can be reported with good, if not perfect accuracy. Previous work has suggested that there might be good memory for stimulus location,5,6 but there do not seem to have been concerted efforts to measure the extent of this memory. Moreover, in our method, observers are recalling where items were located and not merely recognizing the correct location. This is interesting because massive memory is sometimes thought to be limited to recognition tasks based on sense of familiarity.

The tie that binds: temporal coding and adaptive emotion.

Emotions are temporally dynamic, but the persistence of emotions outside of their appropriate temporal context is detrimental to health and well-being. Yet, precisely how temporal coding and emotional processing interact remains unclear. Recently unveiled temporal context representations in the hippocampus, entorhinal cortex (EC), and prefrontal cortex (PFC) support memory for what happened when. Here, we discuss how these neural temporal representations may interact with densely interconnected amygdala circuitry to shape emotional functioning. We propose a neuroanatomically informed framework suggesting that high-fidelity temporal representations linked to dynamic experiences promote emotion regulation and adaptive emotional memories. Then, we discuss how newly-identified synaptic and molecular features of amygdala-hippocampal projections suggest that intense, amygdala-dependent emotional responses may distort temporal-coding mechanisms. We conclude by identifying key avenues for future research.

Temporal order memory impairments in individuals with moderate-severe traumatic brain injury.

INTRODUCTION: Temporal order memory is a core cognitive function that underlies much of our behavior. The ability to bind together information within and across events, and to reconstruct that sequence of information, critically relies upon the hippocampal relational memory system. Recent work has suggested traumatic brain injury (TBI) may particularly impact hippocampally mediated relational memory. However, it is currently unclear whether such deficits extend to temporal order memory, and whether deficits only arise at large memory loads. The present study assessed temporal order memory in individuals with chronic, moderate-severe TBI across multiple set sizes. METHOD: Individuals with TBI and Neurotypical Comparison participants studied sequences of three to nine objects, one a time. At test, all items were re-presented in pseudorandom order, and participants indicated the temporal position (i.e., first, second, etc.) in which each object had appeared. Critically, we assessed both the frequency and the magnitude of errors (i.e., how far from its studied position was an item remembered). RESULTS: Individuals with TBI were not impaired for the smallest set size, but showed significant impairments at 5+ items. Group differences in the error frequency did not increase further with larger set sizes, but group differences in error magnitude did increase with larger memory loads. Individuals with TBI showed spared performance for the first object of each list (primacy) but were impaired on the last object (recency), though error frequency was better for last compared to middle items. CONCLUSIONS: Our findings demonstrate that TBI results in impaired temporal order memory for lists as small as five items, and that impairments are exacerbated with increasing memory loads. Assessments that test only small set sizes may be insufficient to detect these deficits. Further, these data highlight the importance of additional, sensitive measures in the assessment of cognitive impairments in TBI.

Hierarchical intrinsically motivated agent planning behavior with dreaming in grid environments.

Biologically plausible models of learning may provide a crucial insight for building autonomous intelligent agents capable of performing a wide range of tasks. In this work, we propose a hierarchical model of an agent operating in an unfamiliar environment driven by a reinforcement signal. We use temporal memory to learn sparse distributed representation of state-actions and the basal ganglia model to learn effective action policy on different levels of abstraction. The learned model of the environment is utilized to generate an intrinsic motivation signal, which drives the agent in the absence of the extrinsic signal, and through acting in imagination, which we call dreaming. We demonstrate that the proposed architecture enables an agent to effectively reach goals in grid environments.