Foundations of a temporal RL.

Recent advances in neuroscience and psychology show that the brain has access to timelines of both the past and the future. Spiking across populations of neurons in many regions of the mammalian brain maintains a robust temporal memory, a neural timeline of the recent past. Behavioral results demonstrate that people can estimate an extended temporal model of the future, suggesting that the neural timeline of the past could extend through the present into the future. This paper presents a mathematical framework for learning and expressing relationships between events in continuous time. We assume that the brain has access to a temporal memory in the form of the real Laplace transform of the recent past. Hebbian associations with a diversity of synaptic time scales are formed between the past and the present that record the temporal relationships between events. Knowing the temporal relationships between the past and the present allows one to predict relationships between the present and the future, thus constructing an extended temporal prediction for the future. Both memory for the past and the predicted future are represented as the real Laplace transform, expressed as the firing rate over populations of neurons indexed by different rate constants s. The diversity of synaptic timescales allows for a temporal record over the much larger time scale of trial history. In this framework, temporal credit assignment can be assessed via a Laplace temporal difference. The Laplace temporal difference compares the future that actually follows a stimulus to the future predicted just before the stimulus was observed. This computational framework makes a number of specific neurophysiological predictions and, taken together, could provide the basis for a future iteration of RL that incorporates temporal memory as a fundamental building block.

Modeling brain dynamics and gaze behavior: Starting point bias and drift rate relate to frontal midline theta oscillations.

Frontal midline theta oscillatory dynamics have been implicated as an important neural signature of inhibitory control. However, most proactive cognitive control studies rely on behavioral tasks where individual differences are inferred through button presses. We applied computational modeling to further refine our understanding of theta dynamics in a cued anti-saccade task with gaze-contingent eye tracking. Using a drift diffusion model, increased frontal midline theta power during high-conflict, relative to low-conflict, trials predicted a more conservative style of responding through the starting point (bias). During both high- and low-conflict trials, increases in frontal midline theta also predicted improvements in response efficiency (drift rate). Regression analyses provided support for the importance of the starting point bias, which was associated with frontal midline theta over the course of the task above-and-beyond both drift rate and mean reaction time. Our findings provide a more thorough understanding of proactive gaze control by linking trial-by-trial increases of frontal midline theta to a shift in starting point bias facilitating a more neutral style of responding.

Tree Shrews as an Animal Model for Studying Perceptual Decision-Making Reveal a Critical Role of Stimulus-Independent Processes in Guiding Behavior.

Decision-making is an essential cognitive process by which we interact with the external world. However, attempts to understand the neural mechanisms of decision-making are limited by the current available animal models and the technologies that can be applied to them. Here, we build on the renewed interest in using tree shrews (Tupaia belangeri) in vision research and provide strong support for them as a model for studying visual perceptual decision-making. Tree shrews learned very quickly to perform a two-alternative forced choice contrast discrimination task, and they exhibited differences in response time distributions depending on the reward and punishment structure of the task. Specifically, they made occasional fast guesses when incorrect responses are punished by a constant increase in the interval between trials. This behavior was suppressed when faster incorrect responses were discouraged by longer intertrial intervals. By fitting the behavioral data with two variants of racing diffusion decision models, we found that the between-trial delay affected decision-making by modulating the drift rate of a time accumulator. Our results thus provide support for the existence of an internal process that is independent of the evidence accumulation in decision-making and lay a foundation for future mechanistic studies of perceptual decision-making using tree shrews.

Intraobject and extraobject memory binding across early development.

The ability to bind, or link, different aspects of an experience in memory undergoes protracted development across childhood. Most studies of memory binding development have assessed extraobject binding between an object and some external element such as another object, whereas little work has examined the development of intraobject binding, such as between shape and color features within the same object. In this work, we investigate the development of intra- and extraobject memory binding in five-year-olds, eight-year-olds, and young adults with a memory interference paradigm. Between two experiments, we manipulate whether stimuli are presented as coherent objects (Experiment 1: n5-year-olds = 32, 19 males, 13 females; n8-year-olds = 30, 15 males, 15 females; nadults = 30, 15 males, 15 females), requiring intraobject binding between shape and color features, or as spatially separated features (Experiment 2: n5-year-olds = 24, 16 males, 8 females; n8-year-olds = 41, 19 males, 22 females; nadults = 31, 13 males, 18 females), requiring extraobject binding. To estimate the contributions of different binding structures to performance, we present a novel computational model that mathematically instantiates the memory binding, forgetting, and retrieval processes we hypothesize to underlie performance on the task. The results provide evidence of substantial developmental improvements in both intraobject and extraobject binding of shape and color features between 5 and 8 years of age, as well as stronger intraobject compared with extraobject binding of features in all age groups. These findings provide key insights into memory binding across early development. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Transparency, replicability, and discovery in cognitive aging research: A computational modeling approach.

Healthy aging is associated with deficits in performance on episodic memory tasks. Popular verbal theories of the mechanisms underlying this decrement have primarily focused on inferred changes in associative memory. However, performance on any task is the result of interactions between different neurocognitive mechanisms, such as perceptuomotor, memory, and decision-making processes. As a result, age-related differences in performance could arise from multiple processes, which could lead to incomplete or incorrect conclusions about the sources of aging effects. In addition, standard statistical comparisons of group-level summary statistics, such as mean accuracy, may not provide sufficient information to allow detailed mechanistic explanations of age-related change. We argue that these and other drawbacks of relying exclusively on verbal theories can hamper replicability, transparency, and scientific progress in aging research and psychological science more generally, and that computational modeling is a tool that can address many of these limitations. Computational models make mathematically transparent claims about how latent processes give rise to observed behavior and decompose an individual's performance into model parameters governing hypothesized mechanisms. In this work, we present a short memory task designed for and analyzed with mechanistic model-based approaches. We provide an example of a computational model and fit the model to data from young and older adults with hierarchical Bayesian techniques in order to (a) detect differences in latent cognitive processes between young and older adults (as well as individual participants), (b) quantitatively compare models to assess different processes that could underlie performance, and (c) simulate data to make predictions for future experiments based on model mechanisms. We argue that computational modeling is a powerful tool to examine age differences in latent processes, make theories more transparent, and facilitate discovery in cognitive aging research. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Equal evidence perceptual tasks suggest a key role for interactive competition in decision-making.

The dynamics of decision-making have been widely studied over the past several decades through the lens of an overarching theory called sequential sampling theory (SST). Within SST, choices are represented as accumulators, each of which races toward a decision boundary by drawing stochastic samples of evidence through time. Although progress has been made in understanding how decisions are made within the SST framework, considerable debate centers on whether the accumulators exhibit dependency during the evidence accumulation process; namely, whether accumulators are independent, fully dependent, or partially dependent. To evaluate which type of dependency is the most plausible representation of human decision-making, we applied a novel twist on two classic perceptual tasks; namely, in addition to the classic paradigm (i.e., the unequal-evidence conditions), we used stimuli that provided different magnitudes of equal-evidence (i.e., the equal-evidence conditions). In equal-evidence conditions, response times systematically decreased with increase in the magnitude of evidence, whereas in unequal-evidence conditions, response times systematically increased as the difference in evidence between the two alternatives decreased. We designed a spectrum of models that ranged from independent accumulation to fully dependent accumulation, while also examining the effects of within-trial and between-trial variability (BTV). We then fit the set of models to our two experiments and found that models instantiating the principles of partial dependency provided the best fit to the data. Our results further suggest that mechanisms inducing partial dependency, such as lateral inhibition, are beneficial for understanding complex decision-making dynamics, even when the task is relatively simple. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Quantifying mechanisms of cognition with an experiment and modeling ecosystem.

Although there have been major strides toward uncovering the neurobehavioral mechanisms involved in cognitive functions like memory and decision making, methods for measuring behavior and accessing latent processes through computational means remain limited. To this end, we have created SUPREME (Sensing to Understanding and Prediction Realized via an Experiment and Modeling Ecosystem): a toolbox for comprehensive cognitive assessment, provided by a combination of construct-targeted tasks and corresponding computational models. SUPREME includes four tasks, each developed symbiotically with a mechanistic model, which together provide quantified assessments of perception, cognitive control, declarative memory, reward valuation, and frustrative nonreward. In this study, we provide validation analyses for each task using two sessions of data from a cohort of cognitively normal participants (N = 65). Measures of test-retest reliability (r: 0.58-0.75), stability of individual differences (ρ: 0.56-0.70), and internal consistency (α: 0.80-0.86) support the validity of our tasks. After fitting the models to data from individual subjects, we demonstrate each model's ability to capture observed patterns of behavioral results across task conditions. Our computational approaches allow us to decompose behavior into cognitively interpretable subprocesses, which we can compare both within and between participants. We discuss potential future applications of SUPREME, including clinical assessments, longitudinal tracking of cognitive functions, and insight into compensatory mechanisms.

Individual differences in attention allocation during a two-dimensional inhibitory control task.

Visual attention is often through to take the form of a spotlight or zoom lens that gradually focuses on goal-relevant features of a stimulus over the course of a trial. Several lines of evidence suggest that for spatially contiguous stimuli, the spotlight naturally takes on the shape of a horizontally biased ellipse. Analyses of group-level behavior in the presence of horizontally versus vertically configured stimuli, however, potentially obfuscate an important source of between-subject variability in the early stages of attentional processing. In the current study, we used a two-dimensional flanker task paradigm and nested variants of a model of within-trial attention and decision mechanisms to investigate individual differences in spotlight shapes. To account for the influence of distractor stimuli in both horizontal and vertical positions relative to the target, we operationalized the attentional spotlight as the density function for a bivariate normal distribution within our models. Horizontal and vertical shape parameters governing the spotlight were constrained to be equal in one model variant, and were allowed to vary in the other. Within-subject comparisons of Bayesian goodness-of-fit statistics revealed a general preference for an elliptical rather than a circular spotlight. Follow-up analyses, however, demonstrated substantial variability in spotlight shapes across subjects. Although data from most subjects were best captured by a horizontally biased elliptical spotlight, we observed individual differences in the extent of the bias, with some subjects even demonstrating a circular or vertically biased elliptical spotlight.

A model of dynamic, within-trial conflict resolution for decision making.

Growing evidence for moment-to-moment fluctuations in visual attention has led to questions about the impetus and time course of cognitive control. These questions are typically investigated with paradigms like the flanker task, which require participants to inhibit an automatic response before making a decision. Connectionist modeling work suggests that between-trial changes in attention result from fluctuations in conflict-as conflict occurs, attention needs to be upregulated to resolve it. Current sequential sampling models (SSMs) of within-trial effects, however, suggest that attention focuses on a goal-relevant target as a function of time. We propose that within-trial changes in cognitive control and attention are emergent properties of the dynamics of the decision itself. We tested our hypothesis by developing a set of SSMs, each making alternative assumptions about attention modulation and evidence accumulation mechanisms. Combining the SSM framework with likelihood-free Bayesian approximation methods allowed us to conduct quantified comparisons between subject-level fits. Models included either time- or control-based attention mechanisms, and either strongly- (via feedforward inhibition) or weakly correlated (via leak and lateral inhibition) evidence accumulation mechanisms. We fit all models to behavioral data collected in variants of the flanker task, one accompanied by EEG measures. Across three experiments, we found converging evidence that control-based attention processes in combination with evidence accumulation mechanisms governed by leak and lateral inhibition provided the best fits to behavioral data, and uniquely mapped onto observed decision-related signals in the brain. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Cognitive Task Performance During Titration Predicts Deep Brain Stimulation Treatment Efficacy: Evidence From a Case Study.

Device titration is a major challenge when using deep brain stimulation (DBS) to treat behavioral disorders. Unlike in movement disorders, there is no reliable real-time clinical feedback for changes in complex behaviors resulting from DBS. Here, a female patient receiving DBS of the nucleus accumbens for the treatment of morbid obesity underwent cognitive testing via the flanker task alongside traditional methods of device titration. One set of stimulation parameters administered during titration resulted in acute cognitive improvement (p = 0.033) and increased frontal engagement as measured by electroencephalography (left anterior: p = 0.007, right anterior: p = 0.005) relative to DBS-OFF. The same parameters resulted in the most weight-loss during long-term continuous stimulation (47.8 lbs lost in 129 days) compared to the results of other stimulation settings. Diffusion tensor imaging analyses showed increased connectivity to dorsal attention networks and decreased connectivity to the default mode network for optimal parameters (p < 0.01). Our results provide evidence that targeted cognitive testing is a potentially useful tool for capturing acute effects of DBS stimulation during titration and predicting long-term treatment outcomes. CLINICAL TRIAL REGISTRATION: www.ClinicalTrials.gov, identifier: NCT01512134.

A context-change account of temporal distinctiveness.

The distinctiveness effect refers to the finding that items that stand out from other items in a learning set are more likely to be remembered later. Traditionally, distinctiveness has been defined based on item features; specifically, an item is deemed to be distinctive if its features are different from the features of other to-be-learned items. We propose that distinctiveness can be redefined based on context change-distinctive items are those with features that deviate from the others in the current temporal context, a recency-weighted running average of experience-and that this context change modulates learning. We test this account with two novel experiments and introduce a formal mathematical model that instantiates our proposed theory. In the experiments, participants studied lists of words, with each word appearing on one of two background colors. Within each list, each color was used for 50% of the words, but the sequence of the colors was controlled so that runs of the same color for that list were common in Experiment 1 and common, rare, or random in Experiment 2. In both experiments, participants' source memory for background color was enhanced for items where the color changed, especially if the change occurred after a stable run without color changes. Conversely, source memory was not significantly better for nonchanges after runs of alternating colors with each item. This pattern is inconsistent with theories of learning based on prediction error, but is consistent with our context-change account.

Estimating Scale-Invariant Future in Continuous Time.

Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. An important drawback of model-free algorithms is the need to select a timescale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically compressed scale and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation that generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.

The experience of vivid autobiographical reminiscence is supported by subjective content representations in the precuneus.

The human posteromedial cortex, which includes core regions of the default mode network (DMN), is thought to play an important role in episodic memory. However, the nature and functional role of representations in these brain regions remain unspecified. Nine participants (all female) wore smartphone devices to record episodes from their daily lives for multiple weeks, each night indicating the personally-salient attributes of each episode. Participants then relived their experiences in an fMRI scanner cued by images from their own lives. Representational Similarity Analysis revealed a broad network, including parts of the DMN, that represented personal semantics during autobiographical reminiscence. Within this network, activity in the right precuneus reflected more detailed representations of subjective contents during vivid relative to non-vivid, recollection. Our results suggest a more specific mechanism underlying the phenomenology of vivid autobiographical reminiscence, supported by rich subjective content representations in the precuneus, a hub of the DMN previously implicated in metacognitive evaluations during memory retrieval.

Meeting brain-computer interface user performance expectations using a deep neural network decoding framework.

Brain-computer interface (BCI) neurotechnology has the potential to reduce disability associated with paralysis by translating neural activity into control of assistive devices1-9. Surveys of potential end-users have identified key BCI system features10-14, including high accuracy, minimal daily setup, rapid response times, and multifunctionality. These performance characteristics are primarily influenced by the BCI's neural decoding algorithm1,15, which is trained to associate neural activation patterns with intended user actions. Here, we introduce a new deep neural network16 decoding framework for BCI systems enabling discrete movements that addresses these four key performance characteristics. Using intracortical data from a participant with tetraplegia, we provide offline results demonstrating that our decoder is highly accurate, sustains this performance beyond a year without explicit daily retraining by combining it with an unsupervised updating procedure3,17-20, responds faster than competing methods8, and can increase functionality with minimal retraining by using a technique known as transfer learning21. We then show that our participant can use the decoder in real-time to reanimate his paralyzed forearm with functional electrical stimulation (FES), enabling accurate manipulation of three objects from the grasp and release test (GRT)22. These results demonstrate that deep neural network decoders can advance the clinical translation of BCI technology.

Representational differences between line drawings and photographs of natural scenes: A dissociation between multi-voxel pattern analysis and repetition suppression.

Distributed representations of scene categories are consistent between color photographs (CPs) and line drawings (LDs) in the parahippocampal place area (PPA) and the retrosplenial cortex (RSC), as shown using multi-voxel pattern analysis (MVPA). Here, we used repetition suppression (RS) to further investigate the degree of representational convergence between CPs and LDs of natural scenes. MVPA and RS can capture different aspects of visual representations, and RS may prove useful in elucidating important differences in the representations of CPs and LDs of natural scenes. We performed an event-related fMRI experiment, including image-repetitions either within-type (i.e., CP to CP or LD to LD) or between-types (CP to LD, LD to CP). We found significant RS for within-type repetitions in PPA, RSC and the occipital place area (OPA), but did not observe RS for between-types repetitions. By contrast, scene categories were decodable from activity patterns evoked by both CPs and LDs using SVM classification for both within-type decoding and between-types cross-decoding. We conclude that there are representational differences between CPs and LDs in scene-selective cortex despite a category-level correspondence.

Some task demands induce collapsing bounds: Evidence from a behavioral analysis.

Traditional models of choice-response time assume that sensory evidence accumulates for choice alternatives until a threshold amount of evidence has been obtained. Although some researchers have characterized the threshold as varying randomly from trial to trial, these investigations have all assumed that the threshold remains fixed across time within a trial. Despite decades of successful applications of these models to a variety of experimental manipulations, the time-invariance assumption has recently been called into question, and a time-variant alternative implementing collapsing decision thresholds has been proposed instead. Here, we investigated the fidelity of the collapsing threshold assumption by assessing relative model fit to data from a highly constrained experimental design that coupled a within-subject mixture of two classic response time paradigms-interrogation and free response-within a random dot motion (RDM) task. Overall, we identified strong evidence in favor of collapsing decision thresholds, suggesting that subjects may adopt a dynamic decision policy due to task characteristics, specifically to account for the mixture of response time paradigms and motion strengths across trials in the mixed response signal task. We conclude that time-variant mechanisms may serve as a viable explanation for the strategy used by human subjects in our task.

Deep Brain Stimulation of Frontal Lobe Networks to Treat Alzheimer's Disease.

The study objective was to evaluate the safety and efficacy of deep brain stimulation (DBS) at the ventral capsule/ventral striatum (VC/VS) region to specifically modulate frontal lobe behavioral and cognitive networks as a novel treatment approach for Alzheimer's disease (AD) patients. This is a non-randomized phase I prospective open label interventional trial of three subjects with matched comparison groups. AD participants given DBS for at least 18 months at the VC/VS target were compared on the Clinical Dementia Rating-Sum of Boxes (CDR-SB), our primary outcome clinical measure, to matched groups without DBS from the AD Neuroimaging Initiative (ADNI) cohort. Serial 2-Deoxy-2-[18F]fluoro-D-glucose (FDG) positron emission tomography (PET) images of AD participants were also compared longitudinally over time. Three AD DBS participants were matched to subjects from the ADNI cohort. All participants tolerated DBS well without significant adverse events. All three AD DBS participants had less performance decline and two of them meaningfully less decline over time on our primary outcome measure, CDR-SB, relative to matched comparison groups from the ADNI using score trajectory slopes. Minimal changes or increased metabolism on FDG-PET were seen in frontal cortical regions after chronic DBS at the VC/VS target. The first use of DBS in AD at a frontal lobe behavior regulation target (VC/VS) was well-tolerated and revealed less performance decline in CDR-SB. Frontal network modulation to improve executive and behavioral deficits should be furthered studied in AD.

MELD: Mixed effects for large datasets.

Mixed effects models provide significant advantages in sensitivity and flexibility over typical statistical approaches to neural data analysis, but mass univariate application of mixed effects models to large neural datasets is computationally intensive. Threshold free cluster enhancement also provides a significant increase in sensitivity, but requires computationally-intensive permutation-based significance testing. Not surprisingly, the combination of mixed effects models with threshold free cluster enhancement and nonparametric permutation-based significance testing is currently completely impractical. With mixed effects for large datasets (MELD) we circumvent this impasse by means of a singular value decomposition to reduce the dimensionality of neural data while maximizing signal. Singular value decompositions become unstable when there are large numbers of noise features, so we precede it with a bootstrap-based feature selection step employing threshold free cluster enhancement to identify stable features across subjects. By projecting the dependent data into the reduced space of the singular value decomposition we gain the power of a multivariate approach and we can greatly reduce the number of mixed effects models that need to be run, making it feasible to use permutation testing to determine feature level significance. Due to these innovations, MELD is much faster than an element-wise mixed effects analysis, and on simulated data MELD was more sensitive than standard techniques, such as element-wise t-tests combined with threshold-free cluster enhancement. When evaluated on an EEG dataset, MELD identified more significant features than the t-tests with threshold free cluster enhancement in a comparable amount of time.

A single trial analysis of EEG in recognition memory: Tracking the neural correlates of memory strength.

Recent work in perceptual decision-making has shown that although two distinct neural components differentiate experimental conditions (e.g., did you see a face or a car), only one tracked the evidence guiding the decision process. In the memory literature, there is a distinction between a fronto-central evoked potential measured with EEG beginning at 350ms that seems to track familiarity and a late parietal evoked potential that peaks at 600ms that tracks recollection. Here, we applied single-trial regressor analysis (similar to multivariate pattern analysis, MVPA) and diffusion decision modeling to EEG and behavioral data from two recognition memory experiments to test whether these two components contribute to the recognition decision process. The regressor analysis only involved whether an item was studied or not and did not involve any use of the behavioral data. Only late EEG activity distinguishes studied from not studied items that peaks at about 600ms following each test item onset predicted the diffusion model drift rate derived from the behavioral choice and reaction times (but only for studied items). When drift rate was made a linear function of the trial-level regressor values, the estimate for studied items was different than zero. This showed that the later EEG activity indexed the trial-to-trial variability in drift rate for studied items. Our results provide strong evidence that only a single EEG component reflects evidence being used in the recegnition decision process.

Restoring cortical control of functional movement in a human with quadriplegia.

Millions of people worldwide suffer from diseases that lead to paralysis through disruption of signal pathways between the brain and the muscles. Neuroprosthetic devices are designed to restore lost function and could be used to form an electronic 'neural bypass' to circumvent disconnected pathways in the nervous system. It has previously been shown that intracortically recorded signals can be decoded to extract information related to motion, allowing non-human primates and paralysed humans to control computers and robotic arms through imagined movements. In non-human primates, these types of signal have also been used to drive activation of chemically paralysed arm muscles. Here we show that intracortically recorded signals can be linked in real-time to muscle activation to restore movement in a paralysed human. We used a chronically implanted intracortical microelectrode array to record multiunit activity from the motor cortex in a study participant with quadriplegia from cervical spinal cord injury. We applied machine-learning algorithms to decode the neuronal activity and control activation of the participant's forearm muscles through a custom-built high-resolution neuromuscular electrical stimulation system. The system provided isolated finger movements and the participant achieved continuous cortical control of six different wrist and hand motions. Furthermore, he was able to use the system to complete functional tasks relevant to daily living. Clinical assessment showed that, when using the system, his motor impairment improved from the fifth to the sixth cervical (C5-C6) to the seventh cervical to first thoracic (C7-T1) level unilaterally, conferring on him the critical abilities to grasp, manipulate, and release objects. This is the first demonstration to our knowledge of successful control of muscle activation using intracortically recorded signals in a paralysed human. These results have significant implications in advancing neuroprosthetic technology for people worldwide living with the effects of paralysis.