RESUMO
Some neural representations gradually change across multiple timescales. Here we argue that modeling this "drift" could help explain the spacing effect (the long-term benefit of distributed learning), whereby differences between stored and current temporal context activity patterns produce greater error-driven learning. We trained a neurobiologically realistic model of the entorhinal cortex and hippocampus to learn paired associates alongside temporal context vectors that drifted between learning episodes and/or before final retention intervals. In line with spacing effects, greater drift led to better model recall after longer retention intervals. Dissecting model mechanisms revealed that greater drift increased error-driven learning, strengthened weights in slower drifting temporal context neurons (temporal abstraction), and improved direct cue-target associations (decontextualization). Intriguingly, these results suggest that decontextualization-generally ascribed only to the neocortex-can occur within the hippocampus itself. Altogether, our findings provide a mechanistic formalization for established learning concepts such as spacing effects and errors during learning. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
RESUMO
A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience. This has often been framed in terms of a dichotomy between connectionist and symbolic cognitive models. Here, we highlight a recently emerging line of work that suggests a novel reconciliation of these approaches, by exploiting an inductive bias that we term the relational bottleneck. In that approach, neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner, emphasizing their potential as candidate models for the acquisition of abstract concepts in the human mind and brain.
RESUMO
While many theories assume that sleep is critical in stabilizing and strengthening memories, our recent behavioral study (Liu & Ranganath, 2021, Psychonomic Bulletin & Review, 28[6], 2035-2044) suggests that sleep does not simply stabilize memories. Instead, it plays a more complex role, integrating information across two temporally distinct learning episodes. In the current study, we simulated the results of Liu and Ranganath (2021) using our biologically plausible computational model, TEACH, developed based on the complementary learning systems (CLS) framework. Our model suggests that when memories are activated during sleep, the reduced influence of temporal context establishes connections across temporally separated events through mutual training between the hippocampus and neocortex. In addition to providing a compelling mechanistic explanation for the selective effect of sleep, this model offers new examples of the diverse ways in which the cortex and hippocampus can interact during learning.
RESUMO
BACKGROUND AND HYPOTHESIS: The neuronal mechanisms that underlie deficits in effort cost computation in schizophrenia (SZ) are poorly understood. Given the role of frontostriatal circuits in valence-oriented motivation, we hypothesized that these circuits are either dysfunctional in SZ or do not appropriately predict behavior in SZ when task conditions are difficult and good performance is rewarded. STUDY DESIGN: A total of 52 people with recent onset SZ-spectrum disorders and 48 healthy controls (HCs) performed a 3T fMRI task with 2 valence conditions (rewarded vs neutral) and 2 difficulty conditions. Frontostriatal connectivity was extracted during the cue (anticipatory) phase. Individual behavior was fit using a drift-diffusion model, allowing the performance parameter, drift rate (DR), to vary between task conditions. Three models were examined: A group × condition model of DR, a group × condition model of connectivity, and a regression model of connectivity predicting DR depending on group and condition. STUDY RESULTS: DRs showed the expected positive correlation with accuracy and a negative association with reaction time. The SZ group showed a deficit in DR but did not differ in overall connectivity or show a valence-specific deficit in connectivity. Significant group × valence × difficulty interactions, however, were observed on the relationship between right dorsolateral prefrontal (DLPFC)-striatal connectivity and DR (DLPFC-Caudate: Fâ =â 10.92, PFDRâ =â .004; DLPFC-Putamen: Fâ =â 5.14, PFDRâ =â .048) driven by more positive relationships between DR and connectivity during cues for the difficult-rewarded condition in HCs compared to SZ. CONCLUSIONS: These findings suggest that frontostriatal connectivity is less predictive of performance in SZ when task difficulty is increased and a reward incentive is applied.
Assuntos
Esquizofrenia , Humanos , Corpo Estriado/diagnóstico por imagem , Putamen , Imageamento por Ressonância Magnética , Recompensa , Vias Neurais/diagnóstico por imagem , Córtex Pré-Frontal/diagnóstico por imagemRESUMO
The hippocampus plays a critical role in the rapid learning of new episodic memories. Many computational models propose that the hippocampus is an autoassociator that relies on Hebbian learning (i.e., "cells that fire together, wire together"). However, Hebbian learning is computationally suboptimal as it does not learn in a way that is driven toward, and limited by, the objective of achieving effective retrieval. Thus, Hebbian learning results in more interference and a lower overall capacity. Our previous computational models have utilized a powerful, biologically plausible form of error-driven learning in hippocampal CA1 and entorhinal cortex (EC) (functioning as a sparse autoencoder) by contrasting local activity states at different phases in the theta cycle. Based on specific neural data and a recent abstract computational model, we propose a new model called Theremin (Total Hippocampal ERror MINimization) that extends error-driven learning to area CA3-the mnemonic heart of the hippocampal system. In the model, CA3 responds to the EC monosynaptic input prior to the EC disynaptic input through dentate gyrus (DG), giving rise to a temporal difference between these two activation states, which drives error-driven learning in the ECâCA3 and CA3âCA3 projections. In effect, DG serves as a teacher to CA3, correcting its patterns into more pattern-separated ones, thereby reducing interference. Results showed that Theremin, compared with our original Hebbian-based model, has significantly increased capacity and learning speed. The model makes several novel predictions that can be tested in future studies.
Assuntos
Hipocampo , Modelos Neurológicos , Hipocampo/fisiologia , Córtex Entorrinal/fisiologia , Memória/fisiologia , Aprendizagem/fisiologia , Giro Denteado/fisiologiaRESUMO
A hallmark of human intelligence is the ability to adapt to new situations, by applying learned rules to new content (systematicity) and thereby enabling an open-ended number of inferences and actions (generativity). Here, we propose that the human brain accomplishes these feats through pathways in the parietal cortex that encode the abstract structure of space, events, and tasks, and pathways in the temporal cortex that encode information about specific people, places, and things (content). Recent neural network models show how the separation of structure and content might emerge through a combination of architectural biases and learning, and these networks show dramatic improvements in the ability to capture systematic, generative behavior. We close by considering how the hippocampal formation may form integrative memories that enable rapid learning of new structure and content representations.
RESUMO
Neural networks struggle in continual learning settings from catastrophic forgetting: when trials are blocked, new learning can overwrite the learning from previous blocks. Humans learn effectively in these settings, in some cases even showing an advantage of blocking, suggesting the brain contains mechanisms to overcome this problem. Here, we build on previous work and show that neural networks equipped with a mechanism for cognitive control do not exhibit catastrophic forgetting when trials are blocked. We further show an advantage of blocking over interleaving when there is a bias for active maintenance in the control signal, implying a tradeoff between maintenance and the strength of control. Analyses of map-like representations learned by the networks provided additional insights into these mechanisms. Our work highlights the potential of cognitive control to aid continual learning in neural networks, and offers an explanation for the advantage of blocking that has been observed in humans.
RESUMO
The neural mechanisms supporting flexible relational inferences, especially in novel situations, are a major focus of current research. In the complementary learning systems framework, pattern separation in the hippocampus allows rapid learning in novel environments, while slower learning in neocortex accumulates small weight changes to extract systematic structure from well-learned environments. In this work, we adapt this framework to a task from a recent fMRI experiment where novel transitive inferences must be made according to implicit relational structure. We show that computational models capturing the basic cognitive properties of these two systems can explain relational transitive inferences in both familiar and novel environments, and reproduce key phenomena observed in the fMRI experiment.
RESUMO
How do humans learn from raw sensory experience? Throughout life, but most obviously in infancy, we learn without explicit instruction. We propose a detailed biological mechanism for the widely embraced idea that learning is driven by the differences between predictions and actual outcomes (i.e., predictive error-driven learning). Specifically, numerous weak projections into the pulvinar nucleus of the thalamus generate top-down predictions, and sparse driver inputs from lower areas supply the actual outcome, originating in Layer 5 intrinsic bursting neurons. Thus, the outcome representation is only briefly activated, roughly every 100 msec (i.e., 10 Hz, alpha), resulting in a temporal difference error signal, which drives local synaptic changes throughout the neocortex. This results in a biologically plausible form of error backpropagation learning. We implemented these mechanisms in a large-scale model of the visual system and found that the simulated inferotemporal pathway learns to systematically categorize 3-D objects according to invariant shape properties, based solely on predictive learning from raw visual inputs. These categories match human judgments on the same stimuli and are consistent with neural representations in inferotemporal cortex in primates.
Assuntos
Neocórtex , Pulvinar , Córtex Visual , Animais , NeurôniosRESUMO
Compared to our understanding of positive prediction error signals occurring due to unexpected reward outcomes, less is known about the neural circuitry in humans that drives negative prediction errors during omission of expected rewards. While classical learning theories such as Rescorla-Wagner or temporal difference learning suggest that both types of prediction errors result from a simple subtraction, there has been recent evidence suggesting that different brain regions provide input to dopamine neurons which contributes to specific components of this prediction error computation. Here, we focus on the brain regions responding to negative prediction error signals, which has been well-established in animal studies to involve a distinct pathway through the lateral habenula. We examine the activity of this pathway in humans, using a conditioned inhibition paradigm with high-resolution functional MRI. First, participants learned to associate a sensory stimulus with reward delivery. Then, reward delivery was omitted whenever this stimulus was presented simultaneously with a different sensory stimulus, the conditioned inhibitor (CI). Both reward presentation and the reward-predictive cue activated midbrain dopamine regions, insula and orbitofrontal cortex. While we found significant activity at an uncorrected threshold for the CI in the habenula, consistent with our predictions, it did not survive correction for multiple comparisons and awaits further replication. Additionally, the pallidum and putamen regions of the basal ganglia showed modulations of activity for the inhibitor that did not survive the corrected threshold.
RESUMO
We present a theory and neural network model of the neural mechanisms underlying human decision-making. We propose a detailed model of the interaction between brain regions, under a proposer-predictor-actor-criticâ âframework. This theory is based on detailed animal data and theories of action-selection. Those theories are adapted to serial operation to bridge levels of analysis and explain human decision-making. Task-relevant areas of cortex propose a candidate plan using fast, model-free, parallel neural computations. Other areas of cortex and medial temporal lobe can then predictâ likely outcomes of that plan in this situation. This optional prediction- (or model-) based computation can produce better accuracy and generalization, at the expense of speed. Next, linked regions of basal ganglia actâ to accept or reject the proposed plan based on its reward history in similar contexts. If that plan is rejected, the process repeats to consider a new option. The reward-prediction system acts as a criticâ to determine the value of the outcome relative to expectations and produce dopamine as a training signal for cortex and basal ganglia. By operating sequentially and hierarchically, the same mechanisms previously proposed for animal action-selection could explain the most complex human plans and decisions. We discuss explanations of model-based decisions, habitization, and risky behavior based on the computational model.
Assuntos
Gânglios da Base , Tomada de Decisões , Animais , Córtex Cerebral , Humanos , Redes Neurais de Computação , RecompensaRESUMO
Motor dysfunction in youth at clinical high risk (CHR) for psychosis is thought to reflect abnormal neurodevelopment within cortical-subcortical motor circuits and may be important for understanding clinical trajectories of CHR individuals. However, to date, our perspective of brain-behavior relationships has been informed solely by cross-sectional correlational studies linking behavior in the lab to brain structure or respective resting-state network connectivity. Here, we assess movement dysfunction from 2 perspectives: study 1 investigates the longitudinal progression of handwriting variability and positive symptoms in a sample of 91 CHR and healthy controls during a 12-month follow-up and study 2 involves a multiband functional magnetic resonance imaging task exploring the relationship between power grip force stability and motor network brain activation in a subset of participants. In study 1, we found that greater handwriting variability was a stable feature of CHR participants who experienced worse symptom progression. Study 2 results showed that CHR individuals had greater variability in their grip force and greater variability was related to decreased activation in the associative cortico-striatal network compared to controls. Motor variability may be a stable marker of vulnerability for psychosis risk and possible indicator of a vulnerable cortico-striatal brain network functioning in CHR participants, although the effects of antipsychotic medication should be considered.
Assuntos
Córtex Cerebral/fisiopatologia , Corpo Estriado/fisiopatologia , Progressão da Doença , Força da Mão/fisiologia , Escrita Manual , Atividade Motora/fisiologia , Destreza Motora/fisiologia , Rede Nervosa/fisiopatologia , Transtornos Psicóticos/fisiopatologia , Adolescente , Adulto , Córtex Cerebral/diagnóstico por imagem , Criança , Corpo Estriado/diagnóstico por imagem , Suscetibilidade a Doenças , Feminino , Neuroimagem Funcional , Humanos , Estudos Longitudinais , Imageamento por Ressonância Magnética , Masculino , Rede Nervosa/diagnóstico por imagem , Transtornos Psicóticos/diagnóstico por imagem , Risco , Adulto JovemRESUMO
We describe a neurobiologically informed computational model of phasic dopamine signaling to account for a wide range of findings, including many considered inconsistent with the simple reward prediction error (RPE) formalism. The central feature of this PVLV framework is a distinction between a primary value (PV) system for anticipating primary rewards (Unconditioned Stimuli [USs]), and a learned value (LV) system for learning about stimuli associated with such rewards (CSs). The LV system represents the amygdala, which drives phasic bursting in midbrain dopamine areas, while the PV system represents the ventral striatum, which drives shunting inhibition of dopamine for expected USs (via direct inhibitory projections) and phasic pausing for expected USs (via the lateral habenula). Our model accounts for data supporting the separability of these systems, including individual differences in CS-based (sign-tracking) versus US-based learning (goal-tracking). Both systems use competing opponent-processing pathways representing evidence for and against specific USs, which can explain data dissociating the processes involved in acquisition versus extinction conditioning. Further, opponent processing proved critical in accounting for the full range of conditioned inhibition phenomena, and the closely related paradigm of second-order conditioning. Finally, we show how additional separable pathways representing aversive USs, largely mirroring those for appetitive USs, also have important differences from the positive valence case, allowing the model to account for several important phenomena in aversive conditioning. Overall, accounting for all of these phenomena strongly constrains the model, thus providing a well-validated framework for understanding phasic dopamine signaling. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Assuntos
Dopamina , Modelos Neurológicos , Recompensa , Tonsila do Cerebelo/fisiologia , Condicionamento Clássico , Condicionamento Psicológico , Humanos , AprendizagemRESUMO
Motivation plays a central role in human behavior and cognition but is not well captured by widely used artificial intelligence (AI) and computational modeling frameworks. This Opinion article addresses two central questions regarding the nature of motivation: what are the nature and dynamics of the internal goals that drive our motivational system and how can this system be sufficiently flexible to support our ability to rapidly adapt to novel situations, tasks, etc.? In reviewing existing systems and neuroscience research and theorizing on these questions, a wealth of insights to constrain the development of computational models of motivation can be found.
Assuntos
Inteligência Artificial , Motivação , Cognição , Humanos , Córtex Pré-FrontalRESUMO
We address the distinction between habitual/automatic vs. goal-directed/controlled behavior, from the perspective of a computational model of the frontostriatal loops. The model exhibits a continuum of behavior between these poles, as a function of the interactive dynamics among different functionally-specialized brain areas, operating iteratively over multiple sequential steps, and having multiple nested loops of similar decision making circuits. This framework blurs the lines between these traditional distinctions in many ways. For example, although habitual actions have traditionally been considered purely automatic, the outer loop must first decide to allow such habitual actions to proceed. Furthermore, because the part of the brain that generates proposed action plans is common across habitual and controlled/goal-directed behavior, the key differences are instead in how many iterations of sequential decision-making are taken, and to what extent various forms of predictive (model-based) processes are engaged. At the core of every iterative step in our model, the basal ganglia provides a "model-free" dopamine-trained Go/NoGo evaluation of the entire distributed plan/goal/evaluation/prediction state. This evaluation serves as the fulcrum of serializing otherwise parallel neural processing. Goal-based inputs to the nominally model-free basal ganglia system are among several ways in which the popular model-based vs. model-free framework may not capture the most behaviorally and neurally relevant distinctions in this area.
RESUMO
Computational models of frontal function have made important contributions to understanding how the frontal lobes support a wide range of important functions, in their interactions with other brain areas including, critically, the basal ganglia (BG). We focus here on the specific case of how different frontal areas support goal-directed, motivated decision-making, by representing three essential types of information: possible plans of action (in more dorsal and lateral frontal areas), affectively significant outcomes of those action plans (in ventral, medial frontal areas including the orbital frontal cortex), and the overall utility of a given plan compared to other possible courses of action (in anterior cingulate cortex). Computational models of goal-directed action selection at multiple different levels of analysis provide insight into the nature of learning and processing in these areas and the relative contributions of the frontal cortex versus the BG. The most common neurologic disorders implicate these areas, and understanding their precise function and modes of dysfunction can contribute to the new field of computational psychiatry, within the broader field of computational neuroscience.
Assuntos
Simulação por Computador , Lobo Frontal/fisiologia , Modelos Neurológicos , Motivação/fisiologia , HumanosRESUMO
BACKGROUND: Previous research in patients with anorexia nervosa showed heightened brain response during a taste reward conditioning task and heightened sensitivity to rewarding and punishing stimuli. Here we tested the hypothesis that individuals recovered from anorexia nervosa would also experience greater brain activation during this task as well as higher sensitivity to salient stimuli than controls. METHODS: Women recovered from restricting-type anorexia nervosa and healthy control women underwent fMRI during application of a prediction error taste reward learning paradigm. RESULTS: Twenty-four women recovered from anorexia nervosa (mean age 30.3 ± 8.1 yr) and 24 control women (mean age 27.4 ± 6.3 yr) took part in this study. The recovered anorexia nervosa group showed greater left posterior insula activation for the prediction error model analysis than the control group (family-wise error- and small volume-corrected p < 0.05). A group × condition analysis found greater posterior insula response in women recovered from anorexia nervosa than controls for unexpected stimulus omission, but not for unexpected receipt. Sensitivity to punishment was elevated in women recovered from anorexia nervosa. LIMITATIONS: This was a cross-sectional study, and the sample size was modest. CONCLUSION: Anorexia nervosa after recovery is associated with heightened prediction error-related brain response in the posterior insula as well as greater response to unexpected reward stimulus omission. This finding, together with behaviourally increased sensitivity to punishment, could indicate that individuals recovered from anorexia nervosa are particularly responsive to punishment. The posterior insula processes somatosensory stimuli, including unexpected bodily states, and greater response could indicate altered perception or integration of unexpected or maybe unwanted bodily feelings. Whether those findings develop during the ill state or whether they are biological traits requires further study.
Assuntos
Anorexia Nervosa/fisiopatologia , Anorexia Nervosa/psicologia , Antecipação Psicológica/fisiologia , Recompensa , Córtex Somatossensorial/fisiopatologia , Percepção Gustatória/fisiologia , Adulto , Anorexia Nervosa/terapia , Aprendizagem por Associação/fisiologia , Mapeamento Encefálico , Simulação por Computador , Condicionamento Psicológico/fisiologia , Estudos Transversais , Sacarose Alimentar , Feminino , Humanos , Imageamento por Ressonância Magnética , Modelos Neurológicos , Modelos Psicológicos , Testes Neuropsicológicos , Córtex Somatossensorial/diagnóstico por imagem , Percepção Visual/fisiologiaRESUMO
Decades of animal and human neuroimaging research have identified distinct, but overlapping, striatal zones, which are interconnected with separable corticostriatal circuits, and are crucial for the organization of functional systems. Despite continuous efforts to subdivide the human striatum based on anatomical and resting-state functional connectivity, characterizing the different psychological processes related to each zone remains a work in progress. Using an unbiased, data-driven approach, we analyzed large-scale coactivation data from 5,809 human imaging studies. We (i) identified five distinct striatal zones that exhibited discrete patterns of coactivation with cortical brain regions across distinct psychological processes and (ii) identified the different psychological processes associated with each zone. We found that the reported pattern of cortical activation reliably predicted which striatal zone was most strongly activated. Critically, activation in each functional zone could be associated with distinct psychological processes directly, rather than inferred indirectly from psychological functions attributed to associated cortices. Consistent with well-established findings, we found an association of the ventral striatum (VS) with reward processing. Confirming less well-established findings, the VS and adjacent anterior caudate were associated with evaluating the value of rewards and actions, respectively. Furthermore, our results confirmed a sometimes overlooked specialization of the posterior caudate nucleus for executive functions, often considered the exclusive domain of frontoparietal cortical circuits. Our findings provide a precise functional map of regional specialization within the human striatum, both in terms of the differential cortical regions and psychological functions associated with each striatal zone.
Assuntos
Corpo Estriado/fisiologia , Processos Mentais , Humanos , Idioma , Desempenho Psicomotor , Comportamento SocialRESUMO
We present a cerebellar architecture with two main characteristics. The first one is that complex spikes respond to increases in sensory errors. The second one is that cerebellar modules associate particular contexts where errors have increased in the past with corrective commands that stop the increase in error. We analyze our architecture formally and computationally for the case of reaching in a 3D environment. In the case of motor control, we show that there are synergies of this architecture with the Equilibrium-Point hypothesis, leading to novel ways to solve the motor error and distal learning problems. In particular, the presence of desired equilibrium lengths for muscles provides a way to know when the error is increasing, and which corrections to apply. In the context of Threshold Control Theory and Perceptual Control Theory we show how to extend our model so it implements anticipative corrections in cascade control systems that span from muscle contractions to cognitive operations.