RESUMO
Perceptual decisions should depend on sensory evidence. However, such decisions are also influenced by past choices and outcomes. These choice history biases may reflect advantageous strategies to exploit temporal regularities of natural environments. However, it is unclear whether and how observers can adapt their choice history biases to different temporal regularities, to exploit the multitude of temporal correlations that exist in nature. Here, we show that male mice adapt their perceptual choice history biases to different temporal regularities of visual stimuli. This adaptation was slow, evolving over hundreds of trials across several days. It occurred alongside a fast non-adaptive choice history bias, limited to a few trials. Both fast and slow trial history effects are well captured by a normative reinforcement learning algorithm with multi-trial belief states, comprising both current trial sensory and previous trial memory states. We demonstrate that dorsal striatal dopamine tracks predictions of the model and behavior, suggesting that striatal dopamine reports reward predictions associated with adaptive choice history biases. Our results reveal the adaptive nature of perceptual choice history biases and shed light on their underlying computational principles and neural correlates.
Assuntos
Comportamento de Escolha , Corpo Estriado , Dopamina , Animais , Masculino , Dopamina/metabolismo , Camundongos , Corpo Estriado/metabolismo , Corpo Estriado/fisiologia , Comportamento de Escolha/fisiologia , Camundongos Endogâmicos C57BL , Tomada de Decisões/fisiologia , Recompensa , Estimulação Luminosa , Percepção Visual/fisiologia , Reforço PsicológicoRESUMO
The optimal way to make decisions in many circumstances is to track the difference in evidence collected in favor of the options. The drift diffusion model (DDM) implements this approach and provides an excellent account of decisions and response times. However, existing DDM-based models of confidence exhibit certain deficits, and many theories of confidence have used alternative, nonoptimal models of decisions. Motivated by the historical success of the DDM, we ask whether simple extensions to this framework might allow it to better account for confidence. Motivated by the idea that the brain will not duplicate representations of evidence, in all model variants decisions and confidence are based on the same evidence accumulation process. We compare the models to benchmark results, and successfully apply four qualitative tests concerning the relationships between confidence, evidence, and time, in a new preregistered study. Using computationally cheap expressions to model confidence on a trial-by-trial basis, we find that a subset of model variants also provide a very good to excellent account of precise quantitative effects observed in confidence data. Specifically, our results favor the hypothesis that confidence reflects the strength of accumulated evidence penalized by the time taken to reach the decision (Bayesian readout), with the penalty applied not perfectly calibrated to the specific task context. These results suggest there is no need to abandon the DDM or single accumulator models to successfully account for confidence reports. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
RESUMO
Objective.Phase-amplitude coupling (PAC), the coupling of the amplitude of a faster brain rhythm to the phase of a slower brain rhythm, plays a significant role in brain activity and has been implicated in various neurological disorders. For example, in Parkinson's disease, PAC between the beta (13-30 Hz) and gamma (30-100 Hz) rhythms in the motor cortex is exaggerated, while in Alzheimer's disease, PAC between the theta (4-8 Hz) and gamma rhythms is diminished. Modulating PAC (i.e. reducing or enhancing PAC) using brain stimulation could therefore open new therapeutic avenues. However, while it has been previously reported that phase-locked stimulation can increase PAC, it is unclear what the optimal stimulation strategy to modulate PAC might be. Here, we provide a theoretical framework to narrow down the experimental optimisation of stimulation aimed at modulating PAC, which would otherwise rely on trial and error.Approach.We make analytical predictions using a Stuart-Landau model, and confirm these predictions in a more realistic model of coupled neural populations.Main results.Our framework specifies the critical Fourier coefficients of the stimulation waveform which should be tuned to optimally modulate PAC. Depending on the characteristics of the amplitude response curve of the fast population, these components may include the slow frequency, the fast frequency, combinations of these, as well as their harmonics. We also show that the optimal balance of energy between these Fourier components depends on the relative strength of the endogenous slow and fast rhythms, and that the alignment of fast components with the fast rhythm should change throughout the slow cycle. Furthermore, we identify the conditions requiring to phase-lock stimulation to the fast and/or slow rhythms.Significance.Together, our theoretical framework lays the foundation for guiding the development of innovative and more effective brain stimulation aimed at modulating PAC for therapeutic benefit.
Assuntos
Encéfalo , Humanos , Encéfalo/fisiologia , Modelos Neurológicos , Ondas Encefálicas/fisiologia , Simulação por ComputadorRESUMO
One of the key problems the brain faces is inferring the state of the world from a sequence of dynamically changing stimuli, and it is not yet clear how the sensory system achieves this task. A well-established computational framework for describing perceptual processes in the brain is provided by the theory of predictive coding. Although the original proposals of predictive coding have discussed temporal prediction, later work developing this theory mostly focused on static stimuli, and key questions on neural implementation and computational properties of temporal predictive coding networks remain open. Here, we address these questions and present a formulation of the temporal predictive coding model that can be naturally implemented in recurrent networks, in which activity dynamics rely only on local inputs to the neurons, and learning only utilises local Hebbian plasticity. Additionally, we show that temporal predictive coding networks can approximate the performance of the Kalman filter in predicting behaviour of linear systems, and behave as a variant of a Kalman filter which does not track its own subjective posterior variance. Importantly, temporal predictive coding networks can achieve similar accuracy as the Kalman filter without performing complex mathematical operations, but just employing simple computations that can be implemented by biological networks. Moreover, when trained with natural dynamic inputs, we found that temporal predictive coding can produce Gabor-like, motion-sensitive receptive fields resembling those observed in real neurons in visual areas. In addition, we demonstrate how the model can be effectively generalized to nonlinear systems. Overall, models presented in this paper show how biologically plausible circuits can predict future stimuli and may guide research on understanding specific neural circuits in brain areas involved in temporal prediction.
Assuntos
Encéfalo , Modelos Neurológicos , Encéfalo/fisiologia , Aprendizagem , Neurônios/fisiologiaRESUMO
When facing an unfamiliar environment, animals need to explore to gain new knowledge about which actions provide reward, but also put the newly acquired knowledge to use as quickly as possible. Optimal reinforcement learning strategies should therefore assess the uncertainties of these action-reward associations and utilise them to inform decision making. We propose a novel model whereby direct and indirect striatal pathways act together to estimate both the mean and variance of reward distributions, and mesolimbic dopaminergic neurons provide transient novelty signals, facilitating effective uncertainty-driven exploration. We utilised electrophysiological recording data to verify our model of the basal ganglia, and we fitted exploration strategies derived from the neural model to data from behavioural experiments. We also compared the performance of directed exploration strategies inspired by our basal ganglia model with other exploration algorithms including classic variants of upper confidence bound (UCB) strategy in simulation. The exploration strategies inspired by the basal ganglia model can achieve overall superior performance in simulation, and we found qualitatively similar results in fitting model to behavioural data compared with the fitting of more idealised normative models with less implementation level detail. Overall, our results suggest that transient dopamine levels in the basal ganglia that encode novelty could contribute to an uncertainty representation which efficiently drives exploration in reinforcement learning.
Assuntos
Gânglios da Base , Dopamina , Modelos Neurológicos , Recompensa , Dopamina/metabolismo , Dopamina/fisiologia , Incerteza , Animais , Gânglios da Base/fisiologia , Comportamento Exploratório/fisiologia , Reforço Psicológico , Neurônios Dopaminérgicos/fisiologia , Biologia Computacional , Simulação por Computador , Masculino , Algoritmos , Tomada de Decisões/fisiologia , Comportamento Animal/fisiologia , RatosRESUMO
For both humans and machines, the essence of learning is to pinpoint which components in its information processing pipeline are responsible for an error in its output, a challenge that is known as 'credit assignment'. It has long been assumed that credit assignment is best solved by backpropagation, which is also the foundation of modern machine learning. Here, we set out a fundamentally different principle on credit assignment called 'prospective configuration'. In prospective configuration, the network first infers the pattern of neural activity that should result from learning, and then the synaptic weights are modified to consolidate the change in neural activity. We demonstrate that this distinct mechanism, in contrast to backpropagation, (1) underlies learning in a well-established family of models of cortical circuits, (2) enables learning that is more efficient and effective in many contexts faced by biological organisms and (3) reproduces surprising patterns of neural activity and behavior observed in diverse human and rat learning experiments.
Assuntos
Aprendizado de Máquina , Redes Neurais de Computação , Humanos , Ratos , Animais , Estudos Prospectivos , Plasticidade NeuronalRESUMO
OBJECTIVES: The exact mechanisms of deep brain stimulation (DBS) are still an active area of investigation, in spite of its clinical successes. This is due in part to the lack of understanding of the effects of stimulation on neuronal rhythms. Entrainment of brain oscillations has been hypothesised as a potential mechanism of neuromodulation. A better understanding of entrainment might further inform existing methods of continuous DBS, and help refine algorithms for adaptive methods. The purpose of this study is to develop and test a theoretical framework to predict entrainment of cortical rhythms to DBS across a wide range of stimulation parameters. MATERIALS AND METHODS: We fit a model of interacting neural populations to selected features characterising PD patients' off-stimulation finely-tuned gamma rhythm recorded through electrocorticography. Using the fitted models, we predict basal ganglia DBS parameters that would result in 1:2 entrainment, a special case of sub-harmonic entrainment observed in patients and predicted by theory. RESULTS: We show that the neural circuit models fitted to patient data exhibit 1:2 entrainment when stimulation is provided across a range of stimulation parameters. Furthermore, we verify key features of the region of 1:2 entrainment in the stimulation frequency/amplitude space with follow-up recordings from the same patients, such as the loss of 1:2 entrainment above certain stimulation amplitudes. CONCLUSION: Our results reveal that continuous, constant frequency DBS in patients may lead to nonlinear patterns of neuronal entrainment across stimulation parameters, and that these responses can be predicted by modelling. Should entrainment prove to be an important mechanism of therapeutic stimulation, our modelling framework may reduce the parameter space that clinicians must consider when programming devices for optimal benefit.
Assuntos
Estimulação Encefálica Profunda , Doença de Parkinson , Humanos , Doença de Parkinson/terapia , Estimulação Encefálica Profunda/métodos , Gânglios da Base , Modalidades de Fisioterapia , EletrocorticografiaRESUMO
Adapting actions to changing goals and environments is central to intelligent behavior. There is evidence that the basal ganglia play a crucial role in reinforcing or adapting actions depending on their outcome. However, the corresponding electrophysiological correlates in the basal ganglia and the extent to which these causally contribute to action adaptation in humans is unclear. Here, we recorded electrophysiological activity and applied bursts of electrical stimulation to the subthalamic nucleus, a core area of the basal ganglia, in 16 patients with Parkinson's disease (PD) on medication using temporarily externalized deep brain stimulation (DBS) electrodes. Patients as well as 16 age- and gender-matched healthy participants attempted to produce forces as close as possible to a target force to collect a maximum number of points. The target force changed over trials without being explicitly shown on the screen so that participants had to infer target force based on the feedback they received after each movement. Patients and healthy participants were able to adapt their force according to the feedback they received (P < 0.001). At the neural level, decreases in subthalamic beta (13 to 30 Hz) activity reflected poorer outcomes and stronger action adaptation in 2 distinct time windows (Pcluster-corrected < 0.05). Stimulation of the subthalamic nucleus reduced beta activity and led to stronger action adaptation if applied within the time windows when subthalamic activity reflected action outcomes and adaptation (Pcluster-corrected < 0.05). The more the stimulation volume was connected to motor cortex, the stronger was this behavioral effect (Pcorrected = 0.037). These results suggest that dynamic modulation of the subthalamic nucleus and interconnected cortical areas facilitates adaptive behavior.
Assuntos
Estimulação Encefálica Profunda , Doença de Parkinson , Núcleo Subtalâmico , Humanos , Núcleo Subtalâmico/fisiologia , Estimulação Encefálica Profunda/métodos , Doença de Parkinson/terapia , Gânglios da Base , Adaptação PsicológicaRESUMO
INTRODUCTION: We assess risks differently when they are explicitly described, compared to when we learn directly from experience, suggesting dissociable decision-making systems. Our needs, such as hunger, could globally affect our risk preferences, but do they affect described and learned risks equally? On one hand, decision-making from descriptions is often considered flexible and context sensitive, and might therefore be modulated by metabolic needs. On the other hand, preferences learned through reinforcement might be more strongly coupled to biological drives. METHOD: Thirty-two healthy participants (females: 20, mean age: 25.6 ± 6.5 years) with a normal weight (Body Mass Index: 22.9 ± 3.2 kg/m2 ) were tested in a within-subjects counterbalanced, randomized crossover design for the effects of hunger on two separate risk-taking tasks. We asked participants to choose between two options with different risks to obtain monetary outcomes. In one task, the outcome probabilities were described numerically, whereas in a second task, they were learned. RESULT: In agreement with previous studies, we found that rewarding contexts induced risk-aversion when risks were explicitly described (F1,31 = 55.01, p < .0001, ηp 2 = .64), but risk-seeking when they were learned through experience (F1,31 = 10.28, p < .003, ηp 2 = .25). Crucially, hunger attenuated these contextual biases, but only for learned risks (F1,31 = 8.38, p < .007, ηp 2 = .21). CONCLUSION: The results suggest that our metabolic state determines risk-taking biases when we lack explicit descriptions.
Assuntos
Jogo de Azar , Adulto , Feminino , Humanos , Adulto Jovem , Tomada de Decisões , Fome , Probabilidade , Assunção de Riscos , Estômago , Estudos Cross-OverRESUMO
The computational principles adopted by the hippocampus in associative memory (AM) tasks have been one of the most studied topics in computational and theoretical neuroscience. Recent theories suggested that AM and the predictive activities of the hippocampus could be described within a unitary account, and that predictive coding underlies the computations supporting AM in the hippocampus. Following this theory, a computational model based on classical hierarchical predictive networks was proposed and was shown to perform well in various AM tasks. However, this fully hierarchical model did not incorporate recurrent connections, an architectural component of the CA3 region of the hippocampus that is crucial for AM. This makes the structure of the model inconsistent with the known connectivity of CA3 and classical recurrent models such as Hopfield Networks, which learn the covariance of inputs through their recurrent connections to perform AM. Earlier PC models that learn the covariance information of inputs explicitly via recurrent connections seem to be a solution to these issues. Here, we show that although these models can perform AM, they do it in an implausible and numerically unstable way. Instead, we propose alternatives to these earlier covariance-learning predictive coding networks, which learn the covariance information implicitly and plausibly, and can use dendritic structures to encode prediction errors. We show analytically that our proposed models are perfectly equivalent to the earlier predictive coding model learning covariance explicitly, and encounter no numerical issues when performing AM tasks in practice. We further show that our models can be combined with hierarchical predictive coding networks to model the hippocampo-neocortical interactions. Our models provide a biologically plausible approach to modelling the hippocampal network, pointing to a potential computational mechanism during hippocampal memory formation and recall, which employs both predictive coding and covariance learning based on the recurrent network structure of the hippocampus.
Assuntos
Hipocampo , Aprendizagem , Rememoração Mental , Condicionamento Clássico , Modelos NeurológicosRESUMO
Objective.While brain stimulation therapies such as deep brain stimulation for Parkinson's disease (PD) can be effective, they have yet to reach their full potential across neurological disorders. Entraining neuronal rhythms using rhythmic brain stimulation has been suggested as a new therapeutic mechanism to restore neurotypical behaviour in conditions such as chronic pain, depression, and Alzheimer's disease. However, theoretical and experimental evidence indicate that brain stimulation can also entrain neuronal rhythms at sub- and super-harmonics, far from the stimulation frequency. Crucially, these counterintuitive effects could be harmful to patients, for example by triggering debilitating involuntary movements in PD. We therefore seek a principled approach to selectively promote rhythms close to the stimulation frequency, while avoiding potential harmful effects by preventing entrainment at sub- and super-harmonics.Approach.Our open-loop approach to selective entrainment, dithered stimulation, consists in adding white noise to the stimulation period.Main results.We theoretically establish the ability of dithered stimulation to selectively entrain a given brain rhythm, and verify its efficacy in simulations of uncoupled neural oscillators, and networks of coupled neural oscillators. Furthermore, we show that dithered stimulation can be implemented in neurostimulators with limited capabilities by toggling within a finite set of stimulation frequencies.Significance.Likely implementable across a variety of existing brain stimulation devices, dithering-based selective entrainment has potential to enable new brain stimulation therapies, as well as new neuroscientific research exploiting its ability to modulate higher-order entrainment.
Assuntos
Doença de Alzheimer , Doença de Parkinson , Humanos , Técnicas Estereotáxicas , Encéfalo , Doença de Parkinson/terapiaRESUMO
Forming accurate memory of sequential stimuli is a fundamental function of biological agents. However, the computational mechanism underlying sequential memory in the brain remains unclear. Inspired by neuroscience theories and recent successes in applying predictive coding (PC) to static memory tasks, in this work we propose a novel PC-based model for sequential memory, called temporal predictive coding (tPC). We show that our tPC models can memorize and retrieve sequential inputs accurately with a biologically plausible neural implementation. Importantly, our analytical study reveals that tPC can be viewed as a classical Asymmetric Hopfield Network (AHN) with an implicit statistical whitening process, which leads to more stable performance in sequential memory tasks of structured inputs. Moreover, we find that tPC exhibits properties consistent with behavioral observations and theories in neuroscience, thereby strengthening its biological relevance. Our work establishes a possible computational mechanism underlying sequential memory in the brain that can also be theoretically interpreted using existing memory model frameworks.
RESUMO
We introduce a new approach to modelling decision confidence, with the aim of enabling computationally cheap predictions while taking into account, and thereby exploiting, trial-by-trial variability in stochastically fluctuating stimuli. Using the framework of the drift diffusion model of decision making, along with time-dependent thresholds and the idea of a Bayesian confidence readout, we derive expressions for the probability distribution over confidence reports. In line with current models of confidence, the derivations allow for the accumulation of "pipeline" evidence that has been received but not processed by the time of response, the effect of drift rate variability, and metacognitive noise. The expressions are valid for stimuli that change over the course of a trial with normally-distributed fluctuations in the evidence they provide. A number of approximations are made to arrive at the final expressions, and we test all approximations via simulation. The derived expressions contain only a small number of standard functions, and require evaluating only once per trial, making trial-by-trial modelling of confidence data in stochastically fluctuating stimuli tasks more feasible. We conclude by using the expressions to gain insight into the confidence of optimal observers, and empirically observed patterns.
RESUMO
To optimally adjust our behavior to changing environments we need to both adjust the speed of our decisions and movements. Yet little is known about the extent to which these processes are controlled by common or separate mechanisms. Furthermore, while previous evidence from computational models and empirical studies suggests that the basal ganglia play an important role during adjustments of decision-making, it remains unclear how this is implemented. Leveraging the opportunity to directly access the subthalamic nucleus of the basal ganglia in humans undergoing deep brain stimulation surgery, we here combine invasive electrophysiological recordings, electrical stimulation and computational modelling of perceptual decision-making. We demonstrate that, while similarities between subthalamic control of decision- and movement speed exist, the causal contribution of the subthalamic nucleus to these processes can be disentangled. Our results show that the basal ganglia independently control the speed of decisions and movement for each hemisphere during adaptive behavior.
Assuntos
Gânglios da Base , HumanosRESUMO
PURPOSE: Despite the improvement in treatment and prognosis of primary central nervous system lymphoma (PCNSL) over the last decades, the 5-year survival rate is approximately 30%; thus, new therapeutic approaches are needed to improve patient survival. The study's aim was to evaluate the role of surgical resection of PCNSL. METHODS: Primary outcomes were the overall survival (OS) and progression-free survival (PFS) of patients with PCNSL who underwent surgical resection versus biopsy alone. The meta-analysis was conducted to calculate pooled hazard ratios (HRs) under a random-effects model for the time-to-event variables. The odds ratios (ORs) were calculated for binary, secondary outcome parameters. RESULTS: Seven studies (n = 1046) were included. We found that surgical resection was associated with significantly better OS (HR 0.63 [95% CI 0.51-0.77]) when compared with biopsy. PFS was also significantly improved (HR 0.64 [95% CI 0.49-0.85]) in patients who underwent resection compared with those who underwent biopsy. The heterogeneity for OS and PFS was low (I2 = 7% and 24%, respectively). We also found that patients who underwent biopsy more often had multiple (OR 0.38 [95% CI 0.19-0.79]) or deep-seated (OR 0.20 [95% CI 0.12-0.34]) lesions compared with those who underwent surgical resection. There were no significant differences in chemotherapy or radiotherapy use or the occurrence of postoperative complications between the two groups. CONCLUSION: In selected patients, surgical resection of PCNSL is associated with significantly better overall survival and progression-free survival compared with biopsy alone.
Assuntos
Neoplasias do Sistema Nervoso Central , Linfoma , Humanos , Intervalo Livre de Progressão , Biópsia , Terapia Combinada , Linfoma/cirurgia , Linfoma/tratamento farmacológico , Sistema Nervoso CentralRESUMO
To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.
Assuntos
Gânglios da Base , Aprendizagem , Gânglios da Base/fisiologia , Dopamina/fisiologia , Aprendizagem/fisiologia , Reforço Psicológico , Recompensa , IncertezaRESUMO
Making accurate decisions often involves the integration of current and past evidence. Here, we examine the neural correlates of conflict and evidence integration during sequential decision-making. Female and male human patients implanted with deep-brain stimulation (DBS) electrodes and age-matched and gender-matched healthy controls performed an expanded judgment task, in which they were free to choose how many cues to sample. Behaviorally, we found that while patients sampled numerically more cues, they were less able to integrate evidence and showed suboptimal performance. Using recordings of magnetoencephalography (MEG) and local field potentials (LFPs; in patients) in the subthalamic nucleus (STN), we found that ß oscillations signaled conflict between cues within a sequence. Following cues that differed from previous cues, ß power in the STN and cortex first decreased and then increased. Importantly, the conflict signal in the STN outlasted the cortical one, carrying over to the next cue in the sequence. Furthermore, after a conflict, there was an increase in coherence between the dorsal premotor cortex and STN in the ß band. These results extend our understanding of cortico-subcortical dynamics of conflict processing, and do so in a context where evidence must be accumulated in discrete steps, much like in real life. Thus, the present work leads to a more nuanced picture of conflict monitoring systems in the brain and potential changes because of disease.SIGNIFICANCE STATEMENT Decision-making often involves the integration of multiple pieces of information over time to make accurate predictions. We simultaneously recorded whole-head magnetoencephalography (MEG) and local field potentials (LFPs) from the human subthalamic nucleus (STN) in a novel task which required integrating sequentially presented pieces of evidence. Our key finding is prolonged ß oscillations in the STN, with a concurrent increase in communication with frontal cortex, when presented with conflicting information. These neural effects reflect the behavioral profile of reduced tendency to respond after conflict, as well as relate to suboptimal cue integration in patients, which may be directly linked to clinically reported side-effects of deep-brain stimulation (DBS) such as impaired decision-making and impulsivity.
Assuntos
Estimulação Encefálica Profunda , Córtex Motor , Doença de Parkinson , Núcleo Subtalâmico , Ritmo beta , Estimulação Encefálica Profunda/métodos , Feminino , Humanos , Magnetoencefalografia , Masculino , Córtex Motor/fisiologia , Doença de Parkinson/terapia , Núcleo Subtalâmico/fisiologiaRESUMO
A large number of neural network models of associative memory have been proposed in the literature. These include the classical Hopfield networks (HNs), sparse distributed memories (SDMs), and more recently the modern continuous Hopfield networks (MCHNs), which possess close links with self-attention in machine learning. In this paper, we propose a general framework for understanding the operation of such memory networks as a sequence of three operations: similarity, separation, and projection. We derive all these memory models as instances of our general framework with differing similarity and separation functions. We extend the mathematical framework of Krotov & Hopfield (2020) to express general associative memory models using neural network dynamics with local computation, and derive a general energy function that is a Lyapunov function of the dynamics. Finally, using our framework, we empirically investigate the capacity of using different similarity functions for these associative memory models, beyond the dot product similarity measure, and demonstrate empirically that Euclidean or Manhattan distance similarity metrics perform substantially better in practice on many tasks, enabling a more robust retrieval and higher memory capacity than existing models.
RESUMO
INTRODUCTION: Clinical depression is usually treated in primary care with psychological therapies and antidepressant medication. However, when patients do not respond to at least two or more antidepressants within a depressive episode, they are considered to have treatment resistant depression (TRD). Previous small randomised controlled trials suggested that pramipexole, a dopamine D2/3 receptor agonist, may be effective for treating patients with unipolar and bipolar depression as it is known to influence motivational drive and reward processing. PAX-D will compare the effects of pramipexole vs placebo when added to current antidepressant medication for people with TRD. Additionally, PAX-D will investigate the mechanistic effect of pramipexole on reward sensitivity using a probabilistic decision-making task. METHODS AND ANALYSIS: PAX-D will assess effectiveness in the short- term (during the first 12 weeks) and in the longer-term (48 weeks) in patients with TRD from the UK. The primary outcome will be change in self-reported depressive symptoms from baseline to week 12 post-randomisation measured using the Quick Inventory of Depressive Symptomatology Self-Report (QIDS-SR16). Performance on the decision-making task will be measured at week 0, week 2 and week 12. Secondary outcomes include anhedonia, anxiety and health economic measures including quality of life, capability, well-being and costs. PAX-D will also assess the adverse effects of pramipexole including impulse control difficulties. DISCUSSION: Pramipexole is a promising augmentation agent for TRD and may be a useful addition to existing treatment regimes. PAX-D will assess its effectiveness and test for a potential mechanism of action in patients with TRD. TRIAL REGISTRATION NUMBER: ISRCTN84666271.
Assuntos
Transtorno Depressivo Maior , Transtorno Depressivo Resistente a Tratamento , Antidepressivos/efeitos adversos , Depressão/tratamento farmacológico , Transtorno Depressivo Maior/tratamento farmacológico , Transtorno Depressivo Resistente a Tratamento/tratamento farmacológico , Humanos , Pramipexol/farmacologia , Pramipexol/uso terapêutico , Qualidade de Vida , Ensaios Clínicos Controlados Aleatórios como AssuntoRESUMO
Reinforcement learning involves updating estimates of the value of states and actions on the basis of experience. Previous work has shown that in humans, reinforcement learning exhibits a confirmatory bias: when the value of a chosen option is being updated, estimates are revised more radically following positive than negative reward prediction errors, but the converse is observed when updating the unchosen option value estimate. Here, we simulate performance on a multi-arm bandit task to examine the consequences of a confirmatory bias for reward harvesting. We report a paradoxical finding: that confirmatory biases allow the agent to maximize reward relative to an unbiased updating rule. This principle holds over a wide range of experimental settings and is most influential when decisions are corrupted by noise. We show that this occurs because on average, confirmatory biases lead to overestimating the value of more valuable bandits and underestimating the value of less valuable bandits, rendering decisions overall more robust in the face of noise. Our results show how apparently suboptimal learning rules can in fact be reward maximizing if decisions are made with finite computational precision.