Subthalamic control of impulsive actions: insights from deep brain stimulation in Parkinson's disease.

Control of actions allows adaptive, goal-directed behaviour. The basal ganglia, including the subthalamic nucleus, are thought to play a central role in dynamically controlling actions through recurrent negative feedback loops with the cerebral cortex. Here, we summarize recent translational studies that used deep brain stimulation to record neural activity from and apply electrical stimulation to the subthalamic nucleus in people with Parkinson's disease. These studies have elucidated spatial, spectral and temporal features of the neural mechanisms underlying the controlled delay of actions in cortico-subthalamic networks and demonstrated their causal effects on behaviour in distinct processing windows. While these mechanisms have been conceptualized as control signals for suppressing impulsive response tendencies in conflict tasks and as decision threshold adjustments in value-based and perceptual decisions, we propose a common framework linking decision-making, cognition and movement. Within this framework subthalamic deep brain stimulation can lead to suboptimal choices by reducing the time that patients take for deliberation before committing to an action. However, clinical studies have consistently shown that the occurrence of impulse control disorders is reduced, not increased, after subthalamic deep brain stimulation surgery. This apparent contradiction can be reconciled when recognizing the multifaceted nature of impulsivity, its underlying mechanisms and modulation by treatment. While subthalamic deep brain stimulation renders patients susceptible to making decisions without proper forethought, this can be disentangled from effects related to dopamine comprising sensitivity to benefits vs. costs, reward delay aversion and learning from outcomes. Alterations in these dopamine-mediated mechanisms are thought to underlie the development of impulse control disorders, and can be relatively spared with reduced dopaminergic medication after subthalamic deep brain stimulation. Together, results from studies using deep brain stimulation as an experimental tool have improved our understanding of action control in the human brain and have important implications for treatment of patients with Neurological disorders.

Ethological computational psychiatry: Challenges and opportunities.

Studying the intricacies of individual subjects' moods and cognitive processing over extended periods of time presents a formidable challenge in medicine. While much of systems neuroscience appropriately focuses on the link between neural circuit functions and well-constrained behaviors over short timescales (e.g., trials, hours), many mental health conditions involve complex interactions of mood and cognition that are non-stationary across behavioral contexts and evolve over extended timescales. Here, we discuss opportunities, challenges, and possible future directions in computational psychiatry to quantify non-stationary continuously monitored behaviors. We suggest that this exploratory effort may contribute to a more precision-based approach to treating mental disorders and facilitate a more robust reverse translation across animal species. We conclude with ethical considerations for any field that aims to bridge artificial intelligence and patient monitoring.

The challenge of learning adaptive mental behavior.

Many psychotherapies aim to help people replace maladaptive mental behaviors (such as those leading to unproductive worry) with more adaptive ones (such as those leading to active problem solving). Yet, little is known empirically about how challenging it is to learn adaptive mental behaviors. Mental behaviors entail taking mental operations and thus may be more challenging to perform than motor actions; this challenge may enhance or impair learning. In particular, challenge when learning is often desirable because it improves retention. Yet, it is also plausible that the necessity of carrying out mental operations interferes with learning the expected values of mental actions by impeding credit assignment: the process of updating an action's value after reinforcement. Then, it may be more challenging not only to perform-but also to learn the consequences of-mental (vs. motor) behaviors. We designed a task to assess learning to take adaptive mental versus motor actions via matched probabilistic feedback. In two experiments (N = 300), most participants found it more difficult to learn to select optimal mental (vs. motor) actions, as evident in worse accuracy not only in a learning but also test (retention) phase. Computational modeling traced this impairment to an indicator of worse credit assignment (impaired construction and maintenance of expected values) when learning mental actions, accounting for worse accuracy in the learning and retention phases. The results suggest that people have particular difficulty learning adaptive mental behavior and pave the way for novel interventions to scaffold credit assignment and promote adaptive thinking. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Assessing Trial-by-Trial Electrophysiological and Behavioral Markers of Attentional Control and Sensory Precision in Psychotic and Mood Disorders.

BACKGROUND AND HYPOTHESIS: The current study investigated the extent to which changes in attentional control contribute to performance on a visual perceptual discrimination task, on a trial-by-trial basis in a transdiagnostic clinical sample. STUDY DESIGN: Participants with schizophrenia (SZ; N = 58), bipolar disorder (N = 42), major depression disorder (N = 51), and psychiatrically healthy controls (N = 92) completed a visual perception task in which stimuli appeared briefly. The design allowed us to estimate the lapse rate and the precision of perceptual representations of the stimuli. Electroencephalograms (EEG) were recorded to examine pre-stimulus activity in the alpha band (8-13 Hz), overall and in relation to behavior performance on the task. STUDY RESULTS: We found that the attention lapse rate was elevated in the SZ group compared with all other groups. We also observed group differences in pre-stimulus alpha activity, with control participants showing the highest levels of pre-stimulus alpha when averaging across trials. However, trial-by-trial analyses showed within-participant fluctuations in pre-stimulus alpha activity significantly predicted the likelihood of making an error, in all groups. Interestingly, our analysis demonstrated that aperiodic contributions to the EEG signal (which affect power estimates across frequency bands) serve as a significant predictor of behavior as well. CONCLUSIONS: These results confirm the elevated attention lapse rate that has been observed in SZ, validate pre-stimulus EEG markers of attentional control and their use as a predictor of behavior on a trial-by-trial basis, and suggest that aperiodic contributions to the EEG signal are an important target for further research in this area, in addition to alpha-band activity.

Human Curriculum Effects Emerge with In-Context Learning in Neural Networks.

Human learning is sensitive to rule-like structure and the curriculum of examples used for training. In tasks governed by succinct rules, learning is more robust when related examples are blocked across trials, but in the absence of such rules, interleaving is more effective. To date, no neural model has simultaneously captured these seemingly contradictory effects. Here we show that this same tradeoff spontaneously emerges with "in-context learning" (ICL) both in neural networks trained with metalearning and in large language models (LLMs). ICL is the ability to learn new tasks "in context" - without weight changes - via an inner-loop algorithm implemented in activation dynamics. Experiments with pretrained LLMs and metalearning transformers show that ICL exhibits the blocking advantage demonstrated in humans on a task involving rule-like structure, and conversely, that concurrent in-weight learning reproduces the interleaving advantage observed in humans on tasks lacking such structure.

Cognitive Signatures of Depressive and Anhedonic Symptoms and Affective States Using Computational Modeling and Neurocognitive Testing.

BACKGROUND: Deeper phenotyping may improve our understanding of depression. Because depression is heterogeneous, extracting cognitive signatures associated with severity of depressive symptoms, anhedonia, and affective states is a promising approach. METHODS: Sequential sampling models decomposed behavior from an adaptive approach-avoidance conflict task into computational parameters quantifying latent cognitive signatures. Fifty unselected participants completed clinical scales and the approach-avoidance conflict task by either approaching or avoiding trials offering monetary rewards and electric shocks. RESULTS: Decision dynamics were best captured by a sequential sampling model with linear collapsing boundaries varying by net offer values, and with drift rates varying by trial-specific reward and aversion, reflecting net evidence accumulation toward approach or avoidance. Unlike conventional behavioral measures, these computational parameters revealed distinct associations with self-reported symptoms. Specifically, passive avoidance tendencies, indexed by starting point biases, were associated with greater severity of depressive symptoms (R = 0.34, p = .019) and anhedonia (R = 0.49, p = .001). Depressive symptoms were also associated with slower encoding and response execution, indexed by nondecision time (R = 0.37, p = .011). Higher reward sensitivity for offers with negative net values, indexed by drift rates, was linked to more sadness (R = 0.29, p = .042) and lower positive affect (R = -0.33, p = .022). Conversely, higher aversion sensitivity was associated with more tension (R = 0.33, p = .025). Finally, less cautious response patterns, indexed by boundary separation, were linked to more negative affect (R = -0.40, p = .005). CONCLUSIONS: We demonstrated the utility of multidimensional computational phenotyping, which could be applied to clinical samples to improve characterization and treatment selection.

Double dissociation of dopamine and subthalamic nucleus stimulation on effortful cost/benefit decision making.

Deep brain stimulation (DBS) and dopaminergic therapy (DA) are common interventions for Parkinson's disease (PD). Both treatments typically improve patient outcomes, and both can have adverse side effects on decision making (e.g., impulsivity).1,2 Nevertheless, they are thought to act via different mechanisms within basal ganglia circuits.3 Here, we developed and formally evaluated their dissociable predictions within a single cost/benefit effort-based decision-making task. In the same patients, we manipulated DA medication status and subthalamic nucleus (STN) DBS status within and across sessions. Using a series of descriptive and computational modeling analyses of participant choices and their dynamics, we confirm a double dissociation: DA medication asymmetrically altered participants' sensitivities to benefits vs. effort costs of alternative choices (boosting the sensitivity to benefits while simultaneously lowering sensitivity to costs); whereas STN DBS lowered the decision threshold of such choices. To our knowledge, this is the first study to show, using a common modeling framework, a dissociation of DA and DBS within the same participants. As such, this work offers a comprehensive account for how different mechanisms impact decision making, and how impulsive behavior (present in DA-treated patients with PD and DBS patients) may emerge from separate physiological mechanisms.

A Transdiagnostic Study of Effort-Cost Decision-Making in Psychotic and Mood Disorders.

BACKGROUND: Research suggests that effort-cost decision-making (ECDM), the estimation of work required to obtain reward, may be a relevant framework for understanding motivational impairment in psychotic and mood pathology. Specifically, research has suggested that people with psychotic and mood pathology experience effort as more costly than controls, and thus pursue effortful goals less frequently. This study examined ECDM across psychotic and mood pathology. HYPOTHESIS: We hypothesized that patient groups would show reduced willingness to expend effort compared to controls. STUDY DESIGN: People with schizophrenia (N = 33), schizoaffective disorder (N = 28), bipolar disorder (N = 39), major depressive disorder (N = 40), and controls (N = 70) completed a physical ECDM task. Participants decided between completing a low-effort or high-effort option for small or larger rewards, respectively. Reward magnitude, reward probability, and effort magnitude varied trial-by-trial. Data were analyzed using standard and hierarchical logistic regression analyses to assess the subject-specific contribution of various factors to choice. Negative symptoms were measured with a clinician-rated interview. STUDY RESULTS: There was a significant effect of group, driven by reduced choice of high-effort options in schizophrenia. Hierarchical logistic regression revealed that reduced choice of high-effort options in schizophrenia was driven by weaker contributions of probability information. Use of reward information was inversely associated with motivational impairment in schizophrenia. Surprisingly, individuals with major depressive disorder and bipolar disorder did not differ from controls. CONCLUSIONS: Our results provide support for ECDM deficits in schizophrenia. Additionally, differences between groups in ECDM suggest a seemingly similar behavioral phenotype, reduced motivation, could arise from disparate mechanisms.

Multiple routes to enhanced memory for emotionally relevant events.

Events associated with aversive or rewarding outcomes are prioritized in memory. This memory boost is commonly attributed to the elicited affective response, closely linked to noradrenergic and dopaminergic modulation of hippocampal plasticity. Herein we review and compare this 'affect' mechanism to an additional, recently discovered, 'prediction' mechanism whereby memories are strengthened by the extent to which outcomes deviate from expectations, that is, by prediction errors (PEs). The mnemonic impact of PEs is separate from the affective outcome itself and has a distinct neural signature. While both routes enhance memory, these mechanisms are linked to different - and sometimes opposing - predictions for memory integration. We discuss new findings that highlight mechanisms by which emotional events strengthen, integrate, and segment memory.

On the normative advantages of dopamine and striatal opponency for learning and choice.

The basal ganglia (BG) contribute to reinforcement learning (RL) and decision-making, but unlike artificial RL agents, it relies on complex circuitry and dynamic dopamine modulation of opponent striatal pathways to do so. We develop the OpAL* model to assess the normative advantages of this circuitry. In OpAL*, learning induces opponent pathways to differentially emphasize the history of positive or negative outcomes for each action. Dynamic DA modulation then amplifies the pathway most tuned for the task environment. This efficient coding mechanism avoids a vexing explore-exploit tradeoff that plagues traditional RL models in sparse reward environments. OpAL* exhibits robust advantages over alternative models, particularly in environments with sparse reward and large action spaces. These advantages depend on opponent and nonlinear Hebbian plasticity mechanisms previously thought to be pathological. Finally, OpAL* captures risky choice patterns arising from DA and environmental manipulations across species, suggesting that they result from a normative biological mechanism.

Neural Index of Reinforcement Learning Predicts Improved Stimulus-Response Retention under High Working Memory Load.

Human learning and decision-making are supported by multiple systems operating in parallel. Recent studies isolating the contributions of reinforcement learning (RL) and working memory (WM) have revealed a trade-off between the two. An interactive WM/RL computational model predicts that although high WM load slows behavioral acquisition, it also induces larger prediction errors in the RL system that enhance robustness and retention of learned behaviors. Here, we tested this account by parametrically manipulating WM load during RL in conjunction with EEG in both male and female participants and administered two surprise memory tests. We further leveraged single-trial decoding of EEG signatures of RL and WM to determine whether their interaction predicted robust retention. Consistent with the model, behavioral learning was slower for associations acquired under higher load but showed parametrically improved future retention. This paradoxical result was mirrored by EEG indices of RL, which were strengthened under higher WM loads and predictive of more robust future behavioral retention of learned stimulus-response contingencies. We further tested whether stress alters the ability to shift between the two systems strategically to maximize immediate learning versus retention of information and found that induced stress had only a limited effect on this trade-off. The present results offer a deeper understanding of the cooperative interaction between WM and RL and show that relying on WM can benefit the rapid acquisition of choice behavior during learning but impairs retention.SIGNIFICANCE STATEMENT Successful learning is achieved by the joint contribution of the dopaminergic RL system and WM. The cooperative WM/RL model was productive in improving our understanding of the interplay between the two systems during learning, demonstrating that reliance on RL computations is modulated by WM load. However, the role of WM/RL systems in the retention of learned stimulus-response associations remained unestablished. Our results show that increased neural signatures of learning, indicative of greater RL computation, under high WM load also predicted better stimulus-response retention. This result supports a trade-off between the two systems, where degraded WM increases RL processing, which improves retention. Notably, we show that this cooperative interplay remains largely unaffected by acute stress.

Adaptive control of synaptic plasticity integrates micro- and macroscopic network function.

Synaptic plasticity configures interactions between neurons and is therefore likely to be a primary driver of behavioral learning and development. How this microscopic-macroscopic interaction occurs is poorly understood, as researchers frequently examine models within particular ranges of abstraction and scale. Computational neuroscience and machine learning models offer theoretically powerful analyses of plasticity in neural networks, but results are often siloed and only coarsely linked to biology. In this review, we examine connections between these areas, asking how network computations change as a function of diverse features of plasticity and vice versa. We review how plasticity can be controlled at synapses by calcium dynamics and neuromodulatory signals, the manifestation of these changes in networks, and their impacts in specialized circuits. We conclude that metaplasticity-defined broadly as the adaptive control of plasticity-forges connections across scales by governing what groups of synapses can and can't learn about, when, and to what ends. The metaplasticity we discuss acts by co-opting Hebbian mechanisms, shifting network properties, and routing activity within and across brain systems. Asking how these operations can go awry should also be useful for understanding pathology, which we address in the context of autism, schizophrenia and Parkinson's disease.

Cognitive [Computational] Neuroscience Test Reliability and Clinical Applications for Serious Mental Illness (CNTRaCS) Consortium: Progress and Future Directions.

The development of treatments for impaired cognition in schizophrenia has been characterized as the most important challenge facing psychiatry at the beginning of the twenty-first century. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) project was designed to build on the potential benefits of using tasks and tools from cognitive neuroscience to better understanding and treat cognitive impairments in psychosis. These benefits include: (1) the use of fine-grained tasks that measure discrete cognitive processes; (2) the ability to design tasks that distinguish between specific cognitive domain deficits and poor performance due to generalized deficits resulting from sedation, low motivation, poor test taking skills, etc.; and (3) the ability to link cognitive deficits to specific neural systems, using animal models, neuropsychology, and functional imaging. CNTRICS convened a series of meetings to identify paradigms from cognitive neuroscience that maximize these benefits and identified the steps need for translation into use in clinical populations. The Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRaCS) Consortium was developed to help carry out these steps. CNTRaCS consists of investigators at five different sites across the country with diverse expertise relevant to a wide range of the cognitive systems identified as critical as part of CNTRICs. This work reports on the progress and current directions in the evaluation and optimization carried out by CNTRaCS of the tasks identified as part of the original CNTRICs process, as well as subsequent extensions into the Positive Valence systems domain of Research Domain Criteria (RDoC). We also describe the current focus of CNTRaCS, which involves taking a computational psychiatry approach to measuring cognitive and motivational function across the spectrum of psychosis. Specifically, the current iteration of CNTRaCS is using computational modeling to isolate parameters reflecting potentially more specific cognitive and visual processes that may provide greater interpretability in understanding shared and distinct impairments across psychiatric disorders.

Self-judgment dissected: A computational modeling analysis of self-referential processing and its relationship to trait mindfulness facets and depression symptoms.

Cognitive theories of depression, and mindfulness theories of well-being, converge on the notion that self-judgment plays a critical role in mental health. However, these theories have rarely been tested via tasks and computational modeling analyses that can disentangle the information processes operative in self-judgments. We applied a drift-diffusion computational model to the self-referential encoding task (SRET) collected before and after an 8-week mindfulness intervention (n = 96). A drift-rate regression parameter representing positive-relative to negative-self-referential judgment strength positively related to mindful awareness and inversely related to depression, both at baseline and over time; however, this parameter did not significantly relate to the interaction between mindful awareness and nonjudgmentalness. At the level of individual depression symptoms, at baseline, a spectrum of symptoms (inversely) correlated with the drift-rate regression parameter, suggesting that many distinct depression symptoms relate to valenced self-judgment between subjects. By contrast, over the intervention, changes in only a smaller subset of anhedonia-related depression symptoms showed substantial relationships with this parameter. Both behavioral and model-derived measures showed modest split-half and test-retest correlations. Results support cognitive theories that implicate self-judgment in depression and mindfulness theories, which imply that mindful awareness should lead to more positive self-views.

Humans adaptively resolve the explore-exploit dilemma under cognitive constraints: Evidence from a multi-armed bandit task.

When navigating uncertain worlds, humans must balance exploring new options versus exploiting known rewards. Longer horizons and spatially structured option values encourage humans to explore, but the impact of real-world cognitive constraints such as environment size and memory demands on explore-exploit decisions is unclear. In the present study, humans chose between options varying in uncertainty during a multi-armed bandit task with varying environment size and memory demands. Regression and cognitive computational models of choice behavior showed that with a lower cognitive load, humans are more exploratory than a simulated value-maximizing learner, but under cognitive constraints, they adaptively scale down exploration to maintain exploitation. Thus, while humans are curious, cognitive constraints force people to decrease their strategic exploration in a resource-rational-like manner to focus on harvesting known rewards.

Beyond Drift Diffusion Models: Fitting a Broad Class of Decision and Reinforcement Learning Models with HDDM.

Computational modeling has become a central aspect of research in the cognitive neurosciences. As the field matures, it is increasingly important to move beyond standard models to quantitatively assess models with richer dynamics that may better reflect underlying cognitive and neural processes. For example, sequential sampling models (SSMs) are a general class of models of decision-making intended to capture processes jointly giving rise to RT distributions and choice data in n-alternative choice paradigms. A number of model variations are of theoretical interest, but empirical data analysis has historically been tied to a small subset for which likelihood functions are analytically tractable. Advances in methods designed for likelihood-free inference have recently made it computationally feasible to consider a much larger spectrum of SSMs. In addition, recent work has motivated the combination of SSMs with reinforcement learning models, which had historically been considered in separate literatures. Here, we provide a significant addition to the widely used HDDM Python toolbox and include a tutorial for how users can easily fit and assess a (user-extensible) wide variety of SSMs and how they can be combined with reinforcement learning models. The extension comes batteries included, including model visualization tools, posterior predictive checks, and ability to link trial-wise neural signals with model parameters via hierarchical Bayesian regression.

Biophysical and Architectural Mechanisms of Subthalamic Theta under Response Conflict.

The cortico-basal ganglia circuit is needed to suppress prepotent actions and to facilitate controlled behavior. Under conditions of response conflict, the frontal cortex and subthalamic nucleus (STN) exhibit increased spiking and theta band power, which are linked to adaptive regulation of behavioral output. The electrophysiological mechanisms underlying these neural signatures of impulse control remain poorly understood. To address this lacuna, we constructed a novel large-scale, biophysically principled model of the subthalamopallidal (STN-globus pallidus externus) network and examined the mechanisms that modulate theta power and spiking in response to cortical input. Simulations confirmed that theta power does not emerge from intrinsic network dynamics but is robustly elicited in response to cortical input as burst events representing action selection dynamics. Rhythmic burst events of multiple cortical populations, representing a state of conflict where cortical motor plans vacillate in the theta range, led to prolonged STN theta and increased spiking, consistent with empirical literature. Notably, theta band signaling required NMDA, but not AMPA, currents, which were in turn related to a triphasic STN response characterized by spiking, silence, and bursting periods. Finally, theta band resonance was also strongly modulated by architectural connectivity, with maximal theta arising when multiple cortical populations project to individual STN "conflict detector" units because of an NMDA-dependent supralinear response. Our results provide insights into the biophysical principles and architectural constraints that give rise to STN dynamics during response conflict, and how their disruption can lead to impulsivity and compulsivity.SIGNIFICANCE STATEMENT The subthalamic nucleus exhibits theta band power modulation related to cognitive control over motor actions during conditions of response conflict. However, the mechanisms of such dynamics are not understood. Here we developed a novel biophysically detailed and data-constrained large-scale model of the subthalamopallidal network, and examined the impacts of cellular and network architectural properties that give rise to theta dynamics. Our investigations implicate an important role for NMDA receptors and cortico-subthalamic nucleus topographical connectivities in theta power modulation.

Honeycomb: a template for reproducible psychophysiological tasks for clinic, laboratory, and home use

Objective: To improve the ability of psychiatry researchers to build, deploy, maintain, reproduce, and share their own psychophysiological tasks. Psychophysiological tasks are a useful tool for studying human behavior driven by mental processes such as cognitive control, reward evaluation, and learning. Neural mechanisms during behavioral tasks are often studied via simultaneous electrophysiological recordings. Popular online platforms such as Amazon Mechanical Turk (MTurk) and Prolific enable deployment of tasks to numerous participants simultaneously. However, there is currently no task-creation framework available for flexibly deploying tasks both online and during simultaneous electrophysiology. Methods: We developed a task creation template, termed Honeycomb, that standardizes best practices for building jsPsych-based tasks. Honeycomb offers continuous deployment configurations for seamless transition between use in research settings and at home. Further, we have curated a public library, termed BeeHive, of ready-to-use tasks. Results: We demonstrate the benefits of using Honeycomb tasks with a participant in an ongoing study of deep brain stimulation for obsessive compulsive disorder, who completed repeated tasks both in the clinic and at home. Conclusion: Honeycomb enables researchers to deploy tasks online, in clinic, and at home in more ecologically valid environments and during concurrent electrophysiology.

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits.

Adaptive sequential behavior is a hallmark of human cognition. In particular, humans can learn to produce precise spatiotemporal sequences given a certain context. For instance, musicians can not only reproduce learned action sequences in a context-dependent manner, they can also quickly and flexibly reapply them in any desired tempo or rhythm without overwriting previous learning. Existing neural network models fail to account for these properties. We argue that this limitation emerges from the fact that sequence information (i.e., the position of the action) and timing (i.e., the moment of response execution) are typically stored in the same neural network weights. Here, we augment a biologically plausible recurrent neural network of cortical dynamics to include a basal ganglia-thalamic module which uses reinforcement learning to dynamically modulate action. This "associative cluster-dependent chain" (ACDC) model modularly stores sequence and timing information in distinct loci of the network. This feature increases computational power and allows ACDC to display a wide range of temporal properties (e.g., multiple sequences, temporal shifting, rescaling, and compositionality), while still accounting for several behavioral and neurophysiological empirical observations. Finally, we apply this ACDC network to show how it can learn the famous "Thunderstruck" song intro and then flexibly play it in a "bossa nova" rhythm without further training.