Listening to the Data: Computational Approaches to Addiction and Learning.

Computational approaches hold great promise for identifying novel treatment targets and creating translational therapeutics for substance use disorders. From circuitries underlying decision-making to computationally derived neural markers of drug-cue reactivity, this review is a summary of the approaches to data presented at our 2023 Society for Neuroscience Mini-Symposium. Here, we highlight data- and hypothesis-driven computational approaches that recently afforded advancements in addiction and learning neuroscience. First, we discuss the value of hypothesis-driven algorithmic modeling approaches, which integrate behavioral, neural, and cognitive outputs to refine hypothesis testing. Then, we review the advantages of data-driven dimensionality reduction and machine learning methods for uncovering novel predictor variables and elucidating relationships in high-dimensional data. Overall, this review highlights recent breakthroughs in cognitive mapping, model-based analysis of behavior/risky decision-making, patterns of drug taking, relapse, and neuromarker discovery, and showcases the benefits of novel modeling techniques, across both preclinical and clinical data.

Similar network compositions, but distinct neural dynamics underlying belief updating in environments with and without explicit outcomes.

Classic decision theories typically assume the presence of explicit value-based outcomes after action selections to update beliefs about action-outcome contingencies. However, ecological environments are often opaque, and it remains unclear whether the neural dynamics underlying belief updating vary under conditions characterized by the presence or absence of such explicit value-based information, after each choice selection. We investigated this question in healthy humans (n = 28) using Bayesian inference and two multi-option fMRI tasks: a multi-armed bandit task, and a probabilistic perceptual task, respectively with and without explicit value-based feedback after choice selections. Model-based fMRI analysis revealed a network encoding belief updating which did not change depending on the task. More precisely, we found a confidence-building network that included anterior hippocampus, amygdala, and medial prefrontal cortex (mPFC), which became more active as beliefs about action-outcome probabilities were confirmed by newly acquired information. Despite these consistent responses across tasks, dynamic causal modeling estimated that the network dynamics changed depending on the presence or absence of trial-by-trial value-based outcomes. In the task deprived of immediate feedback, the hippocampus increased its influence towards both amygdala and mPFC, in association with increased strength in the confidence signal. However, the opposite causal relations were found (i.e., from both mPFC and amygdala towards the hippocampus), in presence of immediate outcomes. This finding revealed an asymmetric relationship between decision confidence computations, which were based on similar computational models across tasks, and neural implementation, which varied depending on the availability of outcomes after choice selections.

A change of mind: Globus pallidus activity and effective connectivity during changes in choice selections.

The basal ganglia are a group of interconnected subcortical nuclei that plays a key role in multiple motor and cognitive processes, in a close interplay with several cortical regions. Two conflicting theories postulate that the basal ganglia pathways can either foster or suppress the cortico-striatal output or, alternatively, they can stabilize or destabilize the cortico-striatal circuit dynamics. These different approaches significantly impact the understanding of observable behaviours and cognitive processes in healthy, as well as clinical populations. We investigated the predictions of these models in healthy participants (N = 28), using dynamic causal modeling of fMRI BOLD activity to estimate time- and context-dependent changes in the indirect pathway effective connectivity, in association with repetitions or changes of choice selections. We used two multi-option tasks that required the participants to adapt to uncontrollable environmental changes, by performing sequential choice selections, with and without value-based feedbacks. We found that, irrespective of the task, the trials that were characterized by changes in choice selections (switch trials) were associated with a neural response that mostly overlapped with a network commonly described for the encoding of uncertainty. More interestingly, dynamic causal modeling and family-wise model comparison identified with high likelihood a directed causal relation from the external to the internal part of the globus pallidus (i.e., the short indirect pathway in the basal ganglia), in association with the switch trials. This finding supports the hypothesis that the short indirect pathway in the basal ganglia drives instability in the network dynamics, resulting in changes in choice selection.

An insula-driven network computes decision uncertainty and promotes abstinence in chronic cocaine users.

The anterior insular cortex (AIC) and its interconnected brain regions have been associated with both addiction and decision-making under uncertainty. However, the causal interactions in this uncertainty-encoding neurocircuitry and how these neural dynamics impact relapse remain elusive. Here, we used model-based fMRI to measure choice uncertainty in a motor decision task in 61 individuals with cocaine use disorder (CUD) and 25 healthy controls. CUD participants were assessed before discharge from a residential treatment program and followed for up to 24 weeks. We found that choice uncertainty was tracked by the AIC, dorsal anterior cingulate cortex (dACC) and ventral striatum (VS), across participants. Stronger activations in these regions measured pre-discharge predicted longer abstinence after discharge in individuals with CUD. Dynamic causal modeling revealed an AIC-to-dACC-directed connectivity modulated by uncertainty in controls, but a dACC-to-AIC connectivity in CUD participants. This reversal was mostly driven by early relapsers (<30 days). Furthermore, CUD individuals who displayed a stronger AIC-to-dACC excitatory connection during uncertainty encoding remained abstinent for longer periods. These findings reveal a critical role of an AIC-driven, uncertainty-encoding neurocircuitry in protecting against relapse and promoting abstinence.

Value encoding in the globus pallidus: fMRI reveals an interaction effect between reward and dopamine drive.

The external part of the globus pallidus (GPe) is a core nucleus of the basal ganglia (BG) whose activity is disrupted under conditions of low dopamine release, as in Parkinson's disease. Current models assume decreased dopamine release in the dorsal striatum results in deactivation of dorsal GPe, which in turn affects motor expression via a regulatory effect on other nuclei of the BG. However, recent studies in healthy and pathological animal models have reported neural dynamics that do not match with this view of the GPe as a relay in the BG circuit. Thus, the computational role of the GPe in the BG is still to be determined. We previously proposed a neural model that revisits the functions of the nuclei of the BG, and this model predicts that GPe encodes values which are amplified under a condition of low striatal dopaminergic drive. To test this prediction, we used an fMRI paradigm involving a within-subject placebo-controlled design, using the dopamine antagonist risperidone, wherein healthy volunteers performed a motor selection and maintenance task under low and high reward conditions. ROI-based fMRI analysis revealed an interaction between reward and dopamine drive manipulations, with increased BOLD activity in GPe in a high compared to low reward condition, and under risperidone compared to placebo. These results confirm the core prediction of our computational model, and provide a new perspective on neural dynamics in the BG and their effects on motor selection and cognitive disorders.

Towards a neurocomputational account of social controllability: From models to mental health.

Controllability, or the influence one has over their surroundings, is crucial for decision-making and mental health. Traditionally, controllability is operationalized in sensorimotor terms as one's ability to exercise their actions to achieve an intended outcome (also termed "agency"). However, recent social neuroscience research suggests that humans also assess if and how they can exert influence over other people (i.e., their actions, outcomes, beliefs) to achieve desired outcomes ("social controllability"). In this review, we will synthesize empirical findings and neurocomputational frameworks related to social controllability. We first introduce the concepts of contextual and perceived controllability and their respective relevance for decision-making. Then, we outline neurocomputational frameworks that can be used to model social controllability, with a focus on behavioral economic paradigms and reinforcement learning approaches. Finally, we discuss the implications of social controllability for computational psychiatry research, using delusion and obsession-compulsion as examples. Taken together, we propose that social controllability could be a key area of investigation in future social neuroscience and computational psychiatry research.

Computational models of behavioral addictions: State of the art and future directions.

Non-pharmacological behavioral addictions, such as pathological gambling, videogaming, social networking, or internet use, are becoming major public health concerns. It is not yet clear how behavioral addictions could share many major neurobiological and behavioral characteristics with substance use disorders, despite the absence of direct pharmacological influences. A deeper understanding of the neurocognitive mechanisms of addictive behavior is needed, and computational modeling could be one promising approach to explain intricately entwined cognitive and neural dynamics. This review describes computational models of addiction based on reinforcement learning algorithms, Bayesian inference, and biophysical neural simulations. We discuss whether computational frameworks originally conceived to explain maladaptive behavior in substance use disorders can be effectively extended to non-substance-related behavioral addictions. Moreover, we introduce recent studies on behavioral addictions that exemplify the possibility of such extension and propose future directions.

Testing hypotheses about the harm that capitalism causes to the mind and brain: a theoretical framework for neuroscience research.

In this paper, we will attempt to outline the key ideas of a theoretical framework for neuroscience research that reflects critically on the neoliberal capitalist context. We argue that neuroscience can and should illuminate the effects of neoliberal capitalism on the brains and minds of the population living under such socioeconomic systems. Firstly, we review the available empirical research indicating that the socio-economic environment is harmful to minds and brains. We, then, describe the effects of the capitalist context on neuroscience itself by presenting how it has been influenced historically. In order to set out a theoretical framework that can generate neuroscientific hypotheses with regards to the effects of the capitalist context on brains and minds, we suggest a categorization of the effects, namely deprivation, isolation and intersectional effects. We also argue in favor of a neurodiversity perspective [as opposed to the dominant model of conceptualizing neural (mal-)functioning] and for a perspective that takes into account brain plasticity and potential for change and adaptation. Lastly, we discuss the specific needs for future research as well as a frame for post-capitalist research.

Impaired belief updating and devaluation in adult women with bulimia nervosa.

Recent models of bulimia nervosa (BN) propose that binge-purge episodes ultimately become automatic in response to cues and insensitive to negative outcomes. Here, we examined whether women with BN show alterations in instrumental learning and devaluation sensitivity using traditional and computational modeling analyses of behavioral data. Adult women with BN (n = 30) and group-matched healthy controls (n = 31) completed a task in which they first learned stimulus-response-outcome associations. Then, participants were required to repeatedly adjust their responses in a "baseline test", when different sets of stimuli were explicitly devalued, and in a "slips-of-action test", when outcomes instead of stimuli were devalued. The BN group showed intact behavioral sensitivity to outcome devaluation during the slips-of-action test, but showed difficulty overriding previously learned stimulus-response associations on the baseline test. Results from a Bayesian learner model indicated that this impaired performance could be accounted for by a slower pace of belief updating when a new set of previously learned responses had to be inhibited (p = 0.036). Worse performance and a slower belief update in the baseline test were each associated with more frequent binge eating (p = 0.012) and purging (p = 0.002). Our findings suggest that BN diagnosis and severity are associated with deficits in flexibly updating beliefs to withhold previously learned responses to cues. Additional research is needed to determine whether this impaired ability to adjust behavior is responsible for maintaining automatic and persistent binge eating and purging in response to internal and environmental cues.

An Interpretable and Predictive Connectivity-Based Neural Signature for Chronic Cannabis Use.

BACKGROUND: Cannabis is one of the most widely used substances in the world, with usage trending upward in recent years. However, although the psychiatric burden associated with maladaptive cannabis use has been well established, reliable and interpretable biomarkers associated with chronic use remain elusive. In this study, we combine large-scale functional magnetic resonance imaging with machine learning and network analysis and develop an interpretable decoding model that offers both accurate prediction and novel insights into chronic cannabis use. METHODS: Chronic cannabis users (n = 166) and nonusing healthy control subjects (n = 124) completed a cue-elicited craving task during functional magnetic resonance imaging. Linear machine learning methods were used to classify individuals into chronic users and nonusers based on whole-brain functional connectivity. Network analysis was used to identify the most predictive regions and communities. RESULTS: We obtained high (∼80% out-of-sample) accuracy across 4 different classification models, demonstrating that task-evoked connectivity can successfully differentiate chronic cannabis users from nonusers. We also identified key predictive regions implicating motor, sensory, attention, and craving-related areas, as well as a core set of brain networks that contributed to successful classification. The most predictive networks also strongly correlated with cannabis craving within the chronic user group. CONCLUSIONS: This novel approach produced a neural signature of chronic cannabis use that is both accurate in terms of out-of-sample prediction and interpretable in terms of predictive networks and their relation to cannabis craving.

Computational mechanisms underlying illusion of control in delusional individuals.

Humans navigate complex situations that require the accurate estimation of the controllability of the environment. Aberrant controllability computation might lead to maladaptive behaviors and poor mental health outcomes. Illusion of control, which refers to a heightened sense of control while the environment is uncontrollable, is one such manifestation and has been conceptually associated with delusional ideation. Nevertheless, this association has not yet been formally characterized in a computational framework. To address this, we used a computational psychiatry approach to quantify illusion of control in human participants with high (n = 125) or low (n = 126) trait delusion. Participants played a two-party exchange game in which their choices either did ("Controllable condition") or did not ("Uncontrollable condition") influence the future monetary offers made by simulated partners. We found that the two groups behaved similarly in model-agnostic measures (i.e., offer size, rejection rate). However, computational modeling revealed that compared to the low trait delusion group, the high delusion group overestimated their influence ("expected influence" parameter) over the offers made by their partners under the Uncontrollable condition. Highly delusional individuals also reported a stronger sense of control than those with low trait delusion in the Uncontrollable condition. Furthermore, the expected influence parameter and self-reported beliefs about controllability were significantly correlated in the Controllable condition in individuals with low trait delusion, whereas this relationship was diminished in those with high trait delusion. Collectively, these findings demonstrate that delusional ideation is associated with aberrant computation of and belief about environmental controllability, as well as a belief-behavior disconnect.

Humans use forward thinking to exploit social controllability.

The controllability of our social environment has a profound impact on our behavior and mental health. Nevertheless, neurocomputational mechanisms underlying social controllability remain elusive. Here, 48 participants performed a task where their current choices either did (Controllable), or did not (Uncontrollable), influence partners' future proposals. Computational modeling revealed that people engaged a mental model of forward thinking (FT; i.e., calculating the downstream effects of current actions) to estimate social controllability in both Controllable and Uncontrollable conditions. A large-scale online replication study (n=1342) supported this finding. Using functional magnetic resonance imaging (n=48), we further demonstrated that the ventromedial prefrontal cortex (vmPFC) computed the projected total values of current actions during forward planning, supporting the neural realization of the forward-thinking model. These findings demonstrate that humans use vmPFC-dependent FT to estimate and exploit social controllability, expanding the role of this neurocomputational mechanism beyond spatial and cognitive contexts.

Emotional adaptation during a crisis: decline in anxiety and depression after the initial weeks of COVID-19 in the United States.

Crises such as the COVID-19 pandemic are known to exacerbate depression and anxiety, though their temporal trajectories remain under-investigated. The present study aims to investigate fluctuations in depression and anxiety using the COVID-19 pandemic as a model crisis. A total of 1512 adults living in the United States enrolled in this online study beginning April 2, 2020 and were assessed weekly for 10 weeks (until June 4, 2020). We measured depression and anxiety using the Zung Self-Rating Depression scale and State-Trait Anxiety Inventory (state subscale), respectively, along with demographic and COVID-related surveys. Linear mixed-effects models were used to examine factors contributing to longitudinal changes in depression and anxiety. We found that depression and anxiety levels were high in early April, but declined over time. Being female, younger age, lower-income, and previous psychiatric diagnosis correlated with higher overall levels of anxiety and depression; being married additionally correlated with lower overall levels of depression, but not anxiety. Importantly, worsening of COVID-related economic impact and increase in projected pandemic duration exacerbated both depression and anxiety over time. Finally, increasing levels of informedness correlated with decreasing levels of depression, while increased COVID-19 severity (i.e., 7-day change in cases) and social media use were positively associated with anxiety over time. These findings not only provide evidence for overall emotional adaptation during the initial weeks of the pandemic, but also provide insight into overlapping, yet distinct, factors contributing to depression and anxiety throughout the first wave of the pandemic.

Containment of COVID-19: Simulating the impact of different policies and testing capacities for contact tracing, testing, and isolation.

Efficient contact tracing and testing are fundamental tools to contain the transmission of SARS-CoV-2. We used multi-agent simulations to estimate the daily testing capacity required to find and isolate a number of infected agents sufficient to break the chain of transmission of SARS-CoV-2, so decreasing the risk of new waves of infections. Depending on the non-pharmaceutical mitigation policies in place, the size of secondary infection clusters allowed or the percentage of asymptomatic and paucisymptomatic (i.e., subclinical) infections, we estimated that the daily testing capacity required to contain the disease varies between 0.7 and 9.1 tests per thousand agents in the population. However, we also found that if contact tracing and testing efficacy dropped below 60% (e.g. due to false negatives or reduced tracing capability), the number of new daily infections did not always decrease and could even increase exponentially, irrespective of the testing capacity. Under these conditions, we show that population-level information about geographical distribution and travel behaviour could inform sampling policies to aid a successful containment, while avoiding concerns about government-controlled mass surveillance.

Containment of future waves of COVID-19: simulating the impact of different policies and testing capacities for contact tracing, testing, and isolation.

We used multi-agent simulations to estimate the testing capacity required to find and isolate a number of infections sufficient to break the chain of transmission of SARS-CoV-2. Depending on the mitigation policies in place, a daily capacity between 0.7 to 3.6 tests per thousand was required to contain the disease. However, if contact tracing and testing efficacy dropped below 60% (e.g. due to false negatives or reduced tracing capability), the number of infections kept growing exponentially, irrespective of any testing capacity. Under these conditions, the population's geographical distribution and travel behaviour could inform sampling policies to aid a successful containment.

Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality.

Several decision-making vulnerabilities have been identified as underlying causes for addictive behaviours, or the repeated execution of stereotyped actions despite their adverse consequences. These vulnerabilities are mostly associated with brain alterations caused by the consumption of substances of abuse. However, addiction can also happen in the absence of a pharmacological component, such as seen in pathological gambling and videogaming. We use a new reinforcement learning model to highlight a previously neglected vulnerability that we suggest interacts with those already identified, whilst playing a prominent role in non-pharmacological forms of addiction. Specifically, we show that a dual-learning system (i.e. combining model-based and model-free) can be vulnerable to highly rewarding, but suboptimal actions, that are followed by a complex ramification of stochastic adverse effects. This phenomenon is caused by the overload of the capabilities of an agent, as time and cognitive resources required for exploration, deliberation, situation recognition, and habit formation, all increase as a function of the depth and richness of detail of an environment. Furthermore, the cognitive overload can be aggravated due to alterations (e.g. caused by stress) in the bounded rationality, i.e. the limited amount of resources available for the model-based component, in turn increasing the agent's chances to develop or maintain addictive behaviours. Our study demonstrates that, independent of drug consumption, addictive behaviours can arise in the interaction between the environmental complexity and the biologically finite resources available to explore and represent it.

A Multilevel Computational Characterization of Endophenotypes in Addiction.

Addiction is characterized by a profound intersubject (phenotypic) variability in the expression of addictive symptomatology and propensity to relapse following treatment. However, laboratory investigations have primarily focused on common neural substrates in addiction and have not yet been able to identify mechanisms that can account for the multifaceted phenotypic behaviors reported in the literature. To fill this knowledge gap theoretically, here we simulated phenotypic variations in addiction symptomology and responses to putative treatments, using both a neural model, based on cortico-striatal circuit dynamics, and an algorithmic model of reinforcement learning (RL). These simulations rely on the widely accepted assumption that both the ventral, model-based, goal-directed system and the dorsal, model-free, habitual system are vulnerable to extra-physiologic dopamine reinforcements triggered by addictive rewards. We found that endophenotypic differences in the balance between the two circuit or control systems resulted in an inverted-U shape in optimal choice behavior. Specifically, greater unbalance led to a higher likelihood of developing addiction and more severe drug-taking behaviors. Furthermore, endophenotypes with opposite asymmetrical biases among cortico-striatal circuits expressed similar addiction behaviors, but responded differently to simulated treatments, suggesting personalized treatment development could rely on endophenotypic rather than phenotypic differentiations. We propose our simulated results, confirmed across neural and algorithmic levels of analysis, inform on a fundamental and, to date, neglected quantitative method to characterize clinical heterogeneity in addiction.

In silico Interrogation of Insect Central Complex Suggests Computational Roles for the Ellipsoid Body in Spatial Navigation.

The central complex in the insect brain is a composite of midline neuropils involved in processing sensory cues and mediating behavioral outputs to orchestrate spatial navigation. Despite recent advances, however, the neural mechanisms underlying sensory integration and motor action selections have remained largely elusive. In particular, it is not yet understood how the central complex exploits sensory inputs to realize motor functions associated with spatial navigation. Here we report an in silico interrogation of central complex-mediated spatial navigation with a special emphasis on the ellipsoid body. Based on known connectivity and function, we developed a computational model to test how the local connectome of the central complex can mediate sensorimotor integration to guide different forms of behavioral outputs. Our simulations show integration of multiple sensory sources can be effectively performed in the ellipsoid body. This processed information is used to trigger continuous sequences of action selections resulting in self-motion, obstacle avoidance and the navigation of simulated environments of varying complexity. The motor responses to perceived sensory stimuli can be stored in the neural structure of the central complex to simulate navigation relying on a collective of guidance cues, akin to sensory-driven innate or habitual behaviors. By comparing behaviors under different conditions of accessible sources of input information, we show the simulated insect computes visual inputs and body posture to estimate its position in space. Finally, we tested whether the local connectome of the central complex might also allow the flexibility required to recall an intentional behavioral sequence, among different courses of actions. Our simulations suggest that the central complex can encode combined representations of motor and spatial information to pursue a goal and thus successfully guide orientation behavior. Together, the observed computational features identify central complex circuitry, and especially the ellipsoid body, as a key neural correlate involved in spatial navigation.

Computational Psychiatry of ADHD: Neural Gain Impairments across Marrian Levels of Analysis.

Attention-deficit hyperactivity disorder (ADHD), one of the most common psychiatric disorders, is characterised by unstable response patterns across multiple cognitive domains. However, the neural mechanisms that explain these characteristic features remain unclear. Using a computational multilevel approach, we propose that ADHD is caused by impaired gain modulation in systems that generate this phenotypic increased behavioural variability. Using Marr's three levels of analysis as a heuristic framework, we focus on this variable behaviour, detail how it can be explained algorithmically, and how it might be implemented at a neural level through catecholamine influences on corticostriatal loops. This computational, multilevel, approach to ADHD provides a framework for bridging gaps between descriptions of neuronal activity and behaviour, and provides testable predictions about impaired mechanisms.