Opponent learning with different representations in the cortico-basal ganglia pathways can develop obsession-compulsion cycle.

Obsessive-compulsive disorder (OCD) has been suggested to be associated with impairment of model-based behavioral control. Meanwhile, recent work suggested shorter memory trace for negative than positive prediction errors (PEs) in OCD. We explored relations between these two suggestions through computational modeling. Based on the properties of cortico-basal ganglia pathways, we modeled human as an agent having a combination of successor representation (SR)-based system that enables model-based-like control and individual representation (IR)-based system that only hosts model-free control, with the two systems potentially learning from positive and negative PEs in different rates. We simulated the agent's behavior in the environmental model used in the recent work that describes potential development of obsession-compulsion cycle. We found that the dual-system agent could develop enhanced obsession-compulsion cycle, similarly to the agent having memory trace imbalance in the recent work, if the SR- and IR-based systems learned mainly from positive and negative PEs, respectively. We then simulated the behavior of such an opponent SR+IR agent in the two-stage decision task, in comparison with the agent having only SR-based control. Fitting of the agents' behavior by the model weighing model-based and model-free control developed in the original two-stage task study resulted in smaller weights of model-based control for the opponent SR+IR agent than for the SR-only agent. These results reconcile the previous suggestions about OCD, i.e., impaired model-based control and memory trace imbalance, raising a novel possibility that opponent learning in model(SR)-based and model-free controllers underlies obsession-compulsion. Our model cannot explain the behavior of OCD patients in punishment, rather than reward, contexts, but it could be resolved if opponent SR+IR learning operates also in the recently revealed non-canonical cortico-basal ganglia-dopamine circuit for threat/aversiveness, rather than reward, reinforcement learning, and the aversive SR + appetitive IR agent could actually develop obsession-compulsion if the environment is modeled differently.

Computational models of behavioral addictions: State of the art and future directions.

Non-pharmacological behavioral addictions, such as pathological gambling, videogaming, social networking, or internet use, are becoming major public health concerns. It is not yet clear how behavioral addictions could share many major neurobiological and behavioral characteristics with substance use disorders, despite the absence of direct pharmacological influences. A deeper understanding of the neurocognitive mechanisms of addictive behavior is needed, and computational modeling could be one promising approach to explain intricately entwined cognitive and neural dynamics. This review describes computational models of addiction based on reinforcement learning algorithms, Bayesian inference, and biophysical neural simulations. We discuss whether computational frameworks originally conceived to explain maladaptive behavior in substance use disorders can be effectively extended to non-substance-related behavioral addictions. Moreover, we introduce recent studies on behavioral addictions that exemplify the possibility of such extension and propose future directions.

Opponent Learning with Different Representations in the Cortico-Basal Ganglia Circuits.

The direct and indirect pathways of the basal ganglia (BG) have been suggested to learn mainly from positive and negative feedbacks, respectively. Since these pathways unevenly receive inputs from different cortical neuron types and/or regions, they may preferentially use different state/action representations. We explored whether such a combined use of different representations, coupled with different learning rates from positive and negative reward prediction errors (RPEs), has computational benefits. We modeled animal as an agent equipped with two learning systems, each of which adopted individual representation (IR) or successor representation (SR) of states. With varying the combination of IR or SR and also the learning rates from positive and negative RPEs in each system, we examined how the agent performed in a dynamic reward navigation task. We found that combination of SR-based system learning mainly from positive RPEs and IR-based system learning mainly from negative RPEs could achieve a good performance in the task, as compared with other combinations. In such a combination of appetitive SR-based and aversive IR-based systems, both systems show activities of comparable magnitudes with opposite signs, consistent with the suggested profiles of the two BG pathways. Moreover, the architecture of such a combination provides a novel coherent explanation for the functional significance and underlying mechanism of diverse findings about the cortico-BG circuits. These results suggest that particularly combining different representations with appetitive and aversive learning could be an effective learning strategy in certain dynamic environments, and it might actually be implemented in the cortico-BG circuits.

Long term mental health trajectories after disasters and pandemics: A multilingual systematic review of prevalence, risk and protective factors.

The patterns of long-term psychological response after disasters and pandemics remain unclear. We aimed to determine the trajectories for post-traumatic stress symptoms (PTSS), depression and anxiety prevalence following disasters and pandemic exposure; and identify associated risk and protective factors. A systematic review of the English, Chinese, and Japanese longitudinal mental health literature was conducted. We searched Cochrane, MEDLINE, ProQuest, PsycINFO, PubMed, Web of Science, and CINAHL (English), CNKI and SINOMED (Chinese) and CiNii (Japanese) for studies published between January 2000 and May 2022. Following a pre-specified protocol (PROSPERO: CRD42020206424), conditional linear growth curve models and ANOVA analyses were conducted. The search identified 77,891 papers, with a final sample of 234: 206 English, 24 Chinese, and 4 Japanese-language papers. PTSS rates improved for all ages (p = .018, eta2 = 0.035). In contrast, depression and anxiety prevalence remained elevated for years following exposure (p = .424, eta2 = 0.019 and p = .051, eta2 = 0.064, respectively), with significantly higher rates for children and adolescents (p < .005, eta2 > 0.056). Earthquakes and pandemics were associated with higher prevalence of PTSS (p < .019, eta2 > 0.019). Multi-level risk and protective factors were identified. The chronicity of mental health outcomes highlights a critical need for tailored, sustainable mental health services, particularly for children and adolescents, in disaster- and pandemic-affected settings.

Rigid reduced successor representation as a potential mechanism for addiction.

Difficulty in cessation of drinking, smoking, or gambling has been widely recognized. Conventional theories proposed relative dominance of habitual over goal-directed control, but human studies have not convincingly supported them. Referring to the recently suggested "successor representation (SR)" of states that enables partially goal-directed control, we propose a dopamine-related mechanism that makes resistance to habitual reward-obtaining particularly difficult. We considered that long-standing behavior towards a certain reward without resisting temptation can (but not always) lead to a formation of rigid dimension-reduced SR based on the goal state, which cannot be updated. Then, in our model assuming such rigid reduced SR, whereas no reward prediction error (RPE) is generated at the goal while no resistance is made, a sustained large positive RPE is generated upon goal reaching once the person starts resisting temptation. Such sustained RPE is somewhat similar to the hypothesized sustained fictitious RPE caused by drug-induced dopamine. In contrast, if rigid reduced SR is not formed and states are represented individually as in simple reinforcement learning models, no sustained RPE is generated at the goal. Formation of rigid reduced SR also attenuates the resistance-dependent decrease in the value of the cue for behavior, makes subsequent introduction of punishment after the goal ineffective, and potentially enhances the propensity of nonresistance through the influence of RPEs via the spiral striatum-midbrain circuit. These results suggest that formation of rigid reduced SR makes cessation of habitual reward-obtaining particularly difficult and can thus be a mechanism for addiction, common to substance and nonsubstance reward.