Resource-rational psychopathology.

Psychopathology is vast and diverse. Across distinct disease states, individuals exhibit symptoms that appear counter to the standard view of rationality (expected utility maximization). We argue that some aspects of psychopathology can be described as resource-rational, reflecting a rational trade-off between reward and cognitive resources. We review work on two theories of this kind: rational inattention, where a capacity limit applies to perceptual channels, and policy compression, where the capacity limit applies to action channels. We show how these theories can parsimoniously explain many forms of psychopathology, including affective, primary psychotic, and neurodevelopmental disorders, as well as many effects of psychoactive medications on these disorders. While there are important disorder-specific differences and the theories are by no means universal, we argue that resource rationality offers a useful new perspective on psychopathology. By emphasizing the role of cognitive resource constraints, this approach offers a more inclusive picture of rationality. Some aspects of psychopathology may reflect rational trade-offs rather than the breakdown of rationality. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A Bayesian Reanalysis of a Trial of Psilocybin versus Escitalopram for Depression.

Objectives: To perform a Bayesian reanalysis of a recent trial of psilocybin (COMP360) versus escitalopram for Major Depressive Disorder (MDD) in order to provide a more informative interpretation of the indeterminate outcome of a previous frequentist analysis. Design: Reanalysis of a two-arm double-blind placebo controlled trial. Participants: Fifty-nine patients with MDD. Interventions: Two doses of psilocybin 25mg and daily oral placebo versus daily escitalopram and 2 doses of psilocybin 1mg, with psychological support for both groups. Outcome measures: Quick Inventory of Depressive Symptomatology-Self-Report (QIDS SR-16), and three other depression scales as secondary outcomes: HAMD-17, MADRS, and BDI-1A. Results: Using Bayes factors and 'skeptical priors' which bias estimates towards zero, for the hypothesis that psilocybin is superior by any margin, we found indeterminate evidence for QIDS SR-16, strong evidence for BDI-1A and MADRS, and extremely strong evidence for HAMD-17. For the stronger hypothesis that psilocybin is superior by a 'clinically meaningful amount' (using literature defined values of the minimally clinically important difference), we found moderate evidence against it for QIDS SR-16, indeterminate evidence for BDI-1A and MADRS, and moderate evidence supporting it for HAMD-17. Furthermore, across the board we found extremely strong evidence for psilocybin's non-inferiority versus escitalopram. These findings were robust to prior sensitivity analysis. Conclusions: This Bayesian reanalysis supports the following inferences: 1) that psilocybin did indeed outperform escitalopram in this trial, but not to an extent that was clinically meaningful--and 2) that psilocybin is almost certainly non-inferior to escitalopram. The present results provide a more precise and nuanced interpretation to previously reported results from this trial, and support the need for further research into the relative efficacy of psilocybin therapy for depression with respect to current leading treatments. Trial registration number: NCT03429075.

Mild Vitamin C Deficiency Is Common in the Inpatient Psychiatric Setting.

Objective: Mild vitamin C deficiency is a psychiatrically relevant nutritional state, with symptoms including apathy, fatigue, and low mood. Although complete vitamin C deficiency has largely been eradicated, mild deficiency remains common in certain populations. Here, we aimed to identify the prevalence of mild vitamin C deficiency in the inpatient psychiatric setting.Methods: We identified 221 patients with plasma vitamin C levels collected on an inpatient psychiatric unit serving a metropolitan area between January 1, 2015, and March 7, 2022. We identified demographic (age, sex, race, housing status, Area Deprivation Index [an index of neighborhood disadvantage]), substance use (tobacco use, alcohol use), diagnostic (depressive, bipolar, psychotic, anxiety, substance use, catatonia, neurocognitive, autism spectrum), and micronutrient (folate, vitamin B12, vitamin D) risk factors. DSM-5-TR was used as the diagnostic system. Bayesian log-normal regressions were constructed to predict vitamin C as a function of these risk factors. We used these same models to predict vitamin C as a function of significant risk factors.Results: We found that 64% (141 of 221; 95% confidence interval 57%-70%) of patients met criteria for mild vitamin C deficiency. While we did not identify robust demographic, substance use, or diagnostic-based risk factors, we found that folate and vitamin D strongly predicted vitamin C levels. To test the utility of these predictors, we simulated vitamin C as a function of folate and vitamin D and found that predicted deficiency remained high (∼ 50%-55%), even when folate/vitamin D were sufficiently replete.Conclusions: We find that vitamin C deficiency is highly prevalent in the inpatient psychiatric setting and remains high even when the relevant risk factor profile is favorable.

Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys.

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

Undermatching Is a Consequence of Policy Compression.

The matching law describes the tendency of agents to match the ratio of choices allocated to the ratio of rewards received when choosing among multiple options (Herrnstein, 1961). Perfect matching, however, is infrequently observed. Instead, agents tend to undermatch or bias choices toward the poorer option. Overmatching, or the tendency to bias choices toward the richer option, is rarely observed. Despite the ubiquity of undermatching, it has received an inadequate normative justification. Here, we assume agents not only seek to maximize reward, but also seek to minimize cognitive cost, which we formalize as policy complexity (the mutual information between actions and states of the environment). Policy complexity measures the extent to which the policy of an agent is state dependent. Our theory states that capacity-constrained agents (i.e., agents that must compress their policies to reduce complexity) can only undermatch or perfectly match, but not overmatch, consistent with the empirical evidence. Moreover, using mouse behavioral data (male), we validate a novel prediction about which task conditions exaggerate undermatching. Finally, in patients with Parkinson's disease (male and female), we argue that a reduction in undermatching with higher dopamine levels is consistent with an increased policy complexity.SIGNIFICANCE STATEMENT The matching law describes the tendency of agents to match the ratio of choices allocated to different options to the ratio of reward received. For example, if option a yields twice as much reward as option b, matching states that agents will choose option a twice as much. However, agents typically undermatch: they choose the poorer option more frequently than expected. Here, we assume that agents seek to simultaneously maximize reward and minimize the complexity of their action policies. We show that this theory explains when and why undermatching occurs. Neurally, we show that policy complexity, and by extension undermatching, is controlled by tonic dopamine, consistent with other evidence that dopamine plays an important role in cognitive resource allocation.

Reinforcement learning modeling reveals a reward-history-dependent strategy underlying reversal learning in squirrel monkeys.

Insight into psychiatric disease and development of therapeutics relies on behavioral tasks that study similar cognitive constructs in multiple species. The reversal learning task is one popular paradigm that probes flexible behavior, aberrations of which are thought to be important in a number of disease states. Despite widespread use, there is a need for a high-throughput primate model that can bridge the genetic, anatomic, and behavioral gap between rodents and humans. Here, we trained squirrel monkeys, a promising preclinical model, on an image-guided deterministic reversal learning task. We found that squirrel monkeys exhibited two key hallmarks of behavior found in other species: integration of reward history over many trials and a side-specific bias. We adapted a reinforcement learning model and demonstrated that it could simulate squirrel monkey-like behavior, capture training-related trajectories, and provide insight into the strategies animals employed. These results validate squirrel monkeys as a model in which to study behavioral flexibility. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Serotonin neurons modulate learning rate through uncertainty.

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

Entropy-based metrics for predicting choice behavior based on local response to reward.

For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching behavior. To address this, we developed metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct more accurate models of choice. Together, our entropy-based metrics provide a model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.

Aversive stimuli bias corticothalamic responses to motivationally significant cues.

Making predictions about future rewards or punishments is fundamental to adaptive behavior. These processes are influenced by prior experience. For example, prior exposure to aversive stimuli or stressors changes behavioral responses to negative- and positive-value predictive cues. Here, we demonstrate a role for medial prefrontal cortex (mPFC) neurons projecting to the paraventricular nucleus of the thalamus (PVT; mPFC→PVT) in this process. We found that a history of aversive stimuli negatively biased behavioral responses to motivationally relevant cues in mice and that this negative bias was associated with hyperactivity in mPFC→PVT neurons during exposure to those cues. Furthermore, artificially mimicking this hyperactive response with selective optogenetic excitation of the same pathway recapitulated the negative behavioral bias induced by aversive stimuli, whereas optogenetic inactivation of mPFC→PVT neurons prevented the development of the negative bias. Together, our results highlight how information flow within the mPFC→PVT circuit is critical for making predictions about motivationally-relevant outcomes as a function of prior experience.

Subthreshold basis for reward-predictive persistent activity in mouse prefrontal cortex.

Nervous systems maintain information internally using persistent activity changes. The mechanisms by which this activity arises are incompletely understood. We study prefrontal cortex (PFC) in mice performing behaviors in which stimuli predicted rewards at different delays with different probabilities. We measure membrane potential (Vm) from pyramidal neurons across layers. Reward-predictive persistent firing increases arise due to sustained increases in mean and variance of Vm and are terminated by reward or via centrally generated mechanisms based on reward expectation. Other neurons show persistent decreases in firing rates, maintained by persistent hyperpolarization that is robust to intracellular perturbation. Persistent activity is layer (L)- and cell-type-specific. Neurons with persistent depolarization are primarily located in upper L5, whereas those with persistent hyperpolarization are mostly found in lower L5. L2/3 neurons do not show persistent activity. Thus, reward-predictive persistent activity in PFC is spatially organized and conveys information about internal state via synaptic mechanisms.

Dynamic decision making and value computations in medial frontal cortex.

Dynamic decision making requires an intact medial frontal cortex. Recent work has combined theory and single-neuron measurements in frontal cortex to advance models of decision making. We review behavioral tasks that have been used to study dynamic decision making and algorithmic models of these tasks using reinforcement learning theory. We discuss studies linking neurophysiology and quantitative decision variables. We conclude with hypotheses about the role of other cortical and subcortical structures in dynamic decision making, including ascending neuromodulatory systems.

Locus coeruleus spiking differently correlates with S1 cortex activity and pupil diameter in a tactile detection task.

We examined the relationships between activity in the locus coeruleus (LC), activity in the primary somatosensory cortex (S1), and pupil diameter in mice performing a tactile detection task. While LC spiking consistently preceded S1 membrane potential depolarization and pupil dilation, the correlation between S1 and pupil was more heterogeneous. Furthermore, the relationships between LC, S1, and pupil varied on timescales of sub-seconds to seconds within trials. Our data suggest that pupil diameter can be dissociated from LC spiking and cannot be used as a stationary index of LC activity.

A quantitative reward prediction error signal in the ventral pallidum.

The nervous system is hypothesized to compute reward prediction errors (RPEs) to promote adaptive behavior. Correlates of RPEs have been observed in the midbrain dopamine system, but the extent to which RPE signals exist in other reward-processing regions is less well understood. In the present study, we quantified outcome history-based RPE signals in the ventral pallidum (VP), a basal ganglia region functionally linked to reward-seeking behavior. We trained rats to respond to reward-predicting cues, and we fit computational models to predict the firing rates of individual neurons at the time of reward delivery. We found that a subset of VP neurons encoded RPEs and did so more robustly than the nucleus accumbens, an input to the VP. VP RPEs predicted changes in task engagement, and optogenetic manipulation of the VP during reward delivery bidirectionally altered rats' subsequent reward-seeking behavior. Our data suggest a pivotal role for the VP in computing teaching signals that influence adaptive reward seeking.

Stable Representations of Decision Variables for Flexible Behavior.

Decisions occur in dynamic environments. In the framework of reinforcement learning, the probability of performing an action is influenced by decision variables. Discrepancies between predicted and obtained rewards (reward prediction errors) update these variables, but they are otherwise stable between decisions. Although reward prediction errors have been mapped to midbrain dopamine neurons, it is unclear how the brain represents decision variables themselves. We trained mice on a dynamic foraging task in which they chose between alternatives that delivered reward with changing probabilities. Neurons in the medial prefrontal cortex, including projections to the dorsomedial striatum, maintained persistent firing rate changes over long timescales. These changes stably represented relative action values (to bias choices) and total action values (to bias response times) with slow decay. In contrast, decision variables were weakly represented in the anterolateral motor cortex, a region necessary for generating choices. Thus, we define a stable neural mechanism to drive flexible behavior.

Behavioral and electrophysiological effects of cortical microstimulation parameters.

Electrical microstimulation has been widely used to artificially activate neural circuits on fast time scales. Despite the ubiquity of its use, little is known about precisely how it activates neural pathways. Current is typically delivered to neural tissue in a manner that provides a locally balanced injection of positive and negative charge, resulting in negligible net charge delivery to avoid the neurotoxic effects of charge accumulation. Modeling studies have suggested that the most common approach, using a temporally symmetric current pulse waveform as the base unit of stimulation, results in preferential activation of axons, causing diffuse activation of neurons relative to the stimulation site. Altering waveform shape and using an asymmetric current pulse waveform theoretically reverses this bias and preferentially activates cell bodies, providing increased specificity. In separate studies, measurements of downstream cortical activation from sub-cortical microstimulation are consistent with this hypothesis, as are recent measurements of behavioral detection threshold currents from cortical microstimulation. Here, we compared the behavioral and electrophysiological effects of symmetric vs. asymmetric current waveform shape in cortical microstimulation. Using a go/no-go behavioral task, we found that microstimulation waveform shape significantly shifts psychometric performance, where a larger current pulse was necessary when applying an asymmetric waveform to elicit the same behavioral response, across a large range of behaviorally relevant current amplitudes. Using voltage-sensitive dye imaging of cortex in anesthetized animals with simultaneous cortical microstimulation, we found that altering microstimulation waveform shape shifted the cortical activation in a manner that mirrored the behavioral results. Taken together, these results are consistent with the hypothesis that asymmetric stimulation preferentially activates cell bodies, albeit at a higher threshold, as compared to symmetric stimulation. These findings demonstrate the sensitivity of the pathway to varying electrical stimulation parameters and underscore the importance of designing electrical stimuli for optimal activation of neural circuits.

Detection of tactile inputs in the rat vibrissa pathway.

The rapid detection of sensory inputs is crucial for survival. Sensory detection explicitly requires the integration of incoming sensory information and the ability to distinguish between relevant information and ongoing neural activity. In this study, head-fixed rats were trained to detect the presence of a brief deflection of their whiskers resulting from a focused puff of air. The animals showed a monotonic increase in response probability and a decrease in reaction time with increased stimulus strength. High-speed video analysis of whisker motion revealed that animals were more likely to detect the stimulus during periods of reduced self-induced motion of the whiskers, thereby allowing the stimulus-induced whisker motion to exceed the ongoing noise. In parallel, we used voltage-sensitive dye (VSD) imaging of barrel cortex in anesthetized rats receiving the same stimulus set as those in the behavioral portion of this study to assess candidate codes that make use of the full spatiotemporal representation and to compare variability in the trial-by-trial nature of the cortical response and the corresponding variability in the behavioral response. By application of an accumulating evidence framework to the population cortical activity measured in separate animals, a strong correspondence was made between the behavioral output and the neural signaling, in terms of both the response probabilities and the reaction times. Taken together, the results here provide evidence for detection performance that is strongly reliant on the relative strength of signal versus noise, with strong correspondence between behavior and parallel electrophysiological findings.