People construct simplified mental representations to plan.

One of the most striking features of human cognition is the ability to plan. Two aspects of human planning stand out-its efficiency and flexibility. Efficiency is especially impressive because plans must often be made in complex environments, and yet people successfully plan solutions to many everyday problems despite having limited cognitive resources1-3. Standard accounts in psychology, economics and artificial intelligence have suggested that human planning succeeds because people have a complete representation of a task and then use heuristics to plan future actions in that representation4-11. However, this approach generally assumes that task representations are fixed. Here we propose that task representations can be controlled and that such control provides opportunities to quickly simplify problems and more easily reason about them. We propose a computational account of this simplification process and, in a series of preregistered behavioural experiments, show that it is subject to online cognitive control12-14 and that people optimally balance the complexity of a task representation and its utility for planning and acting. These results demonstrate how strategically perceiving and conceiving problems facilitates the effective use of limited cognitive resources.

Having multiple selves helps learning agents explore and adapt in complex changing worlds.

Satisfying a variety of conflicting needs in a changing environment is a fundamental challenge for any adaptive agent. Here, we show that designing an agent in a modular fashion as a collection of subagents, each dedicated to a separate need, powerfully enhanced the agent's capacity to satisfy its overall needs. We used the formalism of deep reinforcement learning to investigate a biologically relevant multiobjective task: continually maintaining homeostasis of a set of physiologic variables. We then conducted simulations in a variety of environments and compared how modular agents performed relative to standard monolithic agents (i.e., agents that aimed to satisfy all needs in an integrated manner using a single aggregate measure of success). Simulations revealed that modular agents a) exhibited a form of exploration that was intrinsic and emergent rather than extrinsically imposed; b) were robust to changes in nonstationary environments, and c) scaled gracefully in their ability to maintain homeostasis as the number of conflicting objectives increased. Supporting analysis suggested that the robustness to changing environments and increasing numbers of needs were due to intrinsic exploration and efficiency of representation afforded by the modular architecture. These results suggest that the normative principles by which agents have adapted to complex changing environments may also explain why humans have long been described as consisting of "multiple selves."

Effort Foraging Task reveals positive correlation between individual differences in the cost of cognitive and physical effort in humans.

Effort-based decisions, in which people weigh potential future rewards against effort costs required to achieve those rewards involve both cognitive and physical effort, though the mechanistic relationship between them is not yet understood. Here, we use an individual differences approach to isolate and measure the computational processes underlying effort-based decisions and test the association between cognitive and physical domains. Patch foraging is an ecologically valid reward rate maximization problem with well-developed theoretical tools. We developed the Effort Foraging Task, which embedded cognitive or physical effort into patch foraging, to quantify the cost of both cognitive and physical effort indirectly, by their effects on foraging choices. Participants chose between harvesting a depleting patch, or traveling to a new patch that was costly in time and effort. Participants' exit thresholds (reflecting the reward they expected to receive by harvesting when they chose to travel to a new patch) were sensitive to cognitive and physical effort demands, allowing us to quantify the perceived effort cost in monetary terms. The indirect sequential choice style revealed effort-seeking behavior in a minority of participants (preferring high over low effort) that has apparently been missed by many previous approaches. Individual differences in cognitive and physical effort costs were positively correlated, suggesting that these are perceived and processed in common. We used canonical correlation analysis to probe the relationship of task measures to self-reported affect and motivation, and found correlations of cognitive effort with anxiety, cognitive function, behavioral activation, and self-efficacy, but no similar correlations with physical effort.

Toward a Rational and Mechanistic Account of Mental Effort.

In spite of its familiar phenomenology, the mechanistic basis for mental effort remains poorly understood. Although most researchers agree that mental effort is aversive and stems from limitations in our capacity to exercise cognitive control, it is unclear what gives rise to those limitations and why they result in an experience of control as costly. The presence of these control costs also raises further questions regarding how best to allocate mental effort to minimize those costs and maximize the attendant benefits. This review explores recent advances in computational modeling and empirical research aimed at addressing these questions at the level of psychological process and neural mechanism, examining both the limitations to mental effort exertion and how we manage those limited cognitive resources. We conclude by identifying remaining challenges for theoretical accounts of mental effort as well as possible applications of the available findings to understanding the causes of and potential solutions for apparent failures to exert the mental effort required of us.

Disentangling Abstraction from Statistical Pattern Matching in Human and Machine Learning.

The ability to acquire abstract knowledge is a hallmark of human intelligence and is believed by many to be one of the core differences between humans and neural network models. Agents can be endowed with an inductive bias towards abstraction through meta-learning, where they are trained on a distribution of tasks that share some abstract structure that can be learned and applied. However, because neural networks are hard to interpret, it can be difficult to tell whether agents have learned the underlying abstraction, or alternatively statistical patterns that are characteristic of that abstraction. In this work, we compare the performance of humans and agents in a meta-reinforcement learning paradigm in which tasks are generated from abstract rules. We define a novel methodology for building "task metamers" that closely match the statistics of the abstract tasks but use a different underlying generative process, and evaluate performance on both abstract and metamer tasks. We find that humans perform better at abstract tasks than metamer tasks whereas common neural network architectures typically perform worse on the abstract tasks than the matched metamers. This work provides a foundation for characterizing differences between humans and machine learning that can be used in future work towards developing machines with more human-like behavior.

Rat Anterior Cingulate Cortex Continuously Signals Decision Variables in a Patch Foraging Task.

In patch foraging tasks, animals must decide whether to remain with a depleting resource or to leave it in search of a potentially better source of reward. In such tasks, animals consistently follow the general predictions of optimal foraging theory (the marginal value theorem; MVT): to leave a patch when the reward rate in the current patch depletes to the average reward rate across patches. Prior studies implicate an important role for the anterior cingulate cortex (ACC) in foraging decisions based on MVT: within single trials, ACC activity increases immediately preceding foraging decisions, and across trials, these dynamics are modulated as the value of staying in the patch depletes to the average reward rate. Here, we test whether these activity patterns reflect dynamic encoding of decision-variables and whether these signals are directly involved in decision-making. We developed a leaky accumulator model based on the MVT that generates estimates of decision variables within and across trials, and tested model predictions against ACC activity recorded from male rats performing a patch foraging task. Model predicted changes in MVT decision variables closely matched rat ACC activity. Next, we pharmacologically inactivated ACC in male rats to test the contribution of these signals to decision-making. ACC inactivation had a profound effect on rats' foraging decisions and response times (RTs) yet rats still followed the MVT decision rule. These findings indicate that the ACC encodes foraging-related variables for reasons unrelated to patch-leaving decisions.SIGNIFICANCE STATEMENT The ability to make adaptive patch-foraging decisions, to remain with a depleting resource or search for better alternatives, is critical to animal well-being. Previous studies have found that anterior cingulate cortex (ACC) activity is modulated at different points in the foraging decision process, raising questions about whether the ACC guides ongoing decisions or serves a more general purpose of regulating cognitive control. To investigate the function of the ACC in foraging, the present study developed a dynamic model of behavior and neural activity, and tested model predictions using recordings and inactivation of ACC. Findings revealed that ACC continuously signals decision variables but that these signals are more likely used to monitor and regulate ongoing processes than to guide foraging decisions.

Associative memory retrieval modulates upcoming perceptual decisions.

Expectations can inform fast, accurate decisions. But what informs expectations? Here we test the hypothesis that expectations are set by dynamic inference from memory. Participants performed a cue-guided perceptual decision task with independently-varying memory and sensory evidence. Cues established expectations by reminding participants of past stimulus-stimulus pairings, which predicted the likely target in a subsequent noisy image stream. Participant's responses used both memory and sensory information, in accordance to their relative reliability. Formal model comparison showed that the sensory inference was best explained when its parameters were set dynamically at each trial by evidence sampled from memory. Supporting this model, neural pattern analysis revealed that responses to the probe were modulated by the specific content and fidelity of memory reinstatement that occurred before the probe appeared. Together, these results suggest that perceptual decisions arise from the continuous sampling of memory and sensory evidence.

Rational Simplification and Rigidity in Human Planning.

Planning underpins the impressive flexibility of goal-directed behavior. However, even when planning, people can display surprising rigidity in how they think about problems (e.g., "functional fixedness") that lead them astray. How can our capacity for behavioral flexibility be reconciled with our susceptibility to conceptual inflexibility? We propose that these tendencies reflect avoidance of two cognitive costs: the cost of representing task details and the cost of switching between representations. To test this hypothesis, we developed a novel paradigm that affords participants opportunities to choose different families of simplified representations to plan. In two preregistered, online studies (Ns = 377 and 294 adults), we found that participants' optimal behavior, suboptimal behavior, and reaction time were explained by a computational model that formalized people's avoidance of representational complexity and switching. These results demonstrate how the selection of simplified, rigid representations leads to the otherwise puzzling combination of flexibility and inflexibility observed in problem solving.

Outcome of Kidney Transplantation in Israel Following Uncontrolled Donation after Cardiocirculatory Determination of Death.

BACKGROUND: A limited program for kidney donation from uncontrolled donation after cardiocirculatory determination of death (uDCDD) was implemented at four hospitals in Israel in close cooperation with Magen David Adom (MDA), the national emergency medical service. OBJECTIVES: To assess the outcome of transplantations performed between January 2017 and June 2022. METHODS: Donor data included age, sex, and cause of death. Recipient data included age, sex, and yearly serum creatinine levels. A retrospective study of out-of-hospital cardiac arrest cases treated by MDA during 2021 were analyzed to assess their compatibility as potential uDCDD donors. RESULTS: In total, 49 potential donors were referred to hospitals by MDA. Consent was obtained in 40 cases (83%), organ retrieval was performed in 28 cases, and 40 kidneys were transplanted from 21 donors (75% retrieval rate). At 1-year follow-up, 36 recipients had a functioning graft (4 returned to dialysis) and mean serum creatinine 1.59 ± 0.92 mg% (90% graft survival). Outcome after transplantation showed serum creatinine levels (mg%) at 2 years 1.41 ± 0.83, n=26; 3 years 1.48 ± 0.99, n=16; 4 years 1.07 ± 1.06, n=7; and 5 years 1.12 ± 0.31, n=5. One patient died of multiple myeloma at 3 years. The MDA audit revealed an unutilized pool of 125 potential cases, 90 of whom were transported to hospitals and 35 were declared dead at the scene. CONCLUSIONS: Transplant outcomes were encouraging, suggesting that more intensive implementation of the program may increase the number of kidneys transplanted, thus shortening recipient waiting lists.

RT-Cloud: A cloud-based software framework to simplify and standardize real-time fMRI.

Real-time fMRI (RT-fMRI) neurofeedback has been shown to be effective in treating neuropsychiatric disorders and holds tremendous promise for future breakthroughs, both with regard to basic science and clinical applications. However, the prevalence of its use has been hampered by computing hardware requirements, the complexity of setting up and running an experiment, and a lack of standards that would foster collaboration. To address these issues, we have developed RT-Cloud (https://github.com/brainiak/rt-cloud), a flexible, cloud-based, open-source Python software package for the execution of RT-fMRI experiments. RT-Cloud uses standardized data formats and adaptable processing streams to support and expand open science in RT-fMRI research and applications. Cloud computing is a key enabling technology for advancing RT-fMRI because it eliminates the need for on-premise technical expertise and high-performance computing; this allows installation, configuration, and maintenance to be automated and done remotely. Furthermore, the scalability of cloud computing makes it easier to deploy computationally-demanding multivariate analyses in real time. In this paper, we describe how RT-Cloud has been integrated with open standards, including the Brain Imaging Data Structure (BIDS) standard and the OpenNeuro database, how it has been applied thus far, and our plans for further development and deployment of RT-Cloud in the coming years.

Brain Death Determination in Israel: The First 9 Years of Experience Following Changes Made by the Brain and Respiratory Death Determination Law.

BACKGROUND: Changes accommodating requirements of religious authorities in Israel resulted in the Brain and Respiratory Death Determination Law (BRDDL), which came into effect in 2009. These included considering patient wishes regarding the brain respiratory death determination (BRDD), mandatory performance of apnea and ancillary testing, establishment of an accreditation committee, and accreditation required for physicians performing BRDD. OBJECTIVES: To assess the impact of the legislation from 2010-2019. METHODS: Data collected included the number of formal BRDDs and accredited physicians. Obstacles to declaring brain death and interventions applied were identified. RESULTS: Obstacles included lack of trained physicians to perform BRDD and interpret ancillary test results, inability to perform apnea or ancillary testing, and non-approach to next-of-kin objecting to BRDD. Interventions included physician training courses, additional ancillary test options, and legal interpretation of patient wishes for non-determination of BRD. As a result, the number of non-determinations related to next-of-kin objecting decreased (26 in 2010 to 5 in 2019), inability to perform apnea or ancillary testing decreased (33 in 2010 to 2 in 2019), and number of physicians receiving accreditation increased (210 in 2010 to 456 in 2019). Last, the consent rate for organ donation increased from 49% to 60% in 2019. CONCLUSIONS: The initial decrease in BRDDs has reversed, thus enabling more approaches for organ donation. The increased consent rate may reflect in part the support of the rabbinate and confidence of the general public that BRDD is performed and monitored according to strict criteria.

SweetPea: A standard language for factorial experimental design.

Experimental design is a key ingredient of reproducible empirical research. Yet, given the increasing complexity of experimental designs, researchers often struggle to implement ones that allow them to measure their variables of interest without confounds. SweetPea ( https://sweetpea-org.github.io/ ) is an open-source declarative language in Python, in which researchers can describe their desired experiment as a set of factors and constraints. The language leverages advances in areas of computer science to sample experiment sequences in an unbiased way. In this article, we provide an overview of SweetPea's capabilities, and demonstrate its application to the design of psychological experiments. Finally, we discuss current limitations of SweetPea, as well as potential applications to other domains of empirical research, such as neuroscience and machine learning.

Refresh my memory: Episodic memory reinstatements intrude on working memory maintenance.

A fundamental question in memory research is how different forms of memory interact. Previous research has shown that people rely on working memory (WM) in short-term recognition tasks; a common view is that episodic memory (EM) only influences performance on these tasks when WM maintenance is disrupted. However, retrieval of memories from EM has been widely observed during brief periods of quiescence, raising the possibility that EM retrievals during maintenance-critically, before a response can be prepared-might affect short-term recognition memory performance even in the absence of distraction. We hypothesized that this influence would be mediated by the lingering presence of reactivated EM content in WM. We obtained support for this hypothesis in three experiments, showing that delay-period EM reactivation introduces incidentally associated information (context) into WM, and that these retrieved associations negatively impact subsequent recognition, leading to substitution errors (Experiment 1) and slowing of accurate responses (Experiment 2). FMRI pattern analysis showed that slowing is mediated by the content of EM reinstatement (Experiment 3). These results expose a previously hidden influence of EM on WM, raising new questions about the adaptive nature of their interaction.

Noise correlations in the human brain and their impact on pattern classification.

Multivariate decoding methods, such as multivoxel pattern analysis (MVPA), are highly effective at extracting information from brain imaging data. Yet, the precise nature of the information that MVPA draws upon remains controversial. Most current theories emphasize the enhanced sensitivity imparted by aggregating across voxels that have mixed and weak selectivity. However, beyond the selectivity of individual voxels, neural variability is correlated across voxels, and such noise correlations may contribute importantly to accurate decoding. Indeed, a recent computational theory proposed that noise correlations enhance multivariate decoding from heterogeneous neural populations. Here we extend this theory from the scale of neurons to functional magnetic resonance imaging (fMRI) and show that noise correlations between heterogeneous populations of voxels (i.e., voxels selective for different stimulus variables) contribute to the success of MVPA. Specifically, decoding performance is enhanced when voxels with high vs. low noise correlations (measured during rest or in the background of the task) are selected during classifier training. Conversely, voxels that are strongly selective for one class in a GLM or that receive high classification weights in MVPA tend to exhibit high noise correlations with voxels selective for the other class being discriminated against. Furthermore, we use simulations to show that this is a general property of fMRI data and that selectivity and noise correlations can have distinguishable influences on decoding. Taken together, our findings demonstrate that if there is signal in the data, the resulting above-chance classification accuracy is modulated by the magnitude of noise correlations.

Catecholamine-Mediated Increases in Gain Enhance the Precision of Cortical Representations.

UNLABELLED: Neurophysiological evidence suggests that neuromodulators, such as norepinephrine and dopamine, increase neural gain in target brain areas. Computational models and prominent theoretical frameworks indicate that this should enhance the precision of neural representations, but direct empirical evidence for this hypothesis is lacking. In two functional MRI studies, we examine the effect of baseline catecholamine levels (as indexed by pupil diameter and manipulated pharmacologically) on the precision of object representations in the human ventral temporal cortex using angular dispersion, a powerful, multivariate metric of representational similarity (precision). We first report the results of computational model simulations indicating that increasing catecholaminergic gain should reduce the angular dispersion, and thus increase the precision, of object representations from the same category, as well as reduce the angular dispersion of object representations from distinct categories when distinct-category representations overlap. In Study 1 (N = 24), we show that angular dispersion covaries with pupil diameter, an index of baseline catecholamine levels. In Study 2 (N = 24), we manipulate catecholamine levels and neural gain using the norepinephrine transporter blocker atomoxetine and demonstrate consistent, causal effects on angular dispersion and brain-wide functional connectivity. Despite the use of very different methods of examining the effect of baseline catecholamine levels, our results show a striking convergence and demonstrate that catecholamines increase the precision of neural representations. SIGNIFICANCE STATEMENT: Norepinephrine and dopamine are among the most widely distributed and ubiquitous neuromodulators in the mammalian brain and have a profound and pervasive impact on cognition. Baseline catecholamine levels tend to increase with increasing task engagement in tasks involving perceptual decisions, yet there is currently no direct evidence of the specific impact of these increases in catecholamine levels on perceptual encoding. Our results fill this void by showing that catecholamines enhance the precision of encoding cortical object representations, and by suggesting that this effect is mediated by increases in neural gain, thus offering a mechanistic account of our key finding.

The integration of social influence and reward: Computational approaches and neural evidence.

Decades of research have established that decision-making is dramatically impacted by both the rewards an individual receives and the behavior of others. How do these distinct influences exert their influence on an individual's actions, and can the resulting behavior be effectively captured in a computational model? To address this question, we employed a novel spatial foraging game in which groups of three participants sought to find the most rewarding location in an unfamiliar two-dimensional space. As the game transitioned from one block to the next, the availability of information regarding other group members was varied systematically, revealing the relative impacts of feedback from the environment and information from other group members on individual decision-making. Both reward-based and socially-based sources of information exerted a significant influence on behavior, and a computational model incorporating these effects was able to recapitulate several key trends in the behavioral data. In addition, our findings suggest how these sources were processed and combined during decision-making. Analysis of reaction time, location of gaze, and functional magnetic resonance imaging (fMRI) data indicated that these distinct sources of information were integrated simultaneously for each decision, rather than exerting their influence in a separate, all-or-none fashion across separate subsets of trials. These findings add to our understanding of how the separate influences of reward from the environment and information derived from other social agents are combined to produce decisions.

Increased locus coeruleus tonic activity causes disengagement from a patch-foraging task.

High levels of locus coeruleus (LC) tonic activity are associated with distraction and poor performance within a task. Adaptive gain theory (AGT; Aston-Jones & Cohen, 2005) suggests that this may reflect an adaptive function of the LC, encouraging search for more remunerative opportunities in times of low utility. Here, we examine whether stimulating LC tonic activity using designer receptors (DREADDs) promotes searching for better opportunities in a patch-foraging task as the value of a patch diminishes. The task required rats to decide repeatedly whether to exploit an immediate but depleting reward within a patch or to incur the cost of a time delay to travel to a new, fuller patch. Similar to behavior associated with high LC tonic activity in other tasks, we found that stimulating LC tonic activity impaired task performance, resulting in reduced task participation and increased response times and omission rates. However, this was accompanied by a more specific, predicted effect: a significant tendency to leave patches earlier, which was best explained by an increase in decision noise rather than a systematic bias to leave earlier (i.e., at higher values). This effect is consistent with the hypothesis that high LC tonic activity favors disengagement from current behavior, and the pursuit of alternatives, by augmenting processing noise. These results provide direct causal evidence for the relationship between LC tonic activity and flexible task switching proposed by AGT.

More Is Meaningful: The Magnitude Effect in Intertemporal Choice Depends on Self-Control.

Impulsivity is a variable behavioral trait that depends on numerous factors. For example, increasing the absolute magnitude of available choice options promotes farsighted decisions. We argue that this magnitude effect arises in part from differential exertion of self-control as the perceived importance of the choice increases. First, we demonstrated that frontal executive-control areas were more engaged for more difficult decisions and that this effect was enhanced for high-magnitude rewards. Second, we showed that increased hunger, which is associated with lower self-control, reduced the magnitude effect. Third, we tested an intervention designed to increase self-control and showed that it reduced the magnitude effect. Taken together, our findings challenge existing theories about the magnitude effect and suggest that visceral and cognitive factors affecting choice may do so by influencing self-control.

Dorsal anterior cingulate and ventromedial prefrontal cortex have inverse roles in both foraging and economic choice.

Recent research has highlighted a distinction between sequential foraging choices and traditional economic choices between simultaneously presented options. This was partly motivated by observations in Kolling, Behrens, Mars, and Rushworth, Science, 336(6077), 95-98 (2012) (hereafter, KBMR) that these choice types are subserved by different circuits, with dorsal anterior cingulate (dACC) preferentially involved in foraging and ventromedial prefrontal cortex (vmPFC) preferentially involved in economic choice. To support this account, KBMR used fMRI to scan human subjects making either a foraging choice (between exploiting a current offer or swapping for potentially better rewards) or an economic choice (between two reward-probability pairs). This study found that dACC better tracked values pertaining to foraging, whereas vmPFC better tracked values pertaining to economic choice. We recently showed that dACC's role in these foraging choices is better described by the difficulty of choosing than by foraging value, when correcting for choice biases and testing a sufficiently broad set of foraging values (Shenhav, Straccia, Cohen, & Botvinick Nature Neuroscience, 17(9), 1249-1254, 2014). Here, we extend these findings in 3 ways. First, we replicate our original finding with a larger sample and a task modified to address remaining methodological gaps between our previous experiments and that of KBMR. Second, we show that dACC activity is best accounted for by choice difficulty alone (rather than in combination with foraging value) during both foraging and economic choices. Third, we show that patterns of vmPFC activity, inverted relative to dACC, also suggest a common function across both choice types. Overall, we conclude that both regions are similarly engaged by foraging-like and economic choice.

Do You See the Forest or the Tree? Neural Gain and Breadth Versus Focus in Perceptual Processing.

When perceiving rich sensory information, some people may integrate its various aspects, whereas other people may selectively focus on its most salient aspects. We propose that neural gain modulates the trade-off between breadth and selectivity, such that high gain focuses perception on those aspects of the information that have the strongest, most immediate influence, whereas low gain allows broader integration of different aspects. We illustrate our hypothesis using a neural-network model of ambiguous-letter perception. We then report an experiment demonstrating that, as predicted by the model, pupil-diameter indices of higher gain are associated with letter perception that is more selectively focused on the letter's shape or, if primed, its semantic content. Finally, we report a recognition-memory experiment showing that the relationship between gain and selective processing also applies when the influence of different stimulus features is voluntarily modulated by task demands.