Integrating Reward Information for Prospective Behavior.

Value-based decision-making is often studied in a static context, where participants decide which option to select from those currently available. However, everyday life often involves an additional dimension: deciding when to select to maximize reward. Recent evidence suggests that agents track the latent reward of an option, updating changes in their latent reward estimate, to achieve appropriate selection timing (latent reward tracking). However, this strategy can be difficult to distinguish from one in which the optimal selection time is estimated in advance, allowing an agent to wait a predetermined amount of time before selecting, without needing to monitor an option's latent reward (distance-to-goal tracking). Here, we show that these strategies can in principle be dissociated. Human brain activity was recorded using electroencephalography (EEG), while female and male participants performed a novel decision task. Participants were shown an option and decided when to select it, as its latent reward changed from trial-to-trial. While the latent reward was uncued, it could be estimated using cued information about the option's starting value and value growth rate. We then used representational similarity analysis (RSA) to assess whether EEG signals more closely resembled latent reward tracking or distance-to-goal tracking. This approach successfully dissociated the strategies in this task. Starting value and growth rate were translated into a distance-to-goal signal, far in advance of selecting the option. Latent reward could not be independently decoded. These results demonstrate the feasibility of using high temporal resolution neural recordings to identify internally computed decision variables in the human brain.SIGNIFICANCE STATEMENT Reward-seeking behavior involves acting at the right time. However, the external world does not always tell us when an action is most rewarding, necessitating internal representations that guide action timing. Specifying this internal neural representation is challenging because it might stem from a variety of strategies, many of which make similar predictions about brain activity. This study used a novel approach to test whether alternative strategies could be dissociated in principle. Using representational similarity analysis (RSA), we were able to distinguish between candidate internal representations for selection timing. This shows how pattern analysis methods can be used to measure latent decision information in noninvasive neural data.

10 Simple Rules for a Supportive Lab Environment.

The transition to principal investigator (PI), or lab leader, can be challenging, partially due to the need to fulfil new managerial and leadership responsibilities. One key aspect of this role, which is often not explicitly discussed, is creating a supportive lab environment. Here, we present ten simple rules to guide the new PI in the development of their own positive and thriving lab atmosphere. These rules were written and voted on collaboratively, by the students and mentees of Professor Mark Stokes, who inspired this piece.

Reward Boosts Neural Coding of Task Rules to Optimize Cognitive Flexibility.

Cognitive flexibility is critical for intelligent behavior. However, its execution is effortful and often suboptimal. Recent work indicates that flexible behavior can be improved by the prospect of reward, which suggests that rewards optimize flexible control processes. Here we investigated how different reward prospects influence neural encoding of task rule information to optimize cognitive flexibility. We applied representational similarity analysis to human electroencephalograms, recorded while female and male participants performed a rule-guided decision-making task. During the task, the prospect of reward varied from trial to trial. Participants made faster, more accurate judgements on high-reward trials. Critically, high reward boosted neural coding of the active task rule, and the extent of this increase was associated with improvements in task performance. Additionally, the effect of high reward on task rule coding was most pronounced on switch trials, where rules were updated relative to the previous trial. These results suggest that reward prospect can promote cognitive performance by strengthening neural coding of task rule information, helping to improve cognitive flexibility during complex behavior.SIGNIFICANCE STATEMENT The importance of motivation is evident in the ubiquity with which reward prospect guides adaptive behavior and the striking number of neurological conditions associated with motivational impairments. In this study, we investigated how dynamic changes in motivation, as manipulated through reward, shape neural coding for task rules during a flexible decision-making task. The results of this work suggest that motivation to obtain reward modulates the encoding of task rules needed for flexible behavior. The extent to which reward increased task rule coding also tracked improvements in behavioral performance under high-reward conditions. These findings help to inform how motivation shapes neural processing in the healthy human brain.

Revisiting foraging approaches in neuroscience.

Many complex real-world decisions, such as deciding which house to buy or whether to switch jobs, involve trying to maximize reward across a sequence of choices. Optimal Foraging Theory is well suited to study these kinds of choices because it provides formal models for reward-maximization in sequential situations. In this article, we review recent insights from foraging neuroscience, behavioral ecology, and computational modelling. We find that a commonly used approach in foraging neuroscience, in which choice items are encountered at random, does not reflect the way animals direct their foraging efforts in certain real-world settings, nor does it reflect efficient reward-maximizing behavior. Based on this, we propose that task designs allowing subjects to encounter choice items strategically will further improve the ecological validity of foraging approaches used in neuroscience, as well as give rise to new behavioral and neural predictions that deepen our understanding of sequential, value-based choice.

Control over patch encounters changes foraging behavior.

Foraging is a common decision problem in natural environments. When new exploitable sites are always available, a simple optimal strategy is to leave a current site when its return falls below a single average reward rate. Here, we examined foraging in a more structured environment, with a limited number of sites that replenished at different rates and had to be revisited. When participants could choose sites, they visited fast-replenishing sites more often, left sites at higher levels of reward, and achieved a higher net reward rate. Decisions to exploit-or-leave a site were best explained with a computational model that included both the average reward rate for the environment and reward information about the unattended sites. This suggests that unattended sites influence leave decisions, in foraging environments where sites can be revisited.

Replay in minds and machines.

Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest that replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.

Medial orbitofrontal cortex modulates associative learning between environmental cues and reward probability.

It has recently been recognized that orbitofrontal cortex has 2 subdivisions that are anatomically and functionally distinct. Most rodent research has focused on the lateral subdivision, leaving the medial subdivision (mOFC) relatively unexplored. We recently showed that inhibiting mOFC neurons eliminated the differential impact of reward probability cues on discrimination accuracy in a sustained attention task. In the present study, we tested whether increasing mOFC neuronal activity in rats would accelerate acquisition of reward contingencies. mOFC neuronal activity was increased using the DREADD (Designer Receptors Exclusively Activated by Designer Drugs) method, in which clozapine-N-oxide administration leads to neuronal modulation by acting on synthetic receptors not normally expressed in the rat brain. We predicted that rats with neuronal activation in mOFC would require fewer sessions than controls for acquisition of a task in which visual cues signal the probability of reward for correct discrimination performance. Contrary to this prediction, mOFC neuronal activation impaired task acquisition, suggesting mOFC may play a role in learning relationships between environmental cues and reward probability or for using that information in adaptive decision-making. In addition, disrupted mOFC activity may contribute to psychiatric conditions in which learning associations between environmental cues and reward probability is impaired. (PsycINFO Database Record