Effects of sleep deprivation on food-related Pavlovian-instrumental transfer: a randomized crossover experiment.

Recent research suggests that insufficient sleep elevates the risk of obesity. Although the mechanisms underlying the relationship between insufficient sleep and obesity are not fully understood, preliminary evidence suggests that insufficient sleep may intensify habitual control of behavior, leading to greater cue-elicited food-seeking behavior that is insensitive to satiation. The present study tested this hypothesis using a within-individual, randomized, crossover experiment. Ninety-six adults underwent a one-night normal sleep duration (NSD) condition and a one-night total sleep deprivation (TSD) condition. They also completed the Pavlovian-instrumental transfer paradigm in which their instrumental responses for food in the presence and absence of conditioned cues were recorded. The sleep × cue × satiation interaction was significant, indicating that the enhancing effect of conditioned cues on food-seeking responses significantly differed across sleep × satiation conditions. However, this effect was observed in NSD but not TSD, and it disappeared after satiation. This finding contradicted the hypothesis but aligned with previous literature on the effect of sleep disruption on appetitive conditioning in animals-sleep disruption following learning impaired the expression of appetitive behavior. The present finding is the first evidence for the role of sleep in Pavlovian-instrumental transfer effects. Future research is needed to further disentangle how sleep influences motivational mechanisms underlying eating.

Encoding of Predictive Associations in Human Prefrontal and Medial Temporal Neurons During Pavlovian Appetitive Conditioning.

Pavlovian conditioning is thought to involve the formation of learned associations between stimuli and values, and between stimuli and specific features of outcomes. Here, we leveraged human single neuron recordings in ventromedial prefrontal, dorsomedial frontal, hippocampus, and amygdala while patients of both sexes performed an appetitive Pavlovian conditioning task probing both stimulus-value and stimulus-stimulus associations. Ventromedial prefrontal cortex encoded predictive value along with the amygdala, and also encoded predictions about the identity of stimuli that would subsequently be presented, suggesting a role for neurons in this region in encoding predictive information beyond value. Unsigned error signals were found in dorsomedial frontal areas and hippocampus, potentially supporting learning of non-value related outcome features. Our findings implicate distinct human prefrontal and medial temporal neuronal populations in mediating predictive associations which could partially support model-based mechanisms during Pavlovian conditioning.

From safety to frustration: The neural substrates of inhibitory learning in aversive and appetitive conditioning procedures.

Inhibitory associative learning counters the effects of excitatory learning, whether appetitively or aversively motivated. Moreover, the affective responses accompanying the inhibitory associations are of opponent valence to the excitatory conditioned responses. Inhibitors for negative aversive outcomes (e.g. shock) signal safety, while inhibitors for appetitive outcomes (e.g. food reward) elicit frustration and/or disappointment. This raises the question as to whether studies using appetitive and aversive conditioning procedures should demonstrate the same neural substrates for inhibitory learning. We review the neural substrates of appetitive and aversive inhibitory learning as measured in different procedural variants and in the context of the underpinning excitatory conditioning on which it depends. The mesocorticolimbic dopamine pathways, retrosplenial cortex and hippocampus are consistently implicated in inhibitory learning. Further neural substrates identified in some procedural variants may be related to the specific motivation of the learning task and modalities of the learning cues. Finally, we consider the translational implications of our understanding of the neural substrates of inhibitory learning, for obesity and addictions as well as for anxiety disorders.

Modulation of aversive value coding in the vertebrate and invertebrate brain.

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

Stress effects on memory retrieval of aversive and appetitive instrumental counterconditioning in men.

Extinction training creates a second inhibitory memory trace and effectively reduces conditioned responding. However, acute stress inhibits the retrieval of this extinction memory trace. It is not known whether this also applies to other forms of associative learning such as instrumental counterconditioning, where previously learned associations are reversed and paired with the opposite valence. Therefore, the current preregistered study investigates whether stress decreases the retrieval of instrumental counterconditioning memories with aversive and appetitive consequences. Fifty-two healthy men were randomly assigned to either a stress or control group and took part in a two-day instrumental learning paradigm. During a first phase, participants learned that pressing specific buttons in response to the presentation of four neutral stimuli either leads to gaining or losing money. During a second phase, two stimuli reversed their contingencies (counterconditioning). One day later, participants were exposed to acute stress or a control condition prior to the same task, which no longer included feedback about gains or losses. Stressed participants showed more approach behavior towards appetitive and less avoidance behavior towards aversive stimuli as compared to non-stressed participants. Our findings indicate that stress effects on memory retrieval differ depending on the associative learning approach in men. These differences might be related to stress effects on decision making and different motivational systems involved.

Gap in the conditioned stimulus: Differential impacts on temporal expectancy in appetitive and aversive conditions in rats.

We analyzed, through a Pavlovian conditioning procedure in rats, the temporal pattern of behavior in appetitive and aversive conditions within subjects, and the difference in inferred temporal working memory functioning with the Gap paradigm. For both conditions, we paired a 60-s conditioned stimulus (CS: tone1 or tone2) with an unconditioned stimulus (US: shock or chocolate pellet) delivered 20s after CS onset. The analyses of mean response rate and individual-trial data were performed during Probe trials, consisting of CS alone, and trials in which gaps of different position or duration were inserted, to assess the effect of the temporal manipulation on behavior. The results showed: (1) An anticipatory peak time in the aversive condition but better accuracy in the appetitive condition, (2) constancy in the Weber fraction suggesting that the difference in peak time was under clock control, (3) a graded effect of gap parameters only in the aversive condition and (4) different gap effects between conditions when a gap was inserted early in the CS. These results highlight behavioral differences between aversive and appetitive conditions and suggest that the temporal working memory mechanism was not engaged in the same manner in each condition.

The potential and realized foraging movements of bees are differentially determined by body size and sociality.

Reversing biodiversity declines requires a better understanding of organismal mobility, as movement processes dictate the scale at which species interact with the environment. Previous studies have demonstrated that species foraging ranges, and therefore, habitat use increases with body size. Yet, foraging ranges are also affected by other life-history traits, such as sociality, which influence the need of and ability to detect resources. We evaluated the effect of body size and sociality on potential and realized foraging ranges using a compiled dataset of 383 measurements for 81 bee species. Potential ranges were larger than realized ranges and increased more steeply with body size. Highly eusocial species had larger realized foraging ranges than primitively eusocial or solitary taxa. We contend that potential ranges describe species movement capabilities, whereas realized ranges depict how foraging movements result from interactions between species traits and environmental conditions. Furthermore, the complex communication strategies and large colony sizes in highly eusocial species may facilitate foraging over wider areas in response to resource depletion. Our findings should contribute to a greater understanding of landscape ecology and conservation, as traits that influence movement mediate species vulnerability to habitat loss and fragmentation.

Harmonics of the social brain: How diverse brain regions coordinate appetitive social behavior.

Socioemotional behaviors rely on the integration of information across multiple systems in the brain. In this issue of Neuron, Mague et al. (2022) characterize a multi-regional functional network that coordinates positively valenced social interactions in mice.

A bottom-up reward pathway mediated by somatostatin neurons in the medial septum complex underlying appetitive learning.

Valence detection and processing are essential for the survival of animals and their life quality in complex environments. Neural circuits underlying the transformation of external sensory signals into positive valence coding to generate appropriate behavioral responses remain not well-studied. Here, we report that somatostatin (SOM) subtype of GABAergic neurons in the mouse medial septum complex (MS), but not parvalbumin subtype or glutamatergic neurons, specifically encode reward signals and positive valence. Through an ascending pathway from the nucleus of solitary tract and then parabrachial nucleus, the MS SOM neurons receive rewarding taste signals and suppress the lateral habenula. They contribute essentially to appetitive associative learning via their projections to the lateral habenula: learning enhances their responses to reward-predictive sensory cues, and suppressing their responses to either conditioned or unconditioned stimulus impairs acquisition of reward learning. Thus, MS serves as a critical hub for transforming bottom-up sensory signals to mediate appetitive behaviors.

Octopaminergic neurons function in appetitive but not aversive olfactory learning and memory in Bactrocera dorsalis.

The biogenic amine octopamine (OA, invertebrate counterpart of noradrenaline) plays critical roles in the regulation of olfactory behavior. Historically, OA has been thought to mediate appetitive but not aversive learning in honeybees, fruit flies (Drosophila), and crickets. However, this viewpoint has recently been challenged because OA activity through a ß-adrenergic-like receptor drives both appetitive and aversive learning. Here, we explored the roles of OA neurons in olfactory learning and memory retrieval in Bactrocera dorsalis. We trained flies to associate an orange odor with a sucrose reward or to associate methyl eugenol, a male lure, with N,N-diethyl-3-methyl benzoyl amide (DEET) punishment. We then treated flies with OA receptor antagonists before appetitive or aversive conditioning and a memory retention test. Injection of OA receptor antagonist mianserin or epinastine into the abdomen of flies led to impaired of appetitive learning and memory retention with a sucrose reward, while aversive learning and memory retention with DEET punishment remained intact. Our results suggest that the OA signaling participates in appetitive but not aversive learning and memory retrieval in B. dorsalis through OA receptors.

The evolutionary maintenance of Lévy flight foraging.

Lévy flight is a type of random walk that characterizes the behaviour of many natural phenomena studied across a multiplicity of academic disciplines; within biology specifically, the behaviour of fish, birds, insects, mollusks, bacteria, plants, slime molds, t-cells, and human populations. The Lévy flight foraging hypothesis states that because Lévy flights can maximize an organism's search efficiency, natural selection should result in Lévy-like behaviour. Empirical and theoretical research has provided ample evidence of Lévy walks in both extinct and extant species, and its efficiency across models with a diversity of resource distributions. However, no model has addressed the maintenance of Lévy flight foraging through evolutionary processes, and existing models lack ecological breadth. We use numerical simulations, including lineage-based models of evolution with a distribution of move lengths as a variable and heritable trait, to test the Lévy flight foraging hypothesis. We include biological and ecological contexts such as population size, searching costs, lifespan, resource distribution, speed, and consider both energy accumulated at the end of a lifespan and averaged over a lifespan. We demonstrate that selection often results in Lévy-like behaviour, although conditional; smaller populations, longer searches, and low searching costs increase the fitness of Lévy-like behaviour relative to Brownian behaviour. Interestingly, our results also evidence a bet-hedging strategy; Lévy-like behaviour reduces fitness variance, thus maximizing geometric mean fitness over multiple generations.

Foraging as sampling without replacement: A Bayesian statistical model for estimating biases in target selection.

Foraging entails finding multiple targets sequentially. In humans and other animals, a key observation has been a tendency to forage in 'runs' of the same target type. This tendency is context-sensitive, and in humans, it is strongest when the targets are difficult to distinguish from the distractors. Many important questions have yet to be addressed about this and other tendencies in human foraging, and a key limitation is a lack of precise measures of foraging behaviour. The standard measures tend to be run statistics, such as the maximum run length and the number of runs. But these measures are not only interdependent, they are also constrained by the number and distribution of targets, making it difficult to make inferences about the effects of these aspects of the environment on foraging. Moreover, run statistics are underspecified about the underlying cognitive processes determining foraging behaviour. We present an alternative approach: modelling foraging as a procedure of generative sampling without replacement, implemented in a Bayesian multilevel model. This allows us to break behaviour down into a number of biases that influence target selection, such as the proximity of targets and a bias for selecting targets in runs, in a way that is not dependent on the number of targets present. Our method thereby facilitates direct comparison of specific foraging tendencies between search environments that differ in theoretically important dimensions. We demonstrate the use of our model with simulation examples and re-analysis of existing data. We believe our model will provide deeper insights into visual foraging and provide a foundation for further modelling work in this area.

Corticostriatal Suppression of Appetitive Pavlovian Conditioned Responding.

The capacity to suppress learned responses is essential for animals to adapt in dynamic environments. Extinction is a process by which animals learn to suppress conditioned responding when an expected outcome is omitted. The infralimbic (IL) cortex to nucleus accumbens shell (NAcS) neural circuit is implicated in suppressing conditioned responding after extinction, especially in the context of operant cocaine-seeking behavior. However, the role of the IL-to-NAcS neural circuit in the extinction of responding to appetitive Pavlovian cues is unknown, and the psychological mechanisms involved in response suppression following extinction are unclear. We trained male Long Evans rats to associate a 10 s auditory conditioned stimulus (CS; 14 trials per session) with a sucrose unconditioned stimulus (US; 0.2 ml per CS) in a specific context, and then following extinction in a different context, precipitated a renewal of CS responding by presenting the CS alone in the original Pavlovian conditioning context. Unilateral, optogenetic stimulation of the IL-to-NAcS circuit selectively during CS trials suppressed renewal. In a separate experiment, IL-to-NAcS stimulation suppressed CS responding regardless of prior extinction and impaired extinction retrieval. Finally, IL-to-NAcS stimulation during the CS did not suppress the acquisition of Pavlovian conditioning but was required for the subsequent expression of CS responding. These results are consistent with multiple studies showing that the IL-to-NAcS neural circuit is involved in the suppression of operant cocaine-seeking, extending these findings to appetitive Pavlovian cues. The suppression of appetitive Pavlovian responding following IL-to-NAcS circuit stimulation, however, does not appear to be an extinction-dependent process.SIGNIFICANCE STATEMENT Extinction is a form of inhibitory learning through which animals learn to suppress conditioned responding in the face of nonreinforcement. We investigated the role of the IL cortex inputs to the NAcS in the extinction of responding to appetitive Pavlovian cues and the psychological mechanisms involved in response suppression following extinction. Using in vivo optogenetics, we found that stimulating the IL-to-NAcS neural circuit suppressed context-induced renewal of conditioned responding after extinction. In a separate experiment, stimulating the IL-to-NAcS circuit suppressed conditioned responding in an extinction-independent manner. These findings can be used by future research aimed at understanding how corticostriatal circuits contribute to behavioral flexibility and mental disorders that involve the suppression of learned behaviors.

How the value of the environment controls persistence in visual search.

Classic foraging theory predicts that humans and animals aim to gain maximum reward per unit time. However, in standard instrumental conditioning tasks individuals adopt an apparently suboptimal strategy: they respond slowly when the expected value is low. This reward-related bias is often explained as reduced motivation in response to low rewards. Here we present evidence this behavior is associated with a complementary increased motivation to search the environment for alternatives. We trained monkeys to search for reward-related visual targets in environments with different values. We found that the reward-related bias scaled with environment value, was consistent with persistent searching after the target was already found, and was associated with increased exploratory gaze to objects in the environment. A novel computational model of foraging suggests that this search strategy could be adaptive in naturalistic settings where both environments and the objects within them provide partial information about hidden, uncertain rewards.

Water-seeking behavior among terrestrial arthropods and mollusks in a cool mesic region: Spatial and temporal patterns.

Dehydration can have negative effects on animal physiological performance, growth, reproduction, and survival, and most animals seek to minimize these effects by reducing water losses or seeking water sources. Much-but not all-of the research on animal water balance comes from dryland ecosystems. However, animals inhabiting mesic regions may also experience desiccating conditions, for example within urban heat islands or during heatwaves and droughts. Here we examined how spatial variation in impervious surface and spatial and temporal variation in microclimate impact water demand behavior of terrestrial arthropods and mollusks in three areas of mesic Northwest Ohio, with analysis of taxa that exhibited the greatest water demand behavior. Water demand behavior was measured as the frequency that individuals were observed at an artificial water source (a moistened pouch), relative to the frequency at a control (a dry pouch). Overall, terrestrial arthropods and mollusks were found about twice as often at the water source than at the control (equivalent to 86 more observations on the wet pouch than on dry at each site, on average), with ants accounting for over 50% of the overall response in urban areas. Daily fluctuations in vapor pressure deficit (VPD) best predicted daily variation in water demand behavior, with increased demand at higher VPD. Mean VPD was generally highest near urbanized areas, but effects of VPD on water demand behavior were generally lower in urbanized areas (possibly related to reductions in overall abundance reducing the potential response). On certain days, VPD was high in natural areas and greenspaces, and this coincided with the highest arthropod water demand behavior observed. Our results suggest that terrestrial arthropod communities do experience periods of water demand within mesic regions, including in greenspaces outside cities, where they appear to respond strongly to short periods of dry conditions-an observation with potential relevance for understanding the effects of climate change.

Inferring predator-prey interaction in the subterranean environment: a case study from Dinaric caves.

Predator-prey interactions are among the most important biotic interactions shaping ecological communities and driving the evolution of defensive traits. These interactions and their effects on species received little attention in extreme and remote environments, where possibilities for direct observations and experimental manipulation of the animals are limited. In this paper, we study such type of environment, namely caves of the Dinarides (Europe), combining spatial and phylogenetic methods. We focused on several species of Niphargus amphipods living in phreatic lakes, as some of them use the dorsal spines as putative morphological defensive traits. We predicted that these spines represent a defense strategy against the olm (Proteus anguinus), a top predator species in the subterranean waters. We tested for spatial overlap of the olm and Niphargus species and showed that spined species live in closer proximity to and co-occur more frequently with the olm than non-spined species. Modeling of the evolution of the spines onto Niphargus phylogeny implies coevolution of this trait in the presence of olm. We conclude that these spines likely evolved as defensive traits in a predator-prey arms race. Combining multiple analyses, we provide an example for a methodological framework to assess predator-prey interactions when in-situ or laboratory observations are not possible.

An operant ethanol self-administration paradigm that discriminates between appetitive and consummatory behaviors reveals distinct behavioral phenotypes in commonly used rat strains.

Alcohol use disorder (AUD) constitutes a major burden to global health. Recently, the translational success of animal models of AUD has come under increased scrutiny. Efforts to refine models to gain a more precise understanding of the neurobiology of addiction are warranted. Appetitive responding for ethanol (seeking) and its consumption (taking) are governed by distinct neurobiological mechanisms. However, consumption is often inferred from appetitive responding in operant ethanol self-administration paradigms, preventing identification of distinct experimental effects on seeking and taking. In the present study, male Long-Evans, Wistar, and Sprague-Dawley rats were trained to lever press for ethanol using a lickometer-equipped system that precisely measures both appetitive and consummatory behavior. Three distinct operant phenotypes emerged during training: 1) Drinkers, who lever press and consume ethanol; 2) Responders, who lever press but consume little to no ethanol; and 3) Non-responders, who do not lever press. While the prevalence of each phenotype differed across strains, appetitive and consummatory behavior was similar across strains within each phenotype. Appetitive and consummatory behaviors were significantly correlated in Drinkers, but not Responders. Analysis of drinking microstructure showed that greater consumption in Drinkers relative to Responders is due to increased incentive for ethanol rather than increased palatability. Importantly, withdrawal from chronic ethanol exposure resulted in a significant increase in appetitive responding in both Drinkers and Responders, but only Drinkers exhibited a concomitant increase in ethanol consumption. Together, these data reveal important strain differences in appetitive and consummatory responding for ethanol and uncover the presence of distinct operant phenotypes.

Choice history effects in mice and humans improve reward harvesting efficiency.

Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias.

Inhibition of serotonergic signaling induces higher consumption of both sucrose solution and toxic baits in carpenter ants.

Biogenic amines play an important role in the regulation of appetitive responses in insects. Among them, serotonin (5-HT) regulates feeding-related processes in numerous insect species. In carpenter ants, 5-HT administration has been shown to depress feeding behavior, thus opening the possibility of using 5-HT modulation in control strategies against those species considered as pest. Here we studied if administration of a 5-HT antagonist, ketanserin, promotes feeding of a sucrose solution and a toxic bait in carpenter ants Camponotus mus. We found that 3 h after a single oral administration of ketanserin, the mass of sucrose solution consumed by carpenter ants increased significantly. A similar effect was found after a chronic administration that lasted 5 days. Yet, ketanserin did neither affect the intake rates nor the activity of the pharyngeal pump that mediates feeding dynamics. In addition, ketanserin promoted the consumption of a toxic bait based on boric acid. Our results thus show that feeding motivation and consumption of both sucrose solution and a toxic bait can be enhanced via prior administration of ketanserin. We discuss the possible mechanisms underlying these effects and conclude that understanding basic physiological and neural principles that underlie feeding motivation allows establishing more efficient control strategies for pest insects.

Lateral hypothalamic LEPR neurons drive appetitive but not consummatory behaviors.

Assigning behavioral roles to genetically defined neurons within the lateral hypothalamus (LH) is an ongoing challenge. We demonstrate that a subpopulation of LH GABAergic neurons expressing leptin receptors (LHLEPR) specifically drives appetitive behaviors in mice. Ablation of LH GABAergic neurons (LHVGAT) decreases weight gain and food intake, whereas LHLEPR ablation does not. Appetitive learning in a Pavlovian conditioning paradigm is delayed in LHVGAT-ablated mice but prevented entirely in LHLEPR-ablated mice. Both LHVGAT and LHLEPR neurons bidirectionally modulate reward-related behaviors, but only LHVGAT neurons affect feeding. In the Pavlovian paradigm, only LHLEPR activity discriminates between conditioned cues. Optogenetic activation or inhibition of either population in this task disrupts discrimination. However, manipulations of LHLEPR→VTA projections evoke divergent effects on responding. Unlike food-oriented learning, chemogenetic inhibition of LHLEPR neurons does not alter cocaine-conditioned place preference but attenuates cocaine sensitization. Thus, LHLEPR neurons may specifically regulate appetitive behaviors toward non-drug reinforcers.