A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information.

Subjects are often willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No Info) vs. a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%) and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) in the anterior cingulate cortex, orbitofrontal cortex, basolateral amygdala, or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of the anterior cingulate cortex destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that previous choice predicted current choice in all conditions, however previously rewarded Info trials strongly predicted preference in all conditions except in female rats following anterior cingulate cortex inhibition. The results reveal a causal, sex-dependent role for the anterior cingulate cortex in decisions involving information.

The effect of age on delay performance and associative learning tasks in pigeons.

Pigeons are commonly utilized in psychological research, and their cognitive abilities have been thoroughly investigated. Yet very little is known about how these abilities change with age. In contrast, age-related changes in humans, nonhuman primates, and rodents are well documented. Mammalian research consistently shows that older subjects show deficits in a variety of learning and memory processes, particularly those that rely on the prefrontal cortex and hippocampus. This research expands the avian aging literature by administering a memory task, the delayed match to sample procedure, and an associative learning task, a conditional or symbolic match to sample procedure, to nine young and 11 old pigeons. Previous research has indicated that these tasks rely on the avian equivalent to the mammalian prefrontal cortex, and we predicted that performance on both tasks would decline with age. In contrast to our predictions, only the associative learning task was sensitive to age-related decline. Performance on the memory task was maintained in older subjects. These results highlight further potential differences in avian versus mammalian aging, particularly when it comes to the prefrontal cortex.

Inhibition and paradoxical choice.

The present study evaluated the role of inhibition in paradoxical choice in pigeons. In a paradoxical choice procedure, pigeons receive a choice between two alternatives. Choosing the "suboptimal" alternative is followed 20% of the time by one cue (the S+) that is always reinforced, and 80% of the time by another cue (S-) that is never reinforced. Thus, this alternative leads to an overall reinforcement rate of 20%. Choosing the "optimal" alternative, however, is followed by one of two cues (S3 or S4), each reinforced 50% of the time. Thus, this alternative leads to an overall reinforcement rate of 50%. González and Blaisdell (2021) reported that development of paradoxical choice was positively correlated to the development of inhibition to the S- (signal that no food will be delivered on that trial) post-choice stimulus. The current experiment tested the hypothesis that inhibition to a post-choice stimulus is causally related to suboptimal preference. Following acquisition of suboptimal preference, pigeons received two manipulations: in one condition one of the cues in the optimal alternative (S4) was extinguished and, in another condition, the S- cue was partially reinforced. When tested on the choice task afterward, both manipulations resulted in a decrement in suboptimal preference. This result is paradoxical given that both manipulations made the suboptimal alternative the richer option. We discuss the implications of our results, arguing that inhibition of a post-choice cue increases attraction to or value of that choice.

Evidence for object-place binding in pigeons in a sequence-learning procedure.

We studied object-location binding in pigeons using a sequence learning procedure. A sequence of four objects was presented, one at a time at one of four locations on a touchscreen. A single peck at the object ended the trial, and food reinforcement was delivered intermittently. In Experiment 1, a between-subjects design was used to present objects, locations, or both in a regular sequence or randomly. Response time costs on nonreinforced probe tests on which object order, location order, or both were disrupted revealed sequence learning effects. Pigeons encoded location order when it was consistent, but not object order when it alone was consistent. When both were consistent, pigeons encoded both, and showed evidence of object-location binding. In Experiment 2, two groups of pigeons received training on sequences where the same object always appeared at the same location. For some pigeons a consistent sequence was used while for others sequence order was randomized. Only when sequence order was consistent was object-location binding found. These experiments are the first demonstrations of strong and lasting feature binding in pigeons and are consistent with a functional account of learning.

Evidence that novel flavors unconditionally suppress weight gain in the absence of flavor-calorie associations.

Beginning with Pavlov (1927), there has been great interest in how associative learning processes affect eating behavior. For instance, flavors can become preferred when paired with calories or, conversely, become aversive when paired with illness. This relationship between flavors and caloric or toxic outcomes has been investigated by a number of theorists. We studied the effect of daily consumption of a flavor that was either paired or unpaired with calories provided by sugar on body weight change and daily food consumption over a 21-day period. Over three experiments, we observed an unanticipated attenuation of weight gain following consumption of flavored liquid solutions, particularly when those solutions were non-caloric. However, we did not find any impact of consuming the flavored liquid solutions on appetite. Given differences in weight gain in the absence of differences in the amount of food consumed, we suggest that unconditioned metabolic responses are elicited to initially novel flavor stimuli, even if those flavors are not followed by caloric outcomes. Potential dieting interventions based on these findings are discussed as is how they inform our understanding of the balance between unconditioned and conditioned responses.

Mental imagery in animals: Learning, memory, and decision-making in the face of missing information.

When we open our eyes, we see a world filled with objects and events. Yet, due to occlusion of some objects by others, we only have partial perceptual access to the events that transpire around us. I discuss the body of research on mental imagery in animals. I first cover prior studies of mental rotation in pigeons and imagery using working memory procedures first developed for human studies. Next, I discuss the seminal work on a type of learning called mediated conditioning in rats. I then provide more in-depth coverage of work from my lab suggesting that rats can use imagery to fill in missing details of the world that are expected but hidden from perception. We have found that rats make use of an active expectation (i.e., an image) of a hidden visual event. I describe the behavioral and neurobiological studies investigating the use of a mental image, its theoretical basis, and its connections to current human cognitive neuroscience research on episodic memory, imagination, and mental simulations. Collectively, the reviewed literature provides insight into the mechanisms that mediate the flexible use of an image during ambiguous situations. I position this work in the broader scientific and philosophical context surrounding the concept of mental imagery in human and nonhuman animals.

ARENA 2.0: The next generation automated remote environmental navigation apparatus to facilitate cross-species comparisons in behavior and cognition.

A series of experiments illustrated the effectiveness and flexibility of a newly developed Automated Remote Environmental Navigation Apparatus (ARENA) as an alternative to traditional operant and open-field procedures. This system improves the concept developed by Badelt and Blaisdell (Behavior Research Methods, 40, 613-621, 2008; see also Leising, Garlick, Parenteau, & Blaisdell in Behavioural Processes, 81, 105-113, 2009), with significant upgrades in flexibility and reliability, as well as a reduction in cost. ARENA is particularly well adapted for open-field studies and eliminates many confounding factors associated with traditional procedures, such as handling effects and physical cues left by the subject. The original system was based on wireless modules with a small stimulus-response well. Nosepokes or pecks within the aperture of the well could be detected and recorded by a computer. ARENA 2.0 increases the flexibility of this system by replacing the modules with stimulus presentation through a data projector mounted on the ceiling and response detection and recording through a video camera system. We report the specifics of this system as well as behavioral tests using rats and pigeons. We demonstrated the feasibility of ARENA 2.0 for the acquisition of conditional approach to a visual target, followed by tests showing generalization of performance to novel locations and visual properties of the target. These experiments support the use of this technology for automated tasks traditionally studied through open-field preparations or using touchscreen-equipped operant chambers. The advantages of ARENA 2.0 over the original system are a significant reduction in cost and increased reliability, ease of use, and flexibility in both stimulus configuration and subject response measures.

Learning history and cholinergic modulation in the dorsal hippocampus are necessary for rats to infer the status of a hidden event.

Identifying statistical patterns between environmental stimuli enables organisms to respond adaptively when cues are later observed. However, stimuli are often obscured from detection, necessitating behavior under conditions of ambiguity. Considerable evidence indicates decisions under ambiguity rely on inference processes that draw on past experiences to generate predictions under novel conditions. Despite the high demand for this process and the observation that it deteriorates disproportionately with age, the underlying mechanisms remain unknown. We developed a rodent model of decision-making during ambiguity to examine features of experience that contribute to inference. Rats learned either a simple (positive patterning) or complex (negative patterning) instrumental discrimination between the illumination of one or two lights. During test, only one light was lit while the other relevant light was blocked from physical detection (covered by an opaque shield, rendering its status ambiguous). We found experience with the complex negative patterning discrimination was necessary for rats to behave sensitively to the ambiguous test situation. These rats behaved as if they inferred the presence of the hidden light, responding differently than when the light was explicitly absent (uncovered and unlit). Differential expression profiles of the immediate early gene cFos indicated hippocampal involvement in the inference process while localized microinfusions of the muscarinic antagonist, scopolamine, into the dorsal hippocampus caused rats to behave as if only one light was present. That is, blocking cholinergic modulation prevented the rat from inferring the presence of the hidden light. Collectively, these results suggest cholinergic modulation mediates recruitment of hippocampal processes related to past experiences and transfer of these processes to make decisions during ambiguous situations. Our results correspond with correlations observed between human brain function and inference abilities, suggesting our experiments may inform interventions to alleviate or prevent cognitive dysfunction. © 2015 Wiley Periodicals, Inc.

Blocking between landmarks during 2-D (touchscreen) and 3-D (ARENA) search tasks with pigeons.

Many studies investigating cue competition have focused on the blocking effect. We investigated the blocking effect with pigeons using a landmark-based spatial search task in both a touchscreen preparation (Exp. 1a) and an automated remote environmental navigation apparatus (Exp. 1b). In Phase 1, two landmarks (LMs: A and Z) appeared on separate trials as colored circles among a row of eight (touchscreen) or six (ARENA) identical response units. Subjects were rewarded for pecking at a target response unit to the right of LM A and to the left of an extraneous LM, Z. During the blocking trials in Phase 2, LM X was presented in compound with a second LM (A) that had been previously trained. On control trials, LM Y was presented in compound with LM B and a target in the same manner as in the trials of AX, except that neither landmark had previously been trained with the target. All subjects were then tested with separate trials of A, X, B, and Y. Testing revealed poor spatial control by X relative to A and Y. We report the first evidence for a spatial-blocking effect in pigeons and additional support for associative effects (e.g., blocking) occurring under similar conditions (e.g., training sessions, spatial relationships, etc.) in 3-D and 2-D search tasks.

Impact of equivalence class training on same/different learning by pigeons.

Separating and isolating the contributions of perception to concept formation in animals has been a long-standing and persistent challenge. Here we describe a novel approach to assessing this question by using equivalence training consisting of unrelated images as the basis for subsequent same/different (S/D) learning. Following equivalence class training, two groups of pigeons attempted to learn a go/no-go discrimination task constructed from these classes. In the go/no-go task, a consistent group was given an S/D assignment that aligned with this prior training (same vs. different classes). An inconsistent group was given go/no-go assignments that were misaligned with their established classes. The consistent group exhibited better learning and stimulus control in their S/D task than did the inconsistent group. These results suggest that pigeons can use trained properties derived from class-based information to learn an S/D task without the aid of perceptual similarity. This novel approach holds promise for helping to evaluate the contribution of perceptual similarity to different types of concept learning. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Differences in paradoxical choice between pigeons (Columba livia) and rats (Rattus norvegicus): The problem of cue trackability.

Organisms are believed to attempt to maximize their net energy intake while foraging. The paradoxical choice task shows that they may instead prefer to obtain information rather than primary reward when the outcome is uncertain. That is, they prefer stimuli that consistently predict food or no food (informative option), to stimuli that inconsistently predict both food and no food in larger amounts (noninformative option). This task also seems to indicate that some species (like pigeons, Columba livia, and starlings, Sturnus vulgaris) are more prone to choose the informative option, while other species (like rats, Rattus norvegicus, and humans, Homo sapiens) tend to favor reward procurement through the noninformative option. There is empirical evidence for and against this view. However, an analysis of the literature suggests that species differences in paradoxical choice might be less pronounced than often believed. We argue that pigeons and rats are usually not tested under conditions that are motivationally equivalent for both species-in particular, the opportunities to track consistent stimulus-food pairings are less often met in the rat studies than in the pigeon studies. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Choice reaction time in pigeons fails to increase as predicted by Hick's law.

Being able to correctly identify a target when presented with multiple possible alternatives, or increasing uncertainty, is highly beneficial in a wide variety of situations. This has been intensely investigated with human participants and results consistently demonstrated that participant reaction time (RT) increases linearly with the number of response alternatives, described as Hick's Law. Yet, the strength of this relationship is impacted by a variety of parameters, including stimulus-response compatibility, stimulus intensity, and practice. Different theories attempt to explain why these parameters affect the time to detect the target, but thus far these theories almost exclusively rely on human and nonhuman primate research. Therefore, it is unclear if these theories are universal or unique to primates, due to the scarcity of other animal models. A previous investigation showed that pigeon RT will increase in accordance with Hick's Law though not as steeply as human RT, potentially due to differences in the procedure used on pigeons. To better understand pigeon RT under uncertainty and facilitate cross species comparisons, these experiments used a procedure that was more similar to what has been given to humans. Surprisingly, pigeon RT did not follow Hick's Law as predicted. In Experiment 1, subjects showed an 'anti-Hick's' effect due to an artefact of stimulus location on the monitor. Subsequent experiments controlled for location, still RT did not increase with the number of choices as predicted by Hick's Law. Procedural changes that may have been responsible for this difference and the role of stimulus-response compatibility are discussed.

The conjunction fallacy in rats.

Humans and other animals are capable of reasoning. However, there are overwhelming examples of errors or anomalies in reasoning. In two experiments, we studied if rats, like humans, estimate the conjunction of two events as more likely than each event independently, a phenomenon that has been called conjunction fallacy. In both experiments, rats learned through food reinforcement to press a lever under some cue conditions but not others. Sound B was rewarded whereas Sound A was not. However, when B was presented with the visual cue Y was not rewarded, whereas AX was rewarded (i.e., A-, AX+, B+, BY-). Both visual cues were presented in the same bulb. After training, rats received test sessions in which A and B were presented with the bulb explicitly off or occluded by a metal piece. Thus, on the occluded condition, it was ambiguous whether the trials were of the elements alone (A or B) or of the compounds (AX or BY). Rats responded on the occluded condition as if the compound cues were most likely present. The second experiment investigated if this error in probability estimation in Experiment 1, could be due to a conjunction fallacy, and if this could be attenuated by increasing the ratio of element/compound trials from the original 50-50 to 70-30 and 90-10. Only the 90-10 condition (where 90% of the training trials were of just A or just B) did not show a conjunction fallacy, though it emerged in all groups with additional training. These findings open new avenues for exploring the mechanisms behind the conjunction fallacy effect.

A special role for anterior cingulate cortex, but not orbitofrontal cortex or basolateral amygdala, in choices involving information.

Subjects often are willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No-info) vs a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%), and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory DREADDs in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), basolateral amygdala (BLA), or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of ACC destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that the previous choice strongly predicted preference in control animals, but not in female rats following ACC inhibition. The results reveal a causal, sex-dependent role for ACC in decisions involving information.

Rats distinguish between absence of events and lack of evidence in contingency learning.

The goal of three experiments was to study whether rats are aware of the difference between absence of events and lack of evidence. We used a Pavlovian extinction paradigm in which lights consistently signaling sucrose were suddenly paired with the absence of sucrose. The crucial manipulation involved the absent outcomes in the extinction phase. Whereas in the Cover conditions, access to the drinking receptacle was blocked by a metal plate, in the No Cover conditions, the drinking receptacle was accessible. The Test phase showed that in the Cover conditions, the measured expectancies of sucrose were clearly at a higher level than in the No Cover conditions. We compare two competing theories potentially explaining the findings. A cognitive theory interprets the observed effect as evidence that the rats were able to understand that the cover blocked informational access to the outcome information, and therefore the changed learning input did not necessarily signify a change of the underlying contingency in the world. An alternative associationist account, renewal theory, might instead explain the relative sparing of extinction in the Cover condition as a consequence of context change. We discuss the merits of both theories as accounts of our data and conclude that the cognitive explanation is in this case preferred.

Elements of a compound elicit little conditioned reinforcement.

The acquisition of instrumental responding can be supported by primary reinforcers or by conditional (also known as secondary) reinforcers that themselves have an association to a primary reinforcer. While primary reinforcement has been heavily studied for the past century, the associative basis of conditioned reinforcement has received comparatively little experimental examination. Yet conditioned reinforcement has been employed as an important behavioral assay in neuroscience studies, and thus an analysis of its associative basis is called for. We evaluated the extent to which an element from a previously trained compound would facilitate conditioned reinforcement. Three groups of rats received Pavlovian conditioning with a visual-auditory compound cue followed by food. After training, a lever was made available that, when pressed, produced the same trained compound (group compound), only the auditory cue (group element), or a novel auditory cue (group control). The rats in group compound pressed the lever at a higher rate than did rats in either group element or group control, demonstrating a strong conditioned reinforcement effect only in group compound. Interestingly, there was almost no difference in responding between group element and group control. The implications of this generalization decrement in conditioned reinforcement are discussed-particularly as they relate to research in behavioral neuroscience. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner.

For over two decades, phasic activity in midbrain dopamine neurons was considered synonymous with the prediction error in temporal-difference reinforcement learning.1-4 Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received, allowing for the calculation of prediction errors.5,6 Phasic firing of dopamine neurons was proposed to reflect this computation,1,2 facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: (1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and (2) that the dopamine signal can only be involved in anticipatory cue-reward learning in which cues or actions precede rewards. Recent work7-15 has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning-when a neutral cue reliably follows a rewarding outcome.16-20 Using a specific Pavlovian-to-instrumental transfer (PIT) procedure,21-23 we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of VTADA neurons timed to the transition between the reward and cue reduces both of these components of backward conditioning. These findings suggest VTADA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data may be in line with suggestions that the VTADA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in myriad psychological disorders that are characterized by very distinct reinforcement-learning deficits.

Pigeon and human performance in a multi-armed bandit task in response to changes in variable interval schedules.

The tension between exploitation of the best options and exploration of alternatives is a ubiquitous problem that all organisms face. To examine this trade-off across species, pigeons and people were trained on an eight-armed bandit task in which the options were rewarded on a variable interval (VI) schedule. At regular intervals, each option's VI changed, thus encouraging dynamic increases in exploration in response to these anticipated changes. Both species showed sensitivity to the payoffs that was often well modeled by Luce's (1963) decision rule. For pigeons, exploration of alternative options was driven by experienced changes in the payoff schedules, not the beginning of a new session, even though each session signaled a new schedule. In contrast, people quickly learned to explore in response to signaled changes in the payoffs.

The role of inhibition in the suboptimal choice task.

Given a choice, pigeons prefer an initial-link stimulus that is followed by reliable signals that food will be delivered (S+) or not (S-) after a delay, over an alternative initial-link stimulus that is followed by unreliable signals of food, even when the former yields a lower overall probability of food. This suboptimal preference has been attributed to the combination of a biased attraction to the S+ and ignoring the S-. We evaluated the inhibitory properties of the S- in three experiments to investigate its role in suboptimal choice. In Experiment 1, pigeons were trained in an autoshaping procedure with the four terminal link stimuli of the suboptimal choice task; S+ was continuously reinforced, S3 and S4 were each partially reinforced on a 50% schedule, and S- was never reinforced. Summation tests showed that S- acquired inhibitory properties during training. Experiment 2 replicated the summation tests after training on the full suboptimal choice procedure and found that S- inhibition was not attributable to external inhibition. In Experiment 3, pigeons were trained on the suboptimal choice procedure and the development of inhibition was assessed throughout training. An analysis of individual differences across birds revealed that the response rates to S- were negatively correlated with the strength of suboptimal preference, both within subject as each bird acquired suboptimal preference, and across subjects once all birds had reached asymptotic levels of suboptimal preference. Thus, rather than ignoring the S-, we found evidence that birds attended to S- as an inhibitory stimulus. Future models explaining performance in the suboptimal choice task should consider inhibition to the S- as a factor in suboptimal choice. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Higher-Order Conditioning and Dopamine: Charting a Path Forward.

Higher-order conditioning involves learning causal links between multiple events, which then allows one to make novel inferences. For example, observing a correlation between two events (e.g., a neighbor wearing a particular sports jersey), later helps one make new predictions based on this knowledge (e.g., the neighbor's wife's favorite sports team). This type of learning is important because it allows one to benefit maximally from previous experiences and perform adaptively in complex environments where many things are ambiguous or uncertain. Two procedures in the lab are often used to probe this kind of learning, second-order conditioning (SOC) and sensory preconditioning (SPC). In second-order conditioning (SOC), we first teach subjects that there is a relationship between a stimulus and an outcome (e.g., a tone that predicts food). Then, an additional stimulus is taught to precede the predictive stimulus (e.g., a light leads to the food-predictive tone). In sensory preconditioning (SPC), this order of training is reversed. Specifically, the two neutral stimuli (i.e., light and tone) are first paired together and then the tone is paired separately with food. Interestingly, in both SPC and SOC, humans, rodents, and even insects, and other invertebrates will later predict that both the light and tone are likely to lead to food, even though they only experienced the tone directly paired with food. While these processes are procedurally similar, a wealth of research suggests they are associatively and neurobiologically distinct. However, midbrain dopamine, a neurotransmitter long thought to facilitate basic Pavlovian conditioning in a relatively simplistic manner, appears critical for both SOC and SPC. These findings suggest dopamine may contribute to learning in ways that transcend differences in associative and neurological structure. We discuss how research demonstrating that dopamine is critical to both SOC and SPC places it at the center of more complex forms of cognition (e.g., spatial navigation and causal reasoning). Further, we suggest that these more sophisticated learning procedures, coupled with recent advances in recording and manipulating dopamine neurons, represent a new path forward in understanding dopamine's contribution to learning and cognition.