Variation in the effectiveness of reinforcement and nonreinforcement in generating different conditioned behaviors.

Rat autoshaping procedures generate two readily measurable conditioned responses: During lever presentations that have previously signaled food, rats approach the food well (called goal-tracking) and interact with the lever itself (called sign-tracking). We investigated how reinforced and nonreinforced trials affect the overall and temporal distributions of these two responses across 10-second lever presentations. In two experiments, reinforced trials generated more goal-tracking than sign-tracking, and nonreinforced trials resulted in a larger reduction in goal-tracking than sign-tracking. The effect of reinforced trials was evident as an increase in goal-tracking and reduction in sign-tracking across the duration of the lever presentations, and nonreinforced trials resulted in this pattern transiently reversing and then becoming less evident with further training. These dissociations are consistent with a recent elaboration of the Rescorla-Wagner model, HeiDI (Honey, R.C., Dwyer, D.M., & Iliescu, A.F. (2020a). HeiDI: A model for Pavlovian learning and performance with reciprocal associations. Psychological Review, 127, 829-852.), a model in which responses related to the nature of the unconditioned stimulus (e.g., goal-tracking) have a different origin than those related to the nature of the conditioned stimulus (e.g., sign-tracking).

Decision Making Within and Outside Standard Operating Procedures: Paradoxical Use of Operational Discretion in Firefighters.

OBJECTIVE: To understand how firefighters' use of rules (i.e., standard operating procedures [SOPs]) and deliberative decision making (i.e., operational discretion [OD]) interacts with acute stress. BACKGROUND: Current operational guidance for UK firefighters combines the provision of SOPs, for routine incidents, with the use of OD, under prescribed conditions (e.g., when there is a risk to human life). However, our understanding of the use of SOPs and OD is limited. METHODS: Incident commanders (ICs; n = 43) responded to simulated emergency incidents, which either licensed the use of OD or required use of a SOP. Video footage of IC behavior was used to code their response as involving a SOP or OD, while levels of acute stress were assessed using a blood-based measure and self-report. RESULTS: ICs were less likely to use OD selectively in the simulated emergency incident that licensed its use than in the one for which use of an SOP was appropriate; IC command level did not affect this pattern of results; and the incident that licensed OD resulted in more acute stress than the incident that required use of a SOP. CONCLUSION: SOPs and OD were not used in the manner prescribed by current operational guidance in simulated emergency incidents. APPLICATION: These results suggest that firefighter training in SOPs and OD should be augmented alongside personal resilience training, given the impact of stress on health and wellbeing, but also to improve the deployment of SOPs and OD under stress.

Conditioning with spatio-temporal patterns: Constraining the contribution of the hippocampus to configural learning.

The conditions under which the hippocampus contributes to learning about spatio-temporal configural patterns are not fully established. The aim of Experiments 1-4 was to investigate the impact of hippocampal lesions on learning about where or when a reinforcer would be delivered. In each experiment, the rats received exposure to an identical set of patterns (i.e., spotted+morning, checked+morning, spotted+afternoon and checked+afternoon); and the contexts (Experiment 1), times of day (Experiment 2), or their configuration (Experiments 3 and 4) signalled whether or not a reinforcer would be delivered. The fact that hippocampal damage did not disrupt the formation of simple or configural associations involving spatio-temporal patterns is surprising, and suggests that the contribution of the hippocampus is restricted to mediated learning (or updating) involving spatio-temporal configurations.

Novel sensory preconditioning procedures identify a specific role for the hippocampus in pattern completion.

Successful retrieval of a memory for an entire pattern of stimulation by the presentation of a fragment of that pattern is a critical facet of memory function. We examined processes of pattern completion using novel sensory preconditioning procedures in rats that had either received sham lesions (group Sham) or lesions of the hippocampus (group HPC). After exposure to two audio-visual patterns (AX and BY) rats received fear conditioning with X (but not Y). Subsequent tests assessed fear to stimulus compounds (e.g., AX versus BX; Experiment 1) or elements (A versus B; Experiment 2). There was more fear to AX than BX in group Sham but not group HPC, while there was more fear to A than B in group HPC, but not in group Sham. This double dissociation suggests that pattern completion can be based upon separable processes that differ in their reliance on the hippocampus.

Configural integration of temporal and contextual information in rats: Automated measurement in appetitive and aversive preparations.

Two experiments investigated the capacity of rats to learn configural discriminations requiring integration of contextual (where) with temporal (when) information. In Experiment 1, during morning training sessions, food was delivered in context A and not in context B, whereas during afternoon sessions food was delivered in context B and not in context A. Rats acquired this discrimination over the course of 20 days. Experiment 2 employed a directly analogous aversive conditioning procedure in which footshock served in place of food. This procedure allowed the acquisition of the discrimination to be assessed through changes in activity to the contextual + temporal configurations (i.e., inactivity or freezing) and modulation of the immediate impact of footshock presentations (i.e., post-shock activity bursts). Both measures provided evidence of configural learning over the course of 12 days, with a final test showing that the presentation of footshock resulted in more post-shock activity in the nonreinforced than reinforced configurations. These behavioral effects reveal important parallels between (i) configural discrimination learning involving components allied to episodic memory and (ii) simple conditioning.

A critical role for the hippocampus and perirhinal cortex in perceptual learning of scenes and faces: complementary findings from amnesia and FMRI.

It is debated whether subregions within the medial temporal lobe (MTL), in particular the hippocampus (HC) and perirhinal cortex (PrC), play domain-sensitive roles in learning. In the present study, two patients with differing degrees of MTL damage were first exposed to pairs of highly similar scenes, faces, and dot patterns and then asked to make repeated same/different decisions to preexposed and nonexposed (novel) pairs from the three categories (Experiment 1). We measured whether patients would show a benefit of prior exposure (preexposed > nonexposed) and whether repetition of nonexposed (and preexposed) pairs at test would benefit discrimination accuracy. Although selective HC damage impaired learning of scenes, but not faces and dot patterns, broader MTL damage involving the HC and PrC compromised discrimination learning of scenes and faces but left dot pattern learning unaffected. In Experiment 2, a similar task was run in healthy young participants in the MRI scanner. Functional region-of-interest analyses revealed that posterior HC and posterior parahippocampal gyrus showed greater activity during scene pattern learning, but not face and dot pattern learning, whereas PrC, anterior HC, and posterior fusiform gyrus were recruited during discrimination learning for faces, but not scenes and dot pattern learning. Critically, activity in posterior HC and PrC, but not the other functional region-of-interest analyses, was modulated by accuracy (correct > incorrect within a preferred category). Therefore, both approaches revealed a key role for the HC and PrC in discrimination learning, which is consistent with representational accounts in which subregions in these MTL structures store complex spatial and object representations, respectively.

Associative structures in animal learning: dissociating elemental and configural processes.

The central concern of associative learning theory is to provide an account of behavioral adaptation that is parsimonious in addressing three key questions: (1) under what conditions does learning occur, (2) what are the associative structures involved, and (3) how do these affect behavior? The principle focus here is on the second question, concerning associative structures, but we will have cause to touch on the others in passing. This question is one that has exercised theorists since Pavlov's descriptions of the conditioning process, where he identifies the shared significance of the study of conditioned reflexes for psychologists and neuroscientists alike.

Instrumental responses and Pavlovian stimuli as temporal referents in a peak procedure.

Three experiments investigated whether the nature of the temporal referent affects timing behaviour in rats. We used a peak procedure and assessed timing of food well activity as a function of whether the referent was an instrumental response (a lever press that resulted in the withdrawal of the lever) or a conditioned stimulus (CS) that was 2 s in Experiment 1, 500 ms in Experiment 2, and 800 ms in Experiment 3. In all experiments, the interval between the offset of the temporal referent and food was 5 s. The curve fits for each experiment revealed no differences in peak time, but magazine responding immediately following the CS was higher than following a lever press. This pattern of results was interpreted as reflecting a combination of (a) ambiguity in which component of the 500 ms-2-s auditory stimulus was serving as the referent and (b) response competition between lever pressing and magazine activity. Critically, these results suggest that peak timing in rats is unaffected by whether a lever press or CS serves as the referent. This conclusion is consistent with theoretical models of timing behaviour, but not with evidence from humans showing that the subjective perception of time is affected by whether the cause of an outcome was self-generated or not.

Prediction error in models of adaptive behavior.

Pavlovian conditioning is evident in every species in which it has been assessed, and there is a consensus about its interpretation across behavioral,1,2 brain,3,4,5,6 and computational analyses7,8,9,10,11: conditioned behavior reflects the formation of a directional associative link from the memory of one stimulus (e.g., a visual stimulus) to another (e.g., food), with learning stopping when there is no error between the prediction generated by the visual stimulus and what happens next (e.g., food). This consensus fails to anticipate the results that we report here. In our experiments with rats, we find that arranging predictive (visual stimulus→food) and nonpredictive (food→visual stimulus) relationships produces marked and sustained changes in conditioned behaviors when the visual stimulus is presented alone. Moreover, the type of relationship affects (1) the distribution of conditioned behaviors related to the properties of both food (called goal-tracking) and the visual stimulus (called sign-tracking) and (2) when in the visual stimulus, these two behaviors are evident. These results represent an impetus for a fundamental shift in how Pavlovian conditioning is interpreted: animals learn about the relationship between two stimuli irrespective of the order in which they are presented, but they exhibit this knowledge in different ways. This interpretation and our new results are captured by a recent model of Pavlovian conditioning,12,13 HeiDI, and both are consistent with the need for animals to represent the fact that the impact of a cause (e.g., the ingestion of nutrients or the bite of a predator) can be felt before or after the cause has been perceived.

Retrieval-mediated learning involving episodes requires synaptic plasticity in the hippocampus.

A novel association can form between two memories even when the events to which they correspond are not physically present. For example, once an integrated memory has formed that binds the (when, where, and what) components of an event together, this memory can be triggered by one of its components, and updated with coincident information in the environment. The neural basis of this form of retrieval-mediated learning is unknown. Here, we show, for the first time, that NMDA receptors in the rat hippocampus are required for retrieval-mediated learning involving episodes, but not for the expression of such learning or for retrieval-mediated learning involving simple associations between the components of episodes. These findings provide a novel insight into learning processes that serve the desirable function of integrating stored information with new information, but whose operation might also provide a substrate for some of the cognitive symptoms of schizophrenia and Alzheimer's disease.

Associative change in Pavlovian conditioning: A reappraisal.

Robert A. Rescorla changed how Pavlovian conditioning was studied and interpreted. His empirical contributions were fundamental and theoretically driven. One involved testing a central tenet of the model that he developed with Allan R. Wagner. The Rescorla-Wagner learning rule uses a pooled error term to determine changes in a directional association between the representations of the conditioned stimulus (CS) and unconditioned stimulus (US). This learning rule predicts that 2 equally salient CSs (A and B) will undergo equivalent associative change when they are conditioned in compound (i.e., AB→US). Rescorla's results suggested that this was not the case (e.g., Rescorla, 2000). Here, we show that these results can be reconciled with a model that uses a learning rule with a pooled error term once that rule is applied equivalently to all of the stimuli presented on a given trial, and the resulting reciprocal associations (directly and indirectly) contribute to performance. This model, called HeiDI, integrates several features of Rescorla's research and theorizing while addressing an issue that he recognized required further analysis: how learning is translated into performance. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Higher-order conditioning: A critical review and computational model.

Higher-order conditioning results from a simple training procedure: Pairing two relatively neutral conditioned stimuli, A and X, allows properties separately conditioned to X (e.g., through pairing it with an unconditioned stimulus, US) to be evident during A. The phenomenon extends the range of ways in which Pavlovian conditioned responding can be expressed and increases its translational relevance. Given this relevance and the wealth of available behavioral analysis, it is a surprisingly underdeveloped territory for formal theoretical analysis. Here, we first provide a critical review of two (informal) classes of account for higher-order conditioning that reflect either: (a) processes that are analogous to Pavlovian conditioning, but involving associatively activated representations (e.g., A→US); or (b) the formation of an associative chain (e.g., A→X, and X→US). Our review first identifies fundamental theoretical and empirical challenges to both classes of account. We then develop a new computational model of higher-order conditioning that meets the challenges identified by showing: how reciprocal associations between A, X, and the US are formed and affect performance; and how the similarity of stimuli, their traces, and associatively retrieved representations modulate this process. The model generates a wealth of novel predictions, providing a platform for further empirical and theoretical analysis. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Perceptual learning after rapidly alternating exposure to taste compounds: Assessment with different indices of generalization.

Exposure to two similar stimuli (AX and BX; e.g., two tastes) reduces the extent to which a conditioned response later established to BX generalizes to AX. This example of perceptual learning is more evident when AX and BX are exposed in an alternating manner (AX, BX, AX, BX,…) than when AX and BX occur in separate blocks (e.g., AX, AX,…BX, BX,…). We examined in male rats (N = 126) the impact of rapid alternation to AX and BX on generalization of a taste aversion from BX to AX. Experiment 1 showed that such alternating presentations (with 5-min intervals between AX and BX) reduced generalization relative to blocked exposure; but only as assessed by consumption levels and not by lick cluster size (an index of hedonic reactions). Experiment 1 also showed that the nature of exposure did not affect how A influenced performance to a novel conditioned taste, Y. Experiment 2 replicated the pattern of results involving the different influences of rapidly alternating and blocked exposure on generalization from BX to AX, and showed that this effect was only evident when rats received access to water during the 5-min intervals between AX and BX. These results reinforce parallels between perceptual learning effects in rats and humans, both at empirical and theoretical levels. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Higher-Order Conditioning: What Is Learnt and How it Is Expressed.

Pairing a neutral conditioned stimulus (CS) with a motivationally significant unconditioned stimulus (US) results in the CS coming to elicit conditioned responses (CRs). The widespread significance and translational value of Pavlovian conditioning are increased by the fact that pairing two neutral CSs (A and X) enables conditioning with X to affect behavior to A. There are two traditional informal accounts of such higher-order conditioning, which build on more formal associative analyses of Pavlovian conditioning. But, higher-order conditioning and Pavlovian conditioning have characteristics that are beyond these accounts: Notably, the two are influenced in different ways by the same experimental manipulations, and both generate conditioned responses that do not reflect the US per se. Here, we present a formal analysis that sought to address these characteristics.

Individual differences in the nature of conditioned behavior across a conditioned stimulus: Adaptation and application of a model.

Pavlovian conditioning procedures produce marked individual differences in the form of conditioned behavior. For example, when rats are given conditioning trials in which the temporary insertion of a lever into an operant chamber (the conditioned stimulus, CS) is paired with the delivery of food (the unconditioned stimulus, US), they exhibit knowledge of the lever-food relationship in different ways. For some rats (known as sign-trackers), interactions with the lever dominate, while for others (goal-trackers), approaching the food well dominates. A formal model of Pavlovian conditioning (HeiDI) attributes such individual differences in behavior to variations in the perceived salience of the CS and US. An application of the model in which the perceived salience of the CS declines (i.e., adapts) across its duration predicts changes in these individual differences within the presentation of the CS: The sign-tracking bias is predicted to decline and goal-tracking bias is predicted to increase across the presentation of a lever. The accuracy of these predictions was confirmed through analysis of archival data from female and male rats. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Elaboration of a model of Pavlovian learning and performance: HeiDI.

The model elaborated here adapts the influential pooled error term, first described by Wagner and Rescorla (Rescorla & Wagner, 1972; Wagner & Rescorla, 1972), to govern the formation of reciprocal associations between any pair of stimuli that are presented on a given trial. In the context of Pavlovian conditioning, these stimuli include various conditioned and unconditioned stimuli. This elaboration enables the model to deal with cue competition phenomena, including the relative validity effect, and evidence implicating separate error terms and attentional processes in association formation. The model also includes a performance rule, which provides a natural basis for (individual) variation in the strength and nature of conditioned behaviors that are observed in Pavlovian conditioning procedures. The new model thereby begins to address theoretical and empirical issues that were apparent when the Rescorla-Wagner model was first described, together with research inspired by the model over the ensuing 50 years. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Individual variation in the vigor and form of Pavlovian conditioned responses: Analysis of a model system.

Pavlovian conditioning results in individual variation in the vigor and form of acquired behaviors. Here, we describe a general-process model of associative learning (HeiDI; How excitation and inhibition determine ideo-motion) that provides an analysis for such variation together with a range of other important group-level phenomena. The model takes as its starting point the idea that pairings of a conditioned stimulus (CS) and an unconditioned stimulus (US) result in the formation of reciprocal associations between their central representations. The asymptotic values of these associations and the rate at which these are reached are held to be influenced by the perceived salience of the CS (αCS) and US (ßUS). Importantly, whether this associative knowledge is exhibited in behavior that reflects the properties of the CS (e.g., sign-tracking) or US (e.g., goal-tracking) is also influenced by the relative values of αCS and ßUS. In this way, HeiDI provides an analysis for both quantitative and qualitative individual differences generated by Pavlovian conditioning procedures.

HeiDI: A model for Pavlovian learning and performance with reciprocal associations.

Associative treatments of how Pavlovian conditioning affects conditioned behavior are rudimentary: A simple ordinal mapping is held to exist between the strength of an association (V) between a conditioned stimulus (CS) and an unconditioned stimulus (US; i.e., VCS-US) and conditioned behavior in a given experimental preparation. The inadequacy of this simplification is highlighted by recent studies that have taken multiple measures of conditioned behavior: Different measures of conditioned behavior provide the basis for drawing opposite conclusions about VCS-US across individual animals. Here, we develop a simple model involving reciprocal associations between the CS and US (VCS-US and VUS-CS) that simulates these qualitative individual differences in conditioned behavior. The new model, HeiDI (How excitation and inhibition Determine Ideo-motion), enables a broad range of phenomena to be accommodated, which are either beyond the scope of extant models or require them to appeal to additional (learning) processes. It also provides an impetus for new lines of inquiry and generates novel predictions. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Cyfip1 haploinsufficient rats show white matter changes, myelin thinning, abnormal oligodendrocytes and behavioural inflexibility.

The biological basis of the increased risk for psychiatric disorders seen in 15q11.2 copy number deletion is unknown. Previous work has shown disturbances in white matter tracts in human carriers of the deletion. Here, in a novel rat model, we recapitulated low dosage of the candidate risk gene CYFIP1 present within the 15q11.2 interval. Using diffusion tensor imaging, we first showed extensive white matter changes in Cyfip1 mutant rats, which were most pronounced in the corpus callosum and external capsule. Transmission electron microscopy showed that these changes were associated with thinning of the myelin sheath in the corpus callosum. Myelin thinning was independent of changes in axon number or diameter but was associated with effects on mature oligodendrocytes, including aberrant intracellular distribution of myelin basic protein. Finally, we demonstrated effects on cognitive phenotypes sensitive to both disruptions in myelin and callosal circuitry.

Representation in development: from a model system to some general processes.

The view that filial imprinting might serve as a useful model system for studying the neurobiological basis of memory was inspired, at least in part, by a simple idea: acquired filial preferences reflect the formation of a memory or representation of the imprinting object itself, as opposed to the change in the efficacy of stimulus-response pathways, for example. We provide a synthesis of the evidence that supports this idea; and show that the processes of memory formation observed in filial imprinting find surprisingly close counterparts in other species, including our own.