Dual-system free-operant avoidance: Extension of a theory.

Our theory of positively reinforced free-operant behavior (Perez & Dickinson, 2020) assumes that responding is controlled by two systems. One system is sensitive to the correlation between response and reinforcement rates and controls goal-directed behavior, whereas a habitual system learns by reward prediction error. We present an extension of this theory to the aversive domain that explains why free-operant avoidance responding increases with both the experienced rate of negative reinforcement and the difference between this rate and that programmed by the avoidance schedule. The theory also assumes that the habitual component is reinforced by the acquisition of aversive inhibitory properties by the feedback stimuli generated by responding, which then act as safety signals that reinforce habit performance. Our analysis suggests that the distinction between habitual and goal-directed control of rewarded behavior can also be applied to the aversive domain. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Evaluation of Insertion Energy as Novel Parameter for Dental Implant Stability.

Insertion energy has been advocated as a novel measure for primary implant stability, but the effect of implant length, diameter, or surgical protocol remains unclear. Twenty implants from one specific bone level implant system were placed in layered polyurethane foam measuring maximum insertion torque, torque-time curves, and primary stability using resonance frequency analysis (RFA). Insertion energy was calculated as area under torque-time curve applying the trapezoidal formula. Statistical analysis was based on analysis of variance, Tukey honest differences tests and Pearson's product moment correlation tests (α = 0.05). Implant stability (p = 0.01) and insertion energy (p < 0.01) differed significantly among groups, while maximum insertion torque did not (p = 0.17). Short implants showed a significant decrease in implant stability (p = 0.01), while reducing implant diameter did not cause any significant effect. Applying the drilling protocol for dense bone resulted in significantly increased insertion energy (p = 0.02) but a significant decrease in implant stability (p = 0.04). Insertion energy was not found to be a more reliable parameter for evaluating primary implant stability when compared to maximum insertion torque and resonance frequency analysis.

A theory of actions and habits: The interaction of rate correlation and contiguity systems in free-operant behavior.

Contemporary theories of instrumental performance assume that responding can be controlled by 2 behavioral systems, 1 goal-directed that encodes the outcome of an action, and 1 habitual that reinforces the response strength of the same action. Here we present a model of free-operant behavior in which goal-directed control is determined by the correlation between the rates of the action and the outcome whereas the total prediction error generated by contiguous reinforcement by the outcome controls habitual response strength. The outputs of these two systems summate to generate a total response strength. This cooperative model addresses the difference in the behavioral impact of ratio and interval schedules, the transition from goal-directed to habitual control with extended training, the persistence of goal-directed control under choice procedures and following extinction, among other phenomena. In these respects, this dual-system model is unique in its account of free-operant behavior. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Human instrumental performance in ratio and interval contingencies: A challenge for associative theory.

Associative learning theories regard the probability of reinforcement as the critical factor determining responding. However, the role of this factor in instrumental conditioning is not completely clear. In fact, free-operant experiments show that participants respond at a higher rate on variable ratio than on variable interval schedules even though the reinforcement probability is matched between the schedules. This difference has been attributed to the differential reinforcement of long inter-response times (IRTs) by interval schedules, which acts to slow responding. In the present study, we used a novel experimental design to investigate human responding under random ratio (RR) and regulated probability interval (RPI) schedules, a type of interval schedule that sets a reinforcement probability independently of the IRT duration. Participants responded on each type of schedule before a final choice test in which they distributed responding between two schedules similar to those experienced during training. Although response rates did not differ during training, the participants responded at a lower rate on the RPI schedule than on the matched RR schedule during the choice test. This preference cannot be attributed to a higher probability of reinforcement for long IRTs and questions the idea that similar associative processes underlie classical and instrumental conditioning.

A re-examination of responding on ratio and regulated-probability interval schedules.

The higher response rates observed on ratio than on matched interval reward schedules has been attributed to the differential reinforcement of longer inter-response times (IRTs) on the interval contingency. Some data, however, seem to contradict this hypothesis, showing that the difference is still observed when the role of IRT reinforcement is neutralized by using a regulated-probability interval schedule (RPI). Given the mixed evidence for these predictions, we re-examined this hypothesis by training three groups of rats to lever press under ratio, interval and RPI schedules across two phases while matching reward rates within triads. At the end of the first phase, the master ratio and RPI groups responded at similar rates. In the second phase, an interval group yoked to the same master ratio group of the first phase responded at a lower rate than the RPI group. Post-hoc analysis showed comparable reward rates for master and yoked schedules. The experienced response-outcome rate correlations were likewise similar and approached zero as training progressed. We discuss these results in terms of a contemporary dual-system model of instrumental conditioning.

Region-Specific Summation Patterns Inform the Role of Cortical Areas in Selecting Motor Plans.

Given an instruction regarding which effector to move and what location to move to, simply adding the effector and spatial signals together will not lead to movement selection. For this, a nonlinearity is required. Thresholds, for example, can be used to select a particular response and reject others. Here we consider another useful nonlinearity, a supralinear multiplicative interaction. To help select a motor plan, spatial and effector signals could multiply and thereby amplify each other. Such an amplification could constitute one step within a distributed network involved in response selection, effectively boosting one response while suppressing others. We therefore asked whether effector and spatial signals sum supralinearly for planning eye versus arm movements from the parietal reach region (PRR), the lateral intraparietal area (LIP), the frontal eye field (FEF), and a portion of area 5 (A5) lying just anterior to PRR. Unlike LIP neurons, PRR, FEF, and, to a lesser extent, A5 neurons show a supralinear interaction. Our results suggest that selecting visually guided eye versus arm movements is likely to be mediated by PRR and FEF but not LIP.

Avoidance behavior: a free-operant lever-press avoidance task for the assessment of the effects of safety signals.

This protocol details a free-operant avoidance paradigm that has been developed to evaluate the relative contribution of different sources of reinforcement of avoidance behavior that may play an important role in the development and maintenance of human anxiety disorders. The task enables the assessment of the effects of safety cues that signal a period free from danger on lever-press avoidance behavior. Avoidance behavior trained using this protocol has been shown to be sensitive to both behavioral and pharmacological manipulations and has been optimized so that it takes approximately 1 month for rats to perform at high levels of stable avoidance responding.

Safety signals as instrumental reinforcers during free-operant avoidance.

Safety signals provide "relief" through predicting the absence of an aversive event. At issue is whether these signals also act as instrumental reinforcers. Four experiments were conducted using a free-operant lever-press avoidance paradigm in which each press avoided shock and was followed by the presentation of a 5-sec auditory safety signal. When given a choice between two levers in Experiment 1, both avoiding shock, rats preferentially responded on the lever that produced the safety signal as feedback, even when footshock was omitted. Following avoidance training with a single lever in Experiment 2, removal of the signal led to a decrease in avoidance responses and an increase in responses during the safety period normally denoted by the signal. These behavioral changes demonstrate the dual conditioned reinforcing and fear inhibiting properties of the safety signal. The associative processes that support the reinforcing properties of a safety signal were tested using a novel revaluation procedure. Prior experience of systemic morphine during safety signal presentations resulted in an increased rate of avoidance responses to produce the safety signal during a drug-free extinction test, a finding not seen with d-amphetamine in Experiment 3. Morphine revaluation of the safety signal was repeated in Experiment 4 followed by a drug-free extinction test in which responses did not produce the signal for the first 10 min of the session. Instrumental avoidance in the absence of the signal was shown to be insensitive to prior signal revaluation, suggesting that the signal reinforces free-operant avoidance behavior through a habit-like mechanism.

Mechanisms of motivation-cognition interaction: challenges and opportunities.

Recent years have seen a rejuvenation of interest in studies of motivation-cognition interactions arising from many different areas of psychology and neuroscience. The present issue of Cognitive, Affective, & Behavioral Neuroscience provides a sampling of some of the latest research from a number of these different areas. In this introductory article, we provide an overview of the current state of the field, in terms of key research developments and candidate neural mechanisms receiving focused investigation as potential sources of motivation-cognition interaction. However, our primary goal is conceptual: to highlight the distinct perspectives taken by different research areas, in terms of how motivation is defined, the relevant dimensions and dissociations that are emphasized, and the theoretical questions being targeted. Together, these distinctions present both challenges and opportunities for efforts aiming toward a more unified and cross-disciplinary approach. We identify a set of pressing research questions calling for this sort of cross-disciplinary approach, with the explicit goal of encouraging integrative and collaborative investigations directed toward them.

Free-operant avoidance behavior by rats after reinforcer revaluation using opioid agonists and D-amphetamine.

The associative processes that support free-operant instrumental avoidance behavior are still unknown. We used a revaluation procedure to determine whether the performance of an avoidance response is sensitive to the current value of the aversive, negative reinforcer. Rats were trained on an unsignaled, free-operant lever press avoidance paradigm in which each response avoided or escaped shock and produced a 5 s feedback stimulus. The revaluation procedure consisted of noncontingent presentations of the shock in the absence of the lever either paired or unpaired with systemic morphine and in a different cohort with systemic d-amphetamine. Rats were then tested drug free during an extinction test. In both the d-amphetamine and morphine groups, pairing of the drug and shock decreased subsequent avoidance responding during the extinction test, suggesting that avoidance behavior was sensitive to the current incentive value of the aversive negative reinforcer. Experiment 2 used central infusions of D-Ala(2), NMe-Phe(4), Gly-ol(5)]-enkephalin (DAMGO), a mu-opioid receptor agonist, in the periacqueductal gray and nucleus accumbens shell to revalue the shock. Infusions of DAMGO in both regions replicated the effects seen with systemic morphine. These results are the first to demonstrate the impact of revaluation of an aversive reinforcer on avoidance behavior using pharmacological agents, thereby providing potential therapeutic targets for the treatment of avoidance behavior symptomatic of anxiety disorders.

Ketamine effects on memory reconsolidation favor a learning model of delusions.

Delusions are the persistent and often bizarre beliefs that characterise psychosis. Previous studies have suggested that their emergence may be explained by disturbances in prediction error-dependent learning. Here we set up complementary studies in order to examine whether such a disturbance also modulates memory reconsolidation and hence explains their remarkable persistence. First, we quantified individual brain responses to prediction error in a causal learning task in 18 human subjects (8 female). Next, a placebo-controlled within-subjects study of the impact of ketamine was set up on the same individuals. We determined the influence of this NMDA receptor antagonist (previously shown to induce aberrant prediction error signal and lead to transient alterations in perception and belief) on the evolution of a fear memory over a 72 hour period: they initially underwent Pavlovian fear conditioning; 24 hours later, during ketamine or placebo administration, the conditioned stimulus (CS) was presented once, without reinforcement; memory strength was then tested again 24 hours later. Re-presentation of the CS under ketamine led to a stronger subsequent memory than under placebo. Moreover, the degree of strengthening correlated with individual vulnerability to ketamine's psychotogenic effects and with prediction error brain signal. This finding was partially replicated in an independent sample with an appetitive learning procedure (in 8 human subjects, 4 female). These results suggest a link between altered prediction error, memory strength and psychosis. They point to a core disruption that may explain not only the emergence of delusional beliefs but also their persistence.

Associative (not Hebbian) learning and the mirror neuron system.

The associative sequence learning (ASL) hypothesis suggests that sensorimotor experience plays an inductive role in the development of the mirror neuron system, and that it can play this crucial role because its effects are mediated by learning that is sensitive to both contingency and contiguity. The Hebbian hypothesis proposes that sensorimotor experience plays a facilitative role, and that its effects are mediated by learning that is sensitive only to contiguity. We tested the associative and Hebbian accounts by computational modelling of automatic imitation data indicating that MNS responsivity is reduced more by contingent and signalled than by non-contingent sensorimotor training (Cook et al. [7]). Supporting the associative account, we found that the reduction in automatic imitation could be reproduced by an existing interactive activation model of imitative compatibility when augmented with Rescorla-Wagner learning, but not with Hebbian or quasi-Hebbian learning. The work argues for an associative, but against a Hebbian, account of the effect of sensorimotor training on automatic imitation. We argue, by extension, that associative learning is potentially sufficient for MNS development.

Resolution of outcome-induced response conflict by humans after extended training.

Studies of incongruent discrimination learning, where the outcome event of one response acts as the discriminative stimulus for the opposite response, suggest that humans rely on habitual stimulus-response (S-R) associations when outcome-response (O-R) associations would cause response conflict. Here, two experiments were conducted to investigate the robustness of this habitual strategy. In Experiment 1, we found that extensive instrumental discrimination training supported learning about the incongruent R → O contingencies, as assessed by an outcome devaluation test. Differential representations of the stimulus and the (associatively retrieved) outcome may have allowed for goal-directed incongruent performance. Experiment 2 failed to provide evidence for this possibility; direct presentation as well as associative retrieval of the incongruent events (by Pavlovian stimuli) activated the response that was associated with each event in its role of stimulus as opposed to outcome. We did find that participants successfully acquired explicit knowledge of the incongruent contingencies, which raises the possibility that propositional encoding allowed them to overcome the response conflict caused by O-R associations. Alternative associative and propositional accounts of successful goal-directed incongruent performance with extensive training will be discussed.

Associative learning and animal cognition.

Associative learning plays a variety of roles in the study of animal cognition from a core theoretical component to a null hypothesis against which the contribution of cognitive processes is assessed. Two developments in contemporary associative learning have enhanced its relevance to animal cognition. The first concerns the role of associatively activated representations, whereas the second is the development of hybrid theories in which learning is determined by prediction errors, both directly and indirectly through associability processes. However, it remains unclear whether these developments allow associative theory to capture the psychological rationality of cognition. I argue that embodying associative processes within specific processing architectures provides mechanisms that can mediate psychological rationality and illustrate such embodiment by discussing the relationship between practical reasoning and the associative-cybernetic model of goal-directed action.

Contextual modulation of mirror and countermirror sensorimotor associations.

Automatic imitation-the unintended copying of observed actions-is thought to be a behavioral product of the mirror neuron system (MNS). Evidence that the MNS develops through associative learning comes from previous research showing that automatic imitation is attenuated by countermirror training, in which the observation of one action is paired contingently with the execution of a different action. If the associative account of the MNS is correct, countermirror training should show context specificity, because countermirror associations render action stimuli ambiguous, and ambiguity promotes contextual control. Two experiments that confirm this prediction are reported. In Experiment 1 we found less residual automatic imitation when human participants were tested in their countermirror training context. In Experiment 2, sensorimotor training where participants made action responses to novel abstract stimuli was insensitive to the same context manipulation, confirming that the former result was not a procedural artifact. Contextual modulation may enable the MNS to function effectively in spite of the fact that action observation often excites multiple conflicting MNS responses.

Rational action selection in 1½- to 3-year-olds following an extended training experience.

Previous studies failed to find evidence for rational action selection in children under 2 years of age. The current study investigated whether younger children required more training to encode the relevant causal relationships. Children between 1½ and 3 years of age were trained over two sessions to perform actions on a touch-sensitive screen to obtain video clips as outcomes. Subsequently, a visual habituation procedure was employed to devalue one of the training outcomes. As in previous studies, 2- and 3-year-olds chose actions associated with an expected valued outcome significantly more often during a subsequent choice test. Moreover, analysis of children's first responses in the post-devaluation test revealed evidence of rational action selection even in the youngest age group (18-23 months). Consistent with dual-process accounts of action control, the findings support the view that the ability to make rational action choices develops gradually.

Choice and goal-directed behavior in preschool children.

Preschool children (3-4 years old) were trained to perform two actions to gain different outcomes, in the form of video clips from different cartoons, before one of these outcomes was devalued by noncontingent exposure. The effect of outcome devaluation was subsequently assessed in an extinction test by giving children the opportunity to perform both actions in the absence of any outcomes. When the two actions were trained concurrently, performance during the test was modulated by outcome value and children showed a preference for the action trained with the currently valued outcome. By contrast, when each action was trained separately on different trials, test performance was insensitive to outcome devaluation. These effects of the training schedules are interpreted in terms of dual-process theories of action control.

Sense of agency, associative learning, and schizotypy.

Despite the fact that the role of learning is recognised in empirical and theoretical work on sense of agency (SoA), the nature of this learning has, rather surprisingly, received little attention. In the present study we consider the contribution of associative mechanisms to SoA. SoA can be measured quantitatively as a temporal linkage between voluntary actions and their external effects. Using an outcome blocking procedure, it was shown that training action-outcome associations under conditions of increased surprise augmented this temporal linkage. Moreover, these effects of surprise were correlated with schizotypy scores, suggesting that individual differences in higher level experiences are related to associative learning and to its impact on SoA. These results are discussed in terms of models of SoA, and our understanding of disrupted SoA in certain disorders.

Stress, habits, and drug addiction: a psychoneuroendocrinological perspective.

It is well known that stress is a significant risk factor for the development of drug addiction and addiction relapse. Remarkably, the cognitive processes involved in the effects of stress on addictive behavior remain poorly understood. Here it is proposed that stress-induced changes in the neural circuits controlling instrumental action provide a potential mechanism by which stress affects the development of addiction and relapse vulnerability. Instrumental action can be controlled by two anatomically distinct systems: a goal-directed system that involves learning of action-outcome associations, and a habit system that learns stimulus-response associations. The transition from initial voluntary drug use to subsequent involuntary, compulsive drug use represents a switch from goal-directed to habitual control of action. Recent evidence indicates that this switch from goal-directed to habit action can be prompted by stress and stress hormones. We argue (i) that acute stressors reinstate habitual responding to drug-related cues and thus trigger relapse to addictive behavior, and (ii) that prolonged or repeated stress may accelerate the transition from voluntary to involuntary drug use and thus promote the development of addiction. The suggested mechanism encompasses cognitive processes that may contribute to the effects of stress on addictive behavior and could have important implications for the treatment of addiction and the prevention of relapse.

Habitual versus goal-directed action control in Parkinson disease.

This study presents the first direct investigation of the hypothesis that dopamine depletion of the dorsal striatum in mild Parkinson disease leads to impaired stimulus-response habit formation, thereby rendering behavior slow and effortful. However, using an instrumental conflict task, we show that patients are able to rely on direct stimulus-response associations when a goal-directed strategy causes response conflict, suggesting that habit formation is not impaired. If anything our results suggest a disease severity-dependent deficit in goal-directed behavior. These results are discussed in the context of Parkinson disease and the neurobiology of habitual and goal-directed behavior.