Instrumental learning and behavioral persistence in children with attention-deficit/hyperactivity-disorder: does reinforcement frequency matter?

BACKGROUND: Prominent theoretical accounts of attention-deficit/hyperactivity-disorder (ADHD) hypothesize that reinforcement learning deficits underlie symptoms of ADHD. The Dynamic Developmental Theory and the Dopamine Transfer Deficit hypothesis assume impairments in both the acquisition and extinction of behavior, especially when learning occurs under partial (non-continuous) reinforcement, and subsequently the Partial Reinforcement Extinction Effect (PREE). Few studies have evaluated instrumental learning in ADHD and the results are inconsistent. The current study investigates instrumental learning under partial and continuous reinforcement schedules and subsequent behavioral persistence when reinforcement is withheld (extinction) in children with and without ADHD. METHODS: Large well-defined samples of children with ADHD (n = 93) and typically developing (TD) children (n = 73) completed a simple instrumental learning task. The children completed acquisition under continuous (100%) or partial (20%) reinforcement, followed by a 4-min extinction phase. Two-way (diagnosis by condition) ANOVAs evaluated responses needed to reach the learning criterion during acquisition, and target and total responses during extinction. RESULTS: Children with ADHD required more trials to reach criterion compared to TD children under both continuous and partial reinforcement. After partial reinforcement, children with ADHD executed fewer target responses during extinction than TD children. Children with ADHD executed more responses than TD children during extinction, irrespective of learning condition. CONCLUSIONS: The findings demonstrate general difficulties in instrumental learning in ADHD, that is, slower learning irrespective of reinforcement schedule. They also show faster extinction following learning under partial reinforcement in those with ADHD, that is, a diminished PREE. Children with ADHD executed more responses during extinction. Results are theoretically important, with clinical implications for understanding and managing learning difficulties in those with ADHD, as they suggest poorer reinforcement learning and lower behavioral persistence.

Overnight fasting affects avoidance learning and relief.

Objectives: prolonged fasting influences threat and reward processing, two fundamental systems underpinning adaptive behaviors. In animals, overnight fasting sensitizes the mesolimbic-dopaminergic activity governing avoidance, reward, and fearextinction learning. Despite evidence that overnight fasting may also affect reward and fear learning in humans, effects on human avoidance learning have not been studied yet. Here, we examined the effects of 16 h-overnight fasting on instrumental avoidance and relief from threat omission.Methods: to this end, 50 healthy women were randomly assigned to a Fasting (N = 25) or a Re-feeding group (N = 25) and performed an Avoidance-Relief Task.Results: we found that fasting decreases unnecessary avoidance during signaled safety; this effect was mediated via a reduction in relief pleasantness during signaled absence of threat. A fasting-induced reduction in relief was also found during fear extinction learning.Discussion: we conclude that fasting optimizes avoidance and safety learning. Future studies should test whether these effects also hold for anxious individuals.

Lack of drug-induced post-retrieval amnesia for auditory fear memories in rats.

BACKGROUND: Long-term memory formation is generally assumed to involve the permanent storage of recently acquired memories, making them relatively insensitive to disruption, a process referred to as memory consolidation. However, when retrieved under specific circumstances, consolidated fear memories are thought to return to a labile state, thereby opening a window for modification (e.g., attenuation) of the memory. Several interventions during a critical time frame after this destabilization seem to be able to alter the retrieved memory, for example by pharmacologically interfering with the restabilization process, either by direct protein synthesis inhibition or indirectly, using drugs that can be safely administered in patients (e.g., propranolol). Here, we find that, contrary to expectations, systemic pharmacological manipulations in auditory fear-conditioned rats do not lead to drug-induced post-retrieval amnesia. RESULTS: In a series of well-powered auditory fear conditioning experiments (four with propranolol, 10 mg/kg, two with rapamycin, 20-40 mg/kg, one with anisomycin, 150 mg/kg and cycloheximide, 1.5 mg/kg), we found no evidence for reduced cued fear memory expression during a drug-free test in adult male Sprague-Dawley rats that had previously received a systemic drug injection upon retrieval of the tone fear memory. All experiments used standard fear conditioning and reactivation procedures with freezing as the behavioral read-out (conceptual or exact replications of published reports) and common pharmacological agents. Additional tests confirmed that the applied drug doses and administration routes were effective in inducing their conventional effects on expression of fear (propranolol, acutely), body weight (rapamycin, anisomycin, cycloheximide), and consolidation of extinction memories (cycloheximide). CONCLUSIONS: In contrast with previously published studies, we did not find evidence for drug-induced post-retrieval amnesia, underlining that this effect, as well as its clinical applicability, may be considerably more constrained and less readily reproduced than what the current literature would suggest.

Perceptual errors are related to shifts in generalization of conditioned responding.

Studies of perceptual generalization have recently demonstrated a close relationship between stimulus perception and conditioned responding, suggesting that incorrect stimulus perception might account for certain characteristics of generalization gradients. In this study, we investigated whether common phenomena, such as the area and peak shift in conditioned responding, relate to perceptual errors. After a differential conditioning procedure, in which one circle was paired with the presentation of an aversive picture whereas a different-sized circle was not, we combined a generalization test with a three-alternative forced-choice perceptual categorization task where participants had to indicate on every trial whether the presented circle was one of the two circles from the conditioning phase or a different one, after which US-expectancy ratings were collected. The typical peak and area shift were observed when conditioned responses were plotted on a physical dimension. However, when stimulus perception was incorporated generalization gradients diverged from the typical gradient. Both the area and peak shift largely disappeared when accounting for perceptual errors. These findings demonstrate the need to incorporate perceptual mechanisms in associative models.

Sleep deprivation increases threat beliefs in human fear conditioning.

Sleep disturbances and anxiety disorders exhibit high comorbidity levels, but it remains unclear whether sleep problems are causes or consequences of increased anxiety. To experimentally probe the aetiological role of sleep disturbances in anxiety, we investigated in healthy participants how total sleep deprivation influences fear expression in a conditioning paradigm. In a fear conditioning procedure, one face stimulus (conditioned stimulus [CS+]) was paired with electric shock, whereas another face stimulus was not (unpaired stimulus [CS-]). Fear expression was tested the next morning using the two face stimuli from the training phase and a generalization stimulus (i.e. a morph between the CS+ and CS- stimuli). Between fear conditioning and test, participants were either kept awake in the laboratory for 12 hr (n = 20) or had a night of sleep at home (n = 20). Irrespective of stimulus type, subjective threat expectancies, but not skin conductance responses, were enhanced after sleep deprivation, relative to regular sleep. These results suggest that sleep disturbances may play a role in anxiety disorders by increasing perceived threat.

Limited replicability of drug-induced amnesia after contextual fear memory retrieval in rats.

With the ultimate goal of investigating boundary conditions for post-reactivation amnesia, we set out to replicate studies in which systemic, post-reactivation administration of midazolam, propranolol, or cycloheximide resulted in amnesia for contextual fear memories. Our experiments involved conceptual as well as exact replications of previously published studies. In most of our experiments, we adopted a procedure that conformed to the standard 3-day protocol typically used in the literature, with contextual fear conditioning on day 1, unreinforced re-exposure to the conditioning context followed by systemic injection of the amnestic drug on day 2, and a memory retention test on day 3. Given the plethora of successful studies with large effects sizes and the absence of any failed replications in the literature, we were surprised to find that we were generally unable to replicate those findings. Our results suggest that post-reactivation amnesia by systemic drug administration in rats is more difficult to obtain than what would be expected based on published empirical reports. At present, it remains unclear which conditions determine the success of this procedure.

Manipulating affective state influences conditioned appetitive responses.

Affective states influence how individuals process information and behave. Some theories predict emotional congruency effects (e.g. preferential processing of negative information in negative affective states). Emotional congruency should theoretically obstruct the learning of reward associations (appetitive learning) and their ability to guide behaviour under negative mood. Two studies tested the effects of the induction of a negative affective state on appetitive Pavlovian learning, in which neutral stimuli were associated with chocolate (Experiment 1) or alcohol (Experiment 2) rewards. In both experiments, participants showed enhanced approach tendencies towards predictors of reward after a negative relative to a positive performance feedback manipulation. This increase was related to a reduction in positive affect in Experiment 1 only. No effects of the manipulation on conditioned reward expectancies, craving, or consumption were observed. Overall, our findings support the idea of counter-regulation, rather than emotional congruency effects. Negative affective states might therefore serve as a vulnerability factor for addiction, through increasing conditioned approach tendencies.

A comparison of behavioral and pharmacological interventions to attenuate reactivated fear memories.

Two experiments using rats in a contextual fear memory preparation compared two approaches to reduce conditioned fear: (1) pharmacological reconsolidation blockade and (2) reactivation-plus-extinction training. In Experiment 1, we explored different combinations of reactivation-plus-extinction parameters to reduce conditioned fear and attenuate reacquisition. In Experiment 2, memory reactivation was followed by extinction training or administration of midazolam (MDZ) (vs. vehicle) to reduce conditioned fear and attenuate spontaneous recovery. We found both treatments to be equally effective in both experiments. This study suggests that parameters leading to memory destabilization during reactivation are critical to observe long-lasting effects of MDZ or reactivation plus extinction.

Paul Eelen: Reflections on Life and Work.

This manuscript is part of a special issue to commemorate professor Paul Eelen, who passed away on August 21, 2016. Paul was a clinically oriented scientist, for whom learning principles (Pavlovian or operant) were more than salivary responses and lever presses. His expertise in learning psychology and his enthusiasm to translate this knowledge to clinical practice inspired many inside and outside academia. Several of his original writings were in the Dutch language. Instead of editing a special issue with contributions of colleagues and friends, we decided to translate a selection of his manuscripts to English to allow wide access to his original insights and opinions. Even though the manuscripts were written more than two decades ago, their content is surprisingly contemporary. This introductory article presents a reflection on Paul's career and legacy and introduces the selected manuscripts that are part of this special issue.

A preregistered, direct replication attempt of the retrieval-extinction effect in cued fear conditioning in rats.

In 2009, Monfils and colleagues proposed a behavioral procedure that was said to result in a permanent attenuation of a previously established fear memory, thereby precluding a possible return of fear after extinction (Monfils, Cowansage, Klann, & LeDoux, 2009). By presenting a single retrieval trial one hour before standard extinction training, they found an enduring reduction of fear. The retrieval-extinction procedure holds great clinical potential, particularly for anxiety patients, but the findings are not undisputed, and several conceptual replications have failed to reproduce the effect. These failures have largely been attributed to small procedural differences. This preregistered study is the first endeavor to exactly replicate three key experiments of the original report by Monfils et al. (2009), thereby gauging the robustness of their seminal findings. Despite adhering to the original procedures as closely as possible, we did not find any evidence for reduced return of fear with the retrieval-extinction procedure relative to regular extinction training, as assessed through spontaneous recovery, reinstatement and renewal. Behavior of animals in the control condition (extinction only) was comparable to that in the original studies and provided an adequate baseline to reveal differences with the retrieval-extinction condition. Our null findings indicate that the effect sizes in the original paper may have been inflated and question the legitimacy of previously proposed moderators of the retrieval-extinction effect. We argue that direct experimental evaluation of purported moderators of the retrieval-extinction effect will be key to shed more light on its nature and prerequisites.

Memory Reconsolidation Interference as an Emerging Treatment for Emotional Disorders: Strengths, Limitations, Challenges, and Opportunities.

Experimental research on emotional memory reconsolidation interference, or the induction of amnesia for previously established emotional memory, has a long tradition, but the potential of that research for the development of novel interventions to treat psychological disorders has been recognized only recently. Here we provide an overview of basic research and clinical studies on emotional memory reconsolidation interference. We point out specific advantages of interventions based on memory reconsolidation interference over traditional treatment for emotional disorders. We also explain how findings from basic research suggest limitations and challenges to clinical translation that may help to understand why clinical trials have met with mixed success so far and how their success can be increased. In closing, we preview new intervention approaches beyond the induction of amnesia that the phenomenon of memory reconsolidation may afford for alleviating the burden imposed by emotional memories and comment on theoretical controversies regarding the nature of memory reconsolidation.

Fearing shades of grey: individual differences in fear responding towards generalisation stimuli.

Individual differences in fear generalisation have been proposed to play a role in the aetiology and/or maintenance of anxiety disorders, but few data are available to directly support that claim. The research that is available has focused mostly on generalisation of peripheral and central physiological fear responses. Far less is known about the generalisation of avoidance, the behavioural component of fear. In two experiments, we evaluated how neuroticism, a known vulnerability factor for anxiety, modulates an array of fear responses, including avoidance tendencies, towards generalisation stimuli (GS). Participants underwent differential fear conditioning, in which one conditioned stimulus (CS+) was repeatedly paired with an aversive outcome (shock; unconditioned stimulus, US), whereas another was not (CS-). Fear generalisation was observed across measures in Experiment 1 (US expectancy and evaluative ratings) and Experiment 2 (US expectancy, evaluative ratings, skin conductance, startle responses, safety behaviours), with overall highest responding to the CS+, lowest to the CS- and intermediate responding to the GSs. Neuroticism had very little impact on fear generalisation (but did affect GS recognition rates in Experiment 1), in line with the idea that fear generalisation is largely an adaptive process.

Bending rules: the shape of the perceptual generalisation gradient is sensitive to inference rules.

Generalising what is learned about one stimulus to other but perceptually related stimuli is a basic behavioural phenomenon. We evaluated whether a rule learning mechanism may serve to explain such generalisation. To this end, we assessed whether inference rules communicated through verbal instructions affect generalisation. Expectancy ratings, but not valence ratings, proved sensitive to this manipulation. In addition to revealing a role for inference rules in generalisation, our study has clinical implications as well. More specifically, we argue that targeting inference rules might prove to be an effective strategy to affect the excessive generalisation that is often observed in psychopathology.

Feature- versus rule-based generalization in rats, pigeons and humans.

Humans can spontaneously create rules that allow them to efficiently generalize what they have learned to novel situations. An enduring question is whether rule-based generalization is uniquely human or whether other animals can also abstract rules and apply them to novel situations. In recent years, there have been a number of high-profile claims that animals such as rats can learn rules. Most of those claims are quite weak because it is possible to demonstrate that simple associative systems (which do not learn rules) can account for the behavior in those tasks. Using a procedure that allows us to clearly distinguish feature-based from rule-based generalization (the Shanks-Darby procedure), we demonstrate that adult humans show rule-based generalization in this task, while generalization in rats and pigeons was based on featural overlap between stimuli. In brief, when learning that a stimulus made of two components ("AB") predicts a different outcome than its elements ("A" and "B"), people spontaneously abstract an opposites rule and apply it to new stimuli (e.g., knowing that "C" and "D" predict one outcome, they will predict that "CD" predicts the opposite outcome). Rats and pigeons show the reverse behavior-they generalize what they have learned, but on the basis of similarity (e.g., "CD" is similar to "C" and "D", so the same outcome is predicted for the compound stimulus as for the components). Genuinely rule-based behavior is observed in humans, but not in rats and pigeons, in the current procedure.

A Bayesian hierarchical diffusion model decomposition of performance in Approach-Avoidance Tasks.

Common methods for analysing response time (RT) tasks, frequently used across different disciplines of psychology, suffer from a number of limitations such as the failure to directly measure the underlying latent processes of interest and the inability to take into account the uncertainty associated with each individual's point estimate of performance. Here, we discuss a Bayesian hierarchical diffusion model and apply it to RT data. This model allows researchers to decompose performance into meaningful psychological processes and to account optimally for individual differences and commonalities, even with relatively sparse data. We highlight the advantages of the Bayesian hierarchical diffusion model decomposition by applying it to performance on Approach-Avoidance Tasks, widely used in the emotion and psychopathology literature. Model fits for two experimental data-sets demonstrate that the model performs well. The Bayesian hierarchical diffusion model overcomes important limitations of current analysis procedures and provides deeper insight in latent psychological processes of interest.

Prediction error demarcates the transition from retrieval, to reconsolidation, to new learning.

Although disrupting reconsolidation is promising in targeting emotional memories, the conditions under which memory becomes labile are still unclear. The current study showed that post-retrieval changes in expectancy as an index for prediction error may serve as a read-out for the underlying processes engaged by memory reactivation. Minor environmental changes define whether retrieval induces memory reconsolidation or the initiation of a new memory trace even before fear extinction can be observed.

Blocking in human causal learning is affected by outcome assumptions manipulated through causal structure.

Additivity-related assumptions have been proven to modulate blocking in human causal learning. Typically, these assumptions are manipulated by means of pretraining phases (including exposure to different outcome magnitudes), or through explicit instructions. In two experiments, we used a different approach that involved neither pretraining nor instructional manipulations. Instead, we manipulated the causal structure in which the cues were embedded, thereby appealing directly to the participants' prior knowledge about causal relations and how causes would add up to yield stronger outcomes. Specifically, in our "different-system" condition, the participants should assume that the outcomes would add up, whereas in our "same-system" condition, a ceiling effect would prevent such an assumption. Consistent with our predictions, Experiment 1 showed that, when two cues from separate causal systems were combined, the participants did expect a stronger outcome on compound trials, and blocking was found, whereas when the cues belonged to the same causal system, the participants did not expect a stronger outcome on compound trials, and blocking was not observed. The results were partially replicated in Experiment 2, in which this pattern was found when the cues were tested for the second time. This evidence supports the claim that prior knowledge about the nature of causal relations can affect human causal learning. In addition, the fact that we did not manipulate causal assumptions through pretraining renders the results hard to account for with associative theories of learning.

What a relief! The pleasure of threat avoidance.

Relief, a pleasurable experience, is often triggered by successful threat avoidance. Although relief is regarded as the positive reinforcer for avoidance behavior, its rewarding nature remains to be demonstrated. In our study, 50 participants responded to cues associated with different magnitudes of monetary values or electrical stimuli. Successful responses to those cues resulted in monetary gains (i.e., rewards) or omissions of electrical stimulation (i.e., relief), followed by a pleasantness rating scale. We also measured physiological arousal via skin conductance. As expected, we found that for reward and relief similarly, higher magnitudes elicited more successful responses, higher pleasantness ratings, and higher skin conductance responses. Moreover, differential reward/relief response patterns predicted later choices between reward and relief cues. These findings indicate that relief induced by threat omissions is functionally equivalent to receiving a reward, confirming that relief is a positive reinforcer for threat avoidance behaviors, which provides a new theoretical perspective on the learning process of active threat avoidance. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A history of avoidance does not impact extinction learning in male rats.

Pervasive avoidance is one of the central symptoms of all anxiety-related disorders. In treatment, avoidance behaviors are typically discouraged because they are assumed to maintain anxiety. Yet, it is not clear if engaging in avoidance is always detrimental. In this study, we used a platform-mediated avoidance task to investigate the influence of avoidance history on extinction learning in male rats. Our results show that having the opportunity to avoid during fear acquisition training does not significantly influence the extinction of auditory-cued fear in rats subjected to this platform-mediated avoidance procedure, which constitutes a realistic approach/avoidance conflict. This holds true irrespective of whether or not avoidance was possible during the extinction phase. This suggests that imposing a realistic cost on avoidance behavior prevents the adverse effects that avoidance has been claimed to have on extinction. However, avoidance does not appear to have clear positive effects on extinction learning nor on retention either.

No harmful effect of propranolol administered prior to fear memory extinction in rats and humans.

Exposure therapy is an evidence-based treatment option for anxiety-related disorders. Many patients also take medication that could, in principle, affect exposure therapy efficacy. Clinical and laboratory evidence indeed suggests that benzodiazepines may have detrimental effects. Large clinical trials with propranolol, a common beta-blocker, are currently lacking, but several preclinical studies do indicate impaired establishment of safety memories. Here, we investigated the effects of propranolol given prior to extinction training in 9 rat studies (N = 215) and one human study (N = 72). A Bayesian meta-analysis of our rat studies provided strong evidence against propranolol-induced extinction memory impairment during a drug-free test, and the human study found no significant difference with placebo. Two of the rat studies actually suggested a small beneficial effect of propranolol. Lastly, two rat studies with a benzodiazepine (midazolam) group provided some evidence for a harmful effect on extinction memory, i.e., impaired extinction retention. In conclusion, our midazolam findings are in line with prior literature (i.e., an extinction retention impairment), but this is not the case for the 10 studies with propranolol. Our data thus support caution regarding the use of benzodiazepines during exposure therapy, but argue against a harmful effect of propranolol on extinction learning.