The Computational and Neural Bases of Context-Dependent Learning.

Flexible behavior requires the creation, updating, and expression of memories to depend on context. While the neural underpinnings of each of these processes have been intensively studied, recent advances in computational modeling revealed a key challenge in context-dependent learning that had been largely ignored previously: Under naturalistic conditions, context is typically uncertain, necessitating contextual inference. We review a theoretical approach to formalizing context-dependent learning in the face of contextual uncertainty and the core computations it requires. We show how this approach begins to organize a large body of disparate experimental observations, from multiple levels of brain organization (including circuits, systems, and behavior) and multiple brain regions (most prominently the prefrontal cortex, the hippocampus, and motor cortices), into a coherent framework. We argue that contextual inference may also be key to understanding continual learning in the brain. This theory-driven perspective places contextual inference as a core component of learning.

Estrogen receptors mediate estradiol's effect on sensitization and CPP to cocaine in female rats: role of contextual cues.

Preclinical studies show that estradiol enhances sensitization to cocaine in females by mechanisms not fully understood. These studies consistently show that ovariectomized (OVX) rats exhibit little or no sensitization to cocaine compared to OVX rats administered estradiol. In this study we varied the dose of cocaine (10, 15, and 30mg/kg), the length of cocaine treatment (from 5 to 10days) and the context of cocaine injections to determine if these factors play a role on estradiol's effects on cocaine sensitization. Because OVX rats are hormonally compromised, they are not representative of the natural state of the animal, and thus the physiological context of these studies remains unclear. To address this issue, we blocked ERs in gonadally intact females by icv administration of the antiestrogen ICI-182,780. Varying the dose or length of exposure to cocaine does not alter estradiol's effect on cocaine sensitization. In contrast, a highly context-dependent sensitization protocol results in robust sensitization even in OVX rats. Interestingly, using this protocol, sensitization in OVX rats diminished with time, suggesting that estradiol is necessary for the maintenance of cocaine sensitization. Blocking brain ERs with ICI completely abolishes the development and expression of cocaine sensitization in gonadally intact female rats, even when tested in a highly context-dependent sensitization protocol. Given these findings, we propose that activation of brain ERs is required for the development and maintenance of sensitization and CPP.

How much to trust the senses: likelihood learning.

Our brain often needs to estimate unknown variables from imperfect information. Our knowledge about the statistical distributions of quantities in our environment (called priors) and currently available information from sensory inputs (called likelihood) are the basis of all Bayesian models of perception and action. While we know that priors are learned, most studies of prior-likelihood integration simply assume that subjects know about the likelihood. However, as the quality of sensory inputs change over time, we also need to learn about new likelihoods. Here, we show that human subjects readily learn the distribution of visual cues (likelihood function) in a way that can be predicted by models of statistically optimal learning. Using a likelihood that depended on color context, we found that a learned likelihood generalized to new priors. Thus, we conclude that subjects learn about likelihood.

Renewal in human fear conditioning: A systematic review and meta-analysis.

Renewal is a 'return of fear' manipulation in human fear conditioning to investigate learning processes underlying anxiety and trauma. Even though renewal paradigms are widely used, no study has compared the strength of different renewal paradigms. We conduct a systematic review (N = 80) and meta-analysis (N = 23) of human fear conditioning studies assessing renewal. Our analysis shows that the classic ABA design is the most effective paradigm, compared to ABC and ABBA designs. We present evidence that conducting extinction in multiple contexts and increasing the similarity between acquisition and extinction contexts reduce renewal. Furthermore, we show that additional cues can be used as safety and 'protection from extinction' cues. The review shows that alcohol weakens the extinction process and that older adults appear less sensitive to context changes and thus show less renewal. The large variability in approaches to study renewal in humans suggests that standardisation of fear conditioning procedures across laboratories would be of great benefit to the field.

Contextual inference in learning and memory.

Context is widely regarded as a major determinant of learning and memory across numerous domains, including classical and instrumental conditioning, episodic memory, economic decision-making, and motor learning. However, studies across these domains remain disconnected due to the lack of a unifying framework formalizing the concept of context and its role in learning. Here, we develop a unified vernacular allowing direct comparisons between different domains of contextual learning. This leads to a Bayesian model positing that context is unobserved and needs to be inferred. Contextual inference then controls the creation, expression, and updating of memories. This theoretical approach reveals two distinct components that underlie adaptation, proper and apparent learning, respectively referring to the creation and updating of memories versus time-varying adjustments in their expression. We review a number of extensions of the basic Bayesian model that allow it to account for increasingly complex forms of contextual learning.

Dynamic top-down biasing implements rapid adaptive changes to individual movements.

Complex behaviors depend on the coordinated activity of neural ensembles in interconnected brain areas. The behavioral function of such coordination, often measured as co-fluctuations in neural activity across areas, is poorly understood. One hypothesis is that rapidly varying co-fluctuations may be a signature of moment-by-moment task-relevant influences of one area on another. We tested this possibility for error-corrective adaptation of birdsong, a form of motor learning which has been hypothesized to depend on the top-down influence of a higher-order area, LMAN (lateral magnocellular nucleus of the anterior nidopallium), in shaping moment-by-moment output from a primary motor area, RA (robust nucleus of the arcopallium). In paired recordings of LMAN and RA in singing birds, we discovered a neural signature of a top-down influence of LMAN on RA, quantified as an LMAN-leading co-fluctuation in activity between these areas. During learning, this co-fluctuation strengthened in a premotor temporal window linked to the specific movement, sequential context, and acoustic modification associated with learning. Moreover, transient perturbation of LMAN activity specifically within this premotor window caused rapid occlusion of pitch modifications, consistent with LMAN conveying a temporally localized motor-biasing signal. Combined, our results reveal a dynamic top-down influence of LMAN on RA that varies on the rapid timescale of individual movements and is flexibly linked to contexts associated with learning. This finding indicates that inter-area co-fluctuations can be a signature of dynamic top-down influences that support complex behavior and its adaptation.

Supplementary dataset of context-dependent conditioned responding to an alcohol-predictive cue in female and male rats.

This supplementary dataset is supportive of the research article entitled 'The role of context on responding to an alcohol-predictive cue in female and male rats' [1]. This article describes the raw data pertaining to the behaviour of male and female rats during intermittent to ethanol and Pavlovian conditioning training and testing procedures. Specifically, the dataset describes the alcohol consumption and ingested-dose of ethanol during home-cage ethanol exposure, as well as the conditioned responding during Pavlovian discrimination training, a test assessing the effect of context on responding to an alcohol-predict cue in the absence of alcohol, and a reinstatement test assessing the effect of context on conditioned responding to an extinguished alcohol-predictive cue.

Gaze-specific motor memories for hand-reaching.

Numerous studies have proposed that our adaptive motor behaviors depend on learning a map between sensory information and limb movement,1-3 called an "internal model." From this perspective, how the brain represents internal models is a critical issue in motor learning, especially regarding their association with spatial frames processed in motor planning.4,5 Extensive experimental evidence suggests that during planning stages for visually guided hand reaching, the brain transforms visual target representations in gaze-centered coordinates to motor commands in limb coordinates, via hand-target vectors in workspace coordinates.6-9 While numerous studies have intensively investigated whether the learning for reaching occurs in workspace or limb coordinates,10-20 the association of the learning with gaze coordinates still remains untested.21 Given the critical role of gaze-related spatial coding in reaching planning,22-26 the potential role of gaze states for learning is worth examining. Here, we show that motor memories for reaching are separately learned according to target location in gaze coordinates. Specifically, two opposing visuomotor rotations, which normally interfere with each other, can be simultaneously learned when each is associated with reaching to a foveal target and peripheral one. We also show that this gaze-dependent learning occurs in force-field adaptation. Furthermore, generalization of gaze-coupled reach adaptation is limited across central, right, and left visual fields. These results suggest that gaze states are available in the formation and recall of multiple internal models for reaching. Our findings provide novel evidence that a gaze-dependent spatial representation can provide a spatial coordinate framework for context-dependent motor learning.

Task-selective place cells show behaviorally driven dynamics during learning and stability during memory recall.

Decades of work propose that hippocampal activity supports internal representation of learned experiences and contexts, allowing individuals to form long-term memories and quickly adapt behavior to changing environments. However, recent studies insinuate hippocampal representations can drift over time, raising the question: how could the hippocampus hold stable memories when activity of its neuronal maps fluctuates? We hypothesized that task-dependent hippocampal maps set by learning rules and structured attention stabilize as a function of behavioral performance. To test this, we imaged hippocampal CA1 pyramidal neurons during learning and memory recall phases of a new task where mice use odor cues to navigate between two reward zones. Across learning, both orthogonal and overlapping task-dependent place maps form rapidly, discriminating trial context with strong correlation to behavioral performance. Once formed, task-selective place maps show increased long-term stability during memory recall phases. We conclude that memory demand and attention stabilize hippocampal activity to maintain contextually rich spatial representations.

Context-Dependent Memory of Motor Sequences.

To examine influences of context changes between encoding and retrieval of motor sequences, we varied a number of encoding and retrieval features in a two lists approach. Participants consecutively learned two sets of three-finger movements at two different computer working places, all enacted with fingers of the right hand. We varied keyboard and display orientation, stimuli, background color, response keys, position of the hand, and the used PC between the two sets. A final free recall test comprised either the same context features as present during study of the first item set or the ones present during study of the second item set or novel test context features. Results showed significant differences in overall recall performance between test conditions, indicating that context features of study episodes guided retrieval of motor sequences. In addition, the number of recalled items varied as a function of output position. Test context elements comprising context features of the first item set study episode were associated with initially lower but subsequently nearby stable recall performance, whereas test features comprising context elements of the second item set study episode were associated with initially higher and subsequently decreasing recall performance. This implies that a context reinstatement for list-1 items during the test phase does not immediately enhance accessibility of those items. However, access is subsequently facilitated over the course of retrieval attempts.

Variable Neural Contributions to Explicit and Implicit Learning During Visuomotor Adaptation.

We routinely make fine motor adjustments to maintain optimal motor performance. These adaptations have been attributed to both implicit, error-based mechanisms, and explicit, strategy-based mechanisms. However, little is known about the neural basis of implicit vs. explicit learning. Here, we aimed to use anodal transcranial direct current stimulation (tDCS) to probe the relationship between different brain regions and learning mechanisms during a visuomotor adaptation task in humans. We hypothesized that anodal tDCS over the cerebellum (CB) should increase implicit learning while anodal tDCS over the dorsolateral prefrontal cortex (dlPFC), a region associated with higher-level cognition, should facilitate explicit learning. Using a horizontal visuomotor adaptation task that measures explicit/implicit contributions to learning (Taylor et al., 2014), we found that dlPFC stimulation significantly improved performance compared to the other groups, and weakly increased explicit learning. However, CB stimulation had no effects on either target error or implicit learning. Previous work showed variable CB stimulation effects only on a vertical visuomotor adaptation task (Jalali et al., 2017), so in Experiment 2, we conducted the same study using a vertical context to see if we could find effects of CB stimulation. We found only weak effects of CB stimulation on target error and implicit learning, and now the dlPFC effect did not replicate. To resolve this discrepancy, in Experiment 3, we examined the effect of context (vertical vs. horizontal) on implicit and explicit contributions and found that individuals performed significantly worse and used greater implicit learning in the vertical screen condition compared to the horizontal screen condition. Across all experiments, however, there was high inter-individual variability, with strong influences of a few individuals, suggesting that these effects are not consistent across individuals. Overall, this work provides preliminary support for the idea that different neural regions can be engaged to improve visuomotor adaptation, but shows that each region's effects are highly context-dependent and not clearly dissociable from one another. This holds implications especially in neurorehabilitation, where an intact neural region could be engaged to potentially compensate if another region is impaired. Future work should examine factors influencing interindividual variability during these processes.

The Effects of Sensory Manipulations on Motor Behavior: From Basic Science to Clinical Rehabilitation.

Modifying sensory aspects of the learning environment can influence motor behavior. Although the effects of sensory manipulations on motor behavior have been widely studied, there still remains a great deal of variability across the field in terms of how sensory information has been manipulated or applied. Here, the authors briefly review and integrate the literature from each sensory modality to gain a better understanding of how sensory manipulations can best be used to enhance motor behavior. Then, they discuss 2 emerging themes from this literature that are important for translating sensory manipulation research into effective interventions. Finally, the authors provide future research directions that may lead to enhanced efficacy of sensory manipulations for motor learning and rehabilitation.

Discrete Circuits Support Generalized versus Context-Specific Vocal Learning in the Songbird.

Motor skills depend on the reuse of individual gestures in multiple sequential contexts (e.g., a single phoneme in different words). Yet optimal performance requires that a given gesture be modified appropriately depending on the sequence in which it occurs. To investigate the neural architecture underlying such context-dependent modifications, we studied Bengalese finch song, which, like speech, consists of variable sequences of "syllables." We found that when birds are instructed to modify a syllable in one sequential context, learning generalizes across contexts; however, if unique instruction is provided in different contexts, learning is specific for each context. Using localized inactivation of a cortical-basal ganglia circuit specialized for song, we show that this balance between generalization and specificity reflects a hierarchical organization of neural substrates. Primary motor circuitry encodes a core syllable representation that contributes to generalization, while top-down input from cortical-basal ganglia circuitry biases this representation to enable context-specific learning.

A Biologically Plausible Architecture of the Striatum to Solve Context-Dependent Reinforcement Learning Tasks.

Basal ganglia circuit is an important subcortical system of the brain thought to be responsible for reward-based learning. Striatum, the largest nucleus of the basal ganglia, serves as an input port that maps cortical information. Microanatomical studies show that the striatum is a mosaic of specialized input-output structures called striosomes and regions of the surrounding matrix called the matrisomes. We have developed a computational model of the striatum using layered self-organizing maps to capture the center-surround structure seen experimentally and explain its functional significance. We believe that these structural components could build representations of state and action spaces in different environments. The striatum model is then integrated with other components of basal ganglia, making it capable of solving reinforcement learning tasks. We have proposed a biologically plausible mechanism of action-based learning where the striosome biases the matrisome activity toward a preferred action. Several studies indicate that the striatum is critical in solving context dependent problems. We build on this hypothesis and the proposed model exploits the modularity of the striatum to efficiently solve such tasks.

Context-Dependent Learning in People With Parkinson's Disease.

Context-dependent learning is a phenomenon in which people demonstrate superior performance in the context in which they originally learned a skill but perform less well in a novel context. This study investigated context-dependent learning in people with Parkinson's disease (PD) and age-matched nondisabled adults. All participants practiced 3 finger sequences, each embedded within a unique context (colors and locations on a computer screen). One day after practice, the participants were tested either under the sequence-context associations remained the same as during practice, or the sequence-context associations were changed (SWITCH). Compared with nondisabled adults, people with PD demonstrated significantly greater decrement in performance (especially movement time) under the SWITCH condition, suggesting that individuals with PD are more context dependent than nondisabled adults.

What determines the impact of context on sequential action?

In the current study we build on earlier observations that memory-based sequential action is better in the original learning context than in other contexts. We examined whether changes in the perceptual context have differential impact across distinct processing phases (preparation versus execution of a motor chunk) within an ongoing movement sequence. Participants were trained on two discrete keying sequences, each of which was systematically presented in its own unique color during a practice session with either limited or extended practice. In a subsequent test session, sequences were performed with the same, with reversed, and with completely novel sequence-specific colors. The results confirm context-dependence in sequential action, the relevance of practice for its development, and its selective expression for the preparation but not the execution of highly practiced motor chunks. As such, the current study provides novel insights into the determinants of context-dependent sequential action. We finish by outlining the overall status of context-dependence in sequential motor behavior, and specify a general working model.