Error-independent effect of sensory uncertainty on motor learning when both feedforward and feedback control processes are engaged.

Integrating sensory information during movement and adapting motor plans over successive movements are both essential for accurate, flexible motor behaviour. When an ongoing movement is off target, feedback control mechanisms update the descending motor commands to counter the sensed error. Over longer timescales, errors induce adaptation in feedforward planning so that future movements become more accurate and require less online adjustment from feedback control processes. Both the degree to which sensory feedback is integrated into an ongoing movement and the degree to which movement errors drive adaptive changes in feedforward motor plans have been shown to scale inversely with sensory uncertainty. However, since these processes have only been studied in isolation from one another, little is known about how they are influenced by sensory uncertainty in real-world movement contexts where they co-occur. Here, we show that sensory uncertainty may impact feedforward adaptation of reaching movements differently when feedback integration is present versus when it is absent. In particular, participants gradually adjust their movements from trial-to-trial in a manner that is well characterised by a slow and consistent envelope of error reduction. Riding on top of this slow envelope, participants exhibit large and abrupt changes in their initial movement vectors that are strongly correlated with the degree of sensory uncertainty present on the previous trial. However, these abrupt changes are insensitive to the magnitude and direction of the sensed movement error. These results prompt important questions for current models of sensorimotor learning under uncertainty and open up new avenues for future exploration in the field.

Camera realignment imposes a cost on laparoscopic performance.

There is an unresolved question about whether realigned visual feedback is beneficial or costly to laparoscopic task performance. We provide evidence that camera realignment imposes a reliable cost on performance across both naive controls and experienced surgeons. This finding clarifies an important ongoing discussion in the literature about the effects of camera realignment, which could inform the strategies that laparoscopic surgeons use in the operating room.

Enhanced visuomotor learning and generalization in expert surgeons.

Although human motor learning has been intensively studied for many decades, it remains unknown whether group differences are present in expert cohorts that must routinely cope with and learn new visuomotor mappings such as expert minimally invasive surgeons. We found that expert surgeons compensate for a visuomotor perturbation more rapidly than naive controls. Modelling indicates that these differences in expert behavioural performance reflects greater trial-to-trial retention, as opposed to greater trial-to-trial learning rate. We also found that surgeons generalize to novel reach directions more broadly than controls, a result which was subsequently confirmed by our modelling. In general, our findings show that minimally invasive surgeons exhibit enhanced visuomotor learning and spatial generalization.

Increased cognitive load enables unlearning in procedural category learning.

Interventions for drug abuse and other maladaptive habitual behaviors may yield temporary success but are often fragile and relapse is common. This implies that current interventions do not erase or substantially modify the representations that support the underlying addictive behavior-that is, they do not cause true unlearning. One example of an intervention that fails to induce true unlearning comes from Crossley, Ashby, and Maddox (2013, Journal of Experimental Psychology: General), who reported that a sudden shift to random feedback did not cause unlearning of category knowledge obtained through procedural systems, and they also reported results suggesting that this failure is because random feedback is noncontingent on behavior. These results imply the existence of a mechanism that (a) estimates feedback contingency and (b) protects procedural learning from modification when feedback contingency is low (i.e., during random feedback). This article reports the results of an experiment in which increasing cognitive load via an explicit dual task during the random feedback period facilitated unlearning. This result is consistent with the hypothesis that the mechanism that protects procedural learning when feedback contingency is low depends on executive function. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Trial-by-trial switching between procedural and declarative categorization systems.

Considerable evidence suggests that human category learning recruits multiple memory systems. A popular assumption is that procedural memory is used to form stimulus-to-response mappings, whereas declarative memory is used to form and test explicit rules about category membership. The multiple systems framework has been successful in motivating and accounting for a broad array of empirical observations over the past 20 years. Even so, only a couple of studies have examined how the different categorization systems interact. Both previous studies suggest that switching between explicit and procedural responding is extremely difficult. But they leave unanswered the critical questions of whether trial-by-trial system switching is possible, and if so, whether it is qualitatively different than trial-by-trial switching between two explicit tasks. The experiment described in this article addressed these questions. The results (1) confirm that effective trial-by-trial system switching, although difficult, is possible; (2) suggest that switching between tasks mediated by different memory systems is more difficult than switching between two declarative memory tasks; and (3) point to a serious shortcoming of current category-learning theories.

Hierarchical control of procedural and declarative category-learning systems.

Substantial evidence suggests that human category learning is governed by the interaction of multiple qualitatively distinct neural systems. In this view, procedural memory is used to learn stimulus-response associations, and declarative memory is used to apply explicit rules and test hypotheses about category membership. However, much less is known about the interaction between these systems: how is control passed between systems as they interact to influence motor resources? Here, we used fMRI to elucidate the neural correlates of switching between procedural and declarative categorization systems. We identified a key region of the cerebellum (left Crus I) whose activity was bidirectionally modulated depending on switch direction. We also identified regions of the default mode network (DMN) that were selectively connected to left Crus I during switching. We propose that the cerebellum-in coordination with the DMN-serves a critical role in passing control between procedural and declarative memory systems.

Trial-by-trial identification of categorization strategy using iterative decision-bound modeling.

Identifying the strategy that participants use in laboratory experiments is crucial in interpreting the results of behavioral experiments. This article introduces a new modeling procedure called iterative decision-bound modeling (iDBM), which iteratively fits decision-bound models to the trial-by-trial responses generated from single participants in perceptual categorization experiments. The goals of iDBM are to identify: (1) all response strategies used by a participant, (2) changes in response strategy, and (3) the trial number at which each change occurs. The new method is validated by testing its ability to identify the response strategies used in noisy simulated data. The benchmark simulation results show that iDBM is able to detect and identify strategy switches during an experiment and accurately estimate the trial number at which the strategy change occurs in low to moderate noise conditions. The new method is then used to reanalyze data from Ell and Ashby (2006). Applying iDBM revealed that increasing category overlap in an information-integration category learning task increased the proportion of participants who abandoned explicit rules, and reduced the number of training trials needed to abandon rules in favor of a procedural strategy. Finally, we discuss new research questions made possible through iDBM.

Credit assignment in movement-dependent reinforcement learning.

When a person fails to obtain an expected reward from an object in the environment, they face a credit assignment problem: Did the absence of reward reflect an extrinsic property of the environment or an intrinsic error in motor execution? To explore this problem, we modified a popular decision-making task used in studies of reinforcement learning, the two-armed bandit task. We compared a version in which choices were indicated by key presses, the standard response in such tasks, to a version in which the choices were indicated by reaching movements, which affords execution failures. In the key press condition, participants exhibited a strong risk aversion bias; strikingly, this bias reversed in the reaching condition. This result can be explained by a reinforcement model wherein movement errors influence decision-making, either by gating reward prediction errors or by modifying an implicit representation of motor competence. Two further experiments support the gating hypothesis. First, we used a condition in which we provided visual cues indicative of movement errors but informed the participants that trial outcomes were independent of their actual movements. The main result was replicated, indicating that the gating process is independent of participants' explicit sense of control. Second, individuals with cerebellar degeneration failed to modulate their behavior between the key press and reach conditions, providing converging evidence of an implicit influence of movement error signals on reinforcement learning. These results provide a mechanistically tractable solution to the credit assignment problem.

Declarative strategies persist under increased cognitive load.

When humans simultaneously execute multiple tasks, performance on individual tasks suffers. Complementing existing theories, this article poses a novel question to investigate interactions between memory systems supporting multi-tasking performance: When a primary and dual task both recruit declarative learning and memory systems, does simultaneous performance of both tasks impair primary task performance because learning in the declarative system is reduced, or because control of the primary task is passed to slower procedural systems? To address this question, participants were trained on either a perceptual categorization task believed to rely on procedural learning or one of three different categorization tasks believed to rely on declarative learning. Task performance was examined with and without a simultaneous dual task thought to recruit working memory and executive attention. To test whether the categories were learned procedurally or declaratively, the response keys were switched after a learning criterion had been reached. Large impairments in performance after switching the response keys are taken to indicate procedural learning, and small impairments are taken to indicate declarative learning. Our results suggest that the declarative memory categorization tasks (regardless of task difficulty) were learned by declarative systems, regardless of whether they were learned under dual-task conditions.

Expanding the role of striatal cholinergic interneurons and the midbrain dopamine system in appetitive instrumental conditioning.

The basal ganglia are a collection of subcortical nuclei thought to underlie a wide variety of vertebrate behavior. Although a great deal is known about the functional and physiological properties of the basal ganglia, relatively few models have been formally developed that have been tested against both behavioral and physiological data. Our previous work (Ashby FG, Crossley MJ. J Cogn Neurosci 23: 1549-1566, 2011) showed that a model grounded in the neurobiology of the basal ganglia could account for basic single-neuron recording data, as well as behavioral phenomena such as fast reacquisition that constrain models of conditioning. In this article we show that this same model accounts for a variety of appetitive instrumental conditioning phenomena, including the partial reinforcement extinction (PRE) effect, rapid and slowed reacquisition following extinction, and renewal of previously extinguished instrumental responses by environmental context cues.

Savings upon Re-Aiming in Visuomotor Adaptation.

Sensorimotor adaptation has traditionally been viewed as a purely error-based process. There is, however, growing appreciation for the idea that performance changes in these tasks can arise from the interplay of error-based adaptation with other learning processes. The challenge is to specify constraints on these different processes, elucidating their respective contributions to performance, as well as the manner in which they interact. We address this question by exploring constraints on savings, the phenomenon in which people show faster performance gains when the same learning task is repeated. In a series of five experiments, we demonstrate that error-based learning associated with sensorimotor adaptation does not contribute to savings. Instead, savings reflects improvements in action selection, rather than motor execution. SIGNIFICANCE STATEMENT: Savings is the phenomenon in which people show faster relearning of a previously forgotten memory. In the motor learning domain, this phenomenon has been a puzzle for learning models that operate exclusively on error-based learning processes. We demonstrate, in a series of experiments, that savings selectively reflects improvements in action selection: Participants are more adept in invoking an appropriate aiming strategy when presented with a previously experienced perturbation. Indeed, improvements in action selection appear to be the sole source of savings in visuomotor adaptation tasks. We observe no evidence of savings in implicit error-based adaptation.

Multiple stages of learning in perceptual categorization: evidence and neurocomputational theory.

Virtually all current theories of category learning assume that humans learn new categories by gradually forming associations directly between stimuli and responses. In information-integration category-learning tasks, this purported process is thought to depend on procedural learning implemented via dopamine-dependent cortical-striatal synaptic plasticity. This article proposes a new, neurobiologically detailed model of procedural category learning that, unlike previous models, does not assume associations are made directly from stimulus to response. Rather, the traditional stimulus-response (S-R) models are replaced with a two-stage learning process. Multiple streams of evidence (behavioral, as well as anatomical and fMRI) are used as inspiration for the new model, which synthesizes evidence of multiple distinct cortical-striatal loops into a neurocomputational theory. An experiment is reported to test a priori predictions of the new model that: (1) recovery from a full reversal should be easier than learning new categories equated for difficulty, and (2) reversal learning in procedural tasks is mediated within the striatum via dopamine-dependent synaptic plasticity. The results confirm the predictions of the new two-stage model and are incompatible with existing S-R models.

Procedural learning during declarative control.

There is now abundant evidence that human learning and memory are governed by multiple systems. As a result, research is now turning to the next question of how these putative systems interact. For instance, how is overall control of behavior coordinated, and does learning occur independently within systems regardless of what system is in control? Behavioral, neuroimaging, and neuroscience data are somewhat mixed with respect to these questions. Human neuroimaging and animal lesion studies suggest independent learning and are mostly agnostic with respect to control. Human behavioral studies suggest active inhibition of behavioral output but have little to say regarding learning. The results of two perceptual category-learning experiments are described that strongly suggest that procedural learning does occur while the explicit system is in control of behavior and that this learning might be just as good as if the procedural system was controlling the response. These results are consistent with the idea that declarative memory systems inhibit the ability of the procedural system to access motor output systems but do not prevent procedural learning.

Context-dependent savings in procedural category learning.

Environmental context can have a profound influence on the efficacy of intervention protocols designed to eliminate undesirable behaviors. This is clearly seen in drug rehabilitation clinics where patients often relapse soon after leaving the context of the treatment facility. A similar pattern is commonly observed in controlled laboratory studies of context-dependent savings in instrumental conditioning, where simply placing an animal back into the original conditioning chamber can renew an extinguished instrumental response. Surprisingly, context-dependent savings in human procedural learning has not been carefully examined in the laboratory. Here, we provide the first known empirical demonstration of context-dependent savings in a perceptual categorization task known to recruit procedural learning. We also present a computational account of these savings using a biologically detailed model in which a key role is played by cholinergic interneurons in the striatum.

Erasing the engram: the unlearning of procedural skills.

Huge amounts of money are spent every year on unlearning programs--in drug-treatment facilities, prisons, psychotherapy clinics, and schools. Yet almost all of these programs fail, since recidivism rates are high in each of these fields. Progress on this problem requires a better understanding of the mechanisms that make unlearning so difficult. Much cognitive neuroscience evidence suggests that an important component of these mechanisms also dictates success on categorization tasks that recruit procedural learning and depend on synaptic plasticity within the striatum. A biologically detailed computational model of this striatal-dependent learning is described (based on Ashby & Crossley, 2011). The model assumes that a key component of striatal-dependent learning is provided by interneurons in the striatum called the tonically active neurons (TANs), which act as a gate for the learning and expression of striatal-dependent behaviors. In their tonically active state, the TANs prevent the expression of any striatal-dependent behavior. However, they learn to pause in rewarding environments and thereby permit the learning and expression of striatal-dependent behaviors. The model predicts that when rewards are no longer contingent on behavior, the TANs cease to pause, which protects striatal learning from decay and prevents unlearning. In addition, the model predicts that when rewards are partially contingent on behavior, the TANs remain partially paused, leaving the striatum available for unlearning. The results from 3 human behavioral studies support the model predictions and suggest a novel unlearning protocol that shows promising initial signs of success.

Implicit and explicit categorization: a tale of four species.

Categorization is essential for survival, and it is a widely studied cognitive adaptation in humans and animals. An influential neuroscience perspective differentiates in humans an explicit, rule-based categorization system from an implicit system that slowly associates response outputs to different regions of perceptual space. This perspective is being extended to study categorization in other vertebrate species, using category tasks that have a one-dimensional, rule-based solution or a two-dimensional, information-integration solution. Humans, macaques, and capuchin monkeys strongly dimensionalize perceptual stimuli and learn rule-based tasks more quickly. In sharp contrast, pigeons learn these two tasks equally quickly. Pigeons represent a cognitive system in which the commitment to dimensional analysis and category rules was not strongly made. Their results may reveal the character of the ancestral vertebrate categorization system from which that of primates emerged. The primate results establish continuity with human cognition, suggesting that nonhuman primates share aspects of humans' capacity for explicit cognition. The emergence of dimensional analysis and rule learning could have been an important step in primates' cognitive evolution.

Procedural learning of unstructured categories.

Unstructured categories are those in which the stimuli are assigned to each contrasting category randomly, and thus there is no rule- or similarity-based strategy for determining category membership. Intuition suggests that unstructured categories are likely to be learned via explicit memorization that is under the control of declarative memory. In contrast to this prediction, neuroimaging studies of unstructured-category learning have reported task-related activation in the striatum, but typically not in the hippocampus--results that seem more consistent with procedural learning than with a declarative-memory strategy. This article reports the first known behavioral test of whether unstructured-category learning is mediated by explicit strategies or by procedural learning. Our results suggest that the feedback-based learning of unstructured categories is mediated by procedural memory.

Implicit and explicit category learning by capuchin monkeys (Cebus apella).

Current theories of human categorization differentiate an explicit, rule-based system of category learning from an implicit system that slowly associates regions of perceptual space with response outputs. The researchers extended this theoretical differentiation to the category learning of New World primates. Four capuchins (Cebus apella) learned categories of circular sine-wave gratings that varied in bar spatial frequency and orientation. The rule-based and information-integration tasks, respectively, had one-dimensional and two-dimensional solutions. Capuchins, like humans, strongly dimensionalized the stimuli and learned the rule-based task more easily. The results strengthen the suggestion that nonhuman primates have some structural components of humans' capacity for explicit categorization, which in humans is linked to declarative cognition and consciousness. The results also strengthen the primate contrast to other vertebrate species that may lack the explicit system. Therefore, the results raise important questions about the origins of the explicit categorization system during cognitive evolution and about its overall phylogenetic distribution.

Automaticity and multiple memory systems.

A large number of criteria have been proposed for determining when a behavior has become automatic. Almost all of these were developed before the widespread acceptance of multiple memory systems. Consequently, popular frameworks for studying automaticity often neglect qualitative differences in how different memory systems guide initial learning. Unfortunately, evidence suggests that automaticity criteria derived from these frameworks consistently misclassify certain sets of initial behaviors as automatic. Specifically, criteria derived from cognitive science mislabel much behavior still under the control of procedural memory as automatic, and criteria derived from animal learning mislabel some behaviors under the control of declarative memory as automatic. Even so, neither set of criteria make the opposite error-that is, both sets correctly identify any automatic behavior as automatic. In fact, evidence suggests that although there are multiple memory systems and therefore multiple routes to automaticity, there might nevertheless be only one common representation for automatic behaviors. A number of possible cognitive and cognitive neuroscience models of this single automaticity system are reviewed. WIREs Cogn Sci 2012, 3:363-376. doi: 10.1002/wcs.1172 For further resources related to this article, please visit the WIREs website.