Humans use local spectrotemporal correlations to detect rising and falling pitch.

To discern speech or appreciate music, the human auditory system detects how pitch increases or decreases over time. However, the algorithms used to detect changes in pitch, or pitch motion, are incompletely understood. Here, using psychophysics, computational modeling, functional neuroimaging, and analysis of recorded speech, we ask if humans detect pitch motion using computations analogous to those used by the visual system. We adapted stimuli from studies of vision to create novel auditory correlated noise stimuli that elicited robust pitch motion percepts. Crucially, these stimuli possess no persistent features across frequency or time, but do possess positive or negative local spectrotemporal correlations in intensity. In psychophysical experiments, we found clear evidence that humans judge pitch direction based on both positive and negative spectrotemporal correlations. The observed sensitivity to negative correlations is a direct analogue of illusory "reverse-phi" motion in vision, and thus constitutes a new auditory illusion. Our behavioral results and computational modeling led us to hypothesize that human auditory processing employs pitch direction opponency. fMRI measurements in auditory cortex supported this hypothesis. To link our psychophysical findings to real-world pitch perception, we analyzed recordings of English and Mandarin speech and discovered that pitch direction was robustly signaled by the same positive and negative spectrotemporal correlations used in our psychophysical tests, suggesting that sensitivity to both positive and negative correlations confers ecological benefits. Overall, this work reveals that motion detection algorithms sensitive to local correlations are deployed by the central nervous system across disparate modalities (vision and audition) and dimensions (space and frequency).

Audiomotor prediction errors drive speech adaptation even in the absence of overt movement.

Observed outcomes of our movements sometimes differ from our expectations. These sensory prediction errors recalibrate the brain's internal models for motor control, reflected in alterations to subsequent movements that counteract these errors (motor adaptation). While leading theories suggest that all forms of motor adaptation are driven by learning from sensory prediction errors, dominant models of speech adaptation argue that adaptation results from integrating time-advanced copies of corrective feedback commands into feedforward motor programs. Here, we tested these competing theories of speech adaptation by inducing planned, but not executed, speech. Human speakers (male and female) were prompted to speak a word and, on a subset of trials, were rapidly cued to withhold the prompted speech. On standard trials, speakers were exposed to real-time playback of their own speech with an auditory perturbation of the first formant to induce single-trial speech adaptation. Speakers experienced a similar sensory error on movement cancelation trials, hearing a perturbation applied to a recording of their speech from a previous trial at the time they would have spoken. Speakers adapted to auditory prediction errors in both contexts, altering the spectral content of spoken vowels to counteract formant perturbations even when no actual movement coincided with the perturbed feedback. These results build upon recent findings in reaching, and suggest that prediction errors, rather than corrective motor commands, drive adaptation in speech.

Fundamental processes in sensorimotor learning: Reasoning, refinement, and retrieval.

Motor learning is often viewed as a unitary process that operates outside of conscious awareness. This perspective has led to the development of sophisticated models designed to elucidate the mechanisms of implicit sensorimotor learning. In this review, we argue for a broader perspective, emphasizing the contribution of explicit strategies to sensorimotor learning tasks. Furthermore, we propose a theoretical framework for motor learning that consists of three fundamental processes: reasoning, the process of understanding action-outcome relationships; refinement, the process of optimizing sensorimotor and cognitive parameters to achieve motor goals; and retrieval, the process of inferring the context and recalling a control policy. We anticipate that this '3R' framework for understanding how complex movements are learned will open exciting avenues for future research at the intersection between cognition and action.

Independent influences of movement distance and visual distance on Fitts' law.

Fitts' Law is one among a small number of psychophysical laws. However, a fundamental variable in Fitts' Law-the movement distance, D-confounds two quantities: The physical distance the effector has to move to reach a goal, and the visually perceived distance to that goal. While these two quantities are functionally equivalent in everyday motor behavior, decoupling them might improve our understanding of the factors that shape speed-accuracy tradeoffs. Here, we leveraged the phenomenon of visuomotor gain adaptation to de-confound movement and visual distance during goal-directed reaching. We found that movement distance and visual distance can influence movement times, supporting a variant of Fitts' Law that considers both. The weighting of movement versus visual distance was modified by restricting movement range and degrading visual feedback. These results may reflect the role of sensory context in early stages of motor planning. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Oblique warping: A general distortion of spatial perception.

There are many putatively distinct phenomena related to perception in the oblique regions of space. For instance, the classic oblique effect describes a deficit in visual acuity for oriented lines in the obliques, and classic "prototype effects" reflect a bias to misplace objects towards the oblique regions of space. Yet these effects are explained in very different terms: The oblique effect itself is often understood as arising from orientation-selective neurons, whereas prototype effects are described as arising from categorical biases. Here, we explore the possibility that these effects (and others) may stem from a single underlying spatial distortion. We show that there is a general distortion of (angular) space in the oblique regions that influences not only orientation judgments, but also location, extent, and size. We argue that these findings reflect oblique warping, a general distortion of spatial representations in the oblique regions which may be the root cause of many oblique effects.

Dissociable Codes in Motor Working Memory.

Working memory has been comprehensively studied in sensory domains, like vision, but little attention has been paid to how motor information (e.g., kinematics of recent movements) is maintained and manipulated in working memory. "Motor working memory" (MWM) is important for short-term behavioral control and skill learning. Here, we employed tasks that required participants to encode and recall reaching movements over short timescales. We conducted three experiments (N = 65 undergraduates) to examine MWM under varying cognitive loads, delays, and degrees of interference. The results support a model of MWM that includes an abstract code that flexibly transfers across effectors, and an effector-specific code vulnerable to interfering movements, even when interfering movements are irrelevant to the task. Neither code was disrupted by increasing visuospatial working memory load. These results echo distinctions between representational formats in other domains, suggesting that MWM shares a basic computational structure with other working memory subsystems.

A common format for representing spatial location in visual and motor working memory.

Does the mind rely on similar systems of spatial representation for both perception and action? Here, we assessed the format of location representations in two simple spatial localization tasks. In one task, participants simply remembered the location of an item based solely on visual input. In another, participants remembered the location of a point in space based solely on kinesthetic input. Participants' recall errors were more consistent with the use of polar coordinates than Cartesian coordinates in both tasks. Moreover, measures of spatial bias and performance were correlated across modalities. In a subsequent study, we tested the flexibility with which people use polar coordinates to represent space; we show that the format in which the information is presented to participants influences how that information is encoded and the errors that are made as a result. We suggest that polar coordinates may be a common means of representing location information across visual and motor modalities, but that these representations are also flexible in form.

Metacognitive judgments during visuomotor learning reflect the integration of error history.

People form metacognitive representations of their own abilities across a range of tasks. How these representations are influenced by errors during learning is poorly understood. Here, we ask how metacognitive confidence judgments of performance during motor learning are shaped by the learner's recent history of errors. Across four motor learning experiments, our computational modeling approach demonstrated that people's confidence judgments are best explained by a recency-weighted averaging of visually observed errors. Moreover, in the formation of these confidence estimates, people appear to reweight observed motor errors according to a subjective cost function. Confidence judgments were adaptive, incorporating recent motor errors in a manner that was sensitive to the volatility of the learning environment, integrating a shallower history when the environment was more volatile. Finally, confidence tracked motor errors in the context of both implicit and explicit motor learning but only showed evidence of influencing behavior in the latter. Our study thus provides a novel descriptive model that successfully approximates the dynamics of metacognitive judgments during motor learning.NEW & NOTEWORTHY This study examined how, during visuomotor learning, people's confidence in their performance is shaped by their recent history of errors. Using computational modeling, we found that confidence incorporated recent error history, tracked subjective error costs, was sensitive to environmental volatility, and in some contexts may influence learning. Together, these results provide a novel model of metacognitive judgments during motor learning that could be applied to future computational and neural studies at the interface of higher-order cognition and motor control.

"Don't [ruminate], be happy": A cognitive perspective linking depression and anhedonia.

Anhedonia, a lack of pleasure in things an individual once enjoyed, and rumination, the process of perseverative and repetitive attention to specific thoughts, are hallmark features of depression. Though these both contribute to the same debilitating disorder, they have often been studied independently and through different theoretical lenses (e.g., biological vs. cognitive). Cognitive theories and research on rumination have largely focused on understanding negative affect in depression with much less focus on the etiology and maintenance of anhedonia. In this paper, we argue that by examining the relation between cognitive constructs and deficits in positive affect, we may better understand anhedonia in depression thereby improving prevention and intervention efforts. We review the extant literature on cognitive deficits in depression and discuss how these dysfunctions may not only lead to sustained negative affect but, importantly, interfere with an ability to attend to social and environmental cues that could restore positive affect. Specifically, we discuss how rumination is associated to deficits in working memory and propose that these deficits in working memory may contribute to anhedonia in depression. We further argue that analytical approaches such as computational modeling are needed to study these questions and, finally, discuss implications for treatment.

Contextual effects in sensorimotor adaptation adhere to associative learning rules.

Traditional associative learning tasks focus on the formation of associations between salient events and arbitrary stimuli that predict those events. This is exemplified in cerebellar-dependent delay eyeblink conditioning, where arbitrary cues such as a tone or light act as conditioned stimuli (CSs) that predict aversive sensations at the cornea (unconditioned stimulus [US]). Here, we ask if a similar framework could be applied to another type of cerebellar-dependent sensorimotor learning - sensorimotor adaptation. Models of sensorimotor adaptation posit that the introduction of an environmental perturbation results in an error signal that is used to update an internal model of a sensorimotor map for motor planning. Here, we take a step toward an integrative account of these two forms of cerebellar-dependent learning, examining the relevance of core concepts from associative learning for sensorimotor adaptation. Using a visuomotor adaptation reaching task, we paired movement-related feedback (US) with neutral auditory or visual contextual cues that served as CSs. Trial-by-trial changes in feedforward movement kinematics exhibited three key signatures of associative learning: differential conditioning, sensitivity to the CS-US interval, and compound conditioning. Moreover, after compound conditioning, a robust negative correlation was observed between responses to the two elemental CSs of the compound (i.e. overshadowing), consistent with the additivity principle posited by theories of associative learning. The existence of associative learning effects in sensorimotor adaptation provides a proof-of-concept for linking cerebellar-dependent learning paradigms within a common theoretical framework.

Motor learning without movement.

Prediction errors guide many forms of learning, providing teaching signals that help us improve our performance. Implicit motor adaptation, for instance, is thought to be driven by sensory prediction errors (SPEs), which occur when the expected and observed consequences of a movement differ. Traditionally, SPE computation is thought to require movement execution. However, recent work suggesting that the brain can generate sensory predictions based on motor imagery or planning alone calls this assumption into question. Here, by measuring implicit motor adaptation during a visuomotor task, we tested whether motor planning and well-timed sensory feedback are sufficient for adaptation. Human participants were cued to reach to a target and were, on a subset of trials, rapidly cued to withhold these movements. Errors displayed both on trials with and without movements induced single-trial adaptation. Learning following trials without movements persisted even when movement trials had never been paired with errors and when the direction of movement and sensory feedback trajectories were decoupled. These observations indicate that the brain can compute errors that drive implicit adaptation without generating overt movements, leading to the adaptation of motor commands that are not overtly produced.

Distinct Neural Signatures of Outcome Monitoring After Selection and Execution Errors.

Losing a point in tennis could result from poor shot selection or faulty stroke execution. To explore how the brain responds to these different types of errors, we examined feedback-locked EEG activity while participants completed a modified version of a standard three-armed bandit probabilistic reward task. Our task framed unrewarded outcomes as the result of either errors of selection or errors of execution. We examined whether amplitude of a medial frontal negativity (the feedback-related negativity [FRN]) was sensitive to the different forms of error attribution. Consistent with previous reports, selection errors elicited a large FRN relative to rewards, and amplitude of this signal correlated with behavioral adjustment after these errors. A different pattern was observed in response to execution errors. These outcomes produced a larger FRN, a frontocentral attenuation in activity preceding this component, and a subsequent enhanced error positivity in parietal sites. Notably, the only correlations with behavioral adjustment were with the early frontocentral attenuation and amplitude of the parietal signal; FRN differences between execution errors and rewarded trials did not correlate with subsequent changes in behavior. Our findings highlight distinct neural correlates of selection and execution error processing, providing insight into how the brain responds to the different classes of error that determine future action.

Continuous manipulation of mental representations is compromised in cerebellar degeneration.

We introduce a novel perspective on how the cerebellum might contribute to cognition, hypothesizing that this structure supports dynamic transformations of mental representations. In support of this hypothesis, we report a series of neuropsychological experiments comparing the performance of individuals with degenerative cerebellar disorders on tasks that either entail continuous, movement-like mental operations or more discrete mental operations. In the domain of visual cognition, the cerebellar disorders group exhibited an impaired rate of mental rotation, an operation hypothesized to require the continuous manipulation of a visual representation. In contrast, the cerebellar disorders group showed a normal processing rate when scanning items in visual working memory, an operation hypothesized to require the maintenance and retrieval of remembered items. In the domain of mathematical cognition, the cerebellar disorders group was impaired at single-digit addition, an operation hypothesized to primarily require iterative manipulations along a mental number-line; this group was not impaired on arithmetic tasks linked to memory retrieval (e.g. single-digit multiplication). These results, obtained in tasks from two disparate domains, point to a potential constraint on the contribution of the cerebellum to cognitive tasks. Paralleling its role in motor control, the cerebellum may be essential for coordinating dynamic, movement-like transformations in a mental workspace.

Going beyond primary motor cortex to improve brain-computer interfaces.

Brain-computer interfaces (BCIs) for movement restoration typically decode the user's intent from neural activity in their primary motor cortex (M1) and use this information to enable 'mental control' of an external device. Here, we argue that activity in M1 has both too little and too much information for optimal decoding: too little, in that many regions beyond it contribute unique motor outputs and have movement-related information that is absent or otherwise difficult to resolve from M1 activity; and too much, in that motor commands are tangled up with nonmotor processes such as attention and feedback processing, potentially hindering decoding. Both challenges might be circumvented, we argue, by integrating additional information from multiple brain regions to develop BCIs that will better interpret the user's intent.

Revisiting the Role of the Medial Temporal Lobe in Motor Learning.

Classic taxonomies of memory distinguish explicit and implicit memory systems, placing motor skills squarely in the latter branch. This assertion is in part a consequence of foundational discoveries showing significant motor learning in amnesics. Those findings suggest that declarative memory processes in the medial temporal lobe (MTL) do not contribute to motor learning. Here, we revisit this issue, testing an individual (L. S. J.) with severe MTL damage on four motor learning tasks and comparing her performance to age-matched controls. Consistent with previous findings in amnesics, we observed that L. S. J. could improve motor performance despite having significantly impaired declarative memory. However, she tended to perform poorly relative to age-matched controls, with deficits apparently related to flexible action selection. Further supporting an action selection deficit, L. S. J. fully failed to learn a task that required the acquisition of arbitrary action-outcome associations. We thus propose a modest revision to the classic taxonomic model: Although MTL-dependent memory processes are not necessary for some motor learning to occur, they play a significant role in the acquisition, implementation, and retrieval of action selection strategies. These findings have implications for our understanding of the neural correlates of motor learning, the psychological mechanisms of skill, and the theory of multiple memory systems.

Post-error Slowing During Instrumental Learning is Shaped by Working Memory-based Choice Strategies.

Post-error slowing (PES) - a relative increase in response time for a decision on trialtgiven an error on trialt - 1 - is a well-known effect in studies of human decision-making. Post-error processing is reflected in neural signatures such as reduced activity in sensorimotor regions and increased activity in medial prefrontal cortex. PES is thought to reflect the deployment of executive resources to get task performance back on track. This provides a general account of PES that cuts across perceptual decision-making, memory, and learning tasks. With respect to PES and learning, things are complicated by the fact that learning often reflectsmultiple qualitatively different processes with distinct neural correlates. It is unclear if multiple processes shape PES during learning, or if PES reflects a policy for reacting to errors generated by one particular process (e.g., cortico-striatal reinforcement learning). Here we provide behavioral and computational evidence that PES is influenced by the operation of multiple distinct processes. Human subjects learned a simple visuomotor skill (arbitrary visuomotor association learning) under low load conditionsmore amenable to simple working memory-based strategies, and high load conditions that were putatively more reliant on trial-by-trial reinforcement learning. PES decreased withload, even when the progress of learning (i.e., reinforcement history) was accounted for. This result suggested that PES during learning is influenced by the recruitment of working memory. Indeed, observed PES effects were approximated by a computational model with parallel working memory and reinforcement learning systems that are differentially recruited according to cognitive load.

Executive Function Assigns Value to Novel Goal-Congruent Outcomes.

People often learn from the outcomes of their actions, even when these outcomes do not involve material rewards or punishments. How does our brain provide this flexibility? We combined behavior, computational modeling, and functional neuroimaging to probe whether learning from abstract novel outcomes harnesses the same circuitry that supports learning from familiar secondary reinforcers. Behavior and neuroimaging revealed that novel images can act as a substitute for rewards during instrumental learning, producing reliable reward-like signals in dopaminergic circuits. Moreover, we found evidence that prefrontal correlates of executive control may play a role in shaping flexible responses in reward circuits. These results suggest that learning from novel outcomes is supported by an interplay between high-level representations in prefrontal cortex and low-level responses in subcortical reward circuits. This interaction may allow for human reinforcement learning over arbitrarily abstract reward functions.

Prolonged response time helps eliminate residual errors in visuomotor adaptation.

One persistent curiosity in visuomotor adaptation tasks is that participants often do not reach maximal performance. This incomplete asymptote has been explained as a consequence of obligatory computations within the implicit adaptation system, such as an equilibrium between learning and forgetting. A body of recent work has shown that in standard adaptation tasks, cognitive strategies operate alongside implicit learning. We reasoned that incomplete learning in adaptation tasks may primarily reflect a speed-accuracy tradeoff on time-consuming motor planning. Across three experiments, we find evidence supporting this hypothesis, showing that hastened motor planning may primarily lead to under-compensation. When an obligatory waiting period was administered before movement start, participants were able to fully counteract imposed perturbations (Experiment 1). Inserting the same delay between trials - rather than during movement planning - did not induce full compensation, suggesting that the motor planning interval influences the learning asymptote (Experiment 2). In the last experiment (Experiment 3), we asked participants to continuously report their movement intent. We show that emphasizing explicit re-aiming strategies (and concomitantly increasing planning time) also lead to complete asymptotic learning. Findings from all experiments support the hypothesis that incomplete adaptation is, in part, the result of an intrinsic speed-accuracy tradeoff, perhaps related to cognitive strategies that require parametric attentional reorienting from the visual target to the goal.

The Role of Executive Function in Shaping Reinforcement Learning.

Reinforcement learning (RL) models have advanced our understanding of how animals learn and make decisions, and how the brain supports some aspects of learning. However, the neural computations that are explained by RL algorithms fall short of explaining many sophisticated aspects of human decision making, including the generalization of learned information, one-shot learning, and the synthesis of task information in complex environments.. Instead, these aspects of instrumental behavior are assumed to be supported by the brain's executive functions (EF). We review recent findings that highlight the importance of EF in learning. Specifically, we advance the theory that EF sets the stage for canonical RL computations in the brain, providing inputs that broaden their flexibility and applicability. Our theory has important implications for how to interpret RL computations in the brain and behavior.