Slowing of Movements in Healthy Aging as a Rational Economic Response to an Elevated Effort Landscape.

Why do we move slower as we grow older? The reward circuits of the brain, which tend to invigorate movements, decline with aging, raising the possibility that reduced vigor is due to the diminishing value that our brain assigns to movements. However, as we grow older, it also becomes more effortful to make movements. Is age-related slowing principally a consequence of increased effort costs from the muscles, or reduced valuation of reward by the brain? Here, we first quantified the cost of reaching via metabolic energy expenditure in human participants (male and female), and found that older adults consumed more energy than the young at a given speed. Thus, movements are objectively more costly for older adults. Next, we observed that when reward increased, older adults, like the young, responded by initiating their movements earlier. Yet, unlike the young, they were unwilling to increase their movement speed. Was their reluctance to reach quicker for rewards due to the increased effort costs, or because they ascribed less value to the movement? Motivated by a mathematical model, we next made the young experience a component of aging by making their movements more effortful. Now the young responded to reward by reacting faster but chose not to increase their movement speed. This suggests that slower movements in older adults are partly driven by an adaptive response to an elevated effort landscape. Moving slower may be a rational economic response the brain is making to mitigate the elevated effort costs that accompany aging.

Disentangling the effects of metabolic cost and accuracy on movement speed.

On any given day, we make countless reaching movements to objects around us. While such ubiquity may suggest uniformity, each movement's speed is unique-why is this? Reach speed is known to be influenced by accuracy; we slow down to sustain high accuracy. However, in other forms of movement like walking or running, metabolic cost is often the primary determinant of movement speed. Here we bridge this gap and ask: how do metabolic cost and accuracy interact to determine speed of reaching movements? First, we systematically measure the effect of increasing mass on the metabolic cost of reaching across a range of movement speeds. Next, in a sequence of three experiments, we examine how added mass affects preferred reaching speed across changing accuracy requirements. We find that, while added mass consistently increases metabolic cost thereby leading to slower metabolically optimal movement speeds, self-selected reach speeds are slower than those predicted by an optimization of metabolic cost alone. We then demonstrate how a single model that considers both accuracy and metabolic costs can explain preferred movement speeds. Together, our findings provide a unifying framework to illuminate the combined effects of metabolic cost and accuracy on movement speed and highlight the integral role metabolic cost plays in determining reach speed.

Common Mechanisms of Learning in Motor and Cognitive Systems.

Rapid progress in our understanding of the brain's learning mechanisms has been accomplished over the past decade, particularly with conceptual advances, including representing behavior as a dynamical system, large-scale neural population recordings, and new methods of analysis of neuronal populations. However, motor and cognitive systems have been traditionally studied with different methods and paradigms. Recently, some common principles, evident in both behavior and neural activity, that underlie these different types of learning have become to emerge. Here we review results from motor and cognitive learning, relying on different techniques and studying different systems to understand the mechanisms of learning. Movement is intertwined with cognitive operations, and its dynamics reflect cognitive variables. Training, in either motor or cognitive tasks, involves recruitment of previously unresponsive neurons and reorganization of neural activity in a low dimensional manifold. Mapping of new variables in neural activity can be very rapid, instantiating flexible learning of new tasks. Communication between areas is just as critical a part of learning as are patterns of activity within an area emerging with learning. Common principles across systems provide a map for future research.

Physical Effort Pre-Crastination Determines Preference in an Isometric Task.

How the brain decides when to invest effort is a central question in neuroscience. When asked to walk a mile to a destination, would you choose a path with a hill at the beginning or the end? The traditional view of effort suggests we should be indifferent-all joules are equal so long as it does not interfere with accomplishing the goal. Yet when total joules are equal across movement decisions, the brain's sensitivity to the temporal profile of effort investment remains poorly understood. Here, we sought to parse the interaction of time and physical effort by comparing subjective preferences in an isometric arm-pushing task that varied the duration and timing of high and low effort. Subjects were presented with a series of two-alternative forced choices, where they chose the force profile they would rather complete. Subjects preferred earlier physical effort (i.e., to pre-crastinate) but were idiosyncratic about preference for task timing. A model of subjective utility that includes physical effort costs, task costs, and independent temporal sensitivity factors described subject preferences best. Interestingly, deliberation time and response vigor covary with the same subjective utility model for preference, suggesting a utility that underlies both decision making and motor control. These results suggest physical effort costs are temporally sensitive, with earlier investment of effort preferred to later investment, and that the representation of effort is based not only on the total energy required but when it is required to invest that energy.

Effects of reward and effort history on decision making and movement vigor during foraging.

During foraging, animals explore a site and harvest reward and then abandon that site and travel to the next opportunity. One aspect of this behavior involves decision making, and the other involves movement control. These two aspects of behavior may be linked via an underlying desire to maximize a single normative utility: the sum of all rewards acquired, minus all efforts expended, divided by time. According to this theory, the history of rewards, and not just its immediate availability, should dictate how long one should stay and harvest reward and how vigorously one should travel to the next opportunity. We tested this theory in a series of experiments in which humans used their hand to harvest tokens at a reward patch and then used their arm to reach toward another patch. After a history of high rewards, the subjects not only shortened their harvest duration but also moved more vigorously toward the next reward opportunity. In contrast, after a history of high effort they lengthened their harvest duration but reduced their movement vigor, reaching more slowly to the next reward site. Thus, a history of high reward or low effort biased decisions by promoting early abandonment of the reward site and biased movements by promoting vigor.NEW & NOTEWORTHY Much of life is spent foraging. Whereas previous work has focused on the decision regarding time spent harvesting from a reward patch, here we test the idea that both decision making and movement control are tuned to optimize the net rate of reward in an environment. Our results show that movement patterns reflect not just immediate expectations but also past experiences in the environment, providing fundamental insight into the factors governing volitional control of arm movements.

Metabolic costs of walking and arm reaching in persons with mild multiple sclerosis.

Movement slowness is a common and disruptive symptom of multiple sclerosis (MS). A potential cause is that individuals with MS slow down to conserve energy as a behavioral adjustment to heightened metabolic costs of movement. To investigate this prospect, we measured the metabolic costs of both walking and seated arm reaching at five speeds in persons with mild MS (pwMS; n = 13; 46.0 ± 7.7 yr) and sex- and age-matched controls (HCs; n = 13; 45.8 ± 7.8 yr). Notably, the cohort of pwMS was highly mobile and no individuals required a cane or aid when walking. We found that the net metabolic power of walking was approximately 20% higher for pwMS across all speeds (P = 0.0185). In contrast, we found no differences in the gross power of reaching between pwMS and HCs (P = 0.492). Collectively, our results suggest that abnormal slowness of movement in MS-particularly reaching-is not the consequence of heightened effort costs and that other sensorimotor mechanisms are playing a considerable role in slowing.NEW & NOTEWORTHY Individuals with multiple sclerosis (MS) often move more slowly than those without the disease. A possible cause is that movements in MS are more energetically expensive and slowing is an adaptation to conserve metabolic resources. Here, we find that while walking is more costly for persons with MS, arm-reaching movements are not. These results bring into question the driving force of movement slowness in MS and implicate other motor-related networks contributing to slowing.

Whole body adaptation to novel dynamics does not transfer between effectors.

How does the brain coordinate concurrent adaptation of arm movements and standing posture? From previous studies, the postural control system can use information about previously adapted arm movement dynamics to plan appropriate postural control; however, it is unclear whether postural control can be adapted and controlled independently of arm control. The present study addresses that question. Subjects practiced planar reaching movements while standing and grasping the handle of a robotic arm, which generated a force field to create novel perturbations. Subjects were divided into two groups, for which perturbations were introduced in either an abrupt or a gradual manner. All subjects adapted to the perturbations while reaching with their dominant (right) arm and then switched to reaching with their nondominant (left) arm. Previous studies of seated reaching movements showed that abrupt perturbation introduction led to transfer of learning between arms, but gradual introduction did not. Interestingly, in this study neither group showed evidence of transferring adapted control of arm or posture between arms. These results suggest primarily that adapted postural control cannot be transferred independently of arm control in this task paradigm. In other words, whole body postural movement planning related to a concurrent arm task is dependent on information about arm dynamics. Finally, we found that subjects were able to adapt to the gradual perturbation while experiencing very small errors, suggesting that both error size and consistency play a role in driving motor adaptation.NEW & NOTEWORTHY This study examined adaptation of arm and postural control to novel dynamics while standing and reaching and subsequent transfer between reaching arms. Neither arm nor postural control was transferred between arms, suggesting that postural planning is highly dependent on the concurrent arm movement.

Control of movement vigor and decision making during foraging.

During foraging, animals decide how long to stay at a patch and harvest reward, and then, they move with certain vigor to another location. How does the brain decide when to leave, and how does it determine the speed of the ensuing movement? Here, we considered the possibility that both the decision-making and the motor control problems aimed to maximize a single normative utility: the sum of all rewards acquired minus all efforts expended divided by total time. This optimization could be achieved if the brain compared a local measure of utility with its history. To test the theory, we examined behavior of people as they gazed at images: they chose how long to look at the image (harvesting information) and then moved their eyes to another image, controlling saccade speed. We varied reward via image content and effort via image eccentricity, and then, we measured how these changes affected decision making (gaze duration) and motor control (saccade speed). After a history of low rewards, people increased gaze duration and decreased saccade speed. In anticipation of future effort, they lowered saccade speed and increased gaze duration. After a history of high effort, they elevated their saccade speed and increased gaze duration. Therefore, the theory presented a principled way with which the brain may control two aspects of behavior: movement speed and harvest duration. Our experiments confirmed many (but not all) of the predictions, suggesting that harvest duration and movement speed, fundamental aspects of behavior during foraging, may be governed by a shared principle of control.

Movement control, decision-making, and the building of Roman roads to link them.

In science, as in life, one can only hope to both inform others, and be informed by them. The commentaries associated with our book Vigor have highlighted the many ways in which the theory that we proposed can be improved. For example, there are a myriad of factors that need to be considered in a fully encompassing objective function. The neural mechanisms underlying the links between movement and decision-making have yet to be unraveled. The implications of a two-way interaction between movement and decisions at both the individual and social levels remain to be understood. The commentaries outline future questions, and encouragingly highlight the diversity of science communities that may be linked via the concept of vigor.

Role of muscle coactivation in adaptation of standing posture during arm reaching.

The ability to maintain stable, upright standing in the face of perturbations is a critical component of daily life. A common strategy for resisting perturbations and maintaining stability is muscle coactivation. Although arm muscle coactivation is often used during adaptation of seated reaching movements, little is known about postural muscle activation during concurrent adaptation of arm and standing posture to novel perturbations. In this study we investigate whether coactivation strategies are employed during adaptation of standing postural control, and how these strategies are prioritized for adaptation of standing posture and arm reaching, in two different postural stability conditions. Healthy adults practiced planar reaching movements while grasping the handle of a robotic arm and standing on a force plate; the robotic arm generated a velocity-dependent force field that created novel perturbations in the forward (more stable) or backward (less stable) direction. Surprisingly, the degree of arm and postural adaptation was not influenced by stability, with similar adaptation observed between conditions in the control of both arm movement and standing posture. We found that an early coactivation strategy can be used in postural adaptation, similar to what is observed in adaptation of arm reaching movements. However, the emergence of a coactivation strategy was dependent on perturbation direction. Despite similar adaptation in both directions, postural coactivation was largely specific to forward perturbations. Backward perturbations led to less coactivation and less modulation of postural muscle activity. These findings provide insight into how postural stability can affect prioritization of postural control objectives and movement adaptation strategies.NEW & NOTEWORTHY Muscle coactivation is a key strategy for modulating movement stability; this is centrally important in the control of standing posture. Our study investigates the little-known role of coactivation in adaptation of whole body standing postural control. We demonstrate that an early coactivation strategy can be used in postural adaptation, but muscle activation strategies may differ depending on postural stability conditions.

Saccade vigor and the subjective economic value of visual stimuli.

Decisions are made based on the subjective value that the brain assigns to options. However, subjective value is a mathematical construct that cannot be measured directly, but rather is inferred from choices. Recent results have demonstrated that reaction time, amplitude, and velocity of movements are modulated by reward, raising the possibility that there is a link between how the brain evaluates an option and how it controls movements toward that option. Here, we asked people to choose among risky options represented by abstract stimuli, some associated with gain (points in a game), and others with loss. From their choices we estimated the subjective value that they assigned to each stimulus. In probe trials, a single stimulus appeared at center, instructing subjects to make a saccade to a peripheral target. We found that the reaction time, peak velocity, and amplitude of the peripherally directed saccade varied roughly linearly with the subjective value that the participant had assigned to the central stimulus: reaction time was shorter, velocity was higher, and amplitude was larger for stimuli that the participant valued more. Naturally, participants differed in how much they valued a given stimulus. Remarkably, those who valued a stimulus more, as evidenced by their choices in decision trials, tended to move with shorter reaction time and greater velocity in response to that stimulus in probe trials. Overall, the reaction time of the saccade in response to a stimulus partly predicted the subjective value that the brain assigned to that stimulus.NEW & NOTEWORTHY Behavioral economics relies on subjective evaluation, an abstract quantity that cannot be measured directly but must be inferred by fitting decision models to the choice patterns. Here, we present a new approach to estimate subjective value: with nothing to fit, we show that it is possible to estimate subjective value based on movement kinematics, providing a modest ability to predict a participant's preferences without prior measurement of their choice patterns.

Précis of Vigor: Neuroeconomics of Movement Control.

Why do we run toward people we love, but only walk toward others? Why do people in New York seem to walk faster than other cities? Why do our eyes linger longer on things we value more? There is a link between how the brain assigns value to things, and how it controls our movements. This link is an ancient one, developed through shared neural circuits that on one hand teach us how to value things, and on the other hand control the vigor with which we move. As a result, when there is damage to systems that signal reward, like dopamine and serotonin, that damage not only affects our mood and patterns of decision-making, but how we move. In this book, we first ask why, in principle, evolution should have developed a shared system of control between valuation and vigor. We then focus on the neural basis of vigor, synthesizing results from experiments that have measured activity in various brain structures and neuromodulators, during tasks in which animals decide how patiently they should wait for reward, and how vigorously they should move to acquire it. Thus, the way we move unmasks one of our well-guarded secrets: how much we value the thing we are moving toward.

Poor estimates of motor variability are associated with longer grooved pegboard times for middle-aged and older adults.

Goal-directed movements that involve greater motor variability are performed with an increased risk that the intended goal will not be achieved. The ability to estimate motor variability during such actions varies across individuals and influences how people decide to move about their environment. The purpose of our study was to identify the decision-making strategies used by middle-aged and older adults when performing two goal-directed motor tasks and to determine if these strategies were associated with the time to complete the grooved pegboard test. Twenty-one middle-aged (48 ± 6 yr; range 40-59 yr, 15 women) and 20 older adults (73 ± 4 yr; range 65-79 yr, 8 women) performed two targeted tasks, each with two normalized target options. Decision-making characteristics were not associated with time to complete the test of manual dexterity when the analysis included all participants, but slower pegboard times were associated with measures of greater movement variability during the target-directed actions. When the data were clustered on the basis of pegboard time rather than age, relatively longer times for the faster group were associated with greater motor variability during the prescribed tasks, whereas longer times for the slower group were associated with increased risk-seeking behavior (α) and greater variability in the targeted actions. NEW & NOTEWORTHY This study was the first to examine the association between decision-making choices and an NIH Toolbox test of manual dexterity (grooved pegboard test) performed by middle-aged and older adults. Significant associations were observed between decision-making choices and time to complete the test when the analyses were based on pegboard times rather than chronological age. This result indicates that decision-making choices of middle-aged and older adults, independent of age, were associated with time to complete a test of manual dexterity.

Vigor of reaching movements: reward discounts the cost of effort.

Making a movement may be thought of as an economic decision in which one spends effort to acquire reward. Time discounts reward, which predicts that the magnitude of reward should affect movement vigor: we should move faster, spending greater effort, when there is greater reward at stake. Indeed, saccade peak velocities are greater and reaction-times shorter when a target is paired with reward. In this study, we focused on human reaching and asked whether movement kinematics were affected by expectation of reward. Participants made out-and-back reaching movements to one of four quadrants of a 14-cm circle. During various periods of the experiment only one of the four quadrants was paired with reward, and the transition from reward to nonreward status of a quadrant occurred randomly. Our experiment design minimized dependence of reward on accuracy, granting the subjects wide latitude in self-selecting their movement speed, amplitude, and variability. When a quadrant was paired with reward, reaching movements had a shorter reaction time, higher peak velocity, and greater amplitude. Despite this greater vigor, movements toward the rewarded quadrant suffered from less variability: both reaction times and reach kinematics were less variable when there was expectation of reward. Importantly, the effect of reward on vigor was specific to the movement component that preceded the time of reward (outward reach), not the movement component that followed it (return reach). Our results suggest that expectation of reward not only increases vigor of human reaching but also decreases its variability. NEW & NOTEWORTHY Movements may be thought of as an economic transaction where the vigor of the movement represents the effort that the brain is willing to expend to acquire a rewarding state. We show that in reaching, reward discounts the cost of effort, producing movements with shorter reaction time, higher velocity, greater amplitude, and reduced reaction-time variability. These results complement earlier observations in saccades, suggesting a common principle of economics across modalities of motor control.

Trial-to-trial adaptation in control of arm reaching and standing posture.

Classical theories of motor learning hypothesize that adaptation is driven by sensorimotor error; this is supported by studies of arm and eye movements that have shown that trial-to-trial adaptation increases with error. Studies of postural control have shown that anticipatory postural adjustments increase with the magnitude of a perturbation. However, differences in adaptation have been observed between the two modalities, possibly due to either the inherent instability or sensory uncertainty in standing posture. Therefore, we hypothesized that trial-to-trial adaptation in posture should be driven by error, similar to what is observed in arm reaching, but the nature of the relationship between error and adaptation may differ. Here we investigated trial-to-trial adaptation of arm reaching and postural control concurrently; subjects made reaching movements in a novel dynamic environment of varying strengths, while standing and holding the handle of a force-generating robotic arm. We found that error and adaptation increased with perturbation strength in both arm and posture. Furthermore, in both modalities, adaptation showed a significant correlation with error magnitude. Our results indicate that adaptation scales proportionally with error in the arm and near proportionally in posture. In posture only, adaptation was not sensitive to small error sizes, which were similar in size to errors experienced in unperturbed baseline movements due to inherent variability. This finding may be explained as an effect of uncertainty about the source of small errors. Our findings suggest that in rehabilitation, postural error size should be considered relative to the magnitude of inherent movement variability.

Rationality in Human Movement.

It long has been appreciated that humans behave irrationally in economic decisions under risk: they fail to objectively consider uncertainty, costs, and rewards and instead exhibit risk-seeking or risk-averse behavior. We hypothesize that poor estimates of motor variability (influenced by motor task) and distorted probability weighting (influenced by relevant emotional processes) contribute to characteristic irrationality in human movement decisions.

Reward feedback accelerates motor learning.

Recent findings have demonstrated that reward feedback alone can drive motor learning. However, it is not yet clear whether reward feedback alone can lead to learning when a perturbation is introduced abruptly, or how a reward gradient can modulate learning. In this study, we provide reward feedback that decays continuously with increasing error. We asked whether it is possible to learn an abrupt visuomotor rotation by reward alone, and if the learning process could be modulated by combining reward and sensory feedback and/or by using different reward landscapes. We designed a novel visuomotor learning protocol during which subjects experienced an abruptly introduced rotational perturbation. Subjects received either visual feedback or reward feedback, or a combination of the two. Two different reward landscapes, where the reward decayed either linearly or cubically with distance from the target, were tested. Results demonstrate that it is possible to learn from reward feedback alone and that the combination of reward and sensory feedback accelerates learning. An analysis of the underlying mechanisms reveals that although reward feedback alone does not allow for sensorimotor remapping, it can nonetheless lead to broad generalization, highlighting a dissociation between remapping and generalization. Also, the combination of reward and sensory feedback accelerates learning without compromising sensorimotor remapping. These findings suggest that the use of reward feedback is a promising approach to either supplement or substitute sensory feedback in the development of improved neurorehabilitation techniques. More generally, they point to an important role played by reward in the motor learning process.

Older adults learn less, but still reduce metabolic cost, during motor adaptation.

The ability to learn new movements and dynamics is important for maintaining independence with advancing age. Age-related sensorimotor changes and increased muscle coactivation likely alter the trial-and-error-based process of adapting to new movement demands (motor adaptation). Here, we asked, to what extent is motor adaptation to novel dynamics maintained in older adults (≥65 yr)? We hypothesized that older adults would adapt to the novel dynamics less well than young adults. Because older adults often use muscle coactivation, we expected older adults to use greater muscle coactivation during motor adaptation than young adults. Nevertheless, we predicted that older adults would reduce muscle activity and metabolic cost with motor adaptation, similar to young adults. Seated older (n = 11, 73.8 ± 5.6 yr) and young (n = 15, 23.8 ± 4.7 yr) adults made targeted reaching movements while grasping a robotic arm. We measured their metabolic rate continuously via expired gas analysis. A force field was used to add novel dynamics. Older adults had greater movement deviations and compensated for just 65% of the novel dynamics compared with 84% in young adults. As expected, older adults used greater muscle coactivation than young adults. Last, older adults reduced muscle activity with motor adaptation and had consistent reductions in metabolic cost later during motor adaptation, similar to young adults. These results suggest that despite increased muscle coactivation, older adults can adapt to the novel dynamics, albeit less accurately. These results also suggest that reductions in metabolic cost may be a fundamental feature of motor adaptation.

Reductions in muscle coactivation and metabolic cost during visuomotor adaptation.

We often have to adapt our movements as we interact with a variety of objects in various conditions on a daily basis. Evidence suggests that motor adaptation relies on a process that minimizes error and effort; however, much of this evidence involved adapting to novel dynamics with physical perturbations to counteract. To examine the generality of the process of minimizing error and effort during motor adaptation, we used a visuomotor adaptation task that did not involve dynamic perturbations. We investigated the time courses of muscle activity, coactivation, and metabolic cost as subjects reached to a target with a visuomotor rotation. We wanted to determine whether subjects would modulate muscle activity, coactivation, and metabolic cost during a visuomotor adaptation task. Interestingly, subjects increased muscle coactivation early during visuomotor adaptation when there were large cursor-trajectory errors but no physical perturbations to reject. As adaptation progressed, muscle activity and coactivation decreased. Metabolic cost followed a similar time course. When the perturbation was removed, typical after-effects were observed: trajectory error increased and then was reduced quickly. This was accompanied by increases in muscle activity, coactivation, and metabolic cost, along with subsequent rapid reductions. These results demonstrate that subjects modulate muscle activity, coactivation, and metabolic cost similarly across different forms of motor adaptation. Overall, our findings suggest that minimization of error and effort may be a general process underlying various forms of motor adaptation.

Transfer of postural adaptation depends on context of prior exposure.

Postural control is significantly affected by the postural base of support; however, the effects on postural adaptation are not well understood. Here we investigated how adaptation and transfer of anticipatory postural control are affected by stance width. Subjects made reaching movements in a novel dynamic environment while holding the handle of a force-generating robotic arm. Each subject initially adapted to the dynamics while standing in a wide stance and then switched to a narrow stance, or vice versa. Our hypothesis is that anticipatory postural control, reflected in center of pressure (COP) movement, is not affected by stance width, as long as the control remains within functional limits; therefore we predicted that subjects in either stance would show similar COP movement by the end of adaptation and immediately upon transfer to the other stance. We found that both groups showed similar adaptation of postural control, by using different muscle activation strategies to account for the differing stance widths. One group, after adapting in wide stance, transferred similar postural control to narrow stance, by modifying their muscle activity to account for the new stance. Interestingly, the other group showed an increase in postural control when transferring from narrow to wide stance, associated with no change in muscle activity. These results confirm that adaptation of anticipatory postural control is not affected by stance width, as long as the control remains within biomechanical limits. However, transfer of control between stance widths is affected by the initial context in which the task is learned.