Reliable population code for subjective economic value from heterogeneous neuronal signals in primate orbitofrontal cortex.

Behavior-related neuronal signals often vary between neurons, which might reflect the unreliability of individual neurons or a truly heterogeneous code. This notion may also apply to economic ("value-based") choices and the underlying reward signals. Reward value is subjective and can be described by a nonlinearly weighted magnitude (utility) and probability. Defining subjective values relies on the continuity axiom, whose testing involves structured variations of a wide range of reward magnitudes and probabilities. Axiom compliance demonstrates understanding of the stimuli and the meaningful character of choices. Using these tests, we investigated the encoding of subjective economic value by neurons in a key economic-decision structure of the monkey brain, the orbitofrontal cortex (OFC). We found that individual neurons carry heterogeneous neuronal value signals that largely fail to match the animal's choices. However, neuronal population signals matched the animal's choices well, suggesting accurate subjective economic value encoding by a heterogeneous population of unreliable neurons.

Systematic comparison of risky choices in humans and monkeys.

The past decades have seen tremendous progress in fundamental studies on economic choice in humans. However, elucidation of the underlying neuronal processes requires invasive neurophysiological studies that are met with difficulties in humans. Monkeys as evolutionary closest relatives offer a solution. The animals display sophisticated and well-controllable behavior that allows to implement key constructs of proven economic choice theories. However, the similarity of economic choice between the two species has never been systematically investigated. We investigated compliance with the independence axiom (IA) of expected utility theory as one of the most demanding choice tests and compared IA violations between humans and monkeys. Using generalized linear modeling and cumulative prospect theory (CPT), we found that humans and monkeys made comparable risky choices, although their subjective values (utilities) differed. These results suggest similar fundamental choice mechanism across these primate species and encourage to study their underlying neurophysiological mechanisms.

Evidence for a we-representation in monkeys when acting together.

A hallmark of human evolution resides in the ability to adapt our actions to those of others. This aptitude optimizes collective behavior, allowing to achieve goals unattainable by acting alone. We have previously shown that macaque monkeys are able to coordinate their actions when engaged in dyadic contexts, therefore they offer a good model to study the roots of joint action. Here, we analyze the behavior of five macaques required to perform visuomotor isometric tasks, either individually or together with a partner. By pre-cueing or not the future action condition (SOLO or TOGETHER) we investigated the existence of a 'We-representation' in monkeys. We found that pre-instructing the action context improves the dyadic performance, thanks to the emergence of an optimal kinematic setting, that facilitates inter-individual motor coordination. Our results offer empirical evidence of a 'We-representation' in macaques, that when evoked provides an overall beneficial effect on joint performance.

Shared population-level dynamics in monkey premotor cortex during solo action, joint action and action observation.

Studies of neural population dynamics of cell activity from monkey motor areas during reaching show that it mostly represents the generation and timing of motor behavior. We compared neural dynamics in dorsal premotor cortex (PMd) during the performance of a visuomotor task executed individually or cooperatively and during an observation task. In the visuomotor conditions, monkeys applied isometric forces on a joystick to guide a visual cursor in different directions, either alone or jointly with a conspecific. In the observation condition, they observed the cursor's motion guided by the partner. We found that in PMd neural dynamics were widely shared across action execution and observation, with cursor motion directions more accurately discriminated than task types. This suggests that PMd encodes spatial aspects irrespective of specific behavioral demands. Furthermore, our results suggest that largest components of premotor population dynamics, which have previously been suggested to reflect a transformation from planning to movement execution, may rather reflect higher cognitive-motor processes, such as the covert representation of actions and goals shared across tasks that require movement and those that do not.

Comparing utility functions between risky and riskless choice in rhesus monkeys.

Decisions can be risky or riskless, depending on the outcomes of the choice. Expected utility theory describes risky choices as a utility maximization process: we choose the option with the highest subjective value (utility), which we compute considering both the option's value and its associated risk. According to the random utility maximization framework, riskless choices could also be based on a utility measure. Neuronal mechanisms of utility-based choice may thus be common to both risky and riskless choices. This assumption would require the existence of a utility function that accounts for both risky and riskless decisions. Here, we investigated whether the choice behavior of two macaque monkeys in risky and riskless decisions could be described by a common underlying utility function. We found that the utility functions elicited in the two choice scenarios were different from each other, even after taking into account the contribution of subjective probability weighting. Our results suggest that distinct utility representations exist for risky and riskless choices, which could reflect distinct neuronal representations of the utility quantities, or distinct brain mechanisms for risky and riskless choices. The different utility functions should be taken into account in neuronal investigations of utility-based choice.

Risky choice: Probability weighting explains independence axiom violations in monkeys.

Expected Utility Theory (EUT) provides axioms for maximizing utility in risky choice. The Independence Axiom (IA) is its most demanding axiom: preferences between two options should not change when altering both options equally by mixing them with a common gamble. We tested common consequence (CC) and common ratio (CR) violations of the IA over several months in thousands of stochastic choices using a large variety of binary option sets. Three monkeys showed consistently few outright Preference Reversals (8%) but substantial graded Preference Changes (46%) between the initial preferred gamble and the corresponding altered gamble. Linear Discriminant Analysis (LDA) indicated that gamble probabilities predicted most Preference Changes in CC (72%) and CR (88%) tests. The Akaike Information Criterion indicated that probability weighting within Cumulative Prospect Theory (CPT) explained choices better than models using Expected Value (EV) or EUT. Fitting by utility and probability weighting functions of CPT resulted in nonlinear and non-parallel indifference curves (IC) in the Marschak-Machina triangle and suggested IA non-compliance of models using EV or EUT. Indeed, CPT models predicted Preference Changes better than EV and EUT models. Indifference points in out-of-sample tests were closer to CPT-estimated ICs than EV and EUT ICs. Finally, while the few outright Preference Reversals may reflect the long experience of our monkeys, their more graded Preference Changes corresponded to those reported for humans. In benefitting from the wide testing possibilities in monkeys, our stringent axiomatic tests contribute critical information about risky decision-making and serves as basis for investigating neuronal decision mechanisms. Supplementary information: The online version contains supplementary material available at 10.1007/s11166-022-09388-7.

Adaptation of utility functions to reward distribution in rhesus monkeys.

This study investigated how the experience of different reward distributions would shape the utility functions that can be inferred from economic choice. Despite the generally accepted notion that utility functions are not insensitive to external references, the exact way in which such changes take place remains largely unknown. Here we benefitted from the capacity to engage in thorough and prolonged empirical tests of economic choice by one of our evolutionary cousins, the rhesus macaque. We analyzed data from thousands of binary choices and found that the animals' preferences changed depending on the statistics of rewards experienced in the past (up to weeks) and that these changes could reflect monkeys' adapting their expectations of reward. The utility functions we elicited from their choices stretched and shifted over several months of sequential changes in the mean and range of rewards that the macaques experienced. However, this adaptation was usually incomplete, suggesting that - even after months - past experiences held weight when monkeys' assigned value to future rewards. Rather than having stable and fixed preferences assumed by normative economic models, our results demonstrate that rhesus macaques flexibly shape their preferences around the past and present statistics of their environment. That is, rather than relying on a singular reference-point, reference-dependent preferences are likely to capture a monkey's range of expectations.

Nonhuman Primates Satisfy Utility Maximization in Compliance with the Continuity Axiom of Expected Utility Theory.

Expected Utility Theory (EUT), the first axiomatic theory of risky choice, describes choices as a utility maximization process: decision makers assign a subjective value (utility) to each choice option and choose the one with the highest utility. The continuity axiom, central to Expected Utility Theory and its modifications, is a necessary and sufficient condition for the definition of numerical utilities. The axiom requires decision makers to be indifferent between a gamble and a specific probabilistic combination of a more preferred and a less preferred gamble. While previous studies demonstrated that monkeys choose according to combinations of objective reward magnitude and probability, a concept-driven experimental approach for assessing the axiomatically defined conditions for maximizing utility by animals is missing. We experimentally tested the continuity axiom for a broad class of gamble types in 4 male rhesus macaque monkeys, showing that their choice behavior complied with the existence of a numerical utility measure as defined by the economic theory. We used the numerical quantity specified in the continuity axiom to characterize subjective preferences in a magnitude-probability space. This mapping highlighted a trade-off relation between reward magnitudes and probabilities, compatible with the existence of a utility function underlying subjective value computation. These results support the existence of a numerical utility function able to describe choices, allowing for the investigation of the neuronal substrates responsible for coding such rigorously defined quantity.SIGNIFICANCE STATEMENT A common assumption of several economic choice theories is that decisions result from the comparison of subjectively assigned values (utilities). This study demonstrated the compliance of monkey behavior with the continuity axiom of Expected Utility Theory, implying a subjective magnitude-probability trade-off relation, which supports the existence of numerical utility directly linked to the theoretical economic framework. We determined a numerical utility measure able to describe choices, which can serve as a correlate for the neuronal activity in the quest for brain structures and mechanisms guiding decisions.

Two Brains in Action: Joint-Action Coding in the Primate Frontal Cortex.

Daily life often requires the coordination of our actions with those of another partner. After 50 years (1968-2018) of behavioral neurophysiology of motor control, the neural mechanisms that allow such coordination in primates are unknown. We studied this issue by recording cell activity simultaneously from dorsal premotor cortex (PMd) of two male interacting monkeys trained to coordinate their hand forces to achieve a common goal. We found a population of "joint-action cells" that discharged preferentially when monkeys cooperated in the task. This modulation was predictive in nature, because in most cells neural activity led in time the changes of the "own" and of the "other" behavior. These neurons encoded the joint-performance more accurately than "canonical action-related cells", activated by the action per se, regardless of the individual versus interactive context. A decoding of joint-action was obtained by combining the two brains' activities, using cells with directional properties distinguished from those associated to the "solo" behaviors. Action observation-related activity studied when one monkey observed the consequences of the partner's behavior, i.e., the cursor's motion on the screen, did not sharpen the accuracy of joint-action cells' representation, suggesting that it plays no major role in encoding joint-action. When monkeys performed with a non-interactive partner, such as a computer, joint-action cells' representation of the other (non-cooperative) behavior was significantly degraded. These findings provide evidence of how premotor neurons integrate the time-varying representation of the self-action with that of a co-actor, thus offering a neural substrate for successful visuomotor coordination between individuals.SIGNIFICANCE STATEMENT The neural bases of intersubject motor coordination were studied by recording cell activity simultaneously from the frontal cortex of two interacting monkeys, trained to coordinate their hand forces to achieve a common goal. We found a new class of cells, preferentially active when the monkeys cooperated, rather than when the same action was performed individually. These "joint-action neurons" offered a neural representation of joint-behaviors by far more accurate than that provided by the "canonical action-related cells", modulated by the action per se regardless of the individual/interactive context. A neural representation of joint-performance was obtained by combining the activity recorded from the two brains. Our findings offer the first evidence concerning neural mechanisms subtending interactive visuomotor coordination between co-acting agents.

Probability Distortion Depends on Choice Sequence in Rhesus Monkeys.

Humans and other primates share many decision biases, among them our subjective distortion of objective probabilities. When making choices between uncertain rewards we typically treat probabilities nonlinearly: overvaluing low probabilities of reward and undervaluing high ones. A growing body of evidence, however, points to a more flexible pattern of distortion than the classical inverse-S one, highlighting the effect of experimental conditions in shifting the weight assigned to probabilities, such as task feedback, learning, and attention. Here we investigated the role of sequence structure (the order in which gambles are presented in a choice task) in shaping the probability distortion patterns of rhesus macaques: we presented 2 male monkeys with binary choice sequences of MIXED or REPEATED gambles against safe rewards. Parametric modeling revealed that choices in each sequence type were guided by significantly different patterns of probability distortion: whereas we elicited the classical inverse-S-shaped probability distortion in pseudorandomly MIXED trial sequences of gamble-safe choices, we found the opposite pattern consisting of S-shaped distortion, with REPEATED sequences. We extended these results to binary choices between two gambles, without a safe option, and confirmed the unique influence of the sequence structure in which the animals make choices. Finally, we showed that the value of gambles experienced in the past had a significant impact on the subjective value of future ones, shaping probability distortion on a trial-by-trial basis. Together, our results suggest that differences in choice sequence are sufficient to reverse the direction of probability distortion.SIGNIFICANCE STATEMENT Our lives are peppered with uncertain, probabilistic choices. Recent studies showed how such probabilities are subjectively distorted. In the present study, we show that probability distortions in macaque monkeys differ significantly between sequences in which single gambles are repeated (S-shaped distortion), as opposed to being pseudorandomly intermixed with other gambles (inverse-S-shaped distortion). Our findings challenge the idea of fixed probability distortions resulting from inflexible computations, and points to a more instantaneous evaluation of probabilistic information. Past trial outcomes appeared to drive the "gap" between probability distortions in different conditions. Our data suggest that, as in most adaptive systems, probability values are slowly but constantly updated from prior experience, driving measures of probability distortion to either side of the S/inverse-S debate.

Modulation of Neural Variability in Premotor, Motor, and Posterior Parietal Cortex during Change of Motor Intention.

The time course of neural variability was studied in three nodes of the parieto-frontal system: the dorsal premotor cortex (PMd, area 6), primary motor cortex (MI, area 4), and posterior parietal cortex (PPC, area 5) while monkeys made either direct reaches to visual targets or changed reach direction in response to an unexpected change of target location. These areas are crucial nodes in the distributed control of reaching and their lesion impairs trajectory formation and correction under different circumstances. During unperturbed reaches, neural variability declined before the onset of hand movement in both frontal and parietal cortex. When the original motor intention suddenly changed, neural variability displayed a complex and area-specific modulation because the perturbation of the motor state was signaled earlier in PMd than in MI and PPC. The comparison of perturbed versus unperturbed reaches revealed that, in the time between the onset of correction signal and trajectory change, identical hand movements were associated with different, therefore context-dependent, patterns of neural variability induced by the instruction to change hand movement direction. In PMd, neural variability was higher before the initiation of hand reach than before its correction, thus providing a neural underpinning to the phenomenon that it takes less time to correct than to initiate hand movement. Furthermore, neural variability was an excellent predictor of slow and fast reach corrections because it was lower during the latter than the former. We conclude that the analysis of neural variability can be an important tool for the study of complex forms of motor cognition. SIGNIFICANCE STATEMENT: No single study has been performed on neural variability during update of motor intention across monkey premotor, motor, and posterior parietal cortex. In perturbed reaches, target location changed unexpectedly during reaction time and the correction of hand trajectory required updating the original motor plan. Comparing unperturbed versus perturbed reaches revealed that neural variability displayed a complex context- and area-dependent pattern of modulation because, before trajectory correction, similar initial hand movements were associated with different patterns of variability depending on the instruction signal, and therefore on the future hand path and final destination. Furthermore, neural variability predicted both slow and fast hand movement corrections, also offering a neural underpinning to the phenomenon that it takes less time to correct than to initiate hand movement.

Posterior Parietal Cortex Encoding of Dynamic Hand Force Underlying Hand-Object Interaction.

Major achievements of primate evolution are skilled hand-object interaction and tool use, both in part dependent on parietal cortex expansion. We recorded spiking activity from macaque inferior parietal cortex during directional manipulation of an isometric tool, which required the application of hand forces to control a cursor's motion on a screen. In areas PFG/PF, the activity of ∼ 70% neurons was modulated by the hand force necessary to implement the desired target motion, reflecting an inverse model, rather than by the intended motion of the visual cursor (forward model). The population vector matched the direction and amplitude of the instantaneous force increments over time. When exposed to a new force condition, that obliged the monkey to change the force output to successfully bring the cursor to the final target, the activity of a consistent subpopulation of neurons changed in an orderly fashion and, at the end of a "Wash-out" session, retained memory of the new learned association, at the service of predictive control of force. Our findings suggest that areas PFG/PF represent a crucial node of the distributed control of hand force, by encoding instantaneous force variations and serving as a memory reservoir of hand dynamics required for object manipulation and tool use. This is coherent with previous studies in humans showing the following: (1) impaired adaptation to a new force field under TMS parietal perturbation; (2) defective control of direction of hand force after parietal lesion; and (3) fMRI activation of parietal cortex during object manipulation requiring control of fine hand forces. SIGNIFICANCE STATEMENT: Skilled object manipulation and tool use are major achievements of primate evolution, both largely dependent on posterior parietal cortex (PPC) expansion. Neurophysiological and fMRI studies in macaque and humans had documented a crucial role of PPC in encoding the hand kinematics underlying these functions, leaving to premotor and motor areas the role of specifying the underlying hand forces. We recorded spiking activity from macaque PPC during manipulation of an isometric tool and found that population activity is not only modulated by the dynamic hand force and its change over time, but also retains memory of the exerted force, as a reservoir to guide of future hand action. This suggests parallel parietal encoding of hand dynamics and kinematics during object manipulation.

Do non-human primates cooperate? Evidences of motor coordination during a joint action task in macaque monkeys.

Humans are intensively social primates, therefore many of their actions are dedicated to communication and interaction with other individuals. Despite the progress in understanding the cognitive and neural processes that allow humans to perform cooperative actions, in non-human primates only few studies have investigated the ability to interact with a partner in order to reach a common goal. These studies have shown that in naturalistic conditions animals engage in various types of social behavior that involve forms of mutual coordination and cooperation. However, little is known on the capacity of non-human primates to actively cooperate in a controlled experimental setting, which allows full characterization of the motor parameters underlying individual action and their change during motor cooperation. To this aim, we analyzed the behavior of three pairs of macaque monkeys trained to perform solo and joint-actions by exerting a force on an isometric joystick, as to move an individual or a common cursor toward visual targets on a screen. We found that during cooperation monkeys reciprocally adapt their behavior by changing the parameters that define the spatial and temporal aspects of their action, as to fine tune their joint effort, and maximize their common performance. Furthermore the results suggest that when acting together the movement parameters that specify each actor's behavior are not only modulated during execution, but also during planning. These findings provide the first quantitative description of action coordination in non-human primates during the performance of a joint action task.

Timing and communication of parietal cortex for visuomotor control.

In both monkeys and humans, motor cognition emerges from a parietal-frontal network containing discrete dominant domains of visual, eye and hand signals, where neurons are responsible for goal and effector selection. Within these domains, the combination of different inputs shape the tuning properties of neurons, while local and long cortico-cortical connections outline the architecture of the distributed network and determine the conduction time underlying eye-hand coordination, necessary for visually guided operations in the action space. The analysis of the communication timing between parietal and frontal nodes of the network helps understanding the sensorimotor cortical delays associated to different functions, such as online control of movement and eye-hand coordination, and opens a new perspective to the study of the parieto-frontal interactions.

Impairment of online control of hand and eye movements in a monkey model of optic ataxia.

The parietal mechanisms for online control of hand trajectory were studied by combining single-cell recording and reversible inactivation of superior parietal area 5 (PE/PEc; SPL) of monkeys while these made reaches and saccades to visual targets, when the target position changed unexpectedly. Neural activity was modulated by hand position, speed, and movement direction, and by pre- and/or postsaccadic signals. After bilateral muscimol injection, an increase in the hand reaction- and movement-time toward both the first and second targets was observed. This caused an increase in the time necessary for the trajectory correction, and therefore an elongation of the hand-path toward the first target location. Furthermore, hand trajectories were different in shape than control ones. An elongation of the eye reaction time to both first and second targets was also observed, which could partially explain the deficit of planning and correction of hand movement. These results identify the superior parietal lobule as a crucial node in the online control of hand and eye movement and highlight the role of the eye impairment in the emergence of the reaching disorder so far regarded as the hallmark of optic ataxia.

Online control of hand trajectory and evolution of motor intention in the parietofrontal system.

The frontal mechanisms of motor intention were studied in dorsal premotor and motor cortex of monkeys making direct reaches to visual targets and online corrections of hand trajectory, whenever a change of the target's location occurred. This study and our previous one of parietal cortex (Archambault et al., 2009) provide a picture on the evolution of motor intention and online control of movement in the parietofrontal system. In frontal cortex, significant relationships were found between neural activity and hand kinematics (position, speed, and movement direction). When a change of motor intention occurred, the activity typical of the movement to the first target smoothly evolved into that associated with the movement toward the second one, as observed during direct reaches. Under these conditions, parietal cells remained a more accurate predictor of hand trajectory than frontal ones. The time lags of neural activity with hand kinematics showed that motor, premotor, and parietal cortex were activated sequentially. After the first target's presentation and its change of location, the population activity signaled the change of motor plan before the hand moved to the initial target's position. This signaling occurred earlier in premotor than in motor and parietal cortex. Thus, premotor cortex encodes a higher-order command for the correction of motor intention, while parietal cortex seems responsible for estimating the kinematics of the motor periphery, an essential step to allow motor cortex to modify the hand trajectory. This indicates that the parietofrontal system can update an original and not-yet-accomplished motor plan during its execution.