Cognitive mechanisms of learning in sequential decision-making under uncertainty: an experimental and theoretical approach.

Learning to make adaptive decisions involves making choices, assessing their consequence, and leveraging this assessment to attain higher rewarding states. Despite vast literature on value-based decision-making, relatively little is known about the cognitive processes underlying decisions in highly uncertain contexts. Real world decisions are rarely accompanied by immediate feedback, explicit rewards, or complete knowledge of the environment. Being able to make informed decisions in such contexts requires significant knowledge about the environment, which can only be gained via exploration. Here we aim at understanding and formalizing the brain mechanisms underlying these processes. To this end, we first designed and performed an experimental task. Human participants had to learn to maximize reward while making sequences of decisions with only basic knowledge of the environment, and in the absence of explicit performance cues. Participants had to rely on their own internal assessment of performance to reveal a covert relationship between their choices and their subsequent consequences to find a strategy leading to the highest cumulative reward. Our results show that the participants' reaction times were longer whenever the decision involved a future consequence, suggesting greater introspection whenever a delayed value had to be considered. The learning time varied significantly across participants. Second, we formalized the neurocognitive processes underlying decision-making within this task, combining mean-field representations of competing neural populations with a reinforcement learning mechanism. This model provided a plausible characterization of the brain dynamics underlying these processes, and reproduced each aspect of the participants' behavior, from their reaction times and choices to their learning rates. In summary, both the experimental results and the model provide a principled explanation to how delayed value may be computed and incorporated into the neural dynamics of decision-making, and to how learning occurs in these uncertain scenarios.

Response inhibition in premotor cortex corresponds to a complex reshuffle of the mesoscopic information network.

Recent studies have explored functional and effective neural networks in animal models; however, the dynamics of information propagation among functional modules under cognitive control remain largely unknown. Here, we addressed the issue using transfer entropy and graph theory methods on mesoscopic neural activities recorded in the dorsal premotor cortex of rhesus monkeys. We focused our study on the decision time of a Stop-signal task, looking for patterns in the network configuration that could influence motor plan maturation when the Stop signal is provided. When comparing trials with successful inhibition to those with generated movement, the nodes of the network resulted organized into four clusters, hierarchically arranged, and distinctly involved in information transfer. Interestingly, the hierarchies and the strength of information transmission between clusters varied throughout the task, distinguishing between generated movements and canceled ones and corresponding to measurable levels of network complexity. Our results suggest a putative mechanism for motor inhibition in premotor cortex: a topological reshuffle of the information exchanged among ensembles of neurons.

In this study, we investigated the dynamics of information transfer among functionally identified neural modules during cognitive motor control. Our focus was on mesoscopic neural activities in the dorsal premotor cortex of rhesus monkeys engaged in a Stop-signal task. Leveraging multivariate transfer entropy and graph theory, we uncovered insights on how behavioral control shapes the topology of information transmission in a local brain network. Task phases modulated the strength and hierarchy of information exchange between modules, revealing the nuanced interplay between neural populations during generated and canceled movements. Notably, during successful inhibition, the network displayed a distinctive configuration, unveiling a novel mechanism for motor inhibition in the premotor cortex: a topological reshuffle of information among neuronal ensembles.

Neural Activity in Quarks Language: Lattice Field Theory for a Network of Real Neurons.

Brain-computer interfaces have seen extraordinary surges in developments in recent years, and a significant discrepancy now exists between the abundance of available data and the limited headway made in achieving a unified theoretical framework. This discrepancy becomes particularly pronounced when examining the collective neural activity at the micro and meso scale, where a coherent formalization that adequately describes neural interactions is still lacking. Here, we introduce a mathematical framework to analyze systems of natural neurons and interpret the related empirical observations in terms of lattice field theory, an established paradigm from theoretical particle physics and statistical mechanics. Our methods are tailored to interpret data from chronic neural interfaces, especially spike rasters from measurements of single neuron activity, and generalize the maximum entropy model for neural networks so that the time evolution of the system is also taken into account. This is obtained by bridging particle physics and neuroscience, paving the way for particle physics-inspired models of the neocortex.

Attentional spatial cueing of the stop-signal affects the ability to suppress behavioural responses.

The ability to adapt to the environment is linked to the possibility of inhibiting inappropriate behaviours, and this ability can be enhanced by attention. Despite this premise, the scientific literature that assesses how attention can influence inhibition is still limited. This study contributes to this topic by evaluating whether spatial and moving attentional cueing can influence inhibitory control. We employed a task in which subjects viewed a vertical bar on the screen that, from a central position, moved either left or right where two circles were positioned. Subjects were asked to respond by pressing a key when the motion of the bar was interrupted close to the circle (go signal). In about 40% of the trials, following the go signal and after a variable delay, a visual target appeared in either one of the circles, requiring response inhibition (stop signal). In most of the trials the stop signal appeared on the same side as the go signal (valid condition), while in the others, it appeared on the opposite side (invalid condition). We found that spatial and moving cueing facilitates inhibitory control in the valid condition. This facilitation was observed especially for stop signals that appeared within 250ms of the presentation of the go signal, thus suggesting an involvement of exogenous attentional orienting. This work demonstrates that spatial and moving cueing can influence inhibitory control, providing a contribution to the investigation of the relationship between spatial attention and inhibitory control.

The transitive inference task to study the neuronal correlates of memory-driven decision making: A monkey neurophysiology perspective.

A vast amount of literature agrees that rank-ordered information as A>B>C>D>E>F is mentally represented in spatially organized schemas after learning. This organization significantly influences the process of decision-making, using the acquired premises, i.e. deciding if B is higher than D is equivalent to comparing their position in this space. The implementation of non-verbal versions of the transitive inference task has provided the basis for ascertaining that different animal species explore a mental space when deciding among hierarchically organized memories. In the present work, we reviewed several studies of transitive inference that highlighted this ability in animals and, consequently, the animal models developed to study the underlying cognitive processes and the main neural structures supporting this ability. Further, we present the literature investigating which are the underlying neuronal mechanisms. Then we discuss how non-human primates represent an excellent model for future studies, providing ideal resources for better understanding the neuronal correlates of decision-making through transitive inference tasks.

Reward prospect affects strategic adjustments in stop signal task.

Interaction with the environment requires us to predict the potential reward that will follow our choices. Rewards could change depending on the context and our behavior adapts accordingly. Previous studies have shown that, depending on reward regimes, actions can be facilitated (i.e., increasing the reward for response) or interfered (i.e., increasing the reward for suppression). Here we studied how the change in reward perspective can influence subjects' adaptation strategy. Students were asked to perform a modified version of the Stop-Signal task. Specifically, at the beginning of each trial, a Cue Signal informed subjects of the value of the reward they would receive; in one condition, Go Trials were rewarded more than Stop Trials, in another, Stop Trials were rewarded more than Go Trials, and in the last, both trials were rewarded equally. Subjects participated in a virtual competition, and the reward consisted of points to be earned to climb the leaderboard and win (as in a video game contest). The sum of points earned was updated with each trial. After a learning phase in which the three conditions were presented separately, each subject performed 600 trials testing phase in which the three conditions were randomly mixed. Based on the previous studies, we hypothesized that subjects could employ different strategies to perform the task, including modulating inhibition efficiency, adjusting response speed, or employing a constant behavior across contexts. We found that to perform the task, subjects preferentially employed a strategy-related speed of response adjustment, while the duration of the inhibition process did not change significantly across the conditions. The investigation of strategic motor adjustments to reward's prospect is relevant not only to understanding how action control is typically regulated, but also to work on various groups of patients who exhibit cognitive control deficits, suggesting that the ability to inhibit can be modulated by employing reward prospects as motivational factors.

Restart errors reaction time of a two-step inhibition process account for the violation of the race model's independence in multi-effector selective stop signal task.

Goal-oriented actions often require the coordinated movement of two or more effectors. Sometimes multi-effector movements need to be adjusted according to a continuously changing environment, requiring stopping an effector without interrupting the movement of the others. This form of control has been investigated by the selective Stop Signal Task (SST), requiring the inhibition of an effector of a multicomponent action. This form of selective inhibition has been hypothesized to act through a two-step process, where a temporary global inhibition deactivating all the ongoing motor responses is followed by a restarting process that reactivates only the moving effector. When this form of inhibition takes place, the reaction time (RT) of the moving effector pays the cost of the previous global inhibition. However, it is poorly investigated if and how this cost delays the RT of the effector that was required to be stopped but was erroneously moved (Stop Error trials). Here we measure the Stop Error RT in a group of participants instructed to simultaneously rotate the wrist and lift the foot when a Go Signal occurred, and interrupt both movements (non-selective Stop version) or only one of them (selective Stop version) when a Stop Signal was presented. We presented this task in two experimental conditions to evaluate how different contexts can influence a possible proactive inhibition on the RT of the moving effector in the selective Stop versions. In one context, we provided the foreknowledge of the effector to be inhibited by presenting the same selective or non-selective Stop versions in the same block of trials. In a different context, while providing no foreknowledge of the effector(s) to be stopped, the selective and non-selective Stop versions were intermingled, and the information on the effector to be stopped was delivered at the time of the Stop Signal presentation. We detected a cost in both Correct and Error selective Stop RTs that was influenced by the different task conditions. Results are discussed within the framework of the race model related to the SST, and its relationship with a restart model developed for selective versions of this paradigm.

Response Preparation Affects Cognitive Motor Control.

OBJECTIVE: We investigated how the ability to control whether or not to inhibit an action is affected by the response preparation. BACKGROUND: The ability to control actions is a central skill to properly behave in complex environments. Increased levels of response preparation are associated with reduced response times, but how they directly affect the ability to control actions is not well explored. We investigated how the response preparation affects the ability to control the generation of actions in the context of a stop selective task. METHOD: Participants performed a visuo-motor stop selective task. RESULTS: We found that an increased level of response preparation reduced the ability to control actions. In the condition with high preparation, we observed shorter response times and increased probability of wrong responses to a request to stop, compared to a condition with a lower level of preparation. CONCLUSION: We demonstrated that high response preparation hinders action control. APPLICATION: Understanding the cognitive factors that affect the ability to properly control actions is crucial to develop devices that can be exploited in different contexts such as the aviation, industrial, and military. We demonstrated that subjects' response preparation is a key factor influencing their ability to flexibly control their reaction to different stimuli. This study offers a suitable paradigm that can be used to investigate which system features in a controlled task promote an optimal balance between response speed and error rate.

Neuronal population dynamics during motor plan cancellation in nonhuman primates.

To understand the cortical neuronal dynamics behind movement generation and control, most studies have focused on tasks where actions were planned and then executed using different instances of visuomotor transformations. However, to fully understand the dynamics related to movement control, one must also study how movements are actively inhibited. Inhibition, indeed, represents the first level of control both when different alternatives are available and only one solution could be adopted and when it is necessary to maintain the current position. We recorded neuronal activity from a multielectrode array in the dorsal premotor cortex (PMd) of monkeys performing a countermanding reaching task that requires, in a subset of trials, them to cancel a planned movement before its onset. In the analysis of the neuronal state space of PMd, we found a subspace in which activities conveying temporal information were confined during active inhibition and position holding. Movement execution required activities to escape from this subspace toward an orthogonal subspace and, furthermore, surpass a threshold associated with the maturation of the motor plan. These results revealed further details in the neuronal dynamics underlying movement control, extending the hypothesis that neuronal computation confined in an "output-null" subspace does not produce movements.

Neuronal Activity in the Premotor Cortex of Monkeys Reflects Both Cue Salience and Motivation for Action Generation and Inhibition.

Reward prospect weighs on motor decision processes, enhancing the selection of appropriate actions and the inhibition of others. While many studies have investigated the neuronal basis of reward representations and of cortical control of actions, the neuronal correlates of the influences of reward prospect on motor decisions are less clear. We recorded from the dorsal premotor cortex (PMd) of 2 male macaque monkeys performing a modified version of the Stop-signal (countermanding) task. This task challenges motor decisions by requiring responding to a frequent Go stimulus, but to suppress this response when a rare Stop signal is presented during the reaction time. We unbalanced the motivation to respond or to suppress the response by presenting a cue informing on three different rewards schedules: in one case, Go trials were rewarded more than Stop trials; in another case, Stop trials were rewarded more than Go trials; in the last case, both types of trials were rewarded equally. Monkeys adopted different strategies according to reward information provided by the cue: the higher the reward for Stop trials, the higher their ability to suppress the response and the slower their response to Go stimuli. PMd neuronal activity evolved in time and correlated with the behavior: PMd signaled first the cue salience, representing the chance to earn the highest reward at stake, then reflected the shaping of the motor choice by the motivation to move or to stop. These findings represent a neuronal correlate of the influence of reward information on motor decision.SIGNIFICANCE STATEMENT The motivation to obtain rewards drives how animals act over their environment. To explore the involvement of motor cortices in motivated behaviors, we recorded high-resolution neuronal activity in the premotor cortex of monkeys performing a task that manipulated the motivation to generate/withhold a movement through different cued reward probabilities. Our results show the presence of neuronal signals dynamically reflecting the salience of the cue, in the time immediately following its presentation, and a motivation-related activity in performing (or cancelling) a motor program, while the behavioral response approached. The encoding of multiple reward-related signals in this region leads to consider an important role of premotor areas in the reward circuitry supporting action.

Neuronal dynamics of signal selective motor plan cancellation in the macaque dorsal premotor cortex.

Primates adopt various strategies to interact with the environment. Yet, no study has examined the effects of behavioural strategies with regard to how movement inhibition is implemented at the neuronal level. We used a modified version of the stop-task by adding an extra signal - termed the Ignore signal - capable of influencing the inhibition of movements only within a specific strategy. We simultaneously recorded multisite neuronal activity from the dorsal premotor (PMd) cortex of macaque monkeys during the task and applied a state-space approach. As a result, we found that movement generation is characterized by neuronal dynamics that evolve between subspaces. When the movement is halted, this evolution is arrested and inverted. Conversely, when the Ignore signal is presented, inversion of the evolution is observed briefly and only when a specific behavioural strategy is adopted. Moreover, neuronal signatures during the inhibitory process were predictive of how PMd processes inhibitory signals, allowing the classification of the resulting behavioural strategy. Our data further corroborate the PMd as a critical node in movement inhibition.

Dorsal Premotor Cortex Neurons Signal the Level of Choice Difficulty during Logical Decisions.

Studies on the neuronal correlates of decision making have demonstrated that the continuous flow of sensorial information is integrated by sensorimotor brain areas in order to select one among simultaneously represented targets and potential actions. In contrast, little is known about how these areas integrate memory information to lead to similar decisions. Using serial order learning, we explore how fragments of information, learned and stored independently (e.g., A > B and B > C), are linked in an abstract representation according to their reciprocal relations (such as A > B > C) and how this representation can be accessed and manipulated to make decisions. We show that manipulating information after learning occurs with increased difficulty as logical relationships get closer in the mental map and that the activity of neurons in the dorsal premotor cortex (PMd) encodes the difficulty level during target selection for motor decision making at the single-neuron and population levels.

The small scale functional topology of movement control: Hierarchical organization of local activity anticipates movement generation in the premotor cortex of primates.

How neurons coordinate their collective activity for behavioural control is an open question in neuroscience. Several studies have progressively proven, on various scales, that the patterns of neural synchronization change accordingly with behavioural events. However, the topological features of the neural dynamics that underlie task-based cognitive decisions on the small scale level are not understood. We analysed the multiunit activity (MUA) from a multielectrode (96 channels) array of the dorsal premotor cortex (PMd) in rhesus monkeys during a countermanding reaching task. Within the framework of graph theory, we found that in the local PMd network motor execution is preceded by the emergence of hubs of anti-correlation that are organized in a hierarchical manner. Conversely, this organization is absent when monkeys correctly inhibit programmed movements. Thus, we interpret the presence of hubs as reflecting the readiness of the motor plan and the irrevocable signature of the onset of the incoming movement.

High Cervical Spinal Cord Stimulation: A One Year Follow-Up Study on Motor and Non-Motor Functions in Parkinson's Disease.

BACKGROUND: The present study investigated the effectiveness of stimulation applied at cervical levels on pain and Parkinson's disease (PD) symptoms using either tonic or burst stimulation mode. METHODS: Tonic high cervical spinal cord stimulation (T-HCSCS) was applied on six PD patients suffering from low back pain and failed back surgery syndrome, while burst HCSCS (B-HCSCS) was applied in twelve PD patients to treat primarily motor deficits. Stimulation was applied percutaneously with quadripolar or octapolar electrodes. Clinical evaluation was assessed by the Unified Parkinson's Disease Rating Scale (UPDRS) and the Hoehn and Yahr (H&Y) scale. Pain was evaluated by a visual analog scale. Evaluations of gait and of performance in a cognitive motor task were performed in some patients subjected to B-HCSCS. One patient who also suffered from severe autonomic cardiovascular dysfunction was investigated to evaluate the effectiveness of B-HCSCS on autonomic functions. RESULTS: B-HCSCS was more effective and had more consistent effects than T-HCSCS in reducing pain. In addition, B-HCSCS improved UPDRS scores, including motor sub-items and tremor and H&Y score. Motor benefits appeared quickly after the beginning of B-HCSCS, in contrast to long latency improvements induced by T-HCSCS. A slight decrease of effectiveness was observed 12 months after implantation. B-HCSCS also improved gait and ability of patients to correctly perform a cognitive-motor task requiring inhibition of a prepared movement. Finally, B-HCSCS ameliorated autonomic control in the investigated patient. CONCLUSIONS: The results support a better usefulness of B-HCSCS compared to T-HCSCS in controlling pain and specific aspects of PD motor and non-motor deficits for at least one year.

The Puzzling Relationship between Attention and Motivation: Do Motor Biases Matter?

The relationship between attention and incentive motivation has been mostly examined by administering Posner style cueing tasks in humans and varying monetary stakes. These studies found that higher incentives improved performance independently of spatial attention. However, the ability of the cueing task to measure actual attentional orienting has been debated by several groups that have highlighted the function of the motor system in affecting the behavioral features that are commonly attributed to spatial attention. To determine the impact of motor influences on the interplay between attention and motivation, we administered 2 reaching versions of a cueing task to monkeys in various motor scenarios. In both tasks, a central stimulus indicated the reward stake and predicted the stimulus target location in 80% of trials. In Experiment 1, subjects were requested to report the detection of a target stimulus in each trial. In Experiment 2, the task was modified to fit a paradigm of Go/NoGo target identification. We found that attention and motivation interacted exclusively in Experiment 2, wherein anticipated motor activation was discouraged and more demanding visual processing was imposed. Consequently, we suggest a protocol that provides novel insights into the study of the relationship between spatial attention and motivation and highlights the influence of the arm motor system in the estimation of the deployment of spatial attention.

Visual salience of the stop signal affects the neuronal dynamics of controlled inhibition.

The voluntary control of movement is often tested by using the countermanding, or stop-signal task that sporadically requires the suppression of a movement in response to an incoming stop-signal. Neurophysiological recordings in monkeys engaged in the countermanding task have shown that dorsal premotor cortex (PMd) is implicated in movement control. An open question is whether and how the perceptual demands inherent the stop-signal affects inhibitory performance and their underlying neuronal correlates. To this aim we recorded multi-unit activity (MUA) from the PMd of two male monkeys performing a countermanding task in which the salience of the stop-signals was modulated. Consistently to what has been observed in humans, we found that less salient stimuli worsened the inhibitory performance. At the neuronal level, these behavioral results were subtended by the following modulations: when the stop-signal was not noticeable compared to the salient condition the preparatory neuronal activity in PMd started to be affected later and with a less sharp dynamic. This neuronal pattern is probably the consequence of a less efficient inhibitory command useful to interrupt the neural dynamic that supports movement generation in PMd.

Coding of Self and Other's Future Choices in Dorsal Premotor Cortex during Social Interaction.

Representing others' intentions is central to primate social life. We explored the role of dorsal premotor cortex (PMd) in discriminating between self and others' behavior while two male rhesus monkeys performed a non-match-to-goal task in a monkey-human paradigm. During each trial, two of four potential targets were randomly presented on the right and left parts of a screen, and the monkey or the human was required to choose the one that did not match the previously chosen target. Each agent had to monitor the other's action in order to select the correct target in that agent's own turn. We report neurons that selectively encoded the future choice of the monkey, the human agent, or both. Our findings suggest that PMd activity shows a high degree of self-other differentiation during face-to-face interactions, leading to an independent representation of what others will do instead of entailing self-centered mental rehearsal or mirror-like activities.

Neural activity in macaque medial frontal cortex represents others' choices.

Predicting the behavior of others is a fundamental skill in primate social life. We tested the role of medial frontal cortex in the prediction of other agents' behavior in two male macaques, using a monkey-human interactive task in which their actor-observer roles were intermixed. In every trial, the observer monitored the actor's choice to reject it for a different one when he became the actor on the subsequent trial. In the delay period preceding the action, we identified neurons modulated by the agent's identity, as well as a group of neurons encoding the agent's future choice, some of which were neurons that showed differential patterns of activity between agents. The ability of these neurons to flexibly move from 'self-oriented' to 'other-oriented' representations could correspond to the "other side of the coin" of the simulative mirroring activity. Neurons that changed coding scheme, together with neurons exclusively involved in the prediction of the other agent's choice, show a neural substrate for predicting or anticipating others' choices beyond simulation.