Formalizing planning and information search in naturalistic decision-making.

Decisions made by mammals and birds are often temporally extended. They require planning and sampling of decision-relevant information. Our understanding of such decision-making remains in its infancy compared with simpler, forced-choice paradigms. However, recent advances in algorithms supporting planning and information search provide a lens through which we can explain neural and behavioral data in these tasks. We review these advances to obtain a clearer understanding for why planning and curiosity originated in certain species but not others; how activity in the medial temporal lobe, prefrontal and cingulate cortices may support these behaviors; and how planning and information search may complement each other as means to improve future action selection.

Anterior cingulate activity during routine and non-routine sequential behaviors in macaques.

Anterior cingulate cortex is important in monitoring action for new challenges. We recorded neuron activity in the anterior cingulate sulcus of macaques while they performed a sequential problem-solving task. By trial and error, animals determined the correct sequence for touching three fixed spatial targets. After the sequence was repeated three times, we then changed the correct solution order, requiring a new search. Irrespective of component movements or their kinematics, task-related neurons encoded the serial order of the sequence. Neurons activated with sequence components (68%) differed in activity between search and repetition. Search-related activity occurred when behavioral flexibility was required and ended as soon as the animal accumulated enough information to infer the solution, but had not yet tested it. Repetition-related activity occurred in a regime of memory-based motor performance in which attention to action is less necessary.

The effects of sequence structure and reward schedule on serial reaction time learning in the monkey.

This research tests the hypothesis that sequence learning performance in non-human primates will be modulated both by the structure of the sequences to be learned and by the schedule of reward applied during learning. Sequence learning in humans has been extensively explored with serial reaction time (SRT) protocols where learning is revealed by reduced reaction times for stimuli presented in repeating sequences vs. stimuli presented in random series. The SRT protocol has been used to demonstrate that different types of sequential structure may be learned under different awareness conditions. Here, we consider surface and abstract structure of sensorimotor sequences such that sequences ABCBAC and DEFEDF (where A to F correspond to spatial locations on a touch sensitive screen) have different serial order or surface structure, but share the same abstract structure 123213, and are thus considered isomorphic. In four experiments, we manipulated the type of sequential structure to be learned, and the schedule of reward in spatial sequence learning tasks. Both of the two monkeys tested demonstrated significant SRT learning for serial order or surface structure, while they failed to learn and transfer abstract structure. Their learning performance was also modulated by the schedule of reward. These results are in support of our hypothesis and are discussed in the context of existing models of sensorimotor sequence learning.

Problem solving and logical reasoning in the macaque monkey.

This study focuses on the performances of monkeys in a spatial problem-solving task that involves working memory. Two monkeys had to find, by trial-and-error, the touching order of 2 or 3 targets in a set of 3 or 4 fixed spatial targets. When a solution was found and performed 6 times, the order was changed and the animal had to resume a new search within the same set of targets. Thus, in a training session, many searches (up to 60) could be initialised. The data show that the animals conducted a methodical search for the hidden order and found the solution in a minimal number of trials. We conclude that the monkey is able to construct complex cognitive structures, similar to logical reasoning, to solve spatial problems of this type.

Automated extraction and variability analysis of sulcal neuroanatomy.

Systematic mapping of the variability in cortical sulcal anatomy is an area of increasing interest which presents numerous methodological challenges. To address these issues, we have implemented sulcal extraction and assisted labeling (SEAL) to automatically extract the two-dimensional (2-D) surface ribbons that represent the median axis of cerebral sulci and to neuroanatomically label these entities. To encode the extracted three-dimensional (3-D) cortical sulcal schematic topography (CSST) we define a relational graph structure composed of two main features: vertices (representing sulci) and arcs (representing the relationships between sulci). Vertices contain a parametric representation of the surface ribbon buried within the sulcus. Points on this surface are expressed in stereotaxic coordinates (i.e., with respect to a standardized brain coordinate system). For each of these vertices, we store length, depth, and orientation as well as anatomical attributes (e.g., hemisphere, lobe, sulcus type, etc.). Each arc stores the 3-D location of the junction between sulci as well as a list of its connecting sulci. Sulcal labeling is performed semiautomatically by selecting a sulcal entity in the CSST and selecting from a menu of candidate sulcus names. In order to help the user in the labeling task, the menu is restricted to the most likely candidates by using priors for the expected sulcal spatial distribution. These priors, i.e., sulcal probabilistic maps, were created from the spatial distribution of 34 sulci traced manually on 36 different subjects. Given these spatial probability maps, the user is provided with the likelihood that the selected entity belongs to a particular sulcus. The cortical structure representation obtained by SEAL is suitable to extract statistical information about both the spatial and the structural composition of the cerebral cortical topography. This methodology allows for the iterative construction of a successively more complete statistical models of the cerebral topography containing spatial distributions of the most important structures, their morphometrics, and their structural components.

Reward encoding in the monkey anterior cingulate cortex.

The anterior cingulate cortex (ACC) is known to play a crucial role in the fast adaptations of behavior based on immediate reward values. What is less certain is whether the ACC is also involved in long-term adaptations to situations with uncertain outcomes. To study this issue, we placed macaque monkeys in a probabilistic context in which the appropriate strategy to maximize reward was to identify the stimulus with the highest reward value (optimal stimulus). Only knowledge of the theoretical average reward value associated with this stimulus--referred to as 'the task value'--was available. Remarkably, in each trial, ACC pre-reward activity correlated with the task value. Importantly, this neuronal activity was observed prior to the discovery of the optimal stimulus. We hypothesize that the received rewards and the task value, constructed a priori through learning, are used to guide behavior and identify the optimal stimulus. We tested this hypothesis by muscimol deactivation of the ACC. As predicted, this inactivation impaired the search for the optimal stimulus. We propose that ACC participates in long-term adaptation of voluntary reward-based behaviors by encoding general task values and received rewards.

Characterization of serial order encoding in the monkey anterior cingulate sulcus.

In a previous report we showed that neurons in the anterior cingulate cortex might encode the serial order of the three components (first, second and third) of motor sequences, irrespective of which component is performed, and irrespective of the component that precedes or follows. Here we further explore these data by comparing the magnitude of cell activity at the different ranks. We also compare the activity recorded in the motor sequences and in tasks with only one motor component. We finally discuss functional hypotheses, which may account for the serial order encoding.

Brain activity during observation of actions. Influence of action content and subject's strategy.

PET was used to map brain regions that are associated with the observation of meaningful and meaningless hand actions. Subjects were scanned under four conditions which consisted of visually presented actions. In each of the four experimental conditions, they were instructed to watch the actions with one of two aims: to be able to recognize or to imitate them later. We found that differences in the meaning of the action, irrespective of the strategy used during observation, lead to different patterns of brain activity and clear left/right asymmetries. Meaningful actions strongly engaged the left hemisphere in frontal and temporal regions while meaningless actions involved mainly the right occipitoparietal pathway. Observing with the intent to recognize activated memory-encoding structures. In contrast, observation with the intent to imitate was associated with activation in the regions involved in the planning and in the generation of actions. Thus, the pattern of brain activation during observation of actions is dependent both on the nature of the required executive processing and the type of the extrinsic properties of the action presented.