Influence of Rule- and Reward-based Strategies on Inferences of Serial Order by Monkeys.

Knowledge of transitive relationships between items can contribute to learning the order of a set of stimuli from pairwise comparisons. However, cognitive mechanisms of transitive inferences based on rank order remain unclear, as are relative contributions of reward associations and rule-based inference. To explore these issues, we created a conflict between rule- and reward-based learning during a serial ordering task. Rhesus macaques learned two lists, each containing five stimuli that were trained exclusively with adjacent pairs. Selection of the higher-ranked item resulted in rewards. "Small reward" lists yielded two drops of fluid reward, whereas "large reward" lists yielded five drops. Following training of adjacent pairs, monkeys were tested on novels pairs. One item was selected from each list, such that a ranking rule could conflict with preferences for large rewards. Differences between the corresponding reward magnitudes had a strong influence on accuracy, but we also observed a symbolic distance effect. That provided evidence of a rule-based influence on decisions. RT comparisons suggested a conflict between rule- and reward-based processes. We conclude that performance reflects the contributions of two strategies and that a model-based strategy is employed in the face of a strong countervailing reward incentive.

Positional inference in rhesus macaques.

Understanding how organisms make transitive inferences is critical to understanding their general ability to learn serial relationships. In this context, transitive inference (TI) can be understood as a specific heuristic that applies broadly to many different serial learning tasks, which have been the focus of hundreds of studies involving dozens of species. In the present study, monkeys learned the order of 7-item lists of photographic stimuli by trial and error, and were then tested on "derived" lists. These derived test lists combined stimuli from multiple training lists in ambiguous ways, sometimes changing their order relative to training. We found that subjects displayed strong preferences when presented with novel test pairs, even when those pairs were drawn from different training lists. These preferences were helpful when test pairs had an ordering congruent with their ranks during training, but yielded consistently below-chance performance when pairs had an incongruent order relative to training. This behavior can be explained by the joint contributions of transitive inference and another heuristic that we refer to as "positional inference." Positional inferences play a complementary role to transitive inferences in facilitating choices between novel pairs of stimuli. The theoretical framework that best explains both transitive and positional inferences is a spatial model that represents both the position of each stimulus and its uncertainty. A computational implementation of this framework yields accurate predictions about both correct responses and errors on derived lists.

Category learning in a transitive inference paradigm.

The implied order of a ranked set of visual images can be learned without reliance on information that explicitly signals their order. Such learning is difficult to explain by associative mechanisms, but can be accounted for by cognitive representations and processes such as transitive inference. Our study sought to determine if those processes also apply to learning categories of images. We asked whether participants can (a) infer that stimulus images belonged to familiar categories, even when the images for each trial were unique, and (b) sort those categories into an ordering that obeys transitivity. Participants received minimal verbal instruction and a single session of training. Despite this, they learned the implied order of lists of fixed stimuli and lists of ordered categories, using trial-unique exemplars. We trained two groups, one for which stimuli were constant throughout training and testing (n = 60), and one for which exemplars of each category were trial-unique (n = 50). Our findings suggest that differing cognitive processes may underpin serial learning when learning about specific stimuli as opposed to stimulus categories.

Inferential Learning of Serial Order of Perceptual Categories by Rhesus Monkeys (Macaca mulatta).

Category learning in animals is typically trained explicitly, in most instances by varying the exemplars of a single category in a matching-to-sample task. Here, we show that male rhesus macaques can learn categories by a transitive inference paradigm in which novel exemplars of five categories were presented throughout training. Instead of requiring decisions about a constant set of repetitively presented stimuli, we studied the macaque's ability to determine the relative order of multiple exemplars of particular stimuli that were rarely repeated. Ordinal decisions generalized both to novel stimuli and, as a consequence, to novel pairings. Thus, we showed that rhesus monkeys could learn to categorize on the basis of implied ordinal position, without prior matching-to-sample training, and that they could then make inferences about category order. Our results challenge the plausibility of association models of category learning and broaden the scope of the transitive inference paradigm.SIGNIFICANCE STATEMENT The cognitive abilities of nonhuman animals are of enduring interest to scientists and the general public because they blur the dividing line between human and nonhuman intelligence. Categorization and sequence learning are highly abstract cognitive abilities each in their own right. This study is the first to provide evidence that visual categories can be ordered serially by macaque monkeys using a behavioral paradigm that provides no explicit feedback about category or serial order. These results strongly challenge accounts of learning based on stimulus-response associations.

Implicit Value Updating Explains Transitive Inference Performance: The Betasort Model.

Transitive inference (the ability to infer that B > D given that B > C and C > D) is a widespread characteristic of serial learning, observed in dozens of species. Despite these robust behavioral effects, reinforcement learning models reliant on reward prediction error or associative strength routinely fail to perform these inferences. We propose an algorithm called betasort, inspired by cognitive processes, which performs transitive inference at low computational cost. This is accomplished by (1) representing stimulus positions along a unit span using beta distributions, (2) treating positive and negative feedback asymmetrically, and (3) updating the position of every stimulus during every trial, whether that stimulus was visible or not. Performance was compared for rhesus macaques, humans, and the betasort algorithm, as well as Q-learning, an established reward-prediction error (RPE) model. Of these, only Q-learning failed to respond above chance during critical test trials. Betasort's success (when compared to RPE models) and its computational efficiency (when compared to full Markov decision process implementations) suggests that the study of reinforcement learning in organisms will be best served by a feature-driven approach to comparing formal models.

Retrospective and prospective metacognitive judgments in rhesus macaques (Macaca mulatta).

A growing body of research suggests that some non-human animals are capable of making accurate metacognitive judgments. In previous studies, non-human animals have made either retrospective or prospective judgments (about how they did on a test or how they will do on a test, respectively). These two types of judgments are dissociable in humans. The current study tested the abilities of two rhesus macaque monkeys to make both retrospective and prospective judgments about their performance on the same memory task. Both monkeys had been trained previously to make retrospective confidence judgments. Both monkeys successfully demonstrated transfer of retrospective metacognitive judgments to the new memory task. Furthermore, both monkeys transferred their retrospective judgments to the prospective task (one, immediately, and one, following the elimination of a response bias). This study is the first to demonstrate both retrospective and prospective monitoring abilities in the same monkeys and on the same task, suggesting a greater level of flexibility in animals' metacognitive monitoring abilities than has been reported previously.

Rapid cognitive flexibility of rhesus macaques performing psychophysical task-switching.

Three rhesus monkeys (Macaca mulatta) performed a simultaneous chaining task in which stimuli had to be sorted according to their visual properties. Each stimulus could vary independently along two dimensions (luminosity and radius), and a cue indicating which dimension to sort by was random trial to trial. These rapid and unpredictable changes constitute a task-switching paradigm, in which subjects must encode task demands and shift to whichever task-set is presently activated. In contrast to the widely reported task-switching delay observed in human studies, our subjects show no appreciable reduction in reaction times following a switch in the task requirements. Also, in contrast to the results of studies on human subjects, monkeys experienced enduring interference from trial-irrelevant stimulus features, even after exhaustive training. These results are consistent with a small but growing body of evidence that task-switching in rhesus macaques differs in basic ways from the pattern of behavior reported in studies of human cognition. Given the importance of task-switching paradigms in cognitive and clinical assessment, and the frequency with which corresponding animal models rely on non-human primates, understanding these differences in behavior is essential to the comparative study of cognitive impairment.

Disruption of component processes of spatial working memory by electroconvulsive shock but not magnetic seizure therapy.

Self-ordered spatial working memory measures provide important information regarding underlying cognitive strategies, such as stereotypy. This strategy is based on repetitive sequential selection of a spatial pattern once a correct sequence has been identified. We previously reported that electroconvulsive shock (ECS) but not magnetic seizure therapy (MST) impaired performance on a spatial working memory task in a preclinical model. Here we tested the hypothesis that ECS disrupted stereotyped patterns in the selection of spatial stimuli. In a within-subject study design, we assessed the effects of ECS, MST, and sham on stereotypy and reaction time in a preclinical model. Stereotypy was assessed by the correlation of actual and predicted response patterns of spatial stimuli. Predicted patterns were based on performance during baseline sessions. ECS resulted in lower correlations between predicted and actual responses to spatial stimuli in two of the three subjects, and it also disrupted stereotypy. For one subject, there was change in the predictability of the spatial locus of responses between experimental conditions. For all three subjects, reaction time was significantly longer in ECS, relative to MST and sham. This is the first study to examine the effect of ECS, and to contrast the effects of ECS and MST, on spatial working memory component processes. Our preliminary findings show that ECS, but not MST decreased stereotypy and increased reaction time. This line of investigation may have significant implications in our understanding cognitive component processes of memory function and impairment.

Monkeys would rather see and do: preference for agentic control in rhesus macaques.

Rhesus monkeys (Macaca mulatta) engaged in a series of computerized tasks modeled on billiards and arcade games in order to determine their degree of preference for scenarios in which food rewards were contingent on their actions, as opposed to those in which outcomes appeared externally caused. Throughout these tasks, subjects showed a consistent preference for "agentic control," a state in which goal-directed behavior is directly responsible for motivating outcomes. Other factors like the frequency and timing of reward deliveries were precisely controlled and did not explain observed preferences.

Intersubjectivity and the Emergence of Words.

Intersubjectivity refers to two non-verbal intersubjective relations infants experience during their first year that are precursors to the emergence of words. Trevarthen, a pioneer in the study of intersubjectivity, referred to those relations as primary and secondary intersubjectivity. The former, a dyadic coordination between the infant and her caregiver, begins at birth. The latter, a triadic coordination that develops around 9 months, allows the infant and a caregiver to share attention to particular features of the environment. Secondary intersubjectivity is crucial for an infant's ability to begin to produce words, at around 12 months. Much research on the social and cognitive origins of language has focused on secondary intersubjectivity. That is unfortunate because it neglects the fact that secondary intersubjectivity and the emergence of words are built on a foundation of primary intersubjectivity. It also ignores the evolutionary origins of intersubjectivity and its uniquely human status. That unique status explains why only humans learn words. This article seeks to address these issues by relating the literature on primary intersubjectivity, particularly research on bi-directional and contingent communication between infants and mothers, to joint attention and ultimately to words. In that context, we also discuss Hrdy's hypothesis about the influence of alloparents on the evolution of intersubjectivity.

Sequential planning in rhesus monkeys (Macaca mulatta).

In the current study, we examined the planning abilities of rhesus monkeys (Macaca mulatta) by training them on a five-item list composed of coloured photographs and then testing them on switch and mask trials. In contrast to previous studies where monkeys made responses using a joystick, in the current study, monkeys made responses directly to a touch screen. On switch trials, after a response to the first list item, the on-screen positions of two list items were exchanged. Performance on trials in which the second and third list items were exchanged was poorer compared to normal (non-switch) trials for all subjects. When the third and fourth items were exchanged, however, only one subject continued to show performance deficits. On mask trials, following a response to the first item, the remaining items were covered by opaque white squares. When two items were masked, all four subjects responded to each masked item at a level significantly above chance. When three items were masked, however, only one subjected was able to respond to all three masked items at a level significantly above chance. The results of the present study indicate that three of our four monkeys planned one response ahead while a single monkey planned two responses ahead. The significance of these findings is discussed in relation to previous studies on planning in chimpanzees and monkeys.

Transitive inference after minimal training in rhesus macaques (Macaca mulatta).

Rhesus macaques, when trained for several hundred trials on adjacent items in an ordered list (e.g., A > B, B > C, C > D), are able to make accurate transitive inferences (TI) about previously untrained pairs (e.g., A > C, B > D). How that learning unfolds during training, however, is not well understood. We sought to measure the relationship between the amount of TI training and the resulting response accuracy in 4 rhesus macaques using seven-item lists. The training conditions included the absolute minimal case of presenting each of the six adjacent pairs only once prior to testing. We also tested transfer to nonadjacent pairs with 24 and 114 training trials. Because performance during and after small amounts of training is expected to be near chance levels, we developed a descriptive statistical model to estimate potentially subtle learning effects in the presence of much larger random response variability and systematic bias. These results suggest that subjects learned serial order in an incremental fashion. Thus, rather than performing transitive inference by a logical process, serial learning in rhesus macaques proceeds in a manner more akin to a statistical inference, with an initial uncertainty about list position that gradually becomes more accurate as evidence accumulates. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Absolute and relative knowledge of ordinal position on implied lists.

Does serial learning result in specific associations between pairs of items, or does it result in a cognitive map based on relations of all items? In 2 experiments, we trained human participants to learn various lists of photographic images. We then tested the participants on new lists of photographic images. These new lists were constructed by selecting only 1 image from each list learned during training. In Experiment 1, participants were trained to choose the earlier (experimenter defined) item when presented with adjacent pairs of items on each of 5 different 5-item lists. Participants were then tested on derived lists, in which each item retained its original ordinal position, even though each of the presented pairs was novel. Participants performed above chance on all of the derived lists. In Experiment 2, a different group of participants received the same training as those of Experiment 1, but the ordinal positions of items were systematically changed on each derived list. The response accuracy for Experiment 2 varied inversely with the degree to which an item's original ordinal position was changed. These results can be explained by a model in which participants learned to make both positional inferences about the absolute rank of each stimulus, and transitive inferences about the relative ranks of pairs of stimuli. These inferences enhanced response accuracy when ordinal position was maintained, but not when it was changed. Our results demonstrate quantitatively that, in addition to item-item associations that participants acquire while learning a list of arbitrary items, they form a cognitive map that represents both experienced and inferred relationships. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Learned Representation of Implied Serial Order in Posterior Parietal Cortex.

Monkeys can learn the implied ranking of pairs of images drawn from an ordered set, despite never seeing all of the images simultaneously and without explicit spatial or temporal cues. We recorded the activity of posterior parietal cortex (including lateral intraparietal area LIP) neurons while monkeys learned 7-item transitive inference (TI) lists with 2 items presented on each trial. Behavior and neuronal activity were significantly influenced by the ordinal relationship of the stimulus pairs, specifically symbolic distance (the difference in rank) and joint rank (the sum of the ranks). Symbolic distance strongly predicted decision accuracy and learning rate. An effect of joint rank on performance was found nested within the symbolic distance effect. Across the population of neurons, there was significant modulation of firing correlated with the relative ranks of the two stimuli presented on each trial. Neurons exhibited selectivity for stimulus rank during learning, but not before or after. The observed behavior is poorly explained by associative or reward mechanisms, and appears more consistent with a mental workspace model in which implied serial order is mapped within a spatial framework. The neural data suggest that posterior parietal cortex supports serial learning by representing information about the ordinal relationship of the stimuli presented during a given trial.

Discovering Implied Serial Order Through Model-Free and Model-Based Learning.

Humans and animals can learn to order a list of items without relying on explicit spatial or temporal cues. To do so, they appear to make use of transitivity, a property of all ordered sets. Here, we summarize relevant research on the transitive inference (TI) paradigm and its relationship to learning the underlying order of an arbitrary set of items. We compare six computational models of TI performance, three of which are model-free (Q-learning, Value Transfer, and REMERGE) and three of which are model-based (RL-Elo, Sequential Monte Carlo, and Betasort). Our goal is to assess the ability of these models to produce empirically observed features of TI behavior. Model-based approaches perform better under a wider range of scenarios, but no single model explains the full scope of behaviors reported in the TI literature.

Reward associations do not explain transitive inference performance in monkeys.

Most accounts of behavior in nonhuman animals assume that they make choices to maximize expected reward value. However, model-free reinforcement learning based on reward associations cannot account for choice behavior in transitive inference paradigms. We manipulated the amount of reward associated with each item of an ordered list, so that maximizing expected reward value was always in conflict with decision rules based on the implicit list order. Under such a schedule, model-free reinforcement algorithms cannot achieve high levels of accuracy, even after extensive training. Monkeys nevertheless learned to make correct rule-based choices. These results show that monkeys' performance in transitive inference paradigms is not driven solely by expected reward and that appropriate inferences are made despite discordant reward incentives. We show that their choices can be explained by an abstract, model-based representation of list order, and we provide a method for inferring the contents of such representations from observed data.

Perceptual category learning of photographic and painterly stimuli in rhesus macaques (Macaca mulatta) and humans.

Humans are highly adept at categorizing visual stimuli, but studies of human categorization are typically validated by verbal reports. This makes it difficult to perform comparative studies of categorization using non-human animals. Interpretation of comparative studies is further complicated by the possibility that animal performance may merely reflect reinforcement learning, whereby discrete features act as discriminative cues for categorization. To assess and compare how humans and monkeys classified visual stimuli, we trained 7 rhesus macaques and 41 human volunteers to respond, in a specific order, to four simultaneously presented stimuli at a time, each belonging to a different perceptual category. These exemplars were drawn at random from large banks of images, such that the stimuli presented changed on every trial. Subjects nevertheless identified and ordered these changing stimuli correctly. Three monkeys learned to order naturalistic photographs; four others, close-up sections of paintings with distinctive styles. Humans learned to order both types of stimuli. All subjects classified stimuli at levels substantially greater than that predicted by chance or by feature-driven learning alone, even when stimuli changed on every trial. However, humans more closely resembled monkeys when classifying the more abstract painting stimuli than the photographic stimuli. This points to a common classification strategy in both species, one that humans can rely on in the absence of linguistic labels for categories.

In the beginning: A review of Robert C. Berwick and Noam Chomsky's Why Only Us.

We review Berwick and Chomsky's Why Only Us, Language and Evolution, a book premised on language as an instrument primarily of thought, only secondarily of communication. The authors conclude that a Universal Grammar can be reduced to three biologically isolated components, whose computational system for syntax was the result of a single mutation that occurred about 80,000 years ago. We question that argument because it ignores the origin of words, even though Berwick and Chomsky acknowledge that words evolved before grammar. It also fails to explain what evolutionary problem language uniquely solved (Wallace's question). To answer that question, we review recent discoveries about the ontogeny and phylogeny of words. Ontogenetically, two modes of nonverbal relation between infant and mother begin at or within 6 months of birth that are crucial antecedents of the infant's first words: intersubjectivity and joint attention. Intersubjectivity refers to rhythmic shared affect between infant and caretaker(s) that develop during the first 6 months. When the infant begins to crawl, they begin to attend jointly to environmental objects. Phylogenetically, Hrdy and Bickerton describe aspects of Homo erectus' ecology and cognition that facilitated the evolution of words. Hrdy shows how cooperative breeding established trust between infant and caretakers, laying the groundwork for a community of mutual trust among adults. Bickerton shows how 'confrontational scavenging' led to displaced reference, whereby an individual communicated the nature of a dead animal and its location to members of the group that could not see it. Thus, both phylogenetically and ontogenetically, the original function of language was primarily an instrument of communication. Rejecting Berwick and Chomsky's answer to Wallace's question that syntax afforded better planning and inference, we endorse Bickerton's view that language enabled speakers to refer to objects not immediately present. Thus arose context-free mental representations, unique to human language and thought.

Transitive inference in humans (Homo sapiens) and rhesus macaques (Macaca mulatta) after massed training of the last two list items.

Transitive inference (TI) is a classic learning paradigm for which the relative contributions of experienced rewards and representation-based inference have been debated vigorously, particularly regarding the notion that animals are capable of logic and reasoning. Rhesus macaque subjects and human participants performed a TI task in which, prior to learning a 7-item list (ABCDEFG), a block of trials presented exclusively the pair FG. Contrary to the expectation of associative models, the high prior rate of reward for F did not disrupt subsequent learning of the entire list. Monkeys (who each completed many sessions with novel stimuli) learned to anticipate that novel stimuli should be preferred over F. We interpret this as evidence of a task representation of TI that generalizes beyond learning about specific stimuli. Humans (who were task-naïve) showed a transitory bias to F when it was paired with novel stimuli, but very rapidly unlearned that bias. Performance with respect to the remaining stimuli was consistent with past reports of TI in both species. These results are difficult to reconcile with any account that assigns the strength of association between individual stimuli and rewards. Instead, they support sophisticated cognitive processes in both species, albeit with some species differences. (PsycINFO Database Record

The simultaneous chain: a new approach to serial learning.

Recent advances have allowed the application of behaviorism's rigor to the control of complex cognitive tasks in animals. This article examines recent research on serially organized behavior in animals. 'Chaining theory', the traditional approach to the study of such behavior, reduces intelligent action to sequences of discrete stimulus-response units in which each overt response is evoked by a particular stimulus. However, such theories are too weak to explain many forms of serially organized cognition, both in humans and animals. By training non-human primates to produce arbitrary sequences that cannot be learned as chains of particular motor responses, the simultaneous chaining paradigm has overcome limitations of chaining theory in experiments on serial expertise, the use of numerical rules, knowledge of ordinal position, and distance and magnitude effects.