Modelling how cleaner fish approach an ephemeral reward task demonstrates a role for ecologically tuned chunking in the evolution of advanced cognition.

What makes cognition "advanced" is an open and not precisely defined question. One perspective involves increasing the complexity of associative learning, from conditioning to learning sequences of events ("chaining") to representing various cue combinations as "chunks." Here we develop a weighted graph model to study the mechanism enabling chunking ability and the conditions for its evolution and success, based on the ecology of the cleaner fish Labroides dimidiatus. In some environments, cleaners must learn to serve visitor clients before resident clients, because a visitor leaves if not attended while a resident waits for service. This challenge has been captured in various versions of the ephemeral reward task, which has been proven difficult for a range of cognitively capable species. We show that chaining is the minimal requirement for solving this task in its common simplified laboratory format that involves repeated simultaneous exposure to an ephemeral and permanent food source. Adding ephemeral-ephemeral and permanent-permanent combinations, as cleaners face in the wild, requires individuals to have chunking abilities to solve the task. Importantly, chunking parameters need to be calibrated to ecological conditions in order to produce adaptive decisions. Thus, it is the fine-tuning of this ability, which may be the major target of selection during the evolution of advanced associative learning.

Social demonstration of colour preference improves the learning of associated demonstrated actions.

We studied how different types of social demonstration improve house sparrows' (Passer domesticus) success in solving a foraging task that requires both operant learning (opening covers) and discrimination learning (preferring covers of the rewarding colour). We provided learners with either paired demonstration (of both cover opening and colour preference), action-only demonstration (of opening white covers only), or no demonstration (a companion bird eating without covers). We found that sparrows failed to learn the two tasks with no demonstration, and learned them best with a paired demonstration. Interestingly, the action of cover opening was learned faster with paired rather than action-only demonstration despite being equally demonstrated in both. We also found that only with paired demonstration, the speed of operant (action) learning was related to the demonstrator's level of activity. Colour preference (i.e. discrimination learning) was eventually acquired by all sparrows that learned to open covers, even without social demonstration of colour preference. Thus, adding a demonstration of colour preference was actually more important for operant learning, possibly as a result of increasing the similarity between the demonstrated and the learned tasks, thereby increasing the learner's attention to the actions of the demonstrator. Giving more attention to individuals in similar settings may be an adaptive strategy directing social learners to focus on ecologically relevant behaviours and on tasks that are likely to be learned successfully.

House sparrows use learned information selectively based on whether reward is hidden or visible.

Based on past experience, food-related-cues can help foragers to predict the presence and the expected quality of food. However, when the food is already visible there is no need to predict its presence or its other visible attributes, but only those that are still cryptic, such as expected handling time or taste. Optimal foragers should therefore use only knowledge that is relevant to the current setting. Nevertheless, the extent to which they do so is not clear. In a set of experiments, we examined how a change in setting, from hidden to visible reward, affects the reliance of house sparrows (Passer domesticus) on three previously learned attributes of food-related cues (sand colors): the setting of the cue (e.g., whether the food was hidden or exposed), the expected amount of the reward (number of seeds), and the expected handling time. We found that sparrows used all three attributes when the rewards were hidden but reached decisions mainly based on handling time when the rewards were visible. This selective use of cue-related information suggests that animals do not simply associate cues with their average expected value but rather learn different attributes of a cue and use all, or only some of them, in a context-appropriate manner.

Crowd wisdom enhanced by costly signaling in a virtual rating system.

Costly signaling theory was developed in both economics and biology and has been used to explain a wide range of phenomena. However, the theory's prediction that signal cost can enforce information quality in the design of new communication systems has never been put to an empirical test. Here we show that imposing time costs on reporting extreme scores can improve crowd wisdom in a previously cost-free rating system. We developed an online game where individuals interacted repeatedly with simulated services and rated them for satisfaction. We associated ratings with differential time costs by endowing the graphical user interface that solicited ratings from the users with "physics," including an initial (default) slider position and friction. When ratings were not associated with differential cost (all scores from 0 to 100 could be given by an equally low-cost click on the screen), scores correlated only weakly with objective service quality. However, introducing differential time costs, proportional to the deviation from the mean score, improved correlations between subjective rating scores and objective service performance and lowered the sample size required for obtaining reliable, averaged crowd estimates. Boosting time costs for reporting extreme scores further facilitated the detection of top performances. Thus, human collective online behavior, which is typically cost-free, can be made more informative by applying costly signaling via the virtual physics of rating devices.

Differential application of cultural practices at the family and individual levels may alter heritability estimates.

Uchiyama et al. emphasize that culture evolves directionally and differentially as a function of selective pressures in different populations. Extending these principles to the level of families, lineages, and individuals exposes additional challenges to estimating heritability. Cultural traits expressed differentially as a function of the genetics whose influence they mask or unmask render inseparable the influences of culture and genetics.

Reinforcement Learning Theory Reveals the Cognitive Requirements for Solving the Cleaner Fish Market Task.

Learning is an adaptation that allows individuals to respond to environmental stimuli in ways that improve their reproductive outcomes. The degree of sophistication in learning mechanisms potentially explains variation in behavioral responses. Here, we present a model of learning that is inspired by documented intra- and interspecific variation in the performance of a simultaneous two-choice task, the biological market task. The task presents a problem that cleaner fish often face in nature: choosing between two client types, one that is willing to wait for inspection and one that may leave if ignored. The cleaner's choice hence influences the future availability of clients (i.e., it influences food availability). We show that learning the preference that maximizes food intake requires subjects to represent in their memory different combinations of pairs of client types rather than just individual client types. In addition, subjects need to account for future consequences of actions, either by estimating expected long-term reward or by experiencing a client leaving as a penalty (negative reward). Finally, learning is influenced by the absolute and relative abundance of client types. Thus, cognitive mechanisms and ecological conditions jointly explain intra- and interspecific variation in the ability to learn the adaptive response.

Competitive advantage of rare behaviours induces adaptive diversity rather than social conformity in skill learning.

Recent studies have emphasized the role of social learning and cultural transmission in promoting conformity and uniformity in animal groups, but little attention has been given to the role of negative frequency-dependent learning in impeding conformity and promoting diversity instead. Here, we show experimentally that under competitive conditions that are common in nature, social foragers (although capable of social learning) are likely to develop diversity in foraging specialization rather than uniformity. Naive house sparrows that were introduced into groups of foraging specialists did not conform to the behaviour of the specialists, but rather learned to use the alternative food-related cues, thus forming groups of complementary specialists. We further show that individuals in such groups may forage more effectively in diverse environments. Our results suggest that when the benefit from socially acquired skills diminishes through competition in a negative frequency-dependent manner, animal societies will become behaviourally diverse rather than uniform.

The evolution of cognitive mechanisms in response to cultural innovations.

When humans and other animals make cultural innovations, they also change their environment, thereby imposing new selective pressures that can modify their biological traits. For example, there is evidence that dairy farming by humans favored alleles for adult lactose tolerance. Similarly, the invention of cooking possibly affected the evolution of jaw and tooth morphology. However, when it comes to cognitive traits and learning mechanisms, it is much more difficult to determine whether and how their evolution was affected by culture or by their use in cultural transmission. Here we argue that, excluding very recent cultural innovations, the assumption that culture shaped the evolution of cognition is both more parsimonious and more productive than assuming the opposite. In considering how culture shapes cognition, we suggest that a process-level model of cognitive evolution is necessary and offer such a model. The model employs relatively simple coevolving mechanisms of learning and data acquisition that jointly construct a complex network of a type previously shown to be capable of supporting a range of cognitive abilities. The evolution of cognition, and thus the effect of culture on cognitive evolution, is captured through small modifications of these coevolving learning and data-acquisition mechanisms, whose coordinated action is critical for building an effective network. We use the model to show how these mechanisms are likely to evolve in response to cultural phenomena, such as language and tool-making, which are associated with major changes in data patterns and with new computational and statistical challenges.

Beyond uncertainty: A broader scope for "incentive hope" mechanisms and its implications.

We propose that food-related uncertainty is but one of multiple cues that predicts harsh conditions and may activate "incentive hope." An evolutionarily adaptive response to these would have been to shift to a behavioral-metabolic phenotype geared toward facing hardship. In modernity, this phenotype may lead to pathologies such as obesity and hoarding. Our perspective suggests a novel therapeutic approach.

Trial-and-error copying of demonstrated actions reveals how fledglings learn to 'imitate' their mothers.

Understanding how humans and other animals learn to perform an act from seeing it done has been a major challenge in the study of social learning. To determine whether this ability is based on 'true imitation', many studies have applied the two-action experimental paradigm, examining whether subjects learn to perform the specific action demonstrated to them. Here, we show that the insights gained from animals' success in two-action experiments may be limited, and that a better understanding is achieved by monitoring subjects' entire behavioural repertoire. Hand-reared house sparrows that followed a model of a mother demonstrator were successful in learning to find seeds hidden under a leaf, using the action demonstrated by the mother (either pushing the leaf or pecking it). However, they also produced behaviours that had not been demonstrated but were nevertheless related to the demonstrated act. This finding suggests that while the learners were clearly influenced by the demonstrator, they did not accurately imitate her. Rather, they used their own behavioural repertoire, gradually fitting it to the demonstrated task solution through trial and error. This process is consistent with recent views on how animals learn to imitate, and may contribute to a unified process-level analysis of social learning mechanisms.

Constructive anthropomorphism: a functional evolutionary approach to the study of human-like cognitive mechanisms in animals.

Anthropomorphism, the attribution of human cognitive processes and emotional states to animals, is commonly viewed as non-scientific and potentially misleading. This is mainly because apparent similarity to humans can usually be explained by alternative, simpler mechanisms in animals, and because there is no explanatory power in analogies to human phenomena when these phenomena are not well understood. Yet, because it is also difficult to preclude real similarity and continuity in the evolution of humans' and animals' cognitive abilities, it may not be productive to completely ignore our understanding of human behaviour when thinking about animals. Here we propose that in applying a functional approach to the evolution of cognitive mechanisms, human cognition may be used to broaden our theoretical thinking and to generate testable hypotheses. Our goal is not to 'elevate' animals, but rather to find the minimal set of mechanistic principles that may explain 'advanced' cognitive abilities in humans, and consider under what conditions these mechanisms were likely to enhance fitness and to evolve in animals. We illustrate this approach, from relatively simple emotional states, to more advanced mechanisms, involved in planning and decision-making, episodic memory, metacognition, theory of mind, and consciousness.

The bottleneck may be the solution, not the problem.

As a highly consequential biological trait, a memory "bottleneck" cannot escape selection pressures. It must therefore co-evolve with other cognitive mechanisms rather than act as an independent constraint. Recent theory and an implemented model of language acquisition suggest that a limit on working memory may evolve to help learning. Furthermore, it need not hamper the use of language for communication.

Evolution of protolinguistic abilities as a by-product of learning to forage in structured environments.

The skills required for the learning and use of language are the focus of extensive research, and their evolutionary origins are widely debated. Using agent-based simulations in a range of virtual environments, we demonstrate that challenges of foraging for food can select for cognitive mechanisms supporting complex, hierarchical, sequential learning, the need for which arises in language acquisition. Building on previous work, where we explored the conditions under which reinforcement learning is out-competed by seldom-reinforced continuous learning that constructs a network model of the environment, we now show that realistic features of the foraging environment can select for two critical advances: (i) chunking of meaningful sequences found in the data, leading to representations composed of units that better fit the prevalent statistical patterns in the environment; and (ii) generalization across units based on their contextual similarity. Importantly, these learning processes, which in our framework evolved for making better foraging decisions, had been earlier shown to reproduce a range of findings in language learning in humans. Thus, our results suggest a possible evolutionary trajectory that may have led from basic learning mechanisms to complex hierarchical sequential learning that can support advanced cognitive abilities of the kind needed for language acquisition.

Evolution of learning and levels of selection: a lesson from avian parent-offspring communication.

In recent years, it has become increasingly clear that the evolution of behavior may be better understood as the evolution of the learning mechanisms that produce it, and that such mechanisms should be modeled and tested explicitly. However, this approach, which has recently been applied to animal foraging and decision-making, has rarely been applied to the social and communicative behaviors that are likely to operate in complex social environments and be subject to multi-level selection. Here we use genetic, agent-based evolutionary simulations to explore how learning mechanisms may evolve to adjust the level of nestling begging (offspring signaling of need), and to examine the possible consequences of this process for parent-offspring conflict and communication. In doing so, we also provide the first step-by-step dynamic model of parent-offspring communication. The results confirm several previous theoretical predictions and demonstrate three novel phenomena. First, negatively frequency-dependent group-level selection can generate a stable polymorphism of learning strategies and parental responses. Second, while conventional reinforcement learning models fail to cope successfully with family dynamics at the nest, a newly developed learning model (incorporating behaviors that are consistent with recent experimental results on learning in nestling begging) produced effective learning, which evolved successfully. Third, while kin-selection affects the frequency of the different learning genes, its impact on begging slope and intensity was unexpectedly negligible, demonstrating that evolution is a complex process, and showing that the effect of kin-selection on behaviors that are shaped by learning may not be predicted by simple application of Hamilton's rule.

Perceptual accuracy and conflicting effects of certainty on risk-taking behaviour.

The 'certainty effect' is a notable violation of expected utility theory by decision makers. It shows that people's tendency to select the safer of two prospects increases when this prospect provides a good outcome with certainty (for example, people prefer a monetary gain of 3 with certainty over 4 with a probability of 0.8, but do not prefer 3 with a probability of 0.25 over 4 with a probability of 0.2). Subsequent work on experience-based decision making in rats extended the certainty effect to other animals, suggesting its generality across different species and different decision-making mechanisms. However, an attempt to replicate this study with human subjects showed a surprising 'reversed certainty effect', namely, the tendency to prefer the safer option decreases when this prospect is associated with certainty (and people now prefer 4 with a probability of 0.8 over 3 with certainty). Here we show that these conflicting results can be explained by perceptual noise and that the certainty effect can be restored experimentally by reducing perceptual accuracy. Using complementary experiments in humans and honeybees (Apis mellifera), we show that by manipulating perceptual accuracy in experience-based tasks, both the certainty and the reversed certainty effects can be exhibited by humans and other animals: the certainty effect emerges when it is difficult to discriminate between the different rewards, whereas the reversed certainty effect emerges when discrimination is easy. Our results fit a simple process-based model of matching behaviour, capable of explaining the certainty effect in humans and other animals that make repeated decisions based on experience. This mechanism should probably be distinguished from those involved in the original certainty effect that was exhibited by human subjects in single description-based problems.

Reconciling genetic evolution and the associative learning account of mirror neurons through data-acquisition mechanisms.

An associative learning account of mirror neurons should not preclude genetic evolution of its underlying mechanisms. On the contrary, an associative learning framework for cognitive development should seek heritable variation in the learning rules and in the data-acquisition mechanisms that construct associative networks, demonstrating how small genetic modifications of associative elements can give rise to the evolution of complex cognition.

Multiple sexual signals and behavioral reproductive isolation in a diverging population.

Sexual trait divergence has been shown to play a role in the evolution of reproductive isolation. While variation in multiple sexual signals is common among closely related species, little is known about the role of these different axes of phenotype variation with respect to the evolution of behavioral reproductive isolation. Here we study a unique population of barn swallows (Hirundo rustica transitiva) that can be distinguished phenotypically from its neighboring populations only on the basis of two features of male plumage: exaggerated expression of both long tail streamers and dark ventral coloration. Using phenotype manipulation experiments, we conducted a paternity study to examine whether both traits are sexually selected. Our results show that an exaggerated form of the local male phenotype (with both tail elongation and color darkening) is favored by local females, whereas males whose phenotypes were manipulated to look like males of neighboring subspecies suffered paternity losses from their social mates. These results confirm the multiple signaling role of the unique tail and color combination in our diverging population and suggest a novel possibility according to which multiple sexual signals may also be used to discriminate among males from nearby populations when prezygotic reproductive isolation is adaptive.

Increased initial task difficulty drives social foragers to develop sub-optimal conformity instead of adaptive diversity.

The extent to which animal societies exhibit social conformity as opposed to behavioural diversity is commonly attributed to adaptive learning strategies. Less attention is given to the possibility that the relative difficulty of learning a task socially as opposed to individually can be critical for social learning dynamics. Here we show that by raising initial task difficulty, house sparrows previously shown to exhibit adaptive social diversity become predominantly conformists. The task we used required opening feeding well covers (easier to learn socially) and to choose the covers with the rewarding cues (easy to learn individually). We replicated a previous study where sparrows exhibited adaptive diversity, but did not pre-train the naive sparrows to open covers, making the task initially more difficult. In sharp contrast to the previous study results, most sparrows continued to conform to the demonstrated cue even after experiencing greater success with the alternative rewarding cue for which competition was less intense. Thus, our study shows that a task's cognitive demands, such as the initial dependency on social demonstration, can change the entire learning dynamics, causing social animals to exhibit sub-optimal social conformity rather than adaptive diversity under otherwise identical conditions.

Evolution of learned strategy choice in a frequency-dependent game.

In frequency-dependent games, strategy choice may be innate or learned. While experimental evidence in the producer-scrounger game suggests that learned strategy choice may be common, a recent theoretical analysis demonstrated that learning by only some individuals prevents learning from evolving in others. Here, however, we model learning explicitly, and demonstrate that learning can easily evolve in the whole population. We used an agent-based evolutionary simulation of the producer-scrounger game to test the success of two general learning rules for strategy choice. We found that learning was eventually acquired by all individuals under a sufficient degree of environmental fluctuation, and when players were phenotypically asymmetric. In the absence of sufficient environmental change or phenotypic asymmetries, the correct target for learning seems to be confounded by game dynamics, and innate strategy choice is likely to be fixed in the population. The results demonstrate that under biologically plausible conditions, learning can easily evolve in the whole population and that phenotypic asymmetry is important for the evolution of learned strategy choice, especially in a stable or mildly changing environment.

Population genetics and morphological comparisons of migratory European (Hirundo rustica rustica) and sedentary East-Mediterranean (Hirundo rustica transitiva) barn swallows.

Speciation processes are largely determined by the relative strength of divergent selection versus the magnitude of gene flow. The barn swallow (Hirundo rustica) has a broad geographic distribution that encompasses substantial geographic variation in morphology and behavior. The European (H. r. rustica) and East-Mediterranean (H. r. transitiva) subspecies are closely related, despite differing in morphological and life-history traits. To explore patterns of genetic differentiation and gene flow, we compared morphological and genetic variation among the nonmigratory breeding population of H. r. transitiva from Israel and the migratory population of H. r. rustica that passes through Israel and compared it with the genetic differentiation between H. r. transitiva from Israel and a breeding population of H. r. rustica from the United Kingdom that uses a different migratory flyway. Mitochondrial haplotype network analysis suggests that the European and East-Mediterranean populations are intermixed, although there was low but significant genetic differentiation between the subspecies based on both mitochondrial (F(ST) = 0.025-0.033) and microsatellite (F(ST) = 0.009-0.014) loci. Coalescent-based analyses suggest recent divergence and substantial gene flow between these populations despite their differences in morphological and behavioral traits. The results suggest that these subspecies are undergoing a differentiation process in the face of gene flow, with selection possibly operating on sexually selected traits.