Social bond dynamics and the evolution of helping.

Empiricists often struggle to apply game theory models to real-life cases of animal cooperation. One reason is that many examples of cooperation occur in stable groups, where individuals form social bonds that influence exchanges of help in ways that are not well described by previous models, including the extent of reciprocity and how relationships are initiated. We present a game theory model exploring the conditions under which social bonds between group members promote cooperation. In the model, bonds build up from exchanges of help in a similar way as the strength of association increases in learning, as in the Rescorla-Wagner rule. The bonds in turn affect partner choice and influence helping amounts. The model has a mechanism of reciprocity for bonded pairs, which can evolve toward either loose or strict reciprocation. Several aspects of the model are inspired by observations of food sharing in vampire bats. We find that small social neighborhoods are required for the evolutionary stability of helping, either as small group sizes, or if bonded members of larger groups can form temporary (daily) smaller groupings. The costs of helping need to be fairly low, while the benefits can be substantial. The form of reciprocity that evolves is neither immediate nor very strict. Individuals in need request help based on bond strength, but there is also an evolved preference for initiating bonds with new group members. In contrast, if different groups come into temporary contact, the evolved tendency is to avoid forming bonds between groups.

Flexible learning in complex worlds.

Cognitive flexibility can enhance the ability to adjust to changing environments. Here, we use learning simulations to investigate the possible advantages of flexible learning in volatile (changing) environments. We compare two established learning mechanisms, one with constant learning rates and one with rates that adjust to volatility. We study an ecologically relevant case of volatility, based on observations of developing cleaner fish Labroides dimidiatus that experience a transition from a simpler to a more complex foraging environment. There are other similar transitions in nature, such as migrating to a new and different habitat. We also examine two traditional approaches to volatile environments in experimental psychology and behavioral ecology: reversal learning, and learning set formation (consisting of a sequence of different discrimination tasks). These provide experimental measures of cognitive flexibility. Concerning transitions to a complex world, we show that both constant and flexible learning rates perform well, losing only a small proportion of available rewards in the period after a transition, but flexible rates perform better than constant rates. For reversal learning, flexible rates improve the performance with each successive reversal because of increasing learning rates, but this does not happen for constant rates. For learning set formation, we find no improvement in performance with successive shifts to new stimuli to discriminate for either flexible or constant learning rates. Flexible learning rates might thus explain increasing performance in reversal learning but not in learning set formation, and this can shed light on the nature of cognitive flexibility in a given system.

Game theory in biology: 50 years and onwards.

Game theory in biology gained prominence 50 years ago, when Maynard Smith & Price formulated the concept of an evolutionarily stable strategy (ESS). Their aim was to explain why conflicts between animals of the same species usually are of a 'limited war' type, not causing serious injury. They emphasized that game theory is an alternative to previous ideas about group selection, which were used by ethologists to explain limited aggression. Subsequently, the ESS concept was applied to many phenomena with frequency dependence in the evolutionary success of strategies, including sex allocation, alternative mating types, contest behaviour and signalling, cooperation, and parental care. Both the analyses of signalling and cooperation were inspired by similar problems in economics and attracted much attention in biology. Here we give a perspective on which of the ambitions in the field have been achieved, with a focus on contest behaviour and cooperation. We evaluate whether the game-theoretical study of the evolution of cooperation has measured up to expectations in explaining the behaviour of non-human animals. We also point to potentially fruitful directions for the field, and emphasize the importance of incorporating realistic behavioural mechanisms into models. This article is part of the theme issue 'Half a century of evolutionary games: a synthesis of theory, application and future directions'.

Effects of local versus global competition on reproductive skew and sex differences in social dominance behaviour.

Social hierarchies are often found in group-living animals. The hierarchy position can influence reproductive success (RS), with a skew towards high-ranking individuals. The amount of aggression in social dominance varies greatly, both between species and between males and females within species. Using game theory we study this variation by taking into account the degree to which reproductive competition in a social group is mainly local to the group, emphasizing within-group relative RS, or global to a larger population, emphasizing an individual's absolute RS. Our model is similar to recent approaches in that reinforcement learning is used as a behavioural mechanism allowing social-hierarchy formation. We test two hypotheses. The first is that local competition should favour the evolution of mating or foraging interference, and thus of reproductive skew. Second, decreases in reproductive output caused by an individual's accumulated fighting damage, such as reduced parenting ability, will favour less intense aggression but should have little influence on reproductive skew. From individual-based simulations of the evolution of social dominance and interference, we find support for both hypotheses. We discuss to what extent our results can explain observed sex differences in reproductive skew and social dominance behaviour.

Behavioural specialization and learning in social networks.

Interactions in social groups can promote behavioural specialization. One way this can happen is when individuals engage in activities with two behavioural options and learn which option to choose. We analyse interactions in groups where individuals learn from playing games with two actions and negatively frequency-dependent payoffs, such as producer-scrounger, caller-satellite, or hawk-dove games. Group members are placed in social networks, characterized by the group size and the number of neighbours to interact with, ranging from just a few neighbours to interactions between all group members. The networks we analyse include ring lattices and the much-studied small-world networks. By implementing two basic reinforcement-learning approaches, action-value learning and actor-critic learning, in different games, we find that individuals often show behavioural specialization. Specialization develops more rapidly when there are few neighbours in a network and when learning rates are high. There can be learned specialization also with many neighbours, but we show that, for action-value learning, behavioural consistency over time is higher with a smaller number of neighbours. We conclude that frequency-dependent competition for resources is a main driver of specialization. We discuss our theoretical results in relation to experimental and field observations of behavioural specialization in social situations.

An evolutionary perspective on stress responses, damage and repair.

Variation in stress responses has been investigated in relation to environmental factors, species ecology, life history and fitness. Moreover, mechanistic studies have unravelled molecular mechanisms of how acute and chronic stress responses cause physiological impacts ('damage'), and how this damage can be repaired. However, it is not yet understood how the fitness effects of damage and repair influence stress response evolution. Here we study the evolution of hormone levels as a function of stressor occurrence, damage and the efficiency of repair. We hypothesise that the evolution of stress responses depends on the fitness consequences of damage and the ability to repair that damage. To obtain some general insights, we model a simplified scenario in which an organism repeatedly encounters a stressor with a certain frequency and predictability (temporal autocorrelation). The organism can defend itself by mounting a stress response (elevated hormone level), but this causes damage that takes time to repair. We identify optimal strategies in this scenario and then investigate how those strategies respond to acute and chronic exposures to the stressor. We find that for higher repair rates, baseline and peak hormone levels are higher. This typically means that the organism experiences higher levels of damage, which it can afford because that damage is repaired more quickly, but for very high repair rates the damage does not build up. With increasing predictability of the stressor, stress responses are sustained for longer, because the animal expects the stressor to persist, and thus damage builds up. This can result in very high (and potentially fatal) levels of damage when organisms are exposed to chronic stressors to which they are not evolutionarily adapted. Overall, our results highlight that at least three factors need to be considered jointly to advance our understanding of how stress physiology has evolved: (i) temporal dynamics of stressor occurrence; (ii) relative mortality risk imposed by the stressor itself versus damage caused by the stress response; and (iii) the efficiency of repair mechanisms.

Reproductive skew, fighting costs and winner-loser effects in social dominance evolution.

Social hierarchies are often found in group-living animals and can be formed through pairwise aggressive interactions. The dominance rank can influence reproductive success (RS) with a skew towards high-ranking individuals. Using game theory, we investigate how the opportunity for differently ranked individuals to achieve RS influences the costs of hierarchy formation and the strength of winner and loser effects. In our model, individuals adjust their aggressive and submissive behaviour towards others through reinforcement learning. The learning is based on rewards and penalties, which depend on relative fighting ability. From individual-based simulations, we determine evolutionary equilibria of traits such as learning rates. We examine situations that differ in the extent of monopolisation of contested RS by dominants and in the proportion of total RS that is contested. The model implements two kinds of fighting costs: a decrease in effective fighting ability from damage (loss of condition) and a risk of mortality that increases with the total accumulated damage. Either of these costs can limit the amount of fighting. We find that individuals form stable dominance hierarchies, with a positive correlation between dominance position and fighting ability. The accumulated costs differ between dominance positions, with the highest costs paid by low or intermediately ranked individuals. Costs tend to be higher in high-skew situations. We identify a 'stay-in, opt-out' syndrome, comprising a range from weaker (stay-in) to stronger (opt-out) winner-loser effects. We interpret the opt-out phenotype to be favoured by selection on lower ranked individuals to opt out of contests over social dominance, because it is more pronounced when more of the total RS is uncontested. We discuss our results in relation to field and experimental observations and argue that there is a need for empirical investigation of the behaviour and reproductive success of lower ranked individuals.

Effects of social experience, aggressiveness and comb size on contest success in male domestic fowl.

The ability to dominate conspecifics and thereby gain access to resources depends on a number of traits and skills. Experience of dominance relationships during development is a potential source of learning such skills. We here study the importance of social experience, aggressiveness and morphological traits for competitiveness in social interactions (contest success) in male domestic fowl (Gallus gallus domesticus). We let males grow up either as a single (dominant) male or as an intermediately ranked male in a group of males, and measured their success in duels against different opponents. We found that single-raised males had lower contest success than group-raised males, and that aggression and comb size correlated positively with contest success. This indicates that experience of dominance interactions with other males increases future success in duels. We similarly studied the consequences of growing up as a dominant or subordinate in a pair of males, finding no statistically significant effect of the dominance position on contest success. Finally, we found that males were consistent over time in contest success. We conclude that social experience increases contest success in male domestic fowl, but that certain behavioural and morphological characteristics have an equal or even stronger covariation with contest success.

The evolution of social learning as phenotypic cue integration.

Most analyses of the origins of cultural evolution focus on when and where social learning prevails over individual learning, overlooking the fact that there are other developmental inputs that influence phenotypic fit to the selective environment. This raises the question of how the presence of other cue 'channels' affects the scope for social learning. Here, we present a model that considers the simultaneous evolution of (i) multiple forms of social learning (involving vertical or horizontal learning based on either prestige or conformity biases) within the broader context of other evolving inputs on phenotype determination, including (ii) heritable epigenetic factors, (iii) individual learning, (iv) environmental and cascading maternal effects, (v) conservative bet-hedging, and (vi) genetic cues. In fluctuating environments that are autocorrelated (and hence predictable), we find that social learning from members of the same generation (horizontal social learning) explains the large majority of phenotypic variation, whereas other cues are much less important. Moreover, social learning based on prestige biases typically prevails in positively autocorrelated environments, whereas conformity biases prevail in negatively autocorrelated environments. Only when environments are unpredictable or horizontal social learning is characterized by an intrinsically low information content, other cues such as conservative bet-hedging or vertical prestige biases prevail. This article is part of the theme issue 'Foundations of cultural evolution'.

The Evolution of Social Dominance through Reinforcement Learning.

AbstractGroups of social animals are often organized into dominance hierarchies that are formed through pairwise interactions. There is much experimental data on hierarchies, examining such things as winner, loser, and bystander effects, as well as the linearity and replicability of hierarchies, but there is a lack evolutionary analyses of these basic observations. Here I present a game theory model of hierarchy formation in which individuals adjust their aggressive behavior toward other group members through reinforcement learning. Individual traits such as the tendency to generalize learning between interactions with different individuals, the rate of learning, and the initial tendency to be aggressive are genetically determined and can be tuned by evolution. I find that evolution favors individuals with high social competence, making use of individual recognition, bystander observational learning, and, to a limited extent, generalizing learned behavior between opponents when adjusting their behavior toward other group members. The results are in qualitative agreement with experimental data, for instance, in finding weaker winner effects compared to loser effects.

Learning, exploitation and bias in games.

We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple 'unbiased' reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual's behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is 'rational'.

Towards an Evolutionary Theory of Stress Responses.

All organisms have a stress response system to cope with environmental threats, yet its precise form varies hugely within and across individuals, populations, and species. While the physiological mechanisms are increasingly understood, how stress responses have evolved remains elusive. Here, we show that important insights can be gained from models that incorporate physiological mechanisms within an evolutionary optimality analysis (the 'evo-mecho' approach). Our approach reveals environmental predictability and physiological constraints as key factors shaping stress response evolution, generating testable predictions about variation across species and contexts. We call for an integrated research programme combining theory, experimental evolution, and comparative analysis to advance scientific understanding of how this core physiological system has evolved.

Reinforcement Learning Theory Reveals the Cognitive Requirements for Solving the Cleaner Fish Market Task.

Learning is an adaptation that allows individuals to respond to environmental stimuli in ways that improve their reproductive outcomes. The degree of sophistication in learning mechanisms potentially explains variation in behavioral responses. Here, we present a model of learning that is inspired by documented intra- and interspecific variation in the performance of a simultaneous two-choice task, the biological market task. The task presents a problem that cleaner fish often face in nature: choosing between two client types, one that is willing to wait for inspection and one that may leave if ignored. The cleaner's choice hence influences the future availability of clients (i.e., it influences food availability). We show that learning the preference that maximizes food intake requires subjects to represent in their memory different combinations of pairs of client types rather than just individual client types. In addition, subjects need to account for future consequences of actions, either by estimating expected long-term reward or by experiencing a client leaving as a penalty (negative reward). Finally, learning is influenced by the absolute and relative abundance of client types. Thus, cognitive mechanisms and ecological conditions jointly explain intra- and interspecific variation in the ability to learn the adaptive response.

Learning leads to bounded rationality and the evolution of cognitive bias in public goods games.

In social interactions, including cooperation and conflict, individuals can adjust their behaviour over the shorter term through learning within a generation, and natural selection can change behaviour over the longer term of many generations. Here we investigate the evolution of cognitive bias by individuals investing into a project that delivers joint benefits. For members of a group that learn how much to invest using the costs and benefits they experience in repeated interactions, we show that overestimation of the cost of investing can evolve. The bias causes individuals to invest less into the project. Our explanation is that learning responds to immediate rather than longer-term rewards. There are thus cognitive limitations in learning, which can be seen as bounded rationality. Over a time horizon of several rounds of interaction, individuals respond to each other's investments, for instance by partially compensating for another's shortfall. However, learning individuals fail to strategically take into account that social partners respond in this way. Learning instead converges to a one-shot Nash equilibrium of a game with perceived rewards as payoffs. Evolution of bias can then compensate for the cognitive limitations of learning.

Generalization of learned preferences covaries with behavioral flexibility in red junglefowl chicks.

The relationship between animal cognition and consistent among-individual behavioral differences (i.e., behavioral types, animal personality, or coping styles), has recently received increased research attention. Focus has mainly been on linking different behavioral types to performance in learning tasks. It has been suggested that behavioral differences could influence also how individuals use previously learnt information to generalize about new stimuli with similar properties. Nonetheless, this has rarely been empirically tested. Here, we therefore explore the possibility that individual variation in generalization is related to variation in behavioral types in red junglefowl chicks (Gallus gallus). We show that more behaviorally flexible chicks have a stronger preference for a novel stimulus that is intermediate between 2 learnt positive stimuli compared to more inflexible chicks. Thus, more flexible and inflexible chicks differ in how they generalize. Further, behavioral flexibility correlates with fearfulness, suggesting a coping style, supporting that variation in generalization is related to variation in behavioral types. How individuals generalize affects decision making and responses to novel situations or objects, and can thus have a broad influence on the life of an individual. Our results add to the growing body of evidence linking cognition to consistent behavioral differences.

Ecological Genetic Conflict: Genetic Architecture Can Shift the Balance between Local Adaptation and Plasticity.

Genetic polymorphism can contribute to local adaptation in heterogeneous habitats, for instance, as a single locus with alleles adapted to different habitats. Phenotypic plasticity can also contribute to trait variation across habitats, through developmental responses to habitat-specific cues. We show that the genetic architecture of genetically polymorphic and plasticity loci may influence the balance between local adaptation and phenotypic plasticity. These effects of genetic architecture are instances of ecological genetic conflict. A reduced effective migration rate for genes tightly linked to a genetic polymorphism provides an explanation for the effects, and they can occur both for a single trait and for a syndrome of coadapted traits. Using individual-based simulations and numerical analysis, we investigate how among-habitat genetic polymorphism and phenotypic plasticity depend on genetic architecture. We also study the evolution of genetic architecture itself, in the form of rates of recombination between genetically polymorphic loci and plasticity loci. Our main result is that for plasticity genes that are unlinked to loci with between-habitat genetic polymorphism, the slope of a reaction norm is steeper in comparison with the slope favored by plasticity genes that are tightly linked to genes for local adaptation.

The relationship between learning speed and personality is age- and task-dependent in red junglefowl.

ABSTRACT: Cognition is fundamental to animals' lives and an important source of phenotypic variation. Nevertheless, research on individual variation in animal cognition is still limited. Further, although individual cognitive abilities have been suggested to be linked to personality (i.e., consistent behavioral differences among individuals), few studies have linked performance across multiple cognitive tasks to personality traits. Thus, the interplays between cognition and personality are still unclear. We therefore investigated the relationships between an important aspect of cognition, learning, and personality, by exposing young and adult red junglefowl (Gallus gallus) to multiple learning tasks (discriminative, reversal, and spatial learning) and personality assays (novel arena, novel object, and tonic immobility). Learning speed was not correlated across learning tasks, and learning speed in discrimination and spatial learning tasks did not co-vary with personality. However, learning speed in reversal tasks was associated with individual variation in exploration, and in an age-dependent manner. More explorative chicks learned the reversal task faster than less explorative ones, while the opposite association was found for adult females (learning speed could not be assayed in adult males). In the same reversal tasks, we also observed a sex difference in learning speed of chicks, with females learning faster than males. Our results suggest that the relationship between cognition and personality is complex, as shown by its task- and age-dependence, and encourage further investigation of the causality and dynamics of this relationship. SIGNIFICANCE STATEMENT: In the ancestor of today's chickens, the red junglefowl, we explored how personality and cognition relate by exposing both chicks and adults to several learning tasks and personality assays. Our birds differed in personality and learning speed, while fast learners in one task did not necessarily learn fast in another (i.e., there were no overall "smarter" birds). Exploration correlated with learning speed in the more complex task of reversal learning: faster exploring chicks, but slower exploring adult females, learned faster, compared to less explorative birds. Other aspects of cognition and personality did not correlate. Our results suggest that cognition and personality are related, and that the relationship can differ depending on task and age of the animal.

Biased generalization of salient traits drives the evolution of warning signals.

The importance of receiver biases in shaping the evolution of many signalling systems is widely acknowledged. Here, we show that receiver bias can explain which traits evolve to become warning signals. For warning coloration, a generalization bias for a signalling trait can result from predators learning to discriminate unprofitable from profitable prey. However, because the colour patterns of prey are complex traits with multiple components, it is crucial to understand which of the many aspects of prey appearance evolve into signals. We provide experimental evidence that the more salient differences in prey traits give rise to greater generalization bias, corresponding to stronger selection towards trait exaggeration. Our results are based on experiments with domestic chickens as predators in a Skinner-box-like setting, and imply that the difference in appearance between profitable and unprofitable prey that is most rapidly learnt produces the greatest generalization bias. As a consequence, certain salient traits of unprofitable prey are selected towards exaggeration to even higher salience, driving the evolution of warning coloration. This general idea may also help to explain the evolution of many other striking signalling traits found in nature.

Learning of salient prey traits explains Batesian mimicry evolution.

Batesian mimicry evolution involves an initial major mutation that produces a rough resemblance to the model, followed by smaller improving changes. To examine the learning psychology of this process, we applied established ideas about mimicry in Papilio polyxenes asterius of the model Battus philenor. We performed experiments with wild birds as predators and butterfly wings as semiartificial prey. Wings of hybrids of P. p. asterius and Papilio machaon were used to approximate the first mutant, with melanism as the hypothesized first mimetic trait. Based on previous results about learning psychology and imperfect mimicry, we predicted that: melanism should have high salience (i.e., being noticeable and prominent), meaning that predators readily discriminate a melanistic mutant from appearances similar to P. machaon; the difference between the first mutant and the model should have intermediate salience to allow further improvement of mimicry; and the final difference in appearance between P. p. asterius and B. philenor should have very low salience, causing improvement to level off. Our results supported both the traditional hypothesis and all our predictions about relative salience. We conclude that there is good agreement between long-held ideas about how Batesian mimicry evolves and recent insights from learning psychology about the role of salience in mimicry evolution.