A general framework for modelling trade-offs in adaptive behaviour.

An animal's behaviour can influence many variables, such as its energy reserves, its risk of injury or mortality, and its rate of reproduction. To identify the optimal action in a given situation, these various effects can be compared in the common currency of reproductive value. While this idea has been widely used to study trade-offs between pairs of variables, e.g. between energy gain versus survival, here we present a unified framework that makes explicit how these various trade-offs fit together. This unification covers a wide range of biological phenomena, highlighting similarities in their logical structure and helping to identify knowledge gaps. To fill one such gap, we present a new model of foraging under the risk of predation and damage accumulation. We conclude by discussing the use and limitations of state-dependent optimisation theory in predicting biological observations.

Game theory in biology: 50 years and onwards.

Game theory in biology gained prominence 50 years ago, when Maynard Smith & Price formulated the concept of an evolutionarily stable strategy (ESS). Their aim was to explain why conflicts between animals of the same species usually are of a 'limited war' type, not causing serious injury. They emphasized that game theory is an alternative to previous ideas about group selection, which were used by ethologists to explain limited aggression. Subsequently, the ESS concept was applied to many phenomena with frequency dependence in the evolutionary success of strategies, including sex allocation, alternative mating types, contest behaviour and signalling, cooperation, and parental care. Both the analyses of signalling and cooperation were inspired by similar problems in economics and attracted much attention in biology. Here we give a perspective on which of the ambitions in the field have been achieved, with a focus on contest behaviour and cooperation. We evaluate whether the game-theoretical study of the evolution of cooperation has measured up to expectations in explaining the behaviour of non-human animals. We also point to potentially fruitful directions for the field, and emphasize the importance of incorporating realistic behavioural mechanisms into models. This article is part of the theme issue 'Half a century of evolutionary games: a synthesis of theory, application and future directions'.

Social interaction can select for reduced ability.

Animals, including humans, differ in a wide range of physical and cognitive abilities ranging from measures of running speed and physical strength to learning ability and intelligence. We consider the evolution of ability when individuals interact pairwise over their contribution to a common good. In this interaction, the contribution of each is assumed to be the best given their own ability and the contribution of their partner. Since there is a tendency for individuals to partially compensate for a low contribution by their partner, low-ability individuals can do well. As a consequence, for benefit and cost structures for which individuals have a strong response to partner's contribution, there can be selection for reduced ability. Furthermore, there can be disruptive selection on ability, leading to a bimodal distribution of ability under some modes of inheritance.

Behavioural specialization and learning in social networks.

Interactions in social groups can promote behavioural specialization. One way this can happen is when individuals engage in activities with two behavioural options and learn which option to choose. We analyse interactions in groups where individuals learn from playing games with two actions and negatively frequency-dependent payoffs, such as producer-scrounger, caller-satellite, or hawk-dove games. Group members are placed in social networks, characterized by the group size and the number of neighbours to interact with, ranging from just a few neighbours to interactions between all group members. The networks we analyse include ring lattices and the much-studied small-world networks. By implementing two basic reinforcement-learning approaches, action-value learning and actor-critic learning, in different games, we find that individuals often show behavioural specialization. Specialization develops more rapidly when there are few neighbours in a network and when learning rates are high. There can be learned specialization also with many neighbours, but we show that, for action-value learning, behavioural consistency over time is higher with a smaller number of neighbours. We conclude that frequency-dependent competition for resources is a main driver of specialization. We discuss our theoretical results in relation to experimental and field observations of behavioural specialization in social situations.

An evolutionary perspective on stress responses, damage and repair.

Variation in stress responses has been investigated in relation to environmental factors, species ecology, life history and fitness. Moreover, mechanistic studies have unravelled molecular mechanisms of how acute and chronic stress responses cause physiological impacts ('damage'), and how this damage can be repaired. However, it is not yet understood how the fitness effects of damage and repair influence stress response evolution. Here we study the evolution of hormone levels as a function of stressor occurrence, damage and the efficiency of repair. We hypothesise that the evolution of stress responses depends on the fitness consequences of damage and the ability to repair that damage. To obtain some general insights, we model a simplified scenario in which an organism repeatedly encounters a stressor with a certain frequency and predictability (temporal autocorrelation). The organism can defend itself by mounting a stress response (elevated hormone level), but this causes damage that takes time to repair. We identify optimal strategies in this scenario and then investigate how those strategies respond to acute and chronic exposures to the stressor. We find that for higher repair rates, baseline and peak hormone levels are higher. This typically means that the organism experiences higher levels of damage, which it can afford because that damage is repaired more quickly, but for very high repair rates the damage does not build up. With increasing predictability of the stressor, stress responses are sustained for longer, because the animal expects the stressor to persist, and thus damage builds up. This can result in very high (and potentially fatal) levels of damage when organisms are exposed to chronic stressors to which they are not evolutionarily adapted. Overall, our results highlight that at least three factors need to be considered jointly to advance our understanding of how stress physiology has evolved: (i) temporal dynamics of stressor occurrence; (ii) relative mortality risk imposed by the stressor itself versus damage caused by the stress response; and (iii) the efficiency of repair mechanisms.

Game Theory in Biology: Moving beyond Functional Accounts.

AbstractThe idea of applying game theory to problems in biology was given a formal basis nearly 50 years ago. Since then, the theory has advanced, and there have been numerous applications of it to a diversity of empirical systems. Most of this work takes a straightforward functional approach, finding a behavioral strategy that is evolutionarily stable in a well-specified specific situation. Relatively little attention has been devoted to the role of phylogeny, the role of learning during development, and the limitations imposed by the psychological and physiological mechanisms that bring about behavior in a complex world. Here I argue that a focus on these elements can improve the link between the theory and empirical systems and hence help us to understand how natural selection has shaped observed behavior.

The evolution of social learning as phenotypic cue integration.

Most analyses of the origins of cultural evolution focus on when and where social learning prevails over individual learning, overlooking the fact that there are other developmental inputs that influence phenotypic fit to the selective environment. This raises the question of how the presence of other cue 'channels' affects the scope for social learning. Here, we present a model that considers the simultaneous evolution of (i) multiple forms of social learning (involving vertical or horizontal learning based on either prestige or conformity biases) within the broader context of other evolving inputs on phenotype determination, including (ii) heritable epigenetic factors, (iii) individual learning, (iv) environmental and cascading maternal effects, (v) conservative bet-hedging, and (vi) genetic cues. In fluctuating environments that are autocorrelated (and hence predictable), we find that social learning from members of the same generation (horizontal social learning) explains the large majority of phenotypic variation, whereas other cues are much less important. Moreover, social learning based on prestige biases typically prevails in positively autocorrelated environments, whereas conformity biases prevail in negatively autocorrelated environments. Only when environments are unpredictable or horizontal social learning is characterized by an intrinsically low information content, other cues such as conservative bet-hedging or vertical prestige biases prevail. This article is part of the theme issue 'Foundations of cultural evolution'.

Learning, exploitation and bias in games.

We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple 'unbiased' reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual's behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is 'rational'.

Matching Behaviours and Rewards.

Matching describes how behaviour is related to rewards. The matching law holds when the ratio of an individual's behaviours equals the ratio of the rewards obtained. From its origins in the study of pigeons working for food in the laboratory, the law has been applied to a range of species, both in the laboratory and outside it (e.g., human sporting decisions). Probability matching occurs when the probability of a behaviour equals the probability of being rewarded. Input matching predicts the distribution of individuals across habitats. We evaluate the rationality of the matching law and probability matching, expose the logic of matching in real-world cases, review how recent neuroscience findings relate to matching, and suggest future research directions.

Towards an Evolutionary Theory of Stress Responses.

All organisms have a stress response system to cope with environmental threats, yet its precise form varies hugely within and across individuals, populations, and species. While the physiological mechanisms are increasingly understood, how stress responses have evolved remains elusive. Here, we show that important insights can be gained from models that incorporate physiological mechanisms within an evolutionary optimality analysis (the 'evo-mecho' approach). Our approach reveals environmental predictability and physiological constraints as key factors shaping stress response evolution, generating testable predictions about variation across species and contexts. We call for an integrated research programme combining theory, experimental evolution, and comparative analysis to advance scientific understanding of how this core physiological system has evolved.

Behavioural flexibility and reputation formation.

Limited flexibility in behaviour gives rise to behavioural consistency, so that past behaviour is partially predictive of current behaviour. The consequences of limits to flexibility are investigated in a population in which pairs of individuals play a game of trust. The game can either be observed by others or not. Reputation is based on trustworthiness when observed and acts as a signal of behaviour in future interactions with others. Individuals use the reputation of partner in deciding whether to trust them, both when observed by others and when not observed. We explore the effects of costs of exhibiting a difference in behaviour between when observed and when not observed (i.e. a cost of flexibility). When costs are low, individuals do not attempt to signal that they will later be trustworthy: their signal should not be believed since it will always pay them to be untrustworthy if trusted. When costs are high, their local optimal behaviour automatically acts as an honest signal. At intermediate costs, individuals are very trustworthy when observed in order to convince others of their trustworthiness when unobserved. It is hypothesized that this type of strong signalling might occur in other settings.

Environmental variability, reliability of information and the timing of migration.

The timing of migration and migratory steps is highly relevant for fitness. Because environmental conditions vary between years, the optimal time for migration varies accordingly. Therefore, migratory animals could clearly benefit from acquiring information as to when it is the best time to migrate in a specific year. Thus, environmental predictability and variability are fundamental characteristics of migration systems but their relationship and consequence for migratory progression has remained unexplored. We develop a simple dynamic model to identify the optimal migration behaviour in environments that differ in predictability, variability and the number of intermediate stop-over sites. Our results indicate that higher predictability along migration routes enables organisms to better time migration when phenology deviates from its long-term average and thus, increases fitness. Information is particularly valuable in highly variable environments and in the final migration-step, i.e. before the destination. Furthermore, we show that a general strategy for obtaining information in relatively uninformative but variable environments is using intermediate stop-over sites that enable migrants to better predict conditions ahead. Our study contributes to a better understanding of the relationship between animal movement and environmental predictability-an important, yet underappreciated factor that strongly influences migratory progression.

Learning leads to bounded rationality and the evolution of cognitive bias in public goods games.

In social interactions, including cooperation and conflict, individuals can adjust their behaviour over the shorter term through learning within a generation, and natural selection can change behaviour over the longer term of many generations. Here we investigate the evolution of cognitive bias by individuals investing into a project that delivers joint benefits. For members of a group that learn how much to invest using the costs and benefits they experience in repeated interactions, we show that overestimation of the cost of investing can evolve. The bias causes individuals to invest less into the project. Our explanation is that learning responds to immediate rather than longer-term rewards. There are thus cognitive limitations in learning, which can be seen as bounded rationality. Over a time horizon of several rounds of interaction, individuals respond to each other's investments, for instance by partially compensating for another's shortfall. However, learning individuals fail to strategically take into account that social partners respond in this way. Learning instead converges to a one-shot Nash equilibrium of a game with perceived rewards as payoffs. Evolution of bias can then compensate for the cognitive limitations of learning.

Ecological Genetic Conflict: Genetic Architecture Can Shift the Balance between Local Adaptation and Plasticity.

Genetic polymorphism can contribute to local adaptation in heterogeneous habitats, for instance, as a single locus with alleles adapted to different habitats. Phenotypic plasticity can also contribute to trait variation across habitats, through developmental responses to habitat-specific cues. We show that the genetic architecture of genetically polymorphic and plasticity loci may influence the balance between local adaptation and phenotypic plasticity. These effects of genetic architecture are instances of ecological genetic conflict. A reduced effective migration rate for genes tightly linked to a genetic polymorphism provides an explanation for the effects, and they can occur both for a single trait and for a syndrome of coadapted traits. Using individual-based simulations and numerical analysis, we investigate how among-habitat genetic polymorphism and phenotypic plasticity depend on genetic architecture. We also study the evolution of genetic architecture itself, in the form of rates of recombination between genetically polymorphic loci and plasticity loci. Our main result is that for plasticity genes that are unlinked to loci with between-habitat genetic polymorphism, the slope of a reaction norm is steeper in comparison with the slope favored by plasticity genes that are tightly linked to genes for local adaptation.

Echoes of Early Life: Recent Insights From Mathematical Modeling.

In the last decades, developmental origins of health and disease (DOHaD) has emerged as a central framework for studying early-life effects, that is, the impact of fetal and early postnatal experience on adult functioning. Apace with empirical progress, theoreticians have built mathematical models that provide novel insights for DOHaD. This article focuses on three of these insights, which show the power of environmental noise (i.e., imperfect indicators of current and future conditions) in shaping development. Such noise can produce: (a) detrimental outcomes even in ontogenetically stable environments, (b) individual differences in sensitive periods, and (c) early-life effects tailored to predicted future somatic states. We argue that these insights extend DOHaD and offer new research directions.

Optimal gut size of small birds and its dependence on environmental and physiological parameters.

Most optimal foraging models assume that the foraging behaviour of small birds depends on a single state variable, their energy reserves in the form of stored fat. Here, we include a second state variable-the contents of the bird's gut-to investigate how a bird should optimise its gut size to minimise its long-term mortality, depending on the availability of food, the size of meal and the bird's digestive constraints. Our results show that (1) the current level of fat is never less important than gut contents in determining the bird's survival; (2) there exists a unique optimal gut size, which is determined by a trade-off between the energetic gains and costs of maintaining a large digestive system; (3) the optimal gut size increases as the bird's digestive cycle becomes slower, allowing the bird to store undigested food; (4) the critical environmental factor for determining the optimal gut size is the mass of food found in a successful foraging effort ("meal size"). We find that when the environment is harsh, it is optimal for the bird to maintain a gut that is larger than the size of a meal. However, the optimal size of the gut in rich environments exactly matches the meal size (i.e. the mass of food that the optimal gut can carry is exactly the mass of food that can be obtained in a successful foraging attempt).

Trust your gut: using physiological states as a source of information is almost as effective as optimal Bayesian learning.

Approaches to understanding adaptive behaviour often assume that animals have perfect information about environmental conditions or are capable of sophisticated learning. If such learning abilities are costly, however, natural selection will favour simpler mechanisms for controlling behaviour when faced with uncertain conditions. Here, we show that, in a foraging context, a strategy based only on current energy reserves often performs almost as well as a Bayesian learning strategy that integrates all previous experiences to form an optimal estimate of environmental conditions. We find that Bayesian learning gives a strong advantage only if fluctuations in the food supply are very strong and reasonably frequent. The performance of both the Bayesian and the reserve-based strategy are more robust to inaccurate knowledge of the temporal pattern of environmental conditions than a strategy that has perfect knowledge about current conditions. Studies assuming Bayesian learning are often accused of being unrealistic; our results suggest that animals can achieve a similar level of performance to Bayesians using much simpler mechanisms based on their physiological state. More broadly, our work suggests that the ability to use internal states as a source of information about recent environmental conditions will have weakened selection for sophisticated learning and decision-making systems.

The erroneous signals of detection theory.

Signal detection theory has influenced the behavioural sciences for over 50 years. The theory provides a simple equation that indicates numerous 'intuitive' results; e.g. prey should be more prone to take evasive action (in response to an ambiguous cue) if predators are more common. Here, we use analytical and computational models to show that, in numerous biological scenarios, the standard results of signal detection theory do not apply; more predators can result in prey being less responsive to such cues. The standard results need not apply when the probability of danger pertains not just to the present, but also to future decisions. We identify how responses to risk should depend on background mortality and autocorrelation, and that predictions in relation to animal welfare can also be reversed from the standard theory.

Towards a behavioural ecology of obesity.

Addressing the obesity epidemic depends on a holistic understanding of the reasons that people become and maintain excessive fat. Theories about the causes of obesity usually focus proximately or evoke evolutionary mismatches, with minimal clinical value. There is potential for substantial progress by adapting strategic body mass regulation models from evolutionary ecology to human obesity by assessing the role of information.

Detection vs. selection: integration of genetic, epigenetic and environmental cues in fluctuating environments.

There are many inputs during development that influence an organism's fit to current or upcoming environments. These include genetic effects, transgenerational epigenetic influences, environmental cues and developmental noise, which are rarely investigated in the same formal framework. We study an analytically tractable evolutionary model, in which cues are integrated to determine mature phenotypes in fluctuating environments. Environmental cues received during development and by the mother as an adult act as detection-based (individually observed) cues. The mother's phenotype and a quantitative genetic effect act as selection-based cues (they correlate with environmental states after selection). We specify when such cues are complementary and tend to be used together, and when using the most informative cue will predominate. Thus, we extend recent analyses of the evolutionary implications of subsets of these effects by providing a general diagnosis of the conditions under which detection and selection-based influences on development are likely to evolve and coexist.