Evaluating the structure-coefficient theorem of evolutionary game theory.

In order to accommodate the empirical fact that population structures are rarely simple, modern studies of evolutionary dynamics allow for complicated and highly heterogeneous spatial structures. As a result, one of the most difficult obstacles lies in making analytical deductions, either qualitative or quantitative, about the long-term outcomes of evolution. The "structure-coefficient" theorem is a well-known approach to this problem for mutation-selection processes under weak selection, but a general method of evaluating the terms it comprises is lacking. Here, we provide such a method for populations of fixed (but arbitrary) size and structure, using easily interpretable demographic measures. This method encompasses a large family of evolutionary update mechanisms and extends the theorem to allow for asymmetric contests to provide a better understanding of the mutation-selection balance under more realistic circumstances. We apply the method to study social goods produced and distributed among individuals in spatially heterogeneous populations, where asymmetric interactions emerge naturally and the outcome of selection varies dramatically, depending on the nature of the social good, the spatial topology, and the frequency with which mutations arise.

Evolution of reciprocity with limited payoff memory.

Direct reciprocity is a mechanism for the evolution of cooperation in repeated social interactions. According to the literature, individuals naturally learn to adopt conditionally cooperative strategies if they have multiple encounters with their partner. Corresponding models have greatly facilitated our understanding of cooperation, yet they often make strong assumptions on how individuals remember and process payoff information. For example, when strategies are updated through social learning, it is commonly assumed that individuals compare their average payoffs. This would require them to compute (or remember) their payoffs against everyone else in the population. To understand how more realistic constraints influence direct reciprocity, we consider the evolution of conditional behaviours when individuals learn based on more recent experiences. Even in the most extreme case that they only take into account their very last interaction, we find that cooperation can still evolve. However, such individuals adopt less generous strategies, and they cooperate less often than in the classical setup with average payoffs. Interestingly, once individuals remember the payoffs of two or three recent interactions, cooperation rates quickly approach the classical limit. These findings contribute to a literature that explores which kind of cognitive capabilities are required for reciprocal cooperation. While our results suggest that some rudimentary form of payoff memory is necessary, it suffices to remember a few interactions.

The coalescent in finite populations with arbitrary, fixed structure.

The coalescent is a stochastic process representing ancestral lineages in a population undergoing neutral genetic drift. Originally defined for a well-mixed population, the coalescent has been adapted in various ways to accommodate spatial, age, and class structure, along with other features of real-world populations. To further extend the range of population structures to which coalescent theory applies, we formulate a coalescent process for a broad class of neutral drift models with arbitrary - but fixed - spatial, age, sex, and class structure, haploid or diploid genetics, and any fixed mating pattern. Here, the coalescent is represented as a random sequence of mappings [Formula: see text] from a finite set G to itself. The set G represents the "sites" (in individuals, in particular locations and/or classes) at which these alleles can live. The state of the coalescent, Ct:G→G, maps each site g∈G to the site containing g's ancestor, t time-steps into the past. Using this representation, we define and analyze coalescence time, coalescence branch length, mutations prior to coalescence, and stationary probabilities of identity-by-descent and identity-by-state. For low mutation, we provide a recipe for computing identity-by-descent and identity-by-state probabilities via the coalescent. Applying our results to a diploid population with arbitrary sex ratio r, we find that measures of genetic dissimilarity, among any set of sites, are scaled by 4r(1-r) relative to the even sex ratio case.

Intriguing effects of selection intensity on the evolution of prosocial behaviors.

In many models of evolving populations, genetic drift has an outsized role relative to natural selection, or vice versa. While there are many scenarios in which one of these two assumptions is reasonable, intermediate balances between these forces are also biologically relevant. In this study, we consider some natural axioms for modeling intermediate selection intensities, and we explore how to quantify the long-term evolutionary dynamics of such a process. To illustrate the sensitivity of evolutionary dynamics to drift and selection, we show that there can be a "sweet spot" for the balance of these two forces, with sufficient noise for rare mutants to become established and sufficient selection to spread. This balance allows prosocial traits to evolve in evolutionary models that were previously thought to be unconducive to the emergence and spread of altruistic behaviors. Furthermore, the effects of selection intensity on long-run evolutionary outcomes in these settings, such as when there is global competition for reproduction, can be highly non-monotonic. Although intermediate selection intensities (neither weak nor strong) are notoriously difficult to study analytically, they are often biologically relevant; and the results we report suggest that they can elicit novel and rich dynamics in the evolution of prosocial behaviors.

Payoff landscapes and the robustness of selfish optimization in iterated games.

In iterated games, a player can unilaterally exert influence over the outcome through a careful choice of strategy. A powerful class of such "payoff control" strategies was discovered by Press and Dyson (2012). Their so-called "zero-determinant" (ZD) strategies allow a player to unilaterally enforce a linear relationship between both players' payoffs. It was subsequently shown by Chen and Zinger (2014) that when the slope of this linear relationship is positive, ZD strategies are robustly effective against a selfishly optimizing co-player, in that all adapting paths of the selfish player lead to the maximal payoffs for both players (at least when there are certain restrictions on the game parameters). In this paper, we investigate the efficacy of selfish learning against a fixed player in more general settings, for both ZD and non-ZD strategies. We first prove that in any symmetric 2[Formula: see text]2 game, the selfish player's final strategy must be of a certain form and cannot be fully stochastic. We then show that there are prisoner's dilemma interactions for which selfish optimization does not always lead to maximal payoffs against fixed ZD strategies with positive slope. We give examples of selfish adapting paths that lead to locally but not globally optimal payoffs, undermining the robustness of payoff control strategies. For non-ZD strategies, these pathologies arise regardless of the original restrictions on the game parameters. Our results illuminate the difficulty of implementing robust payoff control and selfish optimization, even in the simplest context of playing against a fixed strategy.

Evolutionary dynamics with game transitions.

The environment has a strong influence on a population's evolutionary dynamics. Driven by both intrinsic and external factors, the environment is subject to continual change in nature. To capture an ever-changing environment, we consider a model of evolutionary dynamics with game transitions, where individuals' behaviors together with the games that they play in one time step influence the games to be played in the next time step. Within this model, we study the evolution of cooperation in structured populations and find a simple rule: Weak selection favors cooperation over defection if the ratio of the benefit provided by an altruistic behavior, b, to the corresponding cost, c, exceeds [Formula: see text], where k is the average number of neighbors of an individual and [Formula: see text] captures the effects of the game transitions. Even if cooperation cannot be favored in each individual game, allowing for a transition to a relatively valuable game after mutual cooperation and to a less valuable game after defection can result in a favorable outcome for cooperation. In particular, small variations in different games being played can promote cooperation markedly. Our results suggest that simple game transitions can serve as a mechanism for supporting prosocial behaviors in highly connected populations.

The Moran process on 2-chromatic graphs.

Resources are rarely distributed uniformly within a population. Heterogeneity in the concentration of a drug, the quality of breeding sites, or wealth can all affect evolutionary dynamics. In this study, we represent a collection of properties affecting the fitness at a given location using a color. A green node is rich in resources while a red node is poorer. More colors can represent a broader spectrum of resource qualities. For a population evolving according to the birth-death Moran model, the first question we address is which structures, identified by graph connectivity and graph coloring, are evolutionarily equivalent. We prove that all properly two-colored, undirected, regular graphs are evolutionarily equivalent (where "properly colored" means that no two neighbors have the same color). We then compare the effects of background heterogeneity on properly two-colored graphs to those with alternative schemes in which the colors are permuted. Finally, we discuss dynamic coloring as a model for spatiotemporal resource fluctuations, and we illustrate that random dynamic colorings often diminish the effects of background heterogeneity relative to a proper two-coloring.

Fixation probabilities in evolutionary dynamics under weak selection.

In evolutionary dynamics, a key measure of a mutant trait's success is the probability that it takes over the population given some initial mutant-appearance distribution. This "fixation probability" is difficult to compute in general, as it depends on the mutation's effect on the organism as well as the population's spatial structure, mating patterns, and other factors. In this study, we consider weak selection, which means that the mutation's effect on the organism is small. We obtain a weak-selection perturbation expansion of a mutant's fixation probability, from an arbitrary initial configuration of mutant and resident types. Our results apply to a broad class of stochastic evolutionary models, in which the size and spatial structure are arbitrary (but fixed). The problem of whether selection favors a given trait is thereby reduced from exponential to polynomial complexity in the population size, when selection is weak. We conclude by applying these methods to obtain new results for evolutionary dynamics on graphs.

The general form of Hamilton's rule makes no predictions and cannot be tested empirically.

Hamilton's rule asserts that a trait is favored by natural selection if the benefit to others, [Formula: see text], multiplied by relatedness, [Formula: see text], exceeds the cost to self, [Formula: see text] Specifically, Hamilton's rule states that the change in average trait value in a population is proportional to [Formula: see text] This rule is commonly believed to be a natural law making important predictions in biology, and its influence has spread from evolutionary biology to other fields including the social sciences. Whereas many feel that Hamilton's rule provides valuable intuition, there is disagreement even among experts as to how the quantities [Formula: see text], [Formula: see text], and [Formula: see text] should be defined for a given system. Here, we investigate a widely endorsed formulation of Hamilton's rule, which is said to be as general as natural selection itself. We show that, in this formulation, Hamilton's rule does not make predictions and cannot be tested empirically. It turns out that the parameters [Formula: see text] and [Formula: see text] depend on the change in average trait value and therefore cannot predict that change. In this formulation, which has been called "exact and general" by its proponents, Hamilton's rule can "predict" only the data that have already been given.

Asymmetric evolutionary games with environmental feedback.

Models in evolutionary game theory traditionally assume symmetric interactions in homogeneous environments. Here, we consider populations evolving in a heterogeneous environment, which consists of patches of different qualities that are occupied by one individual each. The fitness of individuals is not only determined by interactions with others but also by environmental quality. This heterogeneity results in asymmetric interactions where the characteristics of the interaction may depend on an individual's location. Interestingly, in non-varying heterogeneous environments, the long-term dynamics are the same as for symmetric interactions in an average, homogeneous environment. However, introducing environmental feedback between an individual's strategy and the quality of its patch results in rich eco-evolutionary dynamics. Thus, individuals act as ecosystem engineers. The nature of the feedback and the rate of ecological changes can relax or aggravate social dilemmas and promote persistent periodic oscillations of strategy abundance and environmental quality.

A mathematical formalism for natural selection with arbitrary spatial and genetic structure.

We define a general class of models representing natural selection between two alleles. The population size and spatial structure are arbitrary, but fixed. Genetics can be haploid, diploid, or otherwise; reproduction can be asexual or sexual. Biological events (e.g. births, deaths, mating, dispersal) depend in arbitrary fashion on the current population state. Our formalism is based on the idea of genetic sites. Each genetic site resides at a particular locus and houses a single allele. Each individual contains a number of sites equal to its ploidy (one for haploids, two for diploids, etc.). Selection occurs via replacement events, in which alleles in some sites are replaced by copies of others. Replacement events depend stochastically on the population state, leading to a Markov chain representation of natural selection. Within this formalism, we define reproductive value, fitness, neutral drift, and fixation probability, and prove relationships among them. We identify four criteria for evaluating which allele is selected and show that these become equivalent in the limit of low mutation. We then formalize the method of weak selection. The power of our formalism is illustrated with applications to evolutionary games on graphs and to selection in a haplodiploid population.

Autocratic strategies for iterated games with arbitrary action spaces.

The recent discovery of zero-determinant strategies for the iterated prisoner's dilemma sparked a surge of interest in the surprising fact that a player can exert unilateral control over iterated interactions. These remarkable strategies, however, are known to exist only in games in which players choose between two alternative actions such as "cooperate" and "defect." Here we introduce a broader class of autocratic strategies by extending zero-determinant strategies to iterated games with more general action spaces. We use the continuous donation game as an example, which represents an instance of the prisoner's dilemma that intuitively extends to a continuous range of cooperation levels. Surprisingly, despite the fact that the opponent has infinitely many donation levels from which to choose, a player can devise an autocratic strategy to enforce a linear relationship between his or her payoff and that of the opponent even when restricting his or her actions to merely two discrete levels of cooperation. In particular, a player can use such a strategy to extort an unfair share of the payoffs from the opponent. Therefore, although the action space of the continuous donation game dwarfs that of the classic prisoner's dilemma, players can still devise relatively simple autocratic and, in particular, extortionate strategies.

Public goods games in populations with fluctuating size.

Many mathematical frameworks of evolutionary game dynamics assume that the total population size is constant and that selection affects only the relative frequency of strategies. Here, we consider evolutionary game dynamics in an extended Wright-Fisher process with variable population size. In such a scenario, it is possible that the entire population becomes extinct. Survival of the population may depend on which strategy prevails in the game dynamics. Studying cooperative dilemmas, it is a natural feature of such a model that cooperators enable survival, while defectors drive extinction. Although defectors are favored for any mixed population, random drift could lead to their elimination and the resulting pure-cooperator population could survive. On the other hand, if the defectors remain, then the population will quickly go extinct because the frequency of cooperators steadily declines and defectors alone cannot survive. In a mutation-selection model, we find that (i) a steady supply of cooperators can enable long-term population survival, provided selection is sufficiently strong, and (ii) selection can increase the abundance of cooperators but reduce their relative frequency. Thus, evolutionary game dynamics in populations with variable size generate a multifaceted notion of what constitutes a trait's long-term success.

Autocratic strategies for alternating games.

Repeated games have a long tradition in the behavioral sciences and evolutionary biology. Recently, strategies were discovered that permit an unprecedented level of control over repeated interactions by enabling a player to unilaterally enforce linear constraints on payoffs. Here, we extend this theory of "zero-determinant" (or, more generally, "autocratic") strategies to alternating games, which are often biologically more relevant than traditional synchronous games. Alternating games naturally result in asymmetries between players because the first move matters or because players might not move with equal probabilities. In a strictly-alternating game with two players, X and Y, we give conditions for the existence of autocratic strategies for player X when (i) X moves first and (ii) Y moves first. Furthermore, we show that autocratic strategies exist even for (iii) games with randomly-alternating moves. Particularly important categories of autocratic strategies are extortionate and generous strategies, which enforce unfavorable and favorable outcomes for the opponent, respectively. We illustrate these strategies using the continuous Donation Game, in which a player pays a cost to provide a benefit to the opponent according to a continuous cooperative investment level. Asymmetries due to alternating moves could easily arise from dominance hierarchies, and we show that they can endow subordinate players with more autocratic strategies than dominant players.

Asymmetric Evolutionary Games.

Evolutionary game theory is a powerful framework for studying evolution in populations of interacting individuals. A common assumption in evolutionary game theory is that interactions are symmetric, which means that the players are distinguished by only their strategies. In nature, however, the microscopic interactions between players are nearly always asymmetric due to environmental effects, differing baseline characteristics, and other possible sources of heterogeneity. To model these phenomena, we introduce into evolutionary game theory two broad classes of asymmetric interactions: ecological and genotypic. Ecological asymmetry results from variation in the environments of the players, while genotypic asymmetry is a consequence of the players having differing baseline genotypes. We develop a theory of these forms of asymmetry for games in structured populations and use the classical social dilemmas, the Prisoner's Dilemma and the Snowdrift Game, for illustrations. Interestingly, asymmetric games reveal essential differences between models of genetic evolution based on reproduction and models of cultural evolution based on imitation that are not apparent in symmetric games.

Structure coefficients and strategy selection in multiplayer games.

Evolutionary processes based on two-player games such as the Prisoner's Dilemma or Snowdrift Game are abundant in evolutionary game theory. These processes, including those based on games with more than two strategies, have been studied extensively under the assumption that selection is weak. However, games involving more than two players have not received the same level of attention. To address this issue, and to relate two-player games to multiplayer games, we introduce a notion of reducibility for multiplayer games that captures what it means to break down a multiplayer game into a sequence of interactions with fewer players. We discuss the role of reducibility in structured populations, and we give examples of games that are irreducible in any population structure. Since the known conditions for strategy selection, otherwise known as [Formula: see text]-rules, have been established only for two-player games with multiple strategies and for multiplayer games with two strategies, we extend these rules to multiplayer games with many strategies to account for irreducible games that cannot be reduced to those simpler types of games. In particular, we show that the number of structure coefficients required for a symmetric game with [Formula: see text]-player interactions and [Formula: see text] strategies grows in [Formula: see text] like [Formula: see text]. Our results also cover a type of ecologically asymmetric game based on payoff values that are derived not only from the strategies of the players, but also from their spatial positions within the population.

Imitation dynamics on networks with incomplete information.

Imitation is an important learning heuristic in animal and human societies. Previous explorations report that the fate of individuals with cooperative strategies is sensitive to the protocol of imitation, leading to a conundrum about how different styles of imitation quantitatively impact the evolution of cooperation. Here, we take a different perspective on the personal and external social information required by imitation. We develop a general model of imitation dynamics with incomplete information in networked systems, which unifies classical update rules including the death-birth and pairwise-comparison rule on complex networks. Under pairwise interactions, we find that collective cooperation is most promoted if individuals neglect personal information. If personal information is considered, cooperators evolve more readily with more external information. Intriguingly, when interactions take place in groups on networks with low degrees of clustering, using more personal and less external information better facilitates cooperation. Our unifying perspective uncovers intuition by examining the rate and range of competition induced by different information situations.

Strategy evolution on dynamic networks.

Models of strategy evolution on static networks help us understand how population structure can promote the spread of traits like cooperation. One key mechanism is the formation of altruistic spatial clusters, where neighbors of a cooperative individual are likely to reciprocate, which protects prosocial traits from exploitation. However, most real-world interactions are ephemeral and subject to exogenous restructuring, so that social networks change over time. Strategic behavior on dynamic networks is difficult to study, and much less is known about the resulting evolutionary dynamics. Here we provide an analytical treatment of cooperation on dynamic networks, allowing for arbitrary spatial and temporal heterogeneity. We show that transitions among a large class of network structures can favor the spread of cooperation, even if each individual social network would inhibit cooperation when static. Furthermore, we show that spatial heterogeneity tends to inhibit cooperation, whereas temporal heterogeneity tends to promote it. Dynamic networks can have profound effects on the evolution of prosocial traits, even when individuals have no agency over network structures.

Evolutionary instability of selfish learning in repeated games.

Across many domains of interaction, both natural and artificial, individuals use past experience to shape future behaviors. The results of such learning processes depend on what individuals wish to maximize. A natural objective is one's own success. However, when two such "selfish" learners interact with each other, the outcome can be detrimental to both, especially when there are conflicts of interest. Here, we explore how a learner can align incentives with a selfish opponent. Moreover, we consider the dynamics that arise when learning rules themselves are subject to evolutionary pressure. By combining extensive simulations and analytical techniques, we demonstrate that selfish learning is unstable in most classical two-player repeated games. If evolution operates on the level of long-run payoffs, selection instead favors learning rules that incorporate social (other-regarding) preferences. To further corroborate these results, we analyze data from a repeated prisoner's dilemma experiment. We find that selfish learning is insufficient to explain human behavior when there is a trade-off between payoff maximization and fairness.

Evolution of prosocial behaviours in multilayer populations.

Human societies include diverse social relationships. Friends, family, business colleagues and online contacts can all contribute to one's social life. Individuals may behave differently in different domains, but success in one domain may engender success in another. Here, we study this problem using multilayer networks to model multiple domains of social interactions, in which individuals experience different environments and may express different behaviours. We provide a mathematical analysis and find that coupling between layers tends to promote prosocial behaviour. Even if prosociality is disfavoured in each layer alone, multilayer coupling can promote its proliferation in all layers simultaneously. We apply this analysis to six real-world multilayer networks, ranging from the socio-emotional and professional relationships in a Zambian community, to the online and offline relationships within an academic university. We discuss the implications of our results, which suggest that small modifications to interactions in one domain may catalyse prosociality in a different domain.