Persistence in repeated games encourages the evolution of spite.

Social behavior is divided into four types: altruism, spite, mutualism, and selfishness. The former two are costly to the actor; therefore, from the perspective of natural selection, their existence can be regarded as mysterious. One potential setup which encourages the evolution of altruism and spite is repeated interaction. Players can behave conditionally based on their opponent's previous actions in the repeated interaction. On the one hand, the retaliatory strategy (who behaves altruistically when their opponent behaved altruistically and behaves non-altruistically when the opponent player behaved non-altruistically) is likely to evolve when players choose altruistic or selfish behavior in each round. On the other hand, the anti-retaliatory strategy (who is spiteful when the opponent was not spiteful and is not spiteful when the opponent player was spiteful) is likely to evolve when players opt for spiteful or mutualistic behavior in each round. These successful conditional behaviors can be favored by natural selection. Here, we notice that information on opponent players' actions is not always available. When there is no such information, players cannot determine their behavior according to their opponent's action. By investigating the case of altruism, a previous study (Kurokawa, 2017, Mathematical Biosciences, 286, 94-103) found that persistent altruistic strategies, which choose the same action as the own previous action, are favored by natural selection. How, then, should a spiteful conditional strategy behave when the player does not know what their opponent did? By studying the repeated game, we find that persistent spiteful strategies, which choose the same action as the own previous action, are favored by natural selection. Altruism and spite differ concerning whether retaliatory or anti-retaliatory strategies are favored by natural selection; however, they are identical concerning whether persistent strategies are favored by natural selection.

Evolution of spite versus evolution of altruism through a disbandment mechanism.

Altruism and spite are costly to the actor, making their evolution unlikely without specific mechanisms. Nonetheless, both altruistic and spiteful behaviors are present in individuals, which suggests the existence of an underlying mechanism that drives their evolution. If altruistic individuals are more likely to be recipients of altruism than non-altruistic individuals, then altruism can be favored by natural selection. Similarly, if spiteful individuals are less likely to be recipients of spite than non-spiteful individuals, then spite can be favored by natural selection. Spite is altruism's evil twin, ugly sister of altruism, or a shady relative of altruism. In some mechanisms, such as repeated interactions, if altruism is favored by natural selection, then spite is also favored by natural selection. However, there has been limited investigation into whether both behaviors evolve to the same extent. In this study, we focus on the mechanism by which individuals choose to keep or stop the interaction according to the opponent's behavior. Using the evolutionary game theory, we investigate the evolution of altruism and spite under this mechanism. Our model revealed that the evolution of spite is less likely than the evolution of altruism.

Disbandment rules that most facilitate the evolution of cooperation.

Cooperation is considered a mysterious phenomenon from the perspective of adaptive evolution. However, if an individual can separate from an unsatisfactory group and join another, then this can facilitate positive assortment between cooperative types and promote the evolution of cooperation. What kind of disbandment rule most facilitates the evolution of cooperation? A previous study investigated exogenous disbandment rules and showed that, when games are played between two players, a rule where heterogeneous groups disband facilitates the evolution of cooperation. However, in groups of more than two individuals, a rule strictly requiring homogeneity applied if and only if the expected number of rounds played in a group was greater than some critical value. How large is the critical value? In this study, we make a mathematical analysis using evolutionary game theory. Our results show that the critical number of rounds increases greatly as the group size increases. Consequently, for species with large group sizes, e.g., Homo sapiens, under plausible parameter values, the strict homogeneity rule is unlikely to facilitate the evolution of cooperation. We find instead that a disbandment rule that requires a threshold level of homogeneity outperformed the strict homogeneity rule. Furthermore, we calculate the position of internal equilibria at which cooperators and defectors coexist and show that the initial evolution of cooperation is most encouraged when cooperators are tolerant (intolerant) of defectors if the benefit-to-cost ratio is large (small).

Evolution of spite in an n-player game with an opting-out option.

The evolution of altruism and spite is facilitated by positive and negative assortments, respectively. Animals repeatedly meet the same opponents and can choose to keep or terminate the interaction. Previous studies have showed that if the probability that the interaction stops depends on how the pairs are, then a positive assortment can emerge, encouraging the evolution of altruism in dyadic interactions and in interactions involving more than two individuals. In contrast, according to another previous study if the probability that the interaction stops depends on how the pairs are, then a negative assortment can emerge, encouraging the evolution of spite in dyadic interactions. Is the evolution of spite facilitated by interactions involving more than two individuals, in addition to dyadic interactions? The present study shows that the evolution of spite in interactions involving many individuals is possible by studying the repeated n-player game played using spiteful and nonspiteful strategies with opting-out options. These results suggest that spite in large groups may evolve through an opt-out mechanism. It also promotes the investigation about whether there are any examples of spiteful behavior in large groups that have evolved through negative assortment in opt-out options in nature.

Evolution of cooperation in an n-player game with opting out.

How cooperation could have evolved has been one of the central topics in evolutionary biology. When cooperators are likely to interact with other cooperators, and defectors are likely to interact with other defectors, positive assortment is created, facilitating the evolution of cooperation. Cooperation is observed not only in dyadic interactions but also, sometimes, in sizable groups. Previous studies have found that the opting out rule in which the group is disbanded if and only if the group is heterogeneous, especially facilitates the evolution of cooperation compared to the other opting out rules in multi-player games when the number of rounds is sufficiently large. However, the dynamics between the cooperators and defectors under such an opting out rule have been investigated only in the case where group size is relatively small (e.g., four). In addition, the effect of group size on the evolution of cooperation has not been explored, and considering that humans interact within larger groups, investigation in such large groups is essential. Here, through further algebraic analyses, it is newly revealed that there can be four internal equilibria when the group size is larger than four. In addition, while the effect of group size on cooperation is negative in the case of common goods, it is not straightforward (i.e., can be positive) in the case of public goods.

Evolution of trustfulness in the case where resources for cooperation are sometimes absent.

It is worth investigating the existence of cooperation, which is costly for the actor but beneficial to the recipient (precisely because it is costly for the former). If players, when they approach defectors, stop their relationship with them, cooperation can pay off and favorably emerge in the course of evolutionary dynamics. The present study examines the situation in which animals, even when they want to cooperate, sometimes lack the necessary resources, and are thereby prevented from cooperating with others. In addition, it is also considered that the underlying information about the presence or absence of these resources can be conveyed to the opponent player. Here, the opponent who defects-has no resources for cooperation-may be a cooperator or a defector. Therefore, it is not clear which behavior is more likely to evolve, if it is keeping the interaction with such an opponent (i.e., being trustful) or stopping the interaction with such an opponent (i.e., being not trustful). By using evolutionary game theory, it is revealed that those who want to keep the interaction with those without the resources to cooperate are favored by natural selection. This study sheds new light on the role of keeping and stopping interaction in the evolution of cooperation under variable availability of resources.

Marine Cleaning Mutualism Defies Standard Logic of Supply and Demand.

AbstractSupply and demand affect the values of goods exchanged in cooperative trades. Studies of humans and other species typically describe the standard scenario that an increase in demand leads to a higher price. Here, we challenge the generality of that logic with empirical data and a theoretical model. In our study system, "client" fishes visit cleaner wrasse (Labroides dimidiatus) to have ectoparasites removed, but cleaners prefer client mucus, which constitutes "cheating." We removed 31 of 65 preselected cleaners from a large isolated reef patch. We compared cleaner-client interactions at the reef and a control reef before removal and 4 weeks after removal. Cleaner fish from the experimental treatment site interacted more frequently with large clients (typically visitors with access to alternative cleaning stations), but we did not observe any changes in service quality measures. A game-theoretic analysis revealed that interaction duration and service quality might increase, decrease, or remain unchanged depending on the precise relationships between key parameters, such as the marginal benefits of cheating as a function of satiation or the likelihood of clients responding to cheating as a function of market conditions. The analyses show that the principle of diminishing return may affect exchanges in ways not predicted by supply-to-demand ratios.

Effect of the group size on the evolution of cooperation when an exit option is present.

The evolution of cooperation has been one of the main topics in evolutionary biology. If cooperators maintain interaction with cooperators and halt interaction with defectors, then cooperation can pay and can be favored by natural selection. This is called an exit option. Here, not only cooperation in dyadic interactions but also cooperation in sizable groups can be observed. Rivalry is about whether usage of the benefit by one individual reduces its availability to others or not. A common good is a rivalrous good, whereas a public good is a non-rivalrous good. In this paper, by analyzing n-player prisoner's dilemma games, we examine whether the effect of the group size on cooperation is positive or negative in the context of exit option. When goods are common goods, defectors always dominate cooperators when the group size is infinitely large. Thus, the group size has only negative effects on the evolution of cooperation when goods are common goods. In contrast, when goods are public goods, an increase in group size has positive effects as well as negative effects on the evolution of cooperation. In addition, we reveal that it has both positive and negative effects on the evolution of cooperation for cooperators to tolerate some defection and hope to keep the interaction.

Time to extinction of a cultural trait in an overlapping generation model.

How long a newly emerging trait will stay in a population is a fundamental but rarely asked question in cultural evolution. To tackle this question, the distribution and mean of the time to extinction of a discrete cultural trait are derived for models with overlapping generations, in which trait transmission occurs from multiple role models to a single newborn and may fail with a certain probability. We explore two models. The first is a Moran-type model, which allows us to derive the exact analytical formula for the mean time to extinction of a trait in a finite population. The second is a branching process, which assumes an infinitely large population and allows us to derive approximate analytical formulae for the distribution and mean of the time to extinction in the first model under a large population size. We show that in the first model, the mean time to extinction apparently diverges (becomes so large that even numerical computation is impractical) under a certain parameter condition as the population size tends to infinity. Using the second model, we explain the underlying mechanism of the apparent divergence found in the first model and derive the mathematical condition for this divergence in terms of transmission efficiency and the number of role models per newborn. When this mathematical condition is satisfied in the second model, the probability of extinction is less than 1, and the mean extinction time does not exist. In addition, we find that in both models, the time to extinction of the trait becomes longer as the number of role models per individual increases and as cultural transmission becomes more efficient.

Three-player repeated games with an opt-out option.

The existence of cooperation is problematic for those who view the world through an evolutionary lens. Although natural selection disfavors unconditional cooperation, cooperators who can distinguish cooperators from defectors and discontinue their interactions with the latter can be favored. Economic experiments have indicated that human beings indeed adopt such strategies when they detect defectors. Experiments have also shown that human beings also use those strategies in interactions involving more than two individuals as both cooperators and defectors may be present among opponent players. In such cases, it is not obvious whether it is better to continue or stop the interaction. In this study, I analyzed three-player prisoner's dilemma games. My findings algebraically reveal that both tolerant cooperators, who want to continue interacting with their current partners comprising one cooperator and one defector, and intolerant cooperators, who choose to discontinue such interactions, can survive when the cost of cooperation is small. In addition, my numerical analysis indicates that the coexistence of tolerant and intolerant cooperators cannot be an evolutionary outcome.

Which facilitates the evolution of cooperation more, retaliation or persistence?

The existence of cooperation in this world is a mysterious phenomenon. One of the mechanisms that explain the evolution of cooperation is repeated interaction. If interactions between the same individuals repeat and individuals cooperate conditionally, cooperation can evolve. A previous study pointed out that if individuals have persistence (i.e., imitate its "own" behavior in the last move), cooperation can evolve. However, retaliation and persistence are not mutually exclusive decisions, but rather a trade-off in the decision making process of individuals. Players can refer to the opponent's behavior and if the actor and the opponent opted for the different alternative in the last move, conditional cooperators have to give up either retaliation or persistence. The previous study also investigated this, and has revealed that the individual should give more importance to retaliation than to persistence. However, this study has assumed that the errors in perception are absent. In this world, errors in perception are present, and trying to imitate the opponent player can sometimes end in failure. And, it might be that imitating the focal player, which definitely ends in success, is more beneficial than trying to imitate the opponent player, which can end in failure especially when the error rate in recognition is large. Here, this paper uses evolutionarily stable strategy (ESS) analysis and analyzes the stability for reactive strategies against the invasion by unconditional defectors in the iterated prisoner's dilemma game. And our analysis reveals that even if we take errors in perception into consideration, retaliation facilitates the evolution of cooperation more than persistence unexpectedly. In addition, we analyze the stability for reactive cooperators against the invasion by a strategy other than unconditional defectors. Moreover, we also analyze the deterministic model in which unconditional cooperators, unconditional defectors, and the reactive strategy at the same time.

The extended reciprocity: Strong belief outperforms persistence.

The existence of cooperation is a mysterious phenomenon and demands explanation, and direct reciprocity is one key potential explanation for the evolution of cooperation. Direct reciprocity allows cooperation to evolve for cooperators who switch their behavior on the basis of information about the opponent's behavior. Here, relevant to direct reciprocity is information deficiency. When the opponent's last move is unknown, how should players behave? One possibility is to choose cooperation with some default probability without using any further information. In fact, our previous paper (Kurokawa, 2016a) examined this strategy. However, there might be beneficial information other than the opponent's last move. A subsequent study of ours (Kurokawa, 2017) examined the strategy which uses the own last move when the opponent's last move is unknown, and revealed that referring to the own move and trying to imitate it when information is absent is beneficial. Is there any other beneficial information else? How about strong belief (i.e., have infinite memory and believe that the opponent's behavior is unchanged)? Here, we examine the evolution of strategies with strong belief. Analyzing the repeated prisoner's dilemma game and using evolutionarily stable strategy (ESS) analysis against an invasion by unconditional defectors, we find the strategy with strong belief is more likely to evolve than the strategy which does not use information other than the opponent player's last move and more likely to evolve than the strategy which uses not only the opponent player's last move but also the own last move. Strong belief produces the extended reciprocity and facilitates the evolution of cooperation. Additionally, we consider the two strategies game between strategies with strong belief and any strategy, and we consider the four strategies game in which unconditional cooperators, unconditional defectors, pessimistic reciprocators with strong belief, and optimistic reciprocators with strong belief are present.

Imitation dynamics with time delay.

Based on the classic imitation dynamics (Hofbauer and Sigmund, 1998, Evolutionary Games and Population Dynamics, Cambridge University Press), the imitation dynamics with time delay is investigated, where the probability that an individual will imitate its opponent's own strategy is assumed to depend on the comparison between the past expected payoff of this individual's own strategy and the past expected payoff of its opponent's own strategy, i.e. there is a time delay effect. For the two-phenotype model, we show that if the system has an interior equilibrium and this interior equilibrium is stable when there is no time delay, then there must be a critical value of time delay such that the system tends to a stable periodic solution when the time delay is larger than the critical value. On the other hand, for three-phenotype (rock-scissors-paper) model, the numerical analysis shows that for the stable periodic solution induced by the time delay, the amplitude and the period will increase with the increase of the time delay. These results should help to understand the evolution of behavior based on the imitation dynamics with time delay.

Persistence extends reciprocity.

One key potential explanation for the evolution of cooperation is conditional cooperation. This allows cooperation to evolve for cooperators who switch their behavior on the basis of information about the opponent's behavior or reputation. However, information about the opponent's behavior or reputation is sometimes unavailable, and previous studies have assumed that a player cooperates with some default probability when no information about the opponent's previous behavior or reputation is available. This default probability has been interpreted as the player's "optimism". Here, we make use of the fact that even if a player cannot observe the opponent's previous behavior or reputation, he may still condition his future behavior based on his own past behavior and in such a case, he can behave persistently. In this paper, we especially consider the case where information about the opponent's "behavior" is sometimes absent and the iterated prisoner's dilemma game between the same two individuals is played. Here, we examine the evolution of strategies that can refer to the own behavior in the previous round. Using evolutionarily stable strategy (ESS) analysis and analyzing replicator dynamics, we find that conditioning his future behavior based on his own past behavior is beneficial for the evolution. Persistence facilitates the evolution of cooperation.

Evolution of group-wise cooperation: Is direct reciprocity insufficient?

Group-wise cooperation, or cooperation among three or more individuals, is an integral part of human societies. It is likely that group-wise cooperation also played a crucial role in the survival of early hominins, who were confronted with novel environmental challenges, long before the emergence of Homo sapiens. However, previous theoretical and empirical studies, focusing mainly on modern humans, have tended to suggest that evolution of cooperation in sizable groups cannot be explained by simple direct reciprocity and requires some additional mechanisms (reputation, punishment, etc.), which are cognitively too demanding for early hominins. As a partial resolution of the paradox, our recent analysis of a stochastic evolutionary model, which considers the effect of random drift, has revealed that evolution of group-wise cooperation is more likely to occur in larger groups when an individual's share of the benefit produced by one cooperator does not decrease with increasing group size (i.e., goods are non-rivalrous). In this paper, we further extend our previous analysis to explore possible consequences of introducing rare mistakes in behavior or imperfect information about behavior of others on the model outcome. Analyses of the extended models show that evolution of group-wise cooperation can be facilitated by large group size even when individuals intending to cooperate sometimes fail to do so or when all the information about the past behavior of group members is not available. We argue, therefore, that evolution of cooperation in sizable groups does not necessarily require other mechanisms than direct reciprocity if the goods to be produced via group-wise cooperation are non-rivalrous.

Unified and simple understanding for the evolution of conditional cooperators.

Cooperation is a mysterious phenomenon which is observed in this world. The potential explanation is a repeated interaction. Cooperation is established if individuals meet the same opponent repeatedly and cooperate conditionally. Previous studies have analyzed the following four as characters of conditional cooperators mainly. (i) niceness (i.e., when a conditional cooperator meets an opponent in the first place, he (she) cooperates or defects), (ii) optimism (when a conditional cooperator meets an opponent in the past, but he (she) did not get access to information about the opponent's behavior in the previous round, he (she) cooperates or defects), (iii) generosity (even when a conditional cooperator knows that an opponent defected in the previous round, he (she) cooperates or defects) and (iv) retaliation (a conditional cooperator cooperates with a cooperator with a higher probability than with a defector). Previous works deal with these four characters mainly. However, these four characters basically have been regarded as distinct topics and unified understanding has not been done fully. Here we, by studying the iterated prisoner's dilemma game (in particular, additive games) and using evolutionarily stable strategy (ESS) analysis, find that when retaliation is large, the condition under which conditional cooperators are stable against the invasion by an unconditional defector is loose, while none of "niceness", "optimism", and "generosity" makes impact on the condition under which conditional cooperators are stable against an invasion by an unconditional defector. Furthermore, we show that we can understand "niceness", "optimism", and "generosity" uniformly by using one parameter indicating "cooperative", and when the conditional cooperators have large "retaliation" enough to resist an invasion by an unconditional defector, natural selection favors more "cooperative" conditional cooperators to invade the resident conditional cooperative strategy. Moreover, we show that these results are robust even when taking the existence of mistakes in behavior into consideration.

Payoff non-linearity sways the effect of mistakes on the evolution of reciprocity.

The existence of cooperation is considered to require explanation, and reciprocity is a potential explanatory mechanism. Animals sometimes fail to cooperate even when they attempt to do so, and a reciprocator has an Achilles' heel: it is vulnerable to error (the interaction between two reciprocators can lead to an endless vendetta.). However, the strategy favored by natural selection is determined also by its interaction with other strategies. The relationship between two reciprocators leading to a collapse of cooperation through error does not straightforwardly imply that mistakes make the conditions under which reciprocity evolves stringent. Hence, mistakes may facilitate the evolution of reciprocity. However, it has been shown through the analysis of the interaction between reciprocators and unconditional defectors that the existence of mistakes makes the conditions for reciprocators stable against invasion by an unconditional defector more stringent, which indicates that mistakes discourage the evolution of reciprocity. However, this result is based on the assumption that the effects of cooperation are additive (payoff is linear), while the game played by real animals does not always display this feature. In such cases, the result may be swayed. In this paper, we remove this assumption, reexamining whether mistakes disturb the evolution of reciprocity. Using the analysis of an evolutionarily stable strategy (ESS), we show that when extra fitness costs are present in cases where mutual cooperation is established, mistakes can facilitate the evolution of reciprocity; whereas, when the effect of cooperation is additive, mistakes always disturb the evolution of reciprocity, as has been shown previously.

How Life History Can Sway the Fixation Probability of Mutants.

In this work, we study the effects of demographic structure on evolutionary dynamics when selection acts on reproduction, survival, or both. In contrast to the previously discovered pattern that the fixation probability of a neutral mutant decreases while the population becomes younger, we show that a mutant with a constant selective advantage may have a maximum or a minimum of the fixation probability in populations with an intermediate fraction of young individuals. This highlights the importance of life history and demographic structure in studying evolutionary dynamics. We also illustrate the fundamental differences between selection on reproduction and selection on survival when age structure is present. In addition, we evaluate the relative importance of size and structure of the population in determining the fixation probability of the mutant. Our work lays the foundation for also studying density- and frequency-dependent effects in populations when demographic structures cannot be neglected.

Imperfect information facilitates the evolution of reciprocity.

The existence of cooperation demands explanation since cooperation is costly to the actor. Reciprocity has long been regarded as a potential explanatory mechanism for the existence of cooperation. Reciprocity is a mechanism wherein a cooperator responds to an opponent's behavior by switching his/her own behavior. Hence, a possible problematic case relevant to the theory of reciprocity evolution arises when the mechanism is such that the information regarding an opponent's behavior is imperfect. Although it has been confirmed also by previous theoretical studies that imperfect information interferes with the evolution of reciprocity, this argument is based on the assumption that there are no mistakes in behavior. And, a previous study presumed that it might be expected that when such mistakes occur, reciprocity can more readily evolve in the case of imperfect information than in the case of perfect information. The reason why the previous study considers so is that in the former case, reciprocators can miss defections incurred by other reciprocators' mistakes due to imperfect information, allowing cooperation to persist when such reciprocators meet. However, contrary to this expectation, the previous study has shown that even when mistakes occur, imperfect information interferes with the evolution of reciprocity. Nevertheless, the previous study assumed that payoffs are linear (i.e., that the effect of behavior is additive and there are no synergetic effects). In this study, we revisited the same problem but removed the assumption that payoffs are linear. We used evolutionarily stable strategy analysis to compare the condition for reciprocity to evolve when mistakes occur and information is imperfect with the condition for reciprocity to evolve when mistakes occur and information is perfect. Our study revealed that when payoffs are not linear, imperfect information can facilitate the evolution of reciprocity when mistakes occur; while when payoffs are linear, imperfect information disturbs the evolution of reciprocity even when mistakes occur. Imperfect information can encourage the evolution of cooperation.

Evolution of social behavior in finite populations: a payoff transformation in general n-player games and its implications.

The evolution of social behavior has been the focus of many theoretical investigations, which typically have assumed infinite populations and specific payoff structures. This paper explores the evolution of social behavior in a finite population using a general n-player game. First, we classify social behaviors in a group of n individuals based on their effects on the actor's and the social partner's payoffs, showing that in general such classification is possible only for a given composition of strategies in the group. Second, we introduce a novel transformation of payoffs in the general n-player game to formulate explicitly the effects of a social behavior on the actor's and the social partners' payoffs. Third, using the transformed payoffs, we derive the conditions for a social behavior to be favored by natural selection in a well-mixed population and in the presence of multilevel selection.