The evolution of social behaviors and risk preferences in settings with uncertainty.

Humans update their social behavior in response to past experiences and changing environments. Behavioral decisions are further complicated by uncertainty in the outcome of social interactions. Faced with uncertainty, some individuals exhibit risk aversion while others seek risk. Attitudes toward risk may depend on socioeconomic status; and individuals may update their risk preferences over time, which will feedback on their social behavior. Here, we study how uncertainty and risk preferences shape the evolution of social behaviors. We extend the game-theoretic framework for behavioral evolution to incorporate uncertainty about payoffs and variation in how individuals respond to this uncertainty. We find that different attitudes toward risk can substantially alter behavior and long-term outcomes, as individuals seek to optimize their rewards from social interactions. In a standard setting without risk, for example, defection always overtakes a well-mixed population engaged in the classic Prisoner's Dilemma, whereas risk aversion can reverse the direction of evolution, promoting cooperation over defection. When individuals update their risk preferences along with their strategic behaviors, a population can oscillate between periods dominated by risk-averse cooperators and periods of risk-seeking defectors. Our analysis provides a systematic account of how risk preferences modulate, and even coevolve with, behavior in an uncertain social world.

Emotions and courtship help bonded pairs cooperate, but emotional agents are vulnerable to deceit.

Coordinated pair bonds are common in birds and also occur in many other taxa. How do animals solve the social dilemmas they face in coordinating with a partner? We developed an evolutionary model to explore this question, based on observations that a) neuroendocrine feedback provides emotional bookkeeping which is thought to play a key role in vertebrate social bonds and b) these bonds are developed and maintained via courtship interactions that include low-stakes social dilemmas. Using agent-based simulation, we found that emotional bookkeeping and courtship sustained cooperation in the iterated prisoner's dilemma in noisy environments, especially when combined. However, when deceitful defection was possible at low cost, courtship often increased cooperation, whereas emotional bookkeeping decreased it.

Belief Updating during Social Interactions: Neural Dynamics and Causal Role of Dorsomedial Prefrontal Cortex.

In competitive interactions, humans have to flexibly update their beliefs about another person's intentions in order to adjust their own choice strategy, such as when believing that the other may exploit their cooperativeness. Here we investigate both the neural dynamics and the causal neural substrate of belief updating processes in humans. We used an adapted prisoner's dilemma game in which participants explicitly predicted the coplayer's actions, which allowed us to quantify the prediction error between expected and actual behavior. First, in an EEG experiment, we found a stronger medial frontal negativity (MFN) for negative than positive prediction errors, suggesting that this medial frontal ERP component may encode unexpected defection of the coplayer. The MFN also predicted subsequent belief updating after negative prediction errors. In a second experiment, we used transcranial magnetic stimulation (TMS) to investigate whether the dorsomedial prefrontal cortex (dmPFC) causally implements belief updating after unexpected outcomes. Our results show that dmPFC TMS impaired belief updating and strategic behavioral adjustments after negative prediction errors. Taken together, our findings reveal the time course of the use of prediction errors in social decisions and suggest that the dmPFC plays a crucial role in updating mental representations of others' intentions.

Third-party arbitration and forgiving strategies increase cooperation when perception errors are common.

Humans cooperate in groups in which mutual monitoring is common, and this provides the possibility of third-party arbitration. Third-party arbitration stabilizes reciprocity in at least two ways: first, when it is accurate, it reduces the frequency of misunderstandings resulting from perception errors, and second, even when it is inaccurate, it provides a public signal that allows pairs to align their expectations about how to behave after errors occur. Here, we describe experiments that test for these two effects. We find that in an iterated, sequential Prisoner's Dilemma game with errors, players with the highest average payoffs are those who make use of third-party arbitration and who also employ forgiving strategies. The combination of these two behaviours reduces the detrimental effects of errors on reciprocity, resulting in more cooperation.

Evolutionary branching in multi-level selection models.

We study a model of group-structured populations featuring individual-level birth and death events, and group-level fission and extinction events. Individuals play games within their groups, while groups play games against other groups. Payoffs from individual-level games affect birth rates of individuals, and payoffs from group-level games affect group extinction rates. We focus on the evolutionary dynamics of continuous traits with particular emphasis on the phenomenon of evolutionary diversification. Specifically, we consider two-level processes in which individuals and groups play continuous snowdrift or prisoner's dilemma games. Individual game strategies evolve due to selection pressure from both the individual and group level interactions. The resulting evolutionary dynamics turns out to be very complex, including branching and type-diversification at one level or the other. We observe that a weaker selection pressure at the individual level results in more adaptable groups and sometimes group-level branching. Stronger individual-level selection leads to more effective adaptation within each group while preventing the groups from adapting according to the group-level games.

Children cooperate more with in-group members than with out-group members in an iterated face-to-face Prisoner's Dilemma Game.

Adults are more likely to cooperate with in-group members than with out-group members in the context of social dilemmas, situations in which self-interest is in conflict with collective interest. This bias has the potential to profoundly shape human cooperation, and therefore it is important to understand when it emerges in development. Here we asked whether 6- to 9-year-old children (N = 146) preferentially cooperate with in-group members in the context of a well-studied social dilemma, the iterated Prisoner's Dilemma Game. We assigned children to minimal groups and paired them with unfamiliar same-age and same-gender peers. Consistent with our predictions, children were more likely to cooperate with in-group members than with out-group members in this minimal group context. This finding adds to the current literature on group bias in children's prosocial behavior by showing that it affects decision making in a context that calls on strategic cooperation. In addition, our analyses revealed an effect of gender, with girls more likely to cooperate than boys regardless of the group membership of their partner. Exploring this gender effect further, we found an interaction between gender and age across condition, with older girls showing less sensitivity to the group membership of their partner than younger girls and with older boys showing more sensitivity to the group membership of the partner than younger boys. Our findings suggest that risky cooperation in the face of social dilemmas is shaped by group bias during childhood, highlighting the potentially deeply rooted ties between cooperation and parochialism in humans.

Spatial social dilemmas promote diversity.

Cooperative investments in social dilemmas can spontaneously diversify into stably coexisting high and low contributors in well-mixed populations. Here we extend the analysis to emerging diversity in (spatially) structured populations. Using pair approximation, we derive analytical expressions for the invasion fitness of rare mutants in structured populations, which then yields a spatial adaptive dynamics framework. This allows us to predict changes arising from population structures in terms of existence and location of singular strategies, as well as their convergence and evolutionary stability as compared to well-mixed populations. Based on spatial adaptive dynamics and extensive individual-based simulations, we find that spatial structure has significant and varied impacts on evolutionary diversification in continuous social dilemmas. More specifically, spatial adaptive dynamics suggests that spontaneous diversification through evolutionary branching is suppressed, but simulations show that spatial dimensions offer new modes of diversification that are driven by an interplay of finite-size mutations and population structures. Even though spatial adaptive dynamics is unable to capture these new modes, they can still be understood based on an invasion analysis. In particular, population structures alter invasion fitness and can open up new regions in trait space where mutants can invade, but that may not be accessible to small mutational steps. Instead, stochastically appearing larger mutations or sequences of smaller mutations in a particular direction are required to bridge regions of unfavorable traits. The net effect is that spatial structure tends to promote diversification, especially when selection is strong.

Two-stage strategy update rule based on learning cost in weak prisoner's dilemma.

When players are dissatisfied with their actual payoffs, they will change the actuality by learning strategy of neighbors. The more effort players put in, the more likely they are to succeed in learning. Inspired by this, this paper proposes a two-stage strategy update rule based on learning cost. The players first decide whether to learn strategy according to the updating willingness. If the players imitate the strategy of neighbors, they need to pay the learning cost. Results show that for the well-mixed population, if the updating willingness is homogeneous and remains unchanged, reducing the updating willingness or increasing the learning cost can extend the life cycle of cooperators. If the updating willingness is heterogeneous and dynamically adjusted based on the difference between the actual payoff and the expected payoff, increasing aspiration value and learning cost promotes cooperation. For the structured population, if the updating willingness is homogeneous and remains unchanged, the moderate learning cost is beneficial for cooperators to resist the temptation of defection, and reducing updating willingness makes the system maintain cooperation within a larger parameter range. If the updating willingness is heterogeneous and dynamically adjusted, the larger learning cost and the appropriate aspiration value promote cooperation. This study highlights the complex dynamics of cooperation in paid strategy learning, contributing to the theory of cooperation in the evolutionary game.

Cooperation dynamics of prisoner's dilemma games on an evolutionary weighted network with heterogeneous preferences.

Relationships between people in real life are dynamically changed with the interaction process, and due to the heterogeneous preferences, this change is different from person to person. Based on this observation, we propose a new spatial and weighted prisoner's dilemma game model with heterogeneous individuals. Two types of tags, namely, tag-F (concerned about social fairness) and tag-W (concerned about personal well-being), are introduced to describe individuals' different preferences. The link weights indicating the interaction strength between individuals are updated based on different rules that depend on their tags. Through simulations, we verify that a large link weight control factor and a high proportion of tag-F individuals favor the emergence and persistence of cooperation. In addition, an increase in the link weight sensitivity factor favors the evolution of cooperation when the link weight control factor is small. Moreover, while the level of cooperation increases with the proportion of tag-F type in the population, contrary to our intuition, when the population consists entirely of tag-F individuals, in some cases, cooperation cannot reach a higher level compared with the situation when they are mixed with tag-W type. However, at high dilemma intensities, cooperators emerge only when the entire population consists of tag-F type. These results may provide some new insights into the impact of the evolutionary weighted network with heterogeneous preferences on collective cooperative behavior.

The effect of multi-tasks mechanism on cooperation in evolutionary game.

Human games are inherently diverse, involving more than mere identity interactions. The diversity of game tasks offers a more authentic explanation in the exploration of social dilemmas. Human behavior is also influenced by conformity, and prosociality is a crucial factor in addressing social dilemmas. This study proposes a generalized prisoner's dilemma model of task diversity that incorporates a conformity-driven interaction. Simulation findings indicate that the diversity of multi-tasks and the path dependence contribute to the flourishing of cooperation in games. Conformity-driven interactions also promote cooperation. However, this promotion effect does not increase linearly, and only appropriate task sizes and suitable proportions of conformity-driven interactions yield optimal results. From a broader group perspective, the interplay of network adaptation, task size, and conformity-driven interaction can form a structure of attractors or repellents.

Memory and target payoff enhance cooperation in evolutionary social dilemmas.

We proposed a neighbor selection mechanism based on memory and target payoff, where the target payoff is the maximum value of the group's average expected payoff. According to this mechanism, individuals prioritize selecting neighbors whose average payoffs in the last M rounds are close to the target payoff for strategy learning, aiming to maximize the group's expected payoff. Simulation results on the grid-based Prisoner's Dilemma and Snowdrift games demonstrate that this mechanism can significantly improve the group's payoff and cooperation level. Furthermore, the longer the memory length, the higher the group's payoff and cooperation level. Overall, the combination of memory and target payoff can lead to the emergence and persistence of cooperation in social dilemmas as individuals are motivated to cooperate based on both their past experiences and future goals. This interplay highlights the significance of taking into account numerous variables in comprehending and promoting cooperation within evolutionary frameworks.

Complex pathways to cooperation emergent from asymmetry in heterogeneous populations.

Cooperation within asymmetric populations has garnered significant attention in evolutionary games. This paper explores cooperation evolution in populations with weak and strong players, using a game model where players choose between cooperation and defection. Asymmetry stems from different benefits for strong and weak cooperators, with their benefit ratio indicating the degree of asymmetry. Varied rankings of parameters including the asymmetry degree, cooperation costs, and benefits brought by weak players give rise to scenarios including the prisoner's dilemma (PDG) for both player types, the snowdrift game (SDG), and mixed PDG-SDG interactions. Our results indicate that in an infinite well-mixed population, defection remains the dominant strategy when strong players engage in the prisoner's dilemma game. However, if strong players play snowdrift games, global cooperation increases with the proportion of strong players. In this scenario, strong cooperators can prevail over strong defectors when the proportion of strong players is low, but the prevalence of cooperation among strong players decreases as their proportion increases. In contrast, within a square lattice, the optimum global cooperation emerges at intermediate proportions of strong players with moderate degrees of asymmetry. Additionally, weak players protect cooperative clusters from exploitation by strong defectors. This study highlights the complex dynamics of cooperation in asymmetric interactions, contributing to the theory of cooperation in asymmetric games.

A game of raids: Expanding on a game theoretical approach utilising the prisoner's dilemma and ethnography in situ.

In this commentary, we set out the specifics of how Glowacki's game theoretical framework for the evolution of peace could be incorporated within broader cultural evolutionary approaches. We outline a formal proposal for prisoner's dilemma games investigating raid-based conflict. We also centre an ethnographic lens to understand the norms surrounding war and peace in intergroup interactions in small-scale communities.

Computer anthropomorphisation in a socio-economic dilemma.

In the study of human behaviour, non-social targets are often used as a control for human-to-human interactions. However, the concept of anthropomorphisation suggests that human-like qualities can be attributed to non-human objects. This can prove problematic in psychological experiments, as computers are often used as non-social targets. Here, we assessed the degree of computer anthropomorphisation in a sequential and iterated prisoner's dilemma. Participants (N = 41) faced three opponents in the prisoner's dilemma paradigm-a human, a computer, and a roulette-all represented by images presented at the commencement of each round. Cooperation choice frequencies and transition probabilities were estimated within subjects, in rounds against each opponent. We found that participants anthropomorphised the computer opponent to a high degree, while the same was not found for the roulette (i.e. no cooperation choice difference vs human opponents; p = .99). The difference in participants' behaviour towards the computer vs the roulette was further potentiated by the precedent roulette round, in terms of both cooperation choice (61%, p = .007) and cooperation probability after reciprocated defection (79%, p = .007). This suggests that there could be a considerable anthropomorphisation bias towards computer opponents in social games, even for those without a human-like appearance. Conversely, a roulette may be a preferable non-social control when the opponent's abilities are not explicit or familiar.

Oxytocin and the Punitive Hub-Dynamic Spread of Cooperation in Human Social Networks.

Human society operates on large-scale cooperation. However, individual differences in cooperativeness and incentives to free ride on others' cooperation make large-scale cooperation fragile and can lead to reduced social welfare. Thus, how individual cooperation spreads through human social networks remains puzzling from ecological, evolutionary, and societal perspectives. Here, we identify oxytocin and costly punishment as biobehavioral mechanisms that facilitate the propagation of cooperation in social networks. In three laboratory experiments (n = 870 human participants: 373 males, 497 females), individuals were embedded in heterogeneous networks and made repeated decisions with feedback in games of trust (n = 342), ultimatum bargaining (n = 324), and prisoner's dilemma with punishment (n = 204). In each heterogeneous network, individuals at central positions (hub nodes) were given intranasal oxytocin (or placebo). Giving oxytocin (vs matching placebo) to central individuals increased their trust and enforcement of cooperation norms. Oxytocin-enhanced norm enforcement, but not elevated trust, explained the spreading of cooperation throughout the social network. Moreover, grounded in evolutionary game theory, we simulated computer agents that interacted in heterogeneous networks with central nodes varying in terms of cooperation and punishment levels. Simulation results confirmed that central cooperators' willingness to punish noncooperation allowed the permeation of the network and enabled the evolution of network cooperation. These results identify an oxytocin-initiated proximate mechanism explaining how individual cooperation facilitates network-wide cooperation in human society and shed light on the widespread phenomenon of heterogeneous composition and enforcement systems at all levels of life.SIGNIFICANCE STATEMENT Human society operates on large-scale cooperation. Yet because cooperation is exploitable by free riding, how cooperation in social networks emerges remains puzzling from evolutionary and societal perspectives. Here we identify oxytocin and altruistic punishment as key factors facilitating the propagation of cooperation in human social networks. Individuals played repeated economic games in heterogeneous networks where individuals at central positions were given oxytocin or placebo. Oxytocin-enhanced cooperative norm enforcement, but not elevated trust, explained cooperation spreading throughout the social network. Evolutionary simulations confirmed that central cooperators' willingness to punish noncooperation allowed the permeation of the network and enabled the evolution of cooperation. These results identify an oxytocin-initiated proximate mechanism explaining how individual cooperation facilitates network-wide cooperation in human social networks.

Asymmetry of individual activity promotes cooperation in the spatial prisoner's dilemma game.

We consider an aspiration-based asymmetric individual activity co-evolutionary prisoner's dilemma game model on the square lattice. In detail, each player has an individual weight that evolves with its strategy. We introduce an asymmetric setting that only µ fractions of players in the network chosen to be active can update their individual weights according to whether their payoffs satisfy their aspirations. Therefore, our individual weights can be regarded as a type of intrinsic motivation satisfaction. Through Monte Carlo simulations on the square lattice, our co-evolution mechanism has been proven to significantly promote cooperation. Furthermore, we find that the moderate µ leads to the highest cooperation level. By dividing the players into four types according to their strategies and weights, we prove that the inversion of the dominant relationship between cooperators and defectors with low weights leads to a conversion cycle of the four types of players, which enhances spatial reciprocity to promote cooperation. Moderate µ results in an appropriate number of low-weight players in this cycle. Interestingly, our results show that higher weight heterogeneity does not lead to higher levels of cooperation, which is contrary to the intuition formed in the previous work.

Random migration with tie retention promotes cooperation in the prisoner's dilemma game.

Migration has the potential to induce outbreaks of cooperation, yet little is known about random migration. Does random migration really inhibit cooperation as often as previously thought? Besides, prior literature has often ignored the stickiness of social ties when designing migration protocols and assumed that players always immediately disconnect from their ex-neighbors once they migrate. However, this is not always true. Here, we propose a model where players can still retain some bonds with their ex-partners after they move from one place to another. The results show that maintaining a certain number of social ties, regardless of prosocial, exploitative, or punitive, can nevertheless facilitate cooperation even if migration occurs in a totally random fashion. Notably, it reflects that tie retention can help random migration, previously thought to be harmful to cooperation, restore the ability to spark bursts of cooperation. The maximum number of retained ex-neighbors plays an important role in facilitating cooperation. We analyze the impact of social diversity in terms of the maximum number of retained ex-neighbors and migration probability, and find that the former enhances cooperation while the latter often engenders an optimal dependence between cooperation and migration. Our results instantiate a scenario in which random migration yields the outbreak of cooperation and highlight the importance of social stickiness.

Cooperation driven by alike interactions in presence of social viscosity.

Cooperation observed in nearly all living systems, ranging from human and animal societies down to the scale of bacteria populations, is an astounding process through which individuals act together for mutual benefits. Despite being omnipresent, the mechanism behind the emergence and existence of cooperation in populations of selfish individuals has been a puzzle and exceedingly crucial to investigate. A number of mechanisms have been put forward to explain the stability of cooperation in the last years. In this work, we explore the evolution of cooperation for alike (assortative) interactions in populations subject to social viscosity in terms of zealous individuals. We present a comprehensive study on how a finite fraction of these committed minorities present in both cooperators and defectors govern the evolutionary game dynamics where interactions among the individuals with same strategy are more probable than random interactions. We perform a detailed analysis concerning this synergy between alike interaction and the social viscosity in the opposing individuals. We scrutinize all three principal social dilemmas, namely, the prisoner's dilemma, the stag-hunt, and the snowdrift game, under such evolutionary setting. We have been successful to delineate this evolutionary scenario theoretically based upon the generalized replicator dynamics in the well-mixed regime.

Inhibition and activation of interactions in networked weak prisoner's dilemma.

In the framework of the coevolution dynamics of the weak prisoner's dilemma, inspired by prior empirical research, we present a coevolutionary model with local network dynamics in a static network framework. Viewing the edges of the network as social interactions between individuals, when individuals play the weak prisoner's dilemma game, they accumulate both payoffs and social interaction willingness based on a payoff matrix of the social interaction willingness we constructed. The edges are then inhibiting or activating based on the social interaction willingness of the two individuals, and individuals only interact with others through activated edges, resulting in local network dynamics in a static network framework. Individuals who receive more cooperation will be more likely to activate the edges around them, meaning they will participate in more social interactions. Conversely, individuals who receive more defects will do the opposite. Specifically, we investigate the evolutionary dynamics of cooperation under different levels of sensitivity to social interaction willingness and the temptation to defect. Through the simulation, we find that sparse cooperator clusters can expand greatly when social interaction sensitivity and temptation to defect are low. In contrast, dense cooperator clusters form rapidly in a high social interaction sensitivity, which protects the cooperation from high temptation.

Personal Reflections on Some Early Work in Evolving Strategies in the Iterated Prisoner's Dilemma.

On the occasion of the 30-year anniversary of the Evolutionary Computation journal, I was invited by Professor Hart to offer some reflections on the article on evolving behaviors in the iterated prisoner's dilemma that I contributed to its first issue in 1993. It's an honor to do so. I would like to thank Professor Ken De Jong, the journal's first editor-in-chief, for his vision in creating the journal, and the editors who have followed and maintained that vision. This article contains some personal reflections on the topic and the field as a whole.