Adaptive dynamic programming for containment control with robustness analysis to iterative error: A global Nash equilibrium solution.

Global Nash equilibrium is an optimal solution for each player in a graphical game. This paper proposes an iterative adaptive dynamic programming-based algorithm to solve the global Nash equilibrium solution for optimal containment control problem with robustness analysis to the iterative error. The containment control problem is transferred into the graphical game formulation. Sufficient conditions are given to decouple the Hamilton-Jacobi equations, which guarantee the solvability of the global Nash equilibrium solution. The iterative algorithm is designed to obtain the solution without any knowledge of system dynamics. Conditions of iterative error for global stability are given with rigorous proof. Compared with existing works, the design procedures of control gain and coupling strength are separated, which avoids trivial cases in the design procedure. The robustness analysis exactly quantifies the effect of the iterative error caused by various sources in engineering practice. The theoretical results are validated by two numerical examples with marginally stable and unstable dynamics of the leader.

Optimal synchronization with L2-gain performance: An adaptive dynamic programming approach.

This paper studies an optimal synchronous control protocol design for nonlinear multi-agent systems under partially known dynamics and uncertain external disturbance. Under some mild assumptions, Hamilton-Jacobi-Isaacs equation is derived by the performance index function and system dynamics, which serves as an equivalent formulation. Distributed policy iteration adaptive dynamic programming is developed to obtain the numerical solution to the Hamilton-Jacobi-Isaacs equation. Three theoretical results are given about the proposed algorithm. First, the iterative variables is proved to converge to the solution to Hamilton-Jacobi-Isaacs equation. Second, the L2-gain performance of the closed loop system is achieved. As a special case, the origin of the nominal system is asymptotically stable. Third, the obtained control protocol constitutes an Nash equilibrium solution. Neural network-based implementation is designed following the main results. Finally, two numerical examples are provided to verify the effectiveness of the proposed method.

An evolutionary differential game for regulating the role of monoclonal antibodies in treating signalling pathways in oesophageal cancer.

This work presents a new framework for a competitive evolutionary game between monoclonal antibodies and signalling pathways in oesophageal cancer. The framework is based on a novel dynamical model that takes into account the dynamic progression of signalling pathways, resistance mechanisms and monoclonal antibody therapies. This game involves a scenario in which signalling pathways and monoclonal antibodies are the players competing against each other, where monoclonal antibodies use Brentuximab and Pembrolizumab dosages as strategies to counter the evolutionary resistance strategy implemented by the signalling pathways. Their interactions are described by the dynamical model, which serves as the game's playground. The analysis and computation of two game-theoretic strategies, Stackelberg and Nash equilibria, are conducted within this framework to ascertain the most favourable outcome for the patient. By comparing Stackelberg equilibria with Nash equilibria, numerical experiments show that the Stackelberg equilibria are superior for treating signalling pathways and are critical for the success of monoclonal antibodies in improving oesophageal cancer patient outcomes.

The application of innovative ecosystems to build resilient communities in response to major public health events.

In recent years, major public health events have had a significant and far-reaching impact on communities. As a response, there has been an increasing interest in enhancing community resilience through innovative ecosystems that involve diverse stakeholders with varying needs and demands. This study investigates the application of innovative ecosystems to improve community resilience in the face of major public health events by utilizing a sequential game approach to balance the interests of government, community, and residents. Subsequently, a comprehensive questionnaire survey was conducted among key stakeholders to ascertain their objectives, requirements and concerns for the innovation ecosystem based on the analysis results of the game model. The reliability and effectiveness of the proposed research method were verified through the analysis and verification of the sequence game model and questionnaire survey results. Finally, according to our analysis results, we propose countermeasures for promoting innovative ecosystems to improve community resilience. The research results indicate that the successful implementation of innovative ecosystems requires consideration of the different needs of stakeholders such as government officials, community members, and residents. Combining these perspectives can effectively promote such systems while enhancing the community's resilience to major public health events.

Sensitivity analysis for a dynamic macroeconomic policy game in a monetary union.

In this paper, we examine the sensitivity of the results of an earlier paper which presented and analyzed a dynamic game model of a monetary union with coalitions between governments (fiscal policy makers) and a common central bank (monetary policy maker). Here we examine alternative values of the parameters of the underlying model to show how the earlier results depend on the numerical parameter values chosen, which were obtained by calibration instead of econometric estimation. We demonstrate that the main results are qualitatively the same as in the original model for plausible smaller and larger values of the parameters. For the few cases where they differ, we interpret the deviations in economic terms and illustrate the policies and their macroeconomic effects resulting from the change to the parameter under consideration for one of these cases.

Coverage Strategy for Small-Cell UAV-Based Networks in IoT Environment.

In wireless communication, small cells are low-powered cellular base stations that can be used to enhance the coverage and capacity of wireless networks in areas where traditional cell towers may not be practical or cost-effective. Unmanned aerial vehicles (UAVs) can be used to quickly deploy and position small cells in areas that are difficult to access or where traditional infrastructure is not feasible. UAVs are deployed by telecommunication service providers to provide aerial network access in remote rural areas, disaster-affected areas, or massive-attendance events. In this paper, we focus on the scheduling of beaconing periods as an efficient means of energy consumption optimization. The conducted study provides a sub-modular game perspective of the problem and investigates its structural properties. We also provide a learning algorithm that ensures convergence of the considered UAV network to a Nash equilibrium operating point. Finally, we conduct extensive numerical investigations to assist our claims about the energy and data rate efficiency of the strategic beaconing policy (at Nash equilibrium).

PAC Reinforcement Learning Algorithm for General-Sum Markov Games.

This paper presents a theoretical framework for probably approximately correct (PAC) multi-agent reinforcement learning (MARL) algorithms for Markov games. Using the idea of delayed Q-learning, the paper extends the well-known Nash Q-learning algorithm to build a new PAC MARL algorithm for general-sum Markov games. In addition to guiding the design of a provably PACMARL algorithm, the framework enables checking whether an arbitrary MARL algorithm is PAC. Comparative numerical results demonstrate the algorithm's performance and robustness.

Nash equilibrium realization of population games based on social learning processes.

In the two-population game model, we assume the players have certain imitative learning abilities. To simulate the learning process of the game players, we propose a new swarm intelligence algorithm by combining the particle swarm optimization algorithm, where each player can be considered a particle. We conduct simulations for three typical games: the prisoner's dilemma game (with only one pure-strategy Nash equilibrium), the coin-flip game (with only one fully-mixed Nash equilibrium), and the coordination game (with two pure-strategy Nash equilibria and one fully-mixed Nash equilibrium). The results show that when the game has a pure strategy Nash equilibrium, the algorithm converges to that equilibrium. However, if the game does not have a pure strategy Nash equilibrium, it exhibits periodic convergence to the only mixed-strategy Nash equilibrium. Furthermore, the magnitude of the periodical convergence is inversely proportional to the introspection rate. After conducting experiments, our algorithm outperforms the Meta Equilibrium Q-learning algorithm in realizing mixed-strategy Nash equilibrium.

A Game Model for Analyzing Wireless Sensor Networks of 5G Environment Based on Adaptive Equilibrium Optimizer Algorithm.

Wireless sensors networks (WSNs) play an important role in life. With the development of 5G, its security issues have also raised concerns. Therefore, it is an important topic to study the offense and defense confrontation in WSNs. A complete information static game model is established to analyze the offense and defense confrontation problem of WSNs in 5G. An adaptive equilibrium optimizer algorithm (AEO) based on parameter adaptive strategy is proposed, which can jump out of the local optimal solution better. Experiments show that the optimization ability of AEO outperforms other algorithms on at least 80% of the 23 classical test functions of CEC. The convergence speed of AEO is better in the early stage of population iteration. The optimal offensive and defensive strategy under different offense and defense resources through simulation experiments is analyzed. The conclusion shows that when the offensive resources are large, the offender takes an indiscriminate attack. When the defense resources are small, the defender should defend the most important elements, and when the defense resources are large, the defender should allocate the same resources to defend each element to obtain the maximum benefit. This paper provides new solution ideas for the security problems under the offense and defense game in WSNs.

An impossibility theorem in game dynamics.

The Nash equilibrium-a combination of choices by the players of a game from which no self-interested player would deviate-is the predominant solution concept in game theory. Even though every game has a Nash equilibrium, it is not known whether there are deterministic behaviors of the players who play a game repeatedly that are guaranteed to converge to a Nash equilibrium of the game from all starting points. If one assumes that the players' behavior is a discrete-time or continuous-time rule whereby the current mixed strategy profile is mapped to the next, this question becomes a problem in the theory of dynamical systems. We apply this theory, and in particular Conley index theory, to prove a general impossibility result: There exist games, for which all game dynamics fail to converge to Nash equilibria from all starting points. The games which help prove this impossibility result are degenerate, but we conjecture that the same result holds, under computational complexity assumptions, for nondegenerate games. We also prove a stronger impossibility result for the solution concept of approximate Nash equilibria: For a set of games of positive measure, no game dynamics can converge to the set of approximate Nash equilibria for a sufficiently small yet substantial approximation bound. Our results establish that, although the notions of Nash equilibrium and its computation-inspired approximations are universally applicable in all games, they are fundamentally incomplete as predictors of long-term player behavior.

An adaptive neurodynamic approach for solving nonsmooth N-cluster games.

In this paper, N-cluster games with coupling and private constraints are studied, where each player's cost function is nonsmooth and depends on the actions of all players. In order to seek the generalized Nash equilibrium (GNE) of the nonsmooth N-cluster games, a distributed seeking neurodynamic approach with two-time-scale structure is proposed. An adaptive leader-following consensus technique is adapted to dynamically adjust parameters according to the degree of consensus violation, so as to quickly obtain accurate estimation information of other players' actions which facilitates the evaluation of its own cost. Benefitting from the unique structure of the approach based on primal dual and adaptive penalty methods, the players' actions enter the constraints while completing the seeking for GNE. As a result, the neurodynamic approach is completely distributed, and prior estimation of penalty parameters is avoided. Finally, two engineering examples of power system game and company capacity allocation verify the effectiveness and feasibility of the neurodynamic approach.

Should fiscal policies be centralized in a monetary union? A dynamic game approach.

In this paper we analyze dynamic interactions in a monetary union with three fiscal players (the governments of the countries concerned) and a common central bank in the presence of exogenous shocks. The model is calibrated for the euro area and includes a fiscally more solid core block denoted as country 1 as well as a fiscally less solid periphery block represented by countries 2 and 3. Introducing two periphery countries allows us to capture different attitudes of the periphery countries towards the goal of sustainable fiscal performance. Moreover, different coalition scenarios are modelled in this study including a fiscal union, a coalition of periphery countries and a coalition of fiscal-stability oriented countries. The exogenous shocks are calibrated in such a way as to describe the last major crises in the euro area, namely the financial crisis, the European sovereign debt crisis, the Covid-19 crisis, and the Ukraine war (energy price) crisis. Using the OPTGAME algorithm we calculate a cooperative Pareto and non-cooperative feedback Nash equilibrium solutions for the modelled scenarios. The fully cooperative solution yields the best results. The different non-cooperative scenarios allow insights into the underlying trade-off between economic growth, price stability and fiscal stability.

A game-theoretic model for the classification of selected oil companies' price changes.

One of the essential properties of a machine learning model is to be able to capture nuanced connections within data. This ability can be enhanced by considering alternative solution concepts, such as those offered by game theory. In this article, the Nash equilibrium is used as a solution concept to estimate probit parameters for the binary classification problem. A non-cooperative game is proposed in which data variables are players that attempt to maximize their marginal contribution to the log-likelihood function. A differential evolution algorithm is adapted to solve the proposed game. The new method is used to study the price changes of the Romanian oil company, OMV Petrom SA Romania, relative to the price of oil (crude and Brent) and the evolution of two other major oil companies with influence in the region. Results show that the proposed method outperforms the baseline probit and classical classification approaches in predicting price changes.

An integrated decision support system to achieve sustainable development in transportation routes with traffic flow.

Due to the growing population and demand, transportation planning has received special importance in the context of supply chain management. One of the major challenges in transportation planning is the traffic problem. This challenge affects the safety, environmental, and efficiency factors of transportation systems. Accordingly, in this study, the routes, which are important pillars of transportation planning, are examined from the perspective of sustainability. In this regard, a novel decision support system is developed, wherein at first, some decision-making methods including Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), entropy technique, Nash equilibrium point (NEP), and data envelopment analysis (DEA) are employed to analyze and determine unstable routes. Then, a bi-level leader-follower multi-objective optimization model is developed, based on the vehicle types, to evaluate the routes at different time intervals and identify the most efficient time intervals as a traffic pattern. Finally, the proposed models are implemented in a real case study based on the freeways in Tehran. According to the main finding, it is revealed that heavier and bulkier vehicles have a greater impact on road instability.

Two-stage dual-game model approach to view the difficulty of healthcare accessibility.

This study proposed a two-stage dual-game model methodology to evaluate the existing difficulty of healthcare accessibility in China. First, we analyzed a multi-player El Farol bar game with incomplete information by mixed strategy to explore the Nash equilibrium, and then a weighted El Farol bar game was discussed to identify the existence of a contradiction between supply and demand sides in a tertiary hospital. Second, the overall payoff based on healthcare quality was calculated. In terms of the probability of medical experience reaching that expected level, residents are not optimistic about going to the hospital, and the longer the observation period is, the more pronounced this trend becomes. By adjusting the threshold value to observe the change in the probability of being able to obtain the expected medical experience, it is found that the median number of hospital visits is a key parameter. Going to the hospital did bring benefits to people with consideration of the payoffs, while the benefits varied significantly with the observation period among different months. This study is recommended as a new method and approach to quantitatively assess the tense relationship in access to medical care between the demand and supply sides and a foundation for policy and practice improvements to ensure the efficient delivery of healthcare.

Computation Offloading and User-Clustering Game in Multi-Channel Cellular Networks for Mobile Edge Computing.

Mobile devices may use mobile edge computing to improve energy efficiency and responsiveness by offloading computation tasks to edge servers. However, the transmissions of mobile devices may result in interference that decreases the upload rate and prolongs transmission delay. Clustering has been shown as an effective approach to improve the transmission efficiency for dense devices, but there is no distributed algorithm for the optimization of clustering and computation offloading. In this work, we study the optimization problem of computation offloading to minimize the energy consumption of mobile devices in mobile edge computing by adaptively clustering devices to improve the transmission efficiency. To address the optimization problem in a distributed manner, the decision problem of clustering and computation offloading for mobile devices is formulated as a potential game. We introduce the construction of the potential game and show the existence of Nash equilibrium in the game with a finite enhancement ability. Then, we propose a distributed algorithm of clustering and computation offloading based on game theory. We conducted a simulation to evaluate the proposed algorithm. The numerical results from our simulation show that our algorithm can improve offloading efficiency for mobile devices in mobile edge computing by improving transmission efficiency. By offloading more tasks to edge servers, both the energy efficiency of mobile devices and the responsiveness of computation-intensive applications can be improved simultaneously.

Approximating Nash equilibrium for anti-UAV jamming Markov game using a novel event-triggered multi-agent reinforcement learning.

In the downlink communication, it is currently challenging for ground users to cope with the uncertain interference from aerial intelligent jammers. The cooperation and competition between ground users and unmanned aerial vehicle (UAV) jammers leads to a Markov game problem of anti-UAV jamming. Therefore, a model-free method is adopted based on multi-agent reinforcement learning (MARL) to handle the Markov game. However, the benchmark MARL strategies suffer from dimension explosion and local optimal convergence. To solve these issues, a novel event-triggered multi-agent proximal policy optimization algorithm with Beta strategy (ETMAPPO) is proposed in this paper, which aims to reduce the dimension of information transmission and improve the efficiency of policy convergence. In this event-triggering mechanism, agents can learn to obtain appropriate observation in different moment, thereby reducing the transmission of valueless information. Beta operator is used to optimize the action search. It expands the search scope of policy space. Ablation simulations show that the proposed strategy achieves better global benefits with fewer dimension of information than benchmark algorithms. In addition, the convergence performance verifies that the well-trained ETMAPPO has the capability to achieve stable jamming strategies and stable anti-jamming strategies. This approximately constitutes the Nash equilibrium of the anti-jamming Markov game.

Imperfect vaccine can yield multiple Nash equilibria in vaccination games.

As infectious diseases continue to threaten communities across the globe, people are faced with a choice to vaccinate, or not. Many factors influence this decision, such as the cost of the disease, the chance of contracting the disease, the population vaccination coverage, and the efficacy of the vaccine. While the vaccination games in which individuals decide whether to vaccinate or not based on their own interests are gaining in popularity in recent years, the vaccine imperfection has been an overlooked aspect so far. In this paper we investigate the effects of an imperfect vaccine on the outcomes of a vaccination game. We use a simple SIR compartmental model for the underlying model of disease transmission. We model the vaccine imperfection by adding vaccination at birth and maintain a possibility for the vaccinated individual to become infected. We derive explicit conditions for the existence of different Nash equilibria, the solutions of the vaccination game. The outcomes of the game depend on the complex interplay between disease transmission dynamics (the basic reproduction number), the relative cost of the infection, and the vaccine efficacy. We show that for diseases with relatively low basic reproduction numbers (smaller than about 2.62), there is a little difference between outcomes for perfect or imperfect vaccines and thus the simpler models assuming perfect vaccines are good enough. However, when the basic reproduction number is above 2.62, then, unlike in the case of a perfect vaccine, there can be multiple equilibria. Moreover, unless there is a mandatory vaccination policy in place that would push the vaccination coverage above the value of unstable Nash equilibrium, the population could eventually slip to the "do not vaccinate" state. Thus, for diseases that have relatively high basic reproduction numbers, the potential for the vaccine not being perfect should be explicitly considered in the models.

A policy-making model for evolutionary SME behavior during a pandemic recession supported on game theory approach.

The global economy has experienced a tremendous shock caused by the Covid-19 pandemic and its effects on the normal activities of SMEs, which provide essential driving economic force. Considering that there is currently no precise prediction about the end of this pandemic, many SMEs must make critical decisions about whether to remain in the market during the pandemic or to leave it, investing their assets in a more secure sector of the economy. However, in order to convince SMEs to remain in the market, thus maintaining the damaged economy, governments may variously apply punitive or supportive measures. In this regard, the interaction between SMEs strategies and government measures can be considered as an evolutionary game, in which the governments impose various policies after observing the evolutionary behaviors of SMEs. An evolutionary stable strategy (ESS) is derived through a replicator dynamic system, and the available payoff of each player is calculated by Nash equilibrium (NA). Finally, a numerical example is presented, and related managerial insights are proposed at the end of the current study. For instance, contrary to general belief, it can be inferred from investigating possible scenarios that punitive policies are more effective than supportive measures in convincing SMEs to remain in the market.

Competitive behaviour of major GSM firms' internet data pricing in Nigeria: A game theoretic model approach.

The study investigated the competitive behaviour of major GSM service providers' Internet data pricing in Nigeria. Two null hypotheses were tested: The major GSM service providers in Nigeria do not have a dominant pricing strategy; and there is no Nash Equilibrium for the pricing strategies of the major GSM service providers in Nigeria. The design was a longitudinal study of the active GSM subscribers and Internet data prices of major GSM service providers in Nigeria (MTN and the others). Two pricing strategies were employed, "charge N1000 per data plan" and "charge N1200 per data plan". A zero-sum payoff table was obtained from the data on GSM subscribers and Internet bundle prices in Nigeria. The payoff table was analysed using mixed strategy approach. The results revealed that each player has a dominant strategy. The game has a Pareto optimal Nash equilibrium with disparate dominant strategies for both players. The Nash equilibrium is that the other GSM firms charge N1000 per internet data plan while MTN charges N1200 per Internet data plan. At a price of N1000 and N1200 per Internet data plan, the earnings of the other GSM service providers and MTN are maximised respectively irrespective of what the other player does. The study concludes that major GSM firms can use game theory to model the pricing strategy of competitors and predict the behaviour of Internet data subscribers.