Optimal Evolution Strategy for Continuous Strategy Games on Complex Networks via Reinforcement Learning.

This article presents an optimal evolution strategy for continuous strategy games on complex networks via reinforcement learning (RL). In the past, evolutionary game theory usually assumed that agents use the same selection intensity when interacting, ignoring the differences in their learning abilities and learning willingness. Individuals are reluctant to change their strategies too much. Therefore, we design an adaptive strategy updating framework with various selection intensities for continuous strategy games on complex networks based on imitation dynamics, allowing agents to achieve the optimal state and a higher cooperation level with the minimal strategy changes. The optimal updating strategy is acquired using a coupled Hamilton-Jacobi-Bellman (HJB) equation by minimizing the performance function. This function aims to maximize individual payoffs while minimizing strategy changes. Furthermore, a value iteration (VI) RL algorithm is proposed to approximate the HJB solutions and learn the optimal strategy updating rules. The RL algorithm employs actor and critic neural networks to approximate strategy changes and performance functions, along with the gradient descent weight update approach. Meanwhile, the stability and convergence of the proposed methods have been proved by the designed Lyapunov function. Simulations validate the convergence and effectiveness of the proposed methods in different games and complex networks.

Nanosecond pulsed electric fields prime mesenchymal stem cells to peptide ghrelin and enhance chondrogenesis and osteochondral defect repair in vivo.

Mesenchymal stem cells (MSCs) are important cell sources in cartilage tissue development and homeostasis, and multiple strategies have been developed to improve MSCs chondrogenic differentiation with an aim of promoting cartilage regeneration. Here we report the effects of combining nanosecond pulsed electric fields (nsPEFs) followed by treatment with ghrelin (a hormone that stimulates release of growth hormone) to regulate chondrogenesis of MSCs. nsPEFs and ghrelin were observed to separately enhance the chondrogenesis of MSCs, and the effects were significantly enhanced when the bioelectric stimulation and hormone were combined, which in turn improved osteochondral tissue repair of these cells within Sprague Dawley rats. We further found that nsPEFs can prime MSCs to be more receptive to subsequent stimuli of differentiation by upregulated Oct4/Nanog and activated JNK signaling pathway. Ghrelin initiated chondrogenic differentiation by activation of ERK1/2 signaling pathway, and RNA-seq results indicated 243 genes were regulated, and JAK-STAT signaling pathway was involved. Interestingly, the sequential order of applying these two stimuli is critical, with nsPEFs pretreatment followed by ghrelin enhanced chondrogenesis of MSCs in vitro and subsequent cartilage regeneration in vivo, but not vice versa. This synergistic prochondrogenic effects provide us new insights and strategies for future cell-based therapies.

Incorporating social payoff into reinforcement learning promotes cooperation.

Reinforcement learning has been demonstrated to be an effective approach to investigate the dynamic of strategy updating and the learning process of agents in game theory. Most studies have shown that Q-learning failed to resolve the problem of cooperation in well-mixed populations or homogeneous networks. To this aim, we investigate the self-regarding Q-learning's effect on cooperation in spatial prisoner's dilemma games by incorporating the social payoff. Here, we redefine the reward term of self-regarding Q-learning by involving the social payoff; that is, the reward is defined as a monotonic function of the individual payoff and the social payoff represented by its neighbors' payoff. Numerical simulations reveal that such a framework can facilitate cooperation remarkably because the social payoff ensures agents learn to cooperate toward socially optimal outcomes. Moreover, we find that self-regarding Q-learning is an innovative rule that ensures cooperators coexist with defectors even at high temptations to defection. The investigation of the emergence and stability of the sublattice-ordered structure shows that such a mechanism tends to generate a checkerboard pattern to increase agents' payoff. Finally, the effects of Q-learning parameters are also analyzed, and the robustness of this mechanism is verified on different networks.

Enhancement of the chondrogenic differentiation of mesenchymal stem cells and cartilage repair by ghrelin.

Transforming growth factor beta (TGF-ß) is commonly utilized in chondrogenic differentiation protocols, but this often results in incomplete maturation of the derived chondrocytes. Gene expression analysis, quantitation of sulfated glycosaminoglycan and collagen, and histological staining were performed to assess the effects of ghrelin. The signaling pathways involved were investigated with inhibitors or targeted by shRNAs. Joint cavity delivery of TGF-ß with or without ghrelin, within a rat cartilage defect model was performed to evaluate the in vivo effects of ghrelin. Ghrelin dramatically enhanced gene expression levels of SOX9, ACAN, and COL II and resulted in increased synthesis of sulfated glycosaminoglycan (sGAG) and collagen in vitro. Combined treatment with TGF-ß and ghrelin synergistically enhanced the phosphorylation of ERK1/2 and DMNT3A, which accounted for increased expression of chondrogenic genes. Delivery of ghrelin in combination with TGF-ß after MSC implantation within a rat osteochondral defect model significantly enhanced de novo cartilage regeneration, as compared to delivery with TGF-ß alone. In conclusion, ghrelin could significantly enhance MSC chondrogenic differentiation in vitro and can also enhance cartilage regeneration in vivo when used in combination with TGF-ß. © 2019 Orthopaedic Research Society. Published by Wiley Periodicals, Inc. J Orthop Res 37:1387-1397, 2019.

A 3D Relative-Motion Context Constraint-Based MAP Solution for Multiple-Object Tracking Problems.

Multi-object tracking (MOT), especially by using a moving monocular camera, is a very challenging task in the field of visual object tracking. To tackle this problem, the traditional tracking-by-detection-based method is heavily dependent on detection results. Occlusion and mis-detections will often lead to tracklets or drifting. In this paper, the tasks of MOT and camera motion estimation are formulated as finding a maximum a posteriori (MAP) solution of joint probability and synchronously solved in a unified framework. To improve performance, we incorporate the three-dimensional (3D) relative-motion model into a sequential Bayesian framework to track multiple objects and the camera's ego-motion estimation. A 3D relative-motion model that describes spatial relations among objects is exploited for predicting object states robustly and recovering objects when occlusion and mis-detections occur. Reversible jump Markov chain Monte Carlo (RJMCMC) particle filtering is applied to solve the posteriori estimation problem. Both quantitative and qualitative experiments with benchmark datasets and video collected on campus were conducted, which confirms that the proposed method is outperformed in many evaluation metrics.