Coordination as inference in multi-agent reinforcement learning.

The Centralized Training and Decentralized Execution (CTDE) paradigm, where a centralized critic is allowed to access global information during the training phase while maintaining the learned policies executed with only local information in a decentralized way, has achieved great progress in recent years. Despite the progress, CTDE may suffer from the issue of Centralized-Decentralized Mismatch (CDM): the suboptimality of one agent's policy can exacerbate policy learning of other agents through the centralized joint critic. In contrast to centralized learning, the cooperative model that most closely resembles the way humans cooperate in nature is fully decentralized, i.e. Independent Learning (IL). However, there are still two issues that need to be addressed before agents coordinate through IL: (1) how agents are aware of the presence of other agents, and (2) how to coordinate with other agents to improve joint policy under IL. In this paper, we propose an inference-based coordinated MARL method: Deep Motor System (DMS). DMS first presents the idea of individual intention inference where agents are allowed to disentangle other agents from their environment. Secondly, causal inference was introduced to enhance coordination by reasoning each agent's effect on others' behavior. The proposed model was extensively experimented on a series of Multi-Agent MuJoCo and StarCraftII tasks. Results show that the proposed method outperforms independent learning algorithms and the coordination behavior among agents can be learned even without the CTDE paradigm compared to the state-of-the-art baselines including IPPO and HAPPO.

Effects of different straw biochars on soil organic carbon, nitrogen, available phosphorus, and enzyme activity in paddy soil.

Biochar is widely used as a soil amendment. Enzyme activity is an important factor that reflects soil metabolic activity, and is involved in biochemical processes such as organic matter decomposition and nutrient cycling in soils. However, the effects of biochar prepared for different straw materials on soil enzyme activity and soil nutrients are rarely studied. Through pot experiments, the effects of different straw (wheat, rice, maize) biochars (obtained by pyrolysis at 500 °C) on soil organic carbon, nitrogen, available phosphorus, and enzyme activity were studied in paddy soil. The results showed that the addition of biochar increased the soil organic carbon content, which gradually decreased with the extension of the rice growth period. The soil ammonium nitrogen content gradually decreased as the rice growth period continued; however, the soil nitrate nitrogen content first decreased and then increased over the rice growth period. Soil invertase, phosphatase, and urease activity first increased and then decreased, and the enzyme activity was the highest at the heading stage of rice. At this time, there were also significant correlations between enzyme activity and carbon, nitrogen, and phosphorus levels, except in the case of soil urease activity. The geometric mean of the investigated enzyme activities was the highest after amendment with rice straw biochar. These results indicate that the response of enzyme activity to biochar depends on the biochar feedstock and the rice growth stage.

An outbreak of type π vaccine-derived poliovirus in Sichuan province, China: emergence and circulation in an under-immunized population.

BACKGROUND: During August 2011-February 2012, an outbreak of type Π circulating vaccine-derived poliovirus (cVDPVs) occurred in Sichuan Province, China. METHODS: A field investigation of the outbreak was conducted to characterize outbreak isolates and to guide emergency response. Sequence analysis of poliovirus capsid protein VP1 was performed to determine the viral propagation, and a coverage survey was carried out for risk assessment. RESULTS: One clinical compatible polio case and three VDPV cases were determined in Ngawa County, Ngawa Tibetan and Qiang Autonomous Prefecture, Sichuan Province. Case patients were unimmunized children, 0.8-1 years old. Genetic sequencing showed that the isolates diverged from the VP1 region of the type Π Sabin strain by 5-12 nucleotides (nt) and shared the same 5 nt VP1 substitutions, which indicate single lineage of cVDPVs. Of the 7 acute flaccid paralysis cases (all>6 months) reported in Ngawa Prefecture in 2011, 4 (57.1%) cases (including 2 polio cases) did not receive oral attenuated poliovirus vaccine. Supplementary immunization activities (SIAs) were conducted in February-May, 2012, and the strain has not been isolated since. CONCLUSION: High coverage of routine immunization should be maintained among children until WPV transmission is globally eradicated. Risk assessments should be conducted regularly to pinpoint high risk areas or subpopulations, with SIAs developed if necessary.