An off-policy multi-agent stochastic policy gradient algorithm for cooperative continuous control.

Multi-agent reinforcement learning (MARL) algorithms based on trust regions (TR) have achieved significant success in numerous cooperative multi-agent tasks. These algorithms restrain the Kullback-Leibler (KL) divergence (i.e., TR constraint) between the current and new policies to avoid aggressive update steps and improve learning performance. However, the majority of existing TR-based MARL algorithms are on-policy, meaning that they require new data sampled by current policies for training and cannot utilize off-policy (or historical) data, leading to low sample efficiency. This study aims to enhance the data efficiency of TR-based learning methods. To achieve this, an approximation of the original objective function is designed. In addition, it is proven that as long as the update size of the policy (measured by the KL divergence) is restricted, optimizing the designed objective function using historical data can guarantee the monotonic improvement of the original target. Building on the designed objective, a practical off-policy multi-agent stochastic policy gradient algorithm is proposed within the framework of centralized training with decentralized execution (CTDE). Additionally, policy entropy is integrated into the reward to promote exploration, and consequently, improve stability. Comprehensive experiments are conducted on a representative benchmark for multi-agent MuJoCo (MAMuJoCo), which offers a range of challenging tasks in cooperative continuous multi-agent control. The results demonstrate that the proposed algorithm outperforms all other existing algorithms by a significant margin.

Purification, crystallization and preliminary crystallographic analysis of CYP 195A2, a P450 enzyme from Rhodopseudomonas palustris.

Cytochrome P450 monooxygenases are a superfamily of heme-thiolate proteins involved in the metabolism of a wide variety of endogenous and xenobiotic compounds. The P450 enzyme CYP195A2 from Rhodopseudomonas palustris CGA009, a metabolically versatile bacterium, was overproduced in E. coli and purified. Two distinct crystal forms were obtained under separately optimized conditions by the hanging-drop vapor-diffusion method. Native data sets extending to resolutions of 2.3 A and 2.8 A have been collected and processed in space groups P222 and C2221 respectively.

Purification, crystallization and preliminary crystallographic analysis of cytochrome P450 203A1 from Rhodopseudomonas palustris.

Cytochrome P450 enzymes constitute a large family of haemoproteins that catalyze the monooxygenation of a great variety of endogenous and exogenous organic compounds. Cytochrome P450 203A1 (CYP203A1, RPA1009) from the metabolically versatile organism Rhodopseudomonas palustris binds a broad range of substrates, in particular substituted aromatic compounds. Crystals of CYP203A1 suitable for X-ray crystallography have been obtained and diffraction data were collected in-house to 2.0 A resolution from a single crystal. The crystals belong to space group P222, with unit-cell parameters a = 40.1, b = 95.1, c = 99.0 A, alpha = beta = gamma = 90 degrees. There is one protein molecule per asymmetric unit.