An Improved Approach towards Multi-Agent Pursuit-Evasion Game Decision-Making Using Deep Reinforcement Learning.

A pursuit-evasion game is a classical maneuver confrontation problem in the multi-agent systems (MASs) domain. An online decision technique based on deep reinforcement learning (DRL) was developed in this paper to address the problem of environment sensing and decision-making in pursuit-evasion games. A control-oriented framework developed from the DRL-based multi-agent deep deterministic policy gradient (MADDPG) algorithm was built to implement multi-agent cooperative decision-making to overcome the limitation of the tedious state variables required for the traditionally complicated modeling process. To address the effects of errors between a model and a real scenario, this paper introduces adversarial disturbances. It also proposes a novel adversarial attack trick and adversarial learning MADDPG (A2-MADDPG) algorithm. By introducing an adversarial attack trick for the agents themselves, uncertainties of the real world are modeled, thereby optimizing robust training. During the training process, adversarial learning was incorporated into our algorithm to preprocess the actions of multiple agents, which enabled them to properly respond to uncertain dynamic changes in MASs. Experimental results verified that the proposed approach provides superior performance and effectiveness for pursuers and evaders, and both can learn the corresponding confrontational strategy during training.

Maternal and Neonatal Complications in Patients With Diminished Ovarian Reserve in In-Vitro Fertilization/Intracytoplasmic Sperm Injection Cycles.

Background: Diminished ovarian reserve (DOR) is one of the most intractable clinical issues in human reproduction and is reported to be associated with raised risk of recurrent pregnancy loss and aneuploid blastocysts. In this study, we aimed to explore whether DOR was also associated with maternal and neonatal complications in in-vitro fertilization/intracytoplasmic sperm injection cycles. Methods: A retrospective cohort study including women below 40 years of age who achieved singleton live birth after fresh embryo transfer in in-vitro fertilization/intracytoplasmic sperm injection cycles in a single center from January 2012 to June 2019 was conducted. Participants with DOR, defined as basal follicle-stimulating hormone (FSH) ≥ 10IU/L and antimullerian hormone (AMH) < 1.2ng/ml, were enrolled as the study group. The controls were 1:2 matched by age and body mass index with FSH < 10IU/L and AMH ≥ 1.2ng/ml. Maternal and neonatal complications were compared between the DOR group and the controls. Results: A total of 579 women, 193 in the DOR group and 386 matched as controls, were included in this study. Compared to controls, the incidence of hypertensive disorders of pregnancy was significantly increased in the DOR group (5.7% vs. 2.1%, P = 0.021). DOR patients also presented slightly higher incidences of preterm birth (10.9% vs. 7.5%, P = 0.174) and low birthweight (6.2% vs. 5.4%, P = 0.704) yet without statistical significances. The incidences of gestational diabetes mellitus and placenta previa were comparable between the two groups. Conclusion: Compared to women with normal ovarian reserve, women with diminished ovarian reserve might have elevated incidence of hypertensive disorders of pregnancy. Patients with diminished ovarian reserve might need more strict antenatal care.

Ovarian Reserve Markers in Premature Ovarian Insufficiency: Within Different Clinical Stages and Different Etiologies.

Objective: To characterize the ovarian reserve indicators for premature ovarian insufficiency (POI) at different disease stages and with various etiologies. Methods: According to different FSH levels and menstrual conditions, patients with normal ovarian reserve (NOR with 5 IU/L40 IU/L, n=454) were retrospectively screened and their records were abstracted from Reproductive Hospital Affiliated to Shandong University between 2014 and 2019. Based on the known etiologies, POI patients were subdivided into genetic, iatrogenic, autoimmune and idiopathic subsets according to the known etiologies. The phenotypic features were compared within different subgroups, and the predictive value of ovarian reserve markers was analyzed. Results: The ovarian reserve indicators consecutively deteriorated with the progress of ovarian insufficiency, indicated as an increase of FSH and LH but decrease of AMH, inhibin B, AFC, E2 and T (P<0.01). Most of them changed significantly from NOR to pre-POI while remained relatively stable at a low level or even undetectable at early POI and POF stage. AMH showed the highest predictive value for pre-POI (AUC 0.932, 95% CI 0.918-0.945) and POI (AUC 0.944, 95% CI 0.933-0.954), and the combination of AMH and AFC was highly promising for early prediction. Additionally, significant differences existed in AMH, inhibin B and AFC among women with different etiologies of POI (P<0.05), and the genetic POI presented the worst hormone status. Conclusions: Our study indicated a high heterogeneity of POI in both endocrine hormones and etiological phenotypes. The quantitative changes and cutoff values of AMH and AFC could provide new insights in the prediction and early diagnosis of POI.

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments.

Autonomous motion planning (AMP) of unmanned aerial vehicles (UAVs) is aimed at enabling a UAV to safely fly to the target without human intervention. Recently, several emerging deep reinforcement learning (DRL) methods have been employed to address the AMP problem in some simplified environments, and these methods have yielded good results. This paper proposes a multiple experience pools (MEPs) framework leveraging human expert experiences for DRL to speed up the learning process. Based on the deep deterministic policy gradient (DDPG) algorithm, a MEP-DDPG algorithm was designed using model predictive control and simulated annealing to generate expert experiences. On applying this algorithm to a complex unknown simulation environment constructed based on the parameters of the real UAV, the training experiment results showed that the novel DRL algorithm resulted in a performance improvement exceeding 20% as compared with the state-of-the-art DDPG. The results of the experimental testing indicate that UAVs trained using MEP-DDPG can stably complete a variety of tasks in complex, unknown environments.

Simulation and realization of a second-order quantum-interference-based quantum clock synchronization at the femtosecond level.

Quantum clock synchronization schemes utilizing frequency-entangled pulses have flourished for their potentially superior precision to the classical protocols. In this Letter, a new experimental record based on the second-order quantum interference algorithm is reported, to the best of our knowledge. The synchronization accuracy between two parties separated by a 6 km fiber coiling link, which is evaluated by the time offset shift relative to that with the fibers removed, has been measured to be 13±1 ps. The stability in terms of time deviation (TDEV) of 0.81 ps at an averaging time of 100 s has been achieved. The long-term synchronization stability is seen determined by the measurement device, and a minimum stability of 60 fs has been reached at 25,600 s. Furthermore, for the first time to the best of our knowledge, we quantify the performance of this quantum synchronization scheme, and very good agreements with the experimental results have been achieved. According to the quantum simulation, further improvements for both the synchronizing stability and accuracy can be expected.

Simulation of high SNR photodetector with L-C coupling and transimpedance amplifier circuit and its verification.

In this paper, a model for simulating the optical response and noise performances of photodetectors with L-C coupling and transimpedance amplification circuit is presented. To verify the simulation, two kinds of photodetectors, which are based on the same printed-circuit-board (PCB) designing and PIN photodiode but different operational amplifiers, are developed and experimentally investigated. Through the comparisons between the numerical simulation results and the experimentally obtained data, excellent agreements are achieved, which show that the model provides a highly efficient guide for the development of a high signal to noise ratio photodetector. Furthermore, the parasite capacitances on the developed PCB, which are always hardly measured but play a non-negligible influence on the photodetectors' performances, are estimated.

Demonstration of quantum synchronization based on second-order quantum coherence of entangled photons.

Based on the second-order quantum interference between frequency entangled photons that are generated by parametric down conversion, a quantum strategic algorithm for synchronizing two spatially separated clocks has been recently presented. In the reference frame of a Hong-Ou-Mandel (HOM) interferometer, photon correlations are used to define simultaneous events. Once the HOM interferometer is balanced by use of an adjustable optical delay in one arm, arrival times of simulta- neously generated photons are recorded by each clock. The clock offset is determined by correlation measurement of the recorded arrival times. Utilizing this algorithm, we demonstrate a proof-of-principle experiment for synchronizing two clocks separated by 4 km fiber link. A minimum timing stability of 0.44 ps at averaging time of 16000 s is achieved with an absolute time accuracy of 73.2 ps. The timing stability is verified to be limited by the correlation measurement device and ideally can be better than 10 fs. Such results shine a light to the application of quantum clock synchronization in the real high-accuracy timing system.