On Transforming Reinforcement Learning With Transformers: The Development Trajectory.

Transformers, originally devised for natural language processing (NLP), have also produced significant successes in computer vision (CV). Due to their strong expression power, researchers are investigating ways to deploy transformers for reinforcement learning (RL), and transformer-based models have manifested their potential in representative RL benchmarks. In this paper, we collect and dissect recent advances concerning the transformation of RL with transformers (transformer-based RL (TRL)) to explore the development trajectory and future trends of this field. We group the existing developments into two categories: architecture enhancements and trajectory optimizations, and examine the main applications of TRL in robotic manipulation, text-based games (TBGs), navigation, and autonomous driving. Architecture enhancement methods consider how to apply the powerful transformer structure to RL problems under the traditional RL framework, facilitating more precise modeling of agents and environments compared to traditional deep RL techniques. However, these methods are still limited by the inherent defects of traditional RL algorithms, such as bootstrapping and the "deadly triad". Trajectory optimization methods treat RL problems as sequence modeling problems and train a joint state-action model over entire trajectories under the behavior cloning framework; such approaches are able to extract policies from static datasets and fully use the long-sequence modeling capabilities of transformers. Given these advancements, the limitations and challenges in TRL are reviewed and proposals regarding future research directions are discussed. We hope that this survey can provide a detailed introduction to TRL and motivate future research in this rapidly developing field.

A combination weighting method for debris flow risk assessment based on t-distribution and linear programming optimization algorithm.

Debris flow risk assessment can provide some reference for debris flow prevention and control projects. In risk assessment, researchers often only focus on the impact of objective or subjective indicators. For this purpose, this paper proposed a weight calculation method based on t-distribution and linear programming optimization algorithm (LPOA). Taking 72 mudslides in Beichuan County as an example, this paper used analytic hierarchy process (AHP), entropy weight method (EWM) and variation coefficient method (VCM) to obtain the initial weights. Based on the initial weights, weight intervals with different confidence levels were obtained by t-distribution. Subsequently, the final weights were obtained by LOPA in the 90% confidence interval. Finally, the final weights were used to calculate the risk score for each debris flow, thus delineating the level of risk for each debris flow. The results showed that this paper's method can avoid overemphasizing the importance of a particular indicator compared to EWM and VCM. In contrast, EWM and VCM ignored the effect of debris flow frequency on debris flow risk. The assessment results showed that the 72 debris flows in Beichuan County were mainly dominated by moderate and light risks. Of these, there were 8 high risk debris flows, 24 medium risk debris flows, and 40 light risk debris flows. The excellent triggering conditions provide favorable conditions for the formation of high-risk debris flows. Slightly and moderate risk debris flows are mainly located on both sides of highways and rivers, still posing a minor threat to Beichuan County. The proposed fusion weighting method effectively avoids the limitations of single weight calculating method. Through comparison and data analysis, the rationality of the proposed method is verified, which can provide some reference for combination weighting method and debris flow risk assessment.

Stability analysis of rainfall-induced landslide considering air resistance delay effect and lateral seepage.

Accumulation landslides are prone to occur during the continuous infiltration of heavy rainfall, which seriously threatens the lives and property safety of local residents. In this paper, based on the Green-Ampt (GA) infiltration model, a new slope rainfall infiltration function is derived by combining the effect of air resistance and lateral seepage of saturated zone. Considering that when the soil layer continues to infiltrate after the saturation zone is formed, the air involvement cannot be discharged in time, which delays the infiltration process. Therefore, the influence of air resistance factor in soil pores is added. According to the infiltration characteristics of finite long slope, the lateral seepage of saturated zone is introduced, which makes up for the deficiency that GA model is only applicable to infinite long slope. Finally, based on the seepage characteristics of the previous analysis, the overall shear strength criterion is used to evaluate the stability of the slope. The results show that the safety factor decreases slowly with the increase of size and is inversely correlated with the slope angle and initial moisture content. The time of infiltration at the same depth increases with the increase of size and slope angle, and is inversely correlated with the initial moisture content, but is less affected by rainfall intensity. By comparing with the results of experimental data and other methods, the results of the proposed method are more consistent with the experimental results than other methods.

Tissue-specific accumulation, depuration and histopathological effects of 3,6-dichlorocarbazole and 2,7-dibromocarbazole in adult zebrafish (Danio rerio).

Although polyhalogenated carbazoles have been detected with increasing frequency in aquatic ecosystems, their bioaccumulation in fish and corresponding pathological effects related to bioaccumulation are still unclear. Here, we investigated the tissue-specific accumulation, depuration, and histopathological effects of two typical PHCZs, 3,6-dichlorocarbazole (36-CCZ) and 2,7-dibromocarbazole (27-BCZ), in adult zebrafish at three levels (0, 0.15 µg/L (5 × environmentally relevant level), and 50 µg/L (1/10 LC50). The lowest concentrations of 36-CCZ (1.2 µg/g ww) and 27-BCZ (1.4 µg/g ww) were observed in muscle, and the greatest concentrations of 36-CCZ (3.6 µg/g ww) and 27-BCZ (4 µg/g ww) were detected in intestine among the tested tissues. BCFww of 36-CCZ and 27-BCZ in zebrafish ranged from 172.9 (muscle) to 606.6 (intestine) and 285.2 (muscle) to 987.5 (intestine), respectively, indicating that both 36-CCZ and 27-BCZ have high potential of bioaccumulation in aquatic system. The 0.15 µg/L level of 36-CCZ or 27-BCZ caused lipid accumulation in liver, while 50 µg/L of 36-CCZ or 27-BCZ induced liver lesions such as fibrous septa, cytolysis, and nuclear dissolution. Brain damage such as multinucleated cells and nuclear solidification were only observed at 50 µg/L of 27-BCZ. This study provided valuable information in assessing the health and ecological risks of 36-CCZ and 27-BCZ.

Toxicity Evaluation and Effect-Based Identification of Chlorine Disinfection Products of the Anti-COVID-19 Drug Chloroquine Phosphate.

Antiviral transformation products (TPs) generated during wastewater treatment are an environmental concern, as their discharge, in considerable amounts, into natural waters during a pandemic can pose possible risks to the aquatic environment. Identification of the hazardous TPs generated from antivirals during wastewater treatment is important. Herein, chloroquine phosphate (CQP), which was widely used during the coronavirus disease-19 (COVID-19) pandemic, was selected for research. We investigated the TPs generated from CQP during water chlorination. Zebrafish (Danio rerio) embryos were used to assess the developmental toxicity of CQP after water chlorination, and hazardous TPs were estimated using effect-directed analysis (EDA). Principal component analysis revealed that the developmental toxicity induced by chlorinated samples could be relevant to the formation of some halogenated TPs. Fractionation of the hazardous chlorinated sample, along with the bioassay and chemical analysis, identified halogenated TP387 as the main hazardous TP contributing to the developmental toxicity induced by chlorinated samples. TP387 could also be formed in real wastewater during chlorination in environmentally relevant conditions. This study provides a scientific basis for the further assessment of environmental risks of CQP after water chlorination and describes a method for identifying unknown hazardous TPs generated from pharmaceuticals during wastewater treatment.

Evaluation of degradation performance toward antiviral drug ribavirin using advanced oxidation process and its relations to ecotoxicity evolution.

The rapid spread of coronavirus disease 2019 has increased the consumption of some antiviral drugs, wherein these are discharged into wastewater, posing risks to the ecosystem and human health. Therefore, efforts are being made for the development of advanced oxidation processes (AOPs) to remediate water containing these pharmaceuticals. Here, the toxicity evolution of the antiviral drug ribavirin (RBV) was systematically investigated during its degradation via the UV/TiO2/H2O2 advanced oxidation process. Under optimal conditions, RBV was almost completely eliminated within 20 min, although the mineralization rate was inadequate. Zebrafish embryo testing revealed that the ecotoxicity of the treated RBV solutions increased at some stages and decreased as the reaction time increased, which may be attributed to the formation and decomposition of various transformation products (TPs). Liquid chromatography-mass spectrometry analysis along with density functional theory calculations helped identify possible toxicity increase-causing TPs, and quantitative structure activity relationship prediction revealed that most TPs exhibit higher toxicity than the parent compound. The findings of this study suggest that, in addition to the removal rate of organics, the potential ecotoxicity of treated effluents should also be considered when AOPs are applied in wastewater treatment.

Continuous ultrasonic flow measurement for aerospace small pipelines.

Aerospace explorations stimulate extensive research on innovative propellant flow measurement technologies in microgravity conditions. Ultrasonic-based measurements have advantages of non-invasive and non-moving-component constructions as well as fast responses to bi-directional flow detection, its applications in aerospace explorations have already been reported. To avoid the shortages of pulse ultrasonic measurement configurations, flow measurement of continuous ultrasonic wave propagation is presented to match the requirements of large measurement range and high precision. Fabrication process and laboratory validations using water flow are presented. Ground experiments show that the linearity of the proposed ultrasonic flow meter is obtained in the measurement range [0, 80 ml/s] which is typical requirement in aerospace applications. Meanwhile, the fitted linear feature from the experimental data matches well the theoretical prediction except the flow prediction of stationary fluid. Under specific configurations, the absolute measurement error is significantly affected by the corresponding Reynolds number. Furthermore, the absolute measurement error is smaller when excitation signals with higher frequency are used if the phase tracking performance for different frequencies is identical.

Bioconcentration, depuration and toxicity of Pb in the presence of titanium dioxide nanoparticles in zebrafish larvae.

The interactions between nanoparticles (NPs) and metals in aquatic environments may modify the bioavailability and toxicity of metals to organisms. In this study, we investigated the effects of titanium dioxide NPs (n-TiO2) on the bioconcentration, depuration, and neurotoxic effects of lead (Pb) in zebrafish larvae. Transmission electron microscopy coupled with energy-dispersive X-ray spectroscopy showed that Pb2+ was adsorbed by n-TiO2 to form NP-Pb complexes in suspension, and these complexes were observed in larval tissues. The bioconcentration of Pb in larvae along with the depuration rates of Pb were higher in the presence of n-TiO2 compared to when n-TiO2 was absent. Exposure to Pb alone induced the expression of the biomarker metallothionein, downregulated neurodevelopment-related genes, and reduced swimming activity of larvae. However, the addition of n-TiO2 to the exposure solution alleviated these effects. The results suggest that n-TiO2 can act as a carrier of Pb to increase its bioconcentration; however, the formation of NP-Pb complexes likely reduces the amount of free Pb2+, thereby reducing toxicity to larvae.

Impact of co-exposure to titanium dioxide nanoparticles and Pb on zebrafish embryos.

Complex interactions have been established between nanoparticles (NPs) and heavy metals in real environments. Herein we used zebrafish embryos to investigate the influence of titanium dioxide NPs (n-TiO2) on the uptake, bioconcentration, and depuration, and toxicity of Pb. The formation of n-TiO2-Pb complexes was confirmed in an exposure suspension. An increase in Pb bioconcentration was observed in zebrafish embryos upon co-exposure to n-TiO2 and Pb; moreover, n-TiO2-Pb complexes could be found in the embryos, indicating the bioavailability of NPs. However, there was no difference in the depuration rates of Pb in the presence of n-TiO2. Metallothionein (MT) content was significantly increased upon exposure to Pb alone, and the content significantly increased even further upon co-exposure. A downregulation in the expression levels of the neurodevelopment-related genes gfap, syn2α, and elavl3 was observed in the embryos, and we also noted a reduction in the swimming speed of and the total distance traveled by the larvae. To summarize, our results indicate that n-TiO2 can act as an effective carrier of Pb to enhance its uptake, bioavailability, and toxicity in zebrafish embryos.

The fate and oxidative stress of different sized SiO2 nanoparticles in zebrafish (Danio rerio) larvae.

Nanoparticle such as silicon dioxide nanoparticles (nano-SiO2) are extensively produced and applied in society. Hence there is an increasing concern about their exposure and toxicity to human and wildlife. To understand the effects of sizes of NPs on their bioavailability and toxicity, zebrafish (Danio rerio) embryos (2 h post-fertilization, hpf) were exposed to 25, 50 and 100 mg/L of 15 or 30 nm nano-SiO2 for 5 days respectively. The results showed that SiO2 could be readily uptaken by zebrafish, and the accumulation of SiO2 was significantly higher in 15 nm treatments groups compared to 30 nm nano-SiO2 treated groups. Furthermore, exposure to 15 nm nano-SiO2 at the concentration of 100 mg/L resulted in more significant changes in reactive oxygen species (ROS) levels, perturbation of lipid peroxidative and antioxidant system than the same concentration of 30 nm nano-SiO2, indicating small sized nano-SiO2 evoked severer oxidative stress in zebrafish larvae. In addition, the more significant up-regulation of transcription of genes related to oxidative stress (e.g., nrf2 and sod1) in 15 nm nano-SiO2 at the 100 mg/L treatments groups provided more evidence for this speculation. Given the above, 15 nm nano-SiO2 were more likely to enter and accumulate in zebrafish larvae, thus causing more serious oxidative stress in vivo. These results may provide additional information on the fate and toxicities of different sizes of NPs.

The Forty-Sixth Euro Congress on Drug Synthesis and Analysis: Snapshot †.

The 46th EuroCongress on Drug Synthesis and Analysis (ECDSA-2017) was arranged within the celebration of the 65th Anniversary of the Faculty of Pharmacy at Comenius University in Bratislava, Slovakia from 5-8 September 2017 to get together specialists in medicinal chemistry, organic synthesis, pharmaceutical analysis, screening of bioactive compounds, pharmacology and drug formulations; promote the exchange of scientific results, methods and ideas; and encourage cooperation between researchers from all over the world. The topic of the conference, "Drug Synthesis and Analysis," meant that the symposium welcomed all pharmacists and/or researchers (chemists, analysts, biologists) and students interested in scientific work dealing with investigations of biologically active compounds as potential drugs. The authors of this manuscript were plenary speakers and other participants of the symposium and members of their research teams. The following summary highlights the major points/topics of the meeting.