Design, synthesis and biological activity determination of novel phenylpyrazole protoporphyrinogen oxidase inhibitor herbicides.

Protoporphyrinogen oxidase (PPO, EC 1.3.3.4) is the last common enzyme in the biosynthetic pathway in the synthesis of heme and chlorophyll. The high-frequency use of PPO inhibitor herbicides has led to the gradual exposure of pesticide damage and resistance problems. In order to solve this kind of problem, there is an urgent need to develop new PPO inhibitor herbicides. In this paper, 16 phenylpyrazole derivatives were designed by the principle of active substructure splicing through the electron isosterism of five-membered heterocycles. Greenhouse herbicidal activity experiments and in vitro PPO activity experiments showed that the inhibitory effect of compound 9 on weed growth was comparable to that of pyraflufen-ethyl. Crop safety experiments and cumulative concentration experiments in crops showed that when the spraying concentration was 300 g ai/ha, wheat, corn, rice and other cereal crops were more tolerant to compound 9, among which wheat showed high tolerance, which was comparable to the crop safety of pyraflufen-ethyl. Herbicidal spectrum experiments showed that compound 9 had inhibitory activity against most weeds. Molecular docking results showed that compound 9 formed one hydrogen bond interaction with amino acid residue ARG-98 and two π-π stacking interactions with amino acid residue PHE-392, indicating that compound 9 had better herbicidal activity than pyraflufen-ethyl. It shows that compound 9 is expected to be a lead compound of phenylpyrazole PPO inhibitor herbicide and used as a herbicide in wheat field.

Model-Based Predictive Control and Reinforcement Learning for Planning Vehicle-Parking Trajectories for Vertical Parking Spaces.

This paper proposes a vehicle-parking trajectory planning method that addresses the issues of a long trajectory planning time and difficult training convergence during automatic parking. The process involves two stages: finding a parking space and parking planning. The first stage uses model predictive control (MPC) for trajectory tracking from the initial position of the vehicle to the starting point of the parking operation. The second stage employs the proximal policy optimization (PPO) algorithm to transform the parking behavior into a reinforcement learning process. A four-dimensional reward function is set to evaluate the strategy based on a formal reward, guiding the adjustment of neural network parameters and reducing the exploration of invalid actions. Finally, a simulation environment is built for the parking scene, and a network framework is designed. The proposed method is compared with the deep deterministic policy gradient and double-delay deep deterministic policy gradient algorithms in the same scene. Results confirm that the MPC controller accurately performs trajectory-tracking control with minimal steering wheel angle changes and smooth, continuous movement. The PPO-based reinforcement learning method achieves shorter learning times, totaling only 30% and 37.5% of the deep deterministic policy gradient (DDPG) and twin-delayed deep deterministic policy gradient (TD3), and the number of iterations to reach convergence for the PPO algorithm with the introduction of the four-dimensional evaluation metrics is 75% and 68% shorter compared to the DDPG and TD3 algorithms, respectively. This study demonstrates the effectiveness of the proposed method in addressing a slow convergence and long training times in parking trajectory planning, improving parking timeliness.

Multivariable Coupled System Control Method Based on Deep Reinforcement Learning.

Due to the multi-loop coupling characteristics of multivariable systems, it is difficult for traditional control methods to achieve precise control effects. Therefore, this paper proposes a control method based on deep reinforcement learning to achieve stable and accurate control of multivariable coupling systems. Based on the proximal policy optimization algorithm (PPO), this method selects tanh as the activation function and normalizes the advantage function. At the same time, based on the characteristics of the multivariable coupling system, the reward function and controller are redesigned structures, achieving stable and precise control of the controlled system. In addition, this study used the amplitude of the control quantity output by the controller as an indicator to evaluate the controller's performance. Finally, simulation verification was conducted in MATLAB/Simulink. The experimental results show that compared with decentralized control, decoupled control and traditional PPO control, the method proposed in this article achieves better control effects.

An Overview of Factors Affecting the Functional Quality of Common Wheat (Triticum aestivum L.).

Wheat (Triticum aestivum L.) is one of the most important crops worldwide, and, as a resilient cereal, it grows in various climatic zones. Due to changing climatic conditions and naturally occurring environmental fluctuations, the priority problem in the cultivation of wheat is to improve the quality of the crop. Biotic and abiotic stressors are known factors leading to the deterioration of wheat grain quality and to crop yield reduction. The current state of knowledge on wheat genetics shows significant progress in the analysis of gluten, starch, and lipid genes responsible for the synthesis of the main nutrients in the endosperm of common wheat grain. By identifying these genes through transcriptomics, proteomics, and metabolomics studies, we influence the creation of high-quality wheat. In this review, previous works were assessed to investigate the significance of genes, puroindolines, starches, lipids, and the impact of environmental factors, as well as their effects on the wheat grain quality.

Enzymatic Activity and Its Relationships with the Total Phenolic Content and Color Change in the High Hydrostatic Pressure-Assisted Curing of Vanilla Bean (Vanilla planifolia).

Diverse enzymatic reactions taking place after the killing of green vanilla beans are involved in the flavor and color development of the cured beans. The effects of high hydrostatic pressure (HHP) at 50-400 MPa/5 min and blanching as vanilla killing methods were evaluated on the total phenolic content (TPC), polyphenoloxidase (PPO), and peroxidase (POD) activity and the color change at different curing cycles of sweating-drying (C0-C20) of vanilla beans. The rate constants describing the above parameters during the curing cycles were also obtained. The TPC increased from C1 to C6 compared with the untreated green beans after which it started to decrease. The 400 MPa samples showed the highest rate of phenolic increase. Immediately after the killing (C0), the highest increase in PPO activity was observed at 50 MPa (46%), whereas for POD it was at 400 MPa (25%). Both enzymes showed the maximum activity at C1, after which the activity started to decrease. As expected, the L* color parameter decreased during the entire curing for all treatments. An inverse relationship between the rate of TPC decrease and enzymatic activity loss was found, but the relationship with L* was unclear. HHP appears to be an alternative vanilla killing method; nevertheless, more studies are needed to establish its clear advantages over blanching.

A comprehensive review of polyphenol oxidase in tea (Camellia sinensis): Physiological characteristics, oxidation manufacturing, and biosynthesis of functional constituents.

Polyphenol oxidase (PPO) is a metalloenzyme with a type III copper core that is abundant in nature. As one of the most essential enzymes in the tea plant (Camellia sinensis), the further regulation of PPO is critical for enhancing defensive responses, cultivating high-quality germplasm resources of tea plants, and producing tea products that are both functional and sensory qualities. Due to their physiological and pharmacological values, the constituents from the oxidative polymerization of PPO in tea manufacturing may serve as functional foods to prevent and treat chronic non-communicable diseases. However, current knowledge of the utilization of PPO in the tea industry is only available from scattered sources, and a more comprehensive study is required to reveal the relationship between PPO and tea obviously. A more comprehensive review of the role of PPO in tea was reported for the first time, as its classification, catalytic mechanism, and utilization in modulating tea flavors, compositions, and nutrition, along with the relationships between PPO-mediated enzymatic reactions and the formation of functional constituents in tea, and the techniques for the modification and application of PPO based on modern enzymology and synthetic biology are summarized and suggested in this article.

Discovery of novel phenoxypyridine as promising protoporphyrinogen IX oxidase inhibitors.

Protoporphyrinogen oxidase (PPO, EC 1.3.3.4) is a significant target for the discovery of novel bleaching herbicides. Starting from the active fragments of several known commercial herbicides, a series of PPO inhibitors with diphenyl ether scaffolds were designed and synthesized by substructure splicing and bioisosterism methods. The greenhouse herbicidal activity and the PPO inhibitory activity in vitro were measured. The results showed that the novel synthesized compounds have good PPO inhibitory activity, and the IC50 value against corn PPO ranges from 0.032 ± 0.008 mg/L to 3.245 ± 0.247 mg/L. Among all target compounds, compound P2 showed the best herbicidal activity, with a half inhibitory concentration (IC50) of 0.032 ± 0.008 mg/L. In addition, the molecular docking results showed that the benzene ring part of compound P2 can form a π-π stacking with PHE-392, and the trifluoromethyl group and ARG-98 form two hydrogen bonds. Crop safety experiments and cumulative concentration analysis experiments indicated that compound P2 can be used for weed control in rice, wheat, soybean and corn. Therefore, compound P2 can be selected to develop potential lead compounds for novel PPO inhibitors.

Intelligent Smart Marine Autonomous Surface Ship Decision System Based on Improved PPO Algorithm.

With the development of artificial intelligence technology, the behavior decision-making of an intelligent smart marine autonomous surface ship (SMASS) has become particularly important. This research proposed local path planning and a behavior decision-making approach based on improved Proximal Policy Optimization (PPO), which could drive an unmanned SMASS to the target without requiring any human experiences. In addition, a generalized advantage estimation was added to the loss function of the PPO algorithm, which allowed baselines in PPO algorithms to be self-adjusted. At first, the SMASS was modeled with the Nomoto model in a simulation waterway. Then, distances, obstacles, and prohibited areas were regularized as rewards or punishments, which were used to judge the performance and manipulation decisions of the vessel Subsequently, improved PPO was introduced to learn the action-reward model, and the neural network model after training was used to manipulate the SMASS's movement. To achieve higher reward values, the SMASS could find an appropriate path or navigation strategy by itself. After a sufficient number of rounds of training, a convincing path and manipulation strategies would likely be produced. Compared with the proposed approach of the existing methods, this approach is more effective in self-learning and continuous optimization and thus closer to human manipulation.

Proactive Handover Decision for UAVs with Deep Reinforcement Learning.

The applications of Unmanned Aerial Vehicles (UAVs) are rapidly growing in domains such as surveillance, logistics, and entertainment and require continuous connectivity with cellular networks to ensure their seamless operations. However, handover policies in current cellular networks are primarily designed for ground users, and thus are not appropriate for UAVs due to frequent fluctuations of signal strength in the air. This paper presents a novel handover decision scheme deploying Deep Reinforcement Learning (DRL) to prevent unnecessary handovers while maintaining stable connectivity. The proposed DRL framework takes the UAV state as an input for a proximal policy optimization algorithm and develops a Received Signal Strength Indicator (RSSI) based on a reward function for the online learning of UAV handover decisions. The proposed scheme is evaluated in a 3D-emulated UAV mobility environment where it reduces up to 76 and 73% of unnecessary handovers compared to greedy and Q-learning-based UAV handover decision schemes, respectively. Furthermore, this scheme ensures reliable communication with the UAV by maintaining the RSSI above -75 dBm more than 80% of the time.

Efficient Asynchronous Federated Learning for AUV Swarm.

The development of automatic underwater vehicles (AUVs) has brought about unprecedented profits and opportunities. In order to discover the hidden valuable data detected by an AUV swarm, it is necessary to aggregate the data detected by AUV swarm to generate a powerful machine learning model. Traditional centralized machine learning generates a large number of data exchanges and faces problems of enormous training data, large-scale models, and communication. In underwater environments, radio waves are strongly absorbed, and acoustic communication is the only feasible technology. Unlike electromagnetic wave communication on land, the bandwidth of underwater acoustic communication is extremely limited, with the transmission rate being only 1/105 of the electromagnetic wave. Therefore, traditional centralized machine learning cannot support underwater AUV swarm training. In recent years, federated learning could only interact with model parameters without interacting with data, which greatly reduced communication costs. Therefore, this paper introduces federated learning into the collaboration of an AUV swarm. In order to further reduce the constraints of underwater scarce communication resources on federated learning and alleviate the straggler effect, in this work, we designed an asynchronous federated learning method. Finally, we constructed the optimization problem of minimizing the weighted sum of delay and energy consumption, relying on jointly optimizing the AUV CPU frequency and signal transmission power. In order to solve this complex optimization problem of high-dimensional non-convex time series accumulation, we transformed the problem into a Markov decision process (MDP) and use the proximal policy optimization 2 (PPO2) algorithm to solve this problem. The simulation results demonstrate the effectiveness and superiority of our method.

Polyphenol Oxidase as a Promising Alternative Therapeutic Agent for Cancer Therapy.

Cancers have always been the most difficult to fight, the treatment of cancer is still not considered. Thus, exploring new anticancer drugs is still imminent. Traditional Chinese medicine has played an important role in the treatment of cancer. Polyphenol oxidase (PPO) extracted from Edible mushroom has many related reports on its characteristics, but its role in cancer treatment is still unclear. This study aims to investigate the effects of PPO extracted from Edible mushroom on the proliferation, migration, invasion, and apoptosis of cancer cells in vitro and explore the therapeutic effects of PPO on tumors in vivo. A cell counting kit-8 (CCK8) assay was used to detect the effect of PPO on the proliferation of cancer cells. The effect of PPO on cancer cell migration ability was detected by scratch test. The effect of PPO on the invasion ability of cancer cells was detected by a transwell assay. The effect of PPO on the apoptosis of cancer cells was detected by flow cytometry. Female BALB/c mice (18-25 g, 6-8 weeks) were used for in vivo experiments. The experiments were divided into control group, model group, low-dose group (25 mg/kg), and high-dose group (50 mg/kg). In vitro, PPO extracted from Edible mushroom significantly inhibited the proliferation, migration, and invasion capability of breast cancer cell 4T1, lung cancer cell A549, and prostate cancer cell C4-2, and significantly promoted the apoptosis of 4T1, A549, and C4-2. In vivo experiments showed PPO inhibitory effect on tumor growth. Collectively, the edible fungus extract PPO could play an effective role in treating various cancers, and it may potentially be a promising agent for treating cancers.

Induction of Biosynthesis Antioxidant Molecules in Young Barley Plants by Trioxygen.

Young barley plants are a good source of bioactive compounds. This paper presents the effects of gaseous O3 (trioxygen or ozone) on the biosynthesis of compounds, determining the antioxidant potential of young barley plants. The total content of polyphenols was determined along with their profile, as well as total antioxidant potential and vitamin C content. The highest contents of these compounds were identified in young barley plants exposed to gaseous O3. The main bioactive compound, representing polyphenols, determined in the examined raw materials was saponarin (isovitexin 7-O-glucoside). The induction of increased biosynthesis of these molecules was directly linked to the modification of the activity of selected enzymes. The increased polyphenol content resulted from the modified activities of polyphenol oxidase (PPO) and phenylalanine ammonia lyase (PAL). On the other hand, the oxidative effect of ozone on barley plants was reduced, owing to the modified activities of catalases (CAT), glutathione peroxidases (SOD) and guaiacol peroxidase (GPOX). Analysis of the results showed that by applying gaseous O3 at a dose of 50 ppm for 10 min, the contents of bioactive compounds can be maximised in a residue-free way by activating oxidative stress defence mechanisms.

Genome-wide investigation and expression profiling of polyphenol oxidase (PPO) family genes uncover likely functions in organ development and stress responses in Populus trichocarpa.

BACKGROUND: Trees such as Populus are planted extensively for reforestation and afforestation. However, their successful establishment greatly depends upon ambient environmental conditions and their relative resistance to abiotic and biotic stresses. Polyphenol oxidase (PPO) is a ubiquitous metalloproteinase in plants, which plays crucial roles in mediating plant resistance against biotic and abiotic stresses. Although the whole genome sequence of Populus trichocarpa has long been published, little is known about the PPO genes in Populus, especially those related to drought stress, mechanical damage, and insect feeding. Additionally, there is a paucity of information regarding hormonal responses at the whole genome level. RESULTS: A genome-wide analysis of the poplar PPO family was performed in the present study, and 18 PtrPPO genes were identified. Bioinformatics and qRT-PCR were then used to analyze the gene structure, phylogeny, chromosomal localization, gene replication, cis-elements, and expression patterns of PtrPPOs. Sequence analysis revealed that two-thirds of the PtrPPO genes lacked intronic sequences. Phylogenetic analysis showed that all PPO genes were categorized into 11 groups, and woody plants harbored many PPO genes. Eighteen PtrPPO genes were disproportionally localized on 19 chromosomes, and 3 pairs of segmented replication genes and 4 tandem repeat genomes were detected in poplars. Cis-acting element analysis identified numerous growth and developmental elements, secondary metabolism processes, and stress-related elements in the promoters of different PPO members. Furthermore, PtrPPO genes were expressed preferentially in the tissues and fruits of young plants. In addition, the expression of some PtrPPOs could be significantly induced by polyethylene glycol, abscisic acid, and methyl jasmonate, thereby revealing their potential role in regulating the stress response. Currently, we identified potential upstream TFs of PtrPPOs using bioinformatics. CONCLUSIONS: Comprehensive analysis is helpful for selecting candidate PPO genes for follow-up studies on biological function, and progress in understanding the molecular genetic basis of stress resistance in forest trees might lead to the development of genetic resources.

Design and synthesis of acrylate and acrylamide substituted pyrimidinediones as potential PPO herbicides.

PPO herbicides emerge to be widely use in the agricultural field and a focus of research to many scientists due to its environmentally-friendly properties. In lieu with this, this study presents acrylate and acrylamide substituted pyrimidinediones as PPO herbicide candidates. Most synthesized compounds exhibits herbicidal activities against both monocot and dicot weeds, especially, compound 5a which showed non-selective superior activity against the commercialized, Saflufenacil. Compound 5a was further tested for residual effect and showed promising results as shorter period is needed to cultivate the next crops. The synthesized acrylate and acrylamide substituted pyrimidinediones, especially, 5a could potentially be utilized in the development of commercial protoporphyrinogen oxidase inhibitors with further tests and studies.

Phenoxypyridine derivatives containing natural product coumarins with allelopathy as novel and promising proporphyrin IX oxidase-inhibiting herbicides: Design, synthesis and biological activity study.

To seek novel and safe protoporphyrinogen oxidase (PPO, EC 1.3.3.4) inhibitors with excellent herbicidal activity. A series of novel phenoxypyridine derivatives containing natural product coumarins with allelopathy were designed and synthesized based on bioisosterism and active subunit combination in this research. Compounds W3.1 and W3.4, with the half-maximal inhibitory concentration (IC50) value of 0.02653 mg/L and 0.01937 mg/L, respectively, displayed excellent herbicidal activity in greenhouse. Their herbicidal activity was similar to commercial herbicide oxyfluorfen (IC50 = 0.04943 mg/L). The best field inhibitory effect of compounds W3.1 and W3.4 recorded was at doses of 450 g ai/ha and 300 g ai/ha, respectively. Compound W3.4 had the best herbicidal activity among all the target compounds in this paper. Molecular docking analysis revealed that compounds W3.1 and W3.4 could form a hydrogen bonds with the amino acid AGR-98 and a π-π superposition with the amino acid PHE-398, respectively, which was similar to the oxyfluorfen. The crop selectivity tests results indicated that maize, cotton and soybean showed high tolerance to compound W3.4. Compound W3.4 reduced the Ca and Cb contents of wheat and rice, but had less effect on maize, cotton and soybean. Selectivity of compound W3.4 in maize, cotton and soybean were appeared to be due to reduced absorption of the herbicide compared to wheat and rice. Compound W3.4 deserves further attention as a candidate structure for new herbicides.

In-depth genetic analysis reveals conditioning of polyphenol oxidase activity in wheat grains by cis regulation of TaPPO2A-1 expression level.

Time-dependent darkening and discoloration of wheat product caused by high polyphenol oxidase enzymes (PPO) activity is the most undesirable character in wheat processing industry. We performed GWAS of PPO activity in wheat grains utilizing an association panel and identified 22 significant SNPs. The most significant GWAS peak on chromosome 2A was verified by QTL analysis of PPO activity. The candidate gene for this GWAS peak was identified as TaPPO2A-1, which was the highest expressed PPO gene in wheat grains. The expression level of TaPPO2A-1 was significantly correlated with PPO activity. The most significant association signal for GWAS of the expression values of TaPPO2A-1 pinpointed to the genomic region containing TaPPO2A-1. The results suggested that cis regulation of TaPPO2A-1 expression is the key factor in regulation of PPO activity in wheat grains. The conclusion was further enhanced by haplotype analysis of seven SNPs in the promoter of TaPPO2A-1.

Morphophysiological characteristics of Brazilian bean genotypes related with sulfentrazone tolerance.

In the bean crop there are limited options of herbicides to control eudicotyledons weeds in pre-emergence and sulfentrazone presents potential to be used in a weed management program. The aim of this study was to determine the tolerance of a large number of Brazilian bean genotypes to sulfentrazone and their relation with morphophysiological characteristics, associating it to market classes, seed size and their center of origin. It was evaluated the effect of sulfentrazone application (400 g a.i. ha-1) in pre-emergence of 40 Brazilian bean genotypes, 36 of which were common-bean (Phaseolus vulgaris), two cowpeas (Vigna unguiculata), one adzuki bean (Vigna angularis) and one mung bean (Vigna radiata). There was high tolerance variation of the genotypes to sulfentrazone. Morphological parameters were strongly related with relative tolerance. Common bean genotypes that had medium to large seed size (cranberry and carioca) were more tolerant, while small-sized seed genotypes were more sensitive (black). Andean genotypes were more tolerant than Mesoamerican genotypes. Within the Mesoamerican group, medium-sized seed genotypes (carioca beans) were more tolerant than small-sized seed (black beans). Considering only the Vigna genera, cowpea was highly tolerant to sulfentrazone, while mung bean showed intermediary response and adzuki bean was completely sensible.

MicroRNA528, a hub regulator modulating ROS homeostasis via targeting of a diverse set of genes encoding copper-containing proteins in monocots.

Plant microRNAs (miRNAs) regulate vital cellular processes, including responses to extreme temperatures with which reactive oxygen species (ROS) are often closely associated. In the present study, it was found that aberrant temperatures caused extensive changes in abundance to numerous miRNAs in banana fruit, especially the copper (Cu)-associated miRNAs. Among them, miR528 was significantly downregulated under cold stress and it was found to target genes encoding polyphenol oxidase (PPO), different from those identified in rice and maize. Expression of PPO genes was upregulated by > 100-fold in cold conditions, leading to ROS surge and subsequent peel browning of banana fruit. Extensive comparative genomic analyses revealed that the monocot-specific miR528 can potentially target a large collection of genes encoding Cu-containing proteins. Most of them are actively involved in cellular ROS metabolism, including not only ROS generating oxidases, but also ROS scavenging enzymes. It also was demonstrated that miR528 has evolved a distinct preference of target genes in different monocots, with its target site varying in position among/within gene families, implying a highly dynamic process of target gene diversification. Its broad capacity to target genes encoding Cu-containing protein implicates miR528 as a key regulator for modulating the cellular ROS homeostasis in monocots.

Novel phenoxy-(trifluoromethyl)pyridine-2-pyrrolidinone-based inhibitors of protoporphyrinogen oxidase: Design, synthesis, and herbicidal activity.

As important chemical pesticides, protoporphyrinogen oxidase (PPO, EC 1.3.3.4) herbicides play a vital role in weed management. Herein, in a search for novel PPO herbicides, a series of phenoxypyridine-2-pyrrolidinone derivatives were synthesized and their herbicidal activities were tested. To confirm the structures of the newly synthesized compounds, a colorless single crystal of compound 9d was obtained and crystallographic data collected. PPO inhibition experiments showed that most compounds have PPO inhibitory effects. The half-maximal inhibitory concentration (IC50) of compound 9d and oxyfluorfen were 0.041 mg/L and 0.043 mg/L, respectively, which showed compound 9d was the most potent compound. Compound 9d reduced the Chlorophyll a (Chl a) and Chlorophyll b (Chl b) contents of Abutilon theophrasti (A. theophrasti), to 0.306 and 0.217 mg/g, respectively. Crop selectivity experiments and field trial indicated that compound 9d can potentially be used to develop post-emergence herbicides for weed control in rice, cotton, and peanut. Molecular docking studies showed that both oxyfluorfen and compound 9d can enter the PPO cavity to occupy the active site and compete with the porphyrin to block the chlorophyll synthesis process, affect photosynthesis, and eventually cause weed death. Compound 9d was found to be a promising lead compound for novel herbicide development.

Federated Reinforcement Learning for Training Control Policies on Multiple IoT Devices.

Reinforcement learning has recently been studied in various fields and also used to optimally control IoT devices supporting the expansion of Internet connection beyond the usual standard devices. In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own IoT devices of the same type but with slightly different dynamics. For such multiple IoT devices, there is no guarantee that an agent who interacts only with one IoT device and learns the optimal control policy will also control another IoT device well. Therefore, we may need to apply independent reinforcement learning to each IoT device individually, which requires a costly or time-consuming effort. To solve this problem, we propose a new federated reinforcement learning architecture where each agent working on its independent IoT device shares their learning experience (i.e., the gradient of loss function) with each other, and transfers a mature policy model parameters into other agents. They accelerate its learning process by using mature parameters. We incorporate the actor-critic proximal policy optimization (Actor-Critic PPO) algorithm into each agent in the proposed collaborative architecture and propose an efficient procedure for the gradient sharing and the model transfer. Using multiple rotary inverted pendulum devices interconnected via a network switch, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple IoT devices and that the learning speed can be faster if more agents are involved.