Association between monocyte to lymphocyte ratio and diabetic foot ulcer in the population of the US with diabetes based on the 1999-2004 National Health and Nutrition Examination Survey data: a retrospective cross-sectional study.

Background: Diabetic foot ulcer (DFU) is a severe complication that occurs in patients with diabetes and is a primary factor that necessitates amputation. Therefore, the occurrence and progression of DFU must be predicted at an early stage to improve patient prognosis and outcomes. In this regard, emerging evidence suggests that inflammation-related markers play a significant role in DFU. One such potential marker, the monocyte-lymphocyte ratio (MLR), has not been extensively studied in relation to DFU. This study aimed to define a connection between MLR and DFU. Methods: A cross-sectional study was conducted using National Health and Nutrition Examination Survey (NHANES) data from 1999 to 2004. DFU was defined based on survey questionnaires assessing the presence of nonhealing ulcers in the lower extremities for more than 4 weeks in diabetes patients. The MLR was calculated as the ratio of the monocyte count to the lymphocyte count, which was directly obtained from laboratory data files. Logistic regression analysis was performed to assess the relationship between the MLR and DFU. Stratified analysis according to age, sex, body mass index, blood glucose, hemoglobin, and glycated hemoglobin categories was conducted, and multiple imputations were applied to missing data. Results: In total, 1246 participants were included; the prevalence of DFU was 9.4% (117/1246). A multivariable regression model revealed a significant association between DFU and a 0.1 unit increase in MLR after adjusting for all covariates (adjusted odds ratio=1.16, 95% confidence interval: 1.02-1.33). Subgroup analyses revealed consistent findings regarding the impact of MLR on the presence of DFU (p > 0.05). Conclusion: MLR is significantly associated with DFU in diabetes patients, and can be used as one of the indicators for predicting the occurrence of DFU. MLR assessment may be a valuable component in the follow-up of patients with diabetes.

Syringic acid attenuates acute lung injury by modulating macrophage polarization in LPS-induced mice.

BACKGROUND: Acute lung injury (ALI) is a continuum of lung changes caused by multiple lung injuries, characterized by a syndrome of uncontrolled systemic inflammation that often leads to significant morbidity and death. Anti-inflammatory is one of its treatment methods, but there is no safe and available drug therapy. Syringic acid (SA) is a natural organic compound commonly found in a variety of plants, especially in certain woody plants and fruits. In modern pharmacological studies, SA has anti-inflammatory effects and therefore may be a potentially safe and available compound for the treatment of acute lung injury. PURPOSE: This study attempts to reveal the protective mechanism of SA against ALI by affecting the polarization of macrophages and the activation of NF-κB signaling pathway. Trying to find a safer and more effective drug therapy for clinical use. METHODS: We constructed the ALI model using C57BL/6 mice by intratracheal instillation of LPS (10 mg/kg). Histological analysis was performed with hematoxylin and eosin (H&E). The wet-dry ratio of the whole lung was measured to evaluate pulmonary edema. The effect of SA on macrophage M1-type was detected by flow cytometry. BCA protein quantification method was used to determine the total protein concentration in bronchoalveolar lavage fluid (BALF). The levels of Interleukin (IL)-6, IL-1ß, and tumor necrosis factor (TNF)-α in BALF were determined by the ELISA kits, and RT-qPCR was used to detect the expression levels of IL-6, IL-1ß and TNF-α mRNA of lung tissue. Western blot was used to detect the expression levels of iNOS and COX-2 and the phosphorylation of p65 and IκBα in the NF-κB pathway in lung tissue. In vitro experiments were conducted with RAW267.4 cell inflammation model induced by 100 ng/ml LPS and A549 cell inflammation model induced by 10 µg/ml LPS. The effects of SA on M1-type and M2-type macrophages of RAW267.4 macrophages induced by LPS were detected by flow cytometry. The toxicity of compound SA to A549 cells was detected by MTT method which to determine the safe dose of SA. The expressions of COX-2 and the phosphorylation of p65 and IκBα protein in NF-κB pathway were detected by Western blot. RESULTS: We found that the pre-treatment of SA significantly reduced the degree of lung injury, and the infiltration of neutrophils in the lung interstitium and alveolar space of the lung. The formation of transparent membrane in lung tissue and thickening of alveolar septum were significantly reduced compared with the model group, and the wet-dry ratio of the lung was also reduced. ELISA and RT-qPCR results showed that SA could significantly inhibit the production of IL-6, IL-1ß, TNF-α. At the same time, SA could significantly inhibit the expression of iNOS and COX-2 proteins, and could inhibit the phosphorylation of p65 and IκBα proteins. in a dose-dependent manner. In vitro experiments, we found that flow cytometry showed that SA could significantly inhibit the polarization of macrophages from M0 type macrophages to M1-type macrophages, while SA could promote the polarization of M1-type macrophages to M2-type macrophages. The results of MTT assay showed that SA had no obvious cytotoxicity to A549 cells when the concentration was not higher than 80 µM, while LPS could promote the proliferation of A549 cells. In the study of anti-inflammatory effect, SA can significantly inhibit the expression of COX-2 and the phosphorylation of p65 and IκBα proteins in LPS-induced A549 cells. CONCLUSION: SA has possessed a crucial anti-ALI role in LPS-induced mice. The mechanism was elucidated, suggesting that the inhibition of macrophage polarization to M1-type and the promotion of macrophage polarization to M2-type, as well as the inhibition of NF-κB pathway by SA may be the reasons for its anti-ALI. This finding provides important molecular evidence for the further application of SA in the clinical treatment of ALI.

Learning-Based Hierarchical Decision-Making Framework for Automatic Driving in Incompletely Connected Traffic Scenarios.

The decision-making algorithm serves as a fundamental component for advancing the level of autonomous driving. The end-to-end decision-making algorithm has a strong ability to process the original data, but it has grave uncertainty. However, other learning-based decision-making algorithms rely heavily on ideal state information and are entirely unsuitable for autonomous driving tasks in real-world scenarios with incomplete global information. Addressing this research gap, this paper proposes a stable hierarchical decision-making framework with images as the input. The first step of the framework is a model-based data encoder that converts the input image data into a fixed universal data format. Next is a state machine based on a time series Graph Convolutional Network (GCN), which is used to classify the current driving state. Finally, according to the state's classification, the corresponding rule-based algorithm is selected for action generation. Through verification, the algorithm demonstrates the ability to perform autonomous driving tasks in different traffic scenarios without relying on global network information. Comparative experiments further confirm the effectiveness of the hierarchical framework, model-based image data encoder, and time series GCN.

Detection of Sulfoquinovosidase Activity in Cell Lysates Using Activity-Based Probes.

The sulfolipid sulfoquinovosyl diacylglycerol (SQDG), produced by plants, algae, and cyanobacteria, constitutes a major sulfur reserve in the biosphere. Microbial breakdown of SQDG is critical for the biological utilization of its sulfur. This commences through release of the parent sugar, sulfoquinovose (SQ), catalyzed by sulfoquinovosidases (SQases). These vanguard enzymes are encoded in gene clusters that code for diverse SQ catabolic pathways. To identify, visualize and isolate glycoside hydrolase CAZY-family 31 (GH31) SQases in complex biological environments, we introduce SQ cyclophellitol-aziridine activity-based probes (ABPs). These ABPs label the active site nucleophile of this enzyme family, consistent with specific recognition of the SQ cyclophellitol-aziridine in the active site, as evidenced in the 3D structure of Bacillus megaterium SQase. A fluorescent Cy5-probe enables visualization of SQases in crude cell lysates from bacteria harbouring different SQ breakdown pathways, whilst a biotin-probe enables SQase capture and identification by proteomics. The Cy5-probe facilitates monitoring of active SQase levels during different stages of bacterial growth which show great contrast to more traditional mRNA analysis obtained by RT-qPCR. Given the importance of SQases in global sulfur cycling and in human microbiota, these SQase ABPs provide a new tool with which to study SQase occurrence, activity and stability.

Microplastic separation and enrichment in microchannels under derivative electric field gradient by bipolar electrode reactions.

The decomposed plastic products in the natural environment evolve into tiny plastic particles with characteristics such as small size, lightweight, and difficulty in removal, resulting in a significant pollution issue in aquatic environments. Significant progress has been made in microplastic separation technology benefiting from microfluidic chips in recent years. Based on the mechanisms of microfluidic control technology, this study investigates the enrichment and separation mechanisms of polystyrene particles in an unbuffered solution. The Faraday reaction caused by the bipolar electrodes changes the electric field gradient and improves the separation efficiency. We also propose  an evaluation scheme to measure the separation efficiency. Finite element simulations are conducted to parametrically analyze the influence of applied voltages, channel geometry, and size of electrodes on plastic particle separation. The numerical cases indicate that the electrode-installed microfluidic channels separate microplastic particles effectively and precisely. The electrodes play an important role in local electric field distribution and trigger violent chemical reactions. By optimizing the microchannel structure, applied voltages, and separation channel angle, an optimal solution for separating microplastic particles can be found. This study could supply some references to control microplastic pollution in the future.

Association between Periodontitis and Mild Cognitive Impairment: A Systematic Review and Meta-Analysis.

INTRODUCTION: The connection between periodontitis and mild cognitive impairment (MCI) continues to receive attention. However, whether periodontitis is a risk factor for MCI remains still uncertain. This study aims to systematically analyze the available literature regarding the relationship between periodontitis and the risk of developing MCI and whether the periodontal health of MCI patients is poorer. METHODS: A literature search of PubMed, Scopus, Embase, and Web of Science databases was conducted to include all studies on the relationship between periodontitis and MCI from inception to April 2023. The studies were independently screened by 2 researchers, and those meeting the inclusion criteria were extracted and cross-checked. Pooled odds ratio (OR) or mean difference (MD) with 95% confidence intervals (CI) was calculated using either a fixed-effects or random-effects model. RESULTS: Seven studies with a total of 3,973 participants were included. Meta-analysis results showed a statistically significant higher incidence of MCI in patients with periodontitis (OR, 1.70 (95% CI: 1.24-2.32, p < 0.001) compared to healthy participants. A subgroup meta-analysis showed that the pooled OR for the risk of MCI in patients with severe periodontitis was 2.09 (95% CI: 1.49-2.92, p < 0.001). In addition, attachment loss (MD = 0.44, 95% CI: 0.12-0.75, p < 0.001) and plaque index (MD = 0.72, 95% CI: 0.50-0.93, p < 0.001) were higher in MCI patients compared with the control group, but the pocket probing depth (MD = 0.21, 95% CI: -0.08 to 0.49, p = 0.15) was not significantly different between the two groups. CONCLUSIONS: Patients with periodontitis are at a higher risk of developing MCI, and the periodontal health of MCI patients is generally compromised. However, further well-designed studies should be conducted to confirm this relationship between MCI and periodontitis.

Polarization-insensitive vortex beam generator by the holographic grating on an integrated multi-layer waveguide.

An integrated polarization-insensitive vortex beam generator is proposed in this study. It is composed of a holographic grating on a multi-layer waveguide, which enables conversion of Transverse Electric (TE) and Transverse Magnetic (TM) waveguide modes to y-polarized and x-polarized optical vortex beams, respectively. The conversion efficiency and the phase fidelity are numerically analyzed, and the working bandwidth is about 100 nm from 1500 nm to 1600 nm with a phase fidelity above 0.7. Moreover, the vortex beam with the superposition of the y-polarization and x-polarization states can be obtained with the incident of the superposition of TE and TM waveguide modes.

Efficient nitrite accumulation in partial sulfide autotrophic denitrification (PSAD) system: insights of S/N ratio, pH and temperature.

To provide the necessary nitrite for the Anaerobic Ammonium Oxidation (ANAMMOX) process, the effect of nitrite accumulation in the partial sulfide autotrophic denitrification (PSAD) process was investigated using an SBR reactor. The results revealed that the effectiveness of nitrate removal was unsatisfactory when the S/N ratio (mol/mol) fell below 0.6. The optimal conditions for nitrate removal and nitrite accumulation were achieved within the S/N ratio range of 0.7-0.8, resulting in an average Nitrate Removal Efficiency (NRE) of 95.84%±4.89% and a Nitrite Accumulation Rate (NAR) of 75.31%±6.61%, respectively. It was observed that the nitrate reduction rate was three times faster than that of nitrite reduction during a typical cycle test. Furthermore, batch tests were conducted to assess the influence of pH and temperature conditions. In the pH tests, it became evident that the PSAD process performed more effectively in alkaline environment. The highest levels of nitrate removal and nitrite accumulation were achieved at an initial pH of 8.5, resulting in a NRE of 98.30%±1.93% and a NAR of 85.83%±0.47%, respectively. In the temperature tests, the most favourable outcomes for nitrate removal and nitrite accumulation were observed at 22±1 ℃, with a NRE of 100.00% and a NAR of 81.03%±1.64%, respectively. Moreover, a comparative analysis of 16S rRNA sequencing results between the raw sludge and the sulfide-enriched culture sludge sample showed that Proteobacteria (49.51%) remained the dominant phylum, with Thiobacillus (24.72%), Prosthecobacter (2.55%), Brevundimonas (2.31%) and Ignavibacterium (2.04%) emerging as the dominant genera, assuming the good nitrogen performance of the system.

Molecular dynamics simulation of the interaction between palmitic acid and high pressure CO2.

In this study, molecular dynamics simulation was used to explore the interaction characteristics of palmitic acid and CO2, and the effects of temperature and pressure on the solubility of palmitic acid in CO2 were investigated. In the range of 293-353 K and 5-30 MPa, the snapshot of palmitic acid distribution in CO2 shows that the molecular chain of palmitic acid in high-density CO2 system is more straight and more dispersed than that in low-density CO2 system. The radial distribution function further clearly shows that the solubility of palmitic acid in CO2 decreases with the increase of temperature and increases with the increase of pressure, which is consistent with the fatty acid solubility data reported in the literature and the setting rules of supercritical CO2 extraction process conditions. As the temperature decreases and the pressure increases, the interaction energy between palmitic acid and CO2 increases, which is conducive to overcoming the intermolecular force of palmitic acid and promoting dissolution. The solubility parameters of palmitic acid and CO2 can better reflect the trend of palmitic acid solubility changing with temperature and pressure, which can play a guiding role in the determination of process conditions and even the development of new processes.

Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends.

The proper functioning of connected and autonomous vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaborative decision-making technology for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) methods have become an efficient way in solving decision-making problems. However, with the development of computing technology, graph reinforcement learning (GRL) methods have gradually demonstrated the large potential to further improve the decision-making performance of CAVs, especially in the area of accurately representing the mutual effects of vehicles and modeling dynamic traffic environments. To facilitate the development of GRL-based methods for autonomous driving, this paper proposes a review of GRL-based methods for the decision-making technologies of CAVs. Firstly, a generic GRL framework is proposed in the beginning to gain an overall understanding of the decision-making technology. Then, the GRL-based decision-making technologies are reviewed from the perspective of the construction methods of mixed autonomy traffic, methods for graph representation of the driving environment, and related works about graph neural networks (GNN) and DRL in the field of decision-making for autonomous driving. Moreover, validation methods are summarized to provide an efficient way to verify the performance of decision-making methods. Finally, challenges and future research directions of GRL-based decision-making methods are summarized.

µPADs on Centrifugal Microfluidic Discs for Rapid Sample-to-Answer Salivary Diagnostics.

A fully integrated device for salivary detection with a sample-in-answer-out fashion is critical for noninvasive point-of-care testing (POCT), especially for the screening of contagious disease infection. Microfluidic paper-based analytical devices (µPADs) have demonstrated their huge potential in POCT due to their low cost and easy adaptation with other components. This study developed a generic POCT platform by integrating a centrifugal microfluidic disc with µPADs to realize sample-to-answer salivary diagnostics. Specifically, a custom centrifugal microfluidic disc integrated with µPADs is fabricated, which demonstrated a high efficiency in saliva treatment. To demonstrate the capability of the integrated device for salivary analysis, the SARS-CoV-2 Nucleocapsid (N) protein, a reliable biomarker for SARS-CoV-2 acute infection, is used as the model analyte. By the chemical treatment of the µPAD surface, and by optimizing the protein immobilization conditions, the on-disc µPADs were able to detect the SARS-CoV-2 N protein down to 10 pg mL-1 with a dynamic range of 10-1000 pg mL-1 and an assay time of 8 min. The integrated device was successfully used for the quantification of the N protein of pseudovirus in saliva with high specificity and demonstrated a comparable performance to the commercial paper lateral flow assay test strips.

CD44 connects autophagy decline and ageing in the vascular endothelium.

The decline of endothelial autophagy is closely related to vascular senescence and disease, although the molecular mechanisms connecting these outcomes in vascular endothelial cells (VECs) remain unclear. Here, we identify a crucial role for CD44, a multifunctional adhesion molecule, in controlling autophagy and ageing in VECs. The CD44 intercellular domain (CD44ICD) negatively regulates autophagy by reducing PIK3R4 and PIK3C3 levels and disrupting STAT3-dependent PtdIns3K complexes. CD44 and its homologue clec-31 are increased in ageing vascular endothelium and Caenorhabditis elegans, respectively, suggesting that an age-dependent increase in CD44 induces autophagy decline and ageing phenotypes. Accordingly, CD44 knockdown ameliorates age-associated phenotypes in VECs. The endothelium-specific CD44ICD knock-in mouse is shorter-lived, with VECs exhibiting obvious premature ageing characteristics associated with decreased basal autophagy. Autophagy activation suppresses the premature ageing of human and mouse VECs overexpressing CD44ICD, function conserved in the CD44 homologue clec-31 in C. elegans. Our work describes a mechanism coordinated by CD44 function bridging autophagy decline and ageing.

Nanoscale Investigation of Bubble Nucleation and Boiling on Random Rough Surfaces.

Surface roughness is one of the significant factors affecting liquid-vapor phase change heat transfer. This paper explores the effect of surface roughness on bubble nucleation and boiling heat transfer, as well as the microscopic mechanism, by constructing random rough surfaces using molecular dynamics (MD) simulation. Bubbles randomly nucleate on a flat surface and tend to nucleate in pits on rough surfaces. The pits on the random rough surface gather more argon atoms than the protrusions, forming low potential energy regions on the surface, thus providing stable nucleation sites for bubbles. As the surface roughness increases, bubble generation, merging, and growth are advanced. In addition, rough surfaces offer a larger effective heat transfer area for the heat transfer process, increase the strength of solid-liquid coupling, and obtain smaller solid-liquid interaction energy. The critical heat flux (CHF) value positively correlates with surface roughness. As the roughness increases, the surface superheat at the onset of CHF decreases accordingly. This paper provides new insights into the mechanism of heat transfer enhancement on rough surfaces and surface design in thermal management.

Estimation of Particle Location in Granular Materials Based on Graph Neural Networks.

Particle locations determine the whole structure of a granular system, which is crucial to understanding various anomalous behaviors in glasses and amorphous solids. How to accurately determine the coordinates of each particle in such materials within a short time has always been a challenge. In this paper, we use an improved graph convolutional neural network to estimate the particle locations in two-dimensional photoelastic granular materials purely from the knowledge of the distances for each particle, which can be estimated in advance via a distance estimation algorithm. The robustness and effectiveness of our model are verified by testing other granular systems with different disorder degrees, as well as systems with different configurations. In this study, we attempt to provide a new route to the structural information of granular systems irrelevant to dimensionality, compositions, or other material properties.

Raddeanin A promotes autophagy-induced apoptosis by inactivating PI3K/AKT/mTOR pathway in lung adenocarcinoma cells.

Non-small-cell lung cancer (NSCLC) is the most common cancer in the world. Previous studies have shown that Raddeanin A (RA) exhibited distinct antitumor properties in gastric and colon cancer. This study aimed to investigate the pharmacological actions and intrinsic mechanisms of RA in NSCLC. Through the application of network pharmacology, the potential targets of RA for NSCLC therapy such as SRC, MAPK1, and STAT3 were excavated. Enrichment analyses showed that these targets were concerned with the regulation of cell death, regulation of MAPK cascade, Ras signaling pathway, and PI3K/AKT signaling pathway. Meanwhile, 13 targets of RA were identified as autophagy-related genes. Our experiment data showed that RA effectively inhibited proliferation and induced apoptosis in lung cancer cells A549. We also found that RA could induce autophagy simultaneously. Furthermore, the autophagy induced by RA had a synergistic effect with apoptosis and contributed to cell death. Additionally, RA could downregulate the activity of the PI3K/AKT/mTOR pathway. Generally, our results indicated the antitumor effect and underlying mechanisms of RA on apoptosis and autophagy in A549 cells, suggesting that RA could be used as an effective antineoplastic agent.

The prediction of contact force networks in granular materials based on graph neural networks.

The contact force network, usually organized inhomogeneously by the inter-particle forces on the bases of the contact network topologies, is essential to the rigidity and stability in amorphous solids. How to capture such a "backbone" is crucial to the understanding of various anomalous properties or behaviors in those materials, which remains a central challenge presently in physics, engineering, or material science. Here, we use a novel graph neural network to predict the contact force network in two-dimensional granular materials under uniaxial compression. With the edge classification model in the framework of the deep graph library, we show that the inter-particle contact forces can be accurately estimated purely from the knowledge of the static microstructures, which can be acquired from a discrete element method or directly visualized from experimental methods. By testing the granular packings with different structural disorders and pressure, we further demonstrate the robustness of the optimized graph neural network to changes in various model parameters. Our research tries to provide a new way of extracting the information about the inter-particle forces, which substantially improves the efficiency and reduces the costs compared to the traditional experiments.

Transverse electronic transport properties of single DNA nucleobase pairs between different electrode materials.

Detection of gene mutation through electronic transport properties measurements is an attractive research topic. For this purpose, we computed the current-voltage characteristics of adenine-thymine and guanine-cytosine nucleobase pairs, using a combination method of density-functional theory with non-equilibrium Green's function. Gene mutation was also simulated by structural change in nucleobase pairs by a double proton transfer mechanism. Four different metal electrodes were tested. Comparing the results, nucleobase pairs between platinum surfaces showed distinct electronic transport properties. Such as reverse rectifying direction and negative differential resistance behaviors. The discrepancy can be explained from series of electronic and structural analyses. All these results made identification of structural changes in individual DNA nucleobase pairs possible.

CUR5g, a novel autophagy inhibitor, exhibits potent synergistic anticancer effects with cisplatin against non-small-cell lung cancer.

Autophagy, a highly conserved degradation process of eukaryotic cells, has been proven to be closely related to chemoresistance and metastasis of non-small-cell lung cancer (NSCLC). Autophagy inhibitors, such as chloroquine (CQ) and its derivative hydroxychloroquine (HCQ), has been shown to mediate anticancer effects in preclinical models, especially when combined with chemotherapy. However, the vast majority of autophagy inhibitors, including CQ and HCQ, actually disrupt lysosomal or/and possibly non-lysosomal processes other than autophagy. It is therefore of great significance to discover more specific autophagy inhibitors. In this study, after screening a series of curcumin derivatives synthesized in our laboratory, we found that (3E,5E)-1-methyl-3-(4-hydroxybenzylidene)-5-(3-indolymethylene)-piperidine-4-one (CUR5g) selectively inhibited autophagosome degradation in cancer cells by blocking autophagosome-lysosome fusion. CUR5g did not affect the lysosomal pH and proteolytic function, nor did it disturb cytoskeleton. CUR5g blocked the recruitment of STX17, a soluble N-ethylmaleimide-sensitive factor attachment protein receptor (SNARE) protein, to autophagosomes via a UVRAG-dependent mechanism, resulting in the inability of autophagosomes to fuse with lysosomes. CUR5g alone did not induce apoptosis and necrosis of A549 cells, but significantly inhibited the mobility and colony formation of A549 cells. More excitingly, CUR5g showed no obvious toxicity to normal HUVECs in vitro or mice in vivo. CUR5g enhances the cisplatin sensitivity of A549 cells and effectively inhibited autophagy in tumor tissues in vivo. Collectively, our study identified a new late-stage autophagy inhibitor and provided a novel option for NSCLC treatment, particular when combined with cisplatin.

Generalized Single-Vehicle-Based Graph Reinforcement Learning for Decision-Making in Autonomous Driving.

In the autonomous driving process, the decision-making system is mainly used to provide macro-control instructions based on the information captured by the sensing system. Learning-based algorithms have apparent advantages in information processing and understanding for an increasingly complex driving environment. To incorporate the interactive information between agents in the environment into the decision-making process, this paper proposes a generalized single-vehicle-based graph neural network reinforcement learning algorithm (SGRL algorithm). The SGRL algorithm introduces graph convolution into the traditional deep neural network (DQN) algorithm, adopts the training method for a single agent, designs a more explicit incentive reward function, and significantly improves the dimension of the action space. The SGRL algorithm is compared with the traditional DQN algorithm (NGRL) and the multi-agent training algorithm (MGRL) in the highway ramp scenario. Results show that the SGRL algorithm has outstanding advantages in network convergence, decision-making effect, and training efficiency.

Multi-Agent Decision-Making Modes in Uncertain Interactive Traffic Scenarios via Graph Convolution-Based Deep Reinforcement Learning.

As one of the main elements of reinforcement learning, the design of the reward function is often not given enough attention when reinforcement learning is used in concrete applications, which leads to unsatisfactory performances. In this study, a reward function matrix is proposed for training various decision-making modes with emphasis on decision-making styles and further emphasis on incentives and punishments. Additionally, we model a traffic scene via graph model to better represent the interaction between vehicles, and adopt the graph convolutional network (GCN) to extract the features of the graph structure to help the connected autonomous vehicles perform decision-making directly. Furthermore, we combine GCN with deep Q-learning and multi-step double deep Q-learning to train four decision-making modes, which are named the graph convolutional deep Q-network (GQN) and the multi-step double graph convolutional deep Q-network (MDGQN). In the simulation, the superiority of the reward function matrix is proved by comparing it with the baseline, and evaluation metrics are proposed to verify the performance differences among decision-making modes. Results show that the trained decision-making modes can satisfy various driving requirements, including task completion rate, safety requirements, comfort level, and completion efficiency, by adjusting the weight values in the reward function matrix. Finally, the decision-making modes trained by MDGQN had better performance in an uncertain highway exit scene than those trained by GQN.