Your browser doesn't support javascript.
loading
Show: 20 | 50 | 100
Results 1 - 20 de 40
Filter
1.
Article in English | MEDLINE | ID: mdl-39146157

ABSTRACT

Reinforcement learning (RL) agents are vulnerable to adversarial disturbances, which can deteriorate task performance or break down safety specifications. Existing methods either address safety requirements under the assumption of no adversary (e.g., safe RL) or only focus on robustness against performance adversaries (e.g., robust RL). Learning one policy that is both safe and robust under any adversaries remains a challenging open problem. The difficulty is how to tackle two intertwined aspects in the worst cases: feasibility and optimality. The optimality is only valid inside a feasible region (i.e., robust invariant set), while the identification of maximal feasible region must rely on how to learn the optimal policy. To address this issue, we propose a systematic framework to unify safe RL and robust RL, including the problem formulation, iteration scheme, convergence analysis and practical algorithm design. The unification is built upon constrained two-player zero-sum Markov games, in which the objective for protagonist is twofold. For states inside the maximal robust invariant set, the goal is to pursue rewards under the condition of guaranteed safety; for states outside the maximal robust invariant set, the goal is to reduce the extent of constraint violation. A dual policy iteration scheme is proposed, which simultaneously optimizes a task policy and a safety policy. We prove that the iteration scheme converges to the optimal task policy which maximizes the twofold objective in the worst cases, and the optimal safety policy which stays as far away from the safety boundary. The convergence of safety policy is established by exploiting the monotone contraction property of safety self-consistency operators, and that of task policy depends on the transformation of safety constraints into state-dependent action spaces. By adding two adversarial networks (one is for safety guarantee and the other is for task performance), we propose a practical deep RL algorithm for constrained zero-sum Markov games, called dually robust actor-critic (DRAC). The evaluations with safety-critical benchmarks demonstrate that DRAC achieves high performance and persistent safety under all scenarios (no adversary, safety adversary, performance adversary), outperforming all baselines by a large margin.

2.
Article in English | MEDLINE | ID: mdl-38231811

ABSTRACT

We focus on learning the zero-constraint-violation safe policy in model-free reinforcement learning (RL). Existing model-free RL studies mostly use the posterior penalty to penalize dangerous actions, which means they must experience the danger to learn from the danger. Therefore, they cannot learn a zero-violation safe policy even after convergence. To handle this problem, we leverage the safety-oriented energy functions to learn zero-constraint-violation safe policies and propose the safe set actor-critic (SSAC) algorithm. The energy function is designed to increase rapidly for potentially dangerous actions, locating the safe set on the action space. Therefore, we can identify the dangerous actions prior to taking them and achieve zero-constraint violation. Our major contributions are twofold. First, we use the data-driven methods to learn the energy function, which releases the requirement of known dynamics. Second, we formulate a constrained RL problem to solve the zero-violation policies. We prove that our Lagrangian-based constrained RL solutions converge to the constrained optimal zero-violation policies theoretically. The proposed algorithm is evaluated on the complex simulation environments and a hardware-in-loop (HIL) experiment with a real autonomous vehicle controller. Experimental results suggest that the converged policies in all environments achieve zero-constraint violation and comparable performance with model-based baseline.

3.
J Cell Mol Med ; 28(5): e18070, 2024 Mar.
Article in English | MEDLINE | ID: mdl-38102848

ABSTRACT

Cisplatin-based chemotherapy is often used in advanced gastric cancer (GC) treatment, yet resistance to cisplatin may lead to treatment failure. Mechanisms underlying cisplatin resistance remain unclear. Recent evidence highlighted the role of macrophages in cancer chemoresistance. Macrophage-derived exosomes were shown to facilitate intercellular communication. Here, we investigated the cisplatin resistance mechanism based on macrophage-derived exosomes in gastric cancer. Cell growth and apoptosis detection experiments revealed that M2-polarized macrophages increased the resistance of GC cells to cisplatin. qRT-PCR, RNAase R assay, actinomycin D assay and cell nucleo-cytoplasmic separation experiments confirmed the existence of circTEX2 in macrophage cytoplasm, with a higher expression level in M2 macrophages than that in M1 macrophages. Further experiments showed that circTEX2 acted as microRNA sponges for miR-145 and regulated the expression of ATP Binding Cassette Subfamily C Member 1 (ABCC1). Inhibition of the circTEX2/miR-145/ABCC1 axis blocked the cisplatin resistance of gastric cancer induced by M2 macrophages, as evidenced by in vitro and in vivo experiments. In conclusion, our research suggests that the exosomal transfer of M2 macrophage-derived circTEX2 enhances cisplatin resistance in gastric cancer through miR-145/ABCC1. Additionally, communication between macrophages and cancer cells via exosomes may be a promising therapeutic target for the treatment of cisplatin-resistant gastric cancer.


Subject(s)
Cisplatin , Drug Resistance, Neoplasm , Exosomes , Gene Expression Regulation, Neoplastic , Macrophages , MicroRNAs , Multidrug Resistance-Associated Proteins , RNA, Circular , Stomach Neoplasms , Stomach Neoplasms/genetics , Stomach Neoplasms/pathology , Stomach Neoplasms/drug therapy , Stomach Neoplasms/metabolism , Cisplatin/pharmacology , Drug Resistance, Neoplasm/genetics , Humans , Macrophages/metabolism , Macrophages/drug effects , MicroRNAs/genetics , MicroRNAs/metabolism , Cell Line, Tumor , Animals , Gene Expression Regulation, Neoplastic/drug effects , RNA, Circular/genetics , Exosomes/metabolism , Multidrug Resistance-Associated Proteins/metabolism , Multidrug Resistance-Associated Proteins/genetics , Mice , Apoptosis/drug effects , Cell Proliferation/drug effects , Antineoplastic Agents/pharmacology , Mice, Nude
4.
Article in English | MEDLINE | ID: mdl-38015685

ABSTRACT

Model error and external disturbance have been separately addressed by optimizing the definite H∞ performance in standard linear H∞ control problems. However, the concurrent handling of both introduces uncertainty and nonconvexity into the H∞ performance, posing a huge challenge for solving nonlinear problems. This article introduces an additional cost function in the augmented Hamilton-Jacobi-Isaacs (HJI) equation of zero-sum games to simultaneously manage the model error and external disturbance in nonlinear robust performance problems. For satisfying the Hamilton-Jacobi inequality in nonlinear robust control theory under all considered model errors, the relationship between the additional cost function and model uncertainty is revealed. A critic online learning algorithm, applying Lyapunov stabilizing terms and historical states to reinforce training stability and achieve persistent learning, is proposed to approximate the solution of the augmented HJI equation. By constructing a joint Lyapunov candidate about the critic weight and system state, both stability and convergence are proved by the second method of Lyapunov. Theoretical results also show that introducing historical data reduces the ultimate bounds of system state and critic error. Three numerical examples are conducted to demonstrate the effectiveness of the proposed method.

5.
IEEE Trans Cybern ; PP2023 Oct 26.
Article in English | MEDLINE | ID: mdl-37883283

ABSTRACT

In recent times, significant advancements have been made in delving into the optimization landscape of policy gradient methods for achieving optimal control in linear time-invariant (LTI) systems. Compared with state-feedback control, output-feedback control is more prevalent since the underlying state of the system may not be fully observed in many practical settings. This article analyzes the optimization landscape inherent to policy gradient methods when applied to static output feedback (SOF) control in discrete-time LTI systems subject to quadratic cost. We begin by establishing crucial properties of the SOF cost, encompassing coercivity, L -smoothness, and M -Lipschitz continuous Hessian. Despite the absence of convexity, we leverage these properties to derive novel findings regarding convergence (and nearly dimension-free rate) to stationary points for three policy gradient methods, including the vanilla policy gradient method, the natural policy gradient method, and the Gauss-Newton method. Moreover, we provide proof that the vanilla policy gradient method exhibits linear convergence toward local minima when initialized near such minima. This article concludes by presenting numerical examples that validate our theoretical findings. These results not only characterize the performance of gradient descent for optimizing the SOF problem but also provide insights into the effectiveness of general policy gradient methods within the realm of reinforcement learning.

6.
Gene ; 887: 147733, 2023 Dec 15.
Article in English | MEDLINE | ID: mdl-37625563

ABSTRACT

Cisplatin is the first-line drug for gastric cancer (GC). Cisplatin resistance is the most important cause of poor prognosis for GC. Increasing evidence has identified the important role of macrophage polarization in chemoresistance. CircRNAs are newly discovered non-coding RNAs, characterized by covalently closed loops with high stability. Previous studies have reported a significant difference between circRNA profiles expressed in classically activated M1 macrophages, and those expressed in alternatively activated M2 macrophages. However, the underlying mechanism behind the regulation of GC cisplatin resistance by macrophages remains unclear. In our study, we observed the aberrant high expression of circSOD2 in M1 macrophages derived from THP-1. These expression patterns were confirmed in macrophages from patients with GC. Detection of the M1 and M2 markers confirmed that overexpression of circSOD2 enhances M1 polarization. The viability of cisplatin-treated GC cells was significantly reduced in the presence of macrophages overexpressing circSOD2, and cisplatin-induced apoptosis increased dramatically. In vivo experiments showed that macrophages expressing circSOD2 enhanced the effect of cisplatin. Moreover, we demonstrated that circSOD2 acts as a microRNA sponge for miR-1296 and regulates the expression of its target gene STAT1 (signal transducer and activator of transcription 1). CircSOD2 exerts its function through the miR-1296/STAT1 axis. Inhibition of circSOD2/miR-1296/STAT1 may therefore reduce M1 polarization. Overexpression of circSOD2 promotes the polarization of M1 macrophages and enhances the effect of cisplatin in GC. CircSOD2 is a novel positive regulator of M1 macrophages and may serve as a potential target for GC chemotherapy.


Subject(s)
MicroRNAs , Stomach Neoplasms , Humans , Cisplatin/pharmacology , Cisplatin/therapeutic use , Macrophages/metabolism , MicroRNAs/metabolism , Phenotype , STAT1 Transcription Factor/genetics , STAT1 Transcription Factor/metabolism , Stomach Neoplasms/drug therapy , Stomach Neoplasms/genetics , Stomach Neoplasms/metabolism
7.
Gene ; 888: 147739, 2023 Dec 20.
Article in English | MEDLINE | ID: mdl-37633535

ABSTRACT

The active ingredients of many medicinal plants are the secondary metabolites associated with the growth period. Lonicera japonica Thunb. is an important traditional Chinese medicine, and the flower development stage is an important factor that influences the quality of medicinal ingredients. In this study, transcriptomics and metabolomics were performed to reveal the regulatory mechanism of secondary metabolites during flowering of L. japonica. The results showed that the content of chlorogenic acid (CGA) and luteolin gradually decreased from green bud stage (Sa) to white flower stage (Sc), especially from white flower bud stage (Sb) to Sc. Most of the genes encoding the crucial rate-limiting enzymes, including PAL, C4H, HCT, C3'H, F3'H and FNSII, were down-regulated in three comparisons. Correlation analysis identified some members of the MYB, AP2/ERF, bHLH and NAC transcription factor families that are closely related to CGA and luteolin biosynthesis. Furthermore, differentially expressed genes (DEGs) involved in hormone biosynthesis, signalling pathways and flowering process were analysed in three flower developmental stage.


Subject(s)
Chlorogenic Acid , Lonicera , Chlorogenic Acid/metabolism , Luteolin , Gene Expression Profiling , Lonicera/genetics , Flowers/genetics , Flowers/metabolism , Hormones/metabolism , Transcriptome/genetics
8.
Adv Sci (Weinh) ; 10(22): e2300576, 2023 08.
Article in English | MEDLINE | ID: mdl-37202594

ABSTRACT

Treatment of infected wounds remains a challenge owing to antibiotic resistance; thus, developing smart biomaterials for the healing of infected wounds is urgently needed. In this study, a microneedle (MN) patch system with antimicrobial and immunomodulatory properties is developed to promote and accelerate infected wound healing. In the MN patch (termed PFG/M MNs), a nanoparticle with polydopamine (PDA)-loaded iron oxide is grafted with glucose oxidase (GOx) and hyaluronic acid (HA) and then integrated into the tips, and amine-modified mesoporous silica nanoparticles (AP-MSNs) are incorporated into the bases. Results show that PFG/M MNs eradicate bacterial infections and modulate the immune microenvironment, combining the advantages of chemodynamic therapy, photothermal therapy, and M2 macrophage polarization from Fe/PDA@GOx@HA in the tips as well as anti-inflammatory effect of AP-MSNs from the MN bases. Thus, the PFG/M MN system is a promising clinical candidate for promoting infected wound healing.


Subject(s)
Anti-Infective Agents , Amines , Biocompatible Materials , Drug Delivery Systems , Glucose Oxidase , Hyaluronic Acid
9.
IEEE Trans Neural Netw Learn Syst ; 34(9): 5255-5267, 2023 Sep.
Article in English | MEDLINE | ID: mdl-37015565

ABSTRACT

The Hamilton-Jacobi-Bellman (HJB) equation serves as the necessary and sufficient condition for the optimal solution to the continuous-time (CT) optimal control problem (OCP). Compared with the infinite-horizon HJB equation, the solving of the finite-horizon (FH) HJB equation has been a long-standing challenge, because the partial time derivative of the value function is involved as an additional unknown term. To address this problem, this study first-time bridges the link between the partial time derivative and the terminal-time utility function, and thus it facilitates the use of the policy iteration (PI) technique to solve the CT FH OCPs. Based on this key finding, the FH approximate dynamic programming (ADP) algorithm is proposed leveraging an actor-critic framework. It is shown that the algorithm exhibits important properties in terms of convergence and optimality. Rather importantly, with the use of multilayer neural networks (NNs) in the actor-critic architecture, the algorithm is suitable for CT FH OCPs toward more general nonlinear and complex systems. Finally, the effectiveness of the proposed algorithm is demonstrated by conducting a series of simulations on both a linear quadratic regulator (LQR) problem and a nonlinear vehicle tracking problem.

10.
J Chromatogr Sci ; 61(4): 303-311, 2023 Apr 24.
Article in English | MEDLINE | ID: mdl-36892165

ABSTRACT

Headspace gas chromatography-ion mobility spectrometric (HS-GC-IMS) fingerprint of volatile organic compounds (VOCs) in Lonicerae japonicae flos (LJF, Jinyinhua in Chinese) was developed. This method, combined with chemometrics analysis, was explored in the identification of authentic LJF. Seventy VOCs were identified from LJF, including aldehydes, ketones, esters, etc. The developed volatile-compound fingerprint based on HS-GC-IMS coupled with PCA analysis can successfully discriminate LJF from its adulterant: Lonicerae japonicae(LJ, called Shanyinhua in China) and can equally discriminate the LJF samples from different geographical origins of China. Total of four (compound 120, compound 184, 2-heptanone and 2-heptanone#2) and nine VOCs (styrene, compound 41, 3z-hexenol, methylpyrazine, hexanal#2, compound 78, compound 110, compound 124 and compound 180) were exploited, which might serve as the chemical markers for the difference of LJF, LJ and LJF from different regions of China. The result showed that the fingerprint based on HS-GC-IMS combined with PCA exhibited distinct advantages, such as rapid, intuitive and powerful selectivity, which demonstrated great application potential in the authentic identification of LJF.


Subject(s)
Lonicera , Volatile Organic Compounds , Gas Chromatography-Mass Spectrometry/methods , Chemometrics , Ion Mobility Spectrometry/methods , Ketones/analysis , Lonicera/chemistry , Volatile Organic Compounds/analysis
11.
IEEE Trans Cybern ; 53(2): 859-873, 2023 Feb.
Article in English | MEDLINE | ID: mdl-35439160

ABSTRACT

Decision and control are core functionalities of high-level automated vehicles. Current mainstream methods, such as functional decomposition and end-to-end reinforcement learning (RL), suffer high time complexity or poor interpretability and adaptability on real-world autonomous driving tasks. In this article, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically. First, the static path planning generates several candidate paths only considering static traffic elements. Then, the dynamic optimal tracking is designed to track the optimal path while considering the dynamic obstacles. To that end, we formulate a constrained optimal control problem (OCP) for each candidate path, optimize them separately, and follow the one with the best tracking performance. To unload the heavy online computation, we propose a model-based RL algorithm that can be served as an approximate-constrained OCP solver. Specifically, the OCPs for all paths are considered together to construct a single complete RL problem and then solved offline in the form of value and policy networks for real-time online path selecting and tracking, respectively. We verify our framework in both simulations and the real world. Results show that compared with baseline methods, IDC has an order of magnitude higher online computing efficiency, as well as better driving performance, including traffic efficiency and safety. In addition, it yields great interpretability and adaptability among different driving scenarios and tasks.

12.
J Mol Med (Berl) ; 100(11): 1569-1585, 2022 11.
Article in English | MEDLINE | ID: mdl-36094536

ABSTRACT

With the rapid increase in the incidence of diabetes, non-healing diabetic wounds have posed a huge challenge to public health. Endothelial progenitor cell (EPC) has been widely reported to promote wound repairing, while its number and function were suppressed in diabetes. However, the specific mechanisms and competing endogenous RNA (ceRNA) network of EPCs in diabetes remain largely unknown. Thus, the transcriptome analyses were carried in the present study to clarify the mechanism underlying EPCs dysfunction in diabetes. EPCs were successfully isolated from rats. Compared to the control, diabetic rat-derived EPCs displayed impaired proliferation, migration, and tube formation ability. The differentially expressed (DE) RNAs were successfully identified by RNA sequencing in the control and diabetic groups. Gene Ontology and Kyoto Encyclopedia of Genes and Genomes analyses indicated that DE mRNAs were significantly enriched in terms and pathways involved in the functions of EPCs and wound healing. Protein-protein interaction networks revealed critical DE mRNAs in the above groups. Moreover, the whole lncRNA-miRNA-mRNA ceRNA network was constructed, in which 9 lncRNAs, 9 mRNAs, and 5 miRNAs were further validated by quantitative real-time polymerase chain reaction. Rno-miR-10b-5p and Tgfb2 were identified as key regulators of EPCs dysfunction in diabetes. The present research provided novel insight into the underlying mechanism of EPCs dysfunction in diabetes and prompted potential targets to restore the impaired functions, thus accelerating diabetic wound healing. KEY MESSAGES: • Compared to the control, diabetic rat-derived EPCs displayed impaired proliferation, migration, and tube formation ability. • The DE RNAs were successfully identified by RNA sequencing in the control and diabetic groups and analyzed by DE, GO, and KEGG analysis. • PPI and lncRNA-miRNA-mRNA ceRNA networks were constructed. • 9 lncRNAs, 9 mRNAs, and 5 miRNAs were further validated by qRT-PCR. • Rno-miR-10b-5p and Tgfb2 were identified as key regulators of EPCs dysfunction in diabetes.


Subject(s)
Diabetes Mellitus , Endothelial Progenitor Cells , MicroRNAs , RNA, Long Noncoding , Rats , Animals , RNA, Long Noncoding/genetics , RNA, Long Noncoding/metabolism , MicroRNAs/genetics , RNA, Messenger/genetics , Endothelial Progenitor Cells/metabolism , Gene Regulatory Networks , Sequence Analysis, RNA , Diabetes Mellitus/genetics
13.
Adv Sci (Weinh) ; 9(29): e2202453, 2022 10.
Article in English | MEDLINE | ID: mdl-35981878

ABSTRACT

Smart nanomaterials constitute a new approach toward safer and more effective combined anti-cancer immunotherapy. In this study, polydopamine-multiprotein conjugates (DmPCs) that can be used for targeted delivery of multiple proteins to cells, realize imaging and combine the advantages of multiple treatment methods (photothermal therapy, chemodynamic therapy, and immunotherapy) can be synthesized and characterized. Proteins, as biological agents, are frequently used in this context, given their low toxicity in vivo. To overcome protein instability and short half-life in vivo, the use of several proteins in combination with selected nanomaterials to treat patients with melanoma is proposed. In addition to the synthesis and characterization of protein-bound nanoparticles, it is further demonstrated that several proteins can be efficiently delivered to tumor sites. DmPCs have a wide range of potential adaptability, which provides new opportunities for proteins in the field of treatment and imaging.


Subject(s)
Hyperthermia, Induced , Nanoparticles , Neoplasms , Biological Factors , Humans , Nanoparticles/therapeutic use , Neoplasms/diagnosis , Neoplasms/therapy , Phototherapy , Protein Binding
14.
Article in English | MEDLINE | ID: mdl-35635820

ABSTRACT

Safety is essential for reinforcement learning (RL) applied in the real world. Adding chance constraints (or probabilistic constraints) is a suitable way to enhance RL safety under uncertainty. Existing chance-constrained RL methods, such as the penalty methods and the Lagrangian methods, either exhibit periodic oscillations or learn an overconservative or unsafe policy. In this article, we address these shortcomings by proposing a separated proportional-integral Lagrangian (SPIL) algorithm. We first review the constrained policy optimization process from a feedback control perspective, which regards the penalty weight as the control input and the safe probability as the control output. Based on this, the penalty method is formulated as a proportional controller, and the Lagrangian method is formulated as an integral controller. We then unify them and present a proportional-integral Lagrangian method to get both their merits with an integral separation technique to limit the integral value to a reasonable range. To accelerate training, the gradient of safe probability is computed in a model-based manner. The convergence of the overall algorithm is analyzed. We demonstrate that our method can reduce the oscillations and conservatism of RL policy in a car-following simulation. To prove its practicality, we also apply our method to a real-world mobile robot navigation task, where our robot successfully avoids a moving obstacle with highly uncertain or even aggressive behaviors.

16.
J Nanobiotechnology ; 20(1): 147, 2022 Mar 19.
Article in English | MEDLINE | ID: mdl-35305648

ABSTRACT

Clinical work and research on diabetic wound repair remain challenging globally. Although various conventional wound dressings have been continuously developed, the efficacy is unsatisfactory. The effect of drug delivery is limited by the depth of penetration. The sustained release of biomolecules from biological wound dressings is a promising treatment approach to wound healing. An assortment of cell-derived exosomes (exos) have been proved to be instrumental in tissue regeneration, and researchers are dedicated to developing biomolecules carriers with unique properties. Herein, we reported a methacrylate gelatin (GelMA) microneedles (MNs) patch to achieve transdermal and controlled release of exos and tazarotene. Our MNs patch comprising GelMA/PEGDA hydrogel has distinctive biological features that maintain the biological activity of exos and drugs in vitro. Additionally, its unique physical structure prevents it from being tightly attached to the skin of the wound, it promotes cell migration, angiogenesis by slowly releasing exos and tazarotene in the deep layer of the skin. The full-thickness cutaneous wound on a diabetic mouse model was carried out to demonstrate the therapeutic effects of GelMA/PEGDA@T + exos MNs patch. As a result, our GelMA/PEGDA@T + exos MNs patch presents a potentially valuable method for repairing diabetic wound in clinical applications.


Subject(s)
Diabetes Mellitus , Exosomes , Animals , Gelatin/pharmacology , Mice , Nicotinic Acids , Wound Healing
17.
Int J Biol Macromol ; 202: 657-670, 2022 Mar 31.
Article in English | MEDLINE | ID: mdl-35066024

ABSTRACT

Chronic non-healing diabetic wounds and ulcers can be fatal, lead to amputations, and remain a major challenge to medical, and health care sectors. Susceptibility to infection and impaired angiogenesis are two central reasons for the clinical consequences associated with chronic non-healing diabetic wounds. Herein, we successfully developed calcium ion (Ca2+) cross-linked sodium alginate (SA) hydrogels with both pro-angiogenesis and antibacterial properties. Our results demonstrated that deferoxamine (DFO) and copper nanoparticles (Cu-NPs) worked synergistically to enhance the proliferation, migration, and angiogenesis of human umbilical venous endothelial cells in vitro. Results of colony formation assay indicated Cu-NPs were effective against E. coli and S. aureus in a dose-dependent manner in vitro. An SA hydrogel containing both DFO and Cu-NPs (SA-DFO/Cu) was prepared using a Ca2+ cross-linking method. Cytotoxicity assay and colony formation assay indicated that the hydrogel exhibited beneficial biocompatible and antibacterial properties in vitro. Furthermore, SA-DFO/Cu significantly accelerated diabetic wound healing, improved angiogenesis and reduced long-lasting inflammation in a mouse model of diabetic wound. Mechanistically, DFO and Cu-NPs synergistically stimulated the levels of hypoxia-inducible factor 1α and vascular endothelial growth factor in vivo. Given the pro-angiogenesis, antibacterial and healing properties, the hydrogel possesses high potential for clinical application in refractory wounds.


Subject(s)
Diabetes Mellitus , Nanoparticles , Alginates , Animals , Calcium , Copper , Deferoxamine/pharmacology , Escherichia coli , Human Umbilical Vein Endothelial Cells , Humans , Hydrogels/pharmacology , Mice , Staphylococcus aureus , Vascular Endothelial Growth Factor A , Wound Healing
18.
IEEE Trans Neural Netw Learn Syst ; 33(11): 6584-6598, 2022 Nov.
Article in English | MEDLINE | ID: mdl-34101599

ABSTRACT

In reinforcement learning (RL), function approximation errors are known to easily lead to the Q -value overestimations, thus greatly reducing policy performance. This article presents a distributional soft actor-critic (DSAC) algorithm, which is an off-policy RL method for continuous control setting, to improve the policy performance by mitigating Q -value overestimations. We first discover in theory that learning a distribution function of state-action returns can effectively mitigate Q -value overestimations because it is capable of adaptively adjusting the update step size of the Q -value function. Then, a distributional soft policy iteration (DSPI) framework is developed by embedding the return distribution function into maximum entropy RL. Finally, we present a deep off-policy actor-critic variant of DSPI, called DSAC, which directly learns a continuous return distribution by keeping the variance of the state-action returns within a reasonable range to address exploding and vanishing gradient problems. We evaluate DSAC on the suite of MuJoCo continuous control tasks, achieving the state-of-the-art performance.

19.
Front Oncol ; 11: 625257, 2021.
Article in English | MEDLINE | ID: mdl-34532281

ABSTRACT

Gastric cancer (GC) is the second most common cancer and the third most frequent cause of cancer-related deaths in China. E2Fs are a family of transcription factors reported to be involved in the tumor progression of various cancer types; however, the roles of individual E2Fs are still not known exactly in tumor progression of GC. In this study, we examined the expression of E2Fs to investigate their roles in tumor progression in GC patients using multiple databases, including ONCOMINE, GEPIA2, Kaplan-Meier plotter, cBioPortal, Metascape, LinkedOmics, GeneMANIA, STRING and UCSC Xena. We also performed real-time polymerase chain reaction (RT-PCR) to validate the expression levels of individual E2Fs in several GC cell lines. Our results demonstrated that the mRNA levels of E2F1/2/3/5/8 were significantly higher both in GC tissues and cell lines. The expression levels of E2F1 and E2F4 were correlated with poor overall survival (OS), decreased post-progression survival (PPS), and decreased progression-free survival (FP) in patients with GC. However, overexpression of E2F2, E2F5, E2F7 and E2F8 is significantly associated with disease-free survival and overall survival in patients with GC. In addition, higher E2F3 and E2F6 mRNA expression was found to increase GC patients' OS and PPS. 224 of 415 patients with STAD (54%) had gene mutations that were associated with longer disease-free survival (DFS) but not OS. Cell cycle pathway was closely associated with mRNA level of more than half of E2Fs (E2F1/2/3/7/8). There were close and complicated interactions among E2F family members. Finally, our results indicated the gene expressions of E2Fs had a positive relationship with its copy numbers. Taken together, E2F1/2/3/5/8 can serve as biomarkers for GC patients with high prognostic value for OS of GC patients or therapeutic targets for GC.

20.
Ear Nose Throat J ; 100(4): 260-270, 2021 May.
Article in English | MEDLINE | ID: mdl-33570429

ABSTRACT

BACKGROUND: The impact of obstructive sleep apnea (OSA) on subsequent cardiovascular events in patients with acute coronary syndrome (ACS) remains inconclusive. AIM: Our aim was to systematically assess the relationship between preexisting OSA and adverse cardiovascular events in patients with newly diagnosed ACS by conducting a systematic review and meta-analysis. METHODS: We systematically searched PubMed, EMBASE, and Cochrane library for studies published up to May 1, 2020, that reported any association between OSA and cardiovascular events in patients with newly diagnosed ACS. The main outcomes were a composite of all-cause or cardiovascular death, recurrent myocardial infarction, stroke, repeat revascularization, or heart failure. We conducted a pooled analysis using the random-effects model. We also performed subgroup, sensitivity, heterogeneity analysis, and the assessment of publication bias. RESULTS: We identified 10 studies encompassing 3350 participants. The presence of OSA was associated with increased risk of adverse cardiovascular events in newly prognosed ACS (risk ratio [RR] 2.18, 95% confidence interval [CI]: 1.45-3.26, P < .001, I2 = 64%). Between-study heterogeneity was partially explained by a multicenter study (9 single-center studies, RR 2.33 95% CI 1.69-3.19, I2 =18%), and I2 remarkably decreased from 64% to 18%. Moreover, OSA significantly increased the incidence of repeat revascularization (8 studies) and heart failure (6 studies) in patients with newly diagnosed ACS. CONCLUSION: Patients with preexisting OSA are at greater risk of subsequent cardiovascular events after onset of ACS. Further studies should investigate the treatment of OSA in patient with ACS.


Subject(s)
Acute Coronary Syndrome/complications , Cardiovascular Diseases/epidemiology , Cardiovascular Diseases/etiology , Percutaneous Coronary Intervention/adverse effects , Sleep Apnea, Obstructive/complications , Aged , Female , Heart Disease Risk Factors , Heart Failure/epidemiology , Heart Failure/etiology , Humans , Incidence , Male , Middle Aged , Observational Studies as Topic , Recurrence
SELECTION OF CITATIONS
SEARCH DETAIL