Boosting Reinforcement Learning via Hierarchical Game Playing With State Relay.

Due to its wide application, deep reinforcement learning (DRL) has been extensively studied in the motion planning community in recent years. However, in the current DRL research, regardless of task completion, the state information of the agent will be reset afterward. This leads to a low sample utilization rate and hinders further explorations of the environment. Moreover, in the initial training stage, the agent has a weak learning ability in general, which affects the training efficiency in complex tasks. In this study, a new hierarchical reinforcement learning (HRL) framework dubbed hierarchical learning based on game playing with state relay (HGR) is proposed. In particular, we introduce an auxiliary penalty to regulate task difficulty, and one training mechanism, the state relay mechanism, is designed. The relay mechanism can make full use of the intermediate states of the agent and expand the environment exploration of low-level policy. Our algorithm can improve the sample utilization rate, reduce the sparse reward problem, and thereby enhance the training performance in complex environments. Simulation tests are carried out on two public experiment platforms, i.e., MazeBase and MuJoCo, to verify the effectiveness of the proposed method. The results show that HGR significantly benefits the reinforcement learning (RL) area.

Astrocyte-mediated regulation of BLAWFS1 neurons alleviates risk-assessment deficits in DISC1-N mice.

Assessing and responding to threats is vital in everyday life. Unfortunately, many mental illnesses involve impaired risk assessment, affecting patients, families, and society. The brain processes behind these behaviors are not well understood. We developed a transgenic mouse model (disrupted-in-schizophrenia 1 [DISC1]-N) with a disrupted avoidance response in risky settings. Our study utilized single-nucleus RNA sequencing and path-clamp coupling with real-time RT-PCR to uncover a previously undescribed group of glutamatergic neurons in the basolateral amygdala (BLA) marked by Wolfram syndrome 1 (WFS1) expression, whose activity is modulated by adjacent astrocytes. These neurons in DISC1-N mice exhibited diminished firing ability and impaired communication with the astrocytes. Remarkably, optogenetic activation of these astrocytes reinstated neuronal excitability via D-serine acting on BLAWFS1 neurons' NMDA receptors, leading to improved risk-assessment behavior in the DISC1-N mice. Our findings point to BLA astrocytes as a promising target for treating risk-assessment dysfunctions in mental disorders.

Influence maximization based on threshold models in hypergraphs.

Influence maximization problem has received significant attention in recent years due to its application in various domains, such as product recommendation, public opinion dissemination, and disease propagation. This paper proposes a theoretical analysis framework for collective influence in hypergraphs, focusing on identifying a set of seeds that maximize influence in threshold models. First, we extend the message passing method from pairwise networks to hypergraphs to accurately describe the activation process in threshold models. Then, we introduce the concept of hypergraph collective influence (HCI) to measure the influence of nodes. Subsequently, we design an algorithm, HCI-TM, to select the influence maximization set, taking into account both node and hyperedge activation. Numerical simulations demonstrate that HCI-TM outperforms several competing algorithms in synthetic and real-world hypergraphs. Furthermore, we find that HCI can be used as a tool to predict the occurrence of cascading phenomena. Notably, we find that the HCI-TM algorithm works better for larger average hyperdegrees in Erdös-Rényi hypergraphs and smaller power-law exponents in scale-free hypergraphs.

The Central Nervous Mechanism of Stress-Promoting Cancer Progression.

Evidence shows that stress can promote the occurrence and development of tumors. In recent years, many studies have shown that stress-related hormones or peripheral neurotransmitters can promote the proliferation, survival, and angiogenesis of tumor cells and impair the body's immune response, causing tumor cells to escape the "surveillance" of the immune system. However, the perception of stress occurs in the central nervous system (CNS) and the role of the central nervous system in tumor progression is still unclear, as are the underlying mechanisms. This review summarizes what is known of stress-related CNS-network activation during the stress response and the influence of the CNS on tumors and discusses available adjuvant treatment methods for cancer patients with negative emotional states, such as anxiety and depression.

Activation of dopaminergic VTA inputs to the mPFC ameliorates chronic stress-induced breast tumor progression.

AIMS: Chronic stress plays an important role in promoting the progression and migration of cancers. However, little is known of any direct impact on tumor progression related to the regulation of emotion-related circuitry. The aim of this study was to explore the neural-circuit mechanisms underlying stress-induced progression of cancers and the impact of emotion-related regulation of circuitry on tumor growth. METHODS: Optogenetic manipulation was applied to unpredictable chronic mild stress (UCMS)-treated mice bearing breast tumor cell. The stress-related hormones, tumor-related cytokines, the tyrosine hydroxylase (TH)-positive neurons and their fibers, dopamine receptor-positive cells, and anxiety level were measured using ELISA, immunohistochemical staining, fluorescence in situ hybridization, and behavioral test, respectively. RESULTS: By investigating breast cancer mouse models with a chronic mild stress model, optogenetic stimulation, and behavioral analysis, we show that chronic stress induced anxiety-like behavior in mice and increased serum concentration of norepinephrine and corticosterone, hormones closely related to stress and anxiety. Optogenetic activation of VTA TH terminals in the mPFC rescued anxiety-like behavior induced by chronic stress. Chronic stress resulted in marked progression of breast tumors, and repetitive optogenetic activation of VTA TH terminals in the mPFC significantly attenuated stress-induced progression of breast cancers and reduced serum concentration of norepinephrine and corticosterone. Furthermore, there was a positive correlation between serum norepinephrine or corticosterone concentration and tumor size. CONCLUSIONS: These findings indicate a positive role of an emotion regulation circuit on the progression of breast cancer and reveal a link between stress, emotion regulation, and the progression of breast cancers. Our findings provide new insights pertinent to therapeutic interventions in the treatment of breast cancers.

Facile immobilization of graphene nanosheets onto PBO fibers via MOF-mediated coagulation strategy: Multifunctional interface with self-healing and ultraviolet-resistance performance.

The surface of poly (p-phenylene benzobisoxazole) (PBO) fibers with self-healing and ultraviolet (UV)-resistance performance play the key role in prolonging their service lifespan. Although great advances have been made in the single aspect of above two properties, integration of self-healing and anti-UV performance into the surface of PBO fiber is still a challenge. In this study, the coagulation strategy mediated by metal-organic framework (MOF) is proposed to construct the multifunctional surface of PBO fibers. The spindle-like iron (III)-based MOF (MIL-88B-NH2) nanocrystals are firstly immobilized onto the surface of PBO-COOH through hydrothermal reaction, then serving as the medium layer to further immobilize sufficient graphene oxide (GO) nanosheets. Benefitting from the favorable near-infrared (NIR, 808 nm) photothermal conversion performance of GO nanolayers, the monofilament composite-PBO@Fe-MIL-88B-NH2-GO-TPU (thermoplastic polyurethane) exhibited a stable and high self-healing efficiency (approximately 80%) within five cycle times. Meanwhile, the cooperative adsorption and shielding weaken effects of MOF-GO nanolayers enabled PBO fibers with excellent anti-UV properties that are superior to much reported literatures after 96 h aging time and eventually increased by 75% compared with untreated PBO fiber. In view of the varieties and multifunctionalities of MOFs and carbon nanomaterials, MOF-mediated coagulation strategy would provide guidance for preparing multifunctional composite materials.

Chronic optogenetic manipulation of basolateral amygdala astrocytes rescues stress-induced anxiety.

Chronic exposure to stressors can disrupt normal brain function and induce anxiety-like behavior and neurobiological alterations in the basolateral amygdala (BLA). Here, we showed that unpredictable chronic mild stress (UCMS) induced anxiety-like behavior, lowered glutamatergic neuronal activity and reactive astrocytes in the BLA. Using optogenetic tools, we found that activation of BLA glutamatergic neurons did not rescue anxiety-like behavior in stressed mice. In contrast, however, optogenetic activation of the BLA astrocytes relieved stress-induced anxiety, and, interestingly, chronic optogenetic manipulation fully restored the UCMS-induced behavioral and neurobiological dysfunctions, including anxiety-like behavior, lower c-Fos expression in the BLA, S100 overexpression in the BLA, and higher serum corticosterone concentration. Thus, our findings suggest that chronic manipulation of BLA astrocytes is a potential therapeutic intervention target for pathological anxiety.

Construction of Anti-Ultraviolet "Shielding Clothes" on Poly( p-phenylene benzobisoxazole) Fibers: Metal Organic Framework-Mediated Absorption Strategy.

A metal-organic framework (MOF)-mediated adsorption strategy is first developed for improving the anti-ultraviolet (UV) properties of poly( p-phenylene benzobisoxazole) (PBO) fibers. In this work, UIO-66 was successfully anchored onto the surface of PBO fibers by one-step microwave-assisted heating method. The experimental results showed an obviously enhanced surface energy (91.1%), roughness (268.4%), interfacial shear strength (49.0%), and anti-UV properties (66.7%) compared to pristine PBO fibers. The anti-UV dye (tartrazine) was further immobilized onto the surface of PBO fibers via an adsorption strategy mediated by UIO-66. Interestingly, the PBO@tartrazine fibers demonstrated superior anti-UV performance (further up to 81.5%) compared to PBO@UIO-66 fibers. The extraordinary anti-UV properties of PBO@tartrazine fibers could be rationally ascribed to the synergistic effects of UIO-66 and tartrazine molecules. Considering the diversities and functionalities of MOFs and targeted materials, our work indicates that the MOF-mediated adsorption strategy would promisingly endow PBO fibers with other desired performance and applications such as solar-thermal transition and self-healing abilities.

Light triggered interfacial damage self-healing of poly(p-phenylene benzobisoxazole) fiber composites.

The interfacial microcracks in the resin matrix composites are difficult to be detected and repaired. However, the self-healing concept provides opportunities to fabricate composites with unusual properties. In the present study, photothermal conversion Ag-Cu2S nanoparticles were immobilized onto poly(p-phenylene benzobisoxazole) (PBO) fibers via a polydopamine chemistry. Benefitting from the photothermal effects of Ag-Cu2S, the obtained PBO fibers (Ag-Cu2S-PBO) efficiently converted the light energy into heat under Xenon lamp irradiation. Then, single PBO fiber composites were prepared using thermoplastic polyurethane as the matrix. It was found that the interfacial damage caused by single fiber pull-out was simply self-healed by Xe light irradiation. This wonderful interfacial damage self-healing property was mainly attributed to the in situ heating generation via photothermal effects of Ag-Cu2S in the composite interface. This paper reports a novel strategy to construct advanced composites with light-triggered self-healing properties, which will provide inspiration for preparing high performance composite materials.

Fabrication of a graphene/C60 nanohybrid via γ-cyclodextrin host-guest chemistry for photodynamic and photothermal therapy.

The wonderful chemical structures and characteristics of low-dimensional carbon materials have exciting applications in life sciences. In the present study, we developed a facile strategy to conjugate C60 with graphene via host-guest chemistry for targeted phototherapy. A versatile carrier based on folic acid-functionalized graphene (GO-FA) and comprising γ-cyclodextrin (γ-CD) at its surface was assembled via π-π interaction, creating hybrid structures with drug storage and tumor targeting properties. This γ-CD-modified graphene (GO-FA/Py-γ-CD) is capable of hosting pristine C60 molecules for the fabrication of a GO-FA/Py-γ-CD/C60 nanohybrid. The hybridization of GO-FA, γ-CD, and C60 hinders the aggregation of C60, promotes cellular uptake, enhances light absorption, and finally demonstrates enhanced phototherapy effects of GO-FA/Py-γ-CD/C60. Under Xe lamp irradiation (2 W cm-2) for 4 min, GO-FA/Py-γ-CD/C60 simultaneously causes heating and intracellular ROS production, which further significantly reduces the cell viability to 16.2% at low content of loading (30 µg mL-1). Moreover, it represents an excellent tumor killing efficiency, better than that of the other reported graphene/C60 nanohybrids; thus, this material is suitable for applications in phototherapy of cancer.