Urban Determinants of COVID-19 Spread: a Comparative Study across Three Cities in New York State.

The ongoing pandemic is laying bare dramatic differences in the spread of COVID-19 across seemingly similar urban environments. Identifying the urban determinants that underlie these differences is an open research question, which can contribute to more epidemiologically resilient cities, optimized testing and detection strategies, and effective immunization efforts. Here, we perform a computational analysis of COVID-19 spread in three cities of similar size in New York State (Colonie, New Rochelle, and Utica) aiming to isolate urban determinants of infections and deaths. We develop detailed digital representations of the cities and simulate COVID-19 spread using a complex agent-based model, taking into account differences in spatial layout, mobility, demographics, and occupational structure of the population. By critically comparing pandemic outcomes across the three cities under equivalent initial conditions, we provide compelling evidence in favor of the central role of hospitals. Specifically, with highly efficacious testing and detection, the number and capacity of hospitals, as well as the extent of vaccination of hospital employees are key determinants of COVID-19 spread. The modulating role of these determinants is reduced at lower efficacy of testing and detection, so that the pandemic outcome becomes equivalent across the three cities.

Human motor learning is robust to control-dependent noise.

Noises are ubiquitous in sensorimotor interactions and contaminate the information provided to the central nervous system (CNS) for motor learning. An interesting question is how the CNS manages motor learning with imprecise information. Integrating ideas from reinforcement learning and adaptive optimal control, this paper develops a novel computational mechanism to explain the robustness of human motor learning to the imprecise information, caused by control-dependent noise that exists inherently in the sensorimotor systems. Starting from an initial admissible control policy, in each learning trial the mechanism collects and uses the noisy sensory data (caused by the control-dependent noise) to form an imprecise evaluation of the performance of the current policy and then constructs an updated policy based on the imprecise evaluation. As the number of learning trials increases, the generated policies mathematically provably converge to a (potentially small) neighborhood of the optimal policy under mild conditions, despite the imprecise information in the learning process. The mechanism directly synthesizes the policies from the sensory data, without identifying an internal forward model. Our preliminary computational results on two classic arm reaching tasks are in line with experimental observations reported in the literature. The model-free control principle proposed in the paper sheds more lights into the inherent robustness of human sensorimotor systems to the imprecise information, especially control-dependent noise, in the CNS.

Prediction of internal modification size in glass induced by ultrafast laser scanning.

The modification at the interface between glass plates induced by ultrafast laser is important for the glass welding strength, therefore the relationship between the modification size and processing parameters should be identified. The experimental method has its limitation in understanding the nature of the modification. In this study, a numerical model for the temperature distribution determining the modification size induced by ultrafast laser scanning is developed, in which a three-dimensional steady model for the beam propagation with a transient ionization model is established to estimate the free electron density by the single laser pulse, and then a heat accumulation model for multiple laser pulses is employed to describe energy transportation within the irradiated bulk. The experiment for the internal modifications in single-piece fused silica samples irradiated by a picosecond laser with different pulse energies and scanning velocities is performed to validate the present model.

Model-Free Robust Optimal Feedback Mechanisms of Biological Motor Control.

Sensorimotor tasks that humans perform are often affected by different sources of uncertainty. Nevertheless, the central nervous system (CNS) can gracefully coordinate our movements. Most learning frameworks rely on the internal model principle, which requires a precise internal representation in the CNS to predict the outcomes of our motor commands. However, learning a perfect internal model in a complex environment over a short period of time is a nontrivial problem. Indeed, achieving proficient motor skills may require years of training for some difficult tasks. Internal models alone may not be adequate to explain the motor adaptation behavior during the early phase of learning. Recent studies investigating the active regulation of motor variability, the presence of suboptimal inference, and model-free learning have challenged some of the traditional viewpoints on the sensorimotor learning mechanism. As a result, it may be necessary to develop a computational framework that can account for these new phenomena. Here, we develop a novel theory of motor learning, based on model-free adaptive optimal control, which can bypass some of the difficulties in existing theories. This new theory is based on our recently developed adaptive dynamic programming (ADP) and robust ADP (RADP) methods and is especially useful for accounting for motor learning behavior when an internal model is inaccurate or unavailable. Our preliminary computational results are in line with experimental observations reported in the literature and can account for some phenomena that are inexplicable using existing models.

NEAT1/miR-193a-3p/SOX5 axis regulates cartilage matrix degradation in human osteoarthritis.

Long non-coding RNAs (lncRNAs) were reported to be involved in the progression of osteoarthritis (OA). The aim of this work was to explore the functional role of lncRNA nuclear-enriched abundant transcript 1 (NEAT1) in OA. Reverse-transcription quantitative polymerase chain reaction (RT-qPCR) was employed to analyze the expression of microRNA (miR-193a)-3p, NEAT1, and sex-determining region Y-box protein 5 (SOX5), as well as the levels of pro-inflammatory cytokines interleukin-6 (IL-6), IL-1ß, tumor necrosis factor-α (TNF-α), and IL-8 in OA cartilage tissue and chondrocytes. In addition, flow cytometry was used to measure the apoptosis of chondrocytes. The protein levels of extracellular matrix ACAN, collagen type II α1 chain (Col2a1), matrix metalloproteinase-3 (MMP-3), MMP-13, a disintegrin, and metalloproteinase with thrombospondin motifs (ADAMTS)-5 and SOX5 were determined using western blot analysis. Dual-luciferase reporter assay was performed to determine the target relationship among NEAT1, miR-193a-3p, and SOX5. We found that miR-193a-3p expression was downregulated, while NEAT1 and SOX5 were upregulated in OA cartilage tissue and chondrocytes. Both upregulation of miR-193a-3p and knockdown of NEAT1 suppressed inflammation, apoptosis, and reduced the protein levels of MMP-3, MMP-13, and ADAMTS-5, while elevating ACAN and Col2a1 expression in chondrocytes. NEAT1 targeted miR-193a-3p, and SOX5 was targeted by miR-193a-3p. Silencing of miR-193a-3p reversed the NEAT1 knockdown-mediated effect on the inflammation, apoptosis, and production of the extracellular matrix. The introduction of SOX5 abolished the impact of the upregulation of miR-193a-3p on inflammation, apoptosis, and production of extracellular matrix in chondrocytes. In conclusion, NEAT1/miR-193a-3p/SOX5 axis regulates cartilage matrix degradation in human OA.

Agallolides A-M, including two rearranged ent-atisanes featuring a bicyclo[3.2.1]octane motif, from the Chinese Excoecaria agallocha.

Thirteen new diterpenoid compounds, named agallolides A-M (1-13), including ten ent-atisanes, were isolated from the stems and twigs of the Chinese semi-mangrove plant, Excoecaria agallocha. Most notably, agallolides A (1) and B (2) are two rearranged ent-atisanes featuring a unique 6/6/5/7 tetracyclic carbon skeleton. Agallolides C-J (3-10) are ent-atisanes, among which agallolide C (3) represents the first example of 3,4-seco-17-nor-ent-atisane. Agallolides K (11) and L (12) are two ent-isopimaranes, whereas agallolide M (13) is a rare 3,4-seco-ent-trachylobane. The structures of these diterpenoid compounds were established by HR-ESIMS and extensive 1D and 2D NMR investigations. The absolute configurations of agallolide A (1) and agallolides I-K (9-11) were further confirmed by single-crystal X-ray diffraction analyses with Cu Kα radiation. The plausible biogenetic pathways for agallolides A (1), B (2), and I (9) were proposed. Agallolides I (9) and J (10) exhibited NF-κB inhibitory activity with inhibition rates of 23.4% and 19.4%, respectively, at the concentration of 100.0 µM.

Mangrove Tirucallane- and Apotirucallane-Type Triterpenoids: Structure Diversity of the C-17 Side-Chain and Natural Agonists of Human Farnesoid/Pregnane⁻X⁻Receptor.

Ten new triterpenoid compounds with structure diversity of the C-17 side-chain, including nine tirucallanes, named xylocarpols A⁻E (1⁻5) and agallochols A⁻D (6⁻9), and an apotirucallane, named 25-dehydroxy protoxylogranatin B (10), were isolated from the mangrove plants Xylocarpus granatum, Xylocarpus moluccensis, and Excoecaria agallocha. The structures of these compounds were established by HR-ESIMS and extensive one-dimensional (1D) and two-dimensional (2D) NMR investigations. The absolute configurations of 1 and 2 were unequivocally determined by single-crystal X-ray diffraction analyses, conducted with Cu Kα radiation; whereas those of 4, 6⁻8 were assigned by a modified Mosher's method and the comparison of experimental electronic circular dichroism (ECD) spectra. Most notably, 5, 6, 7, and 9 displayed potent activation effects on farnesoid⁻X⁻receptor (FXR) at the concentration of 10.0 µM; 10 exhibited very significant agonistic effects on pregnane⁻X⁻receptor (PXR) at the concentration of 10.0 nM.

Antiviral Limonoids Including Khayanolides from the Trang Mangrove Plant Xylocarpus moluccensis.

Eight new khayanolides, named thaixylomolins G-N (1-8), two new phragmalins (9 and 10), and two new mexicanolides (11 and 12) were obtained from the seeds of the Trang mangrove plant Xylocarpus moluccensis. The absolute configurations of these limonoids, except for the stereocenter at C-6 of 11 and 12, were assigned by experimental and TDDFT calculated electronic circular dichroism spectra. The khayanolides may be classified into two subclasses, one of which has a C-2 carbonyl and a 3ß-acetoxy group, whereas the other possesses a 2ß-acetoxy and a C-3 carbonyl function. Khayanolides, rearranged phragmalin-type limonoids, are reported for the first time from plants of the mangrove genus Xylocarpus. The structure of moluccensin J, a known 30-ketophragmalin containing a Δ(8(14)) double bond, was revised to be a khayanolide, named thaixylomolin K. The antiviral activities of the isolates against pandemic influenza A virus (subtype H1N1) were tested by the assay of cytopathic effect inhibition. Three khayanolides, viz., thaixylomolins I, K, and M, exhibited moderate anti-H1N1 activities. The most potent one, thaixylomolin I (IC50 = 77.1 ± 8.7 µM), showed stronger inhibitory activity than that of the positive control, ribavirin (IC50 = 185.9 ± 16.8 µM).

Adaptive dynamic programming as a theory of sensorimotor control.

Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.

Marine fungus Aspergillus c1. sp metabolite activates the HSF1/PGC-1α axis, inducing a thermogenic program for treating obesity.

Background and aims: Obesity is one of the most prevalent diseases worldwide with less ideal approved agents in clinic. Activating the HSF1/PGC-1α axis in adipose tissues has been reported to induce thermogenesis in mice, which presents a promising therapeutic avenue for obesity treatment. The present study aimed to identified novel natural HSF1 activator and evaluated the therapeutic effects of the newly discovered compound on obesity-associated metabolic disorders and the molecular mechanisms of these effects. Methods: Our previous reported HSF1/PGC-1α activator screening system was used to identify novel natural HSF1 activator. The PGC-1α luciferase activity, immunoblot, protein nuclear-translocation, immunofluorescence, chromatin immunoprecipitation assays were used to evaluate the activity of compound HN-001 in activating HSF1. The experiments of mitochondrial number measurement, TG assay and imaging, cellular metabolic assay, gene assays, and CRISPR/Cas 9 were applied for investigating the metabolic effect of HN-001 in C3H10-T1/2 adipocytes. The in vivo anti-obesity efficacies and beneficial metabolic effects of HN-001 were evaluated by performing body and fat mass quantification, plasma chemical analysis, GTT, ITT, cold tolerance test, thermogenesis analysis. Results: HN-001 dose- and time-dependently activated HSF1 and induced HSF1 nuclear translocation, resulting in an enhancement in binding with the gene Pgc-1α. This improvement induced activation of adipose thermogenesis and enhancement of mitochondrial oxidation capacity, thus inhibiting adipocyte maturation. Deletion of HSF1 in adipocytes impaired mitochondrial oxidation and abolished the above beneficial metabolic effects of HN-001, including adipocyte browning induction, improvements in mitogenesis and oxidation capacity, and lipid-lowering ability. In mice, HN-001 treatment efficiently alleviated diet-induced obesity and metabolic disorders. These changes were associated with increased body temperature in mice and activation of the HSF1/PGC-1α axis in adipose tissues. UCP1 expression and mitochondrial biogenesis were increased in both white and brown adipose tissues of HN-001-treated mice. Conclusion: These data indicate that HN-001 may have therapeutic potential for obesity-related metabolic diseases by increasing the capacity of energy expenditure in adipose tissues through a mechanism involving the HSF1/PGC-1α axis, which shed new light on the development of novel anti-obesity agents derived from marine sources.

Identification of a natural PLA2 inhibitor from the marine fungus Aspergillus sp. c1 for MAFLD treatment that suppressed lipotoxicity by inhibiting the IRE-1α/XBP-1s axis and JNK signaling.

Lipotoxicity is a pivotal factor that initiates and exacerbates liver injury and is involved in the development of metabolic-associated fatty liver disease (MAFLD). However, there are few reported lipotoxicity inhibitors. Here, we identified a natural anti-lipotoxicity candidate, HN-001, from the marine fungus Aspergillus sp. C1. HN-001 dose- and time- dependently reversed palmitic acid (PA)-induced hepatocyte death. This protection was associated with IRE-1α-mediated XBP-1 splicing inhibition, which resulted in suppression of XBP-1s nuclear translocation and transcriptional regulation. Knockdown of XBP-1s attenuated lipotoxicity, but no additional ameliorative effect of HN-001 on lipotoxicity was observed in XBP-1s knockdown hepatocytes. Notably, the ER stress and lipotoxicity amelioration was associated with PLA2. Both HN-001 and the PLA2 inhibitor MAFP inhibited PLA2 activity, reduced lysophosphatidylcholine (LPC) level, subsequently ameliorated lipotoxicity. In contrast, overexpression of PLA2 caused exacerbation of lipotoxicity and weakened the anti-lipotoxic effects of HN-001. Additionally, HN-001 treatment suppressed the downstream pro-apoptotic JNK pathway. In vivo, chronic administration of HN-001 (i.p.) in mice alleviated all manifestations of MAFLD, including hepatic steatosis, liver injury, inflammation, and fibrogenesis. These effects were correlated with PLA2/IRE-1α/XBP-1s axis and JNK signaling suppression. These data indicate that HN-001 has therapeutic potential for MAFLD because it suppresses lipotoxicity, and provide a natural structural basis for developing anti-MAFLD candidates.

Ent-eudesmane sesquiterpenoids with anti-neuroinflammatory activity from the marine-derived fungus Eutypella sp. F0219.

In this study, twenty-three ent-eudesmane sesquiterpenoids (1-23) including fifteen previously undescribed ones, named eutypelides A-O (1-15) were isolated from the marine-derived fungus Eutypella sp. F0219. Their planar structures and relative configurations were established by HR-ESIMS and extensive 1D and 2D NMR investigations. The absolute configurations of the previously undescribed compounds were determined by single-crystal X-ray diffraction analyses, modified Mosher's method, and ECD calculations. Structurally, eutypelide A (1) is a rare 1,10-seco-ent-eudesmane, whereas 2-15 are typically ent-eudesmanes with 6/6/-fused bicyclic carbon nucleus. The anti-neuroinflammatory activity of all isolated compounds (1-23) was accessed based on their ability to NO production in LPS-stimulated BV2 microglia cells. Compound 16 emerged as the most potent inhibitor. Further mechanistic investigation revealed that compound 16 modulated the inflammatory response by decreasing the protein levels of iNOS and increasing ARG 1 levels, thereby altering the iNOS/ARG 1 ratio and inhibiting macrophage polarization. qRT-PCR analysis showed that compound 16 reversed the LPS-induced upregulation of pro-inflammatory cytokines, including iNOS, TNF-α, IL-6, and IL-1ß, at both the transcriptional and translational levels. These effects were linked to the inhibition of the NF-κB pathway, a key regulator of inflammation. Our findings suggest that compound 16 may be a potential structure basis for developing neuroinflammation-related disease therapeutic agents.

Movement duration, Fitts's law, and an infinite-horizon optimal feedback control model for biological motor systems.

Optimization models explain many aspects of biological goal-directed movements. However, most such models use a finite-horizon formulation, which requires a prefixed movement duration to define a cost function and solve the optimization problem. To predict movement duration, these models have to be run multiple times with different prefixed durations until an appropriate duration is found by trial and error. The constrained minimum time model directly predicts movement duration; however, it does not consider sensory feedback and is thus applicable only to open-loop movements. To address these problems, we analyzed and simulated an infinite-horizon optimal feedback control model, with linear plants, that contains both control-dependent and control-independent noise and optimizes steady-state accuracy and energetic costs per unit time. The model applies the steady-state estimator and controller continuously to guide an effector to, and keep it at, target position. As such, it integrates movement control and posture maintenance without artificially dividing them with a precise, prefixed time boundary. Movement pace is determined by the model parameters, and the duration is an emergent property with trial-to-trial variability. By considering the mean duration, we derived both the log and power forms of Fitts's law as different approximations of the model. Moreover, the model reproduces typically observed velocity profiles and occasional transient overshoots. For unbiased sensory feedback, the effector reaches the target without bias, in contrast to finite-horizon models that systematically undershoot target when energetic cost is considered. Finally, the model does not involve backward and forward sweeps in time, its stability is easily checked, and the same solution applies to movements of different initial conditions and distances. We argue that biological systems could use steady-state solutions as default control mechanisms and might seek additional optimization of transient costs when justified or demanded by task or context.

Reducing Urban Traffic Congestion Using Deep Learning and Model Predictive Control.

This article proposes a deep learning (DL)-based control algorithm-DL velocity-based model predictive control (VMPC)-for reducing traffic congestion with slowly time-varying traffic signal controls. This control algorithm consists of system identification using DL and traffic signal control using VMPC. For the training process of DL, we established a modeling error entropy loss as the criteria inspired by the theory of stochastic distribution control (SDC) originated by the fourth author. Simulation results show that the proposed algorithm can reduce traffic congestion with a slowly varying traffic signal control input. Results of an ablation study demonstrate that this algorithm compares favorably to other model-based controllers in terms of prediction error, signal varying speed, and control effectiveness.

Exploring a COVID-19 Endemic Scenario: High-Resolution Agent-Based Modeling of Multiple Variants.

Our efforts as a society to combat the ongoing COVID-19 pandemic are continuously challenged by the emergence of new variants. These variants can be more infectious than existing strains and many of them are also more resistant to available vaccines. The appearance of these new variants cause new surges of infections, exacerbated by infrastructural difficulties, such as shortages of medical personnel or test kits. In this work, a high-resolution computational framework for modeling the simultaneous spread of two COVID-19 variants: a widely spread base variant and a new one, is established. The computational framework consists of a detailed database of a representative U.S. town and a high-resolution agent-based model that uses the Omicron variant as the base variant and offers flexibility in the incorporation of new variants. The results suggest that the spread of new variants can be contained with highly efficacious tests and mild loss of vaccine protection. However, the aggressiveness of the ongoing Omicron variant and the current waning vaccine immunity point to an endemic phase of COVID-19, in which multiple variants will coexist and residents continue to suffer from infections.

Advanced QuEChERS Method Using Core-Shell Magnetic Molecularly Imprinted Polymers (Fe3O4@MIP) for the Determination of Pesticides in Chlorophyll-Rich Samples.

Graphitized carbon black (GCB) in the traditional QuEChERS (quick, easy, cheap, effective, rugged, and safe) method was used to remove the interfering substance chlorophyll in vegetable and fruit samples for pesticide residues determination. However, it not only adsorbs pigments, but also adsorbs some planar and aromatic pesticides. In order to solve the shortcoming, a core-shell magnetic molecularly imprinted polymer (Fe3O4@MIP) that can specifically recognize and adsorb chlorophyll was synthesized, and an advanced QuEChERS method with the Fe3O4@MIP as a purification material was developed. This advanced method presents detection that is highly sensitive, specific, and reproducible for planar and aromatic pesticides. The limits of detection (LOD) ranged from 0.001-0.002 mg kg-1, and the limit of quantification (LOQ) was 0.005 mg kg-1. The recovery for the planar and aromatic pesticides was within 70-110% with the associated relative standard deviations < 15% in leek samples by the advanced QuEChERS method. However, in the traditional QuEChERS method with GCB, the recovery of most planar and aromatic pesticides was <60%. It may also be useful for the determination of other pesticides in vegetable samples with quick and easy sample purification.

Hainanxylogranolides A-F: New Limonoids isolated from the seeds of Hainan mangrove plant Xylocarpus granatum.

Six new limonoids, named hainanxylogranolides A-F (1-6), together with nineteen known ones (7-25) were isolated from the seeds of a Hainan mangrove Xylocarpus granatum. The structures of the new compounds were established by extensive NMR spectroscopic data combined with the DFT and TDDFT calculated electronic circular dichroism spectra. Hainanxylogranolide A (1) is the aromatic B-ring limonoid containing a central pyridine ring and a C-17 substituted γ(21)-hydroxybutenolide moiety. Hainanxylogranolide B (2) belongs to the small group of mexicanolides containing a C3-O-C8 bridge, whereas hainanxylogranolides C and D (3 and 4) are mexicanolides comprising a C1-O-C8 bridge. Compounds 9 and 25 posed obvious inhibition effect on the tube formation of HUVECs. There are only about 25% tube-like structures were observed at the concentration of 40.0 µM of compound 25. The antiviral activities of the isolates against herpes simplex virus-1 (HSV-1) and severe fever with thrombocytopenia syndrome virus (SFTSV) were tested in vitro. Compound 23 exhibited moderate anti-SFTSV activity with the IC50 value of 29.58 ± 0.73 µM. This is the first report of anti-angiogenic effect and anti-SFTSV activity of limonoids from the genus Xylocarpus.

Reinforcement Learning and Adaptive Optimal Control for Continuous-Time Nonlinear Systems: A Value Iteration Approach.

This article studies the adaptive optimal control problem for continuous-time nonlinear systems described by differential equations. A key strategy is to exploit the value iteration (VI) method proposed initially by Bellman in 1957 as a fundamental tool to solve dynamic programming problems. However, previous VI methods are all exclusively devoted to the Markov decision processes and discrete-time dynamical systems. In this article, we aim to fill up the gap by developing a new continuous-time VI method that will be applied to address the adaptive or nonadaptive optimal control problems for continuous-time systems described by differential equations. Like the traditional VI, the continuous-time VI algorithm retains the nice feature that there is no need to assume the knowledge of an initial admissible control policy. As a direct application of the proposed VI method, a new class of adaptive optimal controllers is obtained for nonlinear systems with totally unknown dynamics. A learning-based control algorithm is proposed to show how to learn robust optimal controllers directly from real-time data. Finally, two examples are given to illustrate the efficacy of the proposed methodology.

Learning-Based Adaptive Optimal Control for Connected Vehicles in Mixed Traffic: Robustness to Driver Reaction Time.

Through vehicle-to-vehicle (V2V) communication, both human-driven and autonomous vehicles can actively exchange data, such as velocities and bumper-to-bumper distances. Employing the shared data, control laws with improved performance can be designed for connected and autonomous vehicles (CAVs). In this article, taking into account human-vehicle interaction and heterogeneous driver behavior, an adaptive optimal control design method is proposed for a platoon mixed with multiple preceding human-driven vehicles and one CAV at the tail. It is shown that by using reinforcement learning and adaptive dynamic programming techniques, a near-optimal controller can be learned from real-time data for the CAV with V2V communications, but without the precise knowledge of the accurate car-following parameters of any driver in the platoon. The proposed method allows the CAV controller to adapt to different platoon dynamics caused by the unknown and heterogeneous driver-dependent parameters. To improve the safety performance during the learning process, our off-policy learning algorithm can leverage both the historical data and the data collected in real time, which leads to considerably reduced learning time duration. The effectiveness and efficiency of our proposed method is demonstrated by rigorous proofs and microscopic traffic simulations.

Reinforcement Learning-Based Cooperative Optimal Output Regulation via Distributed Adaptive Internal Model.

In this article, a data-driven distributed control method is proposed to solve the cooperative optimal output regulation problem of leader-follower multiagent systems. Different from traditional studies on cooperative output regulation, a distributed adaptive internal model is originally developed, which includes a distributed internal model and a distributed observer to estimate the leader's dynamics. Without relying on the dynamics of multiagent systems, we have proposed two reinforcement learning algorithms, policy iteration and value iteration, to learn the optimal controller through online input and state data, and estimated values of the leader's state. By combining these methods, we have established a basis for connecting data-distributed control methods with adaptive dynamic programming approaches in general since these are the theoretical foundation from which they are built.