Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation.

Modeling human motor control and predicting how humans will move in novel environments is a grand scientific challenge. Researchers in the fields of biomechanics and motor control have proposed and evaluated motor control models via neuromechanical simulations, which produce physically correct motions of a musculoskeletal model. Typically, researchers have developed control models that encode physiologically plausible motor control hypotheses and compared the resulting simulation behaviors to measurable human motion data. While such plausible control models were able to simulate and explain many basic locomotion behaviors (e.g. walking, running, and climbing stairs), modeling higher layer controls (e.g. processing environment cues, planning long-term motion strategies, and coordinating basic motor skills to navigate in dynamic and complex environments) remains a challenge. Recent advances in deep reinforcement learning lay a foundation for modeling these complex control processes and controlling a diverse repertoire of human movement; however, reinforcement learning has been rarely applied in neuromechanical simulation to model human control. In this paper, we review the current state of neuromechanical simulations, along with the fundamentals of reinforcement learning, as it applies to human locomotion. We also present a scientific competition and accompanying software platform, which we have organized to accelerate the use of reinforcement learning in neuromechanical simulations. This "Learn to Move" competition was an official competition at the NeurIPS conference from 2017 to 2019 and attracted over 1300 teams from around the world. Top teams adapted state-of-the-art deep reinforcement learning techniques and produced motions, such as quick turning and walk-to-stand transitions, that have not been demonstrated before in neuromechanical simulations without utilizing reference motion data. We close with a discussion of future opportunities at the intersection of human movement simulation and reinforcement learning and our plans to extend the Learn to Move competition to further facilitate interdisciplinary collaboration in modeling human motor control for biomechanics and rehabilitation research.

[Distribution of HCV genotypes in Chinese Han population with chronic hepatitis C].

OBJECTIVE: To investigate the distribution of HCV genotypes in Chinese Han population with chronic hepatitis C (CHC). METHODS: This randomized multicenter study included 1 014 CHC patients from 28 hospitals in different regions of China. SPSS 20.0 was applied to analyze the relationship among region, HCV genotype, gender and the replication level of HCV-RNA. RESULTS: HCV 1 genotype (56.80%) was the most common genotype. The majority of CHC patients were of genotype 1, 2, 3, 6 in the order of frequency, except those in southwestern, southern and central China. HCV 1, 2, 3, 6 genotypes were most common among male patients in southern China; among female patients in northern China; among male patients in northern and northwestern China and among male patients in northwestern China, respectively (all P <0.05). There was no statistical significance between different genders in other regions. The high viral load was more common than the low viral load among HCV 1, 2, 3, 6 genotype-infected patients. CONCLUSION: There are different distributions of HCV genotypes among the different regions. In addition, HCV genotypes are correlated with gender and HCV-RNA load.

The intestinal microbial community dissimilarity in hepatitis B virus-related liver cirrhosis patients with and without at alcohol consumption.

BACKGROUND: Chronic hepatitis B virus (HBV) infection-reduced liver functions are associated with intestinal microbial community dissimilarity. This study aimed to investigate the microbial community dissimilarity in patients with different grades of HBV-related liver cirrhosis. RESULTS: Serum endotoxin was increased with Child-Pugh (CP) class (A, B, and C). Veillonellaceae and Lachnospiraceae families were reduced in patients compared with controls. Megamonas and Veillonella genus was reduced and increased in patients compared with controls, respectively, especially in CPB and CPC groups. Correlation analysis showed that endotoxin content was significantly correlated with alcohol consumption (95% CI 0.100, 0.493), CP class (95% CI 0.289, 0.687) and Lachnospiraceae family level (95% CI - 0.539, - 0.122). Firmicutes/Bacteroidetes ratio was correlated with the level of Lachnospiraceae family (95% CI 0.013, 0.481), Veillonellaceae family (95% CI 0.284, 0.696), Megamonas genus (95% CI 0.101, 0.518) and Veillonella genus (95% CI 0.134, 0.545). All aforementioned bacteria were independent risk or protective factors for hepatitis. Alcohol consumption changed microbial community. CONCLUSIONS: Our study demonstrated that elevated Firmicutes/Bacteroidetes ratio, reduced Megamonas genus level and increased Veillonella genus level were indicators for HBV-related liver cirrhosis. Alcohol-related pathogenesis was associated with the changed microbial community.

Human umbilical cord mesenchymal stem cells inhibit proliferation of hepatic stellate cells in vitro.

The effect of human umbilical cord mesenchymal stem cells (hUC-MSCs) on the proliferation of hepatic stellate cells (HSCs) is largely unknown. The purpose of this study was to explore the mechanism of action of hUC­MSCs on the proliferation of HSCs in vitro. The upper and lower double-cell co-culture system was established between hUC­MSCs and HSCs in the experimental group. HSCs were cultured alone as a negative control group. Cell proliferation and apoptosis were determined by 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide (MTT) assay and flow cytometry, respectively. Cell supernatants were harvested to determine the concentration of transforming growth factor-ß1 (TGF-ß1) by ELISA. mRNA and protein of TGF-ß1, Smad3 and Smad7 in HSCs were determined by reverse transcription-polymerase chain reaction and western blotting, respectively. In the co-culture group, the proliferation of HSCs was significantly inhibited compared with the negative control group at 24 and 48 h (p<0.05). Apoptosis of HSCs in the co-culture group increased compared with that in the negative control group, which was more obvious at 48 h (p<0.05). The concentration of TGF-ß1 in the co-culture group was significantly lower than in the HSCs cultured alone (p<0.05). After HSCs were co-cultured with hUC­MSCs for 48 h, expression of TGF-ß1 and Smad3 mRNA and protein was reduced and expression of Smad7 mRNA and protein was increased compared with the negative control group (p<0.05). hUC­MSCs inhibited proliferation of HSCs, possibly through inhibiting TGF-ß1 and Smad3 expression and increasing Smad7 protein expression.

Bone marrow-derived mesenchymal stem cells inhibit the proliferation of hepatic stellate cells by inhibiting the transforming growth factor ß pathway.

Bone marrow-derived mesenchymal stem cells (BM-MSCs) are considered to be a potential therapy for end-stage liver disease. However, the therapeutic mechanism of BM-MSCs remains unclear. The aim of the current study was to investigate the role of paracrine signaling in BM­MSCs in liver cirrhosis in vitro. Human BM­MSCs and hepatic stellate cells (HSCs) were cultured using a vertical double cell co­culture system. Groups were divided into HSCs alone (control group) and the co­culture system of BM­MSCs with HSCs (experimental group). HSC morphology was observed by inverted phase contrast microscopy. The proliferative capacity of HSCs was measured with the MTT assay and flow cytometry. Hoechst staining was performed to examine the apoptosis of HSCs. Transforming growth factor (TGF)­ß1 and Smad7 mRNA expression were detected by reverse transcription­quantitative polymerase chain reaction and western blotting. BM­MSCs did not inhibit the proliferation of HSCs at 24 h, however significantly inhibited the proliferation of HSCs at 48 and 72 h. BM­MSCs additionally induced the apoptosis of HSCs at 48 h. The concentration of TGF­ß1 in the supernatant at 24 h and 48 h in the co­cultured system was observed to be significantly lower than in the control group (P<0.05). The level of TGF­ß1 mRNA in the experimental group at 48 h was significantly lower than the control group, however Smad7 mRNA levels were significantly greater than in the control group. Additionally, TGF­ß1 protein levels were significantly lower than in the control group, however levels of Smad7 were greater than the control group. It was concluded that BM­MSCs are able to inhibit the proliferation and promote the apoptosis of HSCs. In addition, the mechanism may be associated with inhibition of the TGF-ß1/Smad pathway in HSCs.