Human locomotion with reinforcement learning using bioinspired reward reshaping strategies.

Recent learning strategies such as reinforcement learning (RL) have favored the transition from applied artificial intelligence to general artificial intelligence. One of the current challenges of RL in healthcare relates to the development of a controller to teach a musculoskeletal model to perform dynamic movements. Several solutions have been proposed. However, there is still a lack of investigations exploring the muscle control problem from a biomechanical point of view. Moreover, no studies using biological knowledge to develop plausible motor control models for pathophysiological conditions make use of reward reshaping. Consequently, the objective of the present work was to design and evaluate specific bioinspired reward function strategies for human locomotion learning within an RL framework. The deep deterministic policy gradient (DDPG) method for a single-agent RL problem was applied. A 3D musculoskeletal model (8 DoF and 22 muscles) of a healthy adult was used. A virtual interactive environment was developed and simulated using opensim-rl library. Three reward functions were defined for walking, forward, and side falls. The training process was performed with Google Cloud Compute Engine. The obtained outcomes were compared to the NIPS 2017 challenge outcomes, experimental observations, and literature data. Regarding learning to walk, simulated musculoskeletal models were able to walk from 18 to 20.5 m for the best solutions. A compensation strategy of muscle activations was revealed. Soleus, tibia anterior, and vastii muscles are main actors of the simple forward fall. A higher intensity of muscle activations was also noted after the fall. All kinematics and muscle patterns were consistent with experimental observations and literature data. Regarding the side fall, an intensive level of muscle activation on the expected fall side to unbalance the body was noted. The obtained outcomes suggest that computational and human resources as well as biomechanical knowledge are needed together to develop and evaluate an efficient and robust RL solution. As perspectives, current solutions will be extended to a larger parameter space in 3D. Furthermore, a stochastic reinforcement learning model will be investigated in the future in scope with the uncertainties of the musculoskeletal model and associated environment to provide a general artificial intelligence solution for human locomotion learning. Graphical abstract.

Analysis of IDH mutation, 1p/19q deletion, and PTEN loss delineates prognosis in clinical low-grade diffuse gliomas.

BACKGROUND: Grades II and III gliomas have unpredictable rates of progression, making management decisions difficult. Currently, several clinical and radiological characteristics are utilized to predict progression and survival but collectively are suboptimal. METHODS: In this study, we analyzed a set of 108 nonenhancing hemispheric grade II-III gliomas. Demographic variables, including patient age, tumor diameter, extent of resection, and performance status, were combined with molecular data (IDH mutation status [mIDH], 1p/19q codeletion, PTEN deletion, and EGFR amplification). A complete dataset for all variables was compiled for 70 of the 108 patients. Both univariable and multivariable analyses were performed to determine whether the molecular data singly or in combination offer advantages over tumor type and grade for prediction of overall survival (OS) and/or progression-free rate (PFR). RESULTS: Patient age, clinical variables (tumor diameter, extent of resection, performance status), and pathology (tumor type and grade) were not predictive of OS or PFR. IDH mutation status alone was predictive of longer OS and PFR for the entire group of tumors; 1p/19q deletion alone was predictive of OS but not PFR. In the multivariable analysis, none of the clinical or demographic factors were predictive of OS or PFR. IDH mutation status, 1p/19q codeletion, and PTEN deletion were predictive of OS (P = .003, P = .005, P = .02, respectively). Both mIDH (P < .001) and the interaction term of 1p/19q and PTEN (P < .001) were found to be predictive of PFR. CONCLUSIONS: We conclude that the combination of mIDH, 1p/19q codeletion, and PTEN deletion may be particularly effective in discriminating good prognosis from poor prognosis hemispheric gliomas. We propose that such a scheme merits testing on larger prospective cohorts. Should our findings be confirmed, routine clinical analysis of hemispheric gliomas for mIDH, 1p/19q codeletion, and PTEN deletion would be justified.

Fatores de risco cardiovascular em mulheres climatéricas com doença arterial coronariana / Cardiovascular risk factors in climacteric women with coronary artery disease

Fundamentos: O aumento da incidência de doenças cardiovasculares em mulheres ocorre durante o período do climatério, especialmente após a menopausa. Objetivo: O objetivo deste estudo foi identificar fatores de risco cardiovasculares entre as mulheres climatéricas com e sem doença arterial coronariana (DAC). Métodos: Trata-se de estudo transversal realizado no Serviço de Hemodinâmica do Hospital Universitário da Universidade Federal do Maranhão, no período de março de 2012 a julho de 2013. Foram incluídas 31 mulheres climatéricas que compareceram ao setor para realização do cateterismo cardíaco, separadas em grupos após resultados do cateterismo, Grupo I (com DAC) e Grupo II (sem DAC). Análise estatística: as variáveis categóricas foram descritas por meio de frequências e porcentagem, as numéricas por meio de média ± desvio padrão ou mediana (Quartil.3 ­ Quartil.1); o teste Shapiro-Wilk para verificar a normalidade dos dados quantitativos, o teste Exato de Fisher para comparações de dados categóricos; para dados contínuos o Test-T para amostras não pareadas ou o Mann-Whitney; foi considerado estatisticamente significativo o valor de p < 0,05. Resultados: Avaliaram-se grupos com DAC (n = 13) e sem DAC (n = 18), os resultados apontaram média de idade entre os grupos de 57,92 ± 5,15 e 51,72 ± 4,63 anos, respectivamente; dentre os fatores de risco cardiovasculares, os mais prevalentes entre as mulheres com DAC foram: a menopausa (84,62%), a hipertensão arterial sistêmica (HAS) (69,23%) e o sedentarismo (69,23). Conclusão: Concluiu-se que, além da menopausa propriamente dita, a HAS e o sedentarismo foram os fatores de risco cardiovasculares mais prevalentes entre as mulheres com DAC

Background: The increased incidence of cardiovascular disease in women occurs during the climacteric period, especially after menopause. Objective: The aim of this study was to identify risk factors among climacteric women with and without coronary artery disease (CAD). Method: This cross-sectional study was performed in the Catheterization Laboratory at the Federal University Hospital of Maranhão, in the Northeast region of Brazil, between March 2012 and July 2013. We included 31 climacteric women who went to the care center for cardiac catheterization. They were divided into groups after catheterization results: Group I (with DAC) and Group II (without CAD). Statistical analysis: Categorical variables were described by means of frequencies and percentages, numerical variables by mean ± standard deviation or median (Quartile.3 - Quartile.1); the Shapiro-Wilk test was used to verify the normality of quantitative data. Fisher's exact test was used for categorical data comparisons. For continuous data, we used Student's test or the Mann-Whitney for unpaired samples; statistical significance was set at p < 0.05. Results: We evaluated groups with CAD (n = 13) and without CAD (n = 18). The results showed a mean age between the groups of 57.92 ± 5.15 and 51.72 ± 4.63 years, respectively. Among the cardiovascular risk factors, the most prevalent among women with CAD were menopause (84.62%), systemic arterial hypertension (SAH) (69.23%) and sedentary life style (69.23%). Conclusion: We concluded that, in addition to menopause itself, SAH and sedentary lifestyle were the most prevalent cardiovascular risk factors among women with CAD