Learning agile soccer skills for a bipedal robot with deep reinforcement learning.

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent's tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

Looking back and moving forward: dimensions of coping flexibility divergently predict long-term bereavement outcomes.

BACKGROUND AND OBJECTIVES: Bereavement is a serious public health concern. Some people suffer prolonged and debilitating functional impairment after the death of a loved one. Evidence suggests that flexibility in coping approaches predicts resilience after stressful life events, but its long-term effects after the unique experience of bereavement are unknown. Which strategies of coping flexibility predict better-or worse-adjustment over time for bereaved people and at what times? DESIGN AND METHODS: The present study used path analyses to investigate longitudinal effects of forward-focus and loss-focus coping strategies on symptoms of persistent complex bereavement disorder (PCBD), depression, and posttraumatic stress disorder in a spousally bereaved adult sample (N = 248) at three time-points after the loss (∼3 months, ∼14 months, and ∼25 months). RESULTS: Forward-focus coping demonstrated adaptive utility overall, with sooner effects on PCBD than on depression. By contrast, loss-focus coping demonstrated a delayed-onset, maladaptive pattern. CONCLUSIONS: The findings contribute to the coping flexibility literature by suggesting that the adaptiveness or maladaptiveness of different coping strategies may depend on the context that requires coping. In particular, forward-focus coping may be substantially more advantageous than loss-focus coping in the context of bereavement. Implications, limitations, and future research directions are discussed.

Trajectories of resilience and dysfunction following potential trauma: A review and statistical evaluation.

Given the rapid proliferation of trajectory-based approaches to study clinical consequences to stress and potentially traumatic events (PTEs), there is a need to evaluate emerging findings. This review examined convergence/divergences across 54 studies in the nature and prevalence of response trajectories, and determined potential sources of bias to improve future research. Of the 67 cases that emerged from the 54 studies, the most consistently observed trajectories following PTEs were resilience (observed in: n = 63 cases), recovery (n = 49), chronic (n = 47), and delayed onset (n = 22). The resilience trajectory was the modal response across studies (average of 65.7% across populations, 95% CI [0.616, 0.698]), followed in prevalence by recovery (20.8% [0.162, 0.258]), chronicity (10.6%, [0.086, 0.127]), and delayed onset (8.9% [0.053, 0.133]). Sources of heterogeneity in estimates primarily resulted from substantive population differences rather than bias, which was observed when prospective data is lacking. Overall, prototypical trajectories have been identified across independent studies in relatively consistent proportions, with resilience being the modal response to adversity. Thus, trajectory models robustly identify clinically relevant patterns of response to potential trauma, and are important for studying determinants, consequences, and modifiers of course following potential trauma.

Toward personalizing treatment for depression: predicting diagnosis and severity.

OBJECTIVE: Depression is a prevalent disorder difficult to diagnose and treat. In particular, depressed patients exhibit largely unpredictable responses to treatment. Toward the goal of personalizing treatment for depression, we develop and evaluate computational models that use electronic health record (EHR) data for predicting the diagnosis and severity of depression, and response to treatment. MATERIALS AND METHODS: We develop regression-based models for predicting depression, its severity, and response to treatment from EHR data, using structured diagnosis and medication codes as well as free-text clinical reports. We used two datasets: 35,000 patients (5000 depressed) from the Palo Alto Medical Foundation and 5651 patients treated for depression from the Group Health Research Institute. RESULTS: Our models are able to predict a future diagnosis of depression up to 12 months in advance (area under the receiver operating characteristic curve (AUC) 0.70-0.80). We can differentiate patients with severe baseline depression from those with minimal or mild baseline depression (AUC 0.72). Baseline depression severity was the strongest predictor of treatment response for medication and psychotherapy. CONCLUSIONS: It is possible to use EHR data to predict a diagnosis of depression up to 12 months in advance and to differentiate between extreme baseline levels of depression. The models use commonly available data on diagnosis, medication, and clinical progress notes, making them easily portable. The ability to automatically determine severity can facilitate assembly of large patient cohorts with similar severity from multiple sites, which may enable elucidation of the moderators of treatment response in the future.