Learning agile soccer skills for a bipedal robot with deep reinforcement learning.

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent's tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

From motor control to team play in simulated humanoid football.

Learning to combine control at the level of joint torques with longer-term goal-directed behavior is a long-standing challenge for physically embodied artificial agents. Intelligent behavior in the physical world unfolds across multiple spatial and temporal scales: Although movements are ultimately executed at the level of instantaneous muscle tensions or joint torques, they must be selected to serve goals that are defined on much longer time scales and that often involve complex interactions with the environment and other agents. Recent research has demonstrated the potential of learning-based approaches applied to the respective problems of complex movement, long-term planning, and multiagent coordination. However, their integration traditionally required the design and optimization of independent subsystems and remains challenging. In this work, we tackled the integration of motor control and long-horizon decision-making in the context of simulated humanoid football, which requires agile motor control and multiagent coordination. We optimized teams of agents to play simulated football via reinforcement learning, constraining the solution space to that of plausible movements learned using human motion capture data. They were trained to maximize several environment rewards and to imitate pretrained football-specific skills if doing so led to improved performance. The result is a team of coordinated humanoid football players that exhibit complex behavior at different scales, quantified by a range of analysis and statistics, including those used in real-world sport analytics. Our work constitutes a complete demonstration of learned integrated decision-making at multiple scales in a multiagent setting.

End Point of the Ultraspinning Instability and Violation of Cosmic Censorship.

We determine the end point of the axisymmetric ultraspinning instability of asymptotically flat Myers-Perry black holes in D=6 spacetime dimensions. In the nonlinear regime, this instability gives rise to a sequence of concentric rings connected by segments of black membrane on the rotation plane. The latter become thinner over time, resulting in the formation of a naked singularity in finite asymptotic time and hence a violation of the weak cosmic censorship conjecture in asymptotically flat higher-dimensional spaces.

End Point of Black Ring Instabilities and the Weak Cosmic Censorship Conjecture.

We produce the first concrete evidence that violation of the weak cosmic censorship conjecture can occur in asymptotically flat spaces of five dimensions by numerically evolving perturbed black rings. For certain thin rings, we identify a new, elastic-type instability dominating the evolution, causing the system to settle to a spherical black hole. However, for sufficiently thin rings the Gregory-Laflamme mode is dominant, and the instability unfolds similarly to that of black strings, where the horizon develops a structure of bulges connected by necks which become ever thinner over time.