Outracing champion Gran Turismo drivers with deep reinforcement learning.

Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits1. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world's best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing's important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world's best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

The look-ahead effect of phenotypic mutations.

BACKGROUND: The evolution of complex molecular traits such as disulphide bridges often requires multiple mutations. The intermediate steps in such evolutionary trajectories are likely to be selectively neutral or deleterious. Therefore, large populations and long times may be required to evolve such traits. RESULTS: We propose that errors in transcription and translation may allow selection for the intermediate mutations, if the final trait provides a large enough selective advantage. We test this hypothesis using a population based model of protein evolution. CONCLUSION: If an individual acquires one of two mutations needed for a novel trait, the second mutation can be introduced into the phenotype due to transcription and translation errors. If the novel trait is advantageous enough, the allele with only one mutation will spread through the population, even though the gene sequence does not yet code for the complettrait. Thus, errors allow protein sequences to "look-ahead" for a more direct path to a complex trait.

The AtGenExpress global stress expression data set: protocols, evaluation and model data analysis of UV-B light, drought and cold stress responses.

The tolerance responses of plants to many abiotic stresses are conjectured to be controlled by complex gene networks. In the frame of the AtGenExpress project a comprehensive Arabidopsis thaliana genome transcript expression study was performed using the Affymetrix ATH1 microarray in order to understand these regulatory networks in detail. In contrast to earlier studies, we subjected, side-by-side and in a high-resolution kinetic series, Arabidopsis plants, of identical genotype grown under identical conditions, to different environmental stresses comprising heat, cold, drought, salt, high osmolarity, UV-B light and wounding. Furthermore, the harvesting of tissue and RNA isolation were performed in parallel at the same location using identical experimental protocols. Here we describe the technical performance of the experiments. We also present a general overview of environmental abiotic stress-induced gene expression patterns and the results of a model bioinformatics analysis of gene expression in response to UV-B light, drought and cold stress. Our results suggest that the initial transcriptional stress reaction of Arabidopsis might comprise a set of core environmental stress response genes which, by adjustment of the energy balance, could have a crucial function in various stress responses. In addition, there are indications that systemic signals generated by the tissue exposed to stress play a major role in the coordination and execution of stress responses. In summary, the information reported provides a prime reference point and source for the subsequent exploitation of this important resource for research into plant abiotic stress.