Magnetic control of tokamak plasmas through deep reinforcement learning.

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable1,2, including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and 'snowflake' configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained 'droplets' on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.

Evidence for close-range hunting by last interglacial Neanderthals.

Animal resources have been part of hominin diets since around 2.5 million years ago, with sharp-edged stone tools facilitating access to carcasses. How exactly hominins acquired animal prey and how hunting strategies varied through time and space is far from clear. The oldest possible hunting weapons known from the archaeological record are 300,000 to 400,000-year-old sharpened wooden staves. These may have been used as throwing and/or close-range thrusting spears, but actual data on how such objects were used are lacking, as unambiguous lesions caused by such weapon-like objects are unknown for most of human prehistory. Here, we report perforations observed on two fallow deer skeletons from Neumark-Nord, Germany, retrieved during excavations of 120,000-year-old lake shore deposits with abundant traces of Neanderthal presence. Detailed studies of the perforations, including micro-computed tomography imaging and ballistic experiments, demonstrate that they resulted from the close-range use of thrusting spears. Such confrontational ways of hunting require close cooperation between participants, and over time may have shaped important aspects of hominin biology and behaviour.

Dynamic Monitoring Reveals Motor Task Characteristics in Prehistoric Technical Gestures.

Reconstructing ancient technical gestures associated with simple tool actions is crucial for understanding the co-evolution of the human forelimb and its associated control-related cognitive functions on the one hand, and of the human technological arsenal on the other hand. Although the topic of gesture is an old one in Paleolithic archaeology and in anthropology in general, very few studies have taken advantage of the new technologies from the science of kinematics in order to improve replicative experimental protocols. Recent work in paleoanthropology has shown the potential of monitored replicative experiments to reconstruct tool-use-related motions through the study of fossil bones, but so far comparatively little has been done to examine the dynamics of the tool itself. In this paper, we demonstrate that we can statistically differentiate gestures used in a simple scraping task through dynamic monitoring. Dynamics combines kinematics (position, orientation, and speed) with contact mechanical parameters (force and torque). Taken together, these parameters are important because they play a role in the formation of a visible archaeological signature, use-wear. We present our new affordable, yet precise methodology for measuring the dynamics of a simple hide-scraping task, carried out using a pull-to (PT) and a push-away (PA) gesture. A strain gage force sensor combined with a visual tag tracking system records force, torque, as well as position and orientation of hafted flint stone tools. The set-up allows switching between two tool configurations, one with distal and the other one with perpendicular hafting of the scrapers, to allow for ethnographically plausible reconstructions. The data show statistically significant differences between the two gestures: scraping away from the body (PA) generates higher shearing forces, but requires greater hand torque. Moreover, most benchmarks associated with the PA gesture are more highly variable than in the PT gesture. These results demonstrate that different gestures used in 'common' prehistoric tasks can be distinguished quantitatively based on their dynamic parameters. Future research needs to assess our ability to reconstruct these parameters from observed use-wear patterns.

Resonant neurons and bushcricket behaviour.

The resonant properties of the intrinsic dynamics of single neurons could play a direct role in behaviour. One plausible role is in the recognition of temporal patterns, such as that seen in the auditory communication systems of Orthoptera. Recent behavioural data from bushcrickets suggests that this behaviour has interesting resonance properties, but the underlying mechanism is unknown. Here we show that a very simple and general model for neural resonance could directly account for the different behavioural responses of bushcrickets to different song patterns.

Engineering entrainment and adaptation in limit cycle systems : From biological inspiration to applications in robotics.

Periodic behavior is key to life and is observed in multiple instances and at multiple time scales in our metabolism, our natural environment, and our engineered environment. A natural way of modeling or generating periodic behavior is done by using oscillators, i.e., dynamical systems that exhibit limit cycle behavior. While there is extensive literature on methods to analyze such dynamical systems, much less work has been done on methods to synthesize an oscillator to exhibit some specific desired characteristics. The goal of this article is twofold: (1) to provide a framework for characterizing and designing oscillators and (2) to review how classes of well-known oscillators can be understood and related to this framework. The basis of the framework is to characterize oscillators in terms of their fundamental temporal and spatial behavior and in terms of properties that these two behaviors can be designed to exhibit. This focus on fundamental properties is important because it allows us to systematically compare a large variety of oscillators that might at first sight appear very different from each other. We identify several specifications that are useful for design, such as frequency-locking behavior, phase-locking behavior, and specific output signal shape. We also identify two classes of design methods by which these specifications can be met, namely offline methods and online methods. By relating these specifications to our framework and by presenting several examples of how oscillators have been designed in the literature, this article provides a useful methodology and toolbox for designing oscillators for a wide range of purposes. In particular, the focus on synthesis of limit cycle dynamical systems should be useful both for engineering and for computational modeling of physical or biological phenomena.

Nonlinear modelling of double and triple period pitch breaks in vocal fold vibration.

This paper reports a study on short-time subharmonic pitch breaks in vocal fold vibration, which are found to be a common feature of the human voice in spoken language. The observed pitch breaks correspond to a change in periodicity of the electrolaryngograph (Lx) signal. This paper presents a nonlinear dynamical system capable of producing time-series with subharmonic pitch breaks. The resulting time-series resemble closely Lx recordings of natural speech. The system is developed on the basis of a second order linear system, which is extended with a third dimension and nonlinear coupling terms. It is suggested that improved knowledge about pitch breaks could be used in future speech synthesis systems in order to improve the naturalness of the perceived output.