Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using reinforcement learning.

To find the path that minimizes the time to navigate between two given points in a fluid flow is known as Zermelo's problem. Here, we investigate it by using a Reinforcement Learning (RL) approach for the case of a vessel that has a slip velocity with fixed intensity, Vs, but variable direction and navigating in a 2D turbulent sea. We show that an Actor-Critic RL algorithm is able to find quasioptimal solutions for both time-independent and chaotically evolving flow configurations. For the frozen case, we also compared the results with strategies obtained analytically from continuous Optimal Navigation (ON) protocols. We show that for our application, ON solutions are unstable for the typical duration of the navigation process and are, therefore, not useful in practice. On the other hand, RL solutions are much more robust with respect to small changes in the initial conditions and to external noise, even when Vs is much smaller than the maximum flow velocity. Furthermore, we show how the RL approach is able to take advantage of the flow properties in order to reach the target, especially when the steering speed is small.

The spatio-temporal spectrum of turbulent flows.

Identification and extraction of vortical structures and of waves in a disorganised flow is a mayor challenge in the study of turbulence. We present a study of the spatio-temporal behavior of turbulent flows in the presence of different restitutive forces. We show how to compute and analyse the spatio-temporal spectrum from data stemming from numerical simulations and from laboratory experiments. Four cases are considered: homogeneous and isotropic turbulence, rotating turbulence, stratified turbulence, and water wave turbulence. For homogeneous and isotropic turbulence, the spectrum allows identification of sweeping by the large-scale flow. For rotating and for stratified turbulence, the spectrum allows identification of the waves, precise quantification of the energy in the waves and in the turbulent eddies, and identification of physical mechanisms such as Doppler shift and wave absorption in critical layers. Finally, in water wave turbulence the spectrum shows a transition from gravity-capillary waves to bound waves as the amplitude of the forcing is increased.

Absorption of waves by large-scale winds in stratified turbulence.

The atmosphere is a nonlinear stratified fluid in which internal gravity waves are present. These waves interact with the flow, resulting in wave turbulence that displays important differences with the turbulence observed in isotropic and homogeneous flows. We study numerically the role of these waves and their interaction with the large-scale flow, consisting of vertically sheared horizontal winds. We calculate their space- and time-resolved energy spectrum (a four-dimensional spectrum) and show that most of the energy is concentrated along a dispersion relation that is Doppler shifted by the horizontal winds. We also observe that when uniform winds are let to develop in each horizontal layer of the flow, waves whose phase velocity is equal to the horizontal wind speed have negligible energy. This indicates a nonlocal transfer of their energy to the mean flow. Both phenomena, the Doppler shift and the absorption of waves traveling with the wind speed, are not accounted for in current theories of stratified wave turbulence.

Wave turbulence in shallow water models.

We study wave turbulence in shallow water flows in numerical simulations using two different approximations: the shallow water model and the Boussinesq model with weak dispersion. The equations for both models were solved using periodic grids with up to 2048{2} points. In all simulations, the Froude number varies between 0.015 and 0.05, while the Reynolds number and level of dispersion are varied in a broader range to span different regimes. In all cases, most of the energy in the system remains in the waves, even after integrating the system for very long times. For shallow flows, nonlinear waves are nondispersive and the spectrum of potential energy is compatible with ∼k{-2} scaling. For deeper (Boussinesq) flows, the nonlinear dispersion relation as directly measured from the wave and frequency spectrum (calculated independently) shows signatures of dispersion, and the spectrum of potential energy is compatible with predictions of weak turbulence theory, ∼k{-4/3}. In this latter case, the nonlinear dispersion relation differs from the linear one and has two branches, which we explain with a simple qualitative argument. Finally, we study probability density functions of the surface height and find that in all cases the distributions are asymmetric. The probability density function can be approximated by a skewed normal distribution as well as by a Tayfun distribution.