Physics-informed reinforcement learning for motion control of a fish-like swimming robot.

Motion control of fish-like swimming robots presents many challenges due to the unstructured environment and unmodelled governing physics of the fluid-robot interaction. Commonly used low-fidelity control models using simplified formulas for drag and lift forces do not capture key physics that can play an important role in the dynamics of small-sized robots with limited actuation. Deep Reinforcement Learning (DRL) holds considerable promise for motion control of robots with complex dynamics. Reinforcement learning methods require large amounts of training data exploring a large subset of the relevant state space, which can be expensive, time consuming, or unsafe to obtain. Data from simulations can be used in the initial stages of DRL, but in the case of swimming robots, the complexity of fluid-body interactions makes large numbers of simulations infeasible from the perspective of time and computational resources. Surrogate models that capture the primary physics of the system can be a useful starting point for training a DRL agent which is subsequently transferred to train with a higher fidelity simulation. We demonstrate the utility of such physics-informed reinforcement learning to train a policy that can enable velocity and path tracking for a planar swimming (fish-like) rigid Joukowski hydrofoil. This is done through a curriculum where the DRL agent is first trained to track limit cycles in a velocity space for a representative nonholonomic system, and then transferred to train on a small simulation data set of the swimmer. The results show the utility of physics-informed reinforcement learning for the control of fish-like swimming robots.

Proprioceptive wake classification by a body with a passive tail.

The remarkable ability of some marine animals to identify flow structures and parameters using complex non-visual sensors, such as lateral lines of fish and the whiskers of seals, has been an area of investigation for researchers looking to apply this ability to artificial robotic swimmers, which could lead to improvements in autonomous navigation and efficiency. Several species of fish in particular have been known to school effectively, even when blind. Beyond specialized sensors like the lateral lines, it is now known that some fish use purely proprioceptive sensing, using the kinematics of their fins or tails to sense their surroundings. In this paper we show that the kinematics of a body with a passive tail encode information about the ambient flow, which can be deciphered through machine learning. We demonstrate this with experimental data of the angular velocity of a hydrofoil with a passive tail that lies in the wake generated by an upstream oscillating body. Using convolutional neural networks, we show that with the kinematic data from the downstream body with a tail, the wakes can be better classified than in the case of a body without a tail. This superior sensing ability exists for a body with a tail, even if only the kinematics of the main body are used as input for the machine learning. This shows that beyond generating 'additional inputs', passive tails modulate the response of the main body in manner that is useful for hydrodynamic sensing. These findings have clear application for improving the sensing abilities of bioinspired swimming robots.

Learning hydrodynamic signatures through proprioceptive sensing by bioinspired swimmers.

Objects moving in water or stationary objects in streams create a vortex wake. Such vortex wakes encode information about the objects and the flow conditions. Underwater robots that often function with constrained sensing capabilities can benefit from extracting this information from vortex wakes. Many species of fish do exactly this, by sensing flow features using their lateral lines as part of their multimodal sensing. To replicate such capabilities in robots, significant research has been devoted to developing artificial lateral line sensors that can be placed on the surface of a robot to detect pressure and velocity gradients. We advance an alternative view of embodied sensing in this paper; the kinematics of a swimmer's body in response to the hydrodynamic forcing by the vortex wake can encode information about the vortex wake. Here we show that using artificial neural networks that take the angular velocity of the body as input, fish-like swimmers can be trained to label vortex wakes which are hydrodynamic signatures of other moving bodies and thus acquire a capability to 'blindly' identify them.

Dynamics of groups of magnetically driven artificial microswimmers.

Magnetically driven artificial microswimmers have the potential to revolutionize many biomedical technologies, such as minimally invasive microsurgery, microparticle manipulation, and localized drug delivery. However, many of these applications will require the controlled dynamics of teams of these microrobots with minimal feedback. In this work, we study the motion and fluid dynamics produced by groups of artificial microswimmers driven by a torque induced through a uniform, rotating magnetic field. Through Stokesian dynamics simulations, we show that the swimmer motion produces a rotational velocity field in the plane orthogonal to the direction of the magnetic field's rotation, which causes two interacting swimmers to move in circular trajectories in this plane around a common center. The resulting overall motion is on a helical trajectory for the swimmers. We compare the highly rotational velocity field of the fluid to the velocity field generated by a rotlet, the point-torque singularity of Stokes flows, showing that this is a reasonable approximation on the time average. Finally, we study the motion of larger groups of swimmers, and we show that these groups tend to move coherently, especially when swimmer magnetizations are uniform. This coherence is achieved because the group center remains almost constant in the plane orthogonal to the net motion of the swimmers. The results in the paper will prove useful for controlling the ensemble dynamics of small collections of magnetic swimmers.

Chaotic advection and mixing by a pair of microrotors in a circular domain.

In this work we study chaotic mixing induced by point microrotors in a bounded two-dimensional Stokes flow. The dynamics of the pair of rotors, modeled as rotlets, are non-Hamiltonian in the bounded domain and produce chaotic advection of fluid tracers in subsets of the domain. A complete parametric investigation of the fluid mixing as a function of the initial locations of the rotlets is performed based on pseudophase portraits. The mixing of fluid tracers as a function of relative positions of microrotors is studied using finite-time entropy and locational entropy. The finite-time locational entropy is used to identify regions of the fluid that produce good versus poor mixing, and this is visualized by the stretching and folding of blobs of tracer particles. Unlike the case of the classic blinking vortex dynamics, the velocity field of the flow modeled using rotlets inside a circular boundary is smooth in time and satisfies the no-slip boundary condition. This makes the considered model a more realistic case for studies of mixing in microfluidic devices using magnetic-actuated microspheres.

Particle slip velocity influences inertial focusing of particles in curved microchannels.

Size based separation and identification of particles in microfluidics through purely hydrodynamic means has gained significant interest due to a number of possible biomedical applications. Curved micro-channels, particularly spiral micro-channels with rectangular cross-section and the dynamics of particles in such channels have been extensively researched to achieve size based separation of particles. In this paper we present evidence that sheds new light on the dynamics of particles in such curved channels; that a unique particle slip velocity is associated with the focusing positions in the cross sections, which leads to a balance of forces. Our experiments therefore imply that the forces acting on the particle lead to convergence to an attractor in both the physical space (the cross section of the channel) and the slip velocity space.

Particle segregation by Stokes number for small neutrally buoyant spheres in a fluid.

It is a commonly observed phenomenon that spherical particles with inertia in an incompressible fluid do not behave as ideal tracers. Due to the inertia of the particle, the planar dynamics are described in a four-dimensional phase space and thus can differ considerably from the ideal tracer dynamics. Using finite-time Lyapunov exponents, we compute the sensitivity of the final position of a particle with respect to its initial velocity, relative to the fluid, and thus partition the relative velocity subspace at each point in configuration space. The computations are done at every point in the relative velocity subspace, thus giving a sensitivity field. The Stokes number, being a measure of the independence of the particle from the underlying fluid flow, acts as a parameter in determining the variation in these partitions. We demonstrate how this partition framework can be used to segregate particles by Stokes number in a fluid. The fluid model used for demonstration is a two-dimensional cellular flow.