Using Reinforcement Learning to Develop a Novel Gait for a Bio-Robotic California Sea Lion.

While researchers have made notable progress in bio-inspired swimming robot development, a persistent challenge lies in creating propulsive gaits tailored to these robotic systems. The California sea lion achieves its robust swimming abilities through a careful coordination of foreflippers and body segments. In this paper, reinforcement learning (RL) was used to develop a novel sea lion foreflipper gait for a bio-robotic swimmer using a numerically modelled computational representation of the robot. This model integration enabled reinforcement learning to develop desired swimming gaits in the challenging underwater domain. The novel RL gait outperformed the characteristic sea lion foreflipper gait in the simulated underwater domain. When applied to the real-world robot, the RL constructed novel gait performed as well as or better than the characteristic sea lion gait in many factors. This work shows the potential for using complimentary bio-robotic and numerical models with reinforcement learning to enable the development of effective gaits and maneuvers for underwater swimming vehicles.

Fish robotics: multi-fin propulsion and the coupling of fin phase, spacing, and compliance.

Fish coordinate the motion of their fins and body to create the time-varying forces required for swimming and agile maneuvers. To effectively adapt this biological strategy for underwater robots, it is necessary to understand how the location and coordination of interacting fish-like fins affect the production of propulsive forces. In this study, the impact that phase difference, horizontal and vertical spacing, and compliance of paired fins had on net thrust and lateral forces was investigated using two fish-like robotic swimmers and a series of computational fluid dynamic simulations. The results demonstrated that the propulsive forces created by pairs of fins that interact through wake flows are highly dependent on the fins' spacing and compliance. Changes to fin separation of less than one fin length had a dramatic effect on forces, and on the phase difference at which desired forces would occur. These findings have clear implications when designing multi-finned swimming robots. Well-designed, interacting fins can potentially produce several times more propulsive force than a poorly tuned robot with seemingly small differences in the kinematic, geometric, and mechanical properties.