replicAnt: a pipeline for generating annotated images of animals in complex environments using Unreal Engine.

Deep learning-based computer vision methods are transforming animal behavioural research. Transfer learning has enabled work in non-model species, but still requires hand-annotation of example footage, and is only performant in well-defined conditions. To help overcome these limitations, we developed replicAnt, a configurable pipeline implemented in Unreal Engine 5 and Python, designed to generate large and variable training datasets on consumer-grade hardware. replicAnt places 3D animal models into complex, procedurally generated environments, from which automatically annotated images can be exported. We demonstrate that synthetic data generated with replicAnt can significantly reduce the hand-annotation required to achieve benchmark performance in common applications such as animal detection, tracking, pose-estimation, and semantic segmentation. We also show that it increases the subject-specificity and domain-invariance of the trained networks, thereby conferring robustness. In some applications, replicAnt may even remove the need for hand-annotation altogether. It thus represents a significant step towards porting deep learning-based computer vision tools to the field.

A bio-inspired robotic climbing robot to understand kinematic and morphological determinants for an optimal climbing gait.

Robotic systems for complex tasks, such as search and rescue or exploration, are limited for wheeled designs, thus the study of legged locomotion for robotic applications has become increasingly important. To successfully navigate in regions with rough terrain, a robot must not only be able to negotiate obstacles, but also climb steep inclines. Following the principles of biomimetics, we developed a modular bio-inspired climbing robot, named X4, which mimics the lizard's bauplan including an actuated spine, shoulders, and feet which interlock with the surface via claws. We included the ability to modify gait and hardware parameters and simultaneously collect data with the robot's sensors on climbed distance, slip occurrence and efficiency. We first explored the speed-stability trade-off and its interaction with limb swing phase dynamics, finding a sigmoidal pattern of limb movement resulted in the greatest distance travelled. By modifying foot orientation, we found two optima for both speed and stability, suggesting multiple stable configurations. We varied spine and limb range of motion, again showing two possible optimum configurations, and finally varied the centre of pro- and retraction on climbing performance, showing an advantage for protracted limbs during the stride. We then stacked optimal regions of performance and show that combining optimal dynamic patterns with either foot angles or ROM configurations have the greatest performance, but further optima stacking resulted in a decrease in performance, suggesting complex interactions between kinematic parameters. The search of optimal parameter configurations might not only be beneficial to improve robotic in-field operations but may also further the study of the locomotive evolution of climbing of animals, like lizards or insects.

Using a biologically mimicking climbing robot to explore the performance landscape of climbing in lizards.

Locomotion is a key aspect associated with ecologically relevant tasks for many organisms, therefore, survival often depends on their ability to perform well at these tasks. Despite this significance, we have little idea how different performance tasks are weighted when increased performance in one task comes at the cost of decreased performance in another. Additionally, the ability for natural systems to become optimized to perform a specific task can be limited by structural, historic or functional constraints. Climbing lizards provide a good example of these constraints as climbing ability likely requires the optimization of tasks which may conflict with one another such as increasing speed, avoiding falls and reducing the cost of transport (COT). Understanding how modifications to the lizard bauplan can influence these tasks may allow us to understand the relative weighting of different performance objectives among species. Here, we reconstruct multiple performance landscapes of climbing locomotion using a 10 d.f. robot based upon the lizard bauplan, including an actuated spine, shoulders and feet, the latter which interlock with the surface via claws. This design allows us to independently vary speed, foot angles and range of motion (ROM), while simultaneously collecting data on climbed distance, stability and efficiency. We first demonstrate a trade-off between speed and stability, with high speeds resulting in decreased stability and low speeds an increased COT. By varying foot orientation of fore- and hindfeet independently, we found geckos converge on a narrow optimum of foot angles (fore 20°, hind 100°) for both speed and stability, but avoid a secondary wider optimum (fore -20°, hind -50°) highlighting a possible constraint. Modifying the spine and limb ROM revealed a gradient in performance. Evolutionary modifications in movement among extant species over time appear to follow this gradient towards areas which promote speed and efficiency.