Drosophila flying in augmented reality reveals the vision-based control autonomy of the optomotor response.

For walking, swimming, and flying animals, the optomotor response is essential to stabilize gaze. How flexible is the optomotor response? Classic work in Drosophila has argued that flies adapt flight control under augmented visual feedback conditions during goal-directed bar fixation. However, whether the lower-level, reflexive optomotor response can similarly adapt to augmented visual feedback (partially autonomous) or not (autonomous) over long timescales is poorly understood. To address this question, we developed an augmented reality paradigm to study the vision-based control autonomy of the yaw optomotor response of flying fruit flies (Drosophila). Flies were placed in a flight simulator, which permitted free body rotation about the yaw axis. By feeding back body movements in real time to a visual display, we augmented and inverted visual feedback. Thus, this experimental paradigm caused a constant visual error between expected and actual visual feedback to study potential adaptive visuomotor control. By combining experiments with control theory, we demonstrate that the optomotor response is autonomous during augmented reality flight bouts of up to 30 min, which exceeds the reported learning epoch during bar fixation. Agreement between predictions from linear systems theory and experimental data supports the notion that the optomotor response is approximately linear and time invariant within our experimental assay. Even under positive visual feedback, which revealed the stability limit of flies in augmented reality, the optomotor response was autonomous. Our results support a hierarchical motor control architecture in flies with fast and autonomous reflexes at the bottom and more flexible behavior at higher levels.

Flies adaptively control flight to compensate for added inertia.

Animal locomotion is highly adaptive, displaying a large degree of flexibility, yet how this flexibility arises from the integration of mechanics and neural control remains elusive. For instance, animals require flexible strategies to maintain performance as changes in mass or inertia impact stability. Compensatory strategies to mechanical loading are especially critical for animals that rely on flight for survival. To shed light on the capacity and flexibility of flight neuromechanics to mechanical loading, we pushed the performance of fruit flies (Drosophila) near its limit and implemented a control theoretic framework. Flies with added inertia were placed inside a virtual reality arena which permitted free rotation about the vertical (yaw) axis. Adding inertia increased the fly's response time yet had little influence on overall gaze stabilization performance. Flies maintained stability following the addition of inertia by adaptively modulating both visuomotor gain and damping. By contrast, mathematical modelling predicted a significant decrease in gaze stabilization performance. Adding inertia altered saccades, however, flies compensated for the added inertia by increasing saccade torque. Taken together, in response to added inertia flies increase reaction time but maintain flight performance through adaptive neural control. Overall, adding inertia decreases closed-loop flight robustness. Our work highlights the flexibility and capacity of motor control in flight.

Hawkmoths regulate flight torques with their abdomen for yaw control.

Many animals use body parts such as tails to stabilize posture while moving at high speed. In flying insects, leg or abdominal inertia can influence flight posture. In the hawkmoth Manduca sexta, the abdomen contributes ∼50% of the total body weight and it can therefore serve to inertially redirect flight forces. How do torques generated by the wings and abdomen interact for flight control? We studied the yaw optomotor response of M. sexta by using a torque sensor attached to their thorax. In response to yaw visual motion, the abdomen moved antiphase with the stimulus, head and total torque. By studying moths with ablated wings and a fixed abdomen, we resolved abdomen and wing torques and revealed their individual contribution to total yaw torque production. Frequency-domain analysis revealed that the abdomen torque is overall smaller than wing torque, although the abdomen torque is ∼80% of the wing torque at higher visual stimulus temporal frequency. Experimental data and modeling revealed that the wing and abdomen torque are transmitted linearly to the thorax. By modeling the thorax and abdomen as a two-link system, we show that abdomen flexion can inertially redirect the thorax to add constructively to wing steering efforts. Our work argues for considering the role of the abdomen in tethered insect flight experiments that use force/torque sensors. Taken together, the hawkmoth abdomen can regulate wing torques in free flight, which could modulate flight trajectories and increase maneuverability.

Flies trade off stability and performance via adaptive compensation to wing damage.

Physical injury often impairs mobility, which can have dire consequences for survival in animals. Revealing mechanisms of robust biological intelligence to prevent system failure can provide critical insights into how complex brains generate adaptive movement and inspiration to design fault-tolerant robots. For flying animals, physical injury to a wing can have severe consequences, as flight is inherently unstable. Using a virtual reality flight arena, we studied how flying fruit flies compensate for damage to one wing. By combining experimental and mathematical methods, we show that flies compensate for wing damage by corrective wing movement modulated by closed-loop sensing and robust mechanics. Injured flies actively increase damping and, in doing so, modestly decrease flight performance but fly as stably as uninjured flies. Quantifying responses to injury can uncover the flexibility and robustness of biological systems while informing the development of bio-inspired fault-tolerant strategies.

Nested mechanosensory feedback actively damps visually guided head movements in Drosophila.

Executing agile locomotion requires animals to integrate sensory feedback, often from multiple sources. For example, human gaze is mediated by multiple feedback loops that integrate visual and vestibular information. A central challenge in studying biological feedback loops is that they are nested and dynamically coupled. Here, we develop a framework based on control theory for unraveling nested feedback systems and apply it to study gaze stabilization in the fruit fly (Drosophila). By combining experimental and mathematical methods to manipulate control topologies, we uncovered the role of body-generated mechanosensory feedback nested within visual feedback in the control of head movements. We discovered that visual feedback changed the tuning of head movements across visual motion frequencies whereas mechanosensory feedback damped head movements. Head saccades had slower dynamics when the body was free to move, further pointing to the role of damping via mechanosensory feedback. By comparing head responses between self-generated and externally generated body motion, we revealed a nonlinear gating of mechanosensory feedback that is motor-context dependent. Altogether, our findings reveal the role of nested feedback loops in flies and uncover mechanisms that reconcile differences in head kinematics between body-free and body-fixed flies. Our framework is generalizable to biological and robotic systems relying on nested feedback control for guiding locomotion.

Complementary feedback control enables effective gaze stabilization in animals.

Visually active animals coordinate vision and movement to achieve spectacular tasks. An essential prerequisite to guide agile locomotion is to keep gaze level and stable. Since the eyes, head and body can move independently to control gaze, how does the brain effectively coordinate these distinct motor outputs? Furthermore, since the eyes, head, and body have distinct mechanical constraints (e.g., inertia), how does the nervous system adapt its control to these constraints? To address these questions, we studied gaze control in flying fruit flies (Drosophila) using a paradigm which permitted direct measurement of head and body movements. By combining experiments with mathematical modeling, we show that body movements are sensitive to the speed of visual motion whereas head movements are sensitive to its acceleration. This complementary tuning of the head and body permitted flies to stabilize a broader range of visual motion frequencies. We discovered that flies implement proportional-derivative (PD) control, but unlike classical engineering control systems, relay the proportional and derivative signals in parallel to two distinct motor outputs. This scheme, although derived from flies, recapitulated classic primate vision responses thus suggesting convergent mechanisms across phyla. By applying scaling laws, we quantify that animals as diverse as flies, mice, and humans as well as bio-inspired robots can benefit energetically by having a high ratio between head, body, and eye inertias. Our results provide insights into the mechanical constraints that may have shaped the evolution of active vision and present testable neural control hypotheses for visually guided behavior across phyla.

Mechanisms of punctuated vision in fly flight.

To guide locomotion, animals control gaze via movements of their eyes, head, and/or body, but how the nervous system controls gaze during complex motor tasks remains elusive. In many animals, shifts in gaze consist of periods of smooth movement punctuated by rapid eye saccades. Notably, eye movements are constrained by anatomical limits, which requires resetting eye position. By studying tethered, flying fruit flies (Drosophila), we show that flies perform stereotyped head saccades to reset gaze, analogous to optokinetic nystagmus in primates. Head-reset saccades interrupted head smooth movement for as little as 50 ms-representing less than 5% of the total flight time-thereby enabling punctuated gaze stabilization. By revealing the passive mechanics of the neck joint, we show that head-reset saccades leverage the neck's natural elastic recoil, enabling mechanically assisted redirection of gaze. The consistent head orientation at saccade initiation, the influence of the head's angular position on saccade rate, the decrease in wing saccade frequency in head-fixed flies, and the decrease in head-reset saccade rate in flies with their head range of motion restricted together implicate proprioception as the primary trigger of head-reset saccades. Wing-reset saccades were influenced by head orientation, establishing a causal link between neck sensory signals and the execution of body saccades. Head-reset saccades were abolished when flies switched to a landing state, demonstrating that head movements are gated by behavioral state. We propose a control architecture for active vision systems with limits in sensor range of motion. VIDEO ABSTRACT.

Hybrid visual control in fly flight: insights into gaze shift via saccades.

Flies fly by alternating between periods of fixation and body saccades, analogous to how our own eyes move. Gaze fixation via smooth movement in fly flight has been studied extensively, but comparatively less is known about the mechanism by which flies trigger and control body saccades to shift their gaze. Why do flies implement a hybrid fixate-and-saccade locomotion strategy? Here we review recent developments that provide new insights into this question. We focus on the interplay between smooth movement and saccades, the trigger classes of saccades, and the timeline of saccade execution. We emphasize recent mechanistic advances in Drosophila, where genetic tools have enabled cellular circuit analysis at an unprecedented level in a flying insect. In addition, we review trade-offs in behavioral paradigms used to study saccades. Throughout we highlight exciting avenues for future research in the control of fly flight.

Active vision shapes and coordinates flight motor responses in flies.

Animals use active sensing to respond to sensory inputs and guide future motor decisions. In flight, flies generate a pattern of head and body movements to stabilize gaze. How the brain relays visual information to control head and body movements and how active head movements influence downstream motor control remains elusive. Using a control theoretic framework, we studied the optomotor gaze stabilization reflex in tethered flight and quantified how head movements stabilize visual motion and shape wing steering efforts in fruit flies (Drosophila). By shaping visual inputs, head movements increased the gain of wing steering responses and coordination between stimulus and wings, pointing to a tight coupling between head and wing movements. Head movements followed the visual stimulus in as little as 10 ms-a delay similar to the human vestibulo-ocular reflex-whereas wing steering responses lagged by more than 40 ms. This timing difference suggests a temporal order in the flow of visual information such that the head filters visual information eliciting downstream wing steering responses. Head fixation significantly decreased the mechanical power generated by the flight motor by reducing wingbeat frequency and overall thrust. By simulating an elementary motion detector array, we show that head movements shift the effective visual input dynamic range onto the sensitivity optimum of the motion vision pathway. Taken together, our results reveal a transformative influence of active vision on flight motor responses in flies. Our work provides a framework for understanding how to coordinate moving sensors on a moving body.

Fly eyes are not still: a motion illusion in Drosophila flight supports parallel visual processing.

Most animals shift gaze by a 'fixate and saccade' strategy, where the fixation phase stabilizes background motion. A logical prerequisite for robust detection and tracking of moving foreground objects, therefore, is to suppress the perception of background motion. In a virtual reality magnetic tether system enabling free yaw movement, Drosophila implemented a fixate and saccade strategy in the presence of a static panorama. When the spatial wavelength of a vertical grating was below the Nyquist wavelength of the compound eyes, flies drifted continuously and gaze could not be maintained at a single location. Because the drift occurs from a motionless stimulus - thus any perceived motion stimuli are generated by the fly itself - it is illusory, driven by perceptual aliasing. Notably, the drift speed was significantly faster than under a uniform panorama, suggesting perceptual enhancement as a result of aliasing. Under the same visual conditions in a rigid-tether paradigm, wing steering responses to the unresolvable static panorama were not distinguishable from those to a resolvable static pattern, suggesting visual aliasing is induced by ego motion. We hypothesized that obstructing the control of gaze fixation also disrupts detection and tracking of objects. Using the illusory motion stimulus, we show that magnetically tethered Drosophila track objects robustly in flight even when gaze is not fixated as flies continuously drift. Taken together, our study provides further support for parallel visual motion processing and reveals the critical influence of body motion on visuomotor processing. Motion illusions can reveal important shared principles of information processing across taxa.