Optimization in Visual Motion Estimation.

Sighted animals use visual signals to discern directional motion in their environment. Motion is not directly detected by visual neurons, and it must instead be computed from light signals that vary over space and time. This makes visual motion estimation a near universal neural computation, and decades of research have revealed much about the algorithms and mechanisms that generate directional signals. The idea that sensory systems are optimized for performance in natural environments has deeply impacted this research. In this article, we review the many ways that optimization has been used to quantitatively model visual motion estimation and reveal its underlying principles. We emphasize that no single optimization theory has dominated the literature. Instead, researchers have adeptly incorporated different computational demands and biological constraints that are pertinent to the specific brain system and animal model under study. The successes and failures of the resulting optimization models have thereby provided insights into how computational demands and biological constraints together shape neural computation.

Neural mechanisms to incorporate visual counterevidence in self-movement estimation.

In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can be spuriously triggered by visual motion created by objects moving in the world. Here, we show that stationary patterns on the retina, which constitute evidence against observer rotation, suppress inappropriate stabilizing rotational behavior in the fruit fly Drosophila. In silico experiments show that artificial neural networks (ANNs) that are optimized to distinguish observer movement from external object motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's local motion and optic-flow detectors. Our results show how the fly brain incorporates negative evidence to improve heading stability, exemplifying how a compact brain exploits geometrical constraints of the visual world.

Direct comparison reveals algorithmic similarities in fly and mouse visual motion detection.

Evolution has equipped vertebrates and invertebrates with neural circuits that selectively encode visual motion. While similarities in the computations performed by these circuits in mouse and fruit fly have been noted, direct experimental comparisons have been lacking. Because molecular mechanisms and neuronal morphology in the two species are distinct, we directly compared motion encoding in these two species at the algorithmic level, using matched stimuli and focusing on a pair of analogous neurons, the mouse ON starburst amacrine cell (ON SAC) and Drosophila T4 neurons. We find that the cells share similar spatiotemporal receptive field structures, sensitivity to spatiotemporal correlations, and tuning to sinusoidal drifting gratings, but differ in their responses to apparent motion stimuli. Both neuron types showed a response to summed sinusoids that deviates from models for motion processing in these cells, underscoring the similarities in their processing and identifying response features that remain to be explained.

Long-timescale anti-directional rotation in Drosophila optomotor behavior.

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied Drosophila melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such 'anti-directional turning' is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Long timescale anti-directional rotation in Drosophila optomotor behavior.

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied D. melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such "anti-directional turning" is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Neural mechanisms to incorporate visual counterevidence in self motion estimation.

In selecting appropriate behaviors, animals should weigh sensory evidence both for and against specific beliefs about the world. For instance, animals measure optic flow to estimate and control their own rotation. However, existing models of flow detection can confuse the movement of external objects with genuine self motion. Here, we show that stationary patterns on the retina, which constitute negative evidence against self rotation, are used by the fruit fly Drosophila to suppress inappropriate stabilizing rotational behavior. In silico experiments show that artificial neural networks optimized to distinguish self and world motion similarly detect stationarity and incorporate negative evidence. Employing neural measurements and genetic manipulations, we identified components of the circuitry for stationary pattern detection, which runs parallel to the fly's motion- and optic flow-detectors. Our results exemplify how the compact brain of the fly incorporates negative evidence to improve heading stability, exploiting geometrical constraints of the visual world.

Odour motion sensing enhances navigation of complex plumes.

Odour plumes in the wild are spatially complex and rapidly fluctuating structures carried by turbulent airflows1-4. To successfully navigate plumes in search of food and mates, insects must extract and integrate multiple features of the odour signal, including odour identity5, intensity6 and timing6-12. Effective navigation requires balancing these multiple streams of olfactory information and integrating them with other sensory inputs, including mechanosensory and visual cues9,12,13. Studies dating back a century have indicated that, of these many sensory inputs, the wind provides the main directional cue in turbulent plumes, leading to the longstanding model of insect odour navigation as odour-elicited upwind motion6,8-12,14,15. Here we show that Drosophila melanogaster shape their navigational decisions using an additional directional cue-the direction of motion of odours-which they detect using temporal correlations in the odour signal between their two antennae. Using a high-resolution virtual-reality paradigm to deliver spatiotemporally complex fictive odours to freely walking flies, we demonstrate that such odour-direction sensing involves algorithms analogous to those in visual-direction sensing16. Combining simulations, theory and experiments, we show that odour motion contains valuable directional information that is absent from the airflow alone, and that both Drosophila and virtual agents are aided by that information in navigating naturalistic plumes. The generality of our findings suggests that odour-direction sensing may exist throughout the animal kingdom and could improve olfactory robot navigation in uncertain environments.

Feature maps: How the insect visual system organizes information.

A new study explores how a population of neurons in the insect brain responds to different features of visual scenes and discovers an unusual topographic map that organizes the information they encode.

Excitatory and inhibitory neural dynamics jointly tune motion detection.

Neurons integrate excitatory and inhibitory signals to produce their outputs, but the role of input timing in this integration remains poorly understood. Motion detection is a paradigmatic example of this integration, since theories of motion detection rely on different delays in visual signals. These delays allow circuits to compare scenes at different times to calculate the direction and speed of motion. Different motion detection circuits have different velocity sensitivity, but it remains untested how the response dynamics of individual cell types drive this tuning. Here, we sped up or slowed down specific neuron types in Drosophila's motion detection circuit by manipulating ion channel expression. Altering the dynamics of individual neuron types upstream of motion detectors increased their sensitivity to fast or slow visual motion, exposing distinct roles for excitatory and inhibitory dynamics in tuning directional signals, including a role for the amacrine cell CT1. A circuit model constrained by functional data and anatomy qualitatively reproduced the observed tuning changes. Overall, these results reveal how excitatory and inhibitory dynamics together tune a canonical circuit computation.

Neural mechanisms to exploit positional geometry for collision avoidance.

Visual motion provides rich geometrical cues about the three-dimensional configuration of the world. However, how brains decode the spatial information carried by motion signals remains poorly understood. Here, we study a collision-avoidance behavior in Drosophila as a simple model of motion-based spatial vision. With simulations and psychophysics, we demonstrate that walking Drosophila exhibit a pattern of slowing to avoid collisions by exploiting the geometry of positional changes of objects on near-collision courses. This behavior requires the visual neuron LPLC1, whose tuning mirrors the behavior and whose activity drives slowing. LPLC1 pools inputs from object and motion detectors, and spatially biased inhibition tunes it to the geometry of collisions. Connectomic analyses identified circuitry downstream of LPLC1 that faithfully inherits its response properties. Overall, our results reveal how a small neural circuit solves a specific spatial vision task by combining distinct visual features to exploit universal geometrical constraints of the visual world.

Identifying Inputs to Visual Projection Neurons in Drosophila Lobula by Analyzing Connectomic Data.

Electron microscopy (EM)-based connectomes provide important insights into how visual circuitry of fruit fly Drosophila computes various visual features, guiding and complementing behavioral and physiological studies. However, connectomic analyses of the lobula, a neuropil putatively dedicated to detecting object-like features, remains underdeveloped, largely because of incomplete data on the inputs to the brain region. Here, we attempted to map the columnar inputs into the Drosophila lobula neuropil by performing connectivity-based and morphology-based clustering on a densely reconstructed connectome dataset. While the dataset mostly lacked visual neuropils other than lobula, which would normally help identify inputs to lobula, our clustering analysis successfully extracted clusters of cells with homogeneous connectivity and morphology, likely representing genuine cell types. We were able to draw a correspondence between the resulting clusters and previously identified cell types, revealing previously undocumented connectivity between lobula input and output neurons. While future, more complete connectomic reconstructions are necessary to verify the results presented here, they can serve as a useful basis for formulating hypotheses on mechanisms of visual feature detection in lobula.

Parallel locomotor control strategies in mice and flies.

Our understanding of the neural basis of locomotor behavior can be informed by careful quantification of animal movement. Classical descriptions of legged locomotion have defined discrete locomotor gaits, characterized by distinct patterns of limb movement. Recent technical advances have enabled increasingly detailed characterization of limb kinematics across many species, imposing tighter constraints on neural control. Here, we highlight striking similarities between coordination patterns observed in two genetic model organisms: the laboratory mouse and Drosophila. Both species exhibit continuously-variable coordination patterns with similar low-dimensional structure, suggesting shared principles for limb coordination and descending neural control.

Shallow neural networks trained to detect collisions recover features of visual loom-selective neurons.

Animals have evolved sophisticated visual circuits to solve a vital inference problem: detecting whether or not a visual signal corresponds to an object on a collision course. Such events are detected by specific circuits sensitive to visual looming, or objects increasing in size. Various computational models have been developed for these circuits, but how the collision-detection inference problem itself shapes the computational structures of these circuits remains unknown. Here, inspired by the distinctive structures of LPLC2 neurons in the visual system of Drosophila, we build anatomically-constrained shallow neural network models and train them to identify visual signals that correspond to impending collisions. Surprisingly, the optimization arrives at two distinct, opposing solutions, only one of which matches the actual dendritic weighting of LPLC2 neurons. Both solutions can solve the inference problem with high accuracy when the population size is large enough. The LPLC2-like solutions reproduces experimentally observed LPLC2 neuron responses for many stimuli, and reproduces canonical tuning of loom sensitive neurons, even though the models are never trained on neural data. Thus, LPLC2 neuron properties and tuning are predicted by optimizing an anatomically-constrained neural network to detect impending collisions. More generally, these results illustrate how optimizing inference tasks that are important for an animal's perceptual goals can reveal and explain computational properties of specific sensory neurons.

A metabolic daylength measurement system mediates winter photoperiodism in plants.

Plants have served as a preeminent study system for photoperiodism due to their propensity to flower in concordance with the seasons. A nearly singular focus on understanding photoperiodic flowering has prevented the discovery of other photoperiod measuring systems necessary for vegetative health. Here, we use bioinformatics to identify photoperiod-induced genes in Arabidopsis. We show that one, PP2-A13, is expressed exclusively in, and required for, plant fitness in short, winter-like photoperiods. We create a real-time photoperiod reporter, using the PP2-A13 promoter driving luciferase, and show that photoperiodic regulation is independent of the canonical CO/FT mechanism for photoperiodic flowering. We then reveal that photosynthesis combines with circadian-clock-controlled starch production to regulate cellular sucrose levels to control photoperiodic expression of PP2-A13. This work demonstrates the existence of a photoperiod measuring system housed in the metabolic network of plants that functions to control seasonal cellular health.

Predicting individual neuron responses with anatomically constrained task optimization.

Artificial neural networks trained to solve sensory tasks can develop statistical representations that match those in biological circuits. However, it remains unclear whether they can reproduce properties of individual neurons. Here, we investigated how artificial networks predict individual neuron properties in the visual motion circuits of the fruit fly Drosophila. We trained anatomically constrained networks to predict movement in natural scenes, solving the same inference problem as fly motion detectors. Units in the artificial networks adopted many properties of analogous individual neurons, even though they were not explicitly trained to match these properties. Among these properties was the split into ON and OFF motion detectors, which is not predicted by classical motion detection models. The match between model and neurons was closest when models were trained to be robust to noise. These results demonstrate how anatomical, task, and noise constraints can explain properties of individual neurons in a small neural network.

Walking Drosophila navigate complex plumes using stochastic decisions biased by the timing of odor encounters.

How insects navigate complex odor plumes, where the location and timing of odor packets are uncertain, remains unclear. Here we imaged complex odor plumes simultaneously with freely-walking flies, quantifying how behavior is shaped by encounters with individual odor packets. We found that navigation was stochastic and did not rely on the continuous modulation of speed or orientation. Instead, flies turned stochastically with stereotyped saccades, whose direction was biased upwind by the timing of prior odor encounters, while the magnitude and rate of saccades remained constant. Further, flies used the timing of odor encounters to modulate the transition rates between walks and stops. In more regular environments, flies continuously modulate speed and orientation, even though encounters can still occur randomly due to animal motion. We find that in less predictable environments, where encounters are random in both space and time, walking flies navigate with random walks biased by encounter timing.

When walking along a city street, you might encounter a range of scents and odors, from the smells of coffee and food to those of exhaust fumes and garbage. The odors are swept to your nose by air currents that move scents in two different ways. They carry them downwind in a process called advection, but they also mix them chaotically with clean air in a process called turbulence. What results is an odor plume: a complex ever-changing structure resembling the smoke rising from a chimney. Within a plume, areas of highly concentrated odor particles break up into smaller parcels as they travel further from the odor source. This means that the concentration of the odor does not vary along a smooth gradient. Instead, the odor arrives in brief and unpredictable bursts. Despite this complexity, insects are able to use odor plumes with remarkable ease to navigate towards food sources. But how do they do this? Answering this question has proved challenging because odor plumes are usually invisible. Over the years, scientists have come up with a number of creative solutions to this problem, including releasing soap bubbles together with odors, or using wind tunnels to generate simpler, straight plumes in known locations. These approaches have shown that when insects encounter an odor, they surge upwind towards its source. When they lose track of the odor, they cast themselves crosswind in an effort to regain contact. But this does not explain how insects are able to navigate irregular odor plumes, in which both the timing and location of the odor bursts are unpredictable. Demir, Kadakia et al. have now bridged this gap by showing how fruit flies are attracted to smoke, an odorant that is also visible. By injecting irregular smoke plumes into a custom-built wind tunnel, and then imaging flies as they walked through it, Demir, Kadakia et al. showed that flies make random halts when navigating the plume. Each time they stop, they use the timing of the odor bursts reaching them to decide when to start moving again. Rather than turning every time they detect an odor, flies initiate turns at random times. When several odor bursts arrive in a short time, the flies tend to orient these turns upwind rather than downwind. Flies therefore rely on a different strategy to navigate irregular odor plumes than the 'surge and cast' method they use for regular odor streams. Successful navigation through complex irregular plumes involves a degree of random behavior. This helps the flies gather information about an unpredictable environment as they search for the source of the odor. These findings may help to understand how other insects use odor to navigate in the real world, for example, how mosquitoes track down human hosts.

Mechanism for analogous illusory motion perception in flies and humans.

Visual motion detection is one of the most important computations performed by visual circuits. Yet, we perceive vivid illusory motion in stationary, periodic luminance gradients that contain no true motion. This illusion is shared by diverse vertebrate species, but theories proposed to explain this illusion have remained difficult to test. Here, we demonstrate that in the fruit fly Drosophila, the illusory motion percept is generated by unbalanced contributions of direction-selective neurons' responses to stationary edges. First, we found that flies, like humans, perceive sustained motion in the stationary gradients. The percept was abolished when the elementary motion detector neurons T4 and T5 were silenced. In vivo calcium imaging revealed that T4 and T5 neurons encode the location and polarity of stationary edges. Furthermore, our proposed mechanistic model allowed us to predictably manipulate both the magnitude and direction of the fly's illusory percept by selectively silencing either T4 or T5 neurons. Interestingly, human brains possess the same mechanistic ingredients that drive our model in flies. When we adapted human observers to moving light edges or dark edges, we could manipulate the magnitude and direction of their percepts as well, suggesting that mechanisms similar to the fly's may also underlie this illusion in humans. By taking a comparative approach that exploits Drosophila neurogenetics, our results provide a causal, mechanistic account for a long-known visual illusion. These results argue that this illusion arises from architectures for motion detection that are shared across phyla.

Object-Displacement-Sensitive Visual Neurons Drive Freezing in Drosophila.

Visual systems are often equipped with neurons that detect small moving objects, which may represent prey, predators, or conspecifics. Although the processing properties of those neurons have been studied in diverse organisms, links between the proposed algorithms and animal behaviors or circuit mechanisms remain elusive. Here, we have investigated behavioral function, computational algorithm, and neurochemical mechanisms of an object-selective neuron, LC11, in Drosophila. With genetic silencing and optogenetic activation, we show that LC11 is necessary for a visual object-induced stopping behavior in walking flies, a form of short-term freezing, and its activity can promote stopping. We propose a new quantitative model for small object selectivity based on the physiology and anatomy of LC11 and its inputs. The model accurately reproduces LC11 responses by pooling fast-adapting, tightly size-tuned inputs. Direct visualization of neurotransmitter inputs to LC11 confirmed the model conjectures about upstream processing. Our results demonstrate how adaptation can enhance selectivity for behaviorally relevant, dynamic visual features.

Spatiotemporally precise optogenetic activation of sensory neurons in freely walking Drosophila.

Previous work has characterized how walking Drosophila coordinate the movements of individual limbs (DeAngelis et al., 2019). To understand the circuit basis of this coordination, one must characterize how sensory feedback from each limb affects walking behavior. However, it has remained difficult to manipulate neural activity in individual limbs of freely moving animals. Here, we demonstrate a simple method for optogenetic stimulation with body side-, body segment-, and limb-specificity that does not require real-time tracking. Instead, we activate at random, precise locations in time and space and use post hoc analysis to determine behavioral responses to specific activations. Using this method, we have characterized limb coordination and walking behavior in response to transient activation of mechanosensitive bristle neurons and sweet-sensing chemoreceptor neurons. Our findings reveal that activating these neurons has opposite effects on turning, and that activations in different limbs and body regions produce distinct behaviors.