Reinforcement learning as a robotics-inspired framework for insect navigation: from spatial representations to neural implementation.

Bees are among the master navigators of the insect world. Despite impressive advances in robot navigation research, the performance of these insects is still unrivaled by any artificial system in terms of training efficiency and generalization capabilities, particularly considering the limited computational capacity. On the other hand, computational principles underlying these extraordinary feats are still only partially understood. The theoretical framework of reinforcement learning (RL) provides an ideal focal point to bring the two fields together for mutual benefit. In particular, we analyze and compare representations of space in robot and insect navigation models through the lens of RL, as the efficiency of insect navigation is likely rooted in an efficient and robust internal representation, linking retinotopic (egocentric) visual input with the geometry of the environment. While RL has long been at the core of robot navigation research, current computational theories of insect navigation are not commonly formulated within this framework, but largely as an associative learning process implemented in the insect brain, especially in the mushroom body (MB). Here we propose specific hypothetical components of the MB circuit that would enable the implementation of a certain class of relatively simple RL algorithms, capable of integrating distinct components of a navigation task, reminiscent of hierarchical RL models used in robot navigation. We discuss how current models of insect and robot navigation are exploring representations beyond classical, complete map-like representations, with spatial information being embedded in the respective latent representations to varying degrees.

Displacement experiments provide evidence for path integration in Drosophila.

Like many other animals, insects are capable of returning to previously visited locations using path integration, which is a memory of travelled direction and distance. Recent studies suggest that Drosophila can also use path integration to return to a food reward. However, the existing experimental evidence for path integration in Drosophila has a potential confound: pheromones deposited at the site of reward might enable flies to find previously rewarding locations even without memory. Here, we show that pheromones can indeed cause naïve flies to accumulate where previous flies had been rewarded in a navigation task. Therefore, we designed an experiment to determine if flies can use path integration memory despite potential pheromonal cues by displacing the flies shortly after an optogenetic reward. We found that rewarded flies returned to the location predicted by a memory-based model. Several analyses are consistent with path integration as the mechanism by which flies returned to the reward. We conclude that although pheromones are often important in fly navigation and must be carefully controlled for in future experiments, Drosophila may indeed be capable of performing path integration.

Real-Time Tracking of Multiple Moving Mosquitoes.

Tracking mosquitoes in real time, as opposed to recording video files and performing the tracking step later, is useful for two reasons. The first is efficiency. Real-time tracking requires less storage because video images do not need to be saved and followed by a tracking step. The second is that tracking data can be used to interact with the animal in some way, such as triggering the approach of a looming object. In this protocol, we discuss the use of Braid, free software for performing real-time, multicamera, multianimal tracking. We describe a setup with four cameras capable of tracking the three-dimensional (3D) position of mosquitoes at 100 frames per second in a volume of 30 cm × 30 cm × 60 cm with millimeter accuracy. The specific hardware configuration is flexible and can be substituted using different or additional components to adjust the tracking parameters as needed.

A Rapid, Highly Sensitive and Open-Access SARS-CoV-2 Detection Assay for Laboratory and Home Testing.

RT-qPCR-based diagnostic tests play important roles in combating virus-caused pandemics such as Covid-19. However, their dependence on sophisticated equipment and the associated costs often limits their widespread use. Loop-mediated isothermal amplification after reverse transcription (RT-LAMP) is an alternative nucleic acid detection method that overcomes these limitations. Here, we present a rapid, robust, and sensitive RT-LAMP-based SARS-CoV-2 detection assay. Our 40-min procedure bypasses the RNA isolation step, is insensitive to carryover contamination, and uses a colorimetric readout that enables robust SARS-CoV-2 detection from various sample types. Based on this assay, we have increased sensitivity and scalability by adding a nucleic acid enrichment step (Bead-LAMP), developed a version for home testing (HomeDip-LAMP), and identified open-source RT-LAMP enzymes that can be produced in any molecular biology laboratory. On a dedicated website, rtlamp.org (DOI: 10.5281/zenodo.6033689), we provide detailed protocols and videos. Our optimized, general-purpose RT-LAMP assay is an important step toward population-scale SARS-CoV-2 testing.

The olfactory gating of visual preferences to human skin and visible spectra in mosquitoes.

Mosquitoes track odors, locate hosts, and find mates visually. The color of a food resource, such as a flower or warm-blooded host, can be dominated by long wavelengths of the visible light spectrum (green to red for humans) and is likely important for object recognition and localization. However, little is known about the hues that attract mosquitoes or how odor affects mosquito visual search behaviors. We use a real-time 3D tracking system and wind tunnel that allows careful control of the olfactory and visual environment to quantify the behavior of more than 1.3 million mosquito trajectories. We find that CO2 induces a strong attraction to specific spectral bands, including those that humans perceive as cyan, orange, and red. Sensitivity to orange and red correlates with mosquitoes' strong attraction to the color spectrum of human skin, which is dominated by these wavelengths. The attraction is eliminated by filtering the orange and red bands from the skin color spectrum and by introducing mutations targeting specific long-wavelength opsins or CO2 detection. Collectively, our results show that odor is critical for mosquitoes' wavelength preferences and that the mosquito visual system is a promising target for inhibiting their attraction to human hosts.

Diurnal and nocturnal mosquitoes escape looming threats using distinct flight strategies.

Flying insects have evolved the ability to evade looming objects, such as predators and swatting hands. This is particularly relevant for blood-feeding insects, such as mosquitoes that routinely need to evade the defensive actions of their blood hosts. To minimize the chance of being swatted, a mosquito can use two distinct strategies-continuously exhibiting an unpredictable flight path or maximizing its escape maneuverability. We studied how baseline flight unpredictability and escape maneuverability affect the escape performance of day-active and night-active mosquitoes (Aedes aegypti and Anopheles coluzzii, respectively). We used a multi-camera high-speed videography system to track how freely flying mosquitoes respond to an event-triggered rapidly approaching mechanical swatter, in four different light conditions ranging from pitch darkness to overcast daylight. Results show that both species exhibit enhanced escape performance in their natural blood-feeding light condition (daylight for Aedes and dark for Anopheles). To achieve this, they show strikingly different behaviors. The enhanced escape performance of Anopheles at night is explained by their increased baseline unpredictable erratic flight behavior, whereas the increased escape performance of Aedes in overcast daylight is due to their enhanced escape maneuvers. This shows that both day and night-active mosquitoes modify their flight behavior in response to light intensity such that their escape performance is maximum in their natural blood-feeding light conditions, when these defensive actions by their blood hosts occur most. Because Aedes and Anopheles mosquitoes are major vectors of several deadly human diseases, this knowledge can be used to optimize vector control methods for these specific species.

Review of Methods for Animal Videography Using Camera Systems that Automatically Move to Follow the Animal.

Digital photography and videography provide rich data for the study of animal behavior and are consequently widely used techniques. For fixed, unmoving cameras there is a resolution versus field-of-view tradeoff and motion blur smears the subject on the sensor during exposure. While these fundamental tradeoffs with stationary cameras can be sidestepped by employing multiple cameras and providing additional illumination, this may not always be desirable. An alternative that overcomes these issues of stationary cameras is to direct a high-magnification camera at an animal continually as it moves. Here, we review systems in which automatic tracking is used to maintain an animal in the working volume of a moving optical path. Such methods provide an opportunity to escape the tradeoff between resolution and field of view and also to reduce motion blur while still enabling automated image acquisition. We argue that further development will be useful and outline potential innovations that may improve the technology and lead to more widespread use.

A neurodevelopmental origin of behavioral individuality in the Drosophila visual system.

The genome versus experience dichotomy has dominated understanding of behavioral individuality. By contrast, the role of nonheritable noise during brain development in behavioral variation is understudied. Using Drosophila melanogaster, we demonstrate a link between stochastic variation in brain wiring and behavioral individuality. A visual system circuit called the dorsal cluster neurons (DCN) shows nonheritable, interindividual variation in right/left wiring asymmetry and controls object orientation in freely walking flies. We show that DCN wiring asymmetry instructs an individual's object responses: The greater the asymmetry, the better the individual orients toward a visual object. Silencing DCNs abolishes correlations between anatomy and behavior, whereas inducing DCN asymmetry suffices to improve object responses.

Morphology, muscle capacity, skill, and maneuvering ability in hummingbirds.

How does agility evolve? This question is challenging because natural movement has many degrees of freedom and can be influenced by multiple traits. We used computer vision to record thousands of translations, rotations, and turns from more than 200 hummingbirds from 25 species, revealing that distinct performance metrics are correlated and that species diverge in their maneuvering style. Our analysis demonstrates that the enhanced maneuverability of larger species is explained by their proportionately greater muscle capacity and lower wing loading. Fast acceleration maneuvers evolve by recruiting changes in muscle capacity, whereas fast rotations and sharp turns evolve by recruiting changes in wing morphology. Both species and individuals use turns that play to their strengths. These results demonstrate how both skill and biomechanical traits shape maneuvering behavior.

Virtual reality for freely moving animals.

Standard animal behavior paradigms incompletely mimic nature and thus limit our understanding of behavior and brain function. Virtual reality (VR) can help, but it poses challenges. Typical VR systems require movement restrictions but disrupt sensorimotor experience, causing neuronal and behavioral alterations. We report the development of FreemoVR, a VR system for freely moving animals. We validate immersive VR for mice, flies, and zebrafish. FreemoVR allows instant, disruption-free environmental reconfigurations and interactions between real organisms and computer-controlled agents. Using the FreemoVR platform, we established a height-aversion assay in mice and studied visuomotor effects in Drosophila and zebrafish. Furthermore, by photorealistically mimicking zebrafish we discovered that effective social influence depends on a prospective leader balancing its internally preferred directional choice with social interaction. FreemoVR technology facilitates detailed investigations into neural function and behavior through the precise manipulation of sensorimotor feedback loops in unrestrained animals.

Mechanical Constraints on Flight at High Elevation Decrease Maneuvering Performance of Hummingbirds.

High-elevation habitats offer ecological advantages including reduced competition, predation, and parasitism [1]. However, flying organisms at high elevation also face physiological challenges due to lower air density and oxygen availability [2]. These constraints are expected to affect the flight maneuvers that are required to compete with rivals, capture prey, and evade threats [3-5]. To test how individual maneuvering performance is affected by elevation, we measured the free-flight maneuvers of male Anna's hummingbirds in a large chamber translocated to a high-elevation site and then measured their performance at low elevation. We used a multi-camera tracking system to identify thousands of maneuvers based on body position and orientation [6]. At high elevation, the birds' translational velocities, accelerations, and rotational velocities were reduced, and they used less demanding turns. To determine how mechanical and metabolic constraints independently affect performance, we performed a second experiment to evaluate flight maneuvers in an airtight chamber infused with either normoxic heliox, to lower air density, or nitrogen, to lower oxygen availability. The hypodense treatment caused the birds to reduce their accelerations and rotational velocities, whereas the hypoxic treatment had no significant effect on maneuvering performance. Collectively, these experiments reveal how aerial maneuvering performance changes with elevation, demonstrating that as birds move up in elevation, air density constrains their maneuverability prior to any influence of oxygen availability. Our results support the hypothesis that changes in competitive ability at high elevations are the result of mechanical limits to flight performance [7].

Automatic Segmentation of Drosophila Neural Compartments Using GAL4 Expression Data Reveals Novel Visual Pathways.

Identifying distinct anatomical structures within the brain and developing genetic tools to target them are fundamental steps for understanding brain function. We hypothesize that enhancer expression patterns can be used to automatically identify functional units such as neuropils and fiber tracts. We used two recent, genome-scale Drosophila GAL4 libraries and associated confocal image datasets to segment large brain regions into smaller subvolumes. Our results (available at https://strawlab.org/braincode) support this hypothesis because regions with well-known anatomy, namely the antennal lobes and central complex, were automatically segmented into familiar compartments. The basis for the structural assignment is clustering of voxels based on patterns of enhancer expression. These initial clusters are agglomerated to make hierarchical predictions of structure. We applied the algorithm to central brain regions receiving input from the optic lobes. Based on the automated segmentation and manual validation, we can identify and provide promising driver lines for 11 previously identified and 14 novel types of visual projection neurons and their associated optic glomeruli. The same strategy can be used in other brain regions and likely other species, including vertebrates.

Burst muscle performance predicts the speed, acceleration, and turning performance of Anna's hummingbirds.

Despite recent advances in the study of animal flight, the biomechanical determinants of maneuverability are poorly understood. It is thought that maneuverability may be influenced by intrinsic body mass and wing morphology, and by physiological muscle capacity, but this hypothesis has not yet been evaluated because it requires tracking a large number of free flight maneuvers from known individuals. We used an automated tracking system to record flight sequences from 20 Anna's hummingbirds flying solo and in competition in a large chamber. We found that burst muscle capacity predicted most performance metrics. Hummingbirds with higher burst capacity flew with faster velocities, accelerations, and rotations, and they used more demanding complex turns. In contrast, body mass did not predict variation in maneuvering performance, and wing morphology predicted only the use of arcing turns and high centripetal accelerations. Collectively, our results indicate that burst muscle capacity is a key predictor of maneuverability.

Asymmetric processing of visual motion for simultaneous object and background responses.

Visual object fixation and figure-ground discrimination in Drosophila are robust behaviors requiring sophisticated computation by the visual system, yet the neural substrates remain unknown. Recent experiments in walking flies revealed object fixation behavior mediated by circuitry independent from the motion-sensitive T4-T5 cells required for wide-field motion responses. In tethered flight experiments under closed-loop conditions, we found similar results for one feedback gain, whereas intact T4-T5 cells were necessary for robust object fixation at a higher feedback gain and in figure-ground discrimination tasks. We implemented dynamical models (available at http://strawlab.org/asymmetric-motion/) based on neurons downstream of T4-T5 cells­one a simple phenomenological model and another, physiologically more realistic model­and found that both predict key features of stripe fixation and figure-ground discrimination and are consistent with a classical formulation. Fundamental to both models is motion asymmetry in the responses of model neurons, whereby front-to-back motion elicits stronger responses than back-to-front motion. When a bilateral pair of such model neurons, based on well-understood horizontal system cells, downstream of T4-T5, is coupled to turning behavior, asymmetry leads to object fixation and figure-ground discrimination in the presence of noise. Furthermore, the models also predict fixation in front of a moving background, a behavior previously suggested to require an additional pathway. Thus, the models predict several aspects of object responses on the basis of neurons that are also thought to serve a key role in background stabilization.

Automated image-based tracking and its application in ecology.

The behavior of individuals determines the strength and outcome of ecological interactions, which drive population, community, and ecosystem organization. Bio-logging, such as telemetry and animal-borne imaging, provides essential individual viewpoints, tracks, and life histories, but requires capture of individuals and is often impractical to scale. Recent developments in automated image-based tracking offers opportunities to remotely quantify and understand individual behavior at scales and resolutions not previously possible, providing an essential supplement to other tracking methodologies in ecology. Automated image-based tracking should continue to advance the field of ecology by enabling better understanding of the linkages between individual and higher-level ecological processes, via high-throughput quantitative analysis of complex ecological patterns and processes across scales, including analysis of environmental drivers.

FlyMAD: rapid thermogenetic control of neuronal activity in freely walking Drosophila.

Rapidly and selectively modulating the activity of defined neurons in unrestrained animals is a powerful approach in investigating the circuit mechanisms that shape behavior. In Drosophila melanogaster, temperature-sensitive silencers and activators are widely used to control the activities of genetically defined neuronal cell types. A limitation of these thermogenetic approaches, however, has been their poor temporal resolution. Here we introduce FlyMAD (the fly mind-altering device), which allows thermogenetic silencing or activation within seconds or even fractions of a second. Using computer vision, FlyMAD targets an infrared laser to freely walking flies. As a proof of principle, we demonstrated the rapid silencing and activation of neurons involved in locomotion, vision and courtship. The spatial resolution of the focused beam enabled preferential targeting of neurons in the brain or ventral nerve cord. Moreover, the high temporal resolution of FlyMAD allowed us to discover distinct timing relationships for two neuronal cell types previously linked to courtship song.

Flying Drosophila stabilize their vision-based velocity controller by sensing wind with their antennae.

Flies and other insects use vision to regulate their groundspeed in flight, enabling them to fly in varying wind conditions. Compared with mechanosensory modalities, however, vision requires a long processing delay (~100 ms) that might introduce instability if operated at high gain. Flies also sense air motion with their antennae, but how this is used in flight control is unknown. We manipulated the antennal function of fruit flies by ablating their aristae, forcing them to rely on vision alone to regulate groundspeed. Arista-ablated flies in flight exhibited significantly greater groundspeed variability than intact flies. We then subjected them to a series of controlled impulsive wind gusts delivered by an air piston and experimentally manipulated antennae and visual feedback. The results show that an antenna-mediated response alters wing motion to cause flies to accelerate in the same direction as the gust. This response opposes flying into a headwind, but flies regularly fly upwind. To resolve this discrepancy, we obtained a dynamic model of the fly's velocity regulator by fitting parameters of candidate models to our experimental data. The model suggests that the groundspeed variability of arista-ablated flies is the result of unstable feedback oscillations caused by the delay and high gain of visual feedback. The antenna response drives active damping with a shorter delay (~20 ms) to stabilize this regulator, in exchange for increasing the effect of rapid wind disturbances. This provides insight into flies' multimodal sensory feedback architecture and constitutes a previously unknown role for the antennae.

Circadian and circalunar clock interactions in a marine annelid.

Life is controlled by multiple rhythms. Although the interaction of the daily (circadian) clock with environmental stimuli, such as light, is well documented, its relationship to endogenous clocks with other periods is little understood. We establish that the marine worm Platynereis dumerilii possesses endogenous circadian and circalunar (monthly) clocks and characterize their interactions. The RNAs of likely core circadian oscillator genes localize to a distinct nucleus of the worm's forebrain. The worm's forebrain also harbors a circalunar clock entrained by nocturnal light. This monthly clock regulates maturation and persists even when circadian clock oscillations are disrupted by the inhibition of casein kinase 1δ/ε. Both circadian and circalunar clocks converge on the regulation of transcript levels. Furthermore, the circalunar clock changes the period and power of circadian behavior, although the period length of the daily transcriptional oscillations remains unaltered. We conclude that a second endogenous noncircadian clock can influence circadian clock function.

Discriminating external and internal causes for heading changes in freely flying Drosophila.

As animals move through the world in search of resources, they change course in reaction to both external sensory cues and internally-generated programs. Elucidating the functional logic of complex search algorithms is challenging because the observable actions of the animal cannot be unambiguously assigned to externally- or internally-triggered events. We present a technique that addresses this challenge by assessing quantitatively the contribution of external stimuli and internal processes. We apply this technique to the analysis of rapid turns ("saccades") of freely flying Drosophila melanogaster. We show that a single scalar feature computed from the visual stimulus experienced by the animal is sufficient to explain a majority (93%) of the turning decisions. We automatically estimate this scalar value from the observable trajectory, without any assumption regarding the sensory processing. A posteriori, we show that the estimated feature field is consistent with previous results measured in other experimental conditions. The remaining turning decisions, not explained by this feature of the visual input, may be attributed to a combination of deterministic processes based on unobservable internal states and purely stochastic behavior. We cannot distinguish these contributions using external observations alone, but we are able to provide a quantitative bound of their relative importance with respect to stimulus-triggered decisions. Our results suggest that comparatively few saccades in free-flying conditions are a result of an intrinsic spontaneous process, contrary to previous suggestions. We discuss how this technique could be generalized for use in other systems and employed as a tool for classifying effects into sensory, decision, and motor categories when used to analyze data from genetic behavioral screens.

Active and passive antennal movements during visually guided steering in flying Drosophila.

Insects use feedback from a variety of sensory modalities, including mechanoreceptors on their antennae, to stabilize the direction and speed of flight. Like all arthropod appendages, antennae not only supply sensory information but may also be actively positioned by control muscles. However, how flying insects move their antennae during active turns and how such movements might influence steering responses are currently unknown. Here we examined the antennal movements of flying Drosophila during visually induced turns in a tethered flight arena. In response to both rotational and translational patterns of visual motion, Drosophila actively moved their antennae in a direction opposite to that of the visual motion. We also observed two types of passive antennal movements: small tonic deflections of the antenna and rapid oscillations at wing beat frequency. These passive movements are likely the result of wing-induced airflow and increased in magnitude when the angular distance between the wing and the antenna decreased. In response to rotational visual motion, increases in passive antennal movements appear to trigger a reflex that reduces the stroke amplitude of the contralateral wing, thereby enhancing the visually induced turn. Although the active antennal movements significantly increased antennal oscillation by bringing the arista closer to the wings, it did not significantly affect the turning response in our head-fixed, tethered flies. These results are consistent with the hypothesis that flying Drosophila use mechanosensory feedback to detect changes in the wing induced airflow during visually induced turns and that this feedback plays a role in regulating the magnitude of steering responses.