Sparse spike trains and the limitation of rate codes underlying rapid behaviours.

Animals live in dynamic worlds where they use sensorimotor circuits to rapidly process information and drive behaviours. For example, dragonflies are aerial predators that react to movements of prey within tens of milliseconds. These pursuits are likely controlled by identified neurons in the dragonfly, which have well-characterized physiological responses to moving targets. Predominantly, neural activity in these circuits is interpreted in context of a rate code, where information is conveyed by changes in the number of spikes over a time period. However, such a description of neuronal activity is difficult to achieve in real-world, real-time scenarios. Here, we contrast a neuroscientists' post-hoc view of spiking activity with the information available to the animal in real-time. We describe how performance of a rate code is readily overestimated and outline a rate code's significant limitations in driving rapid behaviours.

Preattentive facilitation of target trajectories in a dragonfly visual neuron.

The ability to pursue targets in visually cluttered and distraction-rich environments is critical for predators such as dragonflies. Previously, we identified Centrifugal Small-Target Motion Detector 1 (CSTMD1), a dragonfly visual neuron likely involved in such target-tracking behaviour. CSTMD1 exhibits facilitated responses to targets moving along a continuous trajectory. Moreover, CSTMD1 competitively selects a single target out of a pair. Here, we conducted in vivo, intracellular recordings from CSTMD1 to examine the interplay between facilitation and selection, in response to the presentation of paired targets. We find that neuronal responses to both individual trajectories of simultaneous, paired targets are facilitated, rather than being constrained to the single, selected target. Additionally, switches in selection elicit suppression which is likely an important attribute underlying target pursuit. However, binocular experiments reveal these results are constrained to paired targets within the same visual hemifield, while selection of a target in one visual hemifield establishes ocular dominance that prevents facilitation or response to contralaterally presented targets. These results reveal that the dragonfly brain preattentively represents more than one target trajectory, to balance between attentional flexibility and resistance against distraction.

Dragonfly Neurons Selectively Attend to Targets Within Natural Scenes.

Aerial predators, such as the dragonfly, determine the position and movement of their prey even when both are moving through complex, natural scenes. This task is likely supported by a group of neurons in the optic lobe which respond to moving targets that subtend less than a few degrees. These Small Target Motion Detector (STMD) neurons are tuned to both target size and velocity, whilst also exhibiting facilitated responses to targets traveling along continuous trajectories. When presented with a pair of targets, some STMDs generate spiking activity that represent a competitive selection of one target, as if the alternative does not exist (i.e., selective attention). Here, we describe intracellular responses of CSTMD1 (an identified STMD) to the visual presentation of targets embedded within cluttered, natural scenes. We examine CSTMD1 response changes to target contrast, as well as a range of target and background velocities. We find that background motion affects CSTMD1 responses via the competitive selection between features within the natural scene. Here, robust discrimination of our artificially embedded "target" is limited to scenarios when its velocity is matched to, or greater than, the background velocity. Additionally, the background's direction of motion affects discriminability, though not in the manner observed in STMDs of other flying insects. Our results highlight that CSTMD1's competitive responses are to those features best matched to the neuron's underlying spatiotemporal tuning, whether from the embedded target or other features in the background clutter. In many scenarios, CSTMD1 responds robustly to targets moving through cluttered scenes. However, whether this neuronal system could underlie the task of competitively selecting slow moving prey against fast-moving backgrounds remains an open question.

Modeling Nonlinear Dendritic Processing of Facilitation in a Dragonfly Target-Tracking Neuron.

Dragonflies are highly skilled and successful aerial predators that are even capable of selectively attending to one target within a swarm. Detection and tracking of prey is likely to be driven by small target motion detector (STMD) neurons identified from several insect groups. Prior work has shown that dragonfly STMD responses are facilitated by targets moving on a continuous path, enhancing the response gain at the present and predicted future location of targets. In this study, we combined detailed morphological data with computational modeling to test whether a combination of dendritic morphology and nonlinear properties of NMDA receptors could explain these observations. We developed a hybrid computational model of neurons within the dragonfly optic lobe, which integrates numerical and morphological components. The model was able to generate potent facilitation for targets moving on continuous trajectories, including a localized spotlight of maximal sensitivity close to the last seen target location, as also measured during in vivo recordings. The model did not, however, include a mechanism capable of producing a traveling or spreading wave of facilitation. Our data support a strong role for the high dendritic density seen in the dragonfly neuron in enhancing non-linear facilitation. An alternative model based on the morphology of an unrelated type of motion processing neuron from a dipteran fly required more than three times higher synaptic gain in order to elicit similar levels of facilitation, despite having only 20% fewer synapses. Our data support a potential role for NMDA receptors in target tracking and also demonstrate the feasibility of combining biologically plausible dendritic computations with more abstract computational models for basic processing as used in earlier studies.

Nonlinear, neuronal adaptation in insect vision models improves target discrimination within repetitively moving backgrounds.

Neurons which respond selectively to small moving targets, even against a cluttered background, have been identified in several insect species. To investigate what underlies these robust and highly selective responses, researchers have probed the neuronal circuitry in target-detecting, visual pathways. Observations in flies reveal nonlinear adaptation over time, composed of a fast onset and gradual decay. This adaptive processing is seen in both of the independent, parallel pathways encoding either luminance increments (ON channel) or decrements (OFF channel). The functional significance of this adaptive phenomenon has not been determined from physiological studies, though the asymmetrical time course suggests a role in suppressing responses to repetitive stimuli. We tested this possibility by comparing an implementation of fast adaptation against alternatives, using a model of insect 'elementary small target motion detectors'. We conducted target-detecting simulations on various natural backgrounds, that were shifted via several movement profiles (and target velocities). Using performance metrics, we confirmed that the fast adaptation observed in neuronal systems enhances target detection against a repetitively moving background. Such background movement would be encountered via natural ego-motion as the insect travels through the world. These findings show that this form of nonlinear, fast-adaptation (suitably implementable via cellular biophysics) plays a role analogous to background subtraction techniques in conventional computer vision.

Spike bursting in a dragonfly target-detecting neuron.

Dragonflies visually detect prey and conspecifics, rapidly pursuing these targets via acrobatic flights. Over many decades, studies have investigated the elaborate neuronal circuits proposed to underlie this rapid behaviour. A subset of dragonfly visual neurons exhibit exquisite tuning to small, moving targets even when presented in cluttered backgrounds. In prior work, these neuronal responses were quantified by computing the rate of spikes fired during an analysis window of interest. However, neuronal systems can utilize a variety of neuronal coding principles to signal information, so a spike train's information content is not necessarily encapsulated by spike rate alone. One example of this is burst coding, where neurons fire rapid bursts of spikes, followed by a period of inactivity. Here we show that the most studied target-detecting neuron in dragonflies, CSTMD1, responds to moving targets with a series of spike bursts. This spiking activity differs from those in other identified visual neurons in the dragonfly, indicative of different physiological mechanisms underlying CSTMD1's spike generation. Burst codes present several advantages and disadvantages compared to other coding approaches. We propose functional implications of CSTMD1's burst coding activity and show that spike bursts enhance the robustness of target-evoked responses.

The visual neuroecology of anisoptera.

Dragonflies belong to the oldest known lineage of flying animals, found across the globe around streams, ponds and forests. They are insect predators, specialising in ambush attack as aquatic larvae and rapid pursuit as adults. Dragonfly adults hunt amidst swarms in conditions that confuse many predatory species, and exhibit capture rates above 90%. Underlying the performance of such a remarkable predator is a finely tuned visual system capable of tracking targets amidst distractors and background clutter. The dragonfly performs a complex repertoire of flight behaviours, from near-motionless hovering to acute turns at high speeds. Here, we review the optical, neuronal, and behavioural adaptations that underlie the dragonflies' ability to achieve such remarkable predatory success.

Differential Tuning to Visual Motion Allows Robust Encoding of Optic Flow in the Dragonfly.

Visual cues provide an important means for aerial creatures to ascertain their self-motion through the environment. In many insects, including flies, moths, and bees, wide-field motion-sensitive neurons in the third optic ganglion are thought to underlie such motion encoding; however, these neurons can only respond robustly over limited speed ranges. The task is more complicated for some species of dragonflies that switch between extended periods of hovering flight and fast-moving pursuit of prey and conspecifics, requiring motion detection over a broad range of velocities. Since little is known about motion processing in these insects, we performed intracellular recordings from hawking, emerald dragonflies (Hemicordulia spp.) and identified a diverse group of motion-sensitive neurons that we named lobula tangential cells (LTCs). Following prolonged visual stimulation with drifting gratings, we observed significant differences in both temporal and spatial tuning of LTCs. Cluster analysis of these changes confirmed several groups of LTCs with distinctive spatiotemporal tuning. These differences were associated with variation in velocity tuning in response to translated, natural scenes. LTCs with differences in velocity tuning ranges and optima may underlie how a broad range of motion velocities are encoded. In the hawking dragonfly, changes in LTC tuning over time are therefore likely to support their extensive range of behaviors, from hovering to fast-speed pursuits.SIGNIFICANCE STATEMENT Understanding how animals navigate the world is an inherently difficult and interesting problem. Insects are useful models for understanding neuronal mechanisms underlying these activities, with neurons that encode wide-field motion previously identified in insects, such as flies, hawkmoths, and butterflies. Like some Dipteran flies, dragonflies exhibit complex aerobatic behaviors, such as hovering, patrolling, and aerial combat. However, dragonflies lack halteres that support such diverse behavior in flies. To understand how dragonflies might address this problem using only visual cues, we recorded from their wide-field motion-sensitive neurons. We found these differ strongly in the ways they respond to sustained motion, allowing them collectively to encode the very broad range of velocities experienced during diverse behavior.

A Target-Detecting Visual Neuron in the Dragonfly Locks on to Selectively Attended Targets.

The visual world projects a complex and rapidly changing image onto the retina of many animal species. This presents computational challenges for those animals reliant on visual processing to provide an accurate representation of the world. One such challenge is parsing a visual scene for the most salient targets, such as the selection of prey amid a swarm. The ability to selectively prioritize processing of some stimuli over others is known as 'selective attention'. We recently identified a dragonfly visual neuron called 'Centrifugal Small Target Motion Detector 1' (CSTMD1) that exhibits selective attention when presented with multiple, equally salient targets. Here we conducted in vivo, electrophysiological recordings from CSTMD1 in wild-caught male dragonflies (Hemicordulia tau), while presenting visual stimuli on an LCD monitor. To identify the target selected in any given trial, we uniquely modulated the intensity of the moving targets (frequency tagging). We found that the frequency information of the selected target is preserved in the neuronal response, while the distracter is completely ignored. We also show that the competitive system that underlies selection in this neuron can be biased by the presentation of a preceding target on the same trajectory, even when it is of lower contrast than an abrupt, novel distracter. With this improved method for identifying and biasing target selection in CSTMD1, the dragonfly provides an ideal animal model system to probe the neuronal mechanisms underlying selective attention.SIGNIFICANCE STATEMENT We present the first application of frequency tagging to intracellular neuronal recordings, demonstrating that the frequency component of a stimulus is encoded in the spiking response of an individual neuron. Using this technique as an identifier, we demonstrate that CSTMD1 'locks on' to a selected target and encodes the absolute strength of this target, even in the presence of abruptly appearing, high-contrast distracters. The underlying mechanism also permits the selection mechanism to switch between targets mid-trial, even among equivalent targets. Together, these results demonstrate greater complexity in this selective attention system than would be expected in a winner-takes-all network. These results are in contrast to typical findings in the primate and avian brain, but display intriguing resemblance to observations in human psychophysics.

Properties of predictive gain modulation in a dragonfly visual neuron.

Dragonflies pursue and capture tiny prey and conspecifics with extremely high success rates. These moving targets represent a small visual signal on the retina and successful chases require accurate detection and amplification by downstream neuronal circuits. This amplification has been observed in a population of neurons called small target motion detectors (STMDs), through a mechanism we term predictive gain modulation. As targets drift through the neuron's receptive field, spike frequency builds slowly over time. This increased likelihood of spiking or gain is modulated across the receptive field, enhancing sensitivity just ahead of the target's path, with suppression of activity in the remaining surround. Whilst some properties of this mechanism have been described, it is not yet known which stimulus parameters modulate the amount of response gain. Previous work suggested that the strength of gain enhancement was predominantly determined by the duration of the target's prior path. Here, we show that predictive gain modulation is more than a slow build-up of responses over time. Rather, the strength of gain is dependent on the velocity of a prior stimulus combined with the current stimulus attributes (e.g. angular size). We also describe response variability as a major challenge of target-detecting neurons and propose that the role of predictive gain modulation is to drive neurons towards response saturation, thus minimising neuronal variability despite noisy visual input signals.

Photoreceptor signalling is sufficient to explain the detectability threshold of insect aerial pursuers.

An essential biological task for many flying insects is the detection of small, moving targets, such as when pursuing prey or conspecifics. Neural pathways underlying such 'target-detecting' behaviours have been investigated for their sensitivity and tuning properties (size, velocity). However, which stage of neuronal processing limits target detection is not yet known. Here, we investigated several skilled, aerial pursuers (males of four insect species), measuring the target-detection limit (signal-to-noise ratio) of light-adapted photoreceptors. We recorded intracellular responses to moving targets of varying size, extended well below the nominal resolution of single ommatidia. We found that the signal detection limit (2× photoreceptor noise) matches physiological or behavioural target-detection thresholds observed in each species. Thus, across a diverse range of flying insects, individual photoreceptor responses to changes in light intensity establish the sensitivity of the feature detection pathway, indicating later stages of processing are dedicated to feature tuning, tracking and selection.

A predictive focus of gain modulation encodes target trajectories in insect vision.

When a human catches a ball, they estimate future target location based on the current trajectory. How animals, small and large, encode such predictive processes at the single neuron level is unknown. Here we describe small target-selective neurons in predatory dragonflies that exhibit localized enhanced sensitivity for targets displaced to new locations just ahead of the prior path, with suppression elsewhere in the surround. This focused region of gain modulation is driven by predictive mechanisms, with the direction tuning shifting selectively to match the target's prior path. It involves a large local increase in contrast gain which spreads forward after a delay (e.g. an occlusion) and can even transfer between brain hemispheres, predicting trajectories moved towards the visual midline from the other eye. The tractable nature of dragonflies for physiological experiments makes this a useful model for studying the neuronal mechanisms underlying the brain's remarkable ability to anticipate moving stimuli.

An autonomous robot inspired by insect neurophysiology pursues moving features in natural environments.

OBJECTIVE: Many computer vision and robotic applications require the implementation of robust and efficient target-tracking algorithms on a moving platform. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. Lightweight and low-powered flying insects, such as dragonflies, track prey or conspecifics within cluttered natural environments, illustrating an efficient biological solution to the target-tracking problem. APPROACH: We used our recent recordings from 'small target motion detector' neurons in the dragonfly brain to inspire the development of a closed-loop target detection and tracking algorithm. This model exploits facilitation, a slow build-up of response to targets which move along long, continuous trajectories, as seen in our electrophysiological data. To test performance in real-world conditions, we implemented this model on a robotic platform that uses active pursuit strategies based on insect behaviour. MAIN RESULTS: Our robot performs robustly in closed-loop pursuit of targets, despite a range of challenging conditions used in our experiments; low contrast targets, heavily cluttered environments and the presence of distracters. We show that the facilitation stage boosts responses to targets moving along continuous trajectories, improving contrast sensitivity and detection of small moving targets against textured backgrounds. Moreover, the temporal properties of facilitation play a useful role in handling vibration of the robotic platform. We also show that the adoption of feed-forward models which predict the sensory consequences of self-movement can significantly improve target detection during saccadic movements. SIGNIFICANCE: Our results provide insight into the neuronal mechanisms that underlie biological target detection and selection (from a moving platform), as well as highlight the effectiveness of our bio-inspired algorithm in an artificial visual system.

Visual acuity of the honey bee retina and the limits for feature detection.

Visual abilities of the honey bee have been studied for more than 100 years, recently revealing unexpectedly sophisticated cognitive skills rivalling those of vertebrates. However, the physiological limits of the honey bee eye have been largely unaddressed and only studied in an unnatural, dark state. Using a bright display and intracellular recordings, we here systematically investigated the angular sensitivity across the light adapted eye of honey bee foragers. Angular sensitivity is a measure of photoreceptor receptive field size and thus small values indicate higher visual acuity. Our recordings reveal a fronto-ventral acute zone in which angular sensitivity falls below 1.9°, some 30% smaller than previously reported. By measuring receptor noise and responses to moving dark objects, we also obtained direct measures of the smallest features detectable by the retina. In the frontal eye, single photoreceptors respond to objects as small as 0.6° × 0.6°, with >99% reliability. This indicates that honey bee foragers possess significantly better resolution than previously reported or estimated behaviourally, and commonly assumed in modelling of bee acuity.

Performance of an insect-inspired target tracker in natural conditions.

Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.

Properties of neuronal facilitation that improve target tracking in natural pursuit simulations.

Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.

Contrast sensitivity and the detection of moving patterns and features.

Theories based on optimal sampling by the retina have been widely applied to visual ecology at the level of the optics of the eye, supported by visual behaviour. This leads to speculation about the additional processing that must lie in between-in the brain itself. But fewer studies have adopted a quantitative approach to evaluating the detectability of specific features in these neural pathways. We briefly review this approach with a focus on contrast sensitivity of two parallel pathways for motion processing in insects, one used for analysis of wide-field optic flow, the other for detection of small features. We further use a combination of optical modelling of image blur and physiological recording from both photoreceptors and higher-order small target motion detector neurons sensitive to small targets to show that such neurons operate right at the limits imposed by the optics of the eye and the noise level of single photoreceptors. Despite this, and the limitation of only being able to use information from adjacent receptors to detect target motion, they achieve a contrast sensitivity that rivals that of wide-field motion sensitive pathways in either insects or vertebrates-among the highest in absolute terms seen in any animal.

Correlation between OFF and ON channels underlies dark target selectivity in an insect visual system.

In both vertebrates and invertebrates, evidence supports separation of luminance increments and decrements (ON and OFF channels) in early stages of visual processing (Hartline, 1938; Joesch et al., 2010); however, less is known about how these parallel pathways are recombined to encode form and motion. In Drosophila, genetic knockdown of inputs to putative ON and OFF pathways and direct recording from downstream neurons in the wide-field motion pathway reveal that local elementary motion detectors exist in pairs that separately correlate contrast polarity channels, ON with ON and OFF with OFF (Joesch et al., 2013). However, behavioral responses to reverse-phi motion of discrete features reveal additional correlations of the opposite signs (Clark et al., 2011). We here present intracellular recordings from feature detecting neurons in the dragonfly that provide direct physiological evidence for the correlation of OFF and ON pathways. These neurons show clear polarity selectivity for feature contrast, responding strongly to targets that are darker than the background and only weakly to dark contrasting edges. These dark target responses are much stronger than the linear combination of responses to ON and OFF edges. We compare these data with output from elementary motion detector-based models (Eichner et al., 2011; Clark et al., 2011), with and without stages of strong center-surround antagonism. Our data support an alternative elementary small target motion detector model, which derives dark target selectivity from the correlation of a delayed OFF with an un-delayed ON signal at each individual visual processing unit (Wiederman et al., 2008, 2009).

Selective attention in an insect visual neuron.

Animals need attention to focus on one target amid alternative distracters. Dragonflies, for example, capture flies in swarms comprising prey and conspecifics, a feat that requires neurons to select one moving target from competing alternatives. Diverse evidence, from functional imaging and physiology to psychophysics, highlights the importance of such "competitive selection" in attention for vertebrates. Analogous mechanisms have been proposed in artificial intelligence and even in invertebrates, yet direct neural correlates of attention are scarce from all animal groups. Here, we demonstrate responses from an identified dragonfly visual neuron that perfectly match a model for competitive selection within limits of neuronal variability (r(2) = 0.83). Responses to individual targets moving at different locations within the receptive field differ in both magnitude and time course. However, responses to two simultaneous targets exclusively track those for one target alone rather than any combination of the pair. Irrespective of target size, contrast, or separation, this neuron selects one target from the pair and perfectly preserves the response, regardless of whether the "winner" is the stronger stimulus if presented alone. This neuron is amenable to electrophysiological recordings, providing neuroscientists with a new model system for studying selective attention.