Sparse spike trains and the limitation of rate codes underlying rapid behaviours.

Animals live in dynamic worlds where they use sensorimotor circuits to rapidly process information and drive behaviours. For example, dragonflies are aerial predators that react to movements of prey within tens of milliseconds. These pursuits are likely controlled by identified neurons in the dragonfly, which have well-characterized physiological responses to moving targets. Predominantly, neural activity in these circuits is interpreted in context of a rate code, where information is conveyed by changes in the number of spikes over a time period. However, such a description of neuronal activity is difficult to achieve in real-world, real-time scenarios. Here, we contrast a neuroscientists' post-hoc view of spiking activity with the information available to the animal in real-time. We describe how performance of a rate code is readily overestimated and outline a rate code's significant limitations in driving rapid behaviours.

Differential Tuning to Visual Motion Allows Robust Encoding of Optic Flow in the Dragonfly.

Visual cues provide an important means for aerial creatures to ascertain their self-motion through the environment. In many insects, including flies, moths, and bees, wide-field motion-sensitive neurons in the third optic ganglion are thought to underlie such motion encoding; however, these neurons can only respond robustly over limited speed ranges. The task is more complicated for some species of dragonflies that switch between extended periods of hovering flight and fast-moving pursuit of prey and conspecifics, requiring motion detection over a broad range of velocities. Since little is known about motion processing in these insects, we performed intracellular recordings from hawking, emerald dragonflies (Hemicordulia spp.) and identified a diverse group of motion-sensitive neurons that we named lobula tangential cells (LTCs). Following prolonged visual stimulation with drifting gratings, we observed significant differences in both temporal and spatial tuning of LTCs. Cluster analysis of these changes confirmed several groups of LTCs with distinctive spatiotemporal tuning. These differences were associated with variation in velocity tuning in response to translated, natural scenes. LTCs with differences in velocity tuning ranges and optima may underlie how a broad range of motion velocities are encoded. In the hawking dragonfly, changes in LTC tuning over time are therefore likely to support their extensive range of behaviors, from hovering to fast-speed pursuits.SIGNIFICANCE STATEMENT Understanding how animals navigate the world is an inherently difficult and interesting problem. Insects are useful models for understanding neuronal mechanisms underlying these activities, with neurons that encode wide-field motion previously identified in insects, such as flies, hawkmoths, and butterflies. Like some Dipteran flies, dragonflies exhibit complex aerobatic behaviors, such as hovering, patrolling, and aerial combat. However, dragonflies lack halteres that support such diverse behavior in flies. To understand how dragonflies might address this problem using only visual cues, we recorded from their wide-field motion-sensitive neurons. We found these differ strongly in the ways they respond to sustained motion, allowing them collectively to encode the very broad range of velocities experienced during diverse behavior.

A Target-Detecting Visual Neuron in the Dragonfly Locks on to Selectively Attended Targets.

The visual world projects a complex and rapidly changing image onto the retina of many animal species. This presents computational challenges for those animals reliant on visual processing to provide an accurate representation of the world. One such challenge is parsing a visual scene for the most salient targets, such as the selection of prey amid a swarm. The ability to selectively prioritize processing of some stimuli over others is known as 'selective attention'. We recently identified a dragonfly visual neuron called 'Centrifugal Small Target Motion Detector 1' (CSTMD1) that exhibits selective attention when presented with multiple, equally salient targets. Here we conducted in vivo, electrophysiological recordings from CSTMD1 in wild-caught male dragonflies (Hemicordulia tau), while presenting visual stimuli on an LCD monitor. To identify the target selected in any given trial, we uniquely modulated the intensity of the moving targets (frequency tagging). We found that the frequency information of the selected target is preserved in the neuronal response, while the distracter is completely ignored. We also show that the competitive system that underlies selection in this neuron can be biased by the presentation of a preceding target on the same trajectory, even when it is of lower contrast than an abrupt, novel distracter. With this improved method for identifying and biasing target selection in CSTMD1, the dragonfly provides an ideal animal model system to probe the neuronal mechanisms underlying selective attention.SIGNIFICANCE STATEMENT We present the first application of frequency tagging to intracellular neuronal recordings, demonstrating that the frequency component of a stimulus is encoded in the spiking response of an individual neuron. Using this technique as an identifier, we demonstrate that CSTMD1 'locks on' to a selected target and encodes the absolute strength of this target, even in the presence of abruptly appearing, high-contrast distracters. The underlying mechanism also permits the selection mechanism to switch between targets mid-trial, even among equivalent targets. Together, these results demonstrate greater complexity in this selective attention system than would be expected in a winner-takes-all network. These results are in contrast to typical findings in the primate and avian brain, but display intriguing resemblance to observations in human psychophysics.

Properties of predictive gain modulation in a dragonfly visual neuron.

Dragonflies pursue and capture tiny prey and conspecifics with extremely high success rates. These moving targets represent a small visual signal on the retina and successful chases require accurate detection and amplification by downstream neuronal circuits. This amplification has been observed in a population of neurons called small target motion detectors (STMDs), through a mechanism we term predictive gain modulation. As targets drift through the neuron's receptive field, spike frequency builds slowly over time. This increased likelihood of spiking or gain is modulated across the receptive field, enhancing sensitivity just ahead of the target's path, with suppression of activity in the remaining surround. Whilst some properties of this mechanism have been described, it is not yet known which stimulus parameters modulate the amount of response gain. Previous work suggested that the strength of gain enhancement was predominantly determined by the duration of the target's prior path. Here, we show that predictive gain modulation is more than a slow build-up of responses over time. Rather, the strength of gain is dependent on the velocity of a prior stimulus combined with the current stimulus attributes (e.g. angular size). We also describe response variability as a major challenge of target-detecting neurons and propose that the role of predictive gain modulation is to drive neurons towards response saturation, thus minimising neuronal variability despite noisy visual input signals.

Photoreceptor signalling is sufficient to explain the detectability threshold of insect aerial pursuers.

An essential biological task for many flying insects is the detection of small, moving targets, such as when pursuing prey or conspecifics. Neural pathways underlying such 'target-detecting' behaviours have been investigated for their sensitivity and tuning properties (size, velocity). However, which stage of neuronal processing limits target detection is not yet known. Here, we investigated several skilled, aerial pursuers (males of four insect species), measuring the target-detection limit (signal-to-noise ratio) of light-adapted photoreceptors. We recorded intracellular responses to moving targets of varying size, extended well below the nominal resolution of single ommatidia. We found that the signal detection limit (2× photoreceptor noise) matches physiological or behavioural target-detection thresholds observed in each species. Thus, across a diverse range of flying insects, individual photoreceptor responses to changes in light intensity establish the sensitivity of the feature detection pathway, indicating later stages of processing are dedicated to feature tuning, tracking and selection.

Correlation between OFF and ON channels underlies dark target selectivity in an insect visual system.

In both vertebrates and invertebrates, evidence supports separation of luminance increments and decrements (ON and OFF channels) in early stages of visual processing (Hartline, 1938; Joesch et al., 2010); however, less is known about how these parallel pathways are recombined to encode form and motion. In Drosophila, genetic knockdown of inputs to putative ON and OFF pathways and direct recording from downstream neurons in the wide-field motion pathway reveal that local elementary motion detectors exist in pairs that separately correlate contrast polarity channels, ON with ON and OFF with OFF (Joesch et al., 2013). However, behavioral responses to reverse-phi motion of discrete features reveal additional correlations of the opposite signs (Clark et al., 2011). We here present intracellular recordings from feature detecting neurons in the dragonfly that provide direct physiological evidence for the correlation of OFF and ON pathways. These neurons show clear polarity selectivity for feature contrast, responding strongly to targets that are darker than the background and only weakly to dark contrasting edges. These dark target responses are much stronger than the linear combination of responses to ON and OFF edges. We compare these data with output from elementary motion detector-based models (Eichner et al., 2011; Clark et al., 2011), with and without stages of strong center-surround antagonism. Our data support an alternative elementary small target motion detector model, which derives dark target selectivity from the correlation of a delayed OFF with an un-delayed ON signal at each individual visual processing unit (Wiederman et al., 2008, 2009).

Temperature modulates the tuning properties of small target motion detector neurons in the dragonfly visual system.

Dragonflies are poikilothermic animals with limited thermoregulation; therefore, their entire bodies, including the brain, experience a range of temperatures during their daily activities.1,2 These flying insects exhibit hunting prowess, pursuing prey or conspecifics whether in direct sunlight or under the cover of cloud.3,4 Likely to underlie these aerobatic feats are the small target motion detector (STMD) neurons.5 These visual neurons are sensitive to target contrast and tuned to the target's size and velocity, with some neurons exhibiting complex predictive and selective properties, well suited for prey interception and feeding amid swarms.3,4,6,7,8,9 Increased temperature can modulate the biochemical processes underlying neuronal processing, increasing sensitivity and quickening the responsiveness of insect photoreceptors and downstream optic flow neurons,10,11,12 while in other neuronal pathways, compensatory processes have been shown to account for temperature changes.13,14 We determined the ethological range of temperatures experienced by the dragonfly, Hemicordulia tau, in its natural environment. Across this behaviorally relevant range, we showed increased temperatures having a large 8.7-fold increase in the contrast sensitivity of STMD neurons. However, suppression of responses to larger targets was unaltered. STMD tuning for target velocities was changed remarkably, not only increasing the optimum but extending the fastest velocities encoded by an order of magnitude. These results caution against interpreting functionality underlying spike rates at constrained, experimental temperatures. Moreover, they raise intriguing new questions about how information is represented within the brain of these flying insects, given the relationship between visual stimulus parameters and neuronal activity varies so dramatically depending on current environmental conditions.

Localized and Long-Lasting Adaptation in Dragonfly Target-Detecting Neurons.

Some visual neurons in the dragonfly (Hemicordulia tau) optic lobe respond to small, moving targets, likely underlying their fast pursuit of prey and conspecifics. In response to repetitive targets presented at short intervals, the spiking activity of these "small target motion detector" (STMD) neurons diminishes over time. Previous experiments limited this adaptation by including intertrial rest periods of varying durations. However, the characteristics of this effect have never been quantified. Here, using extracellular recording techniques lasting for several hours, we quantified both the spatial and temporal properties of STMD adaptation. We found that the time course of adaptation was variable across STMD units. In any one STMD, a repeated series led to more rapid adaptation, a minor accumulative effect more akin to habituation. Following an adapting stimulus, responses recovered quickly, though the rate of recovery decreased nonlinearly over time. We found that the region of adaptation is highly localized, with targets displaced by ∼2.5° eliciting a naive response. Higher frequencies of target stimulation converged to lower levels of sustained response activity. We determined that adaptation itself is a target-tuned property, not elicited by moving bars or luminance flicker. As STMD adaptation is a localized phenomenon, dependent on recent history, it is likely to play an important role in closed-loop behavior where a target is foveated in a localized region for extended periods of the pursuit duration.

Discrimination of features in natural scenes by a dragonfly neuron.

Flying insects engage in spectacular high-speed pursuit of targets, requiring visual discrimination of moving objects against cluttered backgrounds. As a first step toward understanding the neural basis for this complex task, we used computational modeling of insect small target motion detector (STMD) neurons to predict responses to features within natural scenes and then compared this with responses recorded from an identified STMD neuron in the dragonfly brain (Hemicordulia tau). A surprising model prediction confirmed by our electrophysiological recordings is that even heavily cluttered scenes contain very few features that excite these neurons, due largely to their exquisite tuning for small features. We also show that very subtle manipulations of the image cause dramatic changes in the response of this neuron, because of the complex inhibitory and facilitatory interactions within the receptive field.

Assessment outcome is weakly correlated with lecture attendance: influence of learning style and use of alternative materials.

The relation between lecture attendance and learning is surprisingly weak, and the role of learning styles in this is poorly understood. We hypothesized that 1) academic performance is related to lecture attendance and 2) learning style influences lecture attendance and, consequently, affects performance. We also speculated that the availability of alternative resources would affect this relationship. Second-year Bachelor of Science physiology students (n = 120) self-reported their lecture attendance in a block of 21 lectures (attendance not compulsory) and use of alternative resources. Overall self-reported lecture attendance was 73 ± 2%. Female students (n = 71) attended more lectures (16.4 ± 0.6) than male students (14.3 ± 0.08, n = 49) and achieved a higher composite mark in all assessments (73.6% vs. 69.3%, P < 0.02). Marks in the final exam were not statistically different between the sexes and correlated only weakly with lecture attendance (r = 0.29, n = 49, P < 0.04 for male students; r = 0.10, n = 71, P = not significant for female students; and r =0.21, n = 120, P < 0.02 for the whole class). Of the students who passed the exam, poor attenders (<11 lectures) reported significantly more use of lecture recordings (37 ± 8%, n = 15, vs. 10 ± 1%, n = 85, P < 0.001). In a VARK learning style assessment (where V is visual, A is auditory, R is reading/writing, and K is kinesthetic), students were multimodal, although female students had a slightly higher average percentage of the R learning style (preferred read/write) compared with male students (28.9 ± 0.9%, n = 63, vs. 25.3 ± 1.3%, n = 32, P < 0.03). Lecture attendance was not correlated with measured learning style. We concluded that lecture attendance is only weakly correlated with academic performance and is not related to learning style. The substitution of alternative materials for lecture attendance appears to have a greater role than learning style in determining academic outcomes.

Dragonfly Neurons Selectively Attend to Targets Within Natural Scenes.

Aerial predators, such as the dragonfly, determine the position and movement of their prey even when both are moving through complex, natural scenes. This task is likely supported by a group of neurons in the optic lobe which respond to moving targets that subtend less than a few degrees. These Small Target Motion Detector (STMD) neurons are tuned to both target size and velocity, whilst also exhibiting facilitated responses to targets traveling along continuous trajectories. When presented with a pair of targets, some STMDs generate spiking activity that represent a competitive selection of one target, as if the alternative does not exist (i.e., selective attention). Here, we describe intracellular responses of CSTMD1 (an identified STMD) to the visual presentation of targets embedded within cluttered, natural scenes. We examine CSTMD1 response changes to target contrast, as well as a range of target and background velocities. We find that background motion affects CSTMD1 responses via the competitive selection between features within the natural scene. Here, robust discrimination of our artificially embedded "target" is limited to scenarios when its velocity is matched to, or greater than, the background velocity. Additionally, the background's direction of motion affects discriminability, though not in the manner observed in STMDs of other flying insects. Our results highlight that CSTMD1's competitive responses are to those features best matched to the neuron's underlying spatiotemporal tuning, whether from the embedded target or other features in the background clutter. In many scenarios, CSTMD1 responds robustly to targets moving through cluttered scenes. However, whether this neuronal system could underlie the task of competitively selecting slow moving prey against fast-moving backgrounds remains an open question.

Preattentive facilitation of target trajectories in a dragonfly visual neuron.

The ability to pursue targets in visually cluttered and distraction-rich environments is critical for predators such as dragonflies. Previously, we identified Centrifugal Small-Target Motion Detector 1 (CSTMD1), a dragonfly visual neuron likely involved in such target-tracking behaviour. CSTMD1 exhibits facilitated responses to targets moving along a continuous trajectory. Moreover, CSTMD1 competitively selects a single target out of a pair. Here, we conducted in vivo, intracellular recordings from CSTMD1 to examine the interplay between facilitation and selection, in response to the presentation of paired targets. We find that neuronal responses to both individual trajectories of simultaneous, paired targets are facilitated, rather than being constrained to the single, selected target. Additionally, switches in selection elicit suppression which is likely an important attribute underlying target pursuit. However, binocular experiments reveal these results are constrained to paired targets within the same visual hemifield, while selection of a target in one visual hemifield establishes ocular dominance that prevents facilitation or response to contralaterally presented targets. These results reveal that the dragonfly brain preattentively represents more than one target trajectory, to balance between attentional flexibility and resistance against distraction.

Spike bursting in a dragonfly target-detecting neuron.

Dragonflies visually detect prey and conspecifics, rapidly pursuing these targets via acrobatic flights. Over many decades, studies have investigated the elaborate neuronal circuits proposed to underlie this rapid behaviour. A subset of dragonfly visual neurons exhibit exquisite tuning to small, moving targets even when presented in cluttered backgrounds. In prior work, these neuronal responses were quantified by computing the rate of spikes fired during an analysis window of interest. However, neuronal systems can utilize a variety of neuronal coding principles to signal information, so a spike train's information content is not necessarily encapsulated by spike rate alone. One example of this is burst coding, where neurons fire rapid bursts of spikes, followed by a period of inactivity. Here we show that the most studied target-detecting neuron in dragonflies, CSTMD1, responds to moving targets with a series of spike bursts. This spiking activity differs from those in other identified visual neurons in the dragonfly, indicative of different physiological mechanisms underlying CSTMD1's spike generation. Burst codes present several advantages and disadvantages compared to other coding approaches. We propose functional implications of CSTMD1's burst coding activity and show that spike bursts enhance the robustness of target-evoked responses.

Nonlinear, neuronal adaptation in insect vision models improves target discrimination within repetitively moving backgrounds.

Neurons which respond selectively to small moving targets, even against a cluttered background, have been identified in several insect species. To investigate what underlies these robust and highly selective responses, researchers have probed the neuronal circuitry in target-detecting, visual pathways. Observations in flies reveal nonlinear adaptation over time, composed of a fast onset and gradual decay. This adaptive processing is seen in both of the independent, parallel pathways encoding either luminance increments (ON channel) or decrements (OFF channel). The functional significance of this adaptive phenomenon has not been determined from physiological studies, though the asymmetrical time course suggests a role in suppressing responses to repetitive stimuli. We tested this possibility by comparing an implementation of fast adaptation against alternatives, using a model of insect 'elementary small target motion detectors'. We conducted target-detecting simulations on various natural backgrounds, that were shifted via several movement profiles (and target velocities). Using performance metrics, we confirmed that the fast adaptation observed in neuronal systems enhances target detection against a repetitively moving background. Such background movement would be encountered via natural ego-motion as the insect travels through the world. These findings show that this form of nonlinear, fast-adaptation (suitably implementable via cellular biophysics) plays a role analogous to background subtraction techniques in conventional computer vision.

Modeling Nonlinear Dendritic Processing of Facilitation in a Dragonfly Target-Tracking Neuron.

Dragonflies are highly skilled and successful aerial predators that are even capable of selectively attending to one target within a swarm. Detection and tracking of prey is likely to be driven by small target motion detector (STMD) neurons identified from several insect groups. Prior work has shown that dragonfly STMD responses are facilitated by targets moving on a continuous path, enhancing the response gain at the present and predicted future location of targets. In this study, we combined detailed morphological data with computational modeling to test whether a combination of dendritic morphology and nonlinear properties of NMDA receptors could explain these observations. We developed a hybrid computational model of neurons within the dragonfly optic lobe, which integrates numerical and morphological components. The model was able to generate potent facilitation for targets moving on continuous trajectories, including a localized spotlight of maximal sensitivity close to the last seen target location, as also measured during in vivo recordings. The model did not, however, include a mechanism capable of producing a traveling or spreading wave of facilitation. Our data support a strong role for the high dendritic density seen in the dragonfly neuron in enhancing non-linear facilitation. An alternative model based on the morphology of an unrelated type of motion processing neuron from a dipteran fly required more than three times higher synaptic gain in order to elicit similar levels of facilitation, despite having only 20% fewer synapses. Our data support a potential role for NMDA receptors in target tracking and also demonstrate the feasibility of combining biologically plausible dendritic computations with more abstract computational models for basic processing as used in earlier studies.

The visual neuroecology of anisoptera.

Dragonflies belong to the oldest known lineage of flying animals, found across the globe around streams, ponds and forests. They are insect predators, specialising in ambush attack as aquatic larvae and rapid pursuit as adults. Dragonfly adults hunt amidst swarms in conditions that confuse many predatory species, and exhibit capture rates above 90%. Underlying the performance of such a remarkable predator is a finely tuned visual system capable of tracking targets amidst distractors and background clutter. The dragonfly performs a complex repertoire of flight behaviours, from near-motionless hovering to acute turns at high speeds. Here, we review the optical, neuronal, and behavioural adaptations that underlie the dragonflies' ability to achieve such remarkable predatory success.

Visual acuity of the honey bee retina and the limits for feature detection.

Visual abilities of the honey bee have been studied for more than 100 years, recently revealing unexpectedly sophisticated cognitive skills rivalling those of vertebrates. However, the physiological limits of the honey bee eye have been largely unaddressed and only studied in an unnatural, dark state. Using a bright display and intracellular recordings, we here systematically investigated the angular sensitivity across the light adapted eye of honey bee foragers. Angular sensitivity is a measure of photoreceptor receptive field size and thus small values indicate higher visual acuity. Our recordings reveal a fronto-ventral acute zone in which angular sensitivity falls below 1.9°, some 30% smaller than previously reported. By measuring receptor noise and responses to moving dark objects, we also obtained direct measures of the smallest features detectable by the retina. In the frontal eye, single photoreceptors respond to objects as small as 0.6° × 0.6°, with >99% reliability. This indicates that honey bee foragers possess significantly better resolution than previously reported or estimated behaviourally, and commonly assumed in modelling of bee acuity.

Performance of an insect-inspired target tracker in natural conditions.

Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.

An autonomous robot inspired by insect neurophysiology pursues moving features in natural environments.

OBJECTIVE: Many computer vision and robotic applications require the implementation of robust and efficient target-tracking algorithms on a moving platform. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. Lightweight and low-powered flying insects, such as dragonflies, track prey or conspecifics within cluttered natural environments, illustrating an efficient biological solution to the target-tracking problem. APPROACH: We used our recent recordings from 'small target motion detector' neurons in the dragonfly brain to inspire the development of a closed-loop target detection and tracking algorithm. This model exploits facilitation, a slow build-up of response to targets which move along long, continuous trajectories, as seen in our electrophysiological data. To test performance in real-world conditions, we implemented this model on a robotic platform that uses active pursuit strategies based on insect behaviour. MAIN RESULTS: Our robot performs robustly in closed-loop pursuit of targets, despite a range of challenging conditions used in our experiments; low contrast targets, heavily cluttered environments and the presence of distracters. We show that the facilitation stage boosts responses to targets moving along continuous trajectories, improving contrast sensitivity and detection of small moving targets against textured backgrounds. Moreover, the temporal properties of facilitation play a useful role in handling vibration of the robotic platform. We also show that the adoption of feed-forward models which predict the sensory consequences of self-movement can significantly improve target detection during saccadic movements. SIGNIFICANCE: Our results provide insight into the neuronal mechanisms that underlie biological target detection and selection (from a moving platform), as well as highlight the effectiveness of our bio-inspired algorithm in an artificial visual system.

A predictive focus of gain modulation encodes target trajectories in insect vision.

When a human catches a ball, they estimate future target location based on the current trajectory. How animals, small and large, encode such predictive processes at the single neuron level is unknown. Here we describe small target-selective neurons in predatory dragonflies that exhibit localized enhanced sensitivity for targets displaced to new locations just ahead of the prior path, with suppression elsewhere in the surround. This focused region of gain modulation is driven by predictive mechanisms, with the direction tuning shifting selectively to match the target's prior path. It involves a large local increase in contrast gain which spreads forward after a delay (e.g. an occlusion) and can even transfer between brain hemispheres, predicting trajectories moved towards the visual midline from the other eye. The tractable nature of dragonflies for physiological experiments makes this a useful model for studying the neuronal mechanisms underlying the brain's remarkable ability to anticipate moving stimuli.