Spiking Elementary Motion Detector in Neuromorphic Systems.

Apparent motion of the surroundings on an agent's retina can be used to navigate through cluttered environments, avoid collisions with obstacles, or track targets of interest. The pattern of apparent motion of objects, (i.e., the optic flow), contains spatial information about the surrounding environment. For a small, fast-moving agent, as used in search and rescue missions, it is crucial to estimate the distance to close-by objects to avoid collisions quickly. This estimation cannot be done by conventional methods, such as frame-based optic flow estimation, given the size, power, and latency constraints of the necessary hardware. A practical alternative makes use of event-based vision sensors. Contrary to the frame-based approach, they produce so-called events only when there are changes in the visual scene. We propose a novel asynchronous circuit, the spiking elementary motion detector (sEMD), composed of a single silicon neuron and synapse, to detect elementary motion from an event-based vision sensor. The sEMD encodes the time an object's image needs to travel across the retina into a burst of spikes. The number of spikes within the burst is proportional to the speed of events across the retina. A fast but imprecise estimate of the time-to-travel can already be obtained from the first two spikes of a burst and refined by subsequent interspike intervals. The latter encoding scheme is possible due to an adaptive nonlinear synaptic efficacy scaling. We show that the sEMD can be used to compute a collision avoidance direction in the context of robotic navigation in a cluttered outdoor environment and compared the collision avoidance direction to a frame-based algorithm. The proposed computational principle constitutes a generic spiking temporal correlation detector that can be applied to other sensory modalities (e.g., sound localization), and it provides a novel perspective to gating information in spiking neural networks.

Behavioural state affects motion-sensitive neurones in the fly visual system.

The strength of stimulus-induced responses at the neuronal and the behavioural level often depends on the internal state of an animal. Within pathways processing sensory information and eventually controlling behavioural responses, such gain changes can originate at several sites. Using motion-sensitive lobula plate tangential cells (LPTCs) of blowflies, we address whether and in which way information processing changes for two different states of motor activity. We distinguish between the two states on the basis of haltere movements. Halteres are the evolutionarily transformed hindwings of flies. They oscillate when the animals walk or fly. LPTCs mediate, amongst other behaviours, head optomotor responses. These are either of large or small amplitude depending on the state of motor activity. Here we find that LPTC responses also depend on the motor activity of flies. In particular, LPTC responses are enhanced when halteres oscillate. Nevertheless, the response changes of LPTCs do not account for the corresponding large gain changes of head movements. Moreover, haltere activity itself does not change the activity of LPTCs. Instead, we propose that a central signal associated with motor activity changes the gain of head optomotor responses and the response properties of LPTCs.

Variability in spike trains during constant and dynamic stimulation.

In a recent study, it was concluded that natural time-varying stimuli are represented more reliably in the brain than constant stimuli are. The results presented here disagree with this conclusion, although they were obtained from the same identified neuron (H1) in the fly's visual system. For large parts of the neuron's activity range, the variability of the responses was very similar for constant and time-varying stimuli and was considerably smaller than that in many visual interneurons of vertebrates.

Temporal precision of the encoding of motion information by visual interneurons.

BACKGROUND: There is much controversy about the timescale on which neurons process and transmit information. On the one hand, a vast amount of information can be processed by the nervous system if the precise timing of individual spikes on a millisecond timescale is important. On the other hand, neuronal responses to identical stimuli often vary considerably and stochastic response fluctuations can exceed the mean response amplitude. Here, we examined the timescale on which neural responses could be locked to visual motion stimuli. RESULTS: Spikes of motion-sensitive neurons in the visual system of the blowfly are time-locked to visual motion with a precision in the range of several tens of milliseconds. Nevertheless, different motion-sensitive neurons with largely overlapping receptive fields generate a large proportion of spikes almost synchronously. This precision is brought about by stochastic rather than by motion-induced membrane-potential fluctuations elicited by the common peripheral input. The stochastic membrane-potential fluctuations contain more power at frequencies above 30-40 Hz than the motion-induced potential changes. A model of spike generation indicates that such fast membrane-potential changes are a major determinant of the precise timing of spikes. CONCLUSIONS: The timing of spikes in neurons of the motion pathway of the blowfly is controlled on a millisecond timescale by fast membrane-potential fluctuations. Despite this precision, spikes do not lock to motion stimuli on this timescale because visual motion does not induce sufficiently rapid changes in the membrane potential.

Function and coding in the blowfly H1 neuron during naturalistic optic flow.

Naturalistic stimuli, reconstructed from measured eye movements of flying blowflies, were replayed on a panoramic stimulus device. The directional movement-sensitive H1 neuron was recorded from blowflies watching these stimuli. The response of the H1 neuron is dominated by the response to fast saccadic turns into one direction. The response between saccades is mostly inhibited by the front-to-back optic flow caused by the forward translation during flight. To unravel the functional significance of the H1 neuron, we replayed, in addition to the original behaviorally generated stimulus, two targeted stimulus modifications: (1) a stimulus in which flow resulting from translation was removed (this stimulus produced strong intersaccadic responses); and (2) a stimulus in which the saccades were removed by assuming that the head follows the smooth flight trajectory (this stimulus produced alternating zero or nearly saturating spike rates). The responses to the two modified stimuli are strongly different from the response to the original stimulus, showing the importance of translation and saccades for the H1 response to natural optic flow. The response to the original stimulus thus suggests a double function for the H1 neuron, assisting two major classes of movement-sensitive output neurons targeted by H1. First, its strong response to saccades may function as a saccadic suppressor (via one of its target neurons) for cells involved in figure-ground discrimination. Second, its intersaccadic response may increase the signal-to-noise ratio (SNR) of wide-field neurons involved in detecting translational optic flow between saccades, in particular when flying speeds are low or when object distances are large.

On the computations analyzing natural optic flow: quantitative model analysis of the blowfly motion vision pathway.

For many animals, including humans, the optic flow generated on the eyes during locomotion is an important source of information about self-motion and the structure of the environment. The blowfly has been used frequently as a model system for experimental analysis of optic flow processing at the microcircuit level. Here, we describe a model of the computational mechanisms implemented by these circuits in the blowfly motion vision pathway. Although this model was originally proposed based on simple experimenter-designed stimuli, we show that it is also capable to quantitatively predict the responses to the complex dynamic stimuli a blowfly encounters in free flight. In particular, the model visual system exploits the active saccadic gaze and flight strategy of blowflies in a similar way, as does its neuronal counterpart. The model circuit extracts information about translation velocity in the intersaccadic intervals and thus, indirectly, about the three-dimensional layout of the environment. By stepwise dissection of the model circuit, we determine which of its components are essential for these remarkable features. When accounting for the responses to complex natural stimuli, the model is much more robust against parameter changes than when explaining the neuronal responses to simple experimenter-defined stimuli. In contrast to conclusions drawn from experiments with simple stimuli, optimization of the parameter set for different segments of natural optic flow stimuli do not indicate pronounced adaptational changes of these parameters during long-lasting stimulation.

Dendritic processing of synaptic information by sensory interneurons.

One of the most distinguishing features of nerve cells is the vast morphological diversity of their input regions, that is, their dendrites. These range from bulbous structures, with only small protrusions, to large tree-like arborizations. The diversity of nerve cells is further augmented by a continuously increasing number of types of voltage-dependent conductances in dendrites that might alter the postsynaptic signals in a pronounced way. Moreover, intracellular factors such as Ca2+ link electrical activity with biochemical processes, and can induce short and long-term changes in responsiveness. This complexity of neurons in general, and the uniqueness of each cell type, sharply contrasts with the comparatively simple and uniform design principle of the integrate-and-fire units of so-called neuronal net models. This raises the question of which particular structural and physiological details of nerve cells really matter for the performance of neuronal circuits. An answer to this basic problem of computational neurobiology might be given only if the task of the neurons and circuits is known. This review illustrates how the problem can be approached particularly well in sensory interneurons. The functional significance of sensory interneurons can often be assessed more easily than that of central nerve cells because of their vicinity to the sensory surface.

Principles of visual motion detection.

Motion information is required for the solution of many complex tasks of the visual system such as depth perception by motion parallax and figure/ground discrimination by relative motion. However, motion information is not explicitly encoded at the level of the retinal input. Instead, it has to be computed from the time-dependent brightness patterns of the retinal image as sensed by the two-dimensional array of photoreceptors. Different models have been proposed which describe the neural computations underlying motion detection in various ways. To what extent do biological motion detectors approximate any of these models? As will be argued here, there is increasing evidence from the different disciplines studying biological motion vision, that, throughout the animal kingdom ranging from invertebrates to vertebrates including man, the mechanisms underlying motion detection can be attributed to only a few, essentially equivalent computational principles. Motion detection may, therefore, be one of the first examples in computational neurosciences where common principles can be found not only at the cellular level (e.g., dendritic integration, spike propagation, synaptic transmission) but also at the level of computations performed by small neural networks.

Encoding of motion in real time by the fly visual system.

Direction-selective cells in the fly visual system that have large receptive fields play a decisive role in encoding the time-dependent optic flow the animal encounters during locomotion. Recent experiments on the computations performed by these cells have highlighted the significance of dendritic integration and have addressed the role of spikes versus graded membrane potential changes in encoding optic flow information. It is becoming increasingly clear that the way optic flow is encoded in real time is constrained both by the computational needs of the animal in visually guided behaviour as well as by the specific properties of the underlying neuronal hardware.

Neural processing of naturalistic optic flow.

Stimuli traditionally used for analyzing visual information processing are much simpler than what an animal sees in normal life. When characterized with traditional stimuli, neuronal responses were found to depend on various parameters such as contrast, texture, or velocity of motion, and thus were highly ambiguous. In behavioral situations, all of these parameters change simultaneously and differently in different parts of the visual field. Thus it is hardly possible to predict from traditional analyses what information is encoded by neurons in behavioral situations. Therefore, we characterized an identified neuron in the optomotor system of the blowfly with image sequences as they were seen by animals walking in a structured environment. We conclude that during walking, the response of the neuron reflects the animal's turning direction nearly independently of the texture and spatial layout of the environment. Our findings stress the significance of analyzing the performance of neuronal circuits under their natural operating conditions.

Transfer of visual motion information via graded synapses operates linearly in the natural activity range.

Synaptic transmission between a graded potential neuron and a spiking neuron was investigated in vivo using sensory stimulation instead of artificial excitation of the presynaptic neuron. During visual motion stimulation, individual presynaptic and postsynaptic neurons in the brain of the fly were electrophysiologically recorded together with concentration changes of presynaptic calcium (Delta[Ca(2+)](pre)). Preferred-direction motion leads to depolarization of the presynaptic neuron. It also produces pronounced increases in [Ca(2+)](pre) and the postsynaptic spike rate. Motion in the opposite direction was associated with hyperpolarization of the presynaptic cell but only a weak reduction in [Ca(2+)](pre) and the postsynaptic spike rate. Apart from this rectification, the relationships between presynaptic depolarizations, Delta[Ca(2+)](pre), and postsynaptic spike rates are, on average, linear over the entire range of activity levels that can be elicited by sensory stimulation. Thus, the inevitably limited range in which the gain of overall synaptic signal transfer is constant appears to be adjusted to sensory input strengths.

Reliability of a fly motion-sensitive neuron depends on stimulus parameters.

The variability of responses of sensory neurons constrains how reliably animals can respond to stimuli in the outside world. We show for a motion-sensitive visual interneuron of the fly that the variability of spike trains depends on the properties of the motion stimulus, although differently for different stimulus parameters. (1) The spike count variances of responses to constant and to dynamic stimuli lie in the same range. (2) With increasing stimulus size, the variance may slightly decrease. (3) Increasing pattern contrast reduces the variance considerably. For all stimulus conditions, the spike count variance is much smaller than the mean spike count and does not depend much on the mean activity apart from very low activities. Using a model of spike generation, we analyzed how the spike count variance depends on the membrane potential noise and the deterministic membrane potential fluctuations at the spike initiation zone of the neuron. In a physiologically plausible range, the variance is affected only weakly by changes in the dynamics or the amplitude of the deterministic membrane potential fluctuations. In contrast, the amplitude and dynamics of the membrane potential noise strongly influence the spike count variance. The membrane potential noise underlying the variability of the spike responses in the motion-sensitive neuron is concluded to be affected considerably by the contrast of the stimulus but by neither its dynamics nor its size.

Synapse distribution on VCH, an inhibitory, motion-sensitive interneuron in the fly visual system.

In this study, the distribution of synapses in the ventral centrifugal horizontal (VCH)-a nonspiking, inhibitory, motion-sensitive interneuron in the third visual ganglion (lobula plate) of the blowfly Calliphora erythrocephala-was examined by electron microscopy and electrophysiology. The frequency histograms of excitatory and inhibitory postsynaptic potential amplitudes recorded from the VCH during contralateral stimulus presentation suggest the existence of three neurons, two excitatory and one inhibitory, mediating contralateral input. To localize input and output regions of the VCH, we investigated the ultrastructure of its two arborisation areas after intracellular iontophoretic injection of horseradish peroxidase. The VCH has input synapses in its arborisation in the protocerebrum and in its arborisation in the lobula plate. Output synapses were found exclusively in the lobula plate. Thus, the large dendritic arbor of the VCH in the lobula plate serves simultaneously as an input and an output region. There, we found input and output synapses in close vicinity (0.5-1.5 microm) to each other. Taking into account that the VCH receives retinotopicly arranged input from the ipsilateral eye in the lobula plate, the close location of input and output synapses in the VCH suggests that the spatial organization of its retinotopic synaptic input is more or less conserved in its inhibitory output pattern. The VCH has been proposed to inhibit the figure detection 1 (FD1), another neuron of the lobula plate, that responds preferentially to small moving objects. These results suggest that the FD1 may receive inhibitory inputs from the VCH in the lobula plate, where the dendritic arbors of both neurons overlap.

Synaptic transfer of dynamic motion information between identified neurons in the visual system of the blowfly.

Synaptic transmission is usually studied in vitro with electrical stimulation replacing the natural input of the system. In contrast, we analyzed in vivo transfer of visual motion information from graded-potential presynaptic to spiking postsynaptic neurons in the fly. Motion in the null direction leads to hyperpolarization of the presynaptic neuron but does not much influence the postsynaptic cell, because its firing rate is already low during rest, giving only little scope for further reductions. In contrast, preferred-direction motion leads to presynaptic depolarizations and increases the postsynaptic spike rate. Signal transfer to the postsynaptic cell is linear and reliable for presynaptic graded membrane potential fluctuations of up to approximately 10 Hz. This frequency range covers the dynamic range of velocities that is encoded with a high gain by visual motion-sensitive neurons. Hence, information about preferred-direction motion is transmitted largely undistorted ensuring a consistent dependency of neuronal signals on stimulus parameters, such as motion velocity. Postsynaptic spikes are often elicited by rapid presynaptic spike-like depolarizations which superimpose the graded membrane potential. Although the timing of most of these spike-like depolarizations is set by noise and not by the motion stimulus, it is preserved at the synapse with millisecond precision.

Processing of synaptic information depends on the structure of the dendritic tree.

The consequences of dendritic geometry for the processing of synaptic information was analysed in two types of motion-sensitive neurones in the visual system of the fly. These neurones differ conspicuously in the morphology of their dendrites but receive their input from the same type of local motion-sensitive elements. Intracellular recording and activating selected regions of the dendrite by visual motion showed that, in accordance with cable theory of nerve cells, the way in which postsynaptic signals interact is essentially determined by the structure of the dendritic tree.

A preparation of the blowfly (Calliphora erythrocephala) brain for in vitro electrophysiological and pharmacological studies.

We describe a method for the preparation and maintenance of the blowfly (Calliphora erythrocephala) brain in a recording chamber under in vitro conditions in a semi-slice configuration. Large identification neurones in the posterior part of the 3rd optic lobe (lobula plate) can be penetrated easily with microelectrodes. The so-called vertical system (VS) cells which respond to vertical image motion in vivo could be encountered best because their axons are escorted individually by specific tracheae. Fluorescent stained cells show their natural shape as being in vivo. Electrophysiological properties of the cells investigated so far, i.e., resting potential (about -40 mV) and firing properties (single rebound spikes), are comparable to recordings in intact flies. Initial pharmacological experiments on VS cells in this preparation reveal that iontophoretical application of acetylcholine and carbamylcholine results in depolarization. VS cells also respond to bath-applied nicotine (1 microM) with a slow depolarization of their membrane potential in normal fly saline as well as in a Ca(2+)-free saline, suggesting direct cholinergic input via nicotinic receptors. The suitability of the preparation for a wide range of electrophysiological and pharmacological studies is discussed.

Temperature-dependence of neuronal performance in the motion pathway of the blowfly calliphora erythrocephala

Raising the head temperature within a behaviourally relevant range has strong effects on the performance of an identified neuron, the H1 neuron, in the visual motion pathway of blowflies. The effect is seen as an increase in the mean amplitude of the responses to motion under both transient and steady-state conditions, a considerable decrease in the response latency and an improvement in the reliability of the responses to motion. These temperature-dependent effects are independent of whether the animal is exposed to transient temperature changes or is maintained continuously at the same temperature for its entire life. The changes in the neuronal response properties with temperature may be of immediate functional significance for the animal under its normal operating conditions. In particular, the decrease in latency and the improvement in the reliability with increasing temperature may be relevant for the fly when executing its extremely virtuosic flight manoeuvres.

The role of GABA in detecting visual motion.

The basic computations underlying the extraction of motion from the visual environment have been characterized in great detail. A non-linear interaction, such as a multiplication, between neighbouring visual elements was shown to be the core of biological motion detectors in different species ranging from insects to man. GABA (gamma-aminobutyric acid)-ergic inhibitory synapses suppressing the responses to motion in one direction but not in the other are widely accepted to be the cellular basis for this non-linear interaction. Based on model predictions we can show in combined pharmacological and electrophysiological experiments that in the fly motion detection system GABAergic synapses do not play this role but rather are involved in another important step of motion computation. This makes a reconsideration of the role of inhibition in other motion detection systems necessary.

Dendritic integration of motion information in visual interneurons of the blowfly.

Dendritic integration plays a key role in the way information is processed by nerve cells. The large motion-sensitive interneurons of the fly appear to be most appropriate for an investigation of this process. These cells are known to receive input from numerous local motion-sensitive elements and to control visually-guided optomotor responses (e.g., Trends Neurosci., 11 (1988) 351-358; Stavenga and Hardie, Facets of Vision, Springer, 1989). The retinotopic input organization of these cells allows for in vivo stimulation of selected parts of their dendritic tree with their natural excitatory and inhibitory synaptic input signals. By displaying motion in either the cells' preferred or null direction in different regions of the receptive field we found: (i) Responses to combinations of excitatory and inhibitory motion stimuli can be described as the sum of the two response components. (ii) Responses to combination of excitatory stimuli show saturation effects. The deviation from linear superposition depends on the distance and relative position of the activated synaptic sites on the dendrite and makes the responses almost insensitive to the number of activated input channels. (iii) The saturation level depends on different stimulus parameters, e.g. the velocity of the moving pattern. The cell still encodes velocity under conditions of spatial saturation. The results can be understood on the basis of passive dendritic integration of the signals of retinotopically organized local motion-detecting elements with opposite polarity.

Photo-ablation of single neurons in the fly visual system reveals neural circuit for the detection of small moving objects.

Many animals use relative motion to segregate objects from their background. Nerve cells tuned to this visual cue have been found in various animal groups, such as insects, amphibians, birds and mammals. Well examined examples are the figure detection (FD) cells in the visual system of the blowfly. The mechanism that tunes a particular FD-cell, the FD1-cell, to small-field motion is analyzed by injecting individual visual interneurons with a fluorescent dye and ablating them by illumination with a laser beam. In this way, it is shown that the FD1-cell acquires its specific spatial tuning by inhibitory input from an identified GABAergic cell, the ventral centrifugal horizontal (VCH)-cell which is most sensitive to coherent large-field motion in front of both eyes. For the first time, the detection of small objects by evaluation of their motion parallax, thus, can be attributed to synaptic interactions between identified neurons.