An insect-inspired model for visual binding I: learning objects and their characteristics.

Visual binding is the process of associating the responses of visual interneurons in different visual submodalities all of which are responding to the same object in the visual field. Recently identified neuropils in the insect brain termed optic glomeruli reside just downstream of the optic lobes and have an internal organization that could support visual binding. Working from anatomical similarities between optic and olfactory glomeruli, we have developed a model of visual binding based on common temporal fluctuations among signals of independent visual submodalities. Here we describe and demonstrate a neural network model capable both of refining selectivity of visual information in a given visual submodality, and of associating visual signals produced by different objects in the visual field by developing inhibitory neural synaptic weights representing the visual scene. We also show that this model is consistent with initial physiological data from optic glomeruli. Further, we discuss how this neural network model may be implemented in optic glomeruli at a neuronal level.

An insect-inspired model for visual binding II: functional analysis and visual attention.

We have developed a neural network model capable of performing visual binding inspired by neuronal circuitry in the optic glomeruli of flies: a brain area that lies just downstream of the optic lobes where early visual processing is performed. This visual binding model is able to detect objects in dynamic image sequences and bind together their respective characteristic visual features-such as color, motion, and orientation-by taking advantage of their common temporal fluctuations. Visual binding is represented in the form of an inhibitory weight matrix which learns over time which features originate from a given visual object. In the present work, we show that information represented implicitly in this weight matrix can be used to explicitly count the number of objects present in the visual image, to enumerate their specific visual characteristics, and even to create an enhanced image in which one particular object is emphasized over others, thus implementing a simple form of visual attention. Further, we present a detailed analysis which reveals the function and theoretical limitations of the visual binding network and in this context describe a novel network learning rule which is optimized for visual binding.

A visual motion detecting module for dragonfly-controlled robots.

When imitating biological sensors, we have not completely understood the early processing of the input to reproduce artificially. Building hybrid systems with both artificial and real biological components is a promising solution. For example, when a dragonfly is used as a living sensor, the early processing of visual information is performed fully in the brain of the dragonfly. The only significant remaining tasks are recording and processing neural signals in software and/or hardware. Based on existing works which focused on recording neural signals, this paper proposes a software application of neural information processing to design a visual processing module for dragonfly hybrid bio-robots. After a neural signal is recorded in real-time, the action potentials can be detected and matched with predefined templates to detect when and which descending neurons fire. The output of the proposed system will be used to control other parts of the robot platform.

Tracking improves performance of biological collision avoidance models.

Collision avoidance models derived from the study of insect brains do not perform universally well in practical collision scenarios, although the insects themselves may perform well in similar situations. In this article, we present a detailed simulation analysis of two well-known collision avoidance models and illustrate their limitations. In doing so, we present a novel continuous-time implementation of a neuronally based collision avoidance model. We then show that visual tracking can improve performance of these models by allowing an relative computation of the distance between the obstacle and the observer. We compare the results of simulations of the two models with and without tracking to show how tracking improves the ability of the model to detect an imminent collision. We present an implementation of one of these models processing imagery from a camera to show how it performs in real-world scenarios. These results suggest that insects may track looming objects with their gaze.

A neuronally based model of contrast gain adaptation in fly motion vision.

Motion-sensitive neurons in the visual systems of many species, including humans, exhibit a depression of motion responses immediately after being exposed to rapidly moving images. This motion adaptation has been extensively studied in flies, but a neuronal mechanism that explains the most prominent component of adaptation, which occurs regardless of the direction of motion of the visual stimulus, has yet to be proposed. We identify a neuronal mechanism, namely frequency-dependent synaptic depression, which explains a number of the features of adaptation in mammalian motion-sensitive neurons and use it to model fly motion adaptation. While synaptic depression has been studied mainly in spiking cells, we use the same principles to develop a simple model for depression in a graded synapse. By incorporating this synaptic model into a neuronally based model for elementary motion detection, along with the implementation of a center-surround spatial band-pass filtering stage that mimics the interactions among a subset of visual neurons, we show that we can predict with remarkable success most of the qualitative features of adaptation observed in electrophysiological experiments. Our results support the idea that diverse species share common computational principles for processing visual motion and suggest that such principles could be neuronally implemented in very similar ways.

Non-directional motion detectors can be used to mimic optic flow dependent behaviors.

Insect navigational behaviors including obstacle avoidance, grazing landings, and visual odometry are dependent on the ability to estimate flight speed based only on visual cues. In honeybees, this visual estimate of speed is largely independent of both the direction of motion and the spatial frequency content of the image. Electrophysiological recordings from the motion-sensitive cells believed to underlie these behaviors have long supported spatio-temporally tuned correlation-type models of visual motion detection whose speed tuning changes as the spatial frequency of a stimulus is varied. The result is an apparent conflict between behavioral experiments and the electrophysiological and modeling data. In this article, we demonstrate that conventional correlation-type models are sufficient to reproduce some of the speed-dependent behaviors observed in honeybees when square wave gratings are used, contrary to the theoretical predictions. However, these models fail to match the behavioral observations for sinusoidal stimuli. Instead, we show that non-directional motion detectors, which underlie the correlation-based computation of directional motion, can be used to mimic these same behaviors even when narrowband gratings are used. The existence of such non-directional motion detectors is supported both anatomically and electrophysiologically, and they have been hypothesized to be critical in the Dipteran elementary motion detector (EMD) circuit.

The spatial frequency tuning of optic-flow-dependent behaviors in the bumblebee Bombus impatiens.

Insects use visual estimates of flight speed for a variety of behaviors, including visual navigation, odometry, grazing landings and flight speed control, but the neuronal mechanisms underlying speed detection remain unknown. Although many models and theories have been proposed for how the brain extracts the angular speed of the retinal image, termed optic flow, we lack the detailed electrophysiological and behavioral data necessary to conclusively support any one model. One key property by which different models of motion detection can be differentiated is their spatiotemporal frequency tuning. Numerous studies have suggested that optic-flow-dependent behaviors are largely insensitive to the spatial frequency of a visual stimulus, but they have sampled only a narrow range of spatial frequencies, have not always used narrowband stimuli, and have yielded slightly different results between studies based on the behaviors being investigated. In this study, we present a detailed analysis of the spatial frequency dependence of the centering response in the bumblebee Bombus impatiens using sinusoidal and square wave patterns.

A navigation aid for the blind using tactile-visual sensory substitution.

The objective of this study is to improve the quality of life for the visually impaired by restoring their ability to self-navigate. In this paper we describe a compact, wearable device that converts visual information into a tactile signal. This device, constructed entirely from commercially available parts, enables the user to perceive distant objects via a different sensory modality. Preliminary data suggest that this device is useful for object avoidance in simple environments.

An elaborated model of fly small-target tracking.

Flies have the capability to visually track small moving targets, even across cluttered backgrounds. Previous computational models, based on figure detection (FD) cells identified in the fly, have suggested how this may be accomplished at a neuronal level based on information about relative motion between the target and the background. We experimented with the use of this "small-field system model" for the tracking of small moving targets by a simulated fly in a cluttered environment and discovered some functional limitations. As a result of these experiments, we propose elaborations of the original small-field system model to support stronger effects of background motion on small-field responses, proper accounting for more complex optical flow fields, and more direct guidance toward the target. We show that the elaborated model achieves much better tracking performance than the original model in complex visual environments and discuss the biological implications of our elaborations. The elaborated model may help to explain recent electrophysiological data on FD cells that seem to contradict the original model.

The computational basis of an identified neuronal circuit for elementary motion detection in dipterous insects.

Based on comparative anatomical studies and electrophysiological experiments, we have identified a conserved subset of neurons in the lamina, medulla, and lobula of dipterous insects that are involved in retinotopic visual motion direction selectivity. Working from the photoreceptors inward, this neuronal subset includes lamina amacrine (alpha) cells, lamina monopolar (L2) cells, the basket T-cell (T1 or beta), the transmedullary cell Tm1, and the T5 bushy T-cell. Two GABA-immunoreactive neurons, the transmedullary cell Tm9 and a local interneuron at the level of T5 dendrites, are also implicated in the motion computation. We suggest that these neurons comprise the small-field elementary motion detector circuits the outputs of which are integrated by wide-field lobula plate tangential cells. We show that a computational model based on the available data about these neurons is consistent with existing models of biological elementary motion detection, and present a comparable version of the Hassenstein-Reichardt (HR) correlation model. Further, by using the model to synthesize a generic tangential cell, we show that it can account for the responses of lobula plate tangential cells to a wide range of transient stimuli, including responses which cannot be predicted using the HR model. This computational model of elementary motion detection is the first which derives specifically from the functional organization of a subset of retinotopic neurons supplying the lobula plate. A key prediction of this model is that elementary motion detector circuits respond quite differently to small-field transient stimulation than do spatially integrated motion processing neurons as observed in the lobula plate. In addition, this model suggests that the retinotopic motion information provided to wide-field motion-sensitive cells in the lobula is derived from a less refined stage of processing than motion inputs to the lobula plate.

Nondirectional motion may underlie insect behavioral dependence on image speed.

Behavioral experiments suggest that insects make use of the apparent image speed on their compound eyes to navigate through obstacles, control flight speed, land smoothly, and measure the distance they have flown. However, the vast majority of electrophysiological recordings from motion-sensitive insect neurons show responses which are tuned in spatial and temporal frequency and are thus unable to unambiguously represent image speed. We suggest that this contradiction may be resolved at an early stage of visual motion processing using nondirectional motion sensors that respond proportionally to image speed until their peak response. We describe and characterize a computational model of these sensors and propose a model by which a spatial collation of such sensors could be used to generate speed-dependent behavior.