Recognition of natural objects in the archerfish.

Recognition of individual objects and their categorization is a complex computational task. Nevertheless, visual systems can perform this task in a rapid and accurate manner. Humans and other animals can efficiently recognize objects despite countless variations in their projection on the retina due to different viewing angles, distance, illumination conditions and other parameters. To gain a better understanding of the recognition process in teleosts, we explored it in archerfish, a species that hunts by shooting a jet of water at aerial targets and thus can benefit from ecologically relevant recognition of natural objects. We found that archerfish not only can categorize objects into relevant classes but also can do so for novel objects, and additionally they can recognize an individual object presented under different conditions. To understand the mechanisms underlying this capability, we developed a computational model based on object features and a machine learning classifier. The analysis of the model revealed that a small number of features was sufficient for categorization, and the fish were more sensitive to object contours than textures. We tested these predictions in additional behavioral experiments and validated them. Our findings suggest the existence of a complex visual process in the archerfish visual system that enables object recognition and categorization.

Self-motion trajectories can facilitate orientation-based figure-ground segregation.

Segregation of objects from the background is a basic and essential property of the visual system. We studied the neural detection of objects defined by orientation difference from background in barn owls (Tyto alba). We presented wide-field displays of densely packed stripes with a dominant orientation. Visual objects were created by orienting a circular patch differently from the background. In head-fixed conditions, neurons in both tecto- and thalamofugal visual pathways (optic tectum and visual Wulst) were weakly responsive to these objects in their receptive fields. However, notably, in freely viewing conditions, barn owls occasionally perform peculiar side-to-side head motions (peering) when scanning the environment. In the second part of the study we thus recorded the neural response from head-fixed owls while the visual displays replicated the peering conditions; i.e., the displays (objects and backgrounds) were shifted along trajectories that induced a retinal motion identical to sampled peering motions during viewing of a static object. These conditions induced dramatic neural responses to the objects, in the very same neurons that where unresponsive to the objects in static displays. By reverting to circular motions of the display, we show that the pattern of the neural response is mostly shaped by the orientation of the background relative to motion and not the orientation of the object. Thus our findings provide evidence that peering and/or other self-motions can facilitate orientation-based figure-ground segregation through interaction with inhibition from the surround.NEW & NOTEWORTHY Animals frequently move their sensory organs and thereby create motion cues that can enhance object segregation from background. We address a special example of such active sensing, in barn owls. When scanning the environment, barn owls occasionally perform small-amplitude side-to-side head movements called peering. We show that the visual outcome of such peering movements elicit neural detection of objects that are rotated from the dominant orientation of the background scene and which are otherwise mostly undetected. These results suggest a novel role for self-motions in sensing objects that break the regular orientation of elements in the scene.

What pops out for you pops out for fish: Four common visual features.

Visual search is the ability to detect a target of interest against a background of distracting objects. For many animals, performing this task fast and accurately is crucial for survival. Typically, visual-search performance is measured by the time it takes the observer to detect a target against a backdrop of distractors. The efficiency of a visual search depends fundamentally on the features of the target, the distractors, and the interaction between them. Substantial efforts have been devoted to investigating the influence of different visual features on visual-search performance in humans. In particular, it has been demonstrated that color, size, orientation, and motion are efficient visual features to guide attention in humans. However, little is known about which features are efficient and which are not in other vertebrates. Given earlier observations that moving targets elicit pop-out and parallel search in the archerfish during visual-search tasks, here we investigate and confirm that all four of these visual features also facilitate efficient search in the archerfish in a manner comparable to humans. In conjunction with results reported for other species, these finding suggest universality in the way visual search is carried out by animals despite very different brain anatomies and living environments.

Controlled Lighting and Illumination-Independent Target Detection for Real-Time Cost-Efficient Applications. The Case Study of Sweet Pepper Robotic Harvesting.

Current harvesting robots are limited by low detection rates due to the unstructured and dynamic nature of both the objects and the environment. State-of-the-art algorithms include color- and texture-based detection, which are highly sensitive to the illumination conditions. Deep learning algorithms promise robustness at the cost of significant computational resources and the requirement for intensive databases. In this paper we present a Flash-No-Flash (FNF) controlled illumination acquisition protocol that frees the system from most ambient illumination effects and facilitates robust target detection while using only modest computational resources and no supervised training. The approach relies on the simultaneous acquisition of two images-with/without strong artificial lighting ("Flash"/"no-Flash"). The difference between these images represents the appearance of the target scene as if only the artificial light was present, allowing a tight control over ambient light for color-based detection. A performance evaluation database was acquired in greenhouse conditions using an eye-in-hand RGB camera mounted on a robotic manipulator. The database includes 156 scenes with 468 images containing a total of 344 yellow sweet peppers. Performance of both color blob and deep-learning detection algorithms are compared on Flash-only and FNF images. The collected database is made public.

Visual search in barn owls: Task difficulty and saccadic behavior.

How do we find what we are looking for? A target can be in plain view, but it may be detected only after extensive search. During a search we make directed attentional deployments like saccades to segment the scene until we detect the target. Depending on difficulty, the search may be fast with few attentional deployments or slow with many, shorter deployments. Here we study visual search in barn owls by tracking their overt attentional deployments-that is, their head movements-with a camera. We conducted a low-contrast feature search, a high-contrast orientation conjunction search, and a low-contrast orientation conjunction search, each with set sizes varying from 16 to 64 items. The barn owls were able to learn all of these tasks and showed serial search behavior. In a subsequent step, we analyzed how search behavior of owls changes with search complexity. We compared the search mechanisms in these three serial searches with results from pop-out searches our group had reported earlier. Saccade amplitude shortened and fixation duration increased in difficult searches. Also, in conjunction search saccades were guided toward items with shared target features. These data suggest that during visual search, barn owls utilize mechanisms similar to those that humans use.

Specular motion and 3D shape estimation.

Dynamic visual information facilitates three-dimensional shape recognition. It is still unclear, however, whether the motion information generated by moving specularities across a surface is congruent to that available from optic flow produced by a matte-textured shape. Whereas the latter is directly linked to the first-order properties of the shape and its motion relative to the observer, the specular flow, the image flow generated by a specular object, is less sensitive to the object's motion and is tightly related to second-order properties of the shape. We therefore hypothesize that the perceived bumpiness (a perceptual attribute related to curvature magnitude) is more stable to changes in the type of motion in specular objects compared with their matte-textured counterparts. Results from two two-interval forced-choice experiments in which observers judged the perceived bumpiness of perturbed spherelike objects support this idea and provide an additional layer of evidence for the capacity of the visual system to exploit image information for shape inference.

Visual pop-out in barn owls: Human-like behavior in the avian brain.

Visual pop-out is a phenomenon by which the latency to detect a target in a scene is independent of the number of other elements, the distractors. Pop-out is an effective visual-search guidance that occurs typically when the target is distinct in one feature from the distractors, thus facilitating fast detection of predators or prey. However, apart from studies on primates, pop-out has been examined in few species and demonstrated thus far in rats, archer fish, and pigeons only. To fill this gap, here we study pop-out in barn owls. These birds are a unique model system for such exploration because their lack of eye movements dictates visual behavior dominated by head movements. Head saccades and interspersed fixation periods can therefore be tracked and analyzed with a head-mounted wireless microcamera--the OwlCam. Using this methodology we confronted two owls with scenes containing search arrays of one target among varying numbers (15-63) of similar looking distractors. We tested targets distinct either by orientation (Experiment 1) or luminance contrast (Experiment 2). Search time and the number of saccades until the target was fixated remained largely independent of the number of distractors in both experiments. This suggests that barn owls can exhibit pop-out during visual search, thus expanding the group of species and brain structures that can cope with this fundamental visual behavior. The utility of our automatic analysis method is further discussed for other species and scientific questions.

Distance-Dependent Multimodal Image Registration for Agriculture Tasks.

Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibrating a distance-dependent transformation matrix (DDTM) between the sensors; and representing it in a compact way by regressing the distance-dependent coefficients as distance-dependent functions. The DDTM is measured by calculating a projective transformation matrix for varying distances between the sensors and possible targets. To do so we designed a unique experimental setup including unique Artificial Control Points (ACPs) and their detection algorithms for the two sensors. We demonstrate the utility of our approach using different experiments and evaluation criteria.

Overt attention toward oriented objects in free-viewing barn owls.

Visual saliency based on orientation contrast is a perceptual product attributed to the functional organization of the mammalian brain. We examined this visual phenomenon in barn owls by mounting a wireless video microcamera on the owls' heads and confronting them with visual scenes that contained one differently oriented target among similarly oriented distracters. Without being confined by any particular task, the owls looked significantly longer, more often, and earlier at the target, thus exhibiting visual search strategies so far demonstrated in similar conditions only in primates. Given the considerable differences in phylogeny and the structure of visual pathways between owls and humans, these findings suggest that orientation saliency has computational optimality in a wide variety of ecological contexts, and thus constitutes a universal building block for efficient visual information processing in general.

Visual receptive field properties of cells in the optic tectum of the archer fish.

The archer fish is well known for its extreme visual behavior in shooting water jets at prey hanging on vegetation above water. This fish is a promising model in the study of visual system function because it can be trained to respond to artificial targets and thus to provide valuable psychophysical data. Although much behavioral data have indeed been collected over the past two decades, little is known about the functional organization of the main visual area supporting this visual behavior, namely, the fish optic tectum. In this article we focus on a fundamental aspect of this functional organization and provide a detailed analysis of receptive field properties of cells in the archer fish optic tectum. Using extracellular measurements to record activities of single cells, we first measure their retinotectal mapping. We then determine their receptive field properties such as size, selectivity for stimulus direction and orientation, tuning for spatial frequency, and tuning for temporal frequency. Finally, on the basis of all these measurements, we demonstrate that optic tectum cells can be classified into three categories: orientation-tuned cells, direction-tuned cells, and direction-agnostic cells. Our results provide an essential basis for future investigations of information processing in the archer fish visual system.

Orientation saliency without visual cortex and target selection in archer fish.

Our visual attention is attracted by salient stimuli in our environment and affected by primitive features such as orientation, color, and motion. Perceptual saliency due to orientation contrast has been extensively demonstrated in behavioral experiments with humans and other primates and is believed to be facilitated by the functional organization of the primary visual cortex. In behavioral experiments with the archer fish, a proficient hunter with remarkable visual abilities, we found an orientation saliency effect similar to that observed in human subjects. Given the enormous evolutionary distance between humans and archer fish, our findings suggest that orientation-based saliency constitutes a fundamental building block for efficient visual information processing.

Tangent bundle curve completion with locally connected parallel networks.

We propose a theory for cortical representation and computation of visually completed curves that are generated by the visual system to fill in missing visual information (e.g., due to occlusions). Recent computational theories and physiological evidence suggest that although such curves do not correspond to explicit image evidence along their length, their construction emerges from corresponding activation patterns of orientation-selective cells in the primary visual cortex. Previous theoretical work modeled these patterns as least energetic 3D curves in the mathematical continuous space R2 × S1, which abstracts the mammalian striate cortex. Here we discuss the biological plausibility of this theory and present a neural architecture that implements it with locally connected parallel networks. Part of this contribution is also a first attempt to bridge the physiological literature on curve completion with the shape problem and a shape theory. We present completion simulations of our model in natural and synthetic scenes and discuss various observations and predictions that emerge from this theory in the context of curve completion.

Predictive saccade in the absence of smooth pursuit: interception of moving targets in the archer fish.

Interception of fast-moving targets is a demanding task many animals solve. To handle it successfully, mammals employ both saccadic and smooth pursuit eye movements in order to confine the target to their area centralis. But how can non-mammalian vertebrates, which lack smooth pursuit, intercept moving targets? We studied this question by exploring eye movement strategies employed by archer fish, an animal that possesses an area centralis, lacks smooth pursuit eye movements, but can intercept moving targets by shooting jets of water at them. We tracked the gaze direction of fish during interception of moving targets and found that they employ saccadic eye movements based on prediction of target position when it is hit. The fish fixates on the target's initial position for ∼0.2 s from the onset of its motion, a time period used to predict whether a shot can be made before the projection of the target exits the area centralis. If the prediction indicates otherwise, the fish performs a saccade that overshoots the center of gaze beyond the present target projection on the retina, such that after the saccade the moving target remains inside the area centralis long enough to prepare and perform a shot. These results add to the growing body of knowledge on biological target tracking and may shed light on the mechanism underlying this behavior in other animals with no neural system for the generation of smooth pursuit eye movements.

A perceptual paradigm and psychophysical evidence for hierarchy in scene gist processing.

What is the order of processing in scene gist recognition? Following the seminal studies by Rosch (1978) and Tversky and Hemmenway (1983) it has been assumed that basic-level categorization is privileged over the superordinate level because the former maximizes both within-category similarity and between-category variance. However, recent research has begun to challenge this view (Oliva & Torralba, 2001; Joubert, Rousselet, Fize, & Fabre-Thorpe, 2007; Loschky & Larson, 2010). Here we study these directions more fundamentally by investigating the perceptual relations between scene categories in a way that allows us to identify the order of processing of scene categories across taxonomic levels. We introduce the category discrimination paradigm where we briefly present two real scene stimuli simultaneously and ask human observers whether they belong to the same basic-level category or not (i.e., same/different task). As we show, analysis of the obtained data reveals a hierarchical perceptual structure between different scene categories and a corresponding hierarchical structure at the perceptual processing level. In particular, we show a new type of evidence to suggest that the decision whether the scene is manmade or natural is made first, and only then more complicated decisions are taken (such as whether a manmade scene is indoor or outdoor) among a smaller set of viable candidate categories. We argue that this hierarchical structure improves performance and efficiency in both biological and artificial gist recognition systems.

Visual acuity in the archerfish: behavior, anatomy, and neurophysiology.

Archerfish are known for their remarkable behavior of shooting water jets at prey hanging on vegetation above water. Motivated by the fish's capacity to knock down small prey as high as two meters above water level, we studied the role of the retina in facilitating their excellent visual acuity. First, we show behaviorally that archerfish (Toxotes jaculatrix) can detect visual structures with a minimum angle of resolution in the range of 0.075°-0.15°. Then, combining eye movement measurements with a ray tracing method, we show that the image of a target on the retina coincides with the area centralis at the ventro-temporal retina. Moving down to retinal neural circuits, we then examine the ratio by which retinal ganglion cells multiplex visual information from the photoreceptors. Measuring the anatomical densities of both cell types in the area centralis, we found photoreceptor spacing to be 5.8 µm, which supports a minimum angle of resolution as low as 0.073°. Similarly, the average spacing of the ganglion cells was 5.7 µm. Based on electrophysiological measurements we found the smallest receptive fields of ganglion cells in that area to be in the range of 8-16 µm, which translates to an angular width of 0.1°-0.2°. These findings indicate that retinal ganglion cells in the area centralis stream information to the brain at a comparable resolution with which it is sampled by the photoreceptors. Thus, the archerfish can be used as an animal model for studying how visual details are streamed to the brain by retinal output.

From fish out of water to new insights on navigation mechanisms in animals.

Navigation is a critical ability for animal survival and is important for food foraging, finding shelter, seeking mates and a variety of other behaviors. Given their fundamental role and universal function in the animal kingdom, it makes sense to explore whether space representation and navigation mechanisms are dependent on the species, ecological system, brain structures, or whether they share general and universal properties. One way to explore this issue behaviorally is by domain transfer methodology, where one species is embedded in another species' environment and must cope with an otherwise familiar (in our case, navigation) task. Here we push this idea to the limit by studying the navigation ability of a fish in a terrestrial environment. For this purpose, we trained goldfish to use a Fish Operated Vehicle (FOV), a wheeled terrestrial platform that reacts to the fish's movement characteristics, location and orientation in its water tank to change the vehicle's; i.e., the water tank's, position in the arena. The fish were tasked to "drive" the FOV towards a visual target in the terrestrial environment, which was observable through the walls of the tank, and indeed were able to operate the vehicle, explore the new environment, and reach the target regardless of the starting point, all while avoiding dead-ends and correcting location inaccuracies. These results demonstrate how a fish was able to transfer its space representation and navigation skills to a wholly different terrestrial environment, thus supporting the hypothesis that the former possess a universal quality that is species-independent.

Toward Naturalistic Neuroscience of Navigation: Opportunities in Coral Reef Fish.

The ability to navigate in the world is crucial to many species. One of the most fundamental unresolved issues in understanding animal navigation is how the brain represents spatial information. Although navigation has been studied extensively in many taxa, the key efforts to determine the neural basis of navigation have focused on mammals, usually in lab experiments, where the allocated space is typically very small; e.g., up to one order of magnitude the size of the animal, is limited by artificial walls, and contains only a few objects. This type of setting is vastly different from the habitat of animals in the wild, which is open in many cases and is virtually limitless in size compared to its inhabitants. Thus, a fundamental open question in animal navigation is whether small-scale, spatially confined, and artificially crafted lab experiments indeed reveal how navigation is enacted in the real world. This question is difficult to study given the technical problems associated with in vivo electrophysiology in natural settings. Here, we argue that these difficulties can be overcome by implementing state of the art technology when studying the rivulated rabbitfish, Siganus rivulatus as the model animal. As a first step toward this goal, using acoustic tracking of the reef, we demonstrate that individual S. rivulatus have a defined home range of about 200 m in length, from which they seldom venture. They repeatedly visit the same areas and return to the same sleeping grounds, thus providing evidence for their ability to navigate in the reef environment. Using a clustering algorithm to analyze segments of daily trajectories, we found evidence of specific repeating patterns in behavior within the home range of individual fish. Thus, S. rivulatus appears to have the ability to carry out its daily routines and revisit places of interest by employing sophisticated means of navigation while exploring its surroundings. In the future, using novel technologies for wireless recording from single cells of fish brains, S. rivulatus can emerge as an ideal system to study the neural basis of navigation in natural settings and lead to "electrophysiology in the wild."

Vision, cognition, and walking stability in young adults.

Downward gazing is often observed when walking requires guidance. This gaze behavior is thought to promote walking stability through anticipatory stepping control. This study is part of an ongoing effort to investigate whether downward gazing also serves to enhance postural control, which can promote walking stability through a feedback/reactive mechanism. Since gaze behavior alone gives no indication as to what information is gathered and the functions it serves, we aimed to investigate the cognitive demands associated with downward gazing, as they are likely to differ between anticipatory and feedback use of visual input. To do so, we used a novel methodology to compromise walking stability in a manner that could not be resolved through modulation of stepping. Then, using interference methodology and neuroimaging, we tested for (1) interference related to dual tasking, and (2) changes in prefrontal activity. The novel methodology resulted in an increase in the time spent looking at the walking surface. Further, while some dual-task interference was observed, indicating that this gaze behavior is cognitively demanding, several gaze parameters pertaining to downward gazing and prefrontal activity correlated. These correlations revealed that a greater tendency to gaze onto the walking surface was associated with lower PFC activity, as is expected when sensory information is used through highly automatic, and useful, neural circuitry. These results, while not conclusive, do suggest that gazing onto the walking surface can be used for purposes other than anticipatory stepping control, bearing important motor-control and clinical implications.

Stochastic emergence of repeating cortical motifs in spontaneous membrane potential fluctuations in vivo.

It was recently discovered that subthreshold membrane potential fluctuations of cortical neurons can precisely repeat during spontaneous activity, seconds to minutes apart, both in brain slices and in anesthetized animals. These repeats, also called cortical motifs, were suggested to reflect a replay of sequential neuronal firing patterns. We searched for motifs in spontaneous activity, recorded from the rat barrel cortex and from the cat striate cortex of anesthetized animals, and found numerous repeating patterns of high similarity and repetition rates. To test their significance, various statistics were compared between physiological data and three different types of stochastic surrogate data that preserve dynamical characteristics of the recorded data. We found no evidence for the existence of deterministically generated cortical motifs. Rather, the stochastic properties of cortical motifs suggest that they appear by chance, as a result of the constraints imposed by the coarse dynamics of subthreshold ongoing activity.

Gazing down increases standing and walking postural steadiness.

When walking on an uneven surface or complex terrain, humans tend to gaze downward. This behaviour is usually interpreted as an attempt to acquire useful information to guide locomotion. Visual information, however, is not used exclusively for guiding locomotion; it is also useful for postural control. Both locomotive and postural control have been shown to be sensitive to the visual flow arising from the respective motion of the individual and the three-dimensional environment. This flow changes when a person gazes downward and may present information that is more appropriate for postural control. To investigate whether downward gazing can be used for postural control, rather than exclusively for guiding locomotion, we quantified the dynamics of standing and walking posture in healthy adults, under several visual conditions. Through these experiments we were able to demonstrate that gazing downward, just a few steps ahead, resulted in a steadier standing and walking posture. These experiments indicate that gazing downward may serve more than one purpose and provide sufficient evidence of the possible interplay between the visual information used for guiding locomotion and that used for postural control. These findings contribute to our understanding of the control mechanism/s underlying gait and posture and have possible clinical implications.