Toward Naturalistic Neuroscience of Navigation: Opportunities in Coral Reef Fish.

The ability to navigate in the world is crucial to many species. One of the most fundamental unresolved issues in understanding animal navigation is how the brain represents spatial information. Although navigation has been studied extensively in many taxa, the key efforts to determine the neural basis of navigation have focused on mammals, usually in lab experiments, where the allocated space is typically very small; e.g., up to one order of magnitude the size of the animal, is limited by artificial walls, and contains only a few objects. This type of setting is vastly different from the habitat of animals in the wild, which is open in many cases and is virtually limitless in size compared to its inhabitants. Thus, a fundamental open question in animal navigation is whether small-scale, spatially confined, and artificially crafted lab experiments indeed reveal how navigation is enacted in the real world. This question is difficult to study given the technical problems associated with in vivo electrophysiology in natural settings. Here, we argue that these difficulties can be overcome by implementing state of the art technology when studying the rivulated rabbitfish, Siganus rivulatus as the model animal. As a first step toward this goal, using acoustic tracking of the reef, we demonstrate that individual S. rivulatus have a defined home range of about 200 m in length, from which they seldom venture. They repeatedly visit the same areas and return to the same sleeping grounds, thus providing evidence for their ability to navigate in the reef environment. Using a clustering algorithm to analyze segments of daily trajectories, we found evidence of specific repeating patterns in behavior within the home range of individual fish. Thus, S. rivulatus appears to have the ability to carry out its daily routines and revisit places of interest by employing sophisticated means of navigation while exploring its surroundings. In the future, using novel technologies for wireless recording from single cells of fish brains, S. rivulatus can emerge as an ideal system to study the neural basis of navigation in natural settings and lead to "electrophysiology in the wild."

Recognition of natural objects in the archerfish.

Recognition of individual objects and their categorization is a complex computational task. Nevertheless, visual systems can perform this task in a rapid and accurate manner. Humans and other animals can efficiently recognize objects despite countless variations in their projection on the retina due to different viewing angles, distance, illumination conditions and other parameters. To gain a better understanding of the recognition process in teleosts, we explored it in archerfish, a species that hunts by shooting a jet of water at aerial targets and thus can benefit from ecologically relevant recognition of natural objects. We found that archerfish not only can categorize objects into relevant classes but also can do so for novel objects, and additionally they can recognize an individual object presented under different conditions. To understand the mechanisms underlying this capability, we developed a computational model based on object features and a machine learning classifier. The analysis of the model revealed that a small number of features was sufficient for categorization, and the fish were more sensitive to object contours than textures. We tested these predictions in additional behavioral experiments and validated them. Our findings suggest the existence of a complex visual process in the archerfish visual system that enables object recognition and categorization.

Vision, cognition, and walking stability in young adults.

Downward gazing is often observed when walking requires guidance. This gaze behavior is thought to promote walking stability through anticipatory stepping control. This study is part of an ongoing effort to investigate whether downward gazing also serves to enhance postural control, which can promote walking stability through a feedback/reactive mechanism. Since gaze behavior alone gives no indication as to what information is gathered and the functions it serves, we aimed to investigate the cognitive demands associated with downward gazing, as they are likely to differ between anticipatory and feedback use of visual input. To do so, we used a novel methodology to compromise walking stability in a manner that could not be resolved through modulation of stepping. Then, using interference methodology and neuroimaging, we tested for (1) interference related to dual tasking, and (2) changes in prefrontal activity. The novel methodology resulted in an increase in the time spent looking at the walking surface. Further, while some dual-task interference was observed, indicating that this gaze behavior is cognitively demanding, several gaze parameters pertaining to downward gazing and prefrontal activity correlated. These correlations revealed that a greater tendency to gaze onto the walking surface was associated with lower PFC activity, as is expected when sensory information is used through highly automatic, and useful, neural circuitry. These results, while not conclusive, do suggest that gazing onto the walking surface can be used for purposes other than anticipatory stepping control, bearing important motor-control and clinical implications.

From fish out of water to new insights on navigation mechanisms in animals.

Navigation is a critical ability for animal survival and is important for food foraging, finding shelter, seeking mates and a variety of other behaviors. Given their fundamental role and universal function in the animal kingdom, it makes sense to explore whether space representation and navigation mechanisms are dependent on the species, ecological system, brain structures, or whether they share general and universal properties. One way to explore this issue behaviorally is by domain transfer methodology, where one species is embedded in another species' environment and must cope with an otherwise familiar (in our case, navigation) task. Here we push this idea to the limit by studying the navigation ability of a fish in a terrestrial environment. For this purpose, we trained goldfish to use a Fish Operated Vehicle (FOV), a wheeled terrestrial platform that reacts to the fish's movement characteristics, location and orientation in its water tank to change the vehicle's; i.e., the water tank's, position in the arena. The fish were tasked to "drive" the FOV towards a visual target in the terrestrial environment, which was observable through the walls of the tank, and indeed were able to operate the vehicle, explore the new environment, and reach the target regardless of the starting point, all while avoiding dead-ends and correcting location inaccuracies. These results demonstrate how a fish was able to transfer its space representation and navigation skills to a wholly different terrestrial environment, thus supporting the hypothesis that the former possess a universal quality that is species-independent.

Gazing down increases standing and walking postural steadiness.

When walking on an uneven surface or complex terrain, humans tend to gaze downward. This behaviour is usually interpreted as an attempt to acquire useful information to guide locomotion. Visual information, however, is not used exclusively for guiding locomotion; it is also useful for postural control. Both locomotive and postural control have been shown to be sensitive to the visual flow arising from the respective motion of the individual and the three-dimensional environment. This flow changes when a person gazes downward and may present information that is more appropriate for postural control. To investigate whether downward gazing can be used for postural control, rather than exclusively for guiding locomotion, we quantified the dynamics of standing and walking posture in healthy adults, under several visual conditions. Through these experiments we were able to demonstrate that gazing downward, just a few steps ahead, resulted in a steadier standing and walking posture. These experiments indicate that gazing downward may serve more than one purpose and provide sufficient evidence of the possible interplay between the visual information used for guiding locomotion and that used for postural control. These findings contribute to our understanding of the control mechanism/s underlying gait and posture and have possible clinical implications.

Representation of edges, head direction, and swimming kinematics in the brain of freely-navigating fish.

Like most animals, the survival of fish depends on navigation in space. This capacity has been documented in behavioral studies that have revealed navigation strategies. However, little is known about how freely swimming fish represent space and locomotion in the brain to enable successful navigation. Using a wireless neural recording system, we measured the activity of single neurons in the goldfish lateral pallium, a brain region known to be involved in spatial memory and navigation, while the fish swam freely in a two-dimensional water tank. We found that cells in the lateral pallium of the goldfish encode the edges of the environment, the fish head direction, the fish swimming speed, and the fish swimming velocity-vector. This study sheds light on how information related to navigation is represented in the brain of fish and addresses the fundamental question of the neural basis of navigation in this group of vertebrates.

Feature integration theory in non-humans: Spotlight on the archerfish.

The ability to visually search, quickly and accurately, for designated items in cluttered environments is crucial for many species to ensure survival. Feature integration theory, one of the most influential theories of attention, suggests that certain visual features that facilitate this search are extracted pre-attentively in a parallel fashion across the visual field during early visual processing. Hence, if some objects of interest possess such a feature uniquely, it will pop out from the background during the integration stage and draw visual attention immediately and effortlessly. For years, visual search research has explored these ideas by investigating the conditions (and visual features) that characterize efficient versus inefficient visual searches. The bulk of research has focused on human vision, though ecologically there are many reasons to believe that feature integration theory is applicable to other species as well. Here we review the main findings regarding the relevance of feature integration theory to non-human species and expand it to new research on one particular animal model - the archerfish. Specifically, we study both archerfish and humans in an extensive and comparative set of visual-search experiments. The findings indicate that both species exhibit similar behavior in basic feature searches and in conjunction search tasks. In contrast, performance differed in searches defined by shape. These results suggest that evolution pressured many visual features to pop out for both species despite cardinal differences in brain anatomy and living environment, and strengthens the argument that aspects of feature integration theory may be generalizable across the animal kingdom.

Self-motion trajectories can facilitate orientation-based figure-ground segregation.

Segregation of objects from the background is a basic and essential property of the visual system. We studied the neural detection of objects defined by orientation difference from background in barn owls (Tyto alba). We presented wide-field displays of densely packed stripes with a dominant orientation. Visual objects were created by orienting a circular patch differently from the background. In head-fixed conditions, neurons in both tecto- and thalamofugal visual pathways (optic tectum and visual Wulst) were weakly responsive to these objects in their receptive fields. However, notably, in freely viewing conditions, barn owls occasionally perform peculiar side-to-side head motions (peering) when scanning the environment. In the second part of the study we thus recorded the neural response from head-fixed owls while the visual displays replicated the peering conditions; i.e., the displays (objects and backgrounds) were shifted along trajectories that induced a retinal motion identical to sampled peering motions during viewing of a static object. These conditions induced dramatic neural responses to the objects, in the very same neurons that where unresponsive to the objects in static displays. By reverting to circular motions of the display, we show that the pattern of the neural response is mostly shaped by the orientation of the background relative to motion and not the orientation of the object. Thus our findings provide evidence that peering and/or other self-motions can facilitate orientation-based figure-ground segregation through interaction with inhibition from the surround.NEW & NOTEWORTHY Animals frequently move their sensory organs and thereby create motion cues that can enhance object segregation from background. We address a special example of such active sensing, in barn owls. When scanning the environment, barn owls occasionally perform small-amplitude side-to-side head movements called peering. We show that the visual outcome of such peering movements elicit neural detection of objects that are rotated from the dominant orientation of the background scene and which are otherwise mostly undetected. These results suggest a novel role for self-motions in sensing objects that break the regular orientation of elements in the scene.

Controlled Lighting and Illumination-Independent Target Detection for Real-Time Cost-Efficient Applications. The Case Study of Sweet Pepper Robotic Harvesting.

Current harvesting robots are limited by low detection rates due to the unstructured and dynamic nature of both the objects and the environment. State-of-the-art algorithms include color- and texture-based detection, which are highly sensitive to the illumination conditions. Deep learning algorithms promise robustness at the cost of significant computational resources and the requirement for intensive databases. In this paper we present a Flash-No-Flash (FNF) controlled illumination acquisition protocol that frees the system from most ambient illumination effects and facilitates robust target detection while using only modest computational resources and no supervised training. The approach relies on the simultaneous acquisition of two images-with/without strong artificial lighting ("Flash"/"no-Flash"). The difference between these images represents the appearance of the target scene as if only the artificial light was present, allowing a tight control over ambient light for color-based detection. A performance evaluation database was acquired in greenhouse conditions using an eye-in-hand RGB camera mounted on a robotic manipulator. The database includes 156 scenes with 468 images containing a total of 344 yellow sweet peppers. Performance of both color blob and deep-learning detection algorithms are compared on Flash-only and FNF images. The collected database is made public.

What pops out for you pops out for fish: Four common visual features.

Visual search is the ability to detect a target of interest against a background of distracting objects. For many animals, performing this task fast and accurately is crucial for survival. Typically, visual-search performance is measured by the time it takes the observer to detect a target against a backdrop of distractors. The efficiency of a visual search depends fundamentally on the features of the target, the distractors, and the interaction between them. Substantial efforts have been devoted to investigating the influence of different visual features on visual-search performance in humans. In particular, it has been demonstrated that color, size, orientation, and motion are efficient visual features to guide attention in humans. However, little is known about which features are efficient and which are not in other vertebrates. Given earlier observations that moving targets elicit pop-out and parallel search in the archerfish during visual-search tasks, here we investigate and confirm that all four of these visual features also facilitate efficient search in the archerfish in a manner comparable to humans. In conjunction with results reported for other species, these finding suggest universality in the way visual search is carried out by animals despite very different brain anatomies and living environments.

What a predator can teach us about visual processing: a lesson from the archerfish.

The archerfish is a predator with highly unusual visually guided behavior. It is most famous for its ability to hunt by shooting water jets at static or dynamic insect prey, up to two meters above the water's surface. In the lab, the archerfish can learn to distinguish and shoot at artificial targets presented on a computer screen, thus enabling well-controlled experiments. In recent years, these capacities have turned the archerfish into a model animal for studying a variety of visual functions, from visual saliency and visual search, through fast visually guided prediction, and all the way to higher level visual processing such as face recognition. Here we review these recent developments and show how they fall into two emerging lines of research on this animal model. The first is ethologically motivated and emphasizes how the natural environment and habitat of the archerfish interact with its visual processing during predation. The second is driven by parallels to the primate brain and aims to determine whether the latter's characteristic visual information processing capacities can also be found in the qualitatively different fish brain, thereby underscoring the functional universality of certain visual processes. We discuss the differences between these two lines of research and possible future directions.

Visual search in barn owls: Task difficulty and saccadic behavior.

How do we find what we are looking for? A target can be in plain view, but it may be detected only after extensive search. During a search we make directed attentional deployments like saccades to segment the scene until we detect the target. Depending on difficulty, the search may be fast with few attentional deployments or slow with many, shorter deployments. Here we study visual search in barn owls by tracking their overt attentional deployments-that is, their head movements-with a camera. We conducted a low-contrast feature search, a high-contrast orientation conjunction search, and a low-contrast orientation conjunction search, each with set sizes varying from 16 to 64 items. The barn owls were able to learn all of these tasks and showed serial search behavior. In a subsequent step, we analyzed how search behavior of owls changes with search complexity. We compared the search mechanisms in these three serial searches with results from pop-out searches our group had reported earlier. Saccade amplitude shortened and fixation duration increased in difficult searches. Also, in conjunction search saccades were guided toward items with shared target features. These data suggest that during visual search, barn owls utilize mechanisms similar to those that humans use.

Specular motion and 3D shape estimation.

Dynamic visual information facilitates three-dimensional shape recognition. It is still unclear, however, whether the motion information generated by moving specularities across a surface is congruent to that available from optic flow produced by a matte-textured shape. Whereas the latter is directly linked to the first-order properties of the shape and its motion relative to the observer, the specular flow, the image flow generated by a specular object, is less sensitive to the object's motion and is tightly related to second-order properties of the shape. We therefore hypothesize that the perceived bumpiness (a perceptual attribute related to curvature magnitude) is more stable to changes in the type of motion in specular objects compared with their matte-textured counterparts. Results from two two-interval forced-choice experiments in which observers judged the perceived bumpiness of perturbed spherelike objects support this idea and provide an additional layer of evidence for the capacity of the visual system to exploit image information for shape inference.

Visual pop-out in barn owls: Human-like behavior in the avian brain.

Visual pop-out is a phenomenon by which the latency to detect a target in a scene is independent of the number of other elements, the distractors. Pop-out is an effective visual-search guidance that occurs typically when the target is distinct in one feature from the distractors, thus facilitating fast detection of predators or prey. However, apart from studies on primates, pop-out has been examined in few species and demonstrated thus far in rats, archer fish, and pigeons only. To fill this gap, here we study pop-out in barn owls. These birds are a unique model system for such exploration because their lack of eye movements dictates visual behavior dominated by head movements. Head saccades and interspersed fixation periods can therefore be tracked and analyzed with a head-mounted wireless microcamera--the OwlCam. Using this methodology we confronted two owls with scenes containing search arrays of one target among varying numbers (15-63) of similar looking distractors. We tested targets distinct either by orientation (Experiment 1) or luminance contrast (Experiment 2). Search time and the number of saccades until the target was fixated remained largely independent of the number of distractors in both experiments. This suggests that barn owls can exhibit pop-out during visual search, thus expanding the group of species and brain structures that can cope with this fundamental visual behavior. The utility of our automatic analysis method is further discussed for other species and scientific questions.

Tangent Bundle Elastica and Computer Vision.

Visual curve completion, an early visual process that completes the occluded parts between observed boundary fragments (a.k.a. inducers), is a major problem in perceptual organization and a critical step toward higher level visual tasks in both biological and machine vision. Most computational contributions to solving this problem suggest desired perceptual properties that the completed contour should satisfy in the image plane, and then seek the mathematical curves that provide them. Alternatively, few studies (including by the authors) have suggested to frame the problem not in the image plane but rather in the unit tangent bundleR (2) × S(1), the space that abstracts the primary visual cortex, where curve completion allegedly occurs. Combining both schools, here we propose and develop a biologically plausible theory of elastica in the tangent bundle that provides not only perceptually superior completion results but also a rigorous computational prediction that inducer curvatures greatly affects the shape of the completed curve, as indeed indicated by human perception.

Distance-Dependent Multimodal Image Registration for Agriculture Tasks.

Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibrating a distance-dependent transformation matrix (DDTM) between the sensors; and representing it in a compact way by regressing the distance-dependent coefficients as distance-dependent functions. The DDTM is measured by calculating a projective transformation matrix for varying distances between the sensors and possible targets. To do so we designed a unique experimental setup including unique Artificial Control Points (ACPs) and their detection algorithms for the two sensors. We demonstrate the utility of our approach using different experiments and evaluation criteria.

Pop-out in visual search of moving targets in the archer fish.

Pop-out in visual search reflects the capacity of observers to rapidly detect visual targets independent of the number of distracting objects in the background. Although it may be beneficial to most animals, pop-out behaviour has been observed only in mammals, where neural correlates are found in primary visual cortex as contextually modulated neurons that encode aspects of saliency. Here we show that archer fish can also utilize this important search mechanism by exhibiting pop-out of moving targets. We explore neural correlates of this behaviour and report the presence of contextually modulated neurons in the optic tectum that may constitute the neural substrate for a saliency map. Furthermore, we find that both behaving fish and neural responses exhibit additive responses to multiple visual features. These findings suggest that similar neural computations underlie pop-out behaviour in mammals and fish, and that pop-out may be a universal search mechanism across all vertebrates.

Effects of surface reflectance on local second order shape estimation in dynamic scenes.

In dynamic scenes, relative motion between the object, the observer, and/or the environment projects as dynamic visual information onto the retina (optic flow) that facilitates 3D shape perception. When the object is diffusely reflective, e.g. a matte painted surface, this optic flow is directly linked to object shape, a property found at the foundations of most traditional shape-from-motion (SfM) schemes. When the object is specular, the corresponding specular flow is related to shape curvature, a regime change that challenges the visual system to determine concurrently both the shape and the distortions of the (sometimes unknown) environment reflected from its surface. While human observers are able to judge the global 3D shape of most specular objects, shape-from-specular-flow (SFSF) is not veridical. In fact, recent studies have also shown systematic biases in the perceived motion of such objects. Here we focus on the perception of local shape from specular flow and compare it to that of matte-textured rotating objects. Observers judged local surface shape by adjusting a rotation and scale invariant shape index probe. Compared to shape judgments of static objects we find that object motion decreases intra-observer variability in local shape estimation. Moreover, object motion introduces systematic changes in perceived shape between matte-textured and specular conditions. Taken together, this study provides a new insight toward the contribution of motion and surface material to local shape perception.

Visual receptive field properties of cells in the optic tectum of the archer fish.

The archer fish is well known for its extreme visual behavior in shooting water jets at prey hanging on vegetation above water. This fish is a promising model in the study of visual system function because it can be trained to respond to artificial targets and thus to provide valuable psychophysical data. Although much behavioral data have indeed been collected over the past two decades, little is known about the functional organization of the main visual area supporting this visual behavior, namely, the fish optic tectum. In this article we focus on a fundamental aspect of this functional organization and provide a detailed analysis of receptive field properties of cells in the archer fish optic tectum. Using extracellular measurements to record activities of single cells, we first measure their retinotectal mapping. We then determine their receptive field properties such as size, selectivity for stimulus direction and orientation, tuning for spatial frequency, and tuning for temporal frequency. Finally, on the basis of all these measurements, we demonstrate that optic tectum cells can be classified into three categories: orientation-tuned cells, direction-tuned cells, and direction-agnostic cells. Our results provide an essential basis for future investigations of information processing in the archer fish visual system.

A perceptual paradigm and psychophysical evidence for hierarchy in scene gist processing.

What is the order of processing in scene gist recognition? Following the seminal studies by Rosch (1978) and Tversky and Hemmenway (1983) it has been assumed that basic-level categorization is privileged over the superordinate level because the former maximizes both within-category similarity and between-category variance. However, recent research has begun to challenge this view (Oliva & Torralba, 2001; Joubert, Rousselet, Fize, & Fabre-Thorpe, 2007; Loschky & Larson, 2010). Here we study these directions more fundamentally by investigating the perceptual relations between scene categories in a way that allows us to identify the order of processing of scene categories across taxonomic levels. We introduce the category discrimination paradigm where we briefly present two real scene stimuli simultaneously and ask human observers whether they belong to the same basic-level category or not (i.e., same/different task). As we show, analysis of the obtained data reveals a hierarchical perceptual structure between different scene categories and a corresponding hierarchical structure at the perceptual processing level. In particular, we show a new type of evidence to suggest that the decision whether the scene is manmade or natural is made first, and only then more complicated decisions are taken (such as whether a manmade scene is indoor or outdoor) among a smaller set of viable candidate categories. We argue that this hierarchical structure improves performance and efficiency in both biological and artificial gist recognition systems.