Multiple-target tracking in human and machine vision.

Humans are able to track multiple objects at any given time in their daily activities-for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human-computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Effectiveness of "rescue saccades" on the accuracy of tracking multiple moving targets: An eye-tracking study on the effects of target occlusions.

Occlusion is one of the main challenges in tracking multiple moving objects. In almost all real-world scenarios, a moving object or a stationary obstacle occludes targets partially or completely for a short or long time during their movement. A previous study (Zelinsky & Todor, 2010) reported that subjects make timely saccades toward the object in danger of being occluded. Observers make these so-called "rescue saccades" to prevent target swapping. In this study, we examined whether these saccades are helpful. To this aim, we used as the stimuli recorded videos from natural movement of zebrafish larvae swimming freely in a circular container. We considered two main types of occlusion: object-object occlusions that naturally exist in the videos, and object-occluder occlusions created by adding a stationary doughnut-shape occluder in some videos. Four different scenarios were studied: (1) no occlusions, (2) only object-object occlusions, (3) only object-occluder occlusion, or (4) both object-object and object-occluder occlusions. For each condition, two set sizes (two and four) were applied. Participants' eye movements were recorded during tracking, and rescue saccades were extracted afterward. The results showed that rescue saccades are helpful in handling object-object occlusions but had no reliable effect on tracking through object-occluder occlusions. The presence of occlusions generally increased visual sampling of the scenes; nevertheless, tracking accuracy declined due to occlusion.

Brain-inspired multiple-target tracking using Dynamic Neural Fields.

Despite considerable progress in the field of automatic multi-target tracking, several problems such as data association remained challenging. On the other hand, cognitive studies have reported that humans can robustly track several objects simultaneously. Such circumstances happen regularly in daily life, and humans have evolved to handle the associated problems. Accordingly, using brain-inspired processing principles may contribute to significantly increase the performance of automatic systems able to follow the trajectories of multiple objects. In this paper, we propose a multiple-object tracking algorithm based on dynamic neural field theory which has been proven to provide neuro-plausible processing mechanisms for cognitive functions of the brain. We define several input neural fields responsible for representing previous location and orientation information as well as instantaneous linear and angular speed of the objects in successive video frames. Image processing techniques are applied to extract the critical object features including target location and orientation. Two prediction fields anticipate the objects' locations and orientations in the upcoming frame after receiving excitatory and inhibitory inputs from the input fields in a feed-forward architecture. This information is used in the data association and labeling process. We tested the proposed algorithm on a zebrafish larvae segmentation and tracking dataset and an ant-tracking dataset containing non-rigid objects with spiky movements and frequently occurring occlusions. The results showed a significant improvement in tracking metrics compared to state-of-the-art algorithms.

Early Visual Processing of Feature Saliency Tasks: A Review of Psychophysical Experiments.

The visual system is constantly bombarded with information originating from the outside world, but it is unable to process all the received information at any given time. In fact, the most salient parts of the visual scene are chosen to be processed involuntarily and immediately after the first glance along with endogenous signals in the brain. Vision scientists have shown that the early visual system, from retina to lateral geniculate nucleus (LGN) and then primary visual cortex, selectively processes the low-level features of the visual scene. Everything we perceive from the visual scene is based on these feature properties and their subsequent combination in higher visual areas. Different experiments have been designed to investigate the impact of these features on saliency and understand the relative visual mechanisms. In this paper, we review the psychophysical experiments which have been published in the last decades to indicate how the low-level salient features are processed in the early visual cortex and extract the most important and basic information of the visual scene. Important and open questions are discussed in this review as well and one might pursue these questions to investigate the impact of higher level features on saliency in complex scenes or natural images.