idtracker.ai: tracking all individuals in small or large collectives of unmarked animals.

Understanding of animal collectives is limited by the ability to track each individual. We describe an algorithm and software that extract all trajectories from video, with high identification accuracy for collectives of up to 100 individuals. idtracker.ai uses two convolutional networks: one that detects when animals touch or cross and another for animal identification. The tool is trained with a protocol that adapts to video conditions and tracking difficulty.

Deep attention networks reveal the rules of collective motion in zebrafish.

A variety of simple models has been proposed to understand the collective motion of animals. These models can be insightful but may lack important elements necessary to predict the motion of each individual in the collective. Adding more detail increases predictability but can make models too complex to be insightful. Here we report that deep attention networks can obtain a model of collective behavior that is simultaneously predictive and insightful thanks to an organization in modules. When using simulated trajectories, the model recovers the ground-truth interaction rule used to generate them, as well as the number of interacting neighbours. For experimental trajectories of large groups of 60-100 zebrafish, Danio rerio, the model obtains that interactions between pairs can approximately be described as repulsive, attractive or as alignment, but only when moving slowly. At high velocities, interactions correspond only to alignment or alignment mixed with repulsion at close distances. The model also shows that each zebrafish decides where to move by aggregating information from the group as a weighted average over neighbours. Weights are higher for neighbours that are close, in a collision path or moving faster in frontal and lateral locations. The network also extracts that the number of interacting individuals is dynamical and typically in the range 8-22, with 1-10 more important ones. Our results suggest that each animal decides by dynamically selecting information from the collective.

Ontogeny of collective behavior reveals a simple attraction rule.

The striking patterns of collective animal behavior, including ant trails, bird flocks, and fish schools, can result from local interactions among animals without centralized control. Several of these rules of interaction have been proposed, but it has proven difficult to discriminate which ones are implemented in nature. As a method to better discriminate among interaction rules, we propose to follow the slow birth of a rule of interaction during animal development. Specifically, we followed the development of zebrafish, Danio rerio, and found that larvae turn toward each other from 7 days postfertilization and increase the intensity of interactions until 3 weeks. This developmental dataset allows testing the parameter-free predictions of a simple rule in which animals attract each other part of the time, with attraction defined as turning toward another animal chosen at random. This rule makes each individual likely move to a high density of conspecifics, and moving groups naturally emerge. Development of attraction strength corresponds to an increase in the time spent in attraction behavior. Adults were found to follow the same attraction rule, suggesting a potential significance for adults of other species.

idTracker: tracking individuals in a group by automatic identification of unmarked animals.

Animals in groups touch each other, move in paths that cross, and interact in complex ways. Current video tracking methods sometimes switch identities of unmarked individuals during these interactions. These errors propagate and result in random assignments after a few minutes unless manually corrected. We present idTracker, a multitracking algorithm that extracts a characteristic fingerprint from each animal in a video recording of a group. It then uses these fingerprints to identify every individual throughout the video. Tracking by identification prevents propagation of errors, and the correct identities can be maintained indefinitely. idTracker distinguishes animals even when humans cannot, such as for size-matched siblings, and reidentifies animals after they temporarily disappear from view or across different videos. It is robust, easy to use and general. We tested it on fish (Danio rerio and Oryzias latipes), flies (Drosophila melanogaster), ants (Messor structor) and mice (Mus musculus).

Transient behavior in systems with time-delayed feedback.

We investigate the transient times for the onset of control of steady states by time-delayed feedback. The optimization of control by minimizing the transient time before control becomes effective is discussed analytically and numerically, and the competing influences of local and global features are elaborated. We derive an algebraic scaling of the transient time and confirm our findings by numerical simulations in dependence on feedback gain and time delay.