SLEAP: A deep learning system for multi-animal pose tracking.

The desire to understand how the brain generates and patterns behavior has driven rapid methodological innovation in tools to quantify natural animal behavior. While advances in deep learning and computer vision have enabled markerless pose estimation in individual animals, extending these to multiple animals presents unique challenges for studies of social behaviors or animals in their natural environments. Here we present Social LEAP Estimates Animal Poses (SLEAP), a machine learning system for multi-animal pose tracking. This system enables versatile workflows for data labeling, model training and inference on previously unseen data. SLEAP features an accessible graphical user interface, a standardized data model, a reproducible configuration system, over 30 model architectures, two approaches to part grouping and two approaches to identity tracking. We applied SLEAP to seven datasets across flies, bees, mice and gerbils to systematically evaluate each approach and architecture, and we compare it with other existing approaches. SLEAP achieves greater accuracy and speeds of more than 800 frames per second, with latencies of less than 3.5 ms at full 1,024 × 1,024 image resolution. This makes SLEAP usable for real-time applications, which we demonstrate by controlling the behavior of one animal on the basis of the tracking and detection of social interactions with another animal.

Capturing continuous, long timescale behavioral changes in Drosophila melanogaster postural data.

Animal behavior spans many timescales, from short, seconds-scale actions to circadian rhythms over many hours to life-long changes during aging. Most quantitative behavior studies have focused on short-timescale behaviors such as locomotion and grooming. Analysis of these data suggests there exists a hierarchy of timescales; however, the limited duration of these experiments prevents the investigation of the full temporal structure. To access longer timescales of behavior, we continuously recorded individual Drosophila melanogaster at 100 frames per second for up to 7 days at a time in featureless arenas on sucrose-agarose media. We use the deep learning framework SLEAP to produce a full-body postural data set for 47 individuals resulting in nearly 2 billion pose instances. We identify stereotyped behaviors such as grooming, proboscis extension, and locomotion and use the resulting ethograms to explore how the flies' behavior varies across time of day and days in the experiment. We find distinct circadian patterns in all of our stereotyped behavior and also see changes in behavior over the course of the experiment as the flies weaken and die.

NAPS: Integrating pose estimation and tag-based tracking.

1. Significant advances in computational ethology have allowed the quantification of behaviour in unprecedented detail. Tracking animals in social groups, however, remains challenging as most existing methods can either capture pose or robustly retain individual identity over time but not both. 2. To capture finely resolved behaviours while maintaining individual identity, we built NAPS (NAPS is ArUco Plus SLEAP), a hybrid tracking framework that combines state-of-the-art, deep learning-based methods for pose estimation (SLEAP) with unique markers for identity persistence (ArUco). We show that this framework allows the exploration of the social dynamics of the common eastern bumblebee (Bombus impatiens). 3. We provide a stand-alone Python package for implementing this framework along with detailed documentation to allow for easy utilization and expansion. We show that NAPS can scale to long timescale experiments at a high frame rate and that it enables the investigation of detailed behavioural variation within individuals in a group. 4. Expanding the toolkit for capturing the constituent behaviours of social groups is essential for understanding the structure and dynamics of social networks. NAPS provides a key tool for capturing these behaviours and can provide critical data for understanding how individual variation influences collective dynamics.

Isolation disrupts social interactions and destabilizes brain development in bumblebees.

Social isolation, particularly in early life, leads to deleterious physiological and behavioral outcomes. Here, we leverage new high-throughput tools to comprehensively investigate the impact of isolation in the bumblebee, Bombus impatiens, from behavioral, molecular, and neuroanatomical perspectives. We reared newly emerged bumblebees in complete isolation, in small groups, or in their natal colony, and then analyzed their behaviors while alone or paired with another bee. We find that when alone, individuals of each rearing condition show distinct behavioral signatures. When paired with a conspecific, bees reared in small groups or in the natal colony express similar behavioral profiles. Isolated bees, however, showed increased social interactions. To identify the neurobiological correlates of these differences, we quantified brain gene expression and measured the volumes of key brain regions for a subset of individuals from each rearing condition. Overall, we find that isolation increases social interactions and disrupts gene expression and brain development. Limited social experience in small groups is sufficient to preserve typical patterns of brain development and social behavior.

Live imaging and biophysical modeling support a button-based mechanism of somatic homolog pairing in Drosophila.

Three-dimensional eukaryotic genome organization provides the structural basis for gene regulation. In Drosophila melanogaster, genome folding is characterized by somatic homolog pairing, where homologous chromosomes are intimately paired from end to end; however, how homologs identify one another and pair has remained mysterious. Recently, this process has been proposed to be driven by specifically interacting 'buttons' encoded along chromosomes. Here, we turned this hypothesis into a quantitative biophysical model to demonstrate that a button-based mechanism can lead to chromosome-wide pairing. We tested our model using live-imaging measurements of chromosomal loci tagged with the MS2 and PP7 nascent RNA labeling systems. We show solid agreement between model predictions and experiments in the pairing dynamics of individual homologous loci. Our results strongly support a button-based mechanism of somatic homolog pairing in Drosophila and provide a theoretical framework for revealing the molecular identity and regulation of buttons.