LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals.

Markerless three-dimensional (3D) pose estimation has become an indispensable tool for kinematic studies of laboratory animals. Most current methods recover 3D poses by multi-view triangulation of deep network-based two-dimensional (2D) pose estimates. However, triangulation requires multiple synchronized cameras and elaborate calibration protocols that hinder its widespread adoption in laboratory studies. Here we describe LiftPose3D, a deep network-based method that overcomes these barriers by reconstructing 3D poses from a single 2D camera view. We illustrate LiftPose3D's versatility by applying it to multiple experimental systems using flies, mice, rats and macaques, and in circumstances where 3D triangulation is impractical or impossible. Our framework achieves accurate lifting for stereotypical and nonstereotypical behaviors from different camera angles. Thus, LiftPose3D permits high-quality 3D pose estimation in the absence of complex camera arrays and tedious calibration procedures and despite occluded body parts in freely behaving animals.

Ascending neurons convey behavioral state to integrative sensory and action selection brain regions.

Knowing one's own behavioral state has long been theorized as critical for contextualizing dynamic sensory cues and identifying appropriate future behaviors. Ascending neurons (ANs) in the motor system that project to the brain are well positioned to provide such behavioral state signals. However, what ANs encode and where they convey these signals remains largely unknown. Here, through large-scale functional imaging in behaving animals and morphological quantification, we report the behavioral encoding and brain targeting of hundreds of genetically identifiable ANs in the adult fly, Drosophila melanogaster. We reveal that ANs encode behavioral states, specifically conveying self-motion to the anterior ventrolateral protocerebrum, an integrative sensory hub, as well as discrete actions to the gnathal ganglia, a locus for action selection. Additionally, AN projection patterns within the motor system are predictive of their encoding. Thus, ascending populations are well poised to inform distinct brain hubs of self-motion and ongoing behaviors and may provide an important substrate for computations that are required for adaptive behavior.

Microengineered devices enable long-term imaging of the ventral nerve cord in behaving adult Drosophila.

The dynamics and connectivity of neural circuits continuously change on timescales ranging from milliseconds to an animal's lifetime. Therefore, to understand biological networks, minimally invasive methods are required to repeatedly record them in behaving animals. Here we describe a suite of devices that enable long-term optical recordings of the adult Drosophila melanogaster ventral nerve cord (VNC). These consist of transparent, numbered windows to replace thoracic exoskeleton, compliant implants to displace internal organs, a precision arm to assist implantation, and a hinged stage to repeatedly tether flies. To validate and illustrate our toolkit we (i) show minimal impact on animal behavior and survival, (ii) follow the degradation of chordotonal organ mechanosensory nerve terminals over weeks after leg amputation, and (iii) uncover waves of neural activity caffeine ingestion. Thus, our long-term imaging toolkit opens up the investigation of premotor and motor circuit adaptations in response to injury, drug ingestion, aging, learning, and disease.

DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila.

Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in three-dimensional (3D) space. Deep neural networks can estimate two-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here, we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.