Automated, high-throughput quantification of EGFP-expressing neutrophils in zebrafish by machine learning and a highly-parallelized microscope.

Normal development of the immune system is essential for overall health and disease resistance. Bony fish, such as the zebrafish (Danio rerio), possess all the major immune cell lineages as mammals and can be employed to model human host response to immune challenge. Zebrafish neutrophils, for example, are present in the transparent larvae as early as 48 hours post fertilization and have been examined in numerous infection and immunotoxicology reports. One significant advantage of the zebrafish model is the ability to affordably generate high numbers of individual larvae that can be arrayed in multi-well plates for high throughput genetic and chemical exposure screens. However, traditional workflows for imaging individual larvae have been limited to low-throughput studies using traditional microscopes and manual analyses. Using a newly developed, parallelized microscope, the Multi-Camera Array Microscope (MCAM™), we have optimized a rapid, high-resolution algorithmic method to count fluorescently labeled cells in zebrafish larvae in vivo. Using transgenic zebrafish larvae, in which neutrophils express EGFP, we captured 18 gigapixels of images across a full 96-well plate, in 75 seconds, and processed the resulting datastream, counting individual fluorescent neutrophils in all individual larvae in 5 minutes. This automation is facilitated by a machine learning segmentation algorithm that defines the most in-focus view of each larva in each well after which pixel intensity thresholding and blob detection are employed to locate and count fluorescent cells. We validated this method by comparing algorithmic neutrophil counts to manual counts in larvae subjected to changes in neutrophil numbers, demonstrating the utility of this approach for high-throughput genetic and chemical screens where a change in neutrophil number is an endpoint metric. Using the MCAM™ we have been able to, within minutes, acquire both enough data to create an automated algorithm and execute a biological experiment with statistical significance. Finally, we present this open-source software package which allows the user to train and evaluate a custom machine learning segmentation model and use it to localize zebrafish and analyze cell counts within the segmented region of interest. This software can be modified as needed for studies involving other zebrafish cell lineages using different transgenic reporter lines and can also be adapted for studies using other amenable model species.

Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second.

Wide field of view microscopy that can resolve 3D information at high speed and spatial resolution is highly desirable for studying the behaviour of freely moving model organisms. However, it is challenging to design an optical instrument that optimises all these properties simultaneously. Existing techniques typically require the acquisition of sequential image snapshots to observe large areas or measure 3D information, thus compromising on speed and throughput. Here, we present 3D-RAPID, a computational microscope based on a synchronized array of 54 cameras that can capture high-speed 3D topographic videos over an area of 135 cm2, achieving up to 230 frames per second at spatiotemporal throughputs exceeding 5 gigapixels per second. 3D-RAPID employs a 3D reconstruction algorithm that, for each synchronized snapshot, fuses all 54 images into a composite that includes a co-registered 3D height map. The self-supervised 3D reconstruction algorithm trains a neural network to map raw photometric images to 3D topography using stereo overlap redundancy and ray-propagation physics as the only supervision mechanism. The resulting reconstruction process is thus robust to generalization errors and scales to arbitrarily long videos from arbitrarily sized camera arrays. We demonstrate the broad applicability of 3D-RAPID with collections of several freely behaving organisms, including ants, fruit flies, and zebrafish larvae.

Automated, high-throughput quantification of EGFP-expressing neutrophils in zebrafish by machine learning and a highly-parallelized microscope.

Normal development of the immune system is essential for overall health and disease resistance. Bony fish, such as the zebrafish (Danio rerio), possess all the major immune cell lineages as mammals and can be employed to model human host response to immune challenge. Zebrafish neutrophils, for example, are present in the transparent larvae as early as 48 hours post fertilization and have been examined in numerous infection and immunotoxicology reports. One significant advantage of the zebrafish model is the ability to affordably generate high numbers of individual larvae that can be arrayed in multi-well plates for high throughput genetic and chemical exposure screens. However, traditional workflows for imaging individual larvae have been limited to low-throughput studies using traditional microscopes and manual analyses. Using a newly developed, parallelized microscope, the Multi-Camera Array Microscope (MCAM™), we have optimized a rapid, high-resolution algorithmic method to count fluorescently labeled cells in zebrafish larvae in vivo. Using transgenic zebrafish larvae, in which neutrophils express EGFP, we captured 18 gigapixels of images across a full 96-well plate, in 75 seconds, and processed the resulting datastream, counting individual fluorescent neutrophils in all individual larvae in 5 minutes. This automation is facilitated by a machine learning segmentation algorithm that defines the most in-focus view of each larva in each well after which pixel intensity thresholding and blob detection are employed to locate and count fluorescent cells. We validated this method by comparing algorithmic neutrophil counts to manual counts in larvae subjected to changes in neutrophil numbers, demonstrating the utility of this approach for high-throughput genetic and chemical screens where a change in neutrophil number is an endpoint metric. Using the MCAM™ we have been able to, within minutes, acquire both enough data to create an automated algorithm and execute a biological experiment with statistical significance. Finally, we present this open-source software package which allows the user to train and evaluate a custom machine learning segmentation model and use it to localize zebrafish and analyze cell counts within the segmented region of interest. This software can be modified as needed for studies involving other zebrafish cell lineages using different transgenic reporter lines and can also be adapted for studies using other amenable model species.

Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second.

To study the behavior of freely moving model organisms such as zebrafish (Danio rerio) and fruit flies (Drosophila) across multiple spatial scales, it would be ideal to use a light microscope that can resolve 3D information over a wide field of view (FOV) at high speed and high spatial resolution. However, it is challenging to design an optical instrument to achieve all of these properties simultaneously. Existing techniques for large-FOV microscopic imaging and for 3D image measurement typically require many sequential image snapshots, thus compromising speed and throughput. Here, we present 3D-RAPID, a computational microscope based on a synchronized array of 54 cameras that can capture high-speed 3D topographic videos over a 135-cm^2 area, achieving up to 230 frames per second at throughputs exceeding 5 gigapixels (GPs) per second. 3D-RAPID features a 3D reconstruction algorithm that, for each synchronized temporal snapshot, simultaneously fuses all 54 images seamlessly into a globally-consistent composite that includes a coregistered 3D height map. The self-supervised 3D reconstruction algorithm itself trains a spatiotemporally-compressed convolutional neural network (CNN) that maps raw photometric images to 3D topography, using stereo overlap redundancy and ray-propagation physics as the only supervision mechanism. As a result, our end-to-end 3D reconstruction algorithm is robust to generalization errors and scales to arbitrarily long videos from arbitrarily sized camera arrays. The scalable hardware and software design of 3D-RAPID addresses a longstanding problem in the field of behavioral imaging, enabling parallelized 3D observation of large collections of freely moving organisms at high spatiotemporal throughputs, which we demonstrate in ants (Pogonomyrmex barbatus), fruit flies, and zebrafish larvae.