A Foundation Model for Cell Segmentation.

Cells are a fundamental unit of biological organization, and identifying them in imaging data - cell segmentation - is a critical task for various cellular imaging experiments. While deep learning methods have led to substantial progress on this problem, most models in use are specialist models that work well for specific domains. Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive. In this work, we present CellSAM, a foundation model for cell segmentation that generalizes across diverse cellular imaging data. CellSAM builds on top of the Segment Anything Model (SAM) by developing a prompt engineering approach for mask generation. We train an object detector, CellFinder, to automatically detect cells and prompt SAM to generate segmentations. We show that this approach allows a single model to achieve human-level performance for segmenting images of mammalian cells (in tissues and cell culture), yeast, and bacteria collected across various imaging modalities. We show that CellSAM has strong zero-shot performance and can be improved with a few examples via few-shot learning. We also show that CellSAM can unify bioimaging analysis workflows such as spatial transcriptomics and cell tracking. A deployed version of CellSAM is available at https://cellsam.deepcell.org/.

Anatomically distributed neural representations of instincts in the hypothalamus.

Artificial activation of anatomically localized, genetically defined hypothalamic neuron populations is known to trigger distinct innate behaviors, suggesting a hypothalamic nucleus-centered organization of behavior control. To assess whether the encoding of behavior is similarly anatomically confined, we performed simultaneous neuron recordings across twenty hypothalamic regions in freely moving animals. Here we show that distinct but anatomically distributed neuron ensembles encode the social and fear behavior classes, primarily through mixed selectivity. While behavior class-encoding ensembles were spatially distributed, individual ensembles exhibited strong localization bias. Encoding models identified that behavior actions, but not motion-related variables, explained a large fraction of hypothalamic neuron activity variance. These results identify unexpected complexity in the hypothalamic encoding of instincts and provide a foundation for understanding the role of distributed neural representations in the expression of behaviors driven by hardwired circuits.

A multicomponent approach to studying cultural propensities during foraging in the wild.

Determining the cultural propensities or cultural behaviours of a species during foraging entails an investigation of underlying drivers and motivations. In this article, we propose a multicomponent approach involving behaviour, ecology, and physiology to accelerate the study of cultural propensities in the wild. We propose as the first component the use of field experiments that simulate natural contexts, such as foraging behaviours and tool use opportunities, to explore social learning and cultural tendencies in a variety of species. To further accelerate this component, we discuss and advocate for the use of modern machine learning video analysis tools. In conjunction, we examine non-invasive methods to measure ecological influences on foraging such as phenology, fruit availability, dietary intake; and physiological influences such as stress, protein balance, energetics, and metabolism. We feature non-invasive urine sampling to investigate urea, creatinine, ketone bodies, the thyroid hormone triiodothyronine (T3), cortisol and connecting peptides of insulin. To conclude, we highlight the benefits of combining ecological and physiological conditions with behavioural field experiments. This can be done across wild species, and provides the framework needed to test ecological hypotheses related to cultural behaviour.

Deep-learning based identification, tracking, pose estimation, and behavior classification of interacting primates and mice in complex environments.

The quantification of behaviors of interest from video data is commonly used to study brain function, the effects of pharmacological interventions, and genetic alterations. Existing approaches lack the capability to analyze the behavior of groups of animals in complex environments. We present a novel deep learning architecture for classifying individual and social animal behavior, even in complex environments directly from raw video frames, while requiring no intervention after initial human supervision. Our behavioral classifier is embedded in a pipeline (SIPEC) that performs segmentation, identification, pose-estimation, and classification of complex behavior, outperforming the state of the art. SIPEC successfully recognizes multiple behaviors of freely moving individual mice as well as socially interacting non-human primates in 3D, using data only from simple mono-vision cameras in home-cage setups.

An optimized registration workflow and standard geometric space for small animal brain imaging.

The reliability of scientific results critically depends on reproducible and transparent data processing. Cross-subject and cross-study comparability of imaging data in general, and magnetic resonance imaging (MRI) data in particular, is contingent on the quality of registration to a standard reference space. In small animal MRI this is not adequately provided by currently used processing workflows, which utilize high-level scripts optimized for human data, and adapt animal data to fit the scripts, rather than vice-versa. In this fully reproducible article we showcase a generic workflow optimized for the mouse brain, alongside a standard reference space suited to harmonize data between analysis and operation. We introduce four separate metrics for automated quality control (QC), and a visualization method to aid operator inspection. Benchmarking this workflow against common legacy practices reveals that it performs more consistently, better preserves variance across subjects while minimizing variance across sessions, and improves both volume and smoothness conservation RMSE approximately 2-fold. We propose this open source workflow and the QC metrics as a new standard for small animal MRI registration, ensuring workflow robustness, data comparability, and region assignment validity, all of which are indispensable prerequisites for the comparability of scientific results across experiments and centers.

Non-invasive molecularly-specific millimeter-resolution manipulation of brain circuits by ultrasound-mediated aggregation and uncaging of drug carriers.

Non-invasive, molecularly-specific, focal modulation of brain circuits with low off-target effects can lead to breakthroughs in treatments of brain disorders. We systemically inject engineered ultrasound-controllable drug carriers and subsequently apply a novel two-component Aggregation and Uncaging Focused Ultrasound Sequence (AU-FUS) at the desired targets inside the brain. The first sequence aggregates drug carriers with millimeter-precision by orders of magnitude. The second sequence uncages the carrier's cargo locally to achieve high target specificity without compromising the blood-brain barrier (BBB). Upon release from the carriers, drugs locally cross the intact BBB. We show circuit-specific manipulation of sensory signaling in motor cortex in rats by locally concentrating and releasing a GABAA receptor agonist from ultrasound-controlled carriers. Our approach uses orders of magnitude (1300x) less drug than is otherwise required by systemic injection and requires very low ultrasound pressures (20-fold below FDA safety limits for diagnostic imaging). We show that the BBB remains intact using passive cavitation detection (PCD), MRI-contrast agents and, importantly, also by sensitive fluorescent dye extravasation and immunohistochemistry.

An Automated Open-Source Workflow for Standards-Compliant Integration of Small Animal Magnetic Resonance Imaging Data.

Large-scale research integration is contingent on seamless access to data in standardized formats. Standards enable researchers to understand external experiment structures, pool results, and apply homogeneous preprocessing and analysis workflows. Particularly, they facilitate these features without the need for numerous potentially confounding compatibility add-ons. In small animal magnetic resonance imaging, an overwhelming proportion of data is acquired via the ParaVision software of the Bruker Corporation. The original data structure is predominantly transparent, but fundamentally incompatible with modern pipelines. Additionally, it sources metadata from free-field operator input, which diverges strongly between laboratories and researchers. In this article we present an open-source workflow which automatically converts and reposits data from the ParaVision structure into the widely supported and openly documented Brain Imaging Data Structure (BIDS). Complementing this workflow we also present operator guidelines for appropriate ParaVision data input, and a programmatic walk-through detailing how preexisting scans with uninterpretable metadata records can easily be made compliant after the acquisition.