The Cell Tracking Challenge: 10 years of objective benchmarking.

The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

Applying systems-level spectral imaging and analysis to reveal the organelle interactome.

The organization of the eukaryotic cell into discrete membrane-bound organelles allows for the separation of incompatible biochemical processes, but the activities of these organelles must be coordinated. For example, lipid metabolism is distributed between the endoplasmic reticulum for lipid synthesis, lipid droplets for storage and transport, mitochondria and peroxisomes for ß-oxidation, and lysosomes for lipid hydrolysis and recycling. It is increasingly recognized that organelle contacts have a vital role in diverse cellular functions. However, the spatial and temporal organization of organelles within the cell remains poorly characterized, as fluorescence imaging approaches are limited in the number of different labels that can be distinguished in a single image. Here we present a systems-level analysis of the organelle interactome using a multispectral image acquisition method that overcomes the challenge of spectral overlap in the fluorescent protein palette. We used confocal and lattice light sheet instrumentation and an imaging informatics pipeline of five steps to achieve mapping of organelle numbers, volumes, speeds, positions and dynamic inter-organelle contacts in live cells from a monkey fibroblast cell line. We describe the frequency and locality of two-, three-, four- and five-way interactions among six different membrane-bound organelles (endoplasmic reticulum, Golgi, lysosome, peroxisome, mitochondria and lipid droplet) and show how these relationships change over time. We demonstrate that each organelle has a characteristic distribution and dispersion pattern in three-dimensional space and that there is a reproducible pattern of contacts among the six organelles, that is affected by microtubule and cell nutrient status. These live-cell confocal and lattice light sheet spectral imaging approaches are applicable to any cell system expressing multiple fluorescent probes, whether in normal conditions or when cells are exposed to disturbances such as drugs, pathogens or stress. This methodology thus offers a powerful descriptive tool and can be used to develop hypotheses about cellular organization and dynamics.

Calcium-vesicles perform active diffusion in the sea urchin embryo during larval biomineralization.

Biomineralization is the process by which organisms use minerals to harden their tissues and provide them with physical support. Biomineralizing cells concentrate the mineral in vesicles that they secret into a dedicated compartment where crystallization occurs. The dynamics of vesicle motion and the molecular mechanisms that control it, are not well understood. Sea urchin larval skeletogenesis provides an excellent platform for investigating the kinetics of mineral-bearing vesicles. Here we used lattice light-sheet microscopy to study the three-dimensional (3D) dynamics of calcium-bearing vesicles in the cells of normal sea urchin embryos and of embryos where skeletogenesis is blocked through the inhibition of Vascular Endothelial Growth Factor Receptor (VEGFR). We developed computational tools for displaying 3D-volumetric movies and for automatically quantifying vesicle dynamics. Our findings imply that calcium vesicles perform an active diffusion motion in both, calcifying (skeletogenic) and non-calcifying (ectodermal) cells of the embryo. The diffusion coefficient and vesicle speed are larger in the mesenchymal skeletogenic cells compared to the epithelial ectodermal cells. These differences are possibly due to the distinct mechanical properties of the two tissues, demonstrated by the enhanced f-actin accumulation and myosinII activity in the ectodermal cells compared to the skeletogenic cells. Vesicle motion is not directed toward the biomineralization compartment, but the vesicles slow down when they approach it, and probably bind for mineral deposition. VEGFR inhibition leads to an increase of vesicle volume but hardly changes vesicle kinetics and doesn't affect f-actin accumulation and myosinII activity. Thus, calcium vesicles perform an active diffusion motion in the cells of the sea urchin embryo, with diffusion length and speed that inversely correlate with the strength of the actomyosin network. Overall, our studies provide an unprecedented view of calcium vesicle 3D-dynamics and point toward cytoskeleton remodeling as an important effector of the motion of mineral-bearing vesicles.

Hydra image processor: 5-D GPU image analysis library with MATLAB and python wrappers.

SUMMARY: Light microscopes can now capture data in five dimensions at very high frame rates producing terabytes of data per experiment. Five-dimensional data has three spatial dimensions (x, y, z), multiple channels (λ) and time (t). Current tools are prohibitively time consuming and do not efficiently utilize available hardware. The hydra image processor (HIP) is a new library providing hardware-accelerated image processing accessible from interpreted languages including MATLAB and Python. HIP automatically distributes data/computation across system and video RAM allowing hardware-accelerated processing of arbitrarily large images. HIP also partitions compute tasks optimally across multiple GPUs. HIP includes a new kernel renormalization reducing boundary effects associated with widely used padding approaches. AVAILABILITY AND IMPLEMENTATION: HIP is free and open source software released under the BSD 3-Clause License. Source code and compiled binary files will be maintained on http://www.hydraimageprocessor.com. A comprehensive description of all MATLAB and Python interfaces and user documents are provided. HIP includes GPU-accelerated support for most common image processing operations in 2-D and 3-D and is easily extensible. HIP uses the NVIDIA CUDA interface to access the GPU. CUDA is well supported on Windows and Linux with macOS support in the future.

Objective comparison of particle tracking methods.

Particle tracking is of key importance for quantitative analysis of intracellular dynamic processes from time-lapse microscopy image data. Because manually detecting and following large numbers of individual particles is not feasible, automated computational methods have been developed for these tasks by many groups. Aiming to perform an objective comparison of methods, we gathered the community and organized an open competition in which participating teams applied their own methods independently to a commonly defined data set including diverse scenarios. Performance was assessed using commonly defined measures. Although no single method performed best across all scenarios, the results revealed clear differences between the various approaches, leading to notable practical conclusions for users and developers.

LEVER: software tools for segmentation, tracking and lineaging of proliferating cells.

The analysis of time-lapse images showing cells dividing to produce clones of related cells is an important application in biological microscopy. Imaging at the temporal resolution required to establish accurate tracking for vertebrate stem or cancer cells often requires the use of transmitted light or phase-contrast microscopy. Processing these images requires automated segmentation, tracking and lineaging algorithms. There is also a need for any errors in the automated processing to be easily identified and quickly corrected. We have developed LEVER, an open source software tool that combines the automated image analysis for phase-contrast microscopy movies with an easy-to-use interface for validating the results and correcting any errors. AVAILABILITY AND IMPLEMENTATION: LEVER is available free and open source, licensed under the GNU GPLv3. Details on obtaining and using LEVER are available at http://n2t.net/ark:/87918/d9rp4t CONTACT: acohen@coe.drexel.edu.

Visualization and correction of automated segmentation, tracking and lineaging from 5-D stem cell image sequences.

BACKGROUND: Neural stem cells are motile and proliferative cells that undergo mitosis, dividing to produce daughter cells and ultimately generating differentiated neurons and glia. Understanding the mechanisms controlling neural stem cell proliferation and differentiation will play a key role in the emerging fields of regenerative medicine and cancer therapeutics. Stem cell studies in vitro from 2-D image data are well established. Visualizing and analyzing large three dimensional images of intact tissue is a challenging task. It becomes more difficult as the dimensionality of the image data increases to include time and additional fluorescence channels. There is a pressing need for 5-D image analysis and visualization tools to study cellular dynamics in the intact niche and to quantify the role that environmental factors play in determining cell fate. RESULTS: We present an application that integrates visualization and quantitative analysis of 5-D (x,y,z,t,channel) and large montage confocal fluorescence microscopy images. The image sequences show stem cells together with blood vessels, enabling quantification of the dynamic behaviors of stem cells in relation to their vascular niche, with applications in developmental and cancer biology. Our application automatically segments, tracks, and lineages the image sequence data and then allows the user to view and edit the results of automated algorithms in a stereoscopic 3-D window while simultaneously viewing the stem cell lineage tree in a 2-D window. Using the GPU to store and render the image sequence data enables a hybrid computational approach. An inference-based approach utilizing user-provided edits to automatically correct related mistakes executes interactively on the system CPU while the GPU handles 3-D visualization tasks. CONCLUSIONS: By exploiting commodity computer gaming hardware, we have developed an application that can be run in the laboratory to facilitate rapid iteration through biological experiments. We combine unsupervised image analysis algorithms with an interactive visualization of the results. Our validation interface allows for each data set to be corrected to 100% accuracy, ensuring that downstream data analysis is accurate and verifiable. Our tool is the first to combine all of these aspects, leveraging the synergies obtained by utilizing validation information from stereo visualization to improve the low level image processing tasks.

Computational prediction of neural progenitor cell fates.

Understanding how stem and progenitor cells choose between alternative cell fates is a major challenge in developmental biology. Efforts to tackle this problem have been hampered by the scarcity of markers that can be used to predict cell division outcomes. Here we present a computational method, based on algorithmic information theory, to analyze dynamic features of living cells over time. Using this method, we asked whether rat retinal progenitor cells (RPCs) display characteristic phenotypes before undergoing mitosis that could foretell their fate. We predicted whether RPCs will undergo a self-renewing or terminal division with 99% accuracy, or whether they will produce two photoreceptors or another combination of offspring with 87% accuracy. Our implementation can segment, track and generate predictions for 40 cells simultaneously on a standard computer at 5 min per frame. This method could be used to isolate cell populations with specific developmental potential, enabling previously impossible investigations.

Postoperative Opioid Requirements Following Roux-en-Y Gastric Bypass in Patients Receiving Continuous Bupivacaine Through a Pump System: A Retrospective Review.

BACKGROUND: Patients who undergo Roux-en-Y gastric bypass (RYGB) surgery have self-reported considerable postoperative pain, often requiring opioid administration. OBJECTIVE: To determine whether continuous delivery of local anesthetic via an infusion pump system decreased postoperative opioid usage in post-RYGB patients. METHODS: The electronic health record was used to identify and review 289 patients who underwent RYGB at our institution from January 2009 to October 2011. The treatment group received a continuous infusion of 0.375% bupivacaine administered by intraperitoneal soaker catheter for 48 hours via an infusion pump; the control group did not receive a pump or local anesthetic. Both groups received general anesthesia, nausea prophylaxis, and pain medication. Pain management consisted of opioid-containing patient-controlled analgesia (PCA) for the first 24 hours. Patients transitioned to supplemental intravenous opioid boluses, plus an oral opioid, for the remainder of their stay. Opioid use was measured in terms of morphine equivalents. Secondary outcomes included visual analog scale (VAS) pain scores and length of hospitalization. RESULTS: Morphine equivalents over the postoperative time period studied were significantly lower in the bupivacaine group than the control group (133 vs 106 mg, respectively; P = .001). There was no significant difference in VAS scores between the 2 groups (P = .80). Finally, the length of hospitalization between the 2 groups did not differ (P = .77). CONCLUSIONS: We have shown that continuous infusion of bupivacaine, administered via a pain pump system, may have decreased postoperative opioid utilization. There were no differences in VAS scores or length of hospitalization between groups.

The Cluster Structure Function.

For each partition of a data set into a given number of parts there is a partition such that every part is as much as possible a good model (an "algorithmic sufficient statistic") for the data in that part. Since this can be done for every number between one and the number of data, the result is a function, the cluster structure function. It maps the number of parts of a partition to values related to the deficiencies of being good models by the parts. Such a function starts with a value at least zero for no partition of the data set and descents to zero for the partition of the data set into singleton parts. The optimal clustering is the one selected by analyzing the cluster structure function. The theory behind the method is expressed in algorithmic information theory (Kolmogorov complexity). In practice the Kolmogorov complexities involved are approximated by a concrete compressor. We give examples using real data sets: the MNIST handwritten digits and the segmentation of real cells as used in stem cell research.

Automatic detection of spatio-temporal signaling patterns in cell collectives.

Increasing experimental evidence points to the physiological importance of space-time correlations in signaling of cell collectives. From wound healing to epithelial homeostasis to morphogenesis, coordinated activation of biomolecules between cells allows the collectives to perform more complex tasks and to better tackle environmental challenges. To capture this information exchange and to advance new theories of emergent phenomena, we created ARCOS, a computational method to detect and quantify collective signaling. We demonstrate ARCOS on cell and organism collectives with space-time correlations on different scales in 2D and 3D. We made a new observation that oncogenic mutations in the MAPK/ERK and PIK3CA/Akt pathways of MCF10A epithelial cells hyperstimulate intercellular ERK activity waves that are largely dependent on matrix metalloproteinase intercellular signaling. ARCOS is open-source and available as R and Python packages. It also includes a plugin for the napari image viewer to interactively quantify collective phenomena without prior programming experience.

Generation of Rab-based transgenic lines for in vivo studies of endosome biology in zebrafish.

The Rab family of small GTPases function as molecular switches regulating membrane and protein trafficking. Individual Rab isoforms define and are required for specific endosomal compartments. To facilitate in vivo investigation of specific Rab proteins, and endosome biology in general, we have generated transgenic zebrafish lines to mark and manipulate Rab proteins. We also developed software to track and quantify endosome dynamics within time-lapse movies. The established transgenic lines ubiquitously express EGFP fusions of Rab5c (early endosomes), Rab11a (recycling endosomes), and Rab7 (late endosomes) to study localization and dynamics during development. Additionally, we generated UAS-based transgenic lines expressing constitutive active (CA) and dominant-negative (DN) versions for each of these Rab proteins. Predicted localization and functional consequences for each line were verified through a variety of assays, including lipophilic dye uptake and Crumbs2a localization. In summary, we have established a toolset for in vivo analyses of endosome dynamics and functions.

LEVERSC: Cross-Platform Scriptable Multichannel 3-D Visualization for Fluorescence Microscopy Images.

We describe a new open-source program called LEVERSC to address the challenges of visualizing the multi-channel 3-D images prevalent in biological microscopy. LEVERSC uses a custom WebGL hardware-accelerated raycasting engine unique in its combination of rendering quality and performance, particularly for multi-channel data. Key features include platform independence, quantitative visualization through interactive voxel localization, and reproducible dynamic visualization via the scripting interface. LEVERSC is fully scriptable and interactive, and works with MATLAB, Python and Java/ImageJ.

Spatiotemporal control of ERK pulse frequency coordinates fate decisions during mammary acinar morphogenesis.

The signaling events controlling proliferation, survival, and apoptosis during mammary epithelial acinar morphogenesis remain poorly characterized. By imaging single-cell ERK activity dynamics in MCF10A acini, we find that these fates depend on the average frequency of non-periodic ERK pulses. High pulse frequency is observed during initial acinus growth, correlating with rapid cell motility and proliferation. Subsequent decrease in motility correlates with lower ERK pulse frequency and quiescence. Later, during lumen formation, coordinated multicellular ERK waves emerge, correlating with high and low ERK pulse frequencies in outer surviving and inner dying cells, respectively. Optogenetic entrainment of ERK pulses causally connects high ERK pulse frequency with inner cell survival. Acini harboring the PIK3CA H1047R mutation display increased ERK pulse frequency and inner cell survival. Thus, fate decisions during acinar morphogenesis are coordinated by different spatiotemporal modalities of ERK pulse frequency.

Collective ERK/Akt activity waves orchestrate epithelial homeostasis by driving apoptosis-induced survival.

Cell death events continuously challenge epithelial barrier function yet are crucial to eliminate old or critically damaged cells. How such apoptotic events are spatio-temporally organized to maintain epithelial homeostasis remains unclear. We observe waves of extracellular-signal-regulated kinase (ERK) and AKT serine/threonine kinase (Akt) activity pulses that originate from apoptotic cells and propagate radially to healthy surrounding cells. This requires epidermal growth factor receptor (EGFR) and matrix metalloproteinase (MMP) signaling. At the single-cell level, ERK/Akt waves act as spatial survival signals that locally protect cells in the vicinity of the epithelial injury from apoptosis for a period of 3-4 h. At the cell population level, ERK/Akt waves maintain epithelial homeostasis (EH) in response to mild or intense environmental insults. Disruption of this spatial signaling system results in the inability of a model epithelial tissue to ensure barrier function in response to environmental insults.

3D Image Analysis of the Complete Ventricular-Subventricular Zone Stem Cell Niche Reveals Significant Vasculature Changes and Progenitor Deficits in Males Versus Females with Aging.

With age, neural stem cell (NSC) function in the adult ventricular-subventricular zone (V-SVZ) declines, reducing memory and cognitive function in males; however, the impact on females is not well understood. To obtain a global view of how age and sex impact the mouse V-SVZ, we constructed 3D montages after multiplex immunostaining, and used computer-based 3D image analysis to quantify data across the entire niche at 2, 18, and 22 months. We discovered dramatic sex differences in the aging of the V-SVZ niche vasculature, which regulates NSC activity: females showed increased diameter but decreased vessel density with age, while males showed decreased diameter and increased tortuosity and vessel density. Accompanying these vascular changes, males showed significant decline in NSC numbers, progenitor cell proliferation, and more disorganized migrating neuroblast chains with age; however, females did not. By examining the entire 3D niche, we found significant sex differences, with females being relatively spared through very old age.

Human Serum Enhances Biomimicry of Engineered Tissue Models of Bone and Cancer.

For decades, fetal bovine serum (FBS) has been used routinely for culturing many cell types, based on its empirically demonstrated effects on cell growth, and the lack of suitable non-xenogeneic alternatives. The FBS-based culture media do not represent the human physiological conditions, and can compromise biomimicry of preclinical models. To recapitulate in vitro the features of human bone and bone cancer, we investigated the effects of human serum and human platelet lysate on modeling osteogenesis, osteoclastogenesis, and bone cancer in two-dimensional (2D) and three-dimensional (3D) settings. For monitoring tumor growth within tissue-engineered bone in a non-destructive fashion, we generated cancer cell lines expressing and secreting luciferase. Culture media containing human serum enhanced osteogenesis and osteoclasts differentiation, and provided a more realistic in vitro mimic of human cancer cell proliferation. When human serum was used for building 3D engineered bone, the tissue recapitulated bone homeostasis and response to bisphosphonates observed in native bone. We found disparities in cell behavior and drug responses between the metastatic and primary cancer cells cultured in the bone niche, with the effectiveness of bisphosphonates observed only in metastatic models. Overall, these data support the utility of human serum for bioengineering of bone and bone cancers.

Automatic summarization of changes in biological image sequences using algorithmic information theory.

An algorithmic information-theoretic method is presented for object-level summarization of meaningful changes in image sequences. Object extraction and tracking data are represented as an attributed tracking graph (ATG). Time courses of object states are compared using an adaptive information distance measure, aided by a closed-form multidimensional quantization. The notion of meaningful summarization is captured by using the gap statistic to estimate the randomness deficiency from algorithmic statistics. The summary is the clustering result and feature subset that maximize the gap statistic. This approach was validated on four bioimaging applications: 1) It was applied to a synthetic data set containing two populations of cells differing in the rate of growth, for which it correctly identified the two populations and the single feature out of 23 that separated them; 2) it was applied to 59 movies of three types of neuroprosthetic devices being inserted in the brain tissue at three speeds each, for which it correctly identified insertion speed as the primary factor affecting tissue strain; 3) when applied to movies of cultured neural progenitor cells, it correctly distinguished neurons from progenitors without requiring the use of a fixative stain; and 4) when analyzing intracellular molecular transport in cultured neurons undergoing axon specification, it automatically confirmed the role of kinesins in axon specification.

Separating Touching Cells Using Pixel Replicated Elliptical Shape Models.

One of the most important and error-prone tasks in biological image analysis is the segmentation of touching or overlapping cells. Particularly for optical microscopy, including transmitted light and confocal fluorescence microscopy, there is often no consistent discriminative information to separate cells that touch or overlap. It is desired to partition touching foreground pixels into cells using the binary threshold image information only, and optionally incorporating gradient information. The most common approaches for segmenting touching and overlapping cells in these scenarios are based on the watershed transform. We describe a new approach called pixel replication for the task of segmenting elliptical objects that touch or overlap. Pixel replication uses the image Euclidean distance transform in combination with Gaussian mixture models to better exploit practically effective optimization for delineating objects with elliptical decision boundaries. Pixel replication improves significantly on commonly used methods based on watershed transforms, or based on fitting Gaussian mixtures directly to the thresholded image data. Pixel replication works equivalently on both 2-D and 3-D image data, and naturally combines information from multi-channel images. The accuracy of the proposed technique is measured using both the segmentation accuracy on simulated ellipse data and the tracking accuracy on validated stem cell tracking results extracted from hundreds of live-cell microscopy image sequences. Pixel replication is shown to be significantly more accurate compared with other approaches. Variance relationships are derived, allowing a more practically effective Gaussian mixture model to extract cell boundaries for data generated from the threshold image using the uniform elliptical distribution and from the distance transform image using the triangular elliptical distribution.

Contact guidance is cell cycle-dependent.

Cancer cell migration is essential for metastasis, during which cancer cells move through the tumor and reach the blood vessels. In vivo, cancer cells are exposed to contact guidance and chemotactic cues. Depending on the strength of such cues, cells will migrate in a random or directed manner. While similar cues may also stimulate cell proliferation, it is not clear whether cell cycle progression affects migration of cancer cells and whether this effect is different in random versus directed migration. In this study, we tested the effect of cell cycle progression on contact guided migration in 2D and 3D environments, in the breast carcinoma cell line, FUCCI-MDA-MB-231. The results were quantified from live cell microscopy images using the open source lineage editing and validation image analysis tools (LEVER). In 2D, cells were placed inside 10 µm-wide microchannels to stimulate contact guidance, with or without an additional chemotactic gradient of the soluble epidermal growth factor. In 3D, contact guidance was modeled by aligned collagen fibers. In both 2D and 3D, contact guidance was cell cycle-dependent, while the addition of the chemo-attractant gradient in 2D increased cell velocity and persistence in directionally migrating cells, regardless of their cell cycle phases. In both 2D and 3D contact guidance, cells in the G1 phase of the cell cycle outperformed cells in the S/G2 phase in terms of migration persistence and instantaneous velocity. These data suggest that in the presence of contact guidance cues in vivo, breast carcinoma cells in the G1 phase of the cell cycle may be more efficient in reaching the neighboring vasculature.