The Cell Tracking Challenge: 10 years of objective benchmarking.

The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms.

Computational Hyperspectral Microflow Cytometry.

Miniaturized flow cytometry has significant potential for portable applications, such as cell-based diagnostics and the monitoring of therapeutic cell manufacturing, however, the performance of current techniques is often limited by the inability to resolve spectrally-overlapping fluorescence labels. Here, the study presents a computational hyperspectral microflow cytometer (CHC) that enables accurate discrimination of spectrally-overlapping fluorophores labeling single cells. CHC employs a dispersive optical element and an optimization algorithm to detect the full fluorescence emission spectrum from flowing cells, with a high spectral resolution of ≈3 nm in the range from 450 to 650 nm. CHC also includes a dedicated microfluidic device that ensures in-focus imaging through viscoelastic sheathless focusing, thereby enhancing the accuracy and reliability of microflow cytometry analysis. The potential of CHC for analyzing T lymphocyte subpopulations and monitoring changes in cell composition during T cell expansion is demonstrated. Overall, CHC represents a major breakthrough in microflow cytometry and can facilitate its use for immune cell monitoring.

DeepImageJ: A user-friendly environment to run deep learning models in ImageJ.

DeepImageJ is a user-friendly solution that enables the generic use of pre-trained deep learning models for biomedical image analysis in ImageJ. The deepImageJ environment gives access to the largest bioimage repository of pre-trained deep learning models (BioImage Model Zoo). Hence, nonexperts can easily perform common image processing tasks in life-science research with deep learning-based tools including pixel and object classification, instance segmentation, denoising or virtual staining. DeepImageJ is compatible with existing state of the art solutions and it is equipped with utility tools for developers to include new models. Very recently, several training frameworks have adopted the deepImageJ format to deploy their work in one of the most used softwares in the field (ImageJ). Beyond its direct use, we expect deepImageJ to contribute to the broader dissemination and reuse of deep learning models in life sciences applications and bioimage informatics.

Integration of longitudinal deep-radiomics and clinical data improves the prediction of durable benefits to anti-PD-1/PD-L1 immunotherapy in advanced NSCLC patients.

BACKGROUND: Identifying predictive non-invasive biomarkers of immunotherapy response is crucial to avoid premature treatment interruptions or ineffective prolongation. Our aim was to develop a non-invasive biomarker for predicting immunotherapy clinical durable benefit, based on the integration of radiomics and clinical data monitored through early anti-PD-1/PD-L1 monoclonal antibodies treatment in patients with advanced non-small cell lung cancer (NSCLC). METHODS: In this study, 264 patients with pathologically confirmed stage IV NSCLC treated with immunotherapy were retrospectively collected from two institutions. The cohort was randomly divided into a training (n = 221) and an independent test set (n = 43), ensuring the balanced availability of baseline and follow-up data for each patient. Clinical data corresponding to the start of treatment was retrieved from electronic patient records, and blood test variables after the first and third cycles of immunotherapy were also collected. Additionally, traditional radiomics and deep-radiomics features were extracted from the primary tumors of the computed tomography (CT) scans before treatment and during patient follow-up. Random Forest was used to implementing baseline and longitudinal models using clinical and radiomics data separately, and then an ensemble model was built integrating both sources of information. RESULTS: The integration of longitudinal clinical and deep-radiomics data significantly improved clinical durable benefit prediction at 6 and 9 months after treatment in the independent test set, achieving an area under the receiver operating characteristic curve of 0.824 (95% CI: [0.658,0.953]) and 0.753 (95% CI: [0.549,0.931]). The Kaplan-Meier survival analysis showed that, for both endpoints, the signatures significantly stratified high- and low-risk patients (p-value< 0.05) and were significantly correlated with progression-free survival (PFS6 model: C-index 0.723, p-value = 0.004; PFS9 model: C-index 0.685, p-value = 0.030) and overall survival (PFS6 models: C-index 0.768, p-value = 0.002; PFS9 model: C-index 0.736, p-value = 0.023). CONCLUSIONS: Integrating multidimensional and longitudinal data improved clinical durable benefit prediction to immunotherapy treatment of advanced non-small cell lung cancer patients. The selection of effective treatment and the appropriate evaluation of clinical benefit are important for better managing cancer patients with prolonged survival and preserving quality of life.

ABANICCO: A New Color Space for Multi-Label Pixel Classification and Color Analysis.

Classifying pixels according to color, and segmenting the respective areas, are necessary steps in any computer vision task that involves color images. The gap between human color perception, linguistic color terminology, and digital representation are the main challenges for developing methods that properly classify pixels based on color. To address these challenges, we propose a novel method combining geometric analysis, color theory, fuzzy color theory, and multi-label systems for the automatic classification of pixels into 12 conventional color categories, and the subsequent accurate description of each of the detected colors. This method presents a robust, unsupervised, and unbiased strategy for color naming, based on statistics and color theory. The proposed model, "ABANICCO" (AB ANgular Illustrative Classification of COlor), was evaluated through different experiments: its color detection, classification, and naming performance were assessed against the standardized ISCC-NBS color system; its usefulness for image segmentation was tested against state-of-the-art methods. This empirical evaluation provided evidence of ABANICCO's accuracy in color analysis, showing how our proposed model offers a standardized, reliable, and understandable alternative for color naming that is recognizable by both humans and machines. Hence, ABANICCO can serve as a foundation for successfully addressing a myriad of challenges in various areas of computer vision, such as region characterization, histopathology analysis, fire detection, product quality prediction, object description, and hyperspectral imaging.

Reduction of motion effects in myocardial arterial spin labeling.

PURPOSE: To evaluate the accuracy and reproducibility of myocardial blood flow measurements obtained under different breathing strategies and motion correction techniques with arterial spin labeling. METHODS: A prospective cardiac arterial spin labeling study was performed in 12 volunteers at 3 Tesla. Perfusion images were acquired twice under breath-hold, synchronized-breathing, and free-breathing. Motion detection based on the temporal intensity variation of a myocardial voxel, as well as image registration based on pairwise and groupwise approaches, were applied and evaluated in synthetic and in vivo data. A region of interest was drawn over the mean perfusion-weighted image for quantification. Original breath-hold datasets, analyzed with individual regions of interest for each perfusion-weighted image, were considered as reference values. RESULTS: Perfusion measurements in the reference breath-hold datasets were in line with those reported in literature. In original datasets, prior to motion correction, myocardial blood flow quantification was significantly overestimated due to contamination of the myocardial perfusion with the high intensity signal of blood pool. These effects were minimized with motion detection or registration. Synthetic data showed that accuracy of the perfusion measurements was higher with the use of registration, in particular after the pairwise approach, which probed to be more robust to motion. CONCLUSION: Satisfactory results were obtained for the free-breathing strategy after pairwise registration, with higher accuracy and robustness (in synthetic datasets) and higher intrasession reproducibility together with lower myocardial blood flow variability across subjects (in in vivo datasets). Breath-hold and synchronized-breathing after motion correction provided similar results, but these breathing strategies can be difficult to perform by patients.

An objective comparison of cell-tracking algorithms.

We present a combined report on the results of three editions of the Cell Tracking Challenge, an ongoing initiative aimed at promoting the development and objective evaluation of cell segmentation and tracking algorithms. With 21 participating algorithms and a data repository consisting of 13 data sets from various microscopy modalities, the challenge displays today's state-of-the-art methodology in the field. We analyzed the challenge results using performance measures for segmentation and tracking that rank all participating methods. We also analyzed the performance of all of the algorithms in terms of biological measures and practical usability. Although some methods scored high in all technical aspects, none obtained fully correct solutions. We found that methods that either take prior information into account using learning strategies or analyze cells in a global spatiotemporal video context performed better than other methods under the segmentation and tracking scenarios included in the challenge.

Development of an Inverted Epifluorescence Microscope for Long-Term Monitoring of Bacteria in Multiplexed Microfluidic Devices.

Developing more efficient methods for antibiotic susceptibility testing is a pressing issue in novel drug development as bacterial resistance to antibiotics becomes increasingly common. Microfluidic devices have been demonstrated to be powerful platforms that allow researchers to perform multiplexed antibiotic testing. However, the level of multiplexing within microdevices is limited, evidencing the need of creating simple, low-cost and high-resolution imaging systems that can be integrated in antibiotic development pipelines. This paper describes the design and development of an epifluorescence inverted microscope that enables long-term monitoring of bacteria inside multiplexed microfluidic devices. The goal of this work is to provide a simple microscope powerful enough to allow single-cell analysis of bacteria at a reduced cost. This facilitates increasing the number of microscopes that are simultaneously used for antibiotic testing. We prove that the designed system is able to accurately detect fluorescent beads of 100 nm, demonstrating comparable features to high-end commercial microscopes and effectively achieving the resolution required for single-cell analysis of bacteria. The proposed microscope could thus increase the efficiency in antibiotic testing while reducing cost, size, weight, and power requirements, contributing to the successful development of new antibiotic drugs.

Nonlinear Image Registration and Pixel Classification Pipeline for the Study of Tumor Heterogeneity Maps.

We present a novel method to assess the variations in protein expression and spatial heterogeneity of tumor biopsies with application in computational pathology. This was done using different antigen stains for each tissue section and proceeding with a complex image registration followed by a final step of color segmentation to detect the exact location of the proteins of interest. For proper assessment, the registration needs to be highly accurate for the careful study of the antigen patterns. However, accurate registration of histopathological images comes with three main problems: the high amount of artifacts due to the complex biopsy preparation, the size of the images, and the complexity of the local morphology. Our method manages to achieve an accurate registration of the tissue cuts and segmentation of the positive antigen areas.

Full L1-regularized Traction Force Microscopy over whole cells.

BACKGROUND: Traction Force Microscopy (TFM) is a widespread technique to estimate the tractions that cells exert on the surrounding substrate. To recover the tractions, it is necessary to solve an inverse problem, which is ill-posed and needs regularization to make the solution stable. The typical regularization scheme is given by the minimization of a cost functional, which is divided in two terms: the error present in the data or data fidelity term; and the regularization or penalty term. The classical approach is to use zero-order Tikhonov or L2-regularization, which uses the L2-norm for both terms in the cost function. Recently, some studies have demonstrated an improved performance using L1-regularization (L1-norm in the penalty term) related to an increase in the spatial resolution and sensitivity of the recovered traction field. In this manuscript, we present a comparison between the previous two regularization schemes (relying in the L2-norm for the data fidelity term) and the full L1-regularization (using the L1-norm for both terms in the cost function) for synthetic and real data. RESULTS: Our results reveal that L1-regularizations give an improved spatial resolution (more important for full L1-regularization) and a reduction in the background noise with respect to the classical zero-order Tikhonov regularization. In addition, we present an approximation, which makes feasible the recovery of cellular tractions over whole cells on typical full-size microscope images when working in the spatial domain. CONCLUSIONS: The proposed full L1-regularization improves the sensitivity to recover small stress footprints. Moreover, the proposed method has been validated to work on full-field microscopy images of real cells, what certainly demonstrates it is a promising tool for biological applications.

A benchmark for comparison of cell tracking algorithms.

MOTIVATION: Automatic tracking of cells in multidimensional time-lapse fluorescence microscopy is an important task in many biomedical applications. A novel framework for objective evaluation of cell tracking algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2013 Cell Tracking Challenge. In this article, we present the logistics, datasets, methods and results of the challenge and lay down the principles for future uses of this benchmark. RESULTS: The main contributions of the challenge include the creation of a comprehensive video dataset repository and the definition of objective measures for comparison and ranking of the algorithms. With this benchmark, six algorithms covering a variety of segmentation and tracking paradigms have been compared and ranked based on their performance on both synthetic and real datasets. Given the diversity of the datasets, we do not declare a single winner of the challenge. Instead, we present and discuss the results for each individual dataset separately. AVAILABILITY AND IMPLEMENTATION: The challenge Web site (http://www.codesolorzano.com/celltrackingchallenge) provides access to the training and competition datasets, along with the ground truth of the training videos. It also provides access to Windows and Linux executable files of the evaluation software and most of the algorithms that competed in the challenge.

Automated neuromelanin imaging as a diagnostic biomarker for Parkinson's disease.

BACKGROUND: We aimed to analyze the diagnostic accuracy of an automated segmentation and quantification method of the SNc and locus coeruleus (LC) volumes based on neuromelanin (NM)-sensitive MRI (NM-MRI) in patients with idiopathic (iPD) and monogenic (iPD) Parkinson's disease (PD). METHODS: Thirty-six patients (23 idiopathic and 13 monogenic PARKIN or LRRK2 mutations) and 37 age-matched healthy controls underwent 3T-NM-MRI. SNc and LC volumetry were performed using fully automated multi-image atlas segmentation. The diagnostic performance to differentiate PD from controls was measured using the area under the curve (AUC) and likelihood ratios based on receiver operating characteristic (ROC) analyses. RESULTS: We found a significant reduction of SNc and LC volumes in patients, when compared to controls. ROC analysis showed better diagnostic accuracy when using SNc volume than LC volume. Significant differences between ipsilateral and contralateral SNc volumes, in relation to the more clinically affected side, were found in patients with iPD (P = 0.007). Contralateral atrophy in the SNc showed the highest power to discriminate PD subjects from controls (AUC, 0.93-0.94; sensitivity, 91%-92%; specificity, 89%; positive likelihood ratio: 8.4-8.5; negative likelihood ratio: 0.09-0.1 at a single cut-off point). Interval likelihood ratios for contralateral SNc volume improved the diagnostic accuracy of volumetric measurements. CONCLUSION: SNc and LC volumetry based on NM-MRI resulting from the automated segmentation and quantification technique can yield high diagnostic accuracy for differentiating PD from health and might be an unbiased disease biomarker. © 2015 International Parkinson and Movement Disorder Society.

Automatic classification of normal and abnormal cell division using deep learning.

In recent years, there has been a surge in the development of methods for cell segmentation and tracking, with initiatives like the Cell Tracking Challenge driving progress in the field. Most studies focus on regular cell population videos in which cells are segmented and followed, and parental relationships annotated. However, DNA damage induced by genotoxic drugs or ionizing radiation produces additional abnormal events since it leads to behaviors like abnormal cell divisions (resulting in a number of daughters different from two) and cell death. With this in mind, we developed an automatic mitosis classifier to categorize small mitosis image sequences centered around one cell as "Normal" or "Abnormal." These mitosis sequences were extracted from videos of cell populations exposed to varying levels of radiation that affect the cell cycle's development. We explored several deep-learning architectures and found that a network with a ResNet50 backbone and including a Long Short-Term Memory (LSTM) layer produced the best results (mean F1-score: 0.93 ± 0.06). In the future, we plan to integrate this classifier with cell segmentation and tracking to build phylogenetic trees of the population after genomic stress.

Tumor proliferation and invasion are intrinsically coupled and unraveled through tunable spheroid and physics-based models.

Proliferation and invasion are two key drivers of tumor growth that are traditionally considered independent multicellular processes. However, these processes are intrinsically coupled through a maximum carrying capacity, i.e., the maximum spatial cell concentration supported by the tumor volume, total cell count, nutrient access, and mechanical properties of the tissue stroma. We explored this coupling of proliferation and invasion through in vitro and in silico methods where we modulated the mechanical properties of the tumor and the surrounding extracellular matrix. E-cadherin expression and stromal collagen concentration were manipulated in a tunable breast cancer spheroid to determine the overall impacts of these tumor variables on net tumor proliferation and continuum invasion. We integrated these results into a mixed-constitutive formulation to computationally delineate the influences of cellular and extracellular adhesion, stiffness, and mechanical properties of the extracellular matrix on net proliferation and continuum invasion. This framework integrates biological in vitro data into concise computational models of invasion and proliferation to provide more detailed physical insights into the coupling of these key tumor processes and tumor growth. STATEMENT OF SIGNIFICANCE: Tumor growth involves expansion into the collagen-rich stroma through intrinsic coupling of proliferation and invasion within the tumor continuum. These processes are regulated by a maximum carrying capacity that is determined by the total cell count, tumor volume, nutrient access, and mechanical properties of the surrounding stroma. The influences of biomechanical parameters (i.e., stiffness, cell elongation, net proliferation rate and cell-ECM friction) on tumor proliferation or invasion cannot be unraveled using experimental methods alone. By pairing a tunable spheroid system with computational modeling, we delineated the interdependencies of each system parameter on tumor proliferation and continuum invasion, and established a concise computational framework for studying tumor mechanobiology.

MLL1 regulates cytokine-driven cell migration and metastasis.

Cell migration is a critical contributor to metastasis. Cytokine production and its role in cancer cell migration have been traditionally associated with immune cells. We find that the histone methyltransferase Mixed-Lineage Leukemia 1 (MLL1) controls 3D cell migration via cytokines, IL-6, IL-8, and TGF-ß1, secreted by the cancer cells themselves. MLL1, with its scaffold protein Menin, controls actin filament assembly via the IL-6/8/pSTAT3/Arp3 axis and myosin contractility via the TGF-ß1/Gli2/ROCK1/2/pMLC2 axis, which together regulate dynamic protrusion generation and 3D cell migration. MLL1 also regulates cell proliferation via mitosis-based and cell cycle-related pathways. Mice bearing orthotopic MLL1-depleted tumors exhibit decreased lung metastatic burden and longer survival. MLL1 depletion leads to lower metastatic burden even when controlling for the difference in primary tumor growth rates. Combining MLL1-Menin inhibitor with paclitaxel abrogates tumor growth and metastasis, including preexistent metastasis. These results establish MLL1 as a potent regulator of cell migration and highlight the potential of targeting MLL1 in patients with metastatic disease.

Mechanical and Functional Responses in Astrocytes under Alternating Deformation Modes Using Magneto-Active Substrates.

This work introduces NeoMag, a system designed to enhance cell mechanics assays in substrate deformation studies. NeoMag uses multidomain magneto-active materials to mechanically actuate the substrate, transmitting reversible mechanical cues to cells. The system boasts full flexibility in alternating loading substrate deformation modes, seamlessly adapting to both upright and inverted microscopes. The multidomain substrates facilitate mechanobiology assays on 2D and 3D cultures. The integration of the system with nanoindenters allows for precise evaluation of cellular mechanical properties under varying substrate deformation modes. The system is used to study the impact of substrate deformation on astrocytes, simulating mechanical conditions akin to traumatic brain injury and ischemic stroke. The results reveal local heterogeneous changes in astrocyte stiffness, influenced by the orientation of subcellular regions relative to substrate strain. These stiffness variations, exceeding 50% in stiffening and softening, and local deformations significantly alter calcium dynamics. Furthermore, sustained deformations induce actin network reorganization and activate Piezo1 channels, leading to an initial increase followed by a long-term inhibition of calcium events. Conversely, fast and dynamic deformations transiently activate Piezo1 channels and disrupt the actin network, causing long-term cell softening. These findings unveil mechanical and functional alterations in astrocytes during substrate deformation, illustrating the multiple opportunities this technology offers.

Small extracellular vesicles promote stiffness-mediated metastasis.

Tissue stiffness is a critical prognostic factor in breast cancer and is associated with metastatic progression. Here we show an alternative and complementary hypothesis of tumor progression whereby physiological matrix stiffness affects the quantity and protein cargo of small EVs produced by cancer cells, which in turn drive their metastasis. Primary patient breast tissue produces significantly more EVs from stiff tumor tissue than soft tumor adjacent tissue. EVs released by cancer cells on matrices that model human breast tumors (25 kPa; stiff EVs) feature increased adhesion molecule presentation (ITGα 2 ß 1 , ITGα 6 ß 4 , ITGα 6 ß 1 , CD44) compared to EVs from softer normal tissue (0.5 kPa; soft EVs), which facilitates their binding to extracellular matrix (ECM) protein collagen IV, and a 3-fold increase in homing ability to distant organs in mice. In a zebrafish xenograft model, stiff EVs aid cancer cell dissemination through enhanced chemotaxis. Moreover, normal, resident lung fibroblasts treated with stiff and soft EVs change their gene expression profiles to adopt a cancer associated fibroblast (CAF) phenotype. These findings show that EV quantity, cargo, and function depend heavily on the mechanical properties of the extracellular microenvironment.

Small Extracellular Vesicles Promote Stiffness-mediated Metastasis.

Tissue stiffness is a critical prognostic factor in breast cancer and is associated with metastatic progression. Here we show an alternative and complementary hypothesis of tumor progression whereby physiologic matrix stiffness affects the quantity and protein cargo of small extracellular vesicles (EV) produced by cancer cells, which in turn aid cancer cell dissemination. Primary patient breast tissue released by cancer cells on matrices that model human breast tumors (25 kPa; stiff EVs) feature increased adhesion molecule presentation (ITGα2ß1, ITGα6ß4, ITGα6ß1, CD44) compared with EVs from softer normal tissue (0.5 kPa; soft EVs), which facilitates their binding to extracellular matrix proteins including collagen IV, and a 3-fold increase in homing ability to distant organs in mice. In a zebrafish xenograft model, stiff EVs aid cancer cell dissemination. Moreover, normal, resident lung fibroblasts treated with stiff and soft EVs change their gene expression profiles to adopt a cancer-associated fibroblast phenotype. These findings show that EV quantity, cargo, and function depend heavily on the mechanical properties of the extracellular microenvironment. SIGNIFICANCE: Here we show that the quantity, cargo, and function of breast cancer-derived EVs vary with mechanical properties of the extracellular microenvironment.