Emerging machine learning approaches to phenotyping cellular motility and morphodynamics.

Cells respond heterogeneously to molecular and environmental perturbations. Phenotypic heterogeneity, wherein multiple phenotypes coexist in the same conditions, presents challenges when interpreting the observed heterogeneity. Advances in live cell microscopy allow researchers to acquire an unprecedented amount of live cell image data at high spatiotemporal resolutions. Phenotyping cellular dynamics, however, is a nontrivial task and requires machine learning (ML) approaches to discern phenotypic heterogeneity from live cell images. In recent years, ML has proven instrumental in biomedical research, allowing scientists to implement sophisticated computation in which computers learn and effectively perform specific analyses with minimal human instruction or intervention. In this review, we discuss how ML has been recently employed in the study of cell motility and morphodynamics to identify phenotypes from computer vision analysis. We focus on new approaches to extract and learn meaningful spatiotemporal features from complex live cell images for cellular and subcellular phenotyping.

Exploring Deep Learning Applications using Ultrasound Single View Cines in Acute Gallbladder Pathologies: Preliminary results.

RATIONALE AND OBJECTIVES: In this preliminary study, we aimed to develop a deep learning model using ultrasound single view cines that distinguishes between imaging of normal gallbladder, non-urgent cholelithiasis, and acute calculous cholecystitis requiring urgent intervention. METHODS: Adult patients presenting to the emergency department between 2017-2022 with right-upper-quadrant pain were screened, and ultrasound single view cines of normal imaging, non-urgent cholelithiasis, and acute cholecystitis were included based on final clinical diagnosis. Longitudinal-view cines were de-identified and gallbladder pathology was annotated for model training. Cines were randomly sorted into training (70%), validation (10%), and testing (20%) sets and divided into 12-frame segments. The deep learning model classified cines as normal (all segments normal), cholelithiasis (normal and non-urgent cholelithiasis segments), and acute cholecystitis (any cholecystitis segment present). RESULTS: A total of 186 patients with 266 cines were identified: Normal imaging (52 patients; 104 cines), non-urgent cholelithiasis (73;88), and acute cholecystitis (61;74). The model achieved a 91% accuracy for Normal vs. Abnormal imaging and an 82% accuracy for Urgent (acute cholecystitis) vs. Non-urgent (cholelithiasis or normal imaging). Furthermore, the model identified abnormal from normal imaging with 100% specificity, with no false positive results. CONCLUSION: Our deep learning model, using only readily obtained single-view cines, exhibited a high degree of accuracy and specificity in discriminating between non-urgent imaging and acute cholecystitis requiring urgent intervention.

Fine-Grained Forecasting of COVID-19 Trends at the County Level in the United States.

The novel coronavirus (COVID-19) pandemic, first identified in Wuhan China in December 2019, has profoundly impacted various aspects of daily life, society, healthcare systems, and global health policies. There have been more than half a billion human infections and more than 6 million deaths globally attributable to COVID-19. Although treatments and vaccines to protect against COVID-19 are now available, people continue being hospitalized and dying due to COVID-19 infections. Real-time surveillance of population-level infections, hospitalizations, and deaths has helped public health officials better allocate healthcare resources and deploy mitigation strategies. However, producing reliable, real-time, short-term disease activity forecasts (one or two weeks into the future) remains a practical challenge. The recent emergence of robust time-series forecasting methodologies based on deep learning approaches has led to clear improvements in multiple research fields. We propose a recurrent neural network model named Fine-Grained Infection Forecast Network (FIGI-Net), which utilizes a stacked bidirectional LSTM structure designed to leverage fine-grained county-level data, to produce daily forecasts of COVID-19 infection trends up to two weeks in advance. We show that FIGI-Net improves existing COVID-19 forecasting approaches and delivers accurate county-level COVID-19 disease estimates. Specifically, FIGI-Net is capable of anticipating upcoming sudden changes in disease trends such as the onset of a new outbreak or the peak of an ongoing outbreak, a skill that multiple existing state-of-the-art models fail to achieve. This improved performance is observed across locations and periods. Our enhanced forecasting methodologies may help protect human populations against future disease outbreaks.

Unsupervised Contour Tracking of Live Cells by Mechanical and Cycle Consistency Losses.

Analyzing the dynamic changes of cellular morphology is important for understanding the various functions and characteristics of live cells, including stem cells and metastatic cancer cells. To this end, we need to track all points on the highly deformable cellular contour in every frame of live cell video. Local shapes and textures on the contour are not evident, and their motions are complex, often with expansion and contraction of local contour features. The prior arts for optical flow or deep point set tracking are unsuited due to the fluidity of cells, and previous deep contour tracking does not consider point correspondence. We propose the first deep learning-based tracking of cellular (or more generally viscoelastic materials) contours with point correspondence by fusing dense representation between two contours with cross attention. Since it is impractical to manually label dense tracking points on the contour, unsupervised learning comprised of the mechanical and cyclical consistency losses is proposed to train our contour tracker. The mechanical loss forcing the points to move perpendicular to the contour effectively helps out. For quantitative evaluation, we labeled sparse tracking points along the contour of live cells from two live cell datasets taken with phase contrast and confocal fluorescence microscopes. Our contour tracker quantitatively outperforms compared methods and produces qualitatively more favorable results. Our code and data are publicly available at https://github.com/JunbongJang/contour-tracking/.

Screening adequacy of unstained thyroid fine needle aspiration samples using a deep learning-based classifier.

Fine needle aspiration (FNA) biopsy of thyroid nodules is a safe, cost-effective, and accurate diagnostic method for detecting thyroid cancer. However, about 10% of initial FNA biopsy samples from patients are non-diagnostic and require repeated FNA, which delays the diagnosis and appropriate care. On-site evaluation of the FNA sample can be performed to filter out non-diagnostic FNA samples. Unfortunately, it involves a time-consuming staining process, and a cytopathologist has to be present at the time of FNA. To bypass the staining process and expert interpretation of FNA specimens at the clinics, we developed a deep learning-based ensemble model termed FNA-Net that allows in situ screening of adequacy of unstained thyroid FNA samples smeared on a glass slide which can decrease the non-diagnostic rate in thyroid FNA. FNA-Net combines two deep learning models, a patch-based whole slide image classifier and Faster R-CNN, to detect follicular clusters with high precision. Then, FNA-Net classifies sample slides to be non-diagnostic if the total number of detected follicular clusters is less than a predetermined threshold. With bootstrapped sampling, FNA-Net achieved a 0.81 F1 score and 0.84 AUC in the precision-recall curve for detecting the non-diagnostic slides whose follicular clusters are less than six. We expect that FNA-Net can dramatically reduce the diagnostic cost associated with FNA biopsy and improve the quality of patient care.

Protocol for live cell image segmentation to profile cellular morphodynamics using MARS-Net.

Quantitative studies of cellular morphodynamics rely on accurate cell segmentation in live cell images. However, fluorescence and phase contrast imaging hinder accurate edge localization. To address this challenge, we developed MARS-Net, a deep learning model integrating ImageNet-pretrained VGG19 encoder and U-Net decoder trained on the datasets from multiple types of microscopy images. Here, we provide the protocol for installing MARS-Net, labeling images, training MARS-Net for edge localization, evaluating the trained models' performance, and performing the quantitative profiling of cellular morphodynamics. For complete details on the use and execution of this protocol, please refer to Jang et al. (2021).

Unravelling psychiatric heterogeneity and predicting suicide attempts in women with trauma-related dissociation using artificial intelligence.

Background: Suicide is a leading cause of death, and rates of attempted suicide have increased during the COVID-19 pandemic. The under-diagnosed psychiatric phenotype of dissociation is associated with elevated suicidal self-injury; however, it has largely been left out of attempts to predict and prevent suicide.Objective: We designed an artificial intelligence approach to identify dissociative patients and predict prior suicide attempts in an unbiased, data-driven manner.Method: Participants were 30 controls and 93 treatment-seeking female patients with posttraumatic stress disorder (PTSD) and various levels of dissociation, including some with the PTSD dissociative subtype and some with dissociative identity disorder (DID).Results: Unsupervised learning models identified patients along a spectrum of dissociation. Moreover, supervised learning models accurately predicted prior suicide attempts with an F1 score up to 0.83. DID had the highest risk of prior suicide attempts, and distinct subtypes of dissociation predicted suicide attempts in PTSD and DID.Conclusions: These findings expand our understanding of the dissociative phenotype and underscore the urgent need to assess for dissociation to identify individuals at high-risk of suicidal self-injury.

Dissociation, feelings of detachment and disruption in one's sense of self and surroundings, is associated with an elevated risk of suicidal self-injury; however, it has largely been left out of attempts to predict and prevent suicide.Using machine learning techniques, we found dissociative identity disorder had the highest risk of prior suicide attempts, and distinct subtypes of dissociation predicted suicide attempts in posttraumatic stress disorder and dissociative identity disorder.These findings underscore the urgent need to assess for dissociation to identify individuals at high-risk of suicidal self-injury.

A deep learning-based segmentation pipeline for profiling cellular morphodynamics using multiple types of live cell microscopy.

MOTIVATION: Quantitative studies of cellular morphodynamics rely on extracting leading-edge velocity time series based on accurate cell segmentation from live cell imaging. However, live cell imaging has numerous challenging issues regarding accurate edge localization. Fluorescence live cell imaging produces noisy and low-contrast images due to phototoxicity and photobleaching. While phase contrast microscopy is gentle to live cells, it suffers from the halo and shade-off artifacts that cannot be handled by conventional segmentation algorithms. Here, we present a deep learning-based pipeline, termed MARS-Net (Multiple-microscopy-type-based Accurate and Robust Segmentation Network), that utilizes transfer learning and data from multiple types of microscopy to localize cell edges with high accuracy, allowing quantitative profiling of cellular morphodynamics. SUMMARY: To accurately segment cell edges and quantify cellular morphodynamics from live-cell imaging data, we developed a deep learning-based pipeline termed MARS-Net (multiple-microscopy-type-based accurate and robust segmentation network). MARS-Net utilizes transfer learning and data from multiple types of microscopy to localize cell edges with high accuracy. For effective training on distinct types of live-cell microscopy, MARS-Net comprises a pretrained VGG19 encoder with U-Net decoder and dropout layers. We trained MARS-Net on movies from phase-contrast, spinning-disk confocal, and total internal reflection fluorescence microscopes. MARS-Net produced more accurate edge localization than the neural network models trained with single-microscopy-type datasets. We expect that MARS-Net can accelerate the studies of cellular morphodynamics by providing accurate pixel-level segmentation of complex live-cell datasets.

A machine learning pipeline revealing heterogeneous responses to drug perturbations on vascular smooth muscle cell spheroid morphology and formation.

Machine learning approaches have shown great promise in biology and medicine discovering hidden information to further understand complex biological and pathological processes. In this study, we developed a deep learning-based machine learning algorithm to meaningfully process image data and facilitate studies in vascular biology and pathology. Vascular injury and atherosclerosis are characterized by neointima formation caused by the aberrant accumulation and proliferation of vascular smooth muscle cells (VSMCs) within the vessel wall. Understanding how to control VSMC behaviors would promote the development of therapeutic targets to treat vascular diseases. However, the response to drug treatments among VSMCs with the same diseased vascular condition is often heterogeneous. Here, to identify the heterogeneous responses of drug treatments, we created an in vitro experimental model system using VSMC spheroids and developed a machine learning-based computational method called HETEROID (heterogeneous spheroid). First, we established a VSMC spheroid model that mimics neointima-like formation and the structure of arteries. Then, to identify the morphological subpopulations of drug-treated VSMC spheroids, we used a machine learning framework that combines deep learning-based spheroid segmentation and morphological clustering analysis. Our machine learning approach successfully showed that FAK, Rac, Rho, and Cdc42 inhibitors differentially affect spheroid morphology, suggesting that multiple drug responses of VSMC spheroid formation exist. Overall, our HETEROID pipeline enables detailed quantitative drug characterization of morphological changes in neointima formation, that occurs in vivo, by single-spheroid analysis.