Intelligent Image-Activated Cell Sorting.

A fundamental challenge of biology is to understand the vast heterogeneity of cells, particularly how cellular composition, structure, and morphology are linked to cellular physiology. Unfortunately, conventional technologies are limited in uncovering these relations. We present a machine-intelligence technology based on a radically different architecture that realizes real-time image-based intelligent cell sorting at an unprecedented rate. This technology, which we refer to as intelligent image-activated cell sorting, integrates high-throughput cell microscopy, focusing, and sorting on a hybrid software-hardware data-management infrastructure, enabling real-time automated operation for data acquisition, data processing, decision-making, and actuation. We use it to demonstrate real-time sorting of microalgal and blood cells based on intracellular protein localization and cell-cell interaction from large heterogeneous populations for studying photosynthesis and atherothrombosis, respectively. The technology is highly versatile and expected to enable machine-based scientific discovery in biological, pharmaceutical, and medical sciences.

Neuronal sensorimotor integration guiding salt concentration navigation in Caenorhabditis elegans.

Animals navigate their environment by manipulating their movements and adjusting their trajectory which requires a sophisticated integration of sensory data with their current motor status. Here, we utilize the nematode Caenorhabditis elegans to explore the neural mechanisms of processing the sensory and motor information for navigation. We developed a microfluidic device which allows animals to freely move their heads while receiving temporal NaCl stimuli. We found that C. elegans regulates neck bending direction in response to temporal NaCl concentration changes in a way which is consistent with a C. elegans' navigational strategy which regulates traveling direction toward preferred NaCl concentrations. Our analysis also revealed that the activity of a neck motor neuron is significantly correlated with neck bending and activated by the decrease in NaCl concentration in a phase-dependent manner. By combining the analysis of behavioral and neural response to NaCl stimuli and optogenetic perturbation experiments, we revealed that NaCl decrease during ventral bending activates the neck motor neuron which counteracts ipsilateral bending. Simulations further suggest that this phase-dependent response of neck motor neurons can facilitate curving toward preferred salt concentrations.

Intelligent sort-timing prediction for image-activated cell sorting.

Intelligent image-activated cell sorting (iIACS) has enabled high-throughput image-based sorting of single cells with artificial intelligence (AI) algorithms. This AI-on-a-chip technology combines fluorescence microscopy, AI-based image processing, sort-timing prediction, and cell sorting. Sort-timing prediction is particularly essential due to the latency on the order of milliseconds between image acquisition and sort actuation, during which image processing is performed. The long latency amplifies the effects of the fluctuations in the flow speed of cells, leading to fluctuation and uncertainty in the arrival time of cells at the sort point on the microfluidic chip. To compensate for this fluctuation, iIACS measures the flow speed of each cell upstream, predicts the arrival timing of the cell at the sort point, and activates the actuation of the cell sorter appropriately. Here, we propose and demonstrate a machine learning technique to increase the accuracy of the sort-timing prediction that would allow for the improvement of sort event rate, yield, and purity. Specifically, we trained an algorithm to predict the sort timing for morphologically heterogeneous budding yeast cells. The algorithm we developed used cell morphology, position, and flow speed as inputs for prediction and achieved 41.5% lower prediction error compared to the previously employed method based solely on flow speed. As a result, our technique would allow for an increase in the sort event rate of iIACS by a factor of ~2.

Intelligent image-activated sorting of Chlamydomonas reinhardtii by mitochondrial localization.

Organelle positioning in cells is associated with various metabolic functions and signaling in unicellular organisms. Specifically, the microalga Chlamydomonas reinhardtii repositions its mitochondria, depending on the levels of inorganic carbon. Mitochondria are typically randomly distributed in the Chlamydomonas cytoplasm, but relocate toward the cell periphery at low inorganic carbon levels. This mitochondrial relocation is linked with the carbon-concentrating mechanism, but its significance is not yet thoroughly understood. A genotypic understanding of this relocation would require a high-throughput method to isolate rare mutant cells not exhibiting this relocation. However, this task is technically challenging due to the complex intracellular morphological difference between mutant and wild-type cells, rendering conventional non-image-based high-event-rate methods unsuitable. Here, we report our demonstration of intelligent image-activated cell sorting by mitochondrial localization. Specifically, we applied an intelligent image-activated cell sorting system to sort for C. reinhardtii cells displaying no mitochondrial relocation. We trained a convolutional neural network (CNN) to distinguish the cell types based on the complex morphology of their mitochondria. The CNN was employed to perform image-activated sorting for the mutant cell type at 180 events per second, which is 1-2 orders of magnitude faster than automated microscopy with robotic pipetting, resulting in an enhancement of the concentration from 5% to 56.5% corresponding to an enrichment factor of 11.3. These results show the potential of image-activated cell sorting for connecting genotype-phenotype relations for rare-cell populations, which require a high throughput and could lead to a better understanding of metabolic functions in cells.

High-throughput sorting of nanoliter droplets enabled by a sequentially addressable dielectrophoretic array.

Droplet microfluidics has emerged as a powerful tool for a diverse range of biomedical and industrial applications such as single-cell analysis, directed evolution, and metabolic engineering. In these applications, droplet sorting has been effective for isolating small droplets encapsulating molecules, cells, or crystals of interest. Recently, there is an increased interest in extending the applicability of droplet sorting to larger droplets to utilize their size advantage. However, sorting throughputs of large droplets have been limited, hampering their wide adoption. Here, we report our demonstration of high-throughput fluorescence-activated droplet sorting of 1 nL droplets using an upgraded version of the sequentially addressable dielectrophoretic array (SADA), which we reported previously. The SADA is an array of electrodes that are individually and sequentially activated/deactivated according to the speed and position of a droplet passing nearby the array. We upgraded the SADA by increasing the number of driving electrodes constituting the SADA and incorporating a slanted microchannel. By using a ten-electrode SADA with the slanted microchannel, we achieved fluorescence-activated droplet sorting of 1 nL droplets at a record high throughput of 1752 droplets/s, twice as high as the previously reported maximum sorting throughput of 1 nL droplets.

Label-free chemical imaging flow cytometry by high-speed multicolor stimulated Raman scattering.

Combining the strength of flow cytometry with fluorescence imaging and digital image analysis, imaging flow cytometry is a powerful tool in diverse fields including cancer biology, immunology, drug discovery, microbiology, and metabolic engineering. It enables measurements and statistical analyses of chemical, structural, and morphological phenotypes of numerous living cells to provide systematic insights into biological processes. However, its utility is constrained by its requirement of fluorescent labeling for phenotyping. Here we present label-free chemical imaging flow cytometry to overcome the issue. It builds on a pulse pair-resolved wavelength-switchable Stokes laser for the fastest-to-date multicolor stimulated Raman scattering (SRS) microscopy of fast-flowing cells on a 3D acoustic focusing microfluidic chip, enabling an unprecedented throughput of up to ∼140 cells/s. To show its broad utility, we use the SRS imaging flow cytometry with the aid of deep learning to study the metabolic heterogeneity of microalgal cells and perform marker-free cancer detection in blood.

Morphological Indicator for Directed Evolution of Euglena gracilis with a High Heavy Metal Removal Efficiency.

In the past few decades, microalgae-based bioremediation methods for treating heavy metal (HM)-polluted wastewater have attracted much attention by virtue of their environment friendliness, cost efficiency, and sustainability. However, their HM removal efficiency is far from practical use. Directed evolution is expected to be effective for developing microalgae with a much higher HM removal efficiency, but there is no non-invasive or label-free indicator to identify them. Here, we present an intelligent cellular morphological indicator for identifying the HM removal efficiency of Euglena gracilis in a non-invasive and label-free manner. Specifically, we show a strong monotonic correlation (Spearman's ρ = -0.82, P = 2.1 × 10-5) between a morphological meta-feature recognized via our machine learning algorithms and the Cu2+ removal efficiency of 19 E. gracilis clones. Our findings firmly suggest that the morphology of E. gracilis cells can serve as an effective HM removal efficiency indicator and hence have great potential, when combined with a high-throughput image-activated cell sorter, for directed-evolution-based development of E. gracilis with an extremely high HM removal efficiency for practical wastewater treatment worldwide.

Optofluidic time-stretch quantitative phase microscopy.

Innovations in optical microscopy have opened new windows onto scientific research, industrial quality control, and medical practice over the last few decades. One of such innovations is optofluidic time-stretch quantitative phase microscopy - an emerging method for high-throughput quantitative phase imaging that builds on the interference between temporally stretched signal and reference pulses by using dispersive properties of light in both spatial and temporal domains in an interferometric configuration on a microfluidic platform. It achieves the continuous acquisition of both intensity and phase images with a high throughput of more than 10,000 particles or cells per second by overcoming speed limitations that exist in conventional quantitative phase imaging methods. Applications enabled by such capabilities are versatile and include characterization of cancer cells and microalgal cultures. In this paper, we review the principles and applications of optofluidic time-stretch quantitative phase microscopy and discuss its future perspective.

Out-of-plane actuation with a sub-micron initial gap for reconfigurable terahertz micro-electro-mechanical systems metamaterials.

We propose a reconfigurable terahertz (THz) metamaterial that can control the transmittance by out-of-plane actuation with changing the sub-micron gap distance between electrically coupled metamaterial elements. By using the out-of-plane actuation, it was possible to avoid contact between the coupled metamaterial elements across the small initial gap during the adjustment of the gap size. THz spectroscopy was performed during actuation, and the transmission dip frequency was confirmed to be tunable from 0.82 to 0.92 THz for one linear polarization state and from 0.80 to 0.91 THz for the other linear polarization; the two polarizations were orthogonal. The proposed approach will contribute to the development of tunable metamaterials based on structural deformations.

Measurement method for light transmittance of layered metamaterials.

We propose a method to measure light transmittance of layered metamaterials by placing the metamaterials directly on a Si photodiode. Our measurement method enables the direct detection of transmitted light that appears as an evanescent wave in natural materials. Here, we report the transmittance measurements of a typical metamaterial using this method. The metamaterial was composed of Ag/Al(2)O(3) layers and was fabricated by direct evaporation on the Si photodiode. The measured transmittance agrees with the simulated transmittance. Our results confirmed that this measurement method can determine the transmittance properties of metamaterials and that it is applicable to other types of metamaterials.

Is AI essential? Examining the need for deep learning in image-activated sorting of Saccharomyces cerevisiae.

Artificial intelligence (AI) has become a focal point across a multitude of societal sectors, with science not being an exception. Particularly in the life sciences, imaging flow cytometry has increasingly integrated AI for automated management and categorization of extensive cell image data. However, the necessity of AI over traditional classification methods when extending imaging flow cytometry to include cell sorting remains uncertain, primarily due to the time constraints between image acquisition and sorting actuation. AI-enabled image-activated cell sorting (IACS) methods remain substantially limited, even as recent advancements in IACS have found success while largely relying on traditional feature gating strategies. Here we assess the necessity of AI for image classification in IACS by contrasting the performance of feature gating, classical machine learning (ML), and deep learning (DL) with convolutional neural networks (CNNs) in the differentiation of Saccharomyces cerevisiae mutant images. We show that classical ML could only yield a 2.8-fold enhancement in target enrichment capability, albeit at the cost of a 13.7-fold increase in processing time. Conversely, a CNN could offer an 11.0-fold improvement in enrichment capability at an 11.5-fold increase in processing time. We further executed IACS on mixed mutant populations and quantified target strain enrichment via downstream DNA sequencing to substantiate the applicability of DL for the proposed study. Our findings validate the feasibility and value of employing DL in IACS for morphology-based genetic screening of S. cerevisiae, encouraging its incorporation in future advancements of similar technologies.

Virtual-freezing fluorescence imaging flow cytometry with 5-aminolevulinic acid stimulation and antibody labeling for detecting all forms of circulating tumor cells.

Circulating tumor cells (CTCs) are precursors to cancer metastasis. In blood circulation, they take various forms such as single CTCs, CTC clusters, and CTC-leukocyte clusters, all of which have unique characteristics in terms of physiological function and have been a subject of extensive research in the last several years. Unfortunately, conventional methods are limited in accurately analysing the highly heterogeneous nature of CTCs. Here we present an effective strategy for simultaneously analysing all forms of CTCs in blood by virtual-freezing fluorescence imaging (VIFFI) flow cytometry with 5-aminolevulinic acid (5-ALA) stimulation and antibody labeling. VIFFI is an optomechanical imaging method that virtually freezes the motion of fast-flowing cells on an image sensor to enable high-throughput yet sensitive imaging of every single event. 5-ALA stimulates cancer cells to induce the accumulation of protoporphyrin (PpIX), a red fluorescent substance, making it possible to detect all cancer cells even if they show no expression of the epithelial cell adhesion molecule, a typical CTC biomarker. Although PpIX signals are generally weak, VIFFI flow cytometry can detect them by virtue of its high sensitivity. As a proof-of-principle demonstration of the strategy, we applied cancer cells spiked in blood to the strategy to demonstrate image-based detection and accurate classification of single cancer cells, clusters of cancer cells, and clusters of a cancer cell(s) and a leukocyte(s). To show the clinical utility of our method, we used it to evaluate blood samples of four breast cancer patients and four healthy donors and identified EpCAM-positive PpIX-positive cells in one of the patient samples. Our work paves the way toward the determination of cancer prognosis, the guidance and monitoring of treatment, and the design of antitumor strategies for cancer patients.

Deep imaging flow cytometry.

Imaging flow cytometry (IFC) has become a powerful tool for diverse biomedical applications by virtue of its ability to image single cells in a high-throughput manner. However, there remains a challenge posed by the fundamental trade-off between throughput, sensitivity, and spatial resolution. Here we present deep-learning-enhanced imaging flow cytometry (dIFC) that circumvents this trade-off by implementing an image restoration algorithm on a virtual-freezing fluorescence imaging (VIFFI) flow cytometry platform, enabling higher throughput without sacrificing sensitivity and spatial resolution. A key component of dIFC is a high-resolution (HR) image generator that synthesizes "virtual" HR images from the corresponding low-resolution (LR) images acquired with a low-magnification lens (10×/0.4-NA). For IFC, a low-magnification lens is favorable because of reduced image blur of cells flowing at a higher speed, which allows higher throughput. We trained and developed the HR image generator with an architecture containing two generative adversarial networks (GANs). Furthermore, we developed dIFC as a method by combining the trained generator and IFC. We characterized dIFC using Chlamydomonas reinhardtii cell images, fluorescence in situ hybridization (FISH) images of Jurkat cells, and Saccharomyces cerevisiae (budding yeast) cell images, showing high similarities of dIFC images to images obtained with a high-magnification lens (40×/0.95-NA), at a high flow speed of 2 m s-1. We lastly employed dIFC to show enhancements in the accuracy of FISH-spot counting and neck-width measurement of budding yeast cells. These results pave the way for statistical analysis of cells with high-dimensional spatial information.

Intelligent Platelet Morphometry.

Technological advances in image-based platelet analysis or platelet morphometry are critical for a better understanding of the structure and function of platelets in biological research as well as for the development of better clinical strategies in medical practice. Recently, the advent of high-throughput optical imaging and deep learning has boosted platelet morphometry to the next level by providing a new set of capabilities beyond what is achievable with traditional platelet morphometry, shedding light on the unexplored domain of platelet analysis. This Opinion article introduces emerging opportunities in 'intelligent' platelet morphometry, which are expected to pave the way for a new class of diagnostics, pharmacometrics, and therapeutics.

All-dielectric chiral-field-enhanced Raman optical activity.

Raman optical activity (ROA) is effective for studying the conformational structure and behavior of chiral molecules in aqueous solutions and is advantageous over X-ray crystallography and nuclear magnetic resonance spectroscopy in sample preparation and cost performance. However, ROA signals are inherently minuscule; 3-5 orders of magnitude weaker than spontaneous Raman scattering due to the weak chiral light-matter interaction. Localized surface plasmon resonance on metallic nanoparticles has been employed to enhance ROA signals, but suffers from detrimental spectral artifacts due to its photothermal heat generation and inability to efficiently transfer and enhance optical chirality from the far field to the near field. Here we demonstrate all-dielectric chiral-field-enhanced ROA by devising a silicon nanodisk array and exploiting its dark mode to overcome these limitations. Specifically, we use it with pairs of chemical and biological enantiomers to show >100x enhanced chiral light-molecule interaction with negligible artifacts for ROA measurements.

Are droplets really suitable for single-cell analysis? A case study on yeast in droplets.

Single-cell analysis has become one of the main cornerstones of biotechnology, inspiring the advent of various microfluidic compartments for cell cultivation such as microwells, microtrappers, microcapillaries, and droplets. A fundamental assumption for using such microfluidic compartments is that unintended stress or harm to cells derived from the microenvironments is insignificant, which is a crucial condition for carrying out unbiased single-cell studies. Despite the significance of this assumption, simple viability or growth tests have overwhelmingly been the assay of choice for evaluating culture conditions while empirical studies on the sub-lethal effect on cellular functions have been insufficient in many cases. In this work, we assessed the effect of culturing cells in droplets on the cellular function using yeast morphology as an indicator. Quantitative morphological analysis using CalMorph, an image-analysis program, demonstrated that cells cultured in flasks, large droplets, and small droplets significantly differed morphologically. From these differences, we identified that the cell cycle was delayed in droplets during the G1 phase and during the process of bud growth likely due to the checkpoint mechanism and impaired mitochondrial function, respectively. Furthermore, comparing small and large droplets, cells cultured in large droplets were morphologically more similar to those cultured in a flask, highlighting the advantage of increasing the droplet size. These results highlight a potential source of bias in cell analysis using droplets and reinforce the significance of assessing culture conditions of microfluidic cultivation methods for specific study cases.

AI on a chip.

Artificial intelligence (AI) has dramatically changed the landscape of science, industry, defence, and medicine in the last several years. Supported by considerably enhanced computational power and cloud storage, the field of AI has shifted from mostly theoretical studies in the discipline of computer science to diverse real-life applications such as drug design, material discovery, speech recognition, self-driving cars, advertising, finance, medical imaging, and astronomical observation, where AI-produced outcomes have been proven to be comparable or even superior to the performance of human experts. In these applications, what is essentially important for the development of AI is the data needed for machine learning. Despite its prominent importance, the very first process of the AI development, namely data collection and data preparation, is typically the most laborious task and is often a limiting factor of constructing functional AI algorithms. Lab-on-a-chip technology, in particular microfluidics, is a powerful platform for both the construction and implementation of AI in a large-scale, cost-effective, high-throughput, automated, and multiplexed manner, thereby overcoming the above bottleneck. On this platform, high-throughput imaging is a critical tool as it can generate high-content information (e.g., size, shape, structure, composition, interaction) of objects on a large scale. High-throughput imaging can also be paired with sorting and DNA/RNA sequencing to conduct a massive survey of phenotype-genotype relations whose data is too complex to analyze with traditional computational tools, but is analyzable with the power of AI. In addition to its function as a data provider, lab-on-a-chip technology can also be employed to implement the developed AI for accurate identification, characterization, classification, and prediction of objects in mixed, heterogeneous, or unknown samples. In this review article, motivated by the excellent synergy between AI and lab-on-a-chip technology, we outline fundamental elements, recent advances, future challenges, and emerging opportunities of AI with lab-on-a-chip technology or "AI on a chip" for short.

Intelligent image-activated cell sorting 2.0.

The advent of intelligent image-activated cell sorting (iIACS) has enabled high-throughput intelligent image-based sorting of single live cells from heterogeneous populations. iIACS is an on-chip microfluidic technology that builds on a seamless integration of a high-throughput fluorescence microscope, cell focuser, cell sorter, and deep neural network on a hybrid software-hardware data management architecture, thereby providing the combined merits of optical microscopy, fluorescence-activated cell sorting (FACS), and deep learning. Here we report an iIACS machine that far surpasses the state-of-the-art iIACS machine in system performance in order to expand the range of applications and discoveries enabled by the technology. Specifically, it provides a high throughput of ∼2000 events per second and a high sensitivity of ∼50 molecules of equivalent soluble fluorophores (MESFs), both of which are 20 times superior to those achieved in previous reports. This is made possible by employing (i) an image-sensor-based optomechanical flow imaging method known as virtual-freezing fluorescence imaging and (ii) a real-time intelligent image processor on an 8-PC server equipped with 8 multi-core CPUs and GPUs for intelligent decision-making, in order to significantly boost the imaging performance and computational power of the iIACS machine. We characterize the iIACS machine with fluorescent particles and various cell types and show that the performance of the iIACS machine is close to its achievable design specification. Equipped with the improved capabilities, this new generation of the iIACS technology holds promise for diverse applications in immunology, microbiology, stem cell biology, cancer biology, pathology, and synthetic biology.

Raman image-activated cell sorting.

The advent of image-activated cell sorting and imaging-based cell picking has advanced our knowledge and exploitation of biological systems in the last decade. Unfortunately, they generally rely on fluorescent labeling for cellular phenotyping, an indirect measure of the molecular landscape in the cell, which has critical limitations. Here we demonstrate Raman image-activated cell sorting by directly probing chemically specific intracellular molecular vibrations via ultrafast multicolor stimulated Raman scattering (SRS) microscopy for cellular phenotyping. Specifically, the technology enables real-time SRS-image-based sorting of single live cells with a throughput of up to ~100 events per second without the need for fluorescent labeling. To show the broad utility of the technology, we show its applicability to diverse cell types and sizes. The technology is highly versatile and holds promise for numerous applications that are previously difficult or undesirable with fluorescence-based technologies.