Artificial intelligence-enabled quantitative phase imaging methods for life sciences.

Quantitative phase imaging, integrated with artificial intelligence, allows for the rapid and label-free investigation of the physiology and pathology of biological systems. This review presents the principles of various two-dimensional and three-dimensional label-free phase imaging techniques that exploit refractive index as an intrinsic optical imaging contrast. In particular, we discuss artificial intelligence-based analysis methodologies for biomedical studies including image enhancement, segmentation of cellular or subcellular structures, classification of types of biological samples and image translation to furnish subcellular and histochemical information from label-free phase images. We also discuss the advantages and challenges of artificial intelligence-enabled quantitative phase imaging analyses, summarize recent notable applications in the life sciences, and cover the potential of this field for basic and industrial research in the life sciences.

Machine-learning-based diagnosis of thyroid fine-needle aspiration biopsy synergistically by Papanicolaou staining and refractive index distribution.

We developed a machine learning algorithm (MLA) that can classify human thyroid cell clusters by exploiting both Papanicolaou staining and intrinsic refractive index (RI) as correlative imaging contrasts and evaluated the effects of this combination on diagnostic performance. Thyroid fine-needle aspiration biopsy (FNAB) specimens were analyzed using correlative optical diffraction tomography, which can simultaneously measure both, the color brightfield of Papanicolaou staining and three-dimensional RI distribution. The MLA was designed to classify benign and malignant cell clusters using color images, RI images, or both. We included 1535 thyroid cell clusters (benign: malignancy = 1128:407) from 124 patients. Accuracies of MLA classifiers using color images, RI images, and both were 98.0%, 98.0%, and 100%, respectively. As information for classification, the nucleus size was mainly used in the color image; however, detailed morphological information of the nucleus was also used in the RI image. We demonstrate that the present MLA and correlative FNAB imaging approach has the potential for diagnosing thyroid cancer, and complementary information from color and RI images can improve the performance of the MLA.

Label-free histological analysis of retrieved thrombi in acute ischemic stroke using optical diffraction tomography and deep learning.

For patients with acute ischemic stroke, histological quantification of thrombus composition provides evidence for determining appropriate treatment. However, the traditional manual segmentation of stained thrombi is laborious and inconsistent. In this study, we propose a label-free method that combines optical diffraction tomography (ODT) and deep learning (DL) to automate the histological quantification process. The DL model classifies ODT image patches with 95% accuracy, and the collective prediction generates a whole-slide map of red blood cells and fibrin. The resulting whole-slide composition displays an average error of 1.1% and does not experience staining variability, facilitating faster analysis with reduced labor. The present approach will enable rapid and quantitative evaluation of blood clot composition, expediting the preclinical research and diagnosis of cardiovascular diseases.

Rapid species identification of pathogenic bacteria from a minute quantity exploiting three-dimensional quantitative phase imaging and artificial neural network.

The healthcare industry is in dire need of rapid microbial identification techniques for treating microbial infections. Microbial infections are a major healthcare issue worldwide, as these widespread diseases often develop into deadly symptoms. While studies have shown that an early appropriate antibiotic treatment significantly reduces the mortality of an infection, this effective treatment is difficult to practice. The main obstacle to early appropriate antibiotic treatments is the long turnaround time of the routine microbial identification, which includes time-consuming sample growth. Here, we propose a microscopy-based framework that identifies the pathogen from single to few cells. Our framework obtains and exploits the morphology of the limited sample by incorporating three-dimensional quantitative phase imaging and an artificial neural network. We demonstrate the identification of 19 bacterial species that cause bloodstream infections, achieving an accuracy of 82.5% from an individual bacterial cell or cluster. This performance, comparable to that of the gold standard mass spectroscopy under a sufficient amount of sample, underpins the effectiveness of our framework in clinical applications. Furthermore, our accuracy increases with multiple measurements, reaching 99.9% with seven different measurements of cells or clusters. We believe that our framework can serve as a beneficial advisory tool for clinicians during the initial treatment of infections.

Interactions of Nanoparticles with Macrophages and Feasibility of Drug Delivery for Asthma.

Understanding the interaction between nanoparticles and immune cells is essential for the evaluation of nanotoxicity and development of nanomedicines. However, to date, there is little data on the membrane microstructure and biochemical changes in nanoparticle-loaded immune cells. In this study, we observed the microstructure of nanoparticle-loaded macrophages and changes in lipid droplets using holotomography analysis. Quantitatively analyzing the refractive index distribution of nanoparticle-loaded macrophages, we identified the interactions between nanoparticles and macrophages. The results showed that, when nanoparticles were phagocytized by macrophages, the number of lipid droplets and cell volume increased. The volume and mass of the lipid droplets slightly increased, owing to the absorption of nanoparticles. Meanwhile, the number of lipid droplets increased more conspicuously than the other factors. Furthermore, alveolar macrophages are involved in the development and progression of asthma. Studies have shown that macrophages play an essential role in the maintenance of asthma-related inflammation and tissue damage, suggesting that macrophage cells may be applied to asthma target delivery strategies. Therefore, we investigated the target delivery efficiency of gold nanoparticle-loaded macrophages at the biodistribution level, using an ovalbumin-induced asthma mouse model. Normal and severe asthma models were selected to determine the difference in the level of inflammation in the lung. Consequently, macrophages had increased mobility in models of severe asthma, compared to those of normal asthma disease. In this regard, the detection of observable differences in nanoparticle-loaded macrophages may be of primary interest, as an essential endpoint analysis for investigating nanomedical applications and immunotheragnostic strategies.

Label-free multiplexed microtomography of endogenous subcellular dynamics using generalizable deep learning.

Simultaneous imaging of various facets of intact biological systems across multiple spatiotemporal scales is a long-standing goal in biology and medicine, for which progress is hindered by limits of conventional imaging modalities. Here we propose using the refractive index (RI), an intrinsic quantity governing light-matter interaction, as a means for such measurement. We show that major endogenous subcellular structures, which are conventionally accessed via exogenous fluorescence labelling, are encoded in three-dimensional (3D) RI tomograms. We decode this information in a data-driven manner, with a deep learning-based model that infers multiple 3D fluorescence tomograms from RI measurements of the corresponding subcellular targets, thereby achieving multiplexed microtomography. This approach, called RI2FL for refractive index to fluorescence, inherits the advantages of both high-specificity fluorescence imaging and label-free RI imaging. Importantly, full 3D modelling of absolute and unbiased RI improves generalization, such that the approach is applicable to a broad range of new samples without retraining to facilitate immediate applicability. The performance, reliability and scalability of this technology are extensively characterized, and its various applications within single-cell profiling at unprecedented scales (which can generate new experimentally testable hypotheses) are demonstrated.

DeepRegularizer: Rapid Resolution Enhancement of Tomographic Imaging Using Deep Learning.

Optical diffraction tomography measures the three-dimensional refractive index map of a specimen and visualizes biochemical phenomena at the nanoscale in a non-destructive manner. One major drawback of optical diffraction tomography is poor axial resolution due to limited access to the three-dimensional optical transfer function. This missing cone problem has been addressed through regularization algorithms that use a priori information, such as non-negativity and sample smoothness. However, the iterative nature of these algorithms and their parameter dependency make real-time visualization impossible. In this article, we propose and experimentally demonstrate a deep neural network, which we term DeepRegularizer, that rapidly improves the resolution of a three-dimensional refractive index map. Trained with pairs of datasets (a raw refractive index tomogram and a resolution-enhanced refractive index tomogram via the iterative total variation algorithm), the three-dimensional U-net-based convolutional neural network learns a transformation between the two tomogram domains. The feasibility and generalizability of our network are demonstrated using bacterial cells and a human leukaemic cell line, and by validating the model across different samples. DeepRegularizer offers more than an order of magnitude faster regularization performance compared to the conventional iterative method. We envision that the proposed data-driven approach can bypass the high time complexity of various image reconstructions in other imaging modalities.

Label-Free White Blood Cell Classification Using Refractive Index Tomography and Deep Learning.

Objective and Impact Statement. We propose a rapid and accurate blood cell identification method exploiting deep learning and label-free refractive index (RI) tomography. Our computational approach that fully utilizes tomographic information of bone marrow (BM) white blood cell (WBC) enables us to not only classify the blood cells with deep learning but also quantitatively study their morphological and biochemical properties for hematology research. Introduction. Conventional methods for examining blood cells, such as blood smear analysis by medical professionals and fluorescence-activated cell sorting, require significant time, costs, and domain knowledge that could affect test results. While label-free imaging techniques that use a specimen's intrinsic contrast (e.g., multiphoton and Raman microscopy) have been used to characterize blood cells, their imaging procedures and instrumentations are relatively time-consuming and complex. Methods. The RI tomograms of the BM WBCs are acquired via Mach-Zehnder interferometer-based tomographic microscope and classified by a 3D convolutional neural network. We test our deep learning classifier for the four types of bone marrow WBC collected from healthy donors (n=10): monocyte, myelocyte, B lymphocyte, and T lymphocyte. The quantitative parameters of WBC are directly obtained from the tomograms. Results. Our results show >99% accuracy for the binary classification of myeloids and lymphoids and >96% accuracy for the four-type classification of B and T lymphocytes, monocyte, and myelocytes. The feature learning capability of our approach is visualized via an unsupervised dimension reduction technique. Conclusion. We envision that the proposed cell classification framework can be easily integrated into existing blood cell investigation workflows, providing cost-effective and rapid diagnosis for hematologic malignancy.

Calibration-free quantitative phase imaging using data-driven aberration modeling.

We present a data-driven approach to compensate for optical aberrations in calibration-free quantitative phase imaging (QPI). Unlike existing methods that require additional measurements or a background region to correct aberrations, we exploit deep learning techniques to model the physics of aberration in an imaging system. We demonstrate the generation of a single-shot aberration-corrected field image by using a U-net-based deep neural network that learns a translation between an optical field with aberrations and an aberration-corrected field. The high fidelity and stability of our method is demonstrated on 2D and 3D QPI measurements of various confluent eukaryotic cells and microbeads, benchmarking against the conventional method using background subtractions.

Deep learning-based optical field screening for robust optical diffraction tomography.

In tomographic reconstruction, the image quality of the reconstructed images can be significantly degraded by defects in the measured two-dimensional (2D) raw image data. Despite the importance of screening defective 2D images for robust tomographic reconstruction, manual inspection and rule-based automation suffer from low-throughput and insufficient accuracy, respectively. Here, we present deep learning-enabled quality control for holographic data to produce robust and high-throughput optical diffraction tomography (ODT). The key idea is to distil the knowledge of an expert into a deep convolutional neural network. We built an extensive database of optical field images with clean/noisy annotations, and then trained a binary-classification network based upon the data. The trained network outperformed visual inspection by non-expert users and a widely used rule-based algorithm, with >90% test accuracy. Subsequently, we confirmed that the superior screening performance significantly improved the tomogram quality. To further confirm the trained model's performance and generalisability, we evaluated it on unseen biological cell data obtained with a setup that was not used to generate the training dataset. Lastly, we interpreted the trained model using various visualisation techniques that provided the saliency map underlying each model inference. We envision the proposed network would a powerful lightweight module in the tomographic reconstruction pipeline.

Cycle-consistent deep learning approach to coherent noise reduction in optical diffraction tomography.

We present a deep neural network to reduce coherent noise in three-dimensional quantitative phase imaging. Inspired by the cycle generative adversarial network, the denoising network was trained to learn a transform between two image domains: clean and noisy refractive index tomograms. The unique feature of this network, distinct from previous machine learning approaches employed in the optical imaging problem, is that it uses unpaired images. The learned network quantitatively demonstrated its performance and generalization capability through denoising experiments of various samples. We concluded by applying our technique to reduce the temporally changing noise emerging from focal drift in time-lapse imaging of biological cells. This reduction cannot be performed using other optical methods for denoising.

Subsampled phase retrieval for temporal resolution enhancement in lensless on-chip holographic video.

On-chip holographic video is a convenient way to monitor biological samples simultaneously at high spatial resolution and over a wide field-of-view. However, due to the limited readout rate of digital detector arrays, one often faces a tradeoff between the per-frame pixel count and frame rate of the captured video. In this report, we propose a subsampled phase retrieval (SPR) algorithm to overcome the spatial-temporal trade-off in holographic video. Compared to traditional phase retrieval approaches, our SPR algorithm uses over an order of magnitude less pixel measurements while maintaining suitable reconstruction quality. We use an on-chip holographic video setup with pixel sub-sampling to experimentally demonstrate a factor of 5.5 increase in sensor frame rate while monitoring the in vivo movement of Peranema microorganisms.