Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network.

This paper shows that deep learning can eliminate the superimposed twin-image noise in phase images of Gabor holographic setup. This is achieved by the conditional generative adversarial model (C-GAN), trained by input-output pairs of noisy phase images obtained from synthetic Gabor holography and the corresponding quantitative noise-free contrast-phase image obtained by the off-axis digital holography. To train the model, Gabor holograms are generated from digital off-axis holograms with spatial shifting of the real image and twin image in the frequency domain and then adding them with the DC term in the spatial domain. Finally, the digital propagation of the Gabor hologram with Fresnel approximation generates a super-imposed phase image for the C-GAN model input. Two models were trained: a human red blood cell model and an elliptical cancer cell model. Following the training, several quantitative analyses were conducted on the bio-chemical properties and similarity between actual noise-free phase images and the model output. Surprisingly, it is discovered that our model can recover other elliptical cell lines that were not observed during the training. Additionally, some misalignments can also be compensated with the trained model. Particularly, if the reconstruction distance is somewhat incorrect, this model can still retrieve in-focus images.

Secure storage and retrieval schemes for multiple encrypted digital holograms with orthogonal phase encoding multiplexing.

Recent developments in 3D computational optical imaging such as digital holographic microscopy has ushered in a new era for biological research. Therefore, efficient and secure storage and retrieval of digital holograms is a challenging task for future cloud computing services. In this study, we propose a novel scheme to securely store and retrieve multiple encrypted digital holograms by using phase encoding multiplexing. In the proposed schemes, an encrypted hologram can only be accessed using a binary phase mask, which is the key to retrieve the image. In addition, it is possible to independently store, retrieve, and manage the encrypted digital holograms without affecting other groups of the encrypted holograms multiplexed using different sets of binary phase masks, due to the orthogonality properties of the Hadamard matrices with high autocorrelation and low cross-correlation. The desired encrypted holograms may also be searched for, removed, and added independently of other groups of the encrypted holograms. More and more 3D images or digital holograms can be securely and efficiently stored, retrieved, and managed.

Three-dimensional image authentication scheme using sparse phase information in double random phase encoded integral imaging.

In recent years, many studies have focused on authentication of two-dimensional (2D) images using double random phase encryption techniques. However, there has been little research on three-dimensional (3D) imaging systems, such as integral imaging, for 3D image authentication. We propose a 3D image authentication scheme based on a double random phase integral imaging method. All of the 2D elemental images captured through integral imaging are encrypted with a double random phase encoding algorithm and only partial phase information is reserved. All the amplitude and other miscellaneous phase information in the encrypted elemental images is discarded. Nevertheless, we demonstrate that 3D images from integral imaging can be authenticated at different depths using a nonlinear correlation method. The proposed 3D image authentication algorithm can provide enhanced information security because the decrypted 2D elemental images from the sparse phase cannot be easily observed by the naked eye. Additionally, using sparse phase images without any amplitude information can greatly reduce data storage costs and aid in image compression and data transmission.

Lossless and lossy compression of quantitative phase images of red blood cells obtained by digital holographic imaging.

In this paper, we evaluate lossless and lossy compression techniques to compress quantitative phase images of red blood cells (RBCs) obtained by an off-axis digital holographic microscopy (DHM). The RBC phase images are numerically reconstructed from their digital holograms and are stored in 16-bit unsigned integer format. In the case of lossless compression, predictive coding of JPEG lossless (JPEG-LS), JPEG2000, and JP3D are evaluated, and compression ratio (CR) and complexity (compression time) are compared against each other. It turns out that JP2k can outperform other methods by having the best CR. In the lossy case, JP2k and JP3D with different CRs are examined. Because some data is lost in a lossy way, the degradation level is measured by comparing different morphological and biochemical parameters of RBC before and after compression. Morphological parameters are volume, surface area, RBC diameter, sphericity index, and the biochemical cell parameter is mean corpuscular hemoglobin (MCH). Experimental results show that JP2k outperforms JP3D not only in terms of mean square error (MSE) when CR increases, but also in compression time in the lossy compression way. In addition, our compression results with both algorithms demonstrate that with high CR values the three-dimensional profile of RBC can be preserved and morphological and biochemical parameters can still be within the range of reported values.

Automated tracking of temporal displacements of a red blood cell obtained by time-lapse digital holographic microscopy.

Red blood cell (RBC) phase images that are numerically reconstructed by digital holographic microscopy (DHM) can describe the cell structure and dynamics information beneficial for a quantitative analysis of RBCs. However, RBCs investigated with time-lapse DHM undergo temporal displacements when their membranes are loosely attached to the substrate during sedimentation on a glass surface or due to the microscope drift. Therefore, we need to develop a tracking algorithm to localize the same RBC among RBC image sequences and dynamically monitor its biophysical cell parameters; this information is helpful for studies on RBC-related diseases and drug tests. Here, we propose a method, which is a combination of the mean-shift algorithm and Kalman filter, to track a single RBC and demonstrate that the optical path length of the single RBC can be continually extracted from the tracked RBC. The Kalman filter is utilized to predict the target RBC position in the next frame. Then, the mean-shift algorithm starts execution from the predicted location, and a robust kernel, which is adaptive to changes in the RBC scale, shape, and direction, is designed to improve the accuracy of the tracking. Finally, the tracked RBC is segmented and parameters such as the RBC location are extracted to update the Kalman filter and the kernel function for mean-shift tracking; the characteristics of the target RBC are dynamically observed. Experimental results show the feasibility of the proposed algorithm.

Efficient asymmetric image authentication schemes based on photon counting-double random phase encoding and RSA algorithms.

Recently, double random phase encoding (DRPE) has been integrated with the photon counting (PC) imaging technique for the purpose of secure image authentication. In this scheme, the same key should be securely distributed and shared between the sender and receiver, but this is one of the most vexing problems of symmetric cryptosystems. In this study, we propose an efficient asymmetric image authentication scheme by combining the PC-DRPE and RSA algorithms, which solves key management and distribution problems. The retrieved image from the proposed authentication method contains photon-limited encrypted data obtained by means of PC-DRPE. Therefore, the original image can be protected while the retrieved image can be efficiently verified using a statistical nonlinear correlation approach. Experimental results demonstrate the feasibility of our proposed asymmetric image authentication method.

Automated multi-parameter measurement of cardiomyocytes dynamics with digital holographic microscopy.

Compounds tested during drug development may have adverse effects on the heart; therefore all new chemical entities have to undergo extensive preclinical assessment for cardiac liability. Conventional intensity-based imaging techniques are not robust enough to provide detailed information for cell structure and the captured images result in low-contrast, especially to cell with semi-transparent or transparent feature, which would affect the cell analysis. In this paper we show, for the first time, that digital holographic microscopy (DHM) integrated with information processing algorithms automatically provide dynamic quantitative phase profiles of beating cardiomyocytes. We experimentally demonstrate that relevant parameters of cardiomyocytes can be obtained by our automated algorithm based on DHM phase signal analysis and used to characterize the physiological state of resting cardiomyocytes. Our study opens the possibility of automated quantitative analysis of cardiomyocyte dynamics suitable for further drug safety testing and compounds selection as a new paradigm in drug toxicity screens.

Avalanche and bit independence characteristics of double random phase encoding in the Fourier and Fresnel domains.

In this work, we evaluate the avalanche effect and bit independence properties of the double random phase encoding (DRPE) algorithm in the Fourier and Fresnel domains. Experimental results show that DRPE has excellent bit independence characteristics in both the Fourier and Fresnel domains. However, DRPE achieves better avalanche effect results in the Fresnel domain than in the Fourier domain. DRPE gives especially poor avalanche effect results in the Fourier domain when only one bit is changed in the plaintext or in the encryption key. Despite this, DRPE shows satisfactory avalanche effect results in the Fresnel domain when any other number of bits changes in the plaintext or in the encryption key. To the best of our knowledge, this is the first report on the avalanche effect and bit independence behaviors of optical encryption approaches for bit units.

Simultaneous reconstruction of multiple depth images without off-focus points in integral imaging using a graphics processing unit.

The reconstruction of multiple depth images with a ray back-propagation algorithm in three-dimensional (3D) computational integral imaging is computationally burdensome. Further, a reconstructed depth image consists of a focus and an off-focus area. Focus areas are 3D points on the surface of an object that are located at the reconstructed depth, while off-focus areas include 3D points in free-space that do not belong to any object surface in 3D space. Generally, without being removed, the presence of an off-focus area would adversely affect the high-level analysis of a 3D object, including its classification, recognition, and tracking. Here, we use a graphics processing unit (GPU) that supports parallel processing with multiple processors to simultaneously reconstruct multiple depth images using a lookup table containing the shifted values along the x and y directions for each elemental image in a given depth range. Moreover, each 3D point on a depth image can be measured by analyzing its statistical variance with its corresponding samples, which are captured by the two-dimensional (2D) elemental images. These statistical variances can be used to classify depth image pixels as either focus or off-focus points. At this stage, the measurement of focus and off-focus points in multiple depth images is also implemented in parallel on a GPU. Our proposed method is conducted based on the assumption that there is no occlusion of the 3D object during the capture stage of the integral imaging process. Experimental results have demonstrated that this method is capable of removing off-focus points in the reconstructed depth image. The results also showed that using a GPU to remove the off-focus points could greatly improve the overall computational speed compared with using a CPU.

A multispectral photon-counting double random phase encoding scheme for image authentication.

In this paper, we propose a new method for color image-based authentication that combines multispectral photon-counting imaging (MPCI) and double random phase encoding (DRPE) schemes. The sparsely distributed information from MPCI and the stationary white noise signal from DRPE make intruder attacks difficult. In this authentication method, the original multispectral RGB color image is down-sampled into a Bayer image. The three types of color samples (red, green and blue color) in the Bayer image are encrypted with DRPE and the amplitude part of the resulting image is photon counted. The corresponding phase information that has nonzero amplitude after photon counting is then kept for decryption. Experimental results show that the retrieved images from the proposed method do not visually resemble their original counterparts. Nevertheless, the original color image can be efficiently verified with statistical nonlinear correlations. Our experimental results also show that different interpolation algorithms applied to Bayer images result in different verification effects for multispectral RGB color images.

Automated quantitative analysis of 3D morphology and mean corpuscular hemoglobin in human red blood cells stored in different periods.

Quantitative phase (QP) images of red blood cells (RBCs), which are obtained by off-axis digital holographic microscopy, can provide quantitative information about three-dimensional (3D) morphology of human RBCs and the characteristic properties such as mean corpuscular hemoglobin (MCH) and MCH surface density (MCHSD). In this paper, we investigate modifications of the 3D morphology and MCH in RBCs induced by the period of storage time for the purpose of classification of RBCs with different periods of storage by using off-axis digital holographic microscopy. The classification of RBCs based on the duration of storage is highly relevant because a long storage of blood before transfusion may alter the functionality of RBCs and, therefore, cause complications in patients. To analyze any changes in the 3D morphology and MCH of RBCs due to storage, we use data sets from RBC samples stored for 8, 13, 16, 23, 27, 30, 34, 37, 40, 47, and 57 days, respectively. The data sets consist of more than 3,300 blood cells in eleven classes, with more than 300 blood cells per class. The classes indicate the storage period of RBCs and are listed in chronological order. Using the RBCs donated by healthy persons, the off-axis digital holographic microscopy reconstructs several quantitative phase images of RBC samples stored for eleven different periods. We employ marker-controlled watershed transform to remove the background in the RBC quantitative phase images obtained by the off-axis digital holographic microscopy. More than 300 single RBCs are extracted from the segmented quantitative phase images for each class. Such a large number of RBC samples enable us to obtain statistical distributions of the characteristic properties of RBCs after a specific period of storage. Experimental results show that the 3D morphology of the RBCs, in contrast to MCH, is essentially related to the aging of the RBCs.

Equilibrium Optimization Algorithm with Deep Learning Enabled Prostate Cancer Detection on MRI Images.

The enlargement of the prostate gland in the reproductive system of males is considered a form of prostate cancer (PrC). The survival rate is considerably improved with earlier diagnosis of cancer; thus, timely intervention should be administered. In this study, a new automatic approach combining several deep learning (DL) techniques was introduced to detect PrC from MRI and ultrasound (US) images. Furthermore, the presented method describes why a certain decision was made given the input MRI or US images. Many pretrained custom-developed layers were added to the pretrained model and employed in the dataset. The study presents an Equilibrium Optimization Algorithm with Deep Learning-based Prostate Cancer Detection and Classification (EOADL-PCDC) technique on MRIs. The main goal of the EOADL-PCDC method lies in the detection and classification of PrC. To achieve this, the EOADL-PCDC technique applies image preprocessing to improve the image quality. In addition, the EOADL-PCDC technique follows the CapsNet (capsule network) model for the feature extraction model. The EOA is based on hyperparameter tuning used to increase the efficiency of CapsNet. The EOADL-PCDC algorithm makes use of the stacked bidirectional long short-term memory (SBiLSTM) model for prostate cancer classification. A comprehensive set of simulations of the EOADL-PCDC algorithm was tested on the benchmark MRI dataset. The experimental outcome revealed the superior performance of the EOADL-PCDC approach over existing methods in terms of different metrics.

Automated Dual-Mode Cell Monitoring To Simultaneously Explore Calcium Dynamics and Contraction-Relaxation Kinetics within Drug-Treated Stem Cell-Derived Cardiomyocytes.

This manuscript proposes a new dual-mode cell imaging system for studying the relationships between calcium dynamics and the contractility process of cardiomyocytes derived from human-induced pluripotent stem cells. Practically, this dual-mode cell imaging system provides simultaneously both live cell calcium imaging and quantitative phase imaging based on digital holographic microscopy. Specifically, thanks to the development of a robust automated image analysis, simultaneous measurements of both intracellular calcium, a key player of excitation-contraction coupling, and the quantitative phase image-derived dry mass redistribution, reflecting the effective contractility, namely, the contraction and relaxation processes, were achieved. Practically, the relationships between calcium dynamics and the contraction-relaxation kinetics were investigated in particular through the application of two drugs─namely, isoprenaline and E-4031─known to act precisely on calcium dynamics. Specifically, this new dual-mode cell imaging system enabled us to establish that calcium regulation can be divided into two phases, an early phase influencing the occurrence of the relaxation process followed by a late phase, which although not having a significant influence on the relaxation process affects significantly the beat frequency. In combination with cutting-edge technologies allowing the generation of human stem cell-derived cardiomyocytes, this dual-mode cell monitoring approach therefore represents a very promising technique, particularly in the fields of drug discovery and personalized medicine, to identify compounds likely to act more selectively on specific steps that compose the cardiomyocyte contractility.

Automated statistical quantification of three-dimensional morphology and mean corpuscular hemoglobin of multiple red blood cells.

In this paper, we present an automated approach to quantify information about three-dimensional (3D) morphology, hemoglobin content and density of mature red blood cells (RBCs) using off-axis digital holographic microscopy (DHM) and statistical algorithms. The digital hologram of RBCs is recorded by a CCD camera using an off-axis interferometry setup and quantitative phase images of RBCs are obtained by a numerical reconstruction algorithm. In order to remove unnecessary parts and obtain clear targets in the reconstructed phase image with many RBCs, the marker-controlled watershed segmentation algorithm is applied to the phase image. Each RBC in the segmented phase image is three-dimensionally investigated. Characteristic properties such as projected cell surface, average phase, sphericity coefficient, mean corpuscular hemoglobin (MCH) and MCH surface density of each RBC is quantitatively measured. We experimentally demonstrate that joint statistical distributions of the characteristic parameters of RBCs can be obtained by our algorithm and efficiently used as a feature pattern to discriminate between RBC populations that differ in shape and hemoglobin content. Our study opens the possibility of automated RBC quantitative analysis suitable for the rapid classification of a large number of RBCs from an individual blood specimen, which is a fundamental step to develop a diagnostic approach based on DHM.

Photon-counting three-dimensional integral imaging with compression of elemental images.

In this paper, we present lossless compression of elemental images in photon-counting integral imaging. In order to verify the performance of the compression method applied to low light level three-dimensional (3D) integral imaging, we compute the correlation coefficient and peak to mean square error (PSNR) as metrics for 3D scene reconstruction integrity. We show quantitatively via experiments that a considerable compression of the elemental images in photon-counting integral imaging may be achievable without significant loss in the performance in terms of correlation and PSNR metrics. To the best of our knowledge, this is the first report on applying lossless compression algorithms in photon-counting 3D computational integral imaging.

Automatic calculation of tree diameter from stereoscopic image pairs using digital image processing.

Automatic operations play an important role in societies by saving time and improving efficiency. In this paper, we apply the digital image processing method to the field of lumbering to automatically calculate tree diameters in order to reduce culler work and enable a third party to verify tree diameters. To calculate the cross-sectional diameter of a tree, the image was first segmented by the marker-controlled watershed transform algorithm based on the hue saturation intensity (HSI) color model. Then, the tree diameter was obtained by measuring the area of every isolated region in the segmented image. Finally, the true diameter was calculated by multiplying the diameter computed in the image and the scale, which was derived from the baseline and disparity of correspondence points from stereoscopic image pairs captured by rectified configuration cameras.

Automated analysis of human cardiomyocytes dynamics with holographic image-based tracking for cardiotoxicity screening.

This paper proposes a new non-invasive, low-cost, and fully automated platform to quantitatively analyze dynamics of human-induced pluripotent stem cell-derived cardiomyocytes (hiPS-CMs) at the single-cell level by holographic image-based tracking for cardiotoxicity screening. A dense Farneback optical flow method and holographic imaging informatics were combined to characterize the contractile motion of a single CM, which obviates the need for costly equipment to monitor a CM's mechanical beat activity. The reliability of the proposed platform was tested by single-cell motion characterization, synchronization analysis, motion speed measurement of fixed CMs versus live CMs, and noise sensitivity. The applicability of the motion characterization method was tested to determine the pharmacological effects of two cardiovascular drugs, isoprenaline (166 nM) and E-4031 (500 µM). The experiments were done using single CMs and multiple cells, and the results were compared to control conditions. Cardiomyocytes responded to isoprenaline by increasing the action potential (AP) speed and shortening the resting period, thus increasing the beat frequency. In the presence of E-4031, the AP speed was decreased, and the resting period was prolonged, thus decreasing the beat frequency. The findings offer insights into single hiPS-CMs' contractile motion and a deep understanding of their kinetics at the single-cell level for cardiotoxicity screening.

Deep Learning-Based Phenotypic Assessment of Red Cell Storage Lesions for Safe Transfusions.

This study presents a novel approach to automatically perform instant phenotypic assessment of red blood cell (RBC) storage lesion in phase images obtained by digital holographic microscopy. The proposed model combines a generative adversarial network (GAN) with marker-controlled watershed segmentation scheme. The GAN model performed RBC segmentations and classifications to develop ageing markers, and the watershed segmentation was used to completely separate overlapping RBCs. Our approach achieved good segmentation and classification accuracy with a Dice's coefficient of 0.94 at a high throughput rate of about 152 cells per second. These results were compared with other deep neural network architectures. Moreover, our image-based deep learning models recognized the morphological changes that occur in RBCs during storage. Our deep learning-based classification results were in good agreement with previous findings on the changes in RBC markers (dominant shapes) affected by storage duration. We believe that our image-based deep learning models can be useful for automated assessment of RBC quality, storage lesions for safe transfusions, and diagnosis of RBC-related diseases.

Automated phase unwrapping in digital holography with deep learning.

Digital holography can provide quantitative phase images related to the morphology and content of biological samples. After the numerical image reconstruction, the phase values are limited between -π and π; thus, discontinuity may occur due to the modulo 2π operation. We propose a new deep learning model that can automatically reconstruct unwrapped focused-phase images by combining digital holography and a Pix2Pix generative adversarial network (GAN) for image-to-image translation. Compared with numerical phase unwrapping methods, the proposed GAN model overcomes the difficulty of accurate phase unwrapping due to abrupt phase changes and can perform phase unwrapping at a twice faster rate. We show that the proposed model can generalize well to different types of cell images and has high performance compared to recent U-net models. The proposed method can be useful in observing the morphology and movement of biological cells in real-time applications.

High-throughput label-free cell detection and counting from diffraction patterns with deep fully convolutional neural networks.

SIGNIFICANCE: Digital holographic microscopy (DHM) is a promising technique for the study of semitransparent biological specimen such as red blood cells (RBCs). It is important and meaningful to detect and count biological cells at the single cell level in biomedical images for biomarker discovery and disease diagnostics. However, the biological cell analysis based on phase information of images is inefficient due to the complexity of numerical phase reconstruction algorithm applied to raw hologram images. New cell study methods based on diffraction pattern directly are desirable. AIM: Deep fully convolutional networks (FCNs) were developed on raw hologram images directly for high-throughput label-free cell detection and counting to assist the biological cell analysis in the future. APPROACH: The raw diffraction patterns of RBCs were recorded by use of DHM. Ground-truth mask images were labeled based on phase images reconstructed from RBC holograms using numerical reconstruction algorithm. A deep FCN, which is UNet, was trained on the diffraction pattern images to achieve the label-free cell detection and counting. RESULTS: The implemented deep FCNs provide a promising way to high-throughput and label-free counting of RBCs with a counting accuracy of 99% at a throughput rate of greater than 288 cells per second and 200 µm × 200 µm field of view at the single cell level. Compared to convolutional neural networks, the FCNs can get much better results in terms of accuracy and throughput rate. CONCLUSIONS: High-throughput label-free cell detection and counting were successfully achieved from diffraction patterns with deep FCNs. It is a promising approach for biological specimen analysis based on raw hologram directly.