Heart rate estimation from color video sequences with high SNR.

We propose an absorption intensity heartbeat modulation-averaged shifted histogram (AIHM-ASH) method for estimating human heart rate (HR) using color videos of lip image sequences. When heartbeat occurs, AIHM is generated. Based on the AIHM, HR signals can be demodulated by computing the instantaneous HR modulation depth that presents the relative red blood cell (RBC) concentration from the green channel image of the RGB color video. In addition, the ASH algorithm further suppresses the background tissue and vein signals, and increases the signal-to-noise ratio (SNR). The experimental results for flow phantoms, chicken embryos, and human lips validated the proposed method's optimal estimation conditions and effectiveness, where the accuracy and root mean square error (RMSE) were 99.23% and 0.8 bpm, respectively. The proposed HR estimation method has significant potential to advance health monitoring and disease prevention via conventional color video cameras installed in public places.

Multi-Sensor Medical-Image Fusion Technique Based on Embedding Bilateral Filter in Least Squares and Salient Detection.

A multi-sensor medical-image fusion technique, which integrates useful information from different single-modal images of the same tissue and provides a fused image that is more comprehensive and objective than a single-source image, is becoming an increasingly important technique in clinical diagnosis and treatment planning. The salient information in medical images often visually describes the tissue. To effectively embed salient information in the fused image, a multi-sensor medical image fusion method is proposed based on an embedding bilateral filter in least squares and salient detection via a deformed smoothness constraint. First, source images are decomposed into base and detail layers using a bilateral filter in least squares. Then, the detail layers are treated as superpositions of salient regions and background information; a fusion rule for this layer based on the deformed smoothness constraint and guided filtering was designed to successfully conserve the salient structure and detail information of the source images. A base-layer fusion rule based on modified Laplace energy and local energy is proposed to preserve the energy information of these source images. The experimental results demonstrate that the proposed method outperformed nine state-of-the-art methods in both subjective and objective quality assessments on the Harvard Medical School dataset.

Tri-zone flame spatial structure imaging combined with endogenic polarized scattering.

We propose a multi-mode optical imaging method to retrieve the 2D and 3D spatial structures of the preheating, reaction, and recombination zones of an axisymmetric steady flame. In the proposed method, an infrared camera, a visible light monochromatic camera, and a polarization camera are triggered synchronously to capture 2D flame images, and their corresponding 3D images are reconstructed by combining different projection position images. The results of the experiments conducted indicate that the infrared and visible light images represent the flame preheating and flame reaction zones, respectively. The polarized image can be obtained by computing the degree of linear polarization (DOLP) of raw images captured by the polarization camera. We discover that the highlighted regions in the DOLP images lie outside the infrared and visible light zones; they are insensitive to the flame reaction and have different spatial structures for different fuels. We deduce that the combustion product particles cause endogenic polarized scattering, and that the DOLP images represent the flame recombination zone. This study focuses on the combustion mechanisms, such as the formation of combustion products and quantitative flame composition and structure.

Automatic Segmentation and Measurement of Choroid Layer in High Myopia for OCT Imaging Using Deep Learning.

Automatic segmentation and measurement of the choroid layer is useful in studying of related fundus diseases, such as diabetic retinopathy and high myopia. However, most algorithms are not helpful for choroid layer segmentation due to its blurred boundaries and complex gradients. Therefore, this paper aimed to propose a novel choroid segmentation method that combines image enhancement and attention-based dense (AD) U-Net network. The choroidal images obtained from optical coherence tomography (OCT) are pre-enhanced by algorithms that include flattening, filtering, and exponential and linear enhancement to reduce choroid-independent information. Experimental results obtained from 800 OCT B-scans of the choroid layers from both normal eyes and high myopia showed that image enhancement significantly increased the performance of ADU-Net, with an AUC of 99.51% and a DSC of 97.91%. The accuracy of segmentation using the ADU-Net method with image enhancement is superior to that of the existing networks. In addition, we describe some algorithms that can measure automatically choroidal foveal thickness and the volume of adjacent areas. Statistical analyses of the choroidal parameters variation indicated that compared with normal eyes, high myopia has a reduction of 86.3% of the choroidal foveal thickness and 90% of the adjacent volume. It proved that high myopia is likely to cause choroid layer attenuation. These algorithms would have wide application in the diagnosis and precaution of related fundus lesions caused by choroid thinning from high myopia in future studies.

Full-field functional optical angiography.

We propose full-field functional optical angiography for a live biological specimen based on the absorption intensity fluctuation modulation (AIFM) effect. Because of the difference in absorption between red blood cells (RBCs) and the background tissue under low-coherence light illumination, the moving RBCs, which discontinuously pass though the capillary vessels, generate an AIFM effect. This effect offers high contrast of absorption imaging and sensitivity of low-coherence interference between RBCs and the background tissue. It is used to distinguish the signal of RBCs from that of the background tissue. The averaged and real-time modulation depths are computed to obtain full-field label-free optical angiography and measure blood flow velocity simultaneously. The AIFM method could potentially be applied to study the physiological mechanisms of blood circulation systems of near-transparent live biologic samples.

Laser Doppler projection tomography.

We propose a laser Doppler projection tomography (LDPT) method to obtain visualization of three-dimensional (3D) flowing structures. With LDPT, the flowing signal is extracted by a modified laser Doppler method, and the 3D flowing image is reconstructed by the filtered backprojection algorithm. Phantom experiments are performed to demonstrate that LDPT is able to obtain 3D flowing structure with higher signal-to-noise ratio and spatial resolution. Our experiment results display its potentially useful application to develop 3D label-free optical angiography for the circulation system of live small animal models or microfluidic experiments.

Laser speckle projection tomography.

We propose a laser speckle projection tomography (LSPT) method to obtain a three-dimensional (3D) flowing image. The method combines the advantages of optical projection tomography and laser speckle imaging to reconstruct the visualization of 3D flowing structure. With LSPT, the flowing signal is extracted by laser speckle contrast method and the 3D flowing image is reconstructed by the filtered back-projection algorithm. A phantom experiment is performed to demonstrate that LSPT is able to obtain 3D flowing structure, influenced by concentration and the flow speed.

Choroidal vascularity index (CVI)-Net-based automatic assessment of diabetic retinopathy severity using CVI in optical coherence tomography images.

A deep learning model called choroidal vascularity index (CVI)-Net is proposed to automatically segment the choroid layer and its vessels in overall optical coherence tomography (OCT) scans. Clinical parameters are then automatically quantified to determine structural and vascular changes in the choroid with the progression of diabetic retinopathy (DR) severity. The study includes 65 eyes consisting of 34 with proliferative DR (PDR), 17 with nonproliferative DR (NPDR), and 14 healthy controls from two OCT systems. On a dataset of 396 OCT B-scan images with manually annotated ground truths, overall Dice coefficients of 96.6 ± 1.5 and 89.1 ± 3.1 are obtained by CVI-Net for the choroid layer and vessel segmentation, respectively. The mean CVI values among the normal, NPDR, and PDR groups are consistent with reported outcomes. Statistical results indicate that CVI shows a significant negative correlation with DR severity level, and this correlation is independent of changes in other physiological parameters.

Fast and robust Fourier ptychographic microscopy with position misalignment correction.

Significance: Fourier ptychographic microscopy (FPM) is a new, developing computational imaging technology. It can realize the quantitative phase imaging of a wide field of view and high-resolution (HR) simultaneously by means of multi-angle illumination via a light emitting diode (LED) array, combined with a phase recovery algorithm and the synthetic aperture principle. However, in the FPM reconstruction process, LED position misalignment affects the quality of the reconstructed image, and the reconstruction efficiency of the existing LED position correction algorithms needs to be improved. Aim: This study aims to improve the FPM correction method based on simulated annealing (SA) and proposes a position misalignment correction method (AA-C algorithm) using an improved phase recovery strategy. Approach: The spectrum function update strategy was optimized by adding an adaptive control factor, and the reconstruction efficiency of the algorithm was improved. Results: The experimental results show that the proposed method is effective and robust for position misalignment correction of LED arrays in FPM, and the convergence speed can be improved by 21.2% and 54.9% compared with SC-FPM and PC-FPM, respectively. Conclusions: These results can reduce the requirement of the FPM system for LED array accuracy and improve robustness.

Multi-level optical angiography for photodynamic therapy.

Blood flow imaging is widely applied in photodynamic therapy (PDT) to provide vascular morphological and statistical parameters. This approach relies on the intensity of time-domain signal differences between blood vessels and background tissues; therefore, it often ignores differences within the vasculature and cannot accommodate abundant structural information. This study proposes a multi-level optical angiography (MOA) method for PDT. It can enhance capillaries and image vessels at different levels by measuring the signal frequency shift associated with red blood cell motion. The experimental results regarding the PDT-induced chorioallantoic membrane model showed that the proposed method could not only perform multi-level angiography but also provide more accurate quantitative information regarding various vascular parameters. This MOA method has potential applications in PDT studies.

Spatiotemporal absorption fluctuation imaging based on U-Net.

SIGNIFICANCE: Full-field optical angiography is critical for vascular disease research and clinical diagnosis. Existing methods struggle to improve the temporal and spatial resolutions simultaneously. AIM: Spatiotemporal absorption fluctuation imaging (ST-AFI) is proposed to achieve dynamic blood flow imaging with high spatial and temporal resolutions. APPROACH: ST-AFI is a dynamic optical angiography based on a low-coherence imaging system and U-Net. The system was used to acquire a series of dynamic red blood cell (RBC) signals and static background tissue signals, and U-Net is used to predict optical absorption properties and spatiotemporal fluctuation information. U-Net was generally used in two-dimensional blood flow segmentation as an image processing algorithm for biomedical imaging. In the proposed approach, the network simultaneously analyzes the spatial absorption coefficient differences and the temporal dynamic absorption fluctuation. RESULTS: The spatial resolution of ST-AFI is up to 4.33 µm, and the temporal resolution is up to 0.032 s. In vivo experiments on 2.5-day-old chicken embryos were conducted. The results demonstrate that intermittent RBCs flow in capillaries can be resolved, and the blood vessels without blood flow can be suppressed. CONCLUSIONS: Using ST-AFI to achieve convolutional neural network (CNN)-based dynamic angiography is a novel approach that may be useful for several clinical applications. Owing to their strong feature extraction ability, CNNs exhibit the potential to be expanded to other blood flow imaging methods for the prediction of the spatiotemporal optical properties with improved temporal and spatial resolutions.

Retinal cross-section motion correction in three-dimensional retinal optical coherence tomography.

Motion correction is an important issue in ophthalmic optical coherence tomography (OCT), and can improve the ability of data sets to reflect the physiological structures of tissues and make visualization and subsequent analysis easier. In this study, we present a novel method to correct the cross-sectional motion artifacts in retinal OCT volumes. Motion along the x-direction (fast-scan direction) is corrected through the normalized cross-correlation algorithm, while axial motion compensation is performed using the polynomial fitting method on the inner segment/outer segment (IS/OS) layer segmented by the shortest path faster algorithm (SPFA). The results of volunteers with central serous chorioretinopathy demonstrate that the proposed method effectively corrects motion artifacts in OCT volumes and may have potential application value in the evaluation of ophthalmic diseases such as diabetic retinopathy, glaucoma and age-related macular degeneration.

Enhanced temporal and spatial resolution in super-resolution covariance imaging algorithm with deconvolution optimization.

Based on the numerical analysis that covariance exhibits superior statistical precision than cumulant and variance, a new SOFI algorithm by calculating the n orders covariance for each pixel is presented with an almost 2n -fold resolution improvement, which can be enhanced to 2n via deconvolution. An optimized deconvolution is also proposed by calculating the (n + 1) order SD associated with each n order covariance pixel, and introducing the results into the deconvolution as a damping factor to suppress noise generation. Moreover, a re-deconvolution of the covariance image with the covariance-equivalent point spread function is used to further increase the final resolution by above 2-fold. Simulated and experimental results show that this algorithm can significantly increase the temporal-spatial resolution of SOFI, meanwhile, preserve the sample's structure. Thus, a resolution of 58 nm is achieved for 20 experimental images, and the corresponding acquisition time is 0.8 seconds.

Large-depth-of-field full-field optical angiography.

A large-depth-of-field full-field optical angiography (LD-FFOA) method is developed to expand the depth-of-field (DOF) using a contrast pyramid fusion algorithm (CPFA). The absorption intensity fluctuation modulation effect is utilized to obtain full-field optical angiography (FFOA) images at different focus positions. The CPFA is used to process these FFOA images with different focuses. By selecting high-contrast areas, the CPFA can highlight the characteristics and details of blood vessels to obtain LD-FFOA images. In the optimal case of the proposed method, the DOF for FFOA is more than tripled using 10 differently focused FFOA images. Both the phantom and animal experimental results show that the LD-FFOA resolves FFOA defocusing issues induced by surface and thickness inhomogeneities in biological samples. The proposed method can be potentially applied to practical biological experiments.

In vivo full-field functional optical hemocytometer.

We present an in vivo lab-free full-field functional optical hemocytometer (FFOH) for application to the capillaries of a live biological specimen, based on the absorption intensity fluctuation modulation (AIFM) effect. Because of the absorption difference between the red blood cells (RBCs) and background tissue under low-coherence light illumination, an endogenous instantaneous intensity fluctuation is generated by the AIFM effect when RBCs discontinuously traverse the capillary. The AIFM effect is used to highlight the RBC signal relative to the background tissue by computing the real-time modulation depth. FFOH can simultaneously provide a flow video, the flow velocity and the RBC count. Ourexperimental results can potentially be applied to study the physiological mechanisms of the blood circulation systems of near-transparent live biological samples.

Wide-field absolute transverse blood flow velocity mapping in vessel centerline.

We propose a wide-field absolute transverse blood flow velocity measurement method in vessel centerline based on absorption intensity fluctuation modulation effect. The difference between the light absorption capacities of red blood cells and background tissue under low-coherence illumination is utilized to realize the instantaneous and average wide-field optical angiography images. The absolute fuzzy connection algorithm is used for vessel centerline extraction from the average wide-field optical angiography. The absolute transverse velocity in the vessel centerline is then measured by a cross-correlation analysis according to instantaneous modulation depth signal. The proposed method promises to contribute to the treatment of diseases, such as those related to anemia or thrombosis.

High-temporal-resolution, full-field optical angiography based on short-time modulation depth for vascular occlusion tests.

We developed high-temporal-resolution, full-field optical angiography for use in vascular occlusion tests (VOTs). In the proposed method, undersampled signals are acquired by a high-speed digital camera that separates the dynamic and static speckle signals. The two types of speckle signal are used to calculate the short-time modulation depth (STMD) of each of the camera pixels. STMD is then used to realize high-temporal-resolution, full-field optical angiography. Phantom and biological experiments conducted and demonstrated the feasibility of using our proposed method to perform VOTs and to study the reaction kinetics in microfluidic systems.

In vivo label-free microangiography by laser speckle imaging with intensity fluctuation modulation.

We present the theory of laser speckle imaging improved with intensity fluctuation modulation, where the dynamic speckle pattern can be isolated from its stationary counterpart. A series of in vivo experiments demonstrate the effectiveness of our method in achieving microangiography and monitoring vascular self-recovering process. All results show the convincing performance of our imaging method in both structural and functional imaging of blood flow, which may have potential applications in biological research and disease diagnosis.