Heart rate estimation from color video sequences with high SNR.

We propose an absorption intensity heartbeat modulation-averaged shifted histogram (AIHM-ASH) method for estimating human heart rate (HR) using color videos of lip image sequences. When heartbeat occurs, AIHM is generated. Based on the AIHM, HR signals can be demodulated by computing the instantaneous HR modulation depth that presents the relative red blood cell (RBC) concentration from the green channel image of the RGB color video. In addition, the ASH algorithm further suppresses the background tissue and vein signals, and increases the signal-to-noise ratio (SNR). The experimental results for flow phantoms, chicken embryos, and human lips validated the proposed method's optimal estimation conditions and effectiveness, where the accuracy and root mean square error (RMSE) were 99.23% and 0.8 bpm, respectively. The proposed HR estimation method has significant potential to advance health monitoring and disease prevention via conventional color video cameras installed in public places.

Automatic Segmentation and Measurement of Choroid Layer in High Myopia for OCT Imaging Using Deep Learning.

Automatic segmentation and measurement of the choroid layer is useful in studying of related fundus diseases, such as diabetic retinopathy and high myopia. However, most algorithms are not helpful for choroid layer segmentation due to its blurred boundaries and complex gradients. Therefore, this paper aimed to propose a novel choroid segmentation method that combines image enhancement and attention-based dense (AD) U-Net network. The choroidal images obtained from optical coherence tomography (OCT) are pre-enhanced by algorithms that include flattening, filtering, and exponential and linear enhancement to reduce choroid-independent information. Experimental results obtained from 800 OCT B-scans of the choroid layers from both normal eyes and high myopia showed that image enhancement significantly increased the performance of ADU-Net, with an AUC of 99.51% and a DSC of 97.91%. The accuracy of segmentation using the ADU-Net method with image enhancement is superior to that of the existing networks. In addition, we describe some algorithms that can measure automatically choroidal foveal thickness and the volume of adjacent areas. Statistical analyses of the choroidal parameters variation indicated that compared with normal eyes, high myopia has a reduction of 86.3% of the choroidal foveal thickness and 90% of the adjacent volume. It proved that high myopia is likely to cause choroid layer attenuation. These algorithms would have wide application in the diagnosis and precaution of related fundus lesions caused by choroid thinning from high myopia in future studies.

Real-time full-field optical angiography utilizing principal component analysis.

We develop a real-time full-field optical angiography method using principal component analysis (PCA). In our approach, an undersampled laser Doppler method is used to record the raw images. Considering the difference in the signal component contributions, PCA is used to separate the dynamic blood flow and static background signals. The principal advantage of the PCA method is that the choice of a high pixel number can aid in efficiently extracting the blood flow signal with finite frame raw images, which can greatly improve the temporal resolution. Our phantom experimental results validate our choice of the optimal frame number for reconstructing an angiographic image. A vascular occlusion test on a rabbit ear demonstrates that global and simultaneous hemodynamic processes of vessels can be monitored.

In vitro and in vivo brain-targeting chemo-photothermal therapy using graphene oxide conjugated with transferrin for Gliomas.

Current therapies for treating malignant glioma exhibit low therapeutic efficiency because of strong systemic side effects and poor transport across the blood brain barrier (BBB). Herein, we combined targeted chemo-photothermal glioma therapy with a novel multifunctional drug delivery system to overcome these issues. Drug carrier transferrin-conjugated PEGylated nanoscale graphene oxide (TPG) was successfully synthesized and characterized. When loaded on the proposed TPG-based drug delivery (TPGD) system, the anticancer drug doxorubicin could pass through the BBB and improve drug accumulation both in vitro and in vivo. TPGD was found to perform dual functions in chemotherapy and photothermal therapy. Targeted TPGD combination therapy showed higher rates of glioma cell death and prolonged survival of glioma-bearing rats compared with single doxorubicin or PGD therapy. In conclusion, we developed a potential nanoscale drug delivery system for combined therapy of glioma that can effectively decrease side effects and improve therapeutic effects.

Choroidal vascularity index (CVI)-Net-based automatic assessment of diabetic retinopathy severity using CVI in optical coherence tomography images.

A deep learning model called choroidal vascularity index (CVI)-Net is proposed to automatically segment the choroid layer and its vessels in overall optical coherence tomography (OCT) scans. Clinical parameters are then automatically quantified to determine structural and vascular changes in the choroid with the progression of diabetic retinopathy (DR) severity. The study includes 65 eyes consisting of 34 with proliferative DR (PDR), 17 with nonproliferative DR (NPDR), and 14 healthy controls from two OCT systems. On a dataset of 396 OCT B-scan images with manually annotated ground truths, overall Dice coefficients of 96.6 ± 1.5 and 89.1 ± 3.1 are obtained by CVI-Net for the choroid layer and vessel segmentation, respectively. The mean CVI values among the normal, NPDR, and PDR groups are consistent with reported outcomes. Statistical results indicate that CVI shows a significant negative correlation with DR severity level, and this correlation is independent of changes in other physiological parameters.

Deep learning-based signal-independent assessment of macular avascular area on 6×6 mm optical coherence tomography angiogram in diabetic retinopathy: a comparison to instrument-embedded software.

SYNOPSIS: A deep-learning-based macular extrafoveal avascular area (EAA) on a 6×6 mm optical coherence tomography (OCT) angiogram is less dependent on the signal strength and shadow artefacts, providing better diagnostic accuracy for diabetic retinopathy (DR) severity than the commercial software measured extrafoveal vessel density (EVD). AIMS: To compare a deep-learning-based EAA to commercial output EVD in the diagnostic accuracy of determining DR severity levels from 6×6 mm OCT angiography (OCTA) scans. METHODS: The 6×6 mm macular OCTA scans were acquired on one eye of each participant with a spectral-domain OCTA system. After excluding the central 1 mm diameter circle, the EAA on superficial vascular complex was measured with a deep-learning-based algorithm, and the EVD was obtained with commercial software. RESULTS: The study included 34 healthy controls and 118 diabetic patients. EAA and EVD were highly correlated with DR severity (ρ=0.812 and -0.577, respectively, both p<0.001) and visual acuity (r=-0.357 and 0.420, respectively, both p<0.001). EAA had a significantly (p<0.001) higher correlation with DR severity than EVD. With the specificity at 95%, the sensitivities of EAA for differentiating diabetes mellitus (DM), DR and severe DR from control were 80.5%, 92.0% and 100.0%, respectively, significantly higher than those of EVD 11.9% (p=0.001), 13.6% (p<0.001) and 15.8% (p<0.001), respectively. EVD was significantly correlated with signal strength index (SSI) (r=0.607, p<0.001) and shadow area (r=-0.530, p<0.001), but EAA was not (r=-0.044, p=0.805 and r=-0.046, p=0.796, respectively). Adjustment of EVD with SSI and shadow area lowered sensitivities for detection of DM, DR and severe DR. CONCLUSION: Macular EAA on 6×6 mm OCTA measured with a deep learning-based algorithm is less dependent on the signal strength and shadow artefacts, and provides better diagnostic accuracy for DR severity than EVD measured with the instrument-embedded software.

Multi-level optical angiography for photodynamic therapy.

Blood flow imaging is widely applied in photodynamic therapy (PDT) to provide vascular morphological and statistical parameters. This approach relies on the intensity of time-domain signal differences between blood vessels and background tissues; therefore, it often ignores differences within the vasculature and cannot accommodate abundant structural information. This study proposes a multi-level optical angiography (MOA) method for PDT. It can enhance capillaries and image vessels at different levels by measuring the signal frequency shift associated with red blood cell motion. The experimental results regarding the PDT-induced chorioallantoic membrane model showed that the proposed method could not only perform multi-level angiography but also provide more accurate quantitative information regarding various vascular parameters. This MOA method has potential applications in PDT studies.

Spatiotemporal absorption fluctuation imaging based on U-Net.

SIGNIFICANCE: Full-field optical angiography is critical for vascular disease research and clinical diagnosis. Existing methods struggle to improve the temporal and spatial resolutions simultaneously. AIM: Spatiotemporal absorption fluctuation imaging (ST-AFI) is proposed to achieve dynamic blood flow imaging with high spatial and temporal resolutions. APPROACH: ST-AFI is a dynamic optical angiography based on a low-coherence imaging system and U-Net. The system was used to acquire a series of dynamic red blood cell (RBC) signals and static background tissue signals, and U-Net is used to predict optical absorption properties and spatiotemporal fluctuation information. U-Net was generally used in two-dimensional blood flow segmentation as an image processing algorithm for biomedical imaging. In the proposed approach, the network simultaneously analyzes the spatial absorption coefficient differences and the temporal dynamic absorption fluctuation. RESULTS: The spatial resolution of ST-AFI is up to 4.33 µm, and the temporal resolution is up to 0.032 s. In vivo experiments on 2.5-day-old chicken embryos were conducted. The results demonstrate that intermittent RBCs flow in capillaries can be resolved, and the blood vessels without blood flow can be suppressed. CONCLUSIONS: Using ST-AFI to achieve convolutional neural network (CNN)-based dynamic angiography is a novel approach that may be useful for several clinical applications. Owing to their strong feature extraction ability, CNNs exhibit the potential to be expanded to other blood flow imaging methods for the prediction of the spatiotemporal optical properties with improved temporal and spatial resolutions.

Full-field optical multi-functional angiography based on endogenous hemodynamic characteristics.

Blood flow functional imaging is widely applied in biological research to provide vascular morphological and statistical parameters. It relies on the absorption difference and is, therefore, easily affected by complex biological structures, and it cannot accommodate abundant functional information. We propose a full-field multi-functional angiography method to classify arteriovenous vessels and to display flow velocity and vascular diameter distribution simultaneously. Unlike previous methods, an under-sampled laser Doppler acquisition mode is used to record the low-coherence speckle, and multi-functional angiography is achieved by modulating the endogenous hemodynamic characteristics from low-coherence speckle. To demonstrate the combination of classified angiography, blood flow velocity measurement, and vascular diameter measurement realized using our method, we performed experiments on the flow phantom and living chicken embryos and generated multi-functional angiograms. The proposed method can be used as a label-free multi-functional angiography technique in which red blood cells provide a strong endogenous source of naturally hemodynamic characteristics.

Enhanced temporal and spatial resolution in super-resolution covariance imaging algorithm with deconvolution optimization.

Based on the numerical analysis that covariance exhibits superior statistical precision than cumulant and variance, a new SOFI algorithm by calculating the n orders covariance for each pixel is presented with an almost 2n -fold resolution improvement, which can be enhanced to 2n via deconvolution. An optimized deconvolution is also proposed by calculating the (n + 1) order SD associated with each n order covariance pixel, and introducing the results into the deconvolution as a damping factor to suppress noise generation. Moreover, a re-deconvolution of the covariance image with the covariance-equivalent point spread function is used to further increase the final resolution by above 2-fold. Simulated and experimental results show that this algorithm can significantly increase the temporal-spatial resolution of SOFI, meanwhile, preserve the sample's structure. Thus, a resolution of 58 nm is achieved for 20 experimental images, and the corresponding acquisition time is 0.8 seconds.

Automated Segmentation of Retinal Fluid Volumes From Structural and Angiographic Optical Coherence Tomography Using Deep Learning.

Purpose: We proposed a deep convolutional neural network (CNN), named Retinal Fluid Segmentation Network (ReF-Net), to segment retinal fluid in diabetic macular edema (DME) in optical coherence tomography (OCT) volumes. Methods: The 3- × 3-mm OCT scans were acquired on one eye by a 70-kHz OCT commercial AngioVue system (RTVue-XR; Optovue, Inc., Fremont, CA, USA) from 51 participants in a clinical diabetic retinopathy (DR) study (45 with retinal edema and six healthy controls, age 61.3 ± 10.1 (mean ± SD), 33% female, and all DR cases were diagnosed as severe NPDR or PDR). A CNN with U-Net-like architecture was constructed to detect and segment the retinal fluid. Cross-sectional OCT and angiography (OCTA) scans were used for training and testing ReF-Net. The effect of including OCTA data for retinal fluid segmentation was investigated in this study. Volumetric retinal fluid can be constructed using the output of ReF-Net. Area-under-receiver-operating-characteristic-curve, intersection-over-union (IoU), and F1-score were calculated to evaluate the performance of ReF-Net. Results: ReF-Net shows high accuracy (F1 = 0.864 ± 0.084) in retinal fluid segmentation. The performance can be further improved (F1 = 0.892 ± 0.038) by including information from both OCTA and structural OCT. ReF-Net also shows strong robustness to shadow artifacts. Volumetric retinal fluid can provide more comprehensive information than the two-dimensional (2D) area, whether cross-sectional or en face projections. Conclusions: A deep-learning-based method can accurately segment retinal fluid volumetrically on OCT/OCTA scans with strong robustness to shadow artifacts. OCTA data can improve retinal fluid segmentation. Volumetric representations of retinal fluid are superior to 2D projections. Translational Relevance: Using a deep learning method to segment retinal fluid volumetrically has the potential to improve the diagnostic accuracy of diabetic macular edema by OCT systems.

SERS analysis of carcinoma-associated fibroblasts in a tumor microenvironment based on targeted 2D nanosheets.

Carcinoma-associated fibroblasts (CAFs), one of the most important components of a tumor microenvironment (TME), play a significant role in the complex tumorigenesis process. Herein, the evolution of CAFs in TME is elaborately investigated by surface-enhanced Raman spectroscopy (SERS), a molecular fingerprint technique. Two-dimensional (2D) nanocomposites consisting of gold nanoparticles and a supramolecular "PCsheet" self-assembled between 2D nanosheets and oxidized phosphatidylcholine (PC) are fabricated as SERS-active probes to specifically recognize the CD36 receptor on the cytomembrane of the fibroblasts, a reliable landmark of CAF development. The 2D SERS substrates can also illuminate the fingerprint information around the CD36 protein with high detection sensitivity, which helps elucidate the biochemical component transition in the protein mini-domain during carcinoma progression. Visualized data are then supplied by label-free SERS imaging to exploit the distribution of biomolecules on the plasma membrane. In addition, the repressed expression of CD36 in TME is detected in lung metastasis tumor-bearing mice. This study based on the 2D SERS technique opens up an alternative avenue for unveiling carcinoma-associated molecular events.

Photo-induced synthesis of molybdenum oxide quantum dots for surface-enhanced Raman scattering and photothermal therapy.

By means of a simple and photo-induced method, four colors of molybdenum oxide quantum dots (MoOx QDs) have been synthesized, using Mo(CO)6 as the structural guiding agent and molybdenum source. The as-prepared MoOx QDs display diverse optical properties due to the different configurations of oxygen vacancies in various nanostructures. Among them, crystalline molybdenum dioxide (MoO2) with a deep blue color shows the most intense localized surface plasmon resonance effect in the near-infrared (NIR) region. The strong NIR absorption endows MoO2 QDs with a high photothermal conversion efficiency of 66.3%, enabling broad prospects as a photo-responsive nanoagent for photothermal therapy of cancer. Moreover, MoO2 QDs can also serve as a novel semiconductor substrate for ultrasensitive surface-enhanced Raman scattering (SERS) analysis of aromatic molecules, amino acids and antibiotics, with SERS performance comparable to that of noble metal-based substrates. The therapeutic applications of MoO2 QDs open up a new avenue for tumor nanomedicine.

Application of hyperosmotic agent to determine gastric cancer with optical coherence tomography ex vivo in mice.

Noninvasive tumor imaging could lead to the early detection and timely treatment of cancer. Optical coherence tomography (OCT) has been reported as an ideal diagnostic tool for distinguishing tumor tissues from normal tissues based on structural imaging. In this study, the capability of OCT for functional imaging of normal and tumor tissues based on time- and depth-resolved quantification of the permeability of biomolecules through these tissues is investigated. The orthotopic graft model of gastric cancer in nude mice is used, normal and tumor tissues from the gastric wall are imaged, and a diffusion of 20% aqueous solution of glucose in normal stomach tissues and gastric tumor tissues is monitored and quantified as a function of time and tissue depth by an OCT system. Our results show that the permeability coefficient is (0.94+/-0.04)x10(-5) cms in stomach tissues and (5.32+/-0.17)x10(-5) cms in tumor tissues, respectively, and that tumor tissues have a higher permeability coefficient compared to normal tissues in optical coherence tomographic images. From the results, it is found that the accurate and sensitive assessment of the permeability coefficients of normal and tumor tissues offers an effective OCT image method for detection of tumor tissues and clinical diagnosis.

Large-depth-of-field full-field optical angiography.

A large-depth-of-field full-field optical angiography (LD-FFOA) method is developed to expand the depth-of-field (DOF) using a contrast pyramid fusion algorithm (CPFA). The absorption intensity fluctuation modulation effect is utilized to obtain full-field optical angiography (FFOA) images at different focus positions. The CPFA is used to process these FFOA images with different focuses. By selecting high-contrast areas, the CPFA can highlight the characteristics and details of blood vessels to obtain LD-FFOA images. In the optimal case of the proposed method, the DOF for FFOA is more than tripled using 10 differently focused FFOA images. Both the phantom and animal experimental results show that the LD-FFOA resolves FFOA defocusing issues induced by surface and thickness inhomogeneities in biological samples. The proposed method can be potentially applied to practical biological experiments.

Development and validation of a deep learning algorithm for distinguishing the nonperfusion area from signal reduction artifacts on OCT angiography.

The capillary nonperfusion area (NPA) is a key quantifiable biomarker in the evaluation of diabetic retinopathy (DR) using optical coherence tomography angiography (OCTA). However, signal reduction artifacts caused by vitreous floaters, pupil vignetting, or defocus present significant obstacles to accurate quantification. We have developed a convolutional neural network, MEDnet-V2, to distinguish NPA from signal reduction artifacts in 6×6 mm2 OCTA. The network achieves strong specificity and sensitivity for NPA detection across a wide range of DR severity and scan quality.

Potential ability of hematoporphyrin to enhance an optical coherence tomographic image of gastric cancer in vivo in mice.

An ideal diagnostic system for the tumor tissues should be able to detect and define the location of tumor tissues and the early development of malignant diseases. There is great need for enhancement of imaging ability to tumor tissues. Optical coherence tomography (OCT) is used in detection and location of varied tumor tissues. In order to improve the sensitivity and specificity of an OCT image, hematoporphyrin as a new type of contrast agent was used in this study. The orthotopic graft model of gastric cancer in nude mice was used. The image formations of the tumor tissues without and with injection of hematoporphyrin in vivo were obtained by an OCT system at a 1,310 nm central wavelength. The experimental results showed that the tumor tissues accumulated with hematoporphyrin have an ability of light absorption which results in the increase of signal attenuation in the gastric cancer tissues, and that the boundary between the tumor tissues and surrounding normal tissues was perfectly defined owing to the accumulation of hematoporphyrin. From the experimental results, it is found that hematoporphyrin, a photosensitizing agent, could be used as a contrast agent for OCT imaging of tumor tissues, which offer an effective OCT image method for clinical detection and localization of tumor tissues in vivo.

Redox-responsive biodegradable PEGylated nanographene oxide for efficiently chemo-photothermal therapy: a comparative study with non-biodegradable PEGylated nanographene oxide.

Nanographene oxide (NGO) with a non-sheddable poly(ethylene glycol) (PEG) coating has been used for chemo-photothermal therapy. However, the drug release of PEGylated NGO (NGO-PEG) with an amine bond is adversely affected by the diffusion barrier effect of PEG shells. Here, we developed a simple new method for the preparation of biodegradable PEGylated NGO conjugates (NGO-SS-PEG) with cleavable disulfide bonds for rapid drug release and more efficiently chemo-photothermal therapy. The glutathione (GSH)-induced and photothermal-mediated intracellular release of doxorubicin (DOX) from NGO-SS-PEG was studied in A549 cells using confocal laser scanning microscopy and flow cytometry analysis. In vivo cytotoxicity experiments were performed on chemo-photothermal therapy. Furthermore, we presented a comparative study of intracellular drug release and biological efficacy between NGO-SS-PEG/DOX and NGO-PEG/DOX. The results demonstrated that the rapid drug release from the NGO-SS-PEG conjugates with sheddable PEG was triggered upon the stimulus of high GSH levels inside A549 cells. Interesting, the DOX release mediated by the photothermal effect from the NGO-SS-PEG conjugates was found to be more obvious than that for NGO-PEG. Additionally, NGO-SS-PEG showed a higher efficacy than NGO-PEG for anti-tumor therapy compared with NGO-PEG. Thus, NGO-SS-PEG can improve therapeutic efficacy and is an attractive drug nanocarrier.

Study of support vector machine and serum surface-enhanced Raman spectroscopy for noninvasive esophageal cancer detection.

The ability of combining serum surface-enhanced Raman spectroscopy (SERS) with support vector machine (SVM) for improving classification esophageal cancer patients from normal volunteers is investigated. Two groups of serum SERS spectra based on silver nanoparticles (AgNPs) are obtained: one group from patients with pathologically confirmed esophageal cancer (n=30) and the other group from healthy volunteers (n=31). Principal components analysis (PCA), conventional SVM (C-SVM) and conventional SVM combination with PCA (PCA-SVM) methods are implemented to classify the same spectral dataset. Results show that a diagnostic accuracy of 77.0% is acquired for PCA technique, while diagnostic accuracies of 83.6% and 85.2% are obtained for C-SVM and PCA-SVM methods based on radial basis functions (RBF) models. The results prove that RBF SVM models are superior to PCA algorithm in classification serum SERS spectra. The study demonstrates that serum SERS in combination with SVM technique has great potential to provide an effective and accurate diagnostic schema for noninvasive detection of esophageal cancer.

Optical clearing of porcine skin tissue in vitro studied by Raman microspectroscopy.

In present work, we studied the effect of optical clearing on porcine skin in vitro with glycerol by Raman microspectroscopy, denoted as RM, at various time intervals of 0, 15, 30, 45, 60, and 75 min respectively. The results showed that the addition of glycerol significantly improved the depth of RM measurement, and enhanced the recovery of skin tissue Raman spectra that were not overlapped with the glycerol Raman spectra over time. Moreover, it was found that the Raman signals resembled the native spectrum of the molecules in porcine skin with a negligible frequency shift. Furthermore, we evaluated the extent of optical clearing in porcine skin by utilizing various concentrations of 40%, 60%, and 80% glycerol solution. The results demonstrated that with the increase of concentration of glycerol, the optical clearing of porcine skin was much improved.