Revealing speckle obscured living human retinal cells with artificial intelligence assisted adaptive optics optical coherence tomography.

BACKGROUND: In vivo imaging of the human retina using adaptive optics optical coherence tomography (AO-OCT) has transformed medical imaging by enabling visualization of 3D retinal structures at cellular-scale resolution, including the retinal pigment epithelial (RPE) cells, which are essential for maintaining visual function. However, because noise inherent to the imaging process (e.g., speckle) makes it difficult to visualize RPE cells from a single volume acquisition, a large number of 3D volumes are typically averaged to improve contrast, substantially increasing the acquisition duration and reducing the overall imaging throughput. METHODS: Here, we introduce parallel discriminator generative adversarial network (P-GAN), an artificial intelligence (AI) method designed to recover speckle-obscured cellular features from a single AO-OCT volume, circumventing the need for acquiring a large number of volumes for averaging. The combination of two parallel discriminators in P-GAN provides additional feedback to the generator to more faithfully recover both local and global cellular structures. Imaging data from 8 eyes of 7 participants were used in this study. RESULTS: We show that P-GAN not only improves RPE cell contrast by 3.5-fold, but also improves the end-to-end time required to visualize RPE cells by 99-fold, thereby enabling large-scale imaging of cells in the living human eye. RPE cell spacing measured across a large set of AI recovered images from 3 participants were in agreement with expected normative ranges. CONCLUSIONS: The results demonstrate the potential of AI assisted imaging in overcoming a key limitation of RPE imaging and making it more accessible in a routine clinical setting.

The retinal pigment epithelium (RPE) is a single layer of cells within the eye that is crucial for vision. These cells are unhealthy in many eye diseases, and this can result in vision problems, including blindness. Imaging RPE cells in living human eyes is time consuming and difficult with the current technology. Our method substantially speeds up the process of RPE imaging by incorporating artificial intelligence. This enables larger areas of the eye to be imaged more efficiently. Our method could potentially be used in the future during routine eye tests. This could lead to earlier detection and treatment of eye diseases, and the prevention of some causes of blindness.

Visualization of erythrocyte stasis in the living human eye in health and disease.

Blood cells trapped in stasis have been reported within the microcirculation, but their relevance to health and disease has not been established. In this study, we introduce an in vivo imaging approach that reveals the presence of a previously-unknown pool of erythrocytes in stasis, located within capillary segments of the CNS, and present in 100% of subjects imaged. These results provide a key insight that blood cells pause as they travel through the choroidal microvasculature, a vascular structure that boasts the highest blood flow of any tissue in the body. Demonstration of clinical utility using deep learning reveals that erythrocyte stasis is altered in glaucoma, indicating the possibility of more widespread changes in choroidal microvascular than previously realized. The ability to monitor the choroidal microvasculature at the single cell level may lead to novel strategies for tracking microvascular health in glaucoma, age-related macular degeneration, and other neurodegenerative diseases.

Widespread subclinical cellular changes revealed across a neural-epithelial-vascular complex in choroideremia using adaptive optics.

Choroideremia is an X-linked, blinding retinal degeneration with progressive loss of photoreceptors, retinal pigment epithelial (RPE) cells, and choriocapillaris. To study the extent to which these layers are disrupted in affected males and female carriers, we performed multimodal adaptive optics imaging to better visualize the in vivo pathogenesis of choroideremia in the living human eye. We demonstrate the presence of subclinical, widespread enlarged RPE cells present in all subjects imaged. In the fovea, the last area to be affected in choroideremia, we found greater disruption to the RPE than to either the photoreceptor or choriocapillaris layers. The unexpected finding of patches of photoreceptors that were fluorescently-labeled, but structurally and functionally normal, suggests that the RPE blood barrier function may be altered in choroideremia. Finally, we introduce a strategy for detecting enlarged cells using conventional ophthalmic imaging instrumentation. These findings establish that there is subclinical polymegathism of RPE cells in choroideremia.

Visualizing retinal cells with adaptive optics imaging modalities using a translational imaging framework.

Adaptive optics reflectance-based retinal imaging has proved a valuable tool for the noninvasive visualization of cells in the living human retina. Many subcellular features that remain at or below the resolution limit of current in vivo techniques may be more easily visualized with the same modalities in an ex vivo setting. While most microscopy techniques provide significantly higher resolution, enabling the visualization of fine cellular detail in ex vivo retinal samples, they do not replicate the reflectance-based imaging modalities of in vivo retinal imaging. Here, we introduce a strategy for imaging ex vivo samples using the same imaging modalities as those used for in vivo retinal imaging, but with increased resolution. We also demonstrate the ability of this approach to perform protein-specific fluorescence imaging and reflectance imaging simultaneously, enabling the visualization of nearly transparent layers of the retina and the classification of cone photoreceptor types.

Photoreceptor and Retinal Pigment Epithelium Relationships in Eyes With Vitelliform Macular Dystrophy Revealed by Multimodal Adaptive Optics Imaging.

Purpose: To assess the structure of cone photoreceptors and retinal pigment epithelial (RPE) cells in vitelliform macular dystrophy (VMD) arising from various genetic etiologies. Methods: Multimodal adaptive optics (AO) imaging was performed in 11 patients with VMD using a custom-assembled instrument. Non-confocal split detection and AO-enhanced indocyanine green were used to visualize the cone photoreceptor and RPE mosaics, respectively. Cone and RPE densities were measured and compared across BEST1-, PRPH2-, IMPG1-, and IMPG2-related VMD. Results: Within macular lesions associated with VMD, both cone and RPE densities were reduced below normal, to 37% of normal cone density (eccentricity 0.2 mm) and to 8.4% of normal RPE density (eccentricity 0.5 mm). Outside of lesions, cone and RPE densities were slightly reduced (both to 92% of normal values), but with high degree of variability in the individual measurements. Comparison of juxtalesional cone and RPE measurements (<1 mm from the lesion edge) revealed significant differences in RPE density across the four genes (P < 0.05). Overall, cones were affected to a greater extent than RPE in patients with IMPG1 and IMPG2 pathogenic variants, but RPE was affected more than cones in BEST1 and PRPH2 VMD. This trend was observed even in contralateral eyes from a subset of five patients who presented with macular lesions in only one eye. Conclusions: Assessment of cones and RPE in retinal locations outside of the macular lesions reveals a pattern of cone and RPE disruption that appears to be gene dependent in VMD. These findings provide insight into the cellular pathogenesis of disease in VMD.

In-vivo sub-diffraction adaptive optics imaging of photoreceptors in the human eye with annular pupil illumination and sub-Airy detection.

Adaptive optics scanning light ophthalmoscopy (AOSLO) allows non-invasive visualization of the living human eye at the microscopic scale; but even with correction of the ocular wavefront aberrations over a large pupil, the smallest cells in the photoreceptor mosaic cannot always be resolved. Here, we synergistically combine annular pupil illumination with sub-Airy disk confocal detection to demonstrate a 33% improvement in transverse resolution (from 2.36 to 1.58 µm) and a 13% axial resolution enhancement (from 37 to 32 µm), an important step towards the study of the complete photoreceptor mosaic in heath and disease. Interestingly, annular pupil illumination also enhanced the visualization of the photoreceptor mosaic in non-confocal detection schemes such as split detection AOSLO, providing a strategy for enhanced multimodal imaging of the cone and rod photoreceptor mosaic.

Real-time pixelwise phasor analysis for video-rate two-photon fluorescence lifetime imaging microscopy.

Two-photon fluorescence lifetime imaging microscopy (FLIM) is a widely used technique in biomedical optical imaging. Presently, many two-photon time-domain FLIM setups are limited by long acquisition and postprocessing times that decrease data throughput and inhibit the ability to image fast sub-second processes. Here, we present a versatile two-photon FLIM setup capable of video-rate (up to 25 fps) imaging with graphics processing unit (GPU)-accelerated pixelwise phasor analysis displayed and saved simultaneously with acquisition. The system uses an analog output photomultiplier tube in conjunction with 12-bit digitization at 3.2 GHz to overcome the limited maximum acceptable photon rate associated with the photon counting electronics in many FLIM systems. This allows for higher throughput FLIM acquisition and analysis, and additionally enables the user to assess sample fluorescence lifetime in real-time. We further explore the capabilities of the system to examine the kinetics of Rhodamine B uptake by human breast cancer cells and characterize the effect of pixel dwell time on the reduced nicotinamide adenine dinucleotide and reduced nicotinamide adenine dinucleotide phosphate (NAD(P)H) autofluorescence lifetime estimation accuracy.

Integrating adaptive optics-SLO and OCT for multimodal visualization of the human retinal pigment epithelial mosaic.

In vivo imaging of human retinal pigment epithelial (RPE) cells has been demonstrated through multiple adaptive optics (AO)-based modalities. However, whether consistent and complete information regarding the cellular structure of the RPE mosaic is obtained across these modalities remains uncertain due to limited comparisons performed in the same eye. Here, an imaging platform combining multimodal AO-scanning light ophthalmoscopy (AO-SLO) with AO-optical coherence tomography (AO-OCT) is developed to make a side-by-side comparison of the same RPE cells imaged across four modalities: AO-darkfield, AO-enhanced indocyanine green (AO-ICG), AO-infrared autofluorescence (AO-IRAF), and AO-OCT. Co-registered images were acquired in five subjects, including one patient with choroideremia. Multimodal imaging provided multiple perspectives of the RPE mosaic that were used to explore variations in RPE cell contrast in a subject-, location-, and even cell-dependent manner. Estimated cell-to-cell spacing and density were found to be consistent both across modalities and with normative data. Multimodal images from a patient with choroideremia illustrate the benefit of using multiple modalities to infer the cellular structure of the RPE mosaic in an affected eye, in which disruptions to the RPE mosaic may locally alter the signal strength, visibility of individual RPE cells, or even source of contrast in unpredictable ways.

High-speed label-free two-photon fluorescence microscopy of metabolic transients during neuronal activity.

The brain is an especially active metabolic system, requiring a large supply of energy following neuronal activation. However, direct observation of cellular metabolic dynamics associated with neuronal activation is challenging with currently available imaging tools. In this study, an optical imaging approach combining imaging of calcium transients and the metabolic co-enzyme nicotinamide adenine dinucleotide (phosphate) (NAD(P)H) is utilized to track the metabolic dynamics in hippocampal neuron cultures. Results show distinct cellular components for the NAD(P)H response following neuronal activity, where notable differences in the NAD(P)H dynamics between neurons and astrocytes can be directly observed. Additionally, tracking of these responses across a large field of view is demonstrated for metabolic profiling of neuronal activation. Observation of neuronal dynamics using these methods allows for closer examination of the complex metabolic machinery of the brain, and may lead to a better understanding of the cellular metabolism of neuronal activation.

Video-rate multimodal multiphoton imaging and three-dimensional characterization of cellular dynamics in wounded skin.

To date, numerous studies have been performed to elucidate the complex cellular dynamics in skin diseases, but few have attempted to characterize these cellular events under conditions similar to the native environment. To address this challenge, a three-dimensional (3D) multimodal analysis platform was developed for characterizing in vivo cellular dynamics in skin, which was then utilized to process in vivo wound healing data to demonstrate its applicability. Special attention is focused on in vivo biological parameters that are difficult to study with ex vivo analysis, including 3D cell tracking and techniques to connect biological information obtained from different imaging modalities. These results here open new possibilities for evaluating 3D cellular dynamics in vivo, and can potentially provide new tools for characterizing the skin microenvironment and pathologies in the future.

Tracking metabolic dynamics of apoptosis with high-speed two-photon fluorescence lifetime imaging microscopy.

Programmed cell death, or apoptosis, is an essential process in development and homeostasis, and disruptions in associated pathways are responsible for a wide variety of diseases such as cancer, developmental abnormalities, and Alzheimer's disease. On the other hand, cell death, in many cases, is the desired outcome of therapeutic treatments targeting diseases such as cancer. Recently, metabolic imaging based on two-photon fluorescence microscopy has been developed and shown to be highly sensitive to certain cell death processes, most notably apoptosis, thus having the potential as an advanced label-free screening tool. However, the typically low acquisition rates of this imaging technique have resulted in a limited throughput approach, allowing only a small population of cells to be tracked at well-separated time points. To address this limitation, a high-speed two-photon fluorescence lifetime imaging microscopy (2P-FLIM) platform capable of video-rate imaging is applied to study and further characterize the metabolic dynamics associated with cell death. Building upon previous work demonstrating the capabilities of this system, this microscope is utilized to study rapid metabolic changes during cell death induction, such as dose-dependency of metabolic response, response in invasive vs. noninvasive cancer cells, and response in an apoptosis-resistant cell line, which is further shown to undergo autophagy in response to toxic stimuli. Results from these experiments show that the early apoptosis-related metabolic dynamics are strongly correlated with important cellular parameters including responsiveness to apoptosis-inducing stimuli. The high speed and sensitivity of the presented imaging approach enables new investigations into this highly dynamic and complex process.

Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy.

Deep neural networks have been used to map multi-modal, multi-photon microscopy measurements of a label-free tissue sample to its corresponding histologically stained brightfield microscope colour image. It is shown that the extra structural and functional contrasts provided by using two source modes, namely two-photon excitation microscopy and fluorescence lifetime imaging, result in a more faithful reconstruction of the target haematoxylin and eosin stained mode. This modal mapping procedure can aid histopathologists, since it provides access to unobserved imaging modalities, and translates the high-dimensional numerical data generated by multi-modal, multi-photon microscopy into traditionally accepted visual forms. Furthermore, by combining the strengths of traditional chemical staining and modern multi-photon microscopy techniques, modal mapping enables label-free, non-invasive studies of in vivo tissue samples or intravital microscopic imaging inside living animals. The results show that modal co-registration and the inclusion of spatial variations increase the visual accuracy of the mapped results.

Intravital imaging by simultaneous label-free autofluorescence-multiharmonic microscopy.

Intravital microscopy (IVM) emerged and matured as a powerful tool for elucidating pathways in biological processes. Although label-free multiphoton IVM is attractive for its non-perturbative nature, its wide application has been hindered, mostly due to the limited contrast of each imaging modality and the challenge to integrate them. Here we introduce simultaneous label-free autofluorescence-multiharmonic (SLAM) microscopy, a single-excitation source nonlinear imaging platform that uses a custom-designed excitation window at 1110 nm and shaped ultrafast pulses at 10 MHz to enable fast (2-orders-of-magnitude improvement), simultaneous, and efficient acquisition of autofluorescence (FAD and NADH) and second/third harmonic generation from a wide array of cellular and extracellular components (e.g., tumor cells, immune cells, vesicles, and vessels) in living tissue using only 14 mW for extended time-lapse investigations. Our work demonstrates the versatility and efficiency of SLAM microscopy for tracking cellular events in vivo, and is a major enabling advance in label-free IVM.

Wavefront measurement using computational adaptive optics.

In many optical imaging applications, it is necessary to correct for aberrations to obtain high quality images. Optical coherence tomography (OCT) provides access to the amplitude and phase of the backscattered optical field for three-dimensional (3D) imaging samples. Computational adaptive optics (CAO) modifies the phase of the OCT data in the spatial frequency domain to correct optical aberrations without using a deformable mirror, as is commonly done in hardware-based adaptive optics (AO). This provides improvement of image quality throughout the 3D volume, enabling imaging across greater depth ranges and in highly aberrated samples. However, the CAO aberration correction has a complicated relation to the imaging pupil and is not a direct measurement of the pupil aberrations. Here we present new methods for recovering the wavefront aberrations directly from the OCT data without the use of hardware adaptive optics. This enables both computational measurement and correction of optical aberrations.

Investigating the healing mechanisms of an angiogenesis-promoting topical treatment for diabetic wounds using multimodal microscopy.

Impaired skin wound healing is a significant comorbid condition of diabetes that is caused by poor microcirculation, among other factors. Studies have shown that angiogenesis, a critical step in the wound healing process in diabetic wounds, can be promoted under hypoxia. In this study, an angiogenesis-promoting topical treatment for diabetic wounds, which promotes angiogenesis by mimicking a hypoxic environment via inhibition of prolyl hydroxylase resulting in elevation or maintenance of hypoxia-inducible factor, was investigated utilizing a custom-built multimodal microscopy system equipped with phase-variance optical coherence tomography (PV-OCT) and fluorescence lifetime imaging microscopy (FLIM). PV-OCT was used to track the regeneration of the microvasculature network, and FLIM was used to assess the in vivo metabolic response of mouse epidermal keratinocytes to the treatment during healing. Results show a significant decrease in the fluorescence lifetime of intracellular reduced nicotinamide adenine dinucleotide, suggesting a hypoxic-like environment in the wounded skin, followed by a quantitative increase in blood vessel density assessed by PV-OCT. Insights gained in these studies could lead to new endpoints for evaluation of the efficacy and healing mechanisms of wound-healing drugs in a setting where delayed healing does not permit available methods for evaluation to take place.

High-speed imaging of transient metabolic dynamics using two-photon fluorescence lifetime imaging microscopy.

Two-photon fluorescence lifetime imaging microscopy (2P-FLIM) of autofluorescent metabolic coenzymes has been widely used to investigate energetic perturbations in living cells and tissues in a label-free manner with subcellular resolution. While the currently used state-of-the-art instruments are highly sensitive to local molecular changes associated with these metabolic processes, they are inherently slow and limit the study of dynamic metabolic environments. Here, a sustained video-rate 2P-FLIM imaging system is demonstrated for time-lapse lifetime imaging of reduced nicotinamide adenine dinucleotide, an autofluorescent metabolic coenzyme involved in both aerobic and anaerobic processes. This system is sufficiently sensitive to differences in metabolic activity between aggressive and nonaggressive cancer cell lines and is demonstrated for both wide field-of-view autofluorescence imaging as well as sustained video-rate image acquisition of metabolic dynamics following induction of apoptosis. The unique capabilities ofthis imaging platform provide a powerful technological advance to further explore rapid metabolic dynamics in living cells.

In Vivo Assessment of Engineered Skin Cell Delivery with Multimodal Optical Microscopy.

The healing process is often significantly impaired under conditions of chronic or large area wounds, which are often treated clinically using autologous split-thickness skin grafts. However, in many cases, harvesting of donor tissue presents a serious problem such as in the case of very large area burns. In response to this, engineered biomaterials have emerged that attempt to mimic the natural skin environment or deliver a suitable therapy to assist in the healing process. In this study, a custom-built multimodal optical microscope capable of noninvasive structural and functional imaging is used to investigate both the engineered tissue microenvironment and the in vivo wound healing process. Investigation of various engineered scaffolds show the strong relationship among the microenvironment of the scaffold, the organization of the cells within the scaffold, and the delivery pattern of these cells onto the healing wound. Through noninvasive tracking of these processes and parameters, multimodal optical microscopy provides an important tool in the assessment of engineered scaffolds both in vitro and in vivo.

A quantitative framework for the analysis of multimodal optical microscopy images.

BACKGROUND: Multimodal optical microscopy, a set of imaging techniques based on unique, yet complementary contrast mechanisms and spatially and temporally co-registered data acquisition, has emerged as a powerful biomedical tool. However, the analysis of the dense, high-dimensional datasets acquired by these instruments remains mostly qualitative and restricted to analysis of each modality individually. METHODS: Using a custom-built multimodal nonlinear optical microscope, high dimensional datasets were acquired for automated classification of functional cell states as well as identification of histopathological features in tissues slices. Supervised classification of cell death modes was performed through support vector machines (SVM) and semi-supervised classification of tissue slices was performed through the use of the expectation maximization (EM) algorithm. RESULTS: Applications of these techniques to the automated classification of cell death modes as well as to the identification of tissue components in fixed ex vivo tissue slices are presented. The analysis techniques developed provide a direct link between multimodal image contrast and biological structure and function, resulting in highly accurate classification in both settings. CONCLUSIONS: Quantification of multimodal optical microscopy images through statistical modeling of the high dimensional data acquired gives a strong correlation between biological structure and function and image contrast. These methods are sensitive to the identification of diagnostic, cellular-level features important in a variety of clinical settings.

Label-free in vivo cellular-level detection and imaging of apoptosis.

Cell death plays a critical role in health and homeostasis as well as in the pathogenesis and treatment of a broad spectrum of diseases and can be broadly divided into two main categories: apoptosis, or programmed cell death, and necrosis, or acute cell death. While these processes have been characterized extensively in vitro, label-free detection of apoptosis and necrosis at the cellular level in vivo has yet to be shown. In this study, for the first time, fluorescence lifetime imaging microscopy (FLIM) of intracellular reduced nicotinamide adenine dinucleotide (NADH) was utilized to assess the metabolic response of in vivo mouse epidermal keratinocytes following induction of apoptosis and necrosis. Results show significantly elevated levels of both the mean lifetime of NADH and the intracellular ratio of protein bound-to-free NADH in the apoptotic compared to the necrotic tissue. In addition, the longitudinal profiles of these two cell death processes show remarkable differences. By identifying and extracting these temporal metabolic signatures, apoptosis in single cells can be studied in native tissue environments within the living organism.