Interpretable Diabetic Retinopathy Diagnosis Based on Biomarker Activation Map.

OBJECTIVE: Deep learning classifiers provide the most accurate means of automatically diagnosing diabetic retinopathy (DR) based on optical coherence tomography (OCT) and its angiography (OCTA). The power of these models is attributable in part to the inclusion of hidden layers that provide the complexity required to achieve a desired task. However, hidden layers also render algorithm outputs difficult to interpret. Here we introduce a novel biomarker activation map (BAM) framework based on generative adversarial learning that allows clinicians to verify and understand classifiers' decision-making. METHODS: A data set including 456 macular scans were graded as non-referable or referable DR based on current clinical standards. A DR classifier that was used to evaluate our BAM was first trained based on this data set. The BAM generation framework was designed by combing two U-shaped generators to provide meaningful interpretability to this classifier. The main generator was trained to take referable scans as input and produce an output that would be classified by the classifier as non-referable. The BAM is then constructed as the difference image between the output and input of the main generator. To ensure that the BAM only highlights classifier-utilized biomarkers an assistant generator was trained to do the opposite, producing scans that would be classified as referable by the classifier from non-referable scans. RESULTS: The generated BAMs highlighted known pathologic features including nonperfusion area and retinal fluid. CONCLUSION/SIGNIFICANCE: A fully interpretable classifier based on these highlights could help clinicians better utilize and verify automated DR diagnosis.

Deep-Learning-Aided Diagnosis of Diabetic Retinopathy, Age-Related Macular Degeneration, and Glaucoma Based on Structural and Angiographic OCT.

Purpose: Timely diagnosis of eye diseases is paramount to obtaining the best treatment outcomes. OCT and OCT angiography (OCTA) have several advantages that lend themselves to early detection of ocular pathology; furthermore, the techniques produce large, feature-rich data volumes. However, the full clinical potential of both OCT and OCTA is stymied when complex data acquired using the techniques must be manually processed. Here, we propose an automated diagnostic framework based on structural OCT and OCTA data volumes that could substantially support the clinical application of these technologies. Design: Cross sectional study. Participants: Five hundred twenty-six OCT and OCTA volumes were scanned from the eyes of 91 healthy participants, 161 patients with diabetic retinopathy (DR), 95 patients with age-related macular degeneration (AMD), and 108 patients with glaucoma. Methods: The diagnosis framework was constructed based on semisequential 3-dimensional (3D) convolutional neural networks. The trained framework classifies combined structural OCT and OCTA scans as normal, DR, AMD, or glaucoma. Fivefold cross-validation was performed, with 60% of the data reserved for training, 20% for validation, and 20% for testing. The training, validation, and test data sets were independent, with no shared patients. For scans diagnosed as DR, AMD, or glaucoma, 3D class activation maps were generated to highlight subregions that were considered important by the framework for automated diagnosis. Main Outcome Measures: The area under the curve (AUC) of the receiver operating characteristic curve and quadratic-weighted kappa were used to quantify the diagnostic performance of the framework. Results: For the diagnosis of DR, the framework achieved an AUC of 0.95 ± 0.01. For the diagnosis of AMD, the framework achieved an AUC of 0.98 ± 0.01. For the diagnosis of glaucoma, the framework achieved an AUC of 0.91 ± 0.02. Conclusions: Deep learning frameworks can provide reliable, sensitive, interpretable, and fully automated diagnosis of eye diseases. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

A Diabetic Retinopathy Classification Framework Based on Deep-Learning Analysis of OCT Angiography.

Purpose: Reliable classification of referable and vision threatening diabetic retinopathy (DR) is essential for patients with diabetes to prevent blindness. Optical coherence tomography (OCT) and its angiography (OCTA) have several advantages over fundus photographs. We evaluated a deep-learning-aided DR classification framework using volumetric OCT and OCTA. Methods: Four hundred fifty-six OCT and OCTA volumes were scanned from eyes of 50 healthy participants and 305 patients with diabetes. Retina specialists labeled the eyes as non-referable (nrDR), referable (rDR), or vision threatening DR (vtDR). Each eye underwent a 3 × 3-mm scan using a commercial 70 kHz spectral-domain OCT system. We developed a DR classification framework and trained it using volumetric OCT and OCTA to classify eyes into rDR and vtDR. For the scans identified as rDR or vtDR, 3D class activation maps were generated to highlight the subregions which were considered important by the framework for DR classification. Results: For rDR classification, the framework achieved a 0.96 ± 0.01 area under the receiver operating characteristic curve (AUC) and 0.83 ± 0.04 quadratic-weighted kappa. For vtDR classification, the framework achieved a 0.92 ± 0.02 AUC and 0.73 ± 0.04 quadratic-weighted kappa. In addition, the multiple DR classification (non-rDR, rDR but non-vtDR, or vtDR) achieved a 0.83 ± 0.03 quadratic-weighted kappa. Conclusions: A deep learning framework only based on OCT and OCTA can provide specialist-level DR classification using only a single imaging modality. Translational Relevance: The proposed framework can be used to develop clinically valuable automated DR diagnosis system because of the specialist-level performance showed in this study.

Federated Learning for Microvasculature Segmentation and Diabetic Retinopathy Classification of OCT Data.

Purpose: To evaluate the performance of a federated learning framework for deep neural network-based retinal microvasculature segmentation and referable diabetic retinopathy (RDR) classification using OCT and OCT angiography (OCTA). Design: Retrospective analysis of clinical OCT and OCTA scans of control participants and patients with diabetes. Participants: The 153 OCTA en face images used for microvasculature segmentation were acquired from 4 OCT instruments with fields of view ranging from 2 × 2-mm to 6 × 6-mm. The 700 eyes used for RDR classification consisted of OCTA en face images and structural OCT projections acquired from 2 commercial OCT systems. Methods: OCT angiography images used for microvasculature segmentation were delineated manually and verified by retina experts. Diabetic retinopathy (DR) severity was evaluated by retinal specialists and was condensed into 2 classes: non-RDR and RDR. The federated learning configuration was demonstrated via simulation using 4 clients for microvasculature segmentation and was compared with other collaborative training methods. Subsequently, federated learning was applied over multiple institutions for RDR classification and was compared with models trained and tested on data from the same institution (internal models) and different institutions (external models). Main Outcome Measures: For microvasculature segmentation, we measured the accuracy and Dice similarity coefficient (DSC). For severity classification, we measured accuracy, area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve, balanced accuracy, F1 score, sensitivity, and specificity. Results: For both applications, federated learning achieved similar performance as internal models. Specifically, for microvasculature segmentation, the federated learning model achieved similar performance (mean DSC across all test sets, 0.793) as models trained on a fully centralized dataset (mean DSC, 0.807). For RDR classification, federated learning achieved a mean AUROC of 0.954 and 0.960; the internal models attained a mean AUROC of 0.956 and 0.973. Similar results are reflected in the other calculated evaluation metrics. Conclusions: Federated learning showed similar results to traditional deep learning in both applications of segmentation and classification, while maintaining data privacy. Evaluation metrics highlight the potential of collaborative learning for increasing domain diversity and the generalizability of models used for the classification of OCT data.

DcardNet: Diabetic Retinopathy Classification at Multiple Levels Based on Structural and Angiographic Optical Coherence Tomography.

OBJECTIVE: Optical coherence tomography (OCT) and its angiography (OCTA) have several advantages for the early detection and diagnosis of diabetic retinopathy (DR). However, automated, complete DR classification frameworks based on both OCT and OCTA data have not been proposed. In this study, a convolutional neural network (CNN) based method is proposed to fulfill a DR classification framework using en face OCT and OCTA. METHODS: A densely and continuously connected neural network with adaptive rate dropout (DcardNet) is designed for the DR classification. In addition, adaptive label smoothing was proposed and used to suppress overfitting. Three separate classification levels are generated for each case based on the International Clinical Diabetic Retinopathy scale. At the highest level the network classifies scans as referable or non-referable for DR. The second level classifies the eye as non-DR, non-proliferative DR (NPDR), or proliferative DR (PDR). The last level classifies the case as no DR, mild and moderate NPDR, severe NPDR, and PDR. RESULTS: We used 10-fold cross-validation with 10% of the data to assess the network's performance. The overall classification accuracies of the three levels were 95.7%, 85.0%, and 71.0% respectively. CONCLUSION/SIGNIFICANCE: A reliable, sensitive and specific automated classification framework for referral to an ophthalmologist can be a key technology for reducing vision loss related to DR.

Sensorless adaptive-optics optical coherence tomographic angiography.

Optical coherence tomographic angiography (OCTA) can image the retinal blood flow but visualization of the capillary caliber is limited by the low lateral resolution. Adaptive optics (AO) can be used to compensate ocular aberrations when using high numerical aperture (NA), and thus improve image resolution. However, previously reported AO-OCTA instruments were large and complex, and have a small sub-millimeter field of view (FOV) that hinders the extraction of biomarkers with clinical relevance. In this manuscript, we developed a sensorless AO-OCTA prototype with an intermediate numerical aperture to produce depth-resolved angiograms with high resolution and signal-to-noise ratio over a 2 × 2 mm FOV, with a focal spot diameter of 6 µm, which is about 3 times finer than typical commercial OCT systems. We believe these parameters may represent a better tradeoff between resolution and FOV compared to large-NA AO systems, since the spot size matches better that of capillaries. The prototype corrects defocus, astigmatism, and coma using a figure of merit based on the mean reflectance projection of a slab defined with real-time segmentation of retinal layers. AO correction with the ability to optimize focusing in arbitrary retinal depths - particularly the plexuses in the inner retina - could be achieved in 1.35 seconds. The AO-OCTA images showed greater flow signal, signal-to-noise ratio, and finer capillary caliber compared to commercial OCTA. Projection artifacts were also reduced in the intermediate and deep capillary plexuses. The instrument reported here improves OCTA image quality without excessive sacrifice in FOV and device complexity, and thus may have potential for clinical translation.

Automated diagnosis and segmentation of choroidal neovascularization in OCT angiography using deep learning.

Accurate identification and segmentation of choroidal neovascularization (CNV) is essential for the diagnosis and management of exudative age-related macular degeneration (AMD). Projection-resolved optical coherence tomographic angiography (PR-OCTA) enables both cross-sectional and en face visualization of CNV. However, CNV identification and segmentation remains difficult even with PR-OCTA due to the presence of residual artifacts. In this paper, a fully automated CNV diagnosis and segmentation algorithm using convolutional neural networks (CNNs) is described. This study used a clinical dataset, including both scans with and without CNV, and scans of eyes with different pathologies. Furthermore, no scans were excluded due to image quality. In testing, all CNV cases were diagnosed from non-CNV controls with 100% sensitivity and 95% specificity. The mean intersection over union of CNV membrane segmentation was as high as 0.88. By enabling fully automated categorization and segmentation, the proposed algorithm should offer benefits for CNV diagnosis, visualization monitoring.

Measuring Glaucomatous Focal Perfusion Loss in the Peripapillary Retina Using OCT Angiography.

PURPOSE: To measure low perfusion areas (LPAs) and focal perfusion loss (FPL) in the peripapillary retina using OCT angiography (OCTA) in glaucoma. DESIGN: Prospective, observational study. PARTICIPANTS: A total of 47 patients with primary open-angle glaucoma (POAG) and 36 normal participants were analyzed. METHODS: One eye of each subject was scanned using an AngioVue (Optovue, Fremont, CA) 4.5-mm OCTA scan centered on the disc. En face nerve fiber layer (NFL) plexus angiogram was generated. With the use of custom software, a capillary density map was obtained by computing the fraction of area occupied by flow pixels after low-pass filtering by local averaging 21×21 pixels. The low-perfusion map is defined by local capillary density below 0.5 percentile over a contiguous area above 98.5 percentile of the normal reference population. The LPA parameter is the cumulative area, and the FPL is the percent capillary density loss (relative to normal mean) integrated over the LPA. MAIN OUTCOME MEASURES: Peripapillary retinal LPA and FPL. RESULTS: Among patients with POAG, 3 had preperimetric glaucoma and 44 had perimetric glaucoma, with visual field (VF) mean deviation (MD) of -5.14±4.25 decibels (dB). The LPA was 3.40±2.29 mm2 in those with POAG and 0.11±0.18 mm2 in normal subjects (P < 0.001). The FPL was 21.8%±17.0% in those with POAG and 0.3%±0.7% in normal subjects (P < 0.001). The diagnostic accuracy as measured by the area under the receiver operating curve was 0.965 for both LPA and FPL, with a sensitivity of 93.7% at 95% specificity. The repeatability as measured by intraclass correlation coefficient was 0.977 for LPA and 0.958 for FPL. The FPL had excellent correlation with VF MD (Spearman's rho = -0.843), which was significantly (P = 0.008) better than the correlation between NFL thickness and VF MD (rho = 0.760). The hemispheric difference correlation between FPL and VF (Spearman's rho = 0.770) was significantly (P < 0.001) higher than the hemispheric difference correlation between LPA and VF (rho = 0.595). CONCLUSIONS: The low-perfusion map and LPA and FPL parameters are able to assess the location and severity of focal glaucoma damage with good agreement with VF.

DETECTION OF CLINICALLY UNSUSPECTED RETINAL NEOVASCULARIZATION WITH WIDE-FIELD OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY.

PURPOSE: To evaluate wide-field optical coherence tomography angiography (OCTA) for detection of clinically unsuspected neovascularization (NV) in diabetic retinopathy (DR). METHODS: This prospective observational single-center study included adult patients with a clinical diagnosis of nonproliferative DR. Participants underwent a clinical examination, standard 7-field color photography, and OCTA with commercial and prototype swept-source devices. The wide-field OCTA was achieved by montaging five 6 × 10-mm scans from a prototype device into a 25 × 10-mm image and three 6 × 6-mm scans from a commercial device into a 15 × 6-mm image. A masked grader determined the retinopathy severity from color photographs. Two trained readers examined conventional and wide-field OCTA images for the presence of NV. RESULTS: Of 27 participants, photographic grading found 13 mild, 7 moderate, and 7 severe nonproliferative DR. Conventional 6 × 6-mm OCTA detected NV in 2 eyes (7%) and none with 3 × 3-mm scans. Both prototype and commercial wide-field OCTA detected NV in two additional eyes. The mean area of NV was 0.38 mm (range 0.17-0.54 mm). All eyes with OCTA-detected NV were photographically graded as severe nonproliferative DR. CONCLUSION: Wide-field OCTA can detect small NV not seen on clinical examination or color photographs and may improve the clinical evaluation of DR.

Automated segmentation of peripapillary retinal boundaries in OCT combining a convolutional neural network and a multi-weights graph search.

Quantitative analysis of the peripapillary retinal layers and capillary plexuses from optical coherence tomography (OCT) and OCT angiography images depend on two segmentation tasks - delineating the boundary of the optic disc and delineating the boundaries between retinal layers. Here, we present a method combining a neural network and graph search to perform these two tasks. A comparison of this novel method's segmentation of the disc boundary showed good agreement with the ground truth, achieving an overall Dice similarity coefficient of 0.91 ± 0.04 in healthy and glaucomatous eyes. The absolute error of retinal layer boundaries segmentation in the same cases was 4.10 ± 1.25 µm.

Automated boundary detection of the optic disc and layer segmentation of the peripapillary retina in volumetric structural and angiographic optical coherence tomography.

To improve optic disc boundary detection and peripapillary retinal layer segmentation, we propose an automated approach for structural and angiographic optical coherence tomography. The algorithm was performed on radial cross-sectional B-scans. The disc boundary was detected by searching for the position of Bruch's membrane opening, and retinal layer boundaries were detected using a dynamic programming-based graph search algorithm on each B-scan without the disc region. A comparison of the disc boundary using our method with that determined by manual delineation showed good accuracy, with an average Dice similarity coefficient ≥0.90 in healthy eyes and eyes with diabetic retinopathy and glaucoma. The layer segmentation accuracy in the same cases was on average less than one pixel (3.13 µm).

Extended axial imaging range, widefield swept source optical coherence tomography angiography.

We developed a high-speed, swept source OCT system for widefield OCT angiography (OCTA) imaging. The system has an extended axial imaging range of 6.6 mm. An electrical lens is used for fast, automatic focusing. The recently developed split-spectrum amplitude and phase-gradient angiography allow high-resolution OCTA imaging with only two B-scan repetitions. An improved post-processing algorithm effectively removed trigger jitter artifacts and reduced noise in the flow signal. We demonstrated high contrast 3 mm×3 mm OCTA image with 400×400 pixels acquired in 3 seconds and high-definition 8 mm×6 mm and 12 mm×6 mm OCTA images with 850×400 pixels obtained in 4 seconds. A widefield 8 mm×11 mm OCTA image is produced by montaging two 8 mm×6 mm scans. An ultra-widefield (with a maximum of 22 mm along both vertical and horizontal directions) capillary-resolution OCTA image is obtained by montaging six 12 mm×6 mm scans.

Automated three-dimensional registration and volume rebuilding for wide-field angiographic and structural optical coherence tomography.

We propose a three-dimensional (3-D) registration method to correct motion artifacts and construct the volume structure for angiographic and structural optical coherence tomography (OCT). This algorithm is particularly suitable for the nonorthogonal wide-field OCT scan acquired by a ultrahigh-speed swept-source system ( > 200 ?? kHz A-scan rate). First, the transverse motion artifacts are corrected by the between-frame registration based on en face OCT angiography (OCTA). After A-scan transverse translation between B-frames, the axial motions are corrected based on the rebuilt boundary of inner limiting membrane. Finally, a within-frame registration is performed for local optimization based on cross-sectional OCTA. We evaluated this algorithm on retinal volumes of six normal subjects. The results showed significantly improved retinal smoothness in 3-D-registered structural OCT and image contrast on en face OCTA.

Automatic multiorgan segmentation in CT images of the male pelvis using region-specific hierarchical appearance cluster models.

PURPOSE: Accurate segmentation of pelvic organs in CT images is of great importance in external beam radiotherapy for prostate cancer. The aim of this studying is to develop a novel method for automatic, multiorgan segmentation of the male pelvis. METHODS: The authors' segmentation method consists of several stages. First, a pretreatment includes parameterization, principal component analysis (PCA), and an established process of region-specific hierarchical appearance cluster (RSHAC) model which was executed on the training dataset. After the preprocessing, online automatic segmentation of new CT images is achieved by combining the RSHAC model with the PCA-based point distribution model. Fifty pelvic CT from eight prostate cancer patients were used as the training dataset. From another 20 prostate cancer patients, 210 CT images were used for independent validation of the segmentation method. RESULTS: In the training dataset, 15 PCA modes were needed to represent 95% of shape variations of pelvic organs. When tested on the validation dataset, the authors' segmentation method had an average Dice similarity coefficient and mean absolute distance of 0.751 and 0.371 cm, 0.783 and 0.303 cm, 0.573 and 0.604 cm for prostate, bladder, and rectum, respectively. The automated segmentation process took on average 5 min on a personal computer equipped with Core 2 Duo CPU of 2.8 GHz and 8 GB RAM. CONCLUSIONS: The authors have developed an efficient and reliable method for automatic segmentation of multiple organs in the male pelvis. This method should be useful for treatment planning and adaptive replanning for prostate cancer radiotherapy. With this method, the physicist can improve the work efficiency and stability.

Automated motion correction using parallel-strip registration for wide-field en face OCT angiogram.

We propose an innovative registration method to correct motion artifacts for wide-field optical coherence tomography angiography (OCTA) acquired by ultrahigh-speed swept-source OCT (>200 kHz A-scan rate). Considering that the number of A-scans along the fast axis is much higher than the number of positions along slow axis in the wide-field OCTA scan, a non-orthogonal scheme is introduced. Two en face angiograms in the vertical priority (2 y-fast) are divided into microsaccade-free parallel strips. A gross registration based on large vessels and a fine registration based on small vessels are sequentially applied to register parallel strips into a composite image. This technique is extended to automatically montage individual registered, motion-free angiograms into an ultrawide-field view.