End-to-end diabetic retinopathy grading based on fundus fluorescein angiography images using deep learning.

PURPOSE: To develop and validate a deep learning system for diabetic retinopathy (DR) grading based on fundus fluorescein angiography (FFA) images. METHODS: A total of 11,214 FFA images from 705 patients were collected to form the internal dataset. Three convolutional neural networks, namely VGG16, RestNet50, and DenseNet, were trained using a nine-square grid input, and heat maps were generated. Subsequently, a comparison between human graders and the algorithm was performed. Lastly, the best model was tested on two external datasets (Xian dataset and Ningbo dataset). RESULTS: VGG16 performed the best, with a maximum accuracy of 94.17%, and had an AUC of 0.972, 0.922, and 0.994 for levels 1, 2, and 3, respectively. For Xian dataset, our model reached the accuracy of 82.47% and AUC of 0.910, 0.888, and 0.976 for levels 1, 2, and 3. As for Ningbo dataset, the network performed with the accuracy of 88.89% and AUC of 0.972, 0.756, and 0.945 for levels 1, 2, and 3. CONCLUSIONS: A deep learning system for DR staging was trained based on FFA images and evaluated through human-machine comparisons as well as external dataset testing. The proposed system will help clinical practitioners to diagnose and treat DR patients, and lay a foundation for future applications of other ophthalmic or general diseases.

Multi-label classification of retinal lesions in diabetic retinopathy for automatic analysis of fundus fluorescein angiography based on deep learning.

PURPOSE: To automatically detect and classify the lesions of diabetic retinopathy (DR) in fundus fluorescein angiography (FFA) images using deep learning algorithm through comparing 3 convolutional neural networks (CNNs). METHODS: A total of 4067 FFA images from Eye Center at the Second Affiliated Hospital of Zhejiang University School of Medicine were annotated with 4 kinds of lesions of DR, including non-perfusion regions (NP), microaneurysms, leakages, and laser scars. Three CNNs including DenseNet, ResNet50, and VGG16 were trained to achieve multi-label classification, which means the algorithms could identify 4 retinal lesions above at the same time. RESULTS: The area under the curve (AUC) of DenseNet reached 0.8703, 0.9435, 0.9647, and 0.9653 for detecting NP, microaneurysms, leakages, and laser scars, respectively. For ResNet50, AUC was 0.8140 for NP, 0.9097 for microaneurysms, 0.9585 for leakages, and 0.9115 for laser scars. And for VGG16, AUC was 0.7125 for NP, 0.5569 for microaneurysms, 0.9177 for leakages, and 0.8537 for laser scars. CONCLUSIONS: Experimental results demonstrate that DenseNet is a suitable model to automatically detect and distinguish retinal lesions in the FFA images with multi-label classification, which lies the foundation of automatic analysis for FFA images and comprehensive diagnosis and treatment decision-making for DR.

Retina Fundus Photograph-Based Artificial Intelligence Algorithms in Medicine: A Systematic Review.

We conducted a systematic review of research in artificial intelligence (AI) for retinal fundus photographic images. We highlighted the use of various AI algorithms, including deep learning (DL) models, for application in ophthalmic and non-ophthalmic (i.e., systemic) disorders. We found that the use of AI algorithms for the interpretation of retinal images, compared to clinical data and physician experts, represents an innovative solution with demonstrated superior accuracy in identifying many ophthalmic (e.g., diabetic retinopathy (DR), age-related macular degeneration (AMD), optic nerve disorders), and non-ophthalmic disorders (e.g., dementia, cardiovascular disease). There has been a significant amount of clinical and imaging data for this research, leading to the potential incorporation of AI and DL for automated analysis. AI has the potential to transform healthcare by improving accuracy, speed, and workflow, lowering cost, increasing access, reducing mistakes, and transforming healthcare worker education and training.

Automatic interpretation and clinical evaluation for fundus fluorescein angiography images of diabetic retinopathy patients by deep learning.

BACKGROUND/AIMS: Fundus fluorescein angiography (FFA) is an important technique to evaluate diabetic retinopathy (DR) and other retinal diseases. The interpretation of FFA images is complex and time-consuming, and the ability of diagnosis is uneven among different ophthalmologists. The aim of the study is to develop a clinically usable multilevel classification deep learning model for FFA images, including prediagnosis assessment and lesion classification. METHODS: A total of 15 599 FFA images of 1558 eyes from 845 patients diagnosed with DR were collected and annotated. Three convolutional neural network (CNN) models were trained to generate the label of image quality, location, laterality of eye, phase and five lesions. Performance of the models was evaluated by accuracy, F-1 score, the area under the curve and human-machine comparison. The images with false positive and false negative results were analysed in detail. RESULTS: Compared with LeNet-5 and VGG16, ResNet18 got the best result, achieving an accuracy of 80.79%-93.34% for prediagnosis assessment and an accuracy of 63.67%-88.88% for lesion detection. The human-machine comparison showed that the CNN had similar accuracy with junior ophthalmologists. The false positive and false negative analysis indicated a direction of improvement. CONCLUSION: This is the first study to do automated standardised labelling on FFA images. Our model is able to be applied in clinical practice, and will make great contributions to the development of intelligent diagnosis of FFA images.

Multimodal deep learning with feature level fusion for identification of choroidal neovascularization activity in age-related macular degeneration.

PURPOSE: This study aimed to determine the efficacy of a multimodal deep learning (DL) model using optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) images for the assessment of choroidal neovascularization (CNV) in neovascular age-related macular degeneration (AMD). METHODS: This retrospective and cross-sectional study was performed at a multicentre, and the inclusion criteria were age >50 years and a diagnosis of typical neovascular AMD. The OCT and OCTA data for an internal data set and two external data sets were collected. A DL model was developed with a novel feature-level fusion (FLF) method utilized to combine the multimodal data. The results were compared with identification performed by an ophthalmologist. The best model was tested on two external data sets to show its potential for clinical use. RESULTS: Our best model achieved an accuracy of 95.5% and an area under the curve (AUC) of 0.9796 on multimodal data inputs for the internal data set, which is comparable to the performance of retinal specialists. The proposed model reached an accuracy of 100.00% and an AUC of 1.0 for the Ningbo data set, and these performance indicators were 90.48% and an AUC of 0.9727 for the Jinhua data set. CONCLUSION: The FLF method is feasible and highly accurate, and could enhance the power of the existing computer-aided diagnosis systems. The bi-modal computer-aided diagnosis (CADx) system for the automated identification of CNV activity is an accurate and promising tool in the realm of public health.

Clinicopathological analysis of 5146 eyelid tumours and tumour-like lesions in an eye centre in South China, 2000-2018: a retrospective cohort study.

OBJECTIVES: To describe the clinicopathological characteristics of patients with eyelid tumours and tumour-like lesions in South China, investigate possible factors affecting tumour constitution. DESIGN: Retrospective cohort study. SETTING: All patients diagnosed with eyelid tumours were included from a high-volume tertiary eye care centre from South China which caring for over 2000 patients per day. All biopsied specimens were reviewed by two senior pathologists and were classified according to the fourth edition of the WHO Classification of Skin Tumours. PARTICIPANT: A total of 5146 cases of eyelid lesions were reviewed from 2000 to 2018, being classified by histogenesis and pathologic diagnosis, being compared with data from previous literature containing different races. MAIN OUTCOME MEASURES: Age-specific and gender-specific incidence constitutions, time trends, tumour location, distribution in different age groups and relationship with Sociodmographic Index (SDI) were calculated. RESULTS: Benign tumours accounted for 85.08% (4378) of all cases, among which, nevus was most common (33.07%). Eight of top 10 benign lesions had higher occurrence in upper eyelids. The R² value of linear regression in patient annual number of benign lesions were 0.946 (p<0.01) for male and 0.914 (p<0.01) for female. More than 33.60% (1471/4378) were made up by patients younger than 40 years. The number of patients undergoing removal of benign lesions decreased with age. Among the malignant lesions, basal cell carcinoma (BCC) was most prevalent (48.70%), followed by sebaceous gland carcinoma (34.24%) and majority (81.8%) occurred in patients above 60 years. CONCLUSIONS: Over the past 19 years, most eyelid tumours occurred at our centre were benign lesions. The number of patients presenting with benign lesions increased in both genders, especially among young females who were more likely to request surgeries. Among malignant lesions, BCC remains the most common type, appears a higher incidence in countries with higher SDI.

Prediction of response to anti-vascular endothelial growth factor treatment in diabetic macular oedema using an optical coherence tomography-based machine learning method.

PURPOSE: To predict the anti-vascular endothelial growth factor (VEGF) therapeutic response of diabetic macular oedema (DME) patients from optical coherence tomography (OCT) at the initiation stage of treatment using a machine learning-based self-explainable system. METHODS: A total of 712 DME patients were included and classified into poor and good responder groups according to central macular thickness decrease after three consecutive injections. Machine learning models were constructed to make predictions based on related features extracted automatically using deep learning algorithms from OCT scans at baseline. Five-fold cross-validation was applied to optimize and evaluate the models. The model with the best performance was then compared with two ophthalmologists. Feature importance was further investigated, and a Wilcoxon rank-sum test was performed to assess the difference of a single feature between two groups. RESULTS: Of 712 patients, 294 were poor responders and 418 were good responders. The best performance for the prediction task was achieved by random forest (RF), with sensitivity, specificity and area under the receiver operating characteristic curve of 0.900, 0.851 and 0.923. Ophthalmologist 1 and ophthalmologist 2 reached sensitivity of 0.775 and 0.750, and specificity of 0.716 and 0.821, respectively. The sum of hyperreflective dots was found to be the most relevant feature for prediction. CONCLUSION: An RF classifier was constructed to predict the treatment response of anti-VEGF from OCT images of DME patients with high accuracy. The algorithm contributes to predicting treatment requirements in advance and provides an optimal individualized therapeutic regimen.

Automatic detection of non-perfusion areas in diabetic macular edema from fundus fluorescein angiography for decision making using deep learning.

Vision loss caused by diabetic macular edema (DME) can be prevented by early detection and laser photocoagulation. As there is no comprehensive detection technique to recognize NPA, we proposed an automatic detection method of NPA on fundus fluorescein angiography (FFA) in DME. The study included 3,014 FFA images of 221 patients with DME. We use 3 convolutional neural networks (CNNs), including DenseNet, ResNet50, and VGG16, to identify non-perfusion regions (NP), microaneurysms, and leakages in FFA images. The NPA was segmented using attention U-net. To validate its performance, we applied our detection algorithm on 249 FFA images in which the NPA areas were manually delineated by 3 ophthalmologists. For DR lesion classification, area under the curve is 0.8855 for NP regions, 0.9782 for microaneurysms, and 0.9765 for leakage classifier. The average precision of NP region overlap ratio is 0.643. NP regions of DME in FFA images are identified based a new automated deep learning algorithm. This study is an in-depth study from computer-aided diagnosis to treatment, and will be the theoretical basis for the application of intelligent guided laser.