Automated malarial retinopathy detection using transfer learning and multi-camera retinal images.

Cerebral malaria (CM) is a fatal syndrome found commonly in children less than 5 years old in Sub-saharan Africa and Asia. The retinal signs associated with CM are known as malarial retinopathy (MR), and they include highly specific retinal lesions such as whitening and hemorrhages. Detecting these lesions allows the detection of CM with high specificity. Up to 23% of CM, patients are over-diagnosed due to the presence of clinical symptoms also related to pneumonia, meningitis, or others. Therefore, patients go untreated for these pathologies, resulting in death or neurological disability. It is essential to have a low-cost and high-specificity diagnostic technique for CM detection, for which We developed a method based on transfer learning (TL). Models pre-trained with TL select the good quality retinal images, which are fed into another TL model to detect CM. This approach shows a 96% specificity with low-cost retinal cameras.

Diabetic Peripheral Neuropathy Risk Assessment using Digital Fundus Photographs and Machine Learning.

In this work, we demonstrate a novel approach to assessing the risk of Diabetic Peripheral Neuropathy (DPN) using only the retinal images of the patients. Our methodology consists of convolutional neural network feature extraction, dimensionality reduction and feature selection with random projections, combination of image features to case-level representations, and the training and testing of a support vector machine classifier. Using clinical diagnosis as ground truth for DPN, we achieve an overall accuracy of 89% on a held-out test set, with sensitivity reaching 78% and specificity reaching 95%.

Automated Detection of Malarial Retinopathy in Retinal Fundus Images obtained in Clinical Settings.

Cerebral malaria (CM) is a life-threatening clinical syndrome associated with 5-10% of malarial infection cases, most prevalent in Africa. About 23% of cerebral malaria cases are misdiagnosed as false positives, leading to inappropriate treatment and loss of lives. Malarial retinopathy (MR) is a retinal manifestation of CM that presents with a highly specific set of lesions. The detection of MR can reduce the false positive diagnosis of CM and alert physicians to investigate for other possible causes of the clinical symptoms and apply a more appropriate clinical intervention of underlying diseases. In order to facilitate easily accessible and affordable means of MR detection, we have developed an automated software system that detects the retinal lesions specific to MR, whitening and hemorrhages, using retinal color fundus images. The individual lesion detection algorithms were combined into an MR detection model using partial least square classifier. The classifier model was trained and tested on retinal image dataset obtained from 64 patients presenting with clinical signs of CM (44 with MR, 20 without MR). The MR detection model yielded specificity of 92% and sensitivity of 68%, with an AUC of 0.82. The proposed MR detection system demonstrates potential for broad screening of MR and can be integrated with a low-cost and portable retinal camera, to provide a bed-side tool for confirming CM diagnosis.

Automated Detection of Malarial Retinopathy in Digital Fundus Images for Improved Diagnosis in Malawian Children with Clinically Defined Cerebral Malaria.

Cerebral malaria (CM), a complication of malaria infection, is the cause of the majority of malaria-associated deaths in African children. The standard clinical case definition for CM misclassifies ~25% of patients, but when malarial retinopathy (MR) is added to the clinical case definition, the specificity improves from 61% to 95%. Ocular fundoscopy requires expensive equipment and technical expertise not often available in malaria endemic settings, so we developed an automated software system to analyze retinal color images for MR lesions: retinal whitening, vessel discoloration, and white-centered hemorrhages. The individual lesion detection algorithms were combined using a partial least square classifier to determine the presence or absence of MR. We used a retrospective retinal image dataset of 86 pediatric patients with clinically defined CM (70 with MR and 16 without) to evaluate the algorithm performance. Our goal was to reduce the false positive rate of CM diagnosis, and so the algorithms were tuned at high specificity. This yielded sensitivity/specificity of 95%/100% for the detection of MR overall, and 65%/94% for retinal whitening, 62%/100% for vessel discoloration, and 73%/96% for hemorrhages. This automated system for detecting MR using retinal color images has the potential to improve the accuracy of CM diagnosis.

Comparison of the effectiveness of three retinal camera technologies for malarial retinopathy detection in Malawi.

The purpose of this study was to test the suitability of three available camera technologies (desktop, portable, and i-phone based) for imaging comatose children who presented with clinical symptoms of malaria. Ultimately, the results of the project would form the basis for a design of a future camera to screen for malaria retinopathy (MR) in a resource challenged environment. The desktop, portable, and i-phone based cameras were represented by the Topcon, Pictor Plus, and Peek cameras, respectively. These cameras were tested on N=23 children presenting with symptoms of cerebral malaria (CM) at a malaria clinic, Queen Elizabeth Teaching Hospital in Malawi, Africa. Each patient was dilated for binocular indirect ophthalmoscopy (BIO) exam by an ophthalmologist followed by imaging with all three cameras. Each of the cases was graded according to an internationally established protocol and compared to the BIO as the clinical ground truth. The reader used three principal retinal lesions as markers for MR: hemorrhages, retinal whitening, and vessel discoloration. The study found that the mid-priced Pictor Plus hand-held camera performed considerably better than the lower price mobile phone-based camera, and slightly the higher priced table top camera. When comparing the readings of digital images against the clinical reference standard (BIO), the Pictor Plus camera had sensitivity and specificity for MR of 100% and 87%, respectively. This compares to a sensitivity and specificity of 87% and 75% for the i-phone based camera and 100% and 75% for the desktop camera. The drawback of all the cameras were their limited field of view which did not allow complete view of the periphery where vessel discoloration occurs most frequently. The consequence was that vessel discoloration was not addressed in this study. None of the cameras offered real-time image quality assessment to ensure high quality images to afford the best possible opportunity for reading by a remotely located specialist.

A multiscale decomposition approach to detect abnormal vasculature in the optic disc.

This paper presents a multiscale method to detect neovascularization in the optic disc (NVD) using fundus images. Our method is applied to a manually selected region of interest (ROI) containing the optic disc. All the vessels in the ROI are segmented by adaptively combining contrast enhancement methods with a vessel segmentation technique. Textural features extracted using multiscale amplitude-modulation frequency-modulation, morphological granulometry, and fractal dimension are used. A linear SVM is used to perform the classification, which is tested by means of 10-fold cross-validation. The performance is evaluated using 300 images achieving an AUC of 0.93 with maximum accuracy of 88%.

A multiscale optimization approach to detect exudates in the macula.

Pathologies that occur on or near the fovea, such as clinically significant macular edema (CSME), represent high risk for vision loss. The presence of exudates, lipid residues of serous leakage from damaged capillaries, has been associated with CSME, in particular if they are located one optic disc-diameter away from the fovea. In this paper, we present an automatic system to detect exudates in the macula. Our approach uses optimal thresholding of instantaneous amplitude (IA) components that are extracted from multiple frequency scales to generate candidate exudate regions. For each candidate region, we extract color, shape, and texture features that are used for classification. Classification is performed using partial least squares (PLS). We tested the performance of the system on two different databases of 652 and 400 images. The system achieved an area under the receiver operator characteristic curve (AUC) of 0.96 for the combination of both databases and an AUC of 0.97 for each of them when they were evaluated independently.

Detection of hypertensive retinopathy using vessel measurements and textural features.

Features that indicate hypertensive retinopathy have been well described in the medical literature. This paper presents a new system to automatically classify subjects with hypertensive retinopathy (HR) using digital color fundus images. Our method consists of the following steps: 1) normalization and enhancement of the image; 2) determination of regions of interest based on automatic location of the optic disc; 3) segmentation of the retinal vasculature and measurement of vessel width and tortuosity; 4) extraction of color features; 5) classification of vessel segments as arteries or veins; 6) calculation of artery-vein ratios using the six widest (major) vessels for each category; 7) calculation of mean red intensity and saturation values for all arteries; 8) calculation of amplitude-modulation frequency-modulation (AM-FM) features for entire image; and 9) classification of features into HR and non-HR using linear regression. This approach was tested on 74 digital color fundus photographs taken with TOPCON and CANON retinal cameras using leave-one out cross validation. An area under the ROC curve (AUC) of 0.84 was achieved with sensitivity and specificity of 90% and 67%, respectively.

Comprehensive automatic assessment of retinal vascular abnormalities for computer-assisted retinopathy grading.

One of the most important signs of systemic disease that presents on the retina is vascular abnormalities such as in hypertensive retinopathy. Manual analysis of fundus images by human readers is qualitative and lacks in accuracy, consistency and repeatability. Present semi-automatic methods for vascular evaluation are reported to increase accuracy and reduce reader variability, but require extensive reader interaction; thus limiting the software-aided efficiency. Automation thus holds a twofold promise. First, decrease variability while increasing accuracy, and second, increasing the efficiency. In this paper we propose fully automated software as a second reader system for comprehensive assessment of retinal vasculature; which aids the readers in the quantitative characterization of vessel abnormalities in fundus images. This system provides the reader with objective measures of vascular morphology such as tortuosity, branching angles, as well as highlights of areas with abnormalities such as artery-venous nicking, copper and silver wiring, and retinal emboli; in order for the reader to make a final screening decision. To test the efficacy of our system, we evaluated the change in performance of a newly certified retinal reader when grading a set of 40 color fundus images with and without the assistance of the software. The results demonstrated an improvement in reader's performance with the software assistance, in terms of accuracy of detection of vessel abnormalities, determination of retinopathy, and reading time. This system enables the reader in making computer-assisted vasculature assessment with high accuracy and consistency, at a reduced reading time.

Computational basis for risk stratification of peripheral neuropathy from thermal imaging.

The goal of this paper is to present a computer-based system for analyzing thermal images in the detection of preclinical stages of peripheral neuropathy (PN) or diabetic foot. Today, vibration perception threshold (VPT) and sensory tests with a monofilament are used as simple, noninvasive methods for identifying patients who have lost sensation in their feet. These tests are qualitative and are ineffective in stratifying risk for PN in a diabetic patient. In our system a cold stimulus applied to the foot causes a thermoregulatory and corresponding microcirculation response of the foot. A thermal video monitors the recovery of the microcirculation in the foot plantar. Thermal videos for 8 age-matched subjects were analyzed. Six sites were tracked and an average thermal emittance calculated. Characteristics of the recovery curve were extracted using coefficients from an exponential curve fitting process and compared among subjects. The magnitude of the recovery was significantly different for the two classes of subjects. Our system shows evidence of differences between both groups, which could lead to a quantitative test to screen and diagnose peripheral neuropathy.

Detection of neovascularization in the optic disc using an AM-FM representation, granulometry, and vessel segmentation.

Neovascularization, defined as abnormal formation of blood vessels in the retina, is a sight-threatening condition indicative of late-stage diabetic retinopathy (DR). Ischemia due to leakage of blood vessels causes the body to produce new and weak vessels that can lead to complications such as vitreous hemorrhages. Neovascularization on the disc (NVD) is diagnosed when new vessels are located within one disc-diameter of the optic disc. Accurately detecting NVD is important in preventing vision loss due to DR. This paper presents a method for detecting NVD in digital fundus images. First, a region of interest (ROI) containing the optic disc is manually selected from the image. By adaptively combining contrast enhancement methods with a vessel segmentation technique, the ROI is reduced to the regions indicated by the segmented vessels. Textural features extracted by using amplitude-modulation frequency-modulation (AM-FM) techniques and granulometry are used to differentiate NVD from a normal optic disc. Partial least squares is used to perform the final classification. Leave-one-out cross-validation was used to evaluate the performance of the system with 27 NVD and 30 normal cases. We obtained an area under the receiver operator characteristic curve (AUC) of 0.85 by using all features, increasing to 0.94 with feature selection.

Automated analysis of optic nerve images for detection and staging of papilledema.

PURPOSE: To develop an automated system that analyzes digital fundus images for staging and monitoring of optic disc edema (i.e., papilledema), due to raised intracranial pressure. METHODS: A total of 294 retrospective, digital photographs of the right and left eyes of 39 subjects with papilledema acquired over the span of 2 years were used. Software tools were developed to analyze three features of papilledema from digital fundus photographs: (1) sharpness of the optic disc border, (2) discontinuity along major vessels overlying the optic nerve, and (3) texture properties of the peripapillary retinal nerve fiber layer (RNFL). A classifier used these features to assign a grade of papilledema according to a standard protocol used by an expert neuro-ophthalmologist (RK). RESULTS: The algorithm showed substantial agreement (κ = 0.71, P < 0.001) with the neuro-ophthalmologist when grading papilledema per patient. Vessel features showed statistical significance (P < 0.05) in differentiating grades 0, 1, and 2 from grades 3 and 4, whereas disc obscuration differentiated grades 0 or 1 from the rest (P < 0.05). CONCLUSIONS: These results show that this algorithm can be used to automatically grade papilledema. The algorithm provides objective and quantitative assessment of the stage of papilledema with accuracy that is comparable to grading by a neuro-ophthalmologist. One application is in rapid assessment of digital optic nerve photographs acquired in clinical, intensive care, and emergency response settings by nonophthalmologists to evaluate for the presence and severity of papilledema, due to intracranial hypertension.

Automatic detection of diabetic retinopathy and age-related macular degeneration in digital fundus images.

PURPOSE: To describe and evaluate the performance of an algorithm that automatically classifies images with pathologic features commonly found in diabetic retinopathy (DR) and age-related macular degeneration (AMD). METHODS: Retinal digital photographs (N = 2247) of three fields of view (FOV) were obtained of the eyes of 822 patients at two centers: The Retina Institute of South Texas (RIST, San Antonio, TX) and The University of Texas Health Science Center San Antonio (UTHSCSA). Ground truth was provided for the presence of pathologic conditions, including microaneurysms, hemorrhages, exudates, neovascularization in the optic disc and elsewhere, drusen, abnormal pigmentation, and geographic atrophy. The algorithm was used to report on the presence or absence of disease. A detection threshold was applied to obtain different values of sensitivity and specificity with respect to ground truth and to construct a receiver operating characteristic (ROC) curve. RESULTS: The system achieved an average area under the ROC curve (AUC) of 0.89 for detection of DR and of 0.92 for detection of sight-threatening DR (STDR). With a fixed specificity of 0.50, the system's sensitivity ranged from 0.92 for all DR cases to 1.00 for clinically significant macular edema (CSME). CONCLUSIONS: A computer-aided algorithm was trained to detect different types of pathologic retinal conditions. The cases of hard exudates within 1 disc diameter (DD) of the fovea (surrogate for CSME) were detected with very high accuracy (sensitivity = 1, specificity = 0.50), whereas mild nonproliferative DR was the most challenging condition (sensitivity = 0.92, specificity = 0.50). The algorithm was also tested on images with signs of AMD, achieving a performance of AUC of 0.84 (sensitivity = 0.94, specificity = 0.50).

Independent component analysis using prior information for signal detection in a functional imaging system of the retina.

Independent component analysis (ICA) is a statistical technique that estimates a set of sources mixed by an unknown mixing matrix using only a set of observations. For this purpose, the only assumption is that the sources are statistically independent. In many applications, some information about the nature of the unknown signals is available. In this paper we show a method for incorporating prior information about the mixing matrix to increase the levels of detection of responses to visual stimuli. Experimentally, our method matches the performance of known ICA algorithms for high SNR and can greatly improve the performance for low levels of SNR or low levels of signal-to-background ratio (SBR). For the problem of signal extraction, we have achieved detection for signals as small as 0.01% (-40 dB SBR) in hybrid live/synthetic data simulations. In experiments using a functional imager of the retina, measured changes in reflectance in response to visual stimulus are in the order of 0.1-1% of the total pixel intensity value, which makes the functional signal difficult to detect by standard methods. The results of the analysis show that using ICA-P signal levels of 0.1% can be detected. The approach also generalizes the standard Infomax algorithm which can be thought of as a special case of ICA-P when the confidence parameter or a tolerance value is zero. For in vivo animal experiments, we show that signal detection agreement over a range of confidence values parameters can be used to establish reflectance changes in response to the visual stimulus.

Multiscale AM-FM methods for diabetic retinopathy lesion detection.

In this paper, we propose the use of multiscale amplitude-modulation-frequency-modulation (AM-FM) methods for discriminating between normal and pathological retinal images. The method presented in this paper is tested using standard images from the early treatment diabetic retinopathy study. We use 120 regions of 40 x 40 pixels containing four types of lesions commonly associated with diabetic retinopathy (DR) and two types of normal retinal regions that were manually selected by a trained analyst. The region types included microaneurysms, exudates, neovascularization on the retina, hemorrhages, normal retinal background, and normal vessels patterns. The cumulative distribution functions of the instantaneous amplitude, the instantaneous frequency magnitude, and the relative instantaneous frequency angle from multiple scales are used as texture feature vectors. We use distance metrics between the extracted feature vectors to measure interstructure similarity. Our results demonstrate a statistical differentiation of normal retinal structures and pathological lesions based on AM-FM features. We further demonstrate our AM-FM methodology by applying it to classification of retinal images from the MESSIDOR database. Overall, the proposed methodology shows significant capability for use in automatic DR screening.

Noninvasive functional imaging of the retina reveals outer retinal and hemodynamic intrinsic optical signal origins.

We have adapted intrinsic signal optical imaging of neural activity to the noninvasive functional imaging of the retina. Results to date demonstrate the feasibility and potential of this new method of functional assessment of the retina. In response to visual stimuli, we have imaged reflectance changes in the retina that are robust and spatially colocalized to the sites of stimulation. However, the technique is in its infancy and many questions as to the underlying mechanisms remain. In particular, the source and nature of the activity-dependent intrinsic optical signals in the retina need to be characterized and their anatomic origins determined. The studies described here begin to address these issues. The evidence indicates that the imaged signals are driven by the outer retinal layers and have a dominant hemodynamic component.

Stimulus-evoked intrinsic optical signals in the retina: pharmacologic dissection reveals outer retinal origins.

PURPOSE: To elucidate the anatomic origins of stimulus-evoked intrinsic optical signals in the mammalian retina by using selective pharmacologic blockade of specific retinal layers. METHODS: Four adult cats were used to investigate the stimulus-evoked intrinsic signals. The retinas were visually stimulated with a liquid crystal display (LCD) integrated into a modified fundus camera. The evoked signals in the near infrared (NIR) were recorded with a digital camera to image the changes in the optical reflectance of the retinas. Variants of the electroretinogram (pattern ERG and long-pulse ERG) were also recorded as additional measures of retinal function. Specific retinal layers were inactivated via intravitreal injections of the voltage-gated sodium channel blocker, tetrodotoxin (TTX), the metabotropic glutamate receptor (mGluR6) agonist, 2-amino-4-phosphonobutyric acid (APB), and/or the ionotropic glutamate receptor antagonist cis-2,3 piperidinedicarboxylic acid (PDA). The stimulus-evoked intrinsic signals were imaged before and after drug injection. RESULTS: ERG recordings and tests of the consensual pupillary response confirmed the effectiveness of each drug. Yet despite the pharmacologic blockade of the inner retina (TTX) and postreceptoral retinal circuitry (APB and PDA), the stimulus-evoked intrinsic signals remained essentially unaltered from preinjection conditions. Similarly, the time course of the signal did not appreciably shift in time or shape. CONCLUSIONS: The findings demonstrate that stimulus-evoked intrinsic signals persist after injection of APB, PDA, and TTX, drugs that work to suppress inner and postreceptoral retinal circuitry. The persistence of the intrinsic signals after administration of these drugs indicates that the dominant intrinsic signals are likely to arise from the outer retina.

Stimulus-evoked intrinsic optical signals in the retina: spatial and temporal characteristics.

PURPOSE: To characterize the properties of stimulus-evoked retinal intrinsic signals and determine the underlying origins. METHODS: Seven adult cats were anesthetized and paralyzed to maximize imaging stability. The retina was stimulated with a liquid crystal display (LCD) integrated into a modified fundus camera (Topcon, Tokyo, Japan). The LCD presented patterned visual stimuli while the retina was illuminated with near infrared (NIR) light. The peristimulus changes in the NIR reflectance of the retina were recorded with a digital camera. RESULTS: Two stimulus-evoked reflectance signals in the NIR were observed: a positive signal, corresponding to a relative increase in reflectance, and a negative signal, corresponding to a relative decrease in reflectance. When presented with a positive-contrast stimulus, the negative reflectance signals showed a tight spatial coupling with the stimulated region of retina, whereas the positive signals arose in an adjacent region of the retina. Signals remained spatially confined to the stimulated region even when stimuli of much longer duration were used. In addition, the positive and negative signal polarities reversed when the stimulus contrast was inverted. Both signals showed a rise time on the order of seconds, similar to those observed in the mammalian neocortex. The spectral dependency of the signals on illumination was similar to the absorbance spectra of hemoglobin and the oximetric relationship. CONCLUSIONS: The findings characterize the basic properties of stimulus-evoked intrinsic signals of the retina. These signals were generally similar to the more extensively studied cortical signals. Collectively, the data suggest a hemodynamic component to the intrinsic optical signals of the retina.

Spatiotemporal independent component analysis for the detection of functional responses in cat retinal images.

In the early stages of some retinal diseases, such as glaucoma, loss of retinal activity may be difficult to detect with current clinical instruments. Because current instruments require unattainable levels of patient cooperation, high sensitivity and specificity are difficult to attain. We have devised a new retinal imaging system that detects intrinsic optical signals which reflect functional changes in the retina and that do not require patient cooperation. Measured changes in reflectance in response to the visual stimulus are on the order of 0.1%-1% of the total reflected intensity level, which makes the functional signal difficult to detect by standard methods. The desired functional signal is masked by other physiological signals and by imaging system noise. In this paper, we quantify the limits of independent component analysis (ICA) for detecting the low intensity functional signal and apply ICA to 60 video sequences from experiments using an anesthetized cat whose retina is presented with different patterned stimuli. The results of the analysis show that using ICA, in principle, signal levels of 0.1% can be detected. The study found that in 86% of the animal experiments the patterned stimuli effects on the retina can be detected and extracted.