Using a Deep Learning Algorithm and Integrated Gradients Explanation to Assist Grading for Diabetic Retinopathy.

PURPOSE: To understand the impact of deep learning diabetic retinopathy (DR) algorithms on physician readers in computer-assisted settings. DESIGN: Evaluation of diagnostic technology. PARTICIPANTS: One thousand seven hundred ninety-six retinal fundus images from 1612 diabetic patients. METHODS: Ten ophthalmologists (5 general ophthalmologists, 4 retina specialists, 1 retina fellow) read images for DR severity based on the International Clinical Diabetic Retinopathy disease severity scale in each of 3 conditions: unassisted, grades only, or grades plus heatmap. Grades-only assistance comprised a histogram of DR predictions (grades) from a trained deep-learning model. For grades plus heatmap, we additionally showed explanatory heatmaps. MAIN OUTCOME MEASURES: For each experiment arm, we computed sensitivity and specificity of each reader and the algorithm for different levels of DR severity against an adjudicated reference standard. We also measured accuracy (exact 5-class level agreement and Cohen's quadratically weighted κ), reader-reported confidence (5-point Likert scale), and grading time. RESULTS: Readers graded more accurately with model assistance than without for the grades-only condition (P < 0.001). Grades plus heatmaps improved accuracy for patients with DR (P < 0.001), but reduced accuracy for patients without DR (P = 0.006). Both forms of assistance increased readers' sensitivity moderate-or-worse DR: unassisted: mean, 79.4% [95% confidence interval (CI), 72.3%-86.5%]; grades only: mean, 87.5% [95% CI, 85.1%-89.9%]; grades plus heatmap: mean, 88.7% [95% CI, 84.9%-92.5%] without a corresponding drop in specificity (unassisted: mean, 96.6% [95% CI, 95.9%-97.4%]; grades only: mean, 96.1% [95% CI, 95.5%-96.7%]; grades plus heatmap: mean, 95.5% [95% CI, 94.8%-96.1%]). Algorithmic assistance increased the accuracy of retina specialists above that of the unassisted reader or model alone; and increased grading confidence and grading time across all readers. For most cases, grades plus heatmap was only as effective as grades only. Over the course of the experiment, grading time decreased across all conditions, although most sharply for grades plus heatmap. CONCLUSIONS: Deep learning algorithms can improve the accuracy of, and confidence in, DR diagnosis in an assisted read setting. They also may increase grading time, although these effects may be ameliorated with experience.

Monocular and binocular low-contrast visual acuity and optical coherence tomography in pediatric multiple sclerosis.

BACKGROUND: Low-contrast letter acuity and optical coherence tomography (OCT) capture visual dysfunction and axonal loss in adult-onset multiple sclerosis (MS), and have been proposed as secondary outcome metrics for therapeutic trials. Clinical trials will soon be launched in pediatric MS, but such outcome metrics have not been well-validated in this population. OBJECTIVES: To determine whether MS onset during childhood and adolescence is associated with measurable loss of visual acuity and thinning of the retinal nerve fiber layer (RNFL), whether such features are noted only in the context of clinical optic nerve inflammation (optic neuritis, ON) or are a feature of MS even in the absence of optic nerve relapses, and to define the optimal methods for such detection. STUDY DESIGN: Cross-sectional study. METHODS: Monocular and binocular high- and low-contrast letter acuity and contrast sensitivity were assessed in a cross-sectional cohort of children (ages 5 to 17 years) with MS (N=22 patients, 44 eyes; 8 patients with a history of ON) and disease-free controls (N=29 patients; 58 eyes) from three academic centers. Binocular summation was determined by calculating the number of letters correctly identified using the binocular score minus the better eye score for each visual test. RNFL thickness was measured using OCT (Stratus OCT-3). Results were analyzed in terms of "eyes" as: MS ON+, MS ON-, and control eyes. Generalized estimating equation (GEE) regression models were used to compare patients to controls. RESULTS: Traditional high-contrast visual acuity scores did not differ between MS ON+, MS ON-, and controls eyes. MS ON+ eyes had decreased monocular (p<0.001) and decreased binocular (p=0.007) low-contrast letter acuity (Sloan 1.25% contrast charts) scores. Monocular visual acuity did not differ when comparing MS ON- and control eyes. The magnitude of binocular summation using low-contrast charts was similar for pediatric MS participants and controls and was not diminished in children with a history of ON. While the mean RNFL thickness for all MS eyes (103±17 µm) trended lower when compared to corresponding measures in control eyes (109±9 µm, p=0.085), we confirmed a highly significant reduction in mean RNFL thickness in MS eyes with a history of ON (86±22 µm, p<0.001). RNFL thickness of MS ON- eyes in pediatric MS patients (109±11 µm) did not differ from controls (p=0.994). CONCLUSIONS: Low-contrast letter acuity detects subtle visual loss in MS patients with prior ON, consistent with incomplete recovery, a finding further supported by RNFL loss in ON affected eyes. In MS patients with prior unilateral ON, binocular acuity is decreased; however, the magnitude of binocular summation is preserved, unlike adult-onset MS who exhibit a reduced capacity for visual compensation in the context of unilateral injury. Also unlike findings in adult-onset MS, we did not demonstrate RNFL thinning in ON- eyes of children and adolescents with MS. Further validation is required to confirm whether neurodegeneration of visual pathways occurs in the absence of relapse, and thus whether OCT will serve as a sensitive metric for such pathology in the pediatric and adolescent MS context.

Parent and medical professional willingness to enroll children in a hypothetical pediatric optic neuritis treatment trial.

The optic neuritis treatment trial (ONTT) and subsequent studies have had a tremendous impact on the treatment and prognosis of optic neuritis and multiple sclerosis in adults. The results of these studies have been extrapolated to children; however, pediatric data are sparse. Using the method of prospective preference assessment, the willingness of parents and medical professionals to enroll children in a hypothetical Pediatric ONTT was assessed using a mock consent form and questionnaire. A three-arm trial was proposed: (1) intravenous corticosteroids, (2) high-dose oral corticosteroids, and (3) an oral placebo. The forms were completed by 198 parents and 49 physicians. After reviewing the hypothetical scenario, trial design, risks and benefits, and alternatives to the study, 21% of parents would enroll their children in the trial whereas 98% of medical professionals would enroll their patients. With medical professional recommendation, 43% of parents would enroll their children. The manner in which this hypothetical trial was presented to parents, specifically with respect to the recommendation of their child's health care team, influenced a parent's willingness to participate.

Worsening renal function defined as an absolute increase in serum creatinine is a biased metric for the study of cardio-renal interactions.

OBJECTIVES: Worsening renal function (WRF) during the treatment of decompensated heart failure, frequently defined as an absolute increase in serum creatinine >or=0.3 mg/dl, has been reported as a strong adverse prognostic factor in several studies. We hypothesized that this definition of WRF is biased by baseline renal function secondary to the exponential relationship between creatinine and renal function. METHODS: We reviewed consecutive admissions with a discharge diagnosis of heart failure. An increase in creatinine >or=0.3 mg/dl (WRF(CREAT)) was compared to a decrease in GFR >or=20% (WRF(GFR)). RESULTS: Overall, 993 admissions met eligibility. WRF(CREAT) occurred in 31.5% and WRF(GFR) in 32.7%. WRF(CREAT) and WRF(GFR) had opposing relationships with baseline renal function (OR = 1.9 vs. OR = 0.51, respectively, p < 0.001). Both definitions had similar unadjusted associations with death at 30 days [WRF(GFR) OR = 2.3 (95% CI 1.1-4.8), p = 0.026; WRF(CREAT) OR = 2.1 (95% CI 1.0-4.4), p = 0.047]. Controlling for baseline renal insufficiency, WRF(GFR) added incrementally in the prediction of mortality (p = 0.009); however, WRF(CREAT) did not (p = 0.11). CONCLUSIONS: WRF, defined as an absolute change in serum creatinine, is heavily biased by baseline renal function. An alternative definition of WRF should be considered for future studies of cardio-renal interactions.