Object Boundary Detection in Natural Images May Depend on "Incitatory" Cell-Cell Interactions.

Detecting object boundaries is crucial for recognition, but how the process unfolds in visual cortex remains unknown. To study the problem faced by a hypothetical boundary cell, and to predict how cortical circuitry could produce a boundary cell from a population of conventional "simple cells," we labeled 30,000 natural image patches and used Bayes' rule to help determine how a simple cell should influence a nearby boundary cell depending on its relative offset in receptive field position and orientation. We identified the following three basic types of cell-cell interactions: rising and falling interactions with a range of slopes and saturation rates, and nonmonotonic (bump-shaped) interactions with varying modes and amplitudes. Using simple models, we show that a ubiquitous cortical circuit motif consisting of direct excitation and indirect inhibition-a compound effect we call "incitation"-can produce the entire spectrum of simple cell-boundary cell interactions found in our dataset. Moreover, we show that the synaptic weights that parameterize an incitation circuit can be learned by a single-layer "delta" rule. We conclude that incitatory interconnections are a generally useful computing mechanism that the cortex may exploit to help solve difficult natural classification problems.SIGNIFICANCE STATEMENT Simple cells in primary visual cortex (V1) respond to oriented edges and have long been supposed to detect object boundaries, yet the prevailing model of a simple cell-a divisively normalized linear filter-is a surprisingly poor natural boundary detector. To understand why, we analyzed image statistics on and off object boundaries, allowing us to characterize the neural-style computations needed to perform well at this difficult natural classification task. We show that a simple circuit motif known to exist in V1 is capable of extracting high-quality boundary probability signals from local populations of simple cells. Our findings suggest a new, more general way of conceptualizing cell-cell interconnections in the cortex.

Computing local edge probability in natural scenes from a population of oriented simple cells.

A key computation in visual cortex is the extraction of object contours, where the first stage of processing is commonly attributed to V1 simple cells. The standard model of a simple cell-an oriented linear filter followed by a divisive normalization-fits a wide variety of physiological data, but is a poor performing local edge detector when applied to natural images. The brain's ability to finely discriminate edges from nonedges therefore likely depends on information encoded by local simple cell populations. To gain insight into the corresponding decoding problem, we used Bayes's rule to calculate edge probability at a given location/orientation in an image based on a surrounding filter population. Beginning with a set of ∼ 100 filters, we culled out a subset that were maximally informative about edges, and minimally correlated to allow factorization of the joint on- and off-edge likelihood functions. Key features of our approach include a new, efficient method for ground-truth edge labeling, an emphasis on achieving filter independence, including a focus on filters in the region orthogonal rather than tangential to an edge, and the use of a customized parametric model to represent the individual filter likelihood functions. The resulting population-based edge detector has zero parameters, calculates edge probability based on a sum of surrounding filter influences, is much more sharply tuned than the underlying linear filters, and effectively captures fine-scale edge structure in natural scenes. Our findings predict nonmonotonic interactions between cells in visual cortex, wherein a cell may for certain stimuli excite and for other stimuli inhibit the same neighboring cell, depending on the two cells' relative offsets in position and orientation, and their relative activation levels.

Artificial Intelligence Detection of Diabetic Retinopathy: Subgroup Comparison of the EyeArt System with Ophthalmologists' Dilated Examinations.

Objective: To compare general ophthalmologists, retina specialists, and the EyeArt Artificial Intelligence (AI) system to the clinical reference standard for detecting more than mild diabetic retinopathy (mtmDR). Design: Prospective, pivotal, multicenter trial conducted from April 2017 to May 2018. Participants: Participants were aged ≥ 18 years who had diabetes mellitus and underwent dilated ophthalmoscopy. A total of 521 of 893 participants met these criteria and completed the study protocol. Testing: Participants underwent 2-field fundus photography (macula centered, disc centered) for the EyeArt system, dilated ophthalmoscopy, and 4-widefield stereoscopic dilated fundus photography for reference standard grading. Main Outcome Measures: For mtmDR detection, sensitivity and specificity of EyeArt gradings of 2-field, fundus photographs and ophthalmoscopy grading versus a rigorous clinical reference standard comprising Reading Center grading of 4-widefield stereoscopic dilated fundus photographs using the ETDRS severity scale. The AI system provided automatic eye-level results regarding mtmDR. Results: Overall, 521 participants (999 eyes) at 10 centers underwent dilated ophthalmoscopy: 406 by nonretina and 115 by retina specialists. Reading Center graded 207 positive and 792 eyes negative for mtmDR. Of these 999 eyes, 26 eyes were ungradable by the EyeArt system, leaving 973 eyes with both EyeArt and Reading Center gradings. Retina specialists correctly identified 22 of 37 eyes as positive (sensitivity 59.5%) and 182 of 184 eyes as negative (specificity 98.9%) for mtmDR versus the EyeArt AI system that identified 36 of 37 as positive (sensitivity 97%) and 162 of 184 eyes as negative (specificity of 88%) for mtmDR. General ophthalmologists correctly identified 35 of 170 eyes as positive (sensitivity 20.6%) and 607 of 608 eyes as negative (specificity 99.8%) for mtmDR compared with the EyeArt AI system that identified 164 of 170 as positive (sensitivity 96.5%) and 525 of 608 eyes as negative (specificity 86%) for mtmDR. Conclusions: The AI system had a higher sensitivity for detecting mtmDR than either general ophthalmologists or retina specialists compared with the clinical reference standard. It can potentially serve as a low-cost point-of-care diabetic retinopathy detection tool and help address the diabetic eye screening burden.

Comparison of diabetic retinopathy severity grading on ETDRS 7-field versus ultrawide-field assessment.

OBJECTIVES: To compare the diabetic retinopathy (DR) severity level determined when considering only the ETDRS 7-field region versus the entire ultrawidefield (UWF) image. METHODS: In this retrospective, cross-sectional study, UWF pseudocolor images were graded on the Eyenuk image viewing, grading, and annotation platform for the severity of DR considering only the regions within the ETDRS 7-fields as well as the entire UWF image using two different protocols: 1) the simple International Classification of Diabetic Retinopathy (ICDR) scale and 2) the more complex DRCR.net Protocol AA grading scale. RESULTS: A total of 250 eyes from 157 patients were included in this analysis. Six eyes (2.4%) demonstrated a discrepancy in severity level between the ETDRS 7-field region and the entire UWF image when using the ICDR classification system. The discrepancies were due to the presence of lesions [intraretinal haemorrhage (n = 2), neovascular disease (n = 4)] in the peripheral fields which were not identified in the ETDRS 7-fields. Fourteen eyes (5.6%) had a discrepancy in severity level between the ETDRS 7-field region and the entire UWF image when using the ETDRS DRSS Protocol AA grading scale. The discrepancies were due to the presence of a higher level of disease [intraretinal haemorrhage (n = 4), neovascularization (n = 4), preretinal haemorrhage (n = 2), scatter laser scars (n = 4)] in the peripheral fields. CONCLUSION: Although considering regions outside of the ETDRS 7-fields altered the DR severity level assessment in <5% of cases in this cohort, significant and potentially vision-threatening lesions including neovascularization and preretinal haemorrhage were identified in these peripheral regions. This highlights the importance of evaluating the entire UWF region when assessing patients with diabetic retinopathy.

Contour enhancement benefits older adults with simulated central field loss.

PURPOSE: Age-related macular degeneration is the leading cause of vision loss among Americans aged >65 years. Currently, no effective treatment can reverse the central vision loss associated with most age-related macular degeneration. Digital image-processing techniques have been developed to improve image visibility for peripheral vision; however, both the selection and efficacy of such methods are limited. Progress has been difficult for two reasons: the exact nature of image enhancement that might benefit peripheral vision is not well understood, and efficient methods for testing such techniques have been elusive. The current study aims to develop both an effective image enhancement technique for peripheral vision and an efficient means for validating the technique. METHODS: We used a novel contour-detection algorithm to locate shape-defining edges in images based on natural-image statistics. We then enhanced the scene by locally boosting the luminance contrast along such contours. Using a gaze-contingent display, we simulated central visual field loss in normally sighted young (aged 18-30 years) and older adults (aged 58-88 years). Visual search performance was measured as a function of contour enhancement strength ["original" (unenhanced), "medium," and "high"]. For preference task, a separate group of subjects judged which image in a pair "would lead to better search performance." RESULTS: We found that although contour enhancement had no significant effect on search time and accuracy in young adults, Medium enhancement resulted in significantly shorter search time in older adults (about 13% reduction relative to original). Both age-groups preferred images with Medium enhancement over original (2-7 times). Furthermore, across age-groups, image content types, and enhancement strengths, there was a robust correlation between preference and performance. CONCLUSIONS: Our findings demonstrate a beneficial role of contour enhancement in peripheral vision for older adults. Our findings further suggest that task-specific preference judgments can be an efficient surrogate for performance testing.

Effects of contour enhancement on low-vision preference and visual search.

PURPOSE: To determine whether image enhancement improves visual search performance and whether enhanced images were also preferred by subjects with vision impairment. METHODS: Subjects (n = 24) with vision impairment (vision: 20/52 to 20/240) completed visual search and preference tasks for 150 static images that were enhanced to increase object contours' visual saliency. Subjects were divided into two groups and were shown three enhancement levels. Original and medium enhancements were shown to both groups. High enhancement was shown to group 1, and low enhancement was shown to group 2. For search, subjects pointed to an object that matched a search target displayed at the top left of the screen. An "integrated search performance" measure (area under the curve of cumulative correct response rate over search time) quantified performance. For preference, subjects indicated the preferred side when viewing the same image with different enhancement levels on side-by-side high-definition televisions. RESULTS: Contour enhancement did not improve performance in the visual search task. Group 1 subjects significantly (p < 0.001) rejected the High enhancement, and showed no preference for medium enhancement over the original images. Group 2 subjects significantly preferred (p < 0.001) both the medium and the low enhancement levels over original. Contrast sensitivity was correlated with both preference and performance; subjects with worse contrast sensitivity performed worse in the search task (ρ = 0.77, p < 0.001) and preferred more enhancement (ρ = -0.47, p = 0.02). No correlation between visual search performance and enhancement preference was found. However, a small group of subjects (n = 6) in a narrow range of mid-contrast sensitivity performed better with the enhancement, and most (n = 5) also preferred the enhancement. CONCLUSIONS: Preferences for image enhancement can be dissociated from search performance in people with vision impairment. Further investigations are needed to study the relationships between preference and performance for a narrow range of mid-contrast sensitivity where a beneficial effect of enhancement may exist.

Classical-Contextual Interactions in V1 May Rely on Dendritic Computations.

A signature feature of the neocortex is the dense network of horizontal connections (HCs) through which pyramidal neurons (PNs) exchange "contextual" information. In primary visual cortex (V1), HCs are thought to facilitate boundary detection, a crucial operation for object recognition, but how HCs modulate PN responses to boundary cues within their classical receptive fields (CRF) remains unknown. We began by "asking" natural images, through a structured data collection and ground truth labeling process, what function a V1 cell should use to compute boundary probability from aligned edge cues within and outside its CRF. The "answer" was an asymmetric 2-D sigmoidal function, whose nonlinear form provides the first normative account for the "multiplicative" center-flanker interactions previously reported in V1 neurons (Kapadia et al., 1995, 2000; Polat et al., 1998). Using a detailed compartmental model, we then show that this boundary-detecting classical-contextual interaction function can be computed by NMDAR-dependent spatial synaptic interactions within PN dendrites - the site where classical and contextual inputs first converge in the cortex. In additional simulations, we show that local interneuron circuitry activated by HCs can powerfully leverage the nonlinear spatial computing capabilities of PN dendrites, providing the cortex with a highly flexible substrate for integration of classical and contextual information.

Pivotal Evaluation of an Artificial Intelligence System for Autonomous Detection of Referrable and Vision-Threatening Diabetic Retinopathy.

Importance: Diabetic retinopathy (DR) is a leading cause of blindness in adults worldwide. Early detection and intervention can prevent blindness; however, many patients do not receive their recommended annual diabetic eye examinations, primarily owing to limited access. Objective: To evaluate the safety and accuracy of an artificial intelligence (AI) system (the EyeArt Automated DR Detection System, version 2.1.0) in detecting both more-than-mild diabetic retinopathy (mtmDR) and vision-threatening diabetic retinopathy (vtDR). Design, Setting, and Participants: A prospective multicenter cross-sectional diagnostic study was preregistered (NCT03112005) and conducted from April 17, 2017, to May 30, 2018. A total of 942 individuals aged 18 years or older who had diabetes gave consent to participate at 15 primary care and eye care facilities. Data analysis was performed from February 14 to July 10, 2019. Interventions: Retinal imaging for the autonomous AI system and Early Treatment Diabetic Retinopathy Study (ETDRS) reference standard determination. Main Outcomes and Measures: Primary outcome measures included the sensitivity and specificity of the AI system in identifying participants' eyes with mtmDR and/or vtDR by 2-field undilated fundus photography vs a rigorous clinical reference standard comprising reading center grading of 4 wide-field dilated images using the ETDRS severity scale. Secondary outcome measures included the evaluation of imageability, dilated-if-needed analysis, enrichment correction analysis, worst-case imputation, and safety outcomes. Results: Of 942 consenting individuals, 893 patients (1786 eyes) met the inclusion criteria and completed the study protocol. The population included 449 men (50.3%). Mean (SD) participant age was 53.9 (15.2) years (median, 56; range, 18-88 years), 655 were White (73.3%), and 206 had type 1 diabetes (23.1%). Sensitivity and specificity of the AI system were high in detecting mtmDR (sensitivity: 95.5%; 95% CI, 92.4%-98.5% and specificity: 85.0%; 95% CI, 82.6%-87.4%) and vtDR (sensitivity: 95.1%; 95% CI, 90.1%-100% and specificity: 89.0%; 95% CI, 87.0%-91.1%) without dilation. Imageability was high without dilation, with the AI system able to grade 87.4% (95% CI, 85.2%-89.6%) of the eyes with reading center grades. When eyes with ungradable results were dilated per the protocol, the imageability improved to 97.4% (95% CI, 96.4%-98.5%), with the sensitivity and specificity being similar. After correcting for enrichment, the mtmDR specificity increased to 87.8% (95% CI, 86.3%-89.5%) and the sensitivity remained similar; for vtDR, both sensitivity (97.0%; 95% CI, 91.2%-100%) and specificity (90.1%; 95% CI, 89.4%-91.5%) improved. Conclusions and Relevance: This prospective multicenter cross-sectional diagnostic study noted safety and accuracy with use of the EyeArt Automated DR Detection System in detecting both mtmDR and, for the first time, vtDR, without physician assistance. These findings suggest that improved access to accurate, reliable diabetic eye examinations may increase adherence to recommended annual screenings and allow for accelerated referral of patients identified as having vtDR.

Comparison of automated and expert human grading of diabetic retinopathy using smartphone-based retinal photography.

PURPOSE: The aim of this study is to investigate the efficacy of a mobile platform that combines smartphone-based retinal imaging with automated grading for determining the presence of referral-warranted diabetic retinopathy (RWDR). METHODS: A smartphone-based camera (RetinaScope) was used by non-ophthalmic personnel to image the retina of patients with diabetes. Images were analyzed with the Eyenuk EyeArt® system, which generated referral recommendations based on presence of diabetic retinopathy (DR) and/or markers for clinically significant macular oedema. Images were independently evaluated by two masked readers and categorized as refer/no refer. The accuracies of the graders and automated interpretation were determined by comparing results to gold standard clinical diagnoses. RESULTS: A total of 119 eyes from 69 patients were included. RWDR was present in 88 eyes (73.9%) and in 54 patients (78.3%). At the patient-level, automated interpretation had a sensitivity of 87.0% and specificity of 78.6%; grader 1 had a sensitivity of 96.3% and specificity of 42.9%; grader 2 had a sensitivity of 92.5% and specificity of 50.0%. At the eye-level, automated interpretation had a sensitivity of 77.8% and specificity of 71.5%; grader 1 had a sensitivity of 94.0% and specificity of 52.2%; grader 2 had a sensitivity of 89.5% and specificity of 66.9%. DISCUSSION: Retinal photography with RetinaScope combined with automated interpretation by EyeArt achieved a lower sensitivity but higher specificity than trained expert graders. Feasibility testing was performed using non-ophthalmic personnel in a retina clinic with high disease burden. Additional studies are needed to assess efficacy of screening diabetic patients from general population.

The Value of Automated Diabetic Retinopathy Screening with the EyeArt System: A Study of More Than 100,000 Consecutive Encounters from People with Diabetes.

Background: Current manual diabetic retinopathy (DR) screening using eye care experts cannot scale to screen the growing population of diabetes patients who are at risk for vision loss. EyeArt system is an automated, cloud-based artificial intelligence (AI) eye screening technology designed to easily detect referral-warranted DR immediately through automated analysis of patient's retinal images. Methods: This retrospective study assessed the diagnostic efficacy of the EyeArt system v2.0 analyzing 850,908 fundus images from 101,710 consecutive patient visits, collected from 404 primary care clinics. Presence or absence of referral-warranted DR (more than mild nonproliferative DR [NPDR]) was automatically detected by the EyeArt system for each patient encounter, and its performance was compared against a clinical reference standard of quality-assured grading by rigorously trained certified ophthalmologists and optometrists. Results: Of the 101,710 visits, 75.7% were nonreferable, 19.3% were referable to an eye care specialist, and in 5.0%, the DR level was unknown as per the clinical reference standard. EyeArt screening had 91.3% (95% confidence interval [CI]: 90.9-91.7) sensitivity and 91.1% (95% CI: 90.9-91.3) specificity. For 5446 encounters with potentially treatable DR (more than moderate NPDR and/or diabetic macular edema), the system provided a positive "refer" output to 5363 encounters achieving sensitivity of 98.5%. Conclusions: This study captures variations in real-world clinical practice and shows that an AI DR screening system can be safe and effective in the real world. This study demonstrates the value of this easy-to-use, automated tool for endocrinologists, diabetologists, and general practitioners to address the growing need for DR screening and monitoring.

Automated detection of diabetic retinopathy lesions on ultrawidefield pseudocolour images.

PURPOSE: We examined the sensitivity and specificity of an automated algorithm for detecting referral-warranted diabetic retinopathy (DR) on Optos ultrawidefield (UWF) pseudocolour images. METHODS: Patients with diabetes were recruited for UWF imaging. A total of 383 subjects (754 eyes) were enrolled. Nonproliferative DR graded to be moderate or higher on the 5-level International Clinical Diabetic Retinopathy (ICDR) severity scale was considered as grounds for referral. The software automatically detected DR lesions using the previously trained classifiers and classified each image in the test set as referral-warranted or not warranted. Sensitivity, specificity and the area under the receiver operating curve (AUROC) of the algorithm were computed. RESULTS: The automated algorithm achieved a 91.7%/90.3% sensitivity (95% CI 90.1-93.9/80.4-89.4) with a 50.0%/53.6% specificity (95% CI 31.7-72.8/36.5-71.4) for detecting referral-warranted retinopathy at the patient/eye levels, respectively; the AUROC was 0.873/0.851 (95% CI 0.819-0.922/0.804-0.894). CONCLUSION: Diabetic retinopathy (DR) lesions were detected from Optos pseudocolour UWF images using an automated algorithm. Images were classified as referral-warranted DR with a high degree of sensitivity and moderate specificity. Automated analysis of UWF images could be of value in DR screening programmes and could allow for more complete and accurate disease staging.

Automated Diabetic Retinopathy Screening and Monitoring Using Retinal Fundus Image Analysis.

BACKGROUND: Diabetic retinopathy (DR)-a common complication of diabetes-is the leading cause of vision loss among the working-age population in the western world. DR is largely asymptomatic, but if detected at early stages the progression to vision loss can be significantly slowed. With the increasing diabetic population there is an urgent need for automated DR screening and monitoring. To address this growing need, in this article we discuss an automated DR screening tool and extend it for automated estimation of microaneurysm (MA) turnover, a potential biomarker for DR risk. METHODS: The DR screening tool automatically analyzes color retinal fundus images from a patient encounter for the various DR pathologies and collates the information from all the images belonging to a patient encounter to generate a patient-level screening recommendation. The MA turnover estimation tool aligns retinal images from multiple encounters of a patient, localizes MAs, and performs MA dynamics analysis to evaluate new, persistent, and disappeared lesion maps and estimate MA turnover rates. RESULTS: The DR screening tool achieves 90% sensitivity at 63.2% specificity on a data set of 40 542 images from 5084 patient encounters obtained from the EyePACS telescreening system. On a subset of 7 longitudinal pairs the MA turnover estimation tool identifies new and disappeared MAs with 100% sensitivity and average false positives of 0.43 and 1.6 respectively. CONCLUSIONS: The presented automated tools have the potential to address the growing need for DR screening and monitoring, thereby saving vision of millions of diabetic patients worldwide.

Temporal profiles and 2-dimensional oxy-, deoxy-, and total-hemoglobin somatosensory maps in rat versus mouse cortex.

BACKGROUND: Mechanisms of neurovascular coupling-the relationship between neuronal chemoelectrical activity and compensatory metabolic and hemodynamic changes-appear to be preserved across species from rats to humans despite differences in scale. However, previous work suggests that the highly cellular dense mouse somatosensory cortex has different functional hemodynamic changes compared to other species. METHODS: We developed novel hardware and software for 2-dimensional optical spectroscopy (2DOS). Optical changes at four simultaneously recorded wavelengths were measured in both rat and mouse primary somatosensory cortex (S1) evoked by forepaw stimulation to create four spectral maps. The spectral maps were converted to maps of deoxy-, oxy-, and total-hemoglobin (HbR, HbO, and HbT) concentration changes using the modified Beer-Lambert law and phantom HbR and HbO absorption spectra. RESULTS: : Functional hemodynamics were different in mouse versus rat neocortex. On average, hemodynamics were as expected in rat primary somatosensory cortex (S1): the fractional change in the log of HbT concentration increased monophasically 2 s after stimulus, whereas HbO changes mirrored HbR changes, with HbO showing a small initial dip at 0.5 s followed by a large increase 3.0 s post stimulus. In contrast, mouse S1 showed a novel type of stimulus-evoked hemodynamic response, with prolonged, concurrent, monophasic increases in HbR and HbT and a parallel decrease in HbO that all peaked 3.5-4.5 s post stimulus onset. For rats, at any given time point, the average size and shape of HbO and HbR forepaw maps were the same, whereas surface veins distorted the shape of the HbT map. For mice, HbO, HbR, and HbT forepaw maps were generally the same size and shape at any post-stimulus time point. CONCLUSIONS: 2DOS using image splitting optics is feasible across species for brain mapping and quantifying the map topography of cortical hemodynamics. These results suggest that during physiologic stimulation, different species and/or cortical architecture may give rise to different hemodynamic changes during neurovascular coupling.