Cortical Motion Perception Emerges from Dimensionality Reduction with Evolved Spike-Timing-Dependent Plasticity Rules.

The nervous system is under tight energy constraints and must represent information efficiently. This is particularly relevant in the dorsal part of the medial superior temporal area (MSTd) in primates where neurons encode complex motion patterns to support a variety of behaviors. A sparse decomposition model based on a dimensionality reduction principle known as non-negative matrix factorization (NMF) was previously shown to account for a wide range of monkey MSTd visual response properties. This model resulted in sparse, parts-based representations that could be regarded as basis flow fields, a linear superposition of which accurately reconstructed the input stimuli. This model provided evidence that the seemingly complex response properties of MSTd may be a by-product of MSTd neurons performing dimensionality reduction on their input. However, an open question is how a neural circuit could carry out this function. In the current study, we propose a spiking neural network (SNN) model of MSTd based on evolved spike-timing-dependent plasticity and homeostatic synaptic scaling (STDP-H) learning rules. We demonstrate that the SNN model learns compressed and efficient representations of the input patterns similar to the patterns that emerge from NMF, resulting in MSTd-like receptive fields observed in monkeys. This SNN model suggests that STDP-H observed in the nervous system may be performing a similar function as NMF with sparsity constraints, which provides a test bed for mechanistic theories of how MSTd may efficiently encode complex patterns of visual motion to support robust self-motion perception.SIGNIFICANCE STATEMENT The brain may use dimensionality reduction and sparse coding to efficiently represent stimuli under metabolic constraints. Neurons in monkey area MSTd respond to complex optic flow patterns resulting from self-motion. We developed a spiking neural network model that showed MSTd-like response properties can emerge from evolving spike-timing-dependent plasticity with STDP-H parameters of the connections between then middle temporal area and MSTd. Simulated MSTd neurons formed a sparse, reduced population code capable of encoding perceptual variables important for self-motion perception. This model demonstrates that complex neuronal responses observed in MSTd may emerge from efficient coding and suggests that neurobiological plasticity, like STDP-H, may contribute to reducing the dimensions of input stimuli and allowing spiking neurons to learn sparse representations.

Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition.

Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.

A systematic review of extended reality (XR) for understanding and augmenting vision loss.

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.

Learning to see again: Perceptual learning of simulated abnormal on- off-cell population responses in sighted individuals.

Many forms of artificial sight recovery, such as electronic implants and optogenetic proteins, generally cause simultaneous, rather than complementary firing of on- and off-center retinal cells. Here, using virtual patients-sighted individuals viewing distorted input-we examine whether plasticity might compensate for abnormal neuronal population responses. Five participants were dichoptically presented with a combination of original and contrast-reversed images. Each image (I) and its contrast-reverse (I') was filtered using a radial checkerboard (F) in Fourier space and its inverse (F'). [I * F'] + [I' * F] was presented to one eye, and [I * F] + [I' * F'] was presented to the other, such that regions of the image that produced on-center responses in one eye produced off-center responses in the other eye, and vice versa. Participants continuously improved in a naturalistic object discrimination task over 20 one-hour sessions. Pre-training and post-training tests suggest that performance improvements were due to two learning processes: learning to recognize objects with reduced visual information and learning to suppress contrast-reversed image information in a non-eye-selective manner. These results suggest that, with training, it may be possible to adapt to the unnatural on- and off-cell population responses produced by electronic and optogenetic sight recovery technologies.

Neural correlates of sparse coding and dimensionality reduction.

Supported by recent computational studies, there is increasing evidence that a wide range of neuronal responses can be understood as an emergent property of nonnegative sparse coding (NSC), an efficient population coding scheme based on dimensionality reduction and sparsity constraints. We review evidence that NSC might be employed by sensory areas to efficiently encode external stimulus spaces, by some associative areas to conjunctively represent multiple behaviorally relevant variables, and possibly by the basal ganglia to coordinate movement. In addition, NSC might provide a useful theoretical framework under which to understand the often complex and nonintuitive response properties of neurons in other brain areas. Although NSC might not apply to all brain areas (for example, motor or executive function areas) the success of NSC-based models, especially in sensory areas, warrants further investigation for neural correlates in other regions.

3D Visual Response Properties of MSTd Emerge from an Efficient, Sparse Population Code.

UNLABELLED: Neurons in the dorsal subregion of the medial superior temporal (MSTd) area of the macaque respond to large, complex patterns of retinal flow, implying a role in the analysis of self-motion. Some neurons are selective for the expanding radial motion that occurs as an observer moves through the environment ("heading"), and computational models can account for this finding. However, ample evidence suggests that MSTd neurons exhibit a continuum of visual response selectivity to large-field motion stimuli. Furthermore, the underlying computational principles by which these response properties are derived remain poorly understood. Here we describe a computational model of macaque MSTd based on the hypothesis that neurons in MSTd efficiently encode the continuum of large-field retinal flow patterns on the basis of inputs received from neurons in MT with receptive fields that resemble basis vectors recovered with non-negative matrix factorization. These assumptions are sufficient to quantitatively simulate neurophysiological response properties of MSTd cells, such as 3D translation and rotation selectivity, suggesting that these properties might simply be a byproduct of MSTd neurons performing dimensionality reduction on their inputs. At the population level, model MSTd accurately predicts eye velocity and heading using a sparse distributed code, consistent with the idea that biological MSTd might be well equipped to efficiently encode various self-motion variables. The present work aims to add some structure to the often contradictory findings about macaque MSTd, and offers a biologically plausible account of a wide range of visual response properties ranging from single-unit selectivity to population statistics. SIGNIFICANCE STATEMENT: Using a dimensionality reduction technique known as non-negative matrix factorization, we found that a variety of medial superior temporal (MSTd) neural response properties could be derived from MT-like input features. The responses that emerge from this technique, such as 3D translation and rotation selectivity, spiral tuning, and heading selectivity, can account for a number of empirical results. These findings (1) provide a further step toward a scientific understanding of the often nonintuitive response properties of MSTd neurons; (2) suggest that response properties, such as complex motion tuning and heading selectivity, might simply be a byproduct of MSTd neurons performing dimensionality reduction on their inputs; and (3) imply that motion perception in the cortex is consistent with ideas from the efficient-coding and free-energy principles.

Neoadjuvant chemotherapy and extrapleural pneumonectomy of malignant pleural mesothelioma with or without hemithoracic radiotherapy (SAKK 17/04): a randomised, international, multicentre phase 2 trial.

BACKGROUND: Postoperative hemithoracic radiotherapy has been used to treat malignant pleural mesothelioma, but it has not been assessed in a randomised trial. We assessed high-dose hemithoracic radiotherapy after neoadjuvant chemotherapy and extrapleural pneumonectomy in patients with malignant pleural mesothelioma. METHODS: We did this phase 2 trial in two parts at 14 hospitals in Switzerland, Belgium, and Germany. We enrolled patients with pathologically confirmed malignant pleural mesothelioma; resectable TNM stages T1-3 N0-2, M0; WHO performance status 0-1; age 18-70 years. In part 1, patients were given three cycles of neoadjuvant chemotherapy (cisplatin 75 mg/m(2) and pemetrexed 500 mg/m(2) on day 1 given every 3 weeks) and extrapleural pneumonectomy; the primary endpoint was complete macroscopic resection (R0-1). In part 2, participants with complete macroscopic resection were randomly assigned (1:1) to receive high-dose radiotherapy or not. The target volume for radiotherapy encompassed the entire hemithorax, the thoracotomy channel, and mediastinal nodal stations if affected by the disease or violated surgically. A boost was given to areas at high risk for locoregional relapse. The allocation was stratified by centre, histology (sarcomatoid vs epithelioid or mixed), mediastinal lymph node involvement (N0-1 vs N2), and T stage (T1-2 vs T3). The primary endpoint of part 1 was the proportion of patients achieving complete macroscopic resection (R0 and R1). The primary endpoint in part 2 was locoregional relapse-free survival, analysed by intention to treat. The trial is registered with ClinicalTrials.gov, number NCT00334594. FINDINGS: We enrolled patients between Dec 7, 2005, and Oct 17, 2012. Overall, we analysed 151 patients receiving neoadjuvant chemotherapy, of whom 113 (75%) had extrapleural pneumonectomy. Median follow-up was 54·2 months (IQR 32-66). 52 (34%) of 151 patients achieved an objective response. The most common grade 3 or 4 toxic effects were neutropenia (21 [14%] of 151 patients), anaemia (11 [7%]), and nausea or vomiting (eight [5%]). 113 patients had extrapleural pneumonectomy, with complete macroscopic resection achieved in 96 (64%) of 151 patients. We enrolled 54 patients in part 2; 27 in each group. The main reasons for exclusion were patient refusal (n=20) and ineligibility (n=10). 25 of 27 patients completed radiotherapy. Median total radiotherapy dose was 55·9 Gy (IQR 46·8-56·0). Median locoregional relapse-free survival from surgery, was 7·6 months (95% CI 4·5-10·7) in the no radiotherapy group and 9·4 months (6·5-11·9) in the radiotherapy group. The most common grade 3 or higher toxic effects related to radiotherapy were nausea or vomiting (three [11%] of 27 patients), oesophagitis (two [7%]), and pneumonitis (two [7%]). One patient died of pneumonitis. We recorded no toxic effects data for the control group. INTERPRETATION: Our findings do not support the routine use of hemithoracic radiotherapy for malignant pleural mesothelioma after neoadjuvant chemotherapy and extrapleural pneumonectomy. FUNDING: Swiss Group for Clinical Cancer Research, Swiss State Secretariat for Education, Research and Innovation, Eli Lilly.

Label-free detection of tobramycin in serum by transmission-localized surface plasmon resonance.

In order to improve the efficacy and safety of treatments, drug dosage needs to be adjusted to the actual needs of each patient in a truly personalized medicine approach. Key for widespread dosage adjustment is the availability of point-of-care devices able to measure plasma drug concentration in a simple, automated, and cost-effective fashion. In the present work, we introduce and test a portable, palm-sized transmission-localized surface plasmon resonance (T-LSPR) setup, comprised of off-the-shelf components and coupled with DNA-based aptamers specific to the antibiotic tobramycin (467 Da). The core of the T-LSPR setup are aptamer-functionalized gold nanoislands (NIs) deposited on a glass slide covered with fluorine-doped tin oxide (FTO), which acts as a biosensor. The gold NIs exhibit localized plasmon resonance in the visible range matching the sensitivity of the complementary metal oxide semiconductor (CMOS) image sensor employed as a light detector. The combination of gold NIs on the FTO substrate, causing NIs size and pattern irregularity, might reduce the overall sensitivity but confers extremely high stability in high-ionic solutions, allowing it to withstand numerous regeneration cycles without sensing losses. With this rather simple T-LSPR setup, we show real-time label-free detection of tobramycin in buffer, measuring concentrations down to 0.5 µM. We determined an affinity constant of the aptamer-tobramycin pair consistent with the value obtained using a commercial propagating-wave based SPR. Moreover, our label-free system can detect tobramycin in filtered undiluted blood serum, measuring concentrations down to 10 µM with a theoretical detection limit of 3.4 µM. While the association signal of tobramycin onto the aptamer is masked by the serum injection, the quantification of the captured tobramycin is possible during the dissociation phase and leads to a linear calibration curve for the concentrations over the tested range (10-80 µM). The plasmon shift following surface binding is calculated in terms of both plasmon peak location and hue, with the latter allowing faster data elaboration and real-time display of the results. The presented T-LSPR system shows for the first time label-free direct detection and quantification of a small molecule in the complex matrix of filtered undiluted blood serum. Its uncomplicated construction and compact size, together with the remarkable performances, represent a leap forward toward effective point-of-care devices for therapeutic drug concentration monitoring.

Explainable machine learning predictions of perceptual sensitivity for retinal prostheses.

Objective.Retinal prostheses evoke visual precepts by electrically stimulating functioning cells in the retina. Despite high variance in perceptual thresholds across subjects, among electrodes within a subject, and over time, retinal prosthesis users must undergo 'system fitting', a process performed to calibrate stimulation parameters according to the subject's perceptual thresholds. Although previous work has identified electrode-retina distance and impedance as key factors affecting thresholds, an accurate predictive model is still lacking.Approach.To address these challenges, we (1) fitted machine learning models to a large longitudinal dataset with the goal of predicting individual electrode thresholds and deactivation as a function of stimulus, electrode, and clinical parameters ('predictors') and (2) leveraged explainable artificial intelligence (XAI) to reveal which of these predictors were most important.Main results.Our models accounted for up to 76% of the perceptual threshold response variance and enabled predictions of whether an electrode was deactivated in a given trial with F1 and area under the ROC curve scores of up to 0.732 and 0.911, respectively. Our models identified novel predictors of perceptual sensitivity, including subject age, time since blindness onset, and electrode-fovea distance.Significance.Our results demonstrate that routinely collected clinical measures and a single session of system fitting might be sufficient to inform an XAI-based threshold prediction strategy, which has the potential to transform clinical practice in predicting visual outcomes.

Axonal stimulation affects the linear summation of single-point perception in three Argus II users.

Objective.Retinal implants use electrical stimulation to elicit perceived flashes of light ('phosphenes'). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation.Approach.We retrospectively analyzed 3548 phosphene drawings made by three blind participants implanted with an Argus II Retinal Prosthesis. Phosphene shape (characterized by area, perimeter, major and minor axis length) and number of perceived phosphenes were averaged across trials and correlated with the corresponding single-electrode parameters. In addition, the number of phosphenes was correlated with stimulus amplitude and neuroanatomical parameters: electrode-retina and electrode-fovea distance as well as the electrode-electrode distance to ('between-axon') and along axon bundles ('along-axon'). Statistical analyses were conducted using linear regression and partial correlation analysis.Main results.Simple regression revealed that each paired-electrode shape descriptor could be predicted by the sum of the two corresponding single-electrode shape descriptors (p < .001). Multiple regression revealed that paired-electrode phosphene shape was primarily predicted by stimulus amplitude and electrode-fovea distance (p < .05). Interestingly, the number of elicited phosphenes tended to increase with between-axon distance (p < .05), but not with along-axon distance, in two out of three participants.Significance.The shape of phosphenes elicited by paired-electrode stimulation was well predicted by the shape of their corresponding single-electrode phosphenes, suggesting that two-point perception can be expressed as the linear summation of single-point perception. The impact of the between-axon distance on the perceived number of phosphenes provides further evidence in support of the axon map model for epiretinal stimulation. These findings contribute to the growing literature on phosphene perception and have important implications for the design of future retinal prostheses.

Great expectations: Aligning visual prosthetic development with implantee needs.

Purpose: Visual prosthetics have emerged as a promising assistive technology for individuals with vision loss, yet research often overlooks the human aspects of this technology. While previous studies have concentrated on the perceptual experiences of implant recipients (implantees) or the attitudes of potential implantees towards near-future implants, a systematic account of how current implants are being used in everyday life is still lacking. Methods: We interviewed six recipients of the most widely used visual implants (Argus II and Orion) and six leading researchers in the field. Through thematic and statistical analyses, we explored the daily usage of these implants by implantees and compared their responses to the expectations of researchers. We also sought implantees' input on desired features for future versions, aiming to inform the development of the next generation of implants. Results: Although implants are designed to facilitate various daily activities, we found that implantees use them less frequently than researchers expected. This discrepancy primarily stems from issues with usability and reliability, with implantees finding alternative methods to accomplish tasks, reducing the need to rely on the implant. For future implants, implantees emphasized the desire for improved vision, smart integration, and increased independence. Conclusions: Our study reveals a significant gap between researcher expectations and implantee experiences with visual prostheses, underscoring the importance of focusing future research on usability and real-world application. Translational relevance: This work advocates for a better alignment between technology development and implantee needs to enhance clinical relevance and practical utility of visual prosthetics.

Stress affects navigation strategies in immersive virtual reality.

There are known individual differences in both the ability to learn the layout of novel environments and the flexibility of strategies for navigating known environments. However, it is unclear how navigational abilities are impacted by high-stress scenarios. Here we used immersive virtual reality (VR) to develop a novel behavioral paradigm to examine navigation under dynamically changing situations. We recruited 48 participants (24 female; ages 17-32) to navigate a virtual maze (7.5 m × 7.5 m). Participants learned the maze by moving along a fixed path past the maze's landmarks (paintings). Subsequently, participants experienced either a non-stress condition, or a high-stress condition tasking them with navigating the maze. In the high-stress condition, their initial path was blocked, the environment was darkened, threatening music was played, fog obstructed more distal views of the environment, and participants were given a time limit of 20 s with a countdown timer displayed at the top of their screen. On trials where the path was blocked, we found self-reported stress levels and distance traveled increased while trial completion rate decreased (as compared to non-stressed control trials). On unblocked stress trials, participants were less likely to take a shortcut and consequently navigated less efficiently compared to control trials. Participants with more trait spatial anxiety reported more stress and navigated less efficiently. Overall, our results suggest that navigational abilities change considerably under high-stress conditions.

Retinal ganglion cells undergo cell typeâ€"specific functional changes in a biophysically detailed model of retinal degeneration.

Understanding the retina in health and disease is a key issue for neuroscience and neuroengineering applications such as retinal prostheses. During degeneration, the retinal network undergoes complex and multi-stage neuroanatomical alterations, which drastically impact the retinal ganglion cell (RGC) response and are of clinical importance. Here we present a biophysically detailed in silico model of retinal degeneration that simulates the network-level response to both light and electrical stimulation as a function of disease progression. The model is not only able to reproduce common findings about RGC activity in the degenerated retina, such as hyperactivity and increased electrical thresholds, but also offers testable predictions about the underlying neuroanatomical mechanisms. Overall, our findings demonstrate how biophysical changes associated with retinal degeneration affect retinal responses to both light and electrical stimulation, which may further our understanding of visual processing in the retina as well as inform the design and application of retinal prostheses.

Retinal ganglion cells undergo cell type-specific functional changes in a computational model of cone-mediated retinal degeneration.

Introduction: Understanding the retina in health and disease is a key issue for neuroscience and neuroengineering applications such as retinal prostheses. During degeneration, the retinal network undergoes complex and multi-stage neuroanatomical alterations, which drastically impact the retinal ganglion cell (RGC) response and are of clinical importance. Here we present a biophysically detailed in silico model of the cone pathway in the retina that simulates the network-level response to both light and electrical stimulation. Methods: The model included 11, 138 cells belonging to nine different cell types (cone photoreceptors, horizontal cells, ON/OFF bipolar cells, ON/OFF amacrine cells, and ON/OFF ganglion cells) confined to a 300 × 300 × 210µm patch of the parafoveal retina. After verifying that the model reproduced seminal findings about the light response of retinal ganglion cells (RGCs), we systematically introduced anatomical and neurophysiological changes (e.g., reduced light sensitivity of photoreceptor, cell death, cell migration) to the network and studied their effect on network activity. Results: The model was not only able to reproduce common findings about RGC activity in the degenerated retina, such as hyperactivity and increased electrical thresholds, but also offers testable predictions about the underlying neuroanatomical mechanisms. Discussion: Overall, our findings demonstrate how biophysical changes typified by cone-mediated retinal degeneration may impact retinal responses to light and electrical stimulation. These insights may further our understanding of retinal processing and inform the design of retinal prostheses.

Explainable Machine Learning Predictions of Perceptual Sensitivity for Retinal Prostheses.

To provide appropriate levels of stimulation, retinal prostheses must be calibrated to an individual's perceptual thresholds ('system fitting'), despite thresholds varying drastically across subjects, across electrodes within a subject, and over time. Although previous work has identified electrode-retina distance and impedance as key factors affecting thresholds, an accurate predictive model is still lacking. To address these challenges, we 1) fitted machine learning (ML) models to a large longitudinal dataset with the goal of predicting individual electrode thresholds and deactivation as a function of stimulus, electrode, and clinical parameters ('predictors') and 2) leveraged explainable artificial intelligence (XAI) to reveal which of these predictors were most important. Our models accounted for up to 77% of the perceptual threshold response variance and enabled predictions of whether an electrode was deactivated in a given trial with F1 and AUC scores of up to 0.740 and 0.913, respectively. Deactivation and threshold models identified novel predictors of perceptual sensitivity, including subject age, time since blindness onset, and electrode-fovea distance. Our results demonstrate that routinely collected clinical measures and a single session of system fitting might be sufficient to inform an XAI-based threshold prediction strategy, which may transform clinical practice in predicting visual outcomes.

Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice.

Despite their immense success as a model of macaque visual cortex, deep convolutional neural networks (CNNs) have struggled to predict activity in visual cortex of the mouse, which is thought to be strongly dependent on the animal's behavioral state. Furthermore, most computational models focus on predicting neural responses to static images presented under head fixation, which are dramatically different from the dynamic, continuous visual stimuli that arise during movement in the real world. Consequently, it is still unknown how natural visual input and different behavioral variables may integrate over time to generate responses in primary visual cortex (V1). To address this, we introduce a multimodal recurrent neural network that integrates gaze-contingent visual input with behavioral and temporal dynamics to explain V1 activity in freely moving mice. We show that the model achieves state-of-the-art predictions of V1 activity during free exploration and demonstrate the importance of each component in an extensive ablation study. Analyzing our model using maximally activating stimuli and saliency maps, we reveal new insights into cortical function, including the prevalence of mixed selectivity for behavioral variables in mouse V1. In summary, our model offers a comprehensive deep-learning framework for exploring the computational principles underlying V1 neurons in freely-moving animals engaged in natural behavior.

Axonal stimulation affects the linear summation of single-point perception in three Argus II users.

Purpose: Retinal implants use electrical stimulation to elicit perceived flashes of light ("phosphenes"). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation. Methods: We retrospectively analyzed 3548 phosphene drawings made by three blind participants implanted with an Argus II Retinal Prosthesis. Phosphene shape (characterized by area, perimeter, major and minor axis length) and number of perceived phosphenes were averaged across trials and correlated with the corresponding single-electrode parameters. In addition, the number of phosphenes was correlated with stimulus amplitude and neuroanatomical parameters: electrode-retina and electrode-fovea distance as well as the electrode-electrode distance to ("between-axon") and along axon bundles ("along-axon"). Statistical analyses were conducted using linear regression and partial correlation analysis. Results: Simple regression revealed that each paired-electrode shape descriptor could be predicted by the sum of the two corresponding single-electrode shape descriptors (p < .001). Multiple regression revealed that paired-electrode phosphene shape was primarily predicted by stimulus amplitude and electrode-fovea distance (p < .05). Interestingly, the number of elicited phosphenes tended to increase with between-axon distance (p < .05), but not with along-axon distance, in two out of three participants. Conclusions: The shape of phosphenes elicited by paired-electrode stimulation was well predicted by the shape of their corresponding single-electrode phosphenes, suggesting that two-point perception can be expressed as the linear summation of single-point perception. The notable impact of the between-axon distance on the perceived number of phosphenes provides further evidence in support of the axon map model for epiretinal stimulation. These findings contribute to the growing literature on phosphene perception and have important implications for the design of future retinal prostheses.

Genome-wide Association Studies of Retinal Vessel Tortuosity Identify Numerous Novel Loci Revealing Genes and Pathways Associated With Ocular and Cardiometabolic Diseases.

Purpose: To identify novel susceptibility loci for retinal vascular tortuosity, to better understand the molecular mechanisms modulating this trait, and reveal causal relationships with diseases and their risk factors. Design: Genome-wide Association Studies (GWAS) of vascular tortuosity of retinal arteries and veins followed by replication meta-analysis and Mendelian randomization (MR). Participants: We analyzed 116 639 fundus images of suitable quality from 63 662 participants from 3 cohorts, namely the UK Biobank (n = 62 751), the Swiss Kidney Project on Genes in Hypertension (n = 397), and OphtalmoLaus (n = 512). Methods: Using a fully automated retina image processing pipeline to annotate vessels and a deep learning algorithm to determine the vessel type, we computed the median arterial, venous and combined vessel tortuosity measured by the distance factor (the length of a vessel segment over its chord length), as well as by 6 alternative measures that integrate over vessel curvature. We then performed the largest GWAS of these traits to date and assessed gene set enrichment using the novel high-precision statistical method PascalX. Main Outcome Measure: We evaluated the genetic association of retinal tortuosity, measured by the distance factor. Results: Higher retinal tortuosity was significantly associated with higher incidence of angina, myocardial infarction, stroke, deep vein thrombosis, and hypertension. We identified 175 significantly associated genetic loci in the UK Biobank; 173 of these were novel and 4 replicated in our second, much smaller, metacohort. We estimated heritability at ∼25% using linkage disequilibrium score regression. Vessel type specific GWAS revealed 116 loci for arteries and 63 for veins. Genes with significant association signals included COL4A2, ACTN4, LGALS4, LGALS7, LGALS7B, TNS1, MAP4K1, EIF3K, CAPN12, ECH1, and SYNPO2. These tortuosity genes were overexpressed in arteries and heart muscle and linked to pathways related to the structural properties of the vasculature. We demonstrated that retinal tortuosity loci served pleiotropic functions as cardiometabolic disease variants and risk factors. Concordantly, MR revealed causal effects between tortuosity, body mass index, and low-density lipoprotein. Conclusions: Several alleles associated with retinal vessel tortuosity suggest a common genetic architecture of this trait with ocular diseases (glaucoma, myopia), cardiovascular diseases, and metabolic syndrome. Our results shed new light on the genetics of vascular diseases and their pathomechanisms and highlight how GWASs and heritability can be used to improve phenotype extraction from high-dimensional data, such as images. Financial Disclosures: The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Immersive Virtual Reality Simulations of Bionic Vision.

Bionic vision uses neuroprostheses to restore useful vision to people living with incurable blindness. However, a major outstanding challenge is predicting what people "see" when they use their devices. The limited field of view of current devices necessitates head movements to scan the scene, which is difficult to simulate on a computer screen. In addition, many computational models of bionic vision lack biological realism. To address these challenges, we present VR-SPV, an open-source virtual reality toolbox for simulated prosthetic vision that uses a psychophysically validated computational model to allow sighted participants to "see through the eyes" of a bionic eye user. To demonstrate its utility, we systematically evaluated how clinically reported visual distortions affect performance in a letter recognition and an immersive obstacle avoidance task. Our results highlight the importance of using an appropriate phosphene model when predicting visual outcomes for bionic vision.

Towards aSmart Bionic Eye: AI-powered artificial vision for the treatment of incurable blindness.

Objective.How can we return a functional form of sight to people who are living with incurable blindness? Despite recent advances in the development of visual neuroprostheses, the quality of current prosthetic vision is still rudimentary and does not differ much across different device technologies.Approach.Rather than aiming to represent the visual scene as naturally as possible, aSmart Bionic Eyecould provide visual augmentations through the means of artificial intelligence-based scene understanding, tailored to specific real-world tasks that are known to affect the quality of life of people who are blind, such as face recognition, outdoor navigation, and self-care.Main results.Complementary to existing research aiming to restore natural vision, we propose a patient-centered approach to incorporate deep learning-based visual augmentations into the next generation of devices.Significance.The ability of a visual prosthesis to support everyday tasks might make the difference between abandoned technology and a widely adopted next-generation neuroprosthetic device.