Probing Computation in the Primate Visual System at Single-Cone Resolution.

Daylight vision begins when light activates cone photoreceptors in the retina, creating spatial patterns of neural activity. These cone signals are then combined and processed in downstream neural circuits, ultimately producing visual perception. Recent technical advances have made it possible to deliver visual stimuli to the retina that probe this processing by the visual system at its elementary resolution of individual cones. Physiological recordings from nonhuman primate retinas reveal the spatial organization of cone signals in retinal ganglion cells, including how signals from cones of different types are combined to support both spatial and color vision. Psychophysical experiments with human subjects characterize the visual sensations evoked by stimulating a single cone, including the perception of color. Future combined physiological and psychophysical experiments focusing on probing the elementary visual inputs are likely to clarify how neural processing generates our perception of the visual world.

High-Fidelity Reproduction of Visual Signals by Electrical Stimulation in the Central Primate Retina.

Electrical stimulation of retinal ganglion cells (RGCs) with electronic implants provides rudimentary artificial vision to people blinded by retinal degeneration. However, current devices stimulate indiscriminately and therefore cannot reproduce the intricate neural code of the retina. Recent work has demonstrated more precise activation of RGCs using focal electrical stimulation with multielectrode arrays in the peripheral macaque retina, but it is unclear how effective this can be in the central retina, which is required for high-resolution vision. This work probes the neural code and effectiveness of focal epiretinal stimulation in the central macaque retina, using large-scale electrical recording and stimulation ex vivo The functional organization, light response properties, and electrical properties of the major RGC types in the central retina were mostly similar to the peripheral retina, with some notable differences in density, kinetics, linearity, spiking statistics, and correlations. The major RGC types could be distinguished by their intrinsic electrical properties. Electrical stimulation targeting parasol cells revealed similar activation thresholds and reduced axon bundle activation in the central retina, but lower stimulation selectivity. Quantitative evaluation of the potential for image reconstruction from electrically evoked parasol cell signals revealed higher overall expected image quality in the central retina. An exploration of inadvertent midget cell activation suggested that it could contribute high spatial frequency noise to the visual signal carried by parasol cells. These results support the possibility of reproducing high-acuity visual signals in the central retina with an epiretinal implant.SIGNIFICANCE STATEMENT Artificial restoration of vision with retinal implants is a major treatment for blindness. However, present-day implants do not provide high-resolution visual perception, in part because they do not reproduce the natural neural code of the retina. Here, we demonstrate the level of visual signal reproduction that is possible with a future implant by examining how accurately responses to electrical stimulation of parasol retinal ganglion cells can convey visual signals. Although the precision of electrical stimulation in the central retina was diminished relative to the peripheral retina, the quality of expected visual signal reconstruction in parasol cells was greater. These findings suggest that visual signals could be restored with high fidelity in the central retina using a future retinal implant.

Inference of Electrical Stimulation Sensitivity from Recorded Activity of Primate Retinal Ganglion Cells.

High-fidelity electronic implants can in principle restore the function of neural circuits by precisely activating neurons via extracellular stimulation. However, direct characterization of the individual electrical sensitivity of a large population of target neurons, to precisely control their activity, can be difficult or impossible. A potential solution is to leverage biophysical principles to infer sensitivity to electrical stimulation from features of spontaneous electrical activity, which can be recorded relatively easily. Here, this approach is developed and its potential value for vision restoration is tested quantitatively using large-scale multielectrode stimulation and recording from retinal ganglion cells (RGCs) of male and female macaque monkeys ex vivo Electrodes recording larger spikes from a given cell exhibited lower stimulation thresholds across cell types, retinas, and eccentricities, with systematic and distinct trends for somas and axons. Thresholds for somatic stimulation increased with distance from the axon initial segment. The dependence of spike probability on injected current was inversely related to threshold, and was substantially steeper for axonal than somatic compartments, which could be identified by their recorded electrical signatures. Dendritic stimulation was largely ineffective for eliciting spikes. These trends were quantitatively reproduced with biophysical simulations. Results from human RGCs were broadly similar. The inference of stimulation sensitivity from recorded electrical features was tested in a data-driven simulation of visual reconstruction, revealing that the approach could significantly improve the function of future high-fidelity retinal implants.SIGNIFICANCE STATEMENT This study demonstrates that individual in situ primate retinal ganglion cells of different types respond to artificially generated, external electrical fields in a systematic manner, in accordance with theoretical predictions, that allows for prediction of electrical stimulus sensitivity from recorded spontaneous activity. It also provides evidence that such an approach could be immensely helpful in the calibration of clinical retinal implants.

An RF-Ultrasound Relay for Adaptive Wireless Powering Across Tissue Interfaces.

Single modality wireless power transfer has limited depth for mm-sized implants across air / tissue or skull / tissue interfaces because they either suffer from high loss in tissue (RF, Optical) or high reflection at the medium interface (Ultrasound (US)). This paper proposes an RF-US relay chip at the media interface avoiding the reflection at the boundary, and enabling efficient wireless powering to mm-sized deep implants across multiple media. The relay chip rectifies the incoming RF power through an 85.5% efficient RF inductive link (across air) using a multi-output regulating rectifier (MORR) with 81% power conversion efficiency (PCE) at 186 mW load, and transmits ultrasound using adiabatic power amplifiers (PAs) to the implant in order to minimize cascaded power loss. To adapt the US focus to implant movement or placement, beamforming was implemented using 6 channels of US PAs with 2-bit phase control (0, 90, 180, and 270°) and 3 different amplitudes (6-29, 4.5, and 1.8 V) from the MORR. The adiabatic PA contributes a 30-40% increase in efficiency over class-D and beamforming increases the efficiency by 251% at 2.5 cm over fixed focusing. The proof-of-concept powering system for a retinal implant, from an external PA on a pair of glasses to a hydrophone with 1.2 cm (air) + 2.9 cm (agar eyeball phantom in mineral oil) separation distance, had a power delivered to the load (PDL) of 946 µW. The 2.3 × 2 mm2 relay chip was fabricated in a 180 nm high-voltage (HV) BCD process.

Nonlinear Decoding of Natural Images From Large-Scale Primate Retinal Ganglion Recordings.

Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve on existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from more than 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons.

Pathway-Specific Asymmetries between ON and OFF Visual Signals.

Visual processing is largely organized into ON and OFF pathways that signal stimulus increments and decrements, respectively. These pathways exhibit natural pairings based on morphological and physiological similarities, such as ON and OFF α-ganglion cells in the mammalian retina. Several studies have noted asymmetries in the properties of ON and OFF pathways. For example, the spatial receptive fields (RFs) of OFF α-cells are systematically smaller than ON α-cells. Analysis of natural scenes suggests that these asymmetries are optimal for visual encoding. To test the generality of ON/OFF asymmetries, we measured the spatiotemporal RF properties of multiple RGC types in rat retina. Through a quantitative and serial classification, we identified three functional pairs of ON and OFF RGCs. We analyzed the structure of their RFs and compared spatial integration, temporal integration, and gain across ON and OFF pairs. Similar to previous results from the cat and primate, RGC types with larger spatial RFs exhibited briefer temporal integration and higher gain. However, each pair of ON and OFF RGC types exhibited distinct asymmetric relationships between RF properties, some of which were opposite to the findings of previous reports. These results reveal the functional organization of six RGC types in the rodent retina and indicate that ON/OFF asymmetries are pathway specific.SIGNIFICANCE STATEMENT Circuits that process sensory input frequently process increments separately from decrements, so-called ON and OFF responses. Theoretical studies indicate that this separation, and associated asymmetries in ON and OFF pathways, may be beneficial for encoding natural stimuli. However, the generality of ON and OFF pathway asymmetries has not been tested. Here we compare the functional properties of three distinct pairs of ON and OFF pathways in the rodent retina and show that their asymmetries are pathway specific. These results provide a new view on the partitioning of vision across diverse ON and OFF signaling pathways.

Temporal resolution of single-photon responses in primate rod photoreceptors and limits imposed by cellular noise.

Sensory receptor noise corrupts sensory signals, contributing to imperfect perception and dictating central processing strategies. For example, noise in rod phototransduction limits our ability to detect light, and minimizing the impact of this noise requires precisely tuned nonlinear processing by the retina. But detection sensitivity is only one aspect of night vision: prompt and accurate behavior also requires that rods reliably encode the timing of photon arrivals. We show here that the temporal resolution of responses of primate rods is much finer than the duration of the light response and identify the key limiting sources of transduction noise. We also find that the thermal activation rate of rhodopsin is lower than previous estimates, implying that other noise sources are more important than previously appreciated. A model of rod single-photon responses reveals that the limiting noise relevant for behavior depends critically on how rod signals are pooled by downstream neurons. NEW & NOTEWORTHY Many studies have focused on the visual system's ability to detect photons, but not on its ability to encode the relative timing of detected photons. Timing is essential for computations such as determining the velocity of moving objects. Here we examine the timing precision of primate rod photoreceptor responses and show that it is more precise than previously appreciated. This motivates an evaluation of whether scotopic vision approaches limits imposed by rod temporal resolution.

Electrical stimulus artifact cancellation and neural spike detection on large multi-electrode arrays.

Simultaneous electrical stimulation and recording using multi-electrode arrays can provide a valuable technique for studying circuit connectivity and engineering neural interfaces. However, interpreting these measurements is challenging because the spike sorting process (identifying and segregating action potentials arising from different neurons) is greatly complicated by electrical stimulation artifacts across the array, which can exhibit complex and nonlinear waveforms, and overlap temporarily with evoked spikes. Here we develop a scalable algorithm based on a structured Gaussian Process model to estimate the artifact and identify evoked spikes. The effectiveness of our methods is demonstrated in both real and simulated 512-electrode recordings in the peripheral primate retina with single-electrode and several types of multi-electrode stimulation. We establish small error rates in the identification of evoked spikes, with a computational complexity that is compatible with real-time data analysis. This technology may be helpful in the design of future high-resolution sensory prostheses based on tailored stimulation (e.g., retinal prostheses), and for closed-loop neural stimulation at a much larger scale than currently possible.

Activation of ganglion cells and axon bundles using epiretinal electrical stimulation.

Epiretinal prostheses for treating blindness activate axon bundles, causing large, arc-shaped visual percepts that limit the quality of artificial vision. Improving the function of epiretinal prostheses therefore requires understanding and avoiding axon bundle activation. This study introduces a method to detect axon bundle activation on the basis of its electrical signature and uses the method to test whether epiretinal stimulation can directly elicit spikes in individual retinal ganglion cells without activating nearby axon bundles. Combined electrical stimulation and recording from isolated primate retina were performed using a custom multielectrode system (512 electrodes, 10-µm diameter, 60-µm pitch). Axon bundle signals were identified by their bidirectional propagation, speed, and increasing amplitude as a function of stimulation current. The threshold for bundle activation varied across electrodes and retinas, and was in the same range as the threshold for activating retinal ganglion cells near their somas. In the peripheral retina, 45% of electrodes that activated individual ganglion cells (17% of all electrodes) did so without activating bundles. This permitted selective activation of 21% of recorded ganglion cells (7% of expected ganglion cells) over the array. In one recording in the central retina, 75% of electrodes that activated individual ganglion cells (16% of all electrodes) did so without activating bundles. The ability to selectively activate a subset of retinal ganglion cells without axon bundles suggests a possible novel architecture for future epiretinal prostheses.NEW & NOTEWORTHY Large-scale multielectrode recording and stimulation were used to test how selectively retinal ganglion cells can be electrically activated without activating axon bundles. A novel method was developed to identify axon activation on the basis of its unique electrical signature and was used to find that a subset of ganglion cells can be activated at single-cell, single-spike resolution without producing bundle activity in peripheral and central retina. These findings have implications for the development of advanced retinal prostheses.

Anatomical identification of extracellularly recorded cells in large-scale multielectrode recordings.

This study combines for the first time two major approaches to understanding the function and structure of neural circuits: large-scale multielectrode recordings, and confocal imaging of labeled neurons. To achieve this end, we develop a novel approach to the central problem of anatomically identifying recorded cells, based on the electrical image: the spatiotemporal pattern of voltage deflections induced by spikes on a large-scale, high-density multielectrode array. Recordings were performed from identified ganglion cell types in the macaque retina. Anatomical images of cells in the same preparation were obtained using virally transfected fluorescent labeling or by immunolabeling after fixation. The electrical image was then used to locate recorded cell somas, axon initial segments, and axon trajectories, and these signatures were used to identify recorded cells. Comparison of anatomical and physiological measurements permitted visualization and physiological characterization of numerically dominant ganglion cell types with high efficiency in a single preparation.

Functional connectivity in the retina at the resolution of photoreceptors.

To understand a neural circuit requires knowledge of its connectivity. Here we report measurements of functional connectivity between the input and ouput layers of the macaque retina at single-cell resolution and the implications of these for colour vision. Multi-electrode technology was used to record simultaneously from complete populations of the retinal ganglion cell types (midget, parasol and small bistratified) that transmit high-resolution visual signals to the brain. Fine-grained visual stimulation was used to identify the location, type and strength of the functional input of each cone photoreceptor to each ganglion cell. The populations of ON and OFF midget and parasol cells each sampled the complete population of long- and middle-wavelength-sensitive cones. However, only OFF midget cells frequently received strong input from short-wavelength-sensitive cones. ON and OFF midget cells showed a small non-random tendency to selectively sample from either long- or middle-wavelength-sensitive cones to a degree not explained by clumping in the cone mosaic. These measurements reveal computations in a neural circuit at the elementary resolution of individual neurons.

Spatially patterned electrical stimulation to enhance resolution of retinal prostheses.

Retinal prostheses electrically stimulate neurons to produce artificial vision in people blinded by photoreceptor degenerative diseases. The limited spatial resolution of current devices results in indiscriminate stimulation of interleaved cells of different types, precluding veridical reproduction of natural activity patterns in the retinal output. Here we investigate the use of spatial patterns of current injection to increase the spatial resolution of stimulation, using high-density multielectrode recording and stimulation of identified ganglion cells in isolated macaque retina. As previously shown, current passed through a single electrode typically induced a single retinal ganglion cell spike with submillisecond timing precision. Current passed simultaneously through pairs of neighboring electrodes modified the probability of activation relative to injection through a single electrode. This modification could be accurately summarized by a piecewise linear model of current summation, consistent with a simple biophysical model based on multiple sites of activation. The generalizability of the piecewise linear model was tested by using the measured responses to stimulation with two electrodes to predict responses to stimulation with three electrodes. Finally, the model provided an accurate prediction of which among a set of spatial stimulation patterns maximized selective activation of a cell while minimizing activation of a neighboring cell. The results demonstrate that tailored multielectrode stimulation patterns based on a piecewise linear model may be useful in increasing the spatial resolution of retinal prostheses.

A polyaxonal amacrine cell population in the primate retina.

Amacrine cells are the most diverse and least understood cell class in the retina. Polyaxonal amacrine cells (PACs) are a unique subset identified by multiple long axonal processes. To explore their functional properties, populations of PACs were identified by their distinctive radially propagating spikes in large-scale high-density multielectrode recordings of isolated macaque retina. One group of PACs exhibited stereotyped functional properties and receptive field mosaic organization similar to that of parasol ganglion cells. These PACs had receptive fields coincident with their dendritic fields, but much larger axonal fields, and slow radial spike propagation. They also exhibited ON-OFF light responses, transient response kinetics, sparse and coordinated firing during image transitions, receptive fields with antagonistic surrounds and fine spatial structure, nonlinear spatial summation, and strong homotypic neighbor electrical coupling. These findings reveal the functional organization and collective visual signaling by a distinctive, high-density amacrine cell population.

Focal electrical stimulation of major ganglion cell types in the primate retina for the design of visual prostheses.

Electrical stimulation of retinal neurons with an advanced retinal prosthesis may eventually provide high-resolution artificial vision to the blind. However, the success of future prostheses depends on the ability to activate the major parallel visual pathways of the human visual system. Electrical stimulation of the five numerically dominant retinal ganglion cell types was investigated by simultaneous stimulation and recording in isolated peripheral primate (Macaca sp.) retina using multi-electrode arrays. ON and OFF midget, ON and OFF parasol, and small bistratified ganglion cells could all be activated directly to fire a single spike with submillisecond latency using brief pulses of current within established safety limits. Thresholds for electrical stimulation were similar in all five cell types. In many cases, a single cell could be specifically activated without activating neighboring cells of the same type or other types. These findings support the feasibility of direct electrical stimulation of the major visual pathways at or near their native spatial and temporal resolution.

Spatio-temporal correlations and visual signalling in a complete neuronal population.

Statistical dependencies in the responses of sensory neurons govern both the amount of stimulus information conveyed and the means by which downstream neurons can extract it. Although a variety of measurements indicate the existence of such dependencies, their origin and importance for neural coding are poorly understood. Here we analyse the functional significance of correlated firing in a complete population of macaque parasol retinal ganglion cells using a model of multi-neuron spike responses. The model, with parameters fit directly to physiological data, simultaneously captures both the stimulus dependence and detailed spatio-temporal correlations in population responses, and provides two insights into the structure of the neural code. First, neural encoding at the population level is less noisy than one would expect from the variability of individual neurons: spike times are more precise, and can be predicted more accurately when the spiking of neighbouring neurons is taken into account. Second, correlations provide additional sensory information: optimal, model-based decoding that exploits the response correlation structure extracts 20% more information about the visual scene than decoding under the assumption of independence, and preserves 40% more visual information than optimal linear decoding. This model-based approach reveals the role of correlated activity in the retinal coding of visual stimuli, and provides a general framework for understanding the importance of correlated activity in populations of neurons.

Spike sorting in the presence of stimulation artifacts: a dynamical control systems approach.

Objective. Bi-directional electronic neural interfaces, capable of both electrical recording and stimulation, communicate with the nervous system to permit precise calibration of electrical inputs by capturing the evoked neural responses. However, one significant challenge is that stimulation artifacts often mask the actual neural signals. To address this issue, we introduce a novel approach that employs dynamical control systems to detect and decipher electrically evoked neural activity despite the presence of electrical artifacts.Approach. Our proposed method leverages the unique spatiotemporal patterns of neural activity and electrical artifacts to distinguish and identify individual neural spikes. We designed distinctive dynamical models for both the stimulation artifact and each neuron observed during spontaneous neural activity. We can estimate which neurons were active by analyzing the recorded voltage responses across multiple electrodes post-stimulation. This technique also allows us to exclude signals from electrodes heavily affected by stimulation artifacts, such as the stimulating electrode itself, yet still accurately differentiate between evoked spikes and electrical artifacts.Main results. We applied our method to high-density multi-electrode recordings from the primate retina in anex vivosetup, using a grid of 512 electrodes. Through repeated electrical stimulations at varying amplitudes, we were able to construct activation curves for each neuron. The curves obtained with our method closely resembled those derived from manual spike sorting. Additionally, the stimulation thresholds we estimated strongly agreed with those determined through manual analysis, demonstrating high reliability (R2=0.951for human 1 andR2=0.944for human 2).Significance. Our method can effectively separate evoked neural spikes from stimulation artifacts by exploiting the distinct spatiotemporal propagation patterns captured by a dense, large-scale multi-electrode array. This technique holds promise for future applications in real-time closed-loop stimulation systems and for managing multi-channel stimulation strategies.

Modeling responses of macaque and human retinal ganglion cells to natural images using a convolutional neural network.

Linear-nonlinear (LN) cascade models provide a simple way to capture retinal ganglion cell (RGC) responses to artificial stimuli such as white noise, but their ability to model responses to natural images is limited. Recently, convolutional neural network (CNN) models have been shown to produce light response predictions that were substantially more accurate than those of a LN model. However, this modeling approach has not yet been applied to responses of macaque or human RGCs to natural images. Here, we train and test a CNN model on responses to natural images of the four numerically dominant RGC types in the macaque and human retina - ON parasol, OFF parasol, ON midget and OFF midget cells. Compared with the LN model, the CNN model provided substantially more accurate response predictions. Linear reconstructions of the visual stimulus were more accurate for CNN compared to LN model-generated responses, relative to reconstructions obtained from the recorded data. These findings demonstrate the effectiveness of a CNN model in capturing light responses of major RGC types in the macaque and human retinas in natural conditions.

Efficient coding of spatial information in the primate retina.

Sensory neurons have been hypothesized to efficiently encode signals from the natural environment subject to resource constraints. The predictions of this efficient coding hypothesis regarding the spatial filtering properties of the visual system have been found consistent with human perception, but they have not been compared directly with neural responses. Here, we analyze the information that retinal ganglion cells transmit to the brain about the spatial information in natural images subject to three resource constraints: the number of retinal ganglion cells, their total response variances, and their total synaptic strengths. We derive a model that optimizes the transmitted information and compare it directly with measurements of complete functional connectivity between cone photoreceptors and the four major types of ganglion cells in the primate retina, obtained at single-cell resolution. We find that the ganglion cell population exhibited 80% efficiency in transmitting spatial information relative to the model. Both the retina and the model exhibited high redundancy (~30%) among ganglion cells of the same cell type. A novel and unique prediction of efficient coding, the relationships between projection patterns of individual cones to all ganglion cells, was consistent with the observed projection patterns in the retina. These results indicate a high level of efficiency with near-optimal redundancy in visual signaling by the retina.

Monte Carlo methods for localization of cones given multielectrode retinal ganglion cell recordings.

It has recently become possible to identify cone photoreceptors in primate retina from multi-electrode recordings of ganglion cell spiking driven by visual stimuli of sufficiently high spatial resolution. In this paper we present a statistical approach to the problem of identifying the number, locations, and color types of the cones observed in this type of experiment. We develop an adaptive Markov Chain Monte Carlo (MCMC) method that explores the space of cone configurations, using a Linear-Nonlinear-Poisson (LNP) encoding model of ganglion cell spiking output, while analytically integrating out the functional weights between cones and ganglion cells. This method provides information about our posterior certainty about the inferred cone properties, and additionally leads to improvements in both the speed and quality of the inferred cone maps, compared to earlier "greedy" computational approaches.

Data Compression Versus Signal Fidelity Tradeoff in Wired-OR Analog-to-Digital Compressive Arrays for Neural Recording.

Future high-density and high channel count neural interfaces that enable simultaneous recording of tens of thousands of neurons will provide a gateway to study, restore and augment neural functions. However, building such technology within the bit-rate limit and power budget of a fully implantable device is challenging. The wired-OR compressive readout architecture addresses the data deluge challenge of a high channel count neural interface using lossy compression at the analog-to-digital interface. In this article, we assess the suitability of wired-OR for several steps that are important for neuroengineering, including spike detection, spike assignment and waveform estimation. For various wiring configurations of wired-OR and assumptions about the quality of the underlying signal, we characterize the trade-off between compression ratio and task-specific signal fidelity metrics. Using data from 18 large-scale microelectrode array recordings in macaque retina ex vivo, we find that for an event SNR of 7-10, wired-OR correctly detects and assigns at least 80% of the spikes with at least 50× compression. The wired-OR approach also robustly encodes action potential waveform information, enabling downstream processing such as cell-type classification. Finally, we show that by applying an LZ77-based lossless compressor (gzip) to the output of the wired-OR architecture, 1000× compression can be achieved over the baseline recordings.