Scalable Gaussian process inference of neural responses to natural images.

Predicting the responses of sensory neurons is a long-standing neuroscience goal. However, while there has been much progress in modeling neural responses to simple and/or artificial stimuli, predicting responses to natural stimuli remains an ongoing challenge. On the one hand, deep neural networks perform very well on certain datasets but can fail when data are limited. On the other hand, Gaussian processes (GPs) perform well on limited data but are poor at predicting responses to high-dimensional stimuli, such as natural images. Here, we show how structured priors, e.g., for local and smooth receptive fields, can be used to scale up GPs to model neural responses to high-dimensional stimuli. With this addition, GPs largely outperform a deep neural network trained to predict retinal responses to natural images, with the largest differences observed when both models are trained on a small dataset. Further, since they allow us to quantify the uncertainty in their predictions, GPs are well suited to closed-loop experiments, where stimuli are chosen actively so as to collect "informative" neural data. We show how GPs can be used to actively select which stimuli to present, so as to i) efficiently learn a model of retinal responses to natural images, using few data, and ii) rapidly distinguish between competing models (e.g., a linear vs. a nonlinear model). In the future, our approach could be applied to other sensory areas, beyond the retina.

Context-dependent selectivity to natural images in the retina.

Retina ganglion cells extract specific features from natural scenes and send this information to the brain. In particular, they respond to local light increase (ON responses), and/or decrease (OFF). However, it is unclear if this ON-OFF selectivity, characterized with synthetic stimuli, is maintained under natural scene stimulation. Here we recorded ganglion cell responses to natural images slightly perturbed by random noise patterns to determine their selectivity during natural stimulation. The ON-OFF selectivity strongly depended on the specific image. A single ganglion cell can signal luminance increase for one image, and luminance decrease for another. Modeling and experiments showed that this resulted from the non-linear combination of different retinal pathways. Despite the versatility of the ON-OFF selectivity, a systematic analysis demonstrated that contrast was reliably encoded in these responses. Our perturbative approach uncovered the selectivity of retinal ganglion cells to more complex features than initially thought.