Mice Preferentially Use Increases in Cerebral Cortex Spiking to Detect Changes in Visual Stimuli.

Whenever the retinal image changes, some neurons in visual cortex increase their rate of firing whereas others decrease their rate of firing. Linking specific sets of neuronal responses with perception and behavior is essential for understanding mechanisms of neural circuit computation. We trained mice of both sexes to perform visual detection tasks and used optogenetic perturbations to increase or decrease neuronal spiking primary visual cortex (V1). Perceptual reports were always enhanced by increments in V1 spike counts and impaired by decrements, even when increments and decrements in spiking were generated in the same neuronal populations. Moreover, detecting changes in cortical activity depended on spike count integration rather than instantaneous changes in spiking. Recurrent neural networks trained in the task similarly relied on increments in neuronal activity when activity has costs. This work clarifies neuronal decoding strategies used by cerebral cortex to translate cortical spiking into percepts that can be used to guide behavior.SIGNIFICANCE STATEMENT Visual responses in the primary visual cortex (V1) are diverse, in that neurons can be either excited or inhibited by the onset of a visual stimulus. We selectively potentiated or suppressed V1 spiking in mice while they performed contrast change detection tasks. In other experiments, excitation or inhibition was delivered to V1 independent of visual stimuli. Mice readily detected increases in V1 spiking while equivalent reductions in V1 spiking suppressed the probability of detection, even when increases and decreases in V1 spiking were generated in the same neuronal populations. Our data raise the striking possibility that only increments in spiking are used to render information to structures downstream of V1.

Visual feature tuning of superior colliculus neural reafferent responses after fixational microsaccades.

The primate superior colliculus (SC) is causally involved in microsaccade generation. Moreover, visually responsive SC neurons across this structure's topographic map, even at peripheral eccentricities much larger than the tiny microsaccade amplitudes, exhibit significant modulations of evoked response sensitivity when stimuli appear perimicrosaccadically. However, during natural viewing, visual stimuli are normally stably present in the environment and are only shifted on the retina by eye movements. Here we investigated this scenario for the case of microsaccades, asking whether and how SC neurons respond to microsaccade-induced image jitter. We recorded neural activity from two male rhesus macaque monkeys. Within the response field (RF) of a neuron, there was a stable stimulus consisting of a grating of one of three possible spatial frequencies. The grating was stable on the display, but microsaccades periodically jittered the retinotopic RF location over it. We observed clear short-latency visual reafferent responses after microsaccades. These responses were weaker, but earlier (relative to new fixation onset after microsaccade end), than responses to sudden stimulus onsets without microsaccades. The reafferent responses clearly depended on microsaccade amplitude as well as microsaccade direction relative to grating orientation. Our results indicate that one way for microsaccades to influence vision is through modulating how the spatio-temporal landscape of SC visual neural activity represents stable stimuli in the environment. Such representation depends on the specific pattern of temporal luminance modulations expected from the relative relationship between eye movement vector (size and direction) on one hand and spatial visual pattern layout on the other.NEW & NOTEWORTHY Despite being diminutive, microsaccades still jitter retinal images. We investigated how such jitter affects superior colliculus (SC) activity. We found that SC neurons exhibit short-latency visual reafferent bursts after microsaccades. These bursts reflect not only the spatial luminance profiles of visual patterns but also how such profiles are shifted by eye movement size and direction. These results indicate that the SC continuously represents visual patterns, even as they are jittered by the smallest possible saccades.

Superior colliculus encodes visual saliency during smooth pursuit eye movements.

The saliency map has played a long-standing role in models and theories of visual attention, and it is now supported by neurobiological evidence from several cortical and subcortical brain areas. While visual saliency is computed during moments of active fixation, it is not known whether the same is true while engaged in smooth pursuit of a moving stimulus, which is very common in real-world vision. Here, we examined extrafoveal saliency coding in the superior colliculus, a midbrain area associated with attention and gaze, during smooth pursuit eye movements. We found that SC neurons from the superficial visual layers showed a robust representation of peripheral saliency evoked by a conspicuous stimulus embedded in a wide-field array of goal-irrelevant stimuli. In contrast, visuomotor neurons from the intermediate saccade-related layers showed a poor saliency representation, even though most of these neurons were visually responsive during smooth pursuit. These results confirm and extend previous findings that place the SCs in a unique role as a saliency map that monitors peripheral vision during foveation of stationary and now moving objects.

Non-accidental properties, metric invariance, and encoding by neurons in a model of ventral stream visual object recognition, VisNet.

When objects transform into different views, some properties are maintained, such as whether the edges are convex or concave, and these non-accidental properties are likely to be important in view-invariant object recognition. The metric properties, such as the degree of curvature, may change with different views, and are less likely to be useful in object recognition. It is shown that in a model of invariant visual object recognition in the ventral visual stream, VisNet, non-accidental properties are encoded much more than metric properties by neurons. Moreover, it is shown how with the temporal trace rule training in VisNet, non-accidental properties of objects become encoded by neurons, and how metric properties are treated invariantly. We also show how VisNet can generalize between different objects if they have the same non-accidental property, because the metric properties are likely to overlap. VisNet is a 4-layer unsupervised model of visual object recognition trained by competitive learning that utilizes a temporal trace learning rule to implement the learning of invariance using views that occur close together in time. A second crucial property of this model of object recognition is, when neurons in the level corresponding to the inferior temporal visual cortex respond selectively to objects, whether neurons in the intermediate layers can respond to combinations of features that may be parts of two or more objects. In an investigation using the four sides of a square presented in every possible combination, it was shown that even though different layer 4 neurons are tuned to encode each feature or feature combination orthogonally, neurons in the intermediate layers can respond to features or feature combinations present is several objects. This property is an important part of the way in which high capacity can be achieved in the four-layer ventral visual cortical pathway. These findings concerning non-accidental properties and the use of neurons in intermediate layers of the hierarchy help to emphasise fundamental underlying principles of the computations that may be implemented in the ventral cortical visual stream used in object recognition.

Computational assessment of visual coding across mouse brain areas and behavioural states.

Introduction: Our brain is bombarded by a diverse range of visual stimuli, which are converted into corresponding neuronal responses and processed throughout the visual system. The neural activity patterns that result from these external stimuli vary depending on the object or scene being observed, but they also change as a result of internal or behavioural states. This raises the question of to what extent it is possible to predict the presented visual stimuli from neural activity across behavioural states, and how this varies in different brain regions. Methods: To address this question, we assessed the computational capacity of decoders to extract visual information in awake behaving mice, by analysing publicly available standardised datasets from the Allen Brain Institute. We evaluated how natural movie frames can be distinguished based on the activity of units recorded in distinct brain regions and under different behavioural states. This analysis revealed the spectrum of visual information present in different brain regions in response to binary and multiclass classification tasks. Results: Visual cortical areas showed highest classification accuracies, followed by thalamic and midbrain regions, with hippocampal regions showing close to chance accuracy. In addition, we found that behavioural variability led to a decrease in decoding accuracy, whereby large behavioural changes between train and test sessions reduced the classification performance of the decoders. A generalised linear model analysis suggested that this deterioration in classification might be due to an independent modulation of neural activity by stimulus and behaviour. Finally, we reconstructed the natural movie frames from optimal linear classifiers, and observed a strong similarity between reconstructed and actual movie frames. However, the similarity was significantly higher when the decoders were trained and tested on sessions with similar behavioural states. Conclusion: Our analysis provides a systematic assessment of visual coding in the mouse brain, and sheds light on the spectrum of visual information present across brain areas and behavioural states.

Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks.

Traditional models of retinal system identification analyze the neural response to artificial stimuli using models consisting of predefined components. The model design is limited to prior knowledge, and the artificial stimuli are too simple to be compared with stimuli processed by the retina. To fill in this gap with an explainable model that reveals how a population of neurons work together to encode the larger field of natural scenes, here we used a deep-learning model for identifying the computational elements of the retinal circuit that contribute to learning the dynamics of natural scenes. Experimental results verify that the recurrent connection plays a key role in encoding complex dynamic visual scenes while learning biological computational underpinnings of the retinal circuit. In addition, the proposed models reveal both the shapes and the locations of the spatiotemporal receptive fields of ganglion cells.

Neuronal Activities in the Mouse Visual Cortex Predict Patterns of Sensory Stimuli.

Visual cortex forms the basis of visual processing and plays important roles in visual encoding. By using the recently published Allen Brain Observatory dataset consisting of large-scale calcium imaging of mouse V1 activities under visual stimuli, we were able to obtain high-quality data capturing simultaneous neuronal activities at multiple sub-areas and cortical depths of V1. Using prediction models, we analyzed the activity profiles related to static and drifting grating stimuli. We conducted a comprehensive survey of the coding ability of multiple cortical locations toward different stimulus attributes. Specifically, we focused on orientations and spatial frequencies (for static stimuli), as well as moving directions and speed (for drifting stimuli). By using results produced from a prediction model, we quantified the decoding performance profile at different sub-areas and layers of V1. In addition, we analyzed the interactions and interference between different stimulus attributes. The insights obtained from these discoveries would contribute to more precise and quantitative understanding of V1 coding mechanisms.

Contrasting group analysis of Brazilian students with dyslexia and good readers using the computerized reading and writing assessment battery "BALE".

The analysis of cognitive processes underpinning reading and writing skills may help to distinguish different reading ability profiles. The present study used a Brazilian reading and writing battery to compare performance of students with dyslexia with two individually matched control groups: one contrasting on reading competence but not age and the other group contrasting on age but not reading competence. Participants were 28 individuals with dyslexia (19 boys) with a mean age of 9.82 (SD ± 1.44) drawn from public and private schools. These were matched to: (1) an age control group (AC) of 26 good readers with a mean age of 9.77 (SD ± 1.44) matched by age, sex, years of schooling, and type of school; (2) reading control group (RC) of 28 younger controls with a mean age of 7.82 (SD ± 1.06) matched by sex, type of school, and reading level. All groups were tested on four tasks from the Brazilian Reading and Writing Assessment battery ("BALE"): Written Sentence Comprehension Test (WSCT); Spoken Sentence Comprehension Test (OSCT); Picture-Print Writing Test (PPWT 1.1-Writing); and the Reading Competence Test (RCT). These tasks evaluate reading and listening comprehension for sentences, spelling, and reading isolated words and pseudowords (non-words). The dyslexia group scored lower and took longer to complete tasks than the AC group. Compared with the RC group, there were no differences in total scores on reading or oral comprehension tasks. However, dyslexics presented slower reading speeds, longer completion times, and lower scores on spelling tasks, even compared with younger controls. Analysis of types of errors on word and pseudoword reading items showed students with dyslexia scoring lower for pseudoword reading than the other two groups. These findings suggest that the dyslexics overall scores were similar to those of younger readers. However, specific phonological and visual decoding deficits showed that the two groups differ in terms of underpinning reading strategies.

An illusory distortion of moving form driven by motion deblurring.

Many visual processes integrate information over protracted periods, a process known as temporal integration. One consequence of this is that objects that cast images that move across the retinal surfaces can generate blurred form signals, similar to the motion blur that can be captured in photographs taken with slow shutter speeds. Subjectively, retinal motion blur signals are suppressed from awareness, such that moving objects seem sharply defined. One suggestion has been that this subjective impression is due to humans not being able to distinguish between focussed and blurred moving objects. Contrary to this suggestion, here we report a novel illusion, and consequent experiments, that implicate a suppressive mechanism. We find that the apparent shape of circular moving objects can be distorted when their rear edges lag leading edges by ∼60 ms. Moreover, we find that sensitivity for detecting blur, and for discriminating between blur intensities, is uniformly worse for physical blurs added behind moving objects, as opposed to in-front. Also, it was easier to differentiate between slight and slightly greater physical blurs than it was to differentiate between slight blur and the absence of blur, both behind and in-front of moving edges. These 'dipper' functions suggest that blur signals must reach a threshold intensity before they can be detected, and that the relevant threshold is effectively elevated for blur signals trailing behind moving contours. In combination, these data suggest moving objects look sharply defined, at least in part, because of a functional adaptation that actively suppresses motion blur signals from awareness.

Exploring the function of neural oscillations in early sensory systems.

Neuronal oscillations appear throughout the nervous system, in structures as diverse as the cerebral cortex, hippocampus, subcortical nuclei and sense organs. Whether neural rhythms contribute to normal function, are merely epiphenomena, or even interfere with physiological processing are topics of vigorous debate. Sensory pathways are ideal for investigation of oscillatory activity because their inputs can be defined. Thus, we will focus on sensory systems as we ask how neural oscillations arise and how they might encode information about the stimulus. We will highlight recent work in the early visual pathway that shows how oscillations can multiplex different types of signals to increase the amount of information that spike trains encode and transmit. Last, we will describe oscillation-based models of visual processing and explore how they might guide further research.

Retinal oscillations carry visual information to cortex.

Thalamic relay cells fire action potentials that transmit information from retina to cortex. The amount of information that spike trains encode is usually estimated from the precision of spike timing with respect to the stimulus. Sensory input, however, is only one factor that influences neural activity. For example, intrinsic dynamics, such as oscillations of networks of neurons, also modulate firing pattern. Here, we asked if retinal oscillations might help to convey information to neurons downstream. Specifically, we made whole-cell recordings from relay cells to reveal retinal inputs (EPSPs) and thalamic outputs (spikes) and then analyzed these events with information theory. Our results show that thalamic spike trains operate as two multiplexed channels. One channel, which occupies a low frequency band (<30 Hz), is encoded by average firing rate with respect to the stimulus and carries information about local changes in the visual field over time. The other operates in the gamma frequency band (40-80 Hz) and is encoded by spike timing relative to retinal oscillations. At times, the second channel conveyed even more information than the first. Because retinal oscillations involve extensive networks of ganglion cells, it is likely that the second channel transmits information about global features of the visual scene.