A neural model of modified excitation/inhibition and feedback levels in schizophrenia.

Introduction: The strength of certain visual illusions, including contrast-contrast and apparent motion, is weakened in individuals with schizophrenia. Such phenomena have been interpreted as the impaired integration of inhibitory and excitatory neural responses, and impaired top-down feedback mechanisms. Methods: To investigate whether and how these factors influence the perceived contrast-contrast and apparent motion illusions in individuals with schizophrenia, we propose a two-layer network, with top-down feedback from layer 2 to layer 1 that can model visual receptive fields (RFs) and their inhibitory and excitatory subfields. Results: Our neural model suggests that illusion perception changes in individuals with schizophrenia can be influenced by altered top-down mechanisms and the organization of the on-center off-surround receptive fields. Alteration of the RF inhibitory surround and/or the excitatory center can replicate the difference of illusion precepts between individuals with schizophrenia within certain clinical states and normal controls. The results show that the simulated top-down feedback modulation enlarges the difference of the model illusion representations, replicating the difference between the two groups. Discussion: We propose that the heterogeneity of visual and in general sensory processing in certain clinical states of schizophrenia can be largely explained by the degree of top-down feedback reduction, emphasizing the critical role of top-down feedback in illusion perception, and to a lesser extent on the imbalance of excitation/inhibition. Our neural model provides a mechanistic explanation for the modulated visual percepts of contrast-contrast and apparent motion in schizophrenia with findings that can explain a broad range of visual perceptual observations in previous studies. The two-layer motif of the current model provides a general framework that can be tailored to investigate subcortico-cortical (such as thalamocortical) and cortico-cortical networks, bridging neurobiological changes in schizophrenia and perceptual processing.

A neural model of modified excitation/inhibition and feedback levels in schizophrenia.

The strength of certain visual illusions is weakened in individuals with schizophrenia. Such phenomena have been interpreted as the impaired integration of inhibitory and excitatory neural responses, and impaired top-down feedback mechanisms. To investigate whether and how these factors influence the perceived illusions in individuals with schizophrenia, we propose a two-layer network that can model visual receptive fields (RFs), their inhibitory and excitatory subfields, and the top-down feedback. Our neural model suggests that illusion perception changes in individuals with schizophrenia can be influenced by altered top-down mechanisms and the organization of the on-center off-surround receptive fields. Alteration of the RF inhibitory surround and/or the excitatory center can replicate the difference of illusion precepts between individuals with schizophrenia and normal controls. The results show that the simulated top-down feedback modulation enlarges the difference of the model illusion representations, replicating the difference between the two groups. We propose that the heterogeneity of visual and in general sensory processing in schizophrenia can be largely explained by the degree of top-down feedback reduction, emphasizing the critical role of top-down feedback in illusion perception, and to a lesser extent on the imbalance of excitation/inhibition. Our neural model provides a mechanistic explanation for the modulated visual percepts in schizophrenia with findings that can explain a broad range of visual perceptual observations in previous studies. The two-layer motif of the current model provides a general framework that can be tailored to investigate subcortico-cortical (such as thalamocortical) and cortico-cortical networks, bridging neurobiological changes in schizophrenia and perceptual processing.

Inter-individual deep image reconstruction via hierarchical neural code conversion.

The sensory cortex is characterized by general organizational principles such as topography and hierarchy. However, measured brain activity given identical input exhibits substantially different patterns across individuals. Although anatomical and functional alignment methods have been proposed in functional magnetic resonance imaging (fMRI) studies, it remains unclear whether and how hierarchical and fine-grained representations can be converted between individuals while preserving the encoded perceptual content. In this study, we trained a method of functional alignment called neural code converter that predicts a target subject's brain activity pattern from a source subject given the same stimulus, and analyzed the converted patterns by decoding hierarchical visual features and reconstructing perceived images. The converters were trained on fMRI responses to identical sets of natural images presented to pairs of individuals, using the voxels on the visual cortex that covers from V1 through the ventral object areas without explicit labels of the visual areas. We decoded the converted brain activity patterns into the hierarchical visual features of a deep neural network using decoders pre-trained on the target subject and then reconstructed images via the decoded features. Without explicit information about the visual cortical hierarchy, the converters automatically learned the correspondence between visual areas of the same levels. Deep neural network feature decoding at each layer showed higher decoding accuracies from corresponding levels of visual areas, indicating that hierarchical representations were preserved after conversion. The visual images were reconstructed with recognizable silhouettes of objects even with relatively small numbers of data for converter training. The decoders trained on pooled data from multiple individuals through conversions led to a slight improvement over those trained on a single individual. These results demonstrate that the hierarchical and fine-grained representation can be converted by functional alignment, while preserving sufficient visual information to enable inter-individual visual image reconstruction.

Computational components of visual predictive coding circuitry.

If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.

Tuning landscapes of the ventral stream.

A goal in visual neuroscience is to explain how neurons respond to natural scenes. However, neurons are generally tested using simpler stimuli, often because they can be transformed smoothly, allowing the measurement of tuning functions (i.e., response peaks and slopes). Here, we test the idea that all classic tuning curves can be viewed as slices of a higher-dimensional tuning landscape. We use activation-maximizing stimuli ("prototypes") as landmarks in a generative image space and map tuning functions around these peaks. We find that neurons show smooth bell-shaped tuning consistent with radial basis functions, spanning a vast image transformation range, with systematic differences in landscape geometry from V1 to inferotemporal cortex. By modeling these trends, we infer that neurons in the higher visual cortex have higher intrinsic feature dimensionality. Overall, these results suggest that visual neurons are better viewed as signaling distances to prototypes on an image manifold.

Predicting the retinotopic organization of human visual cortex from anatomy using geometric deep learning.

Whether it be in a single neuron or a more complex biological system like the human brain, form and function are often directly related. The functional organization of human visual cortex, for instance, is tightly coupled with the underlying anatomy with cortical shape having been shown to be a useful predictor of the retinotopic organization in early visual cortex. Although the current state-of-the-art in predicting retinotopic maps is able to account for gross individual differences, such models are unable to account for any idiosyncratic differences in the structure-function relationship from anatomical information alone due to their initial assumption of a template. Here we developed a geometric deep learning model capable of exploiting the actual structure of the cortex to learn the complex relationship between brain function and anatomy in human visual cortex such that more realistic and idiosyncratic maps could be predicted. We show that our neural network was not only able to predict the functional organization throughout the visual cortical hierarchy, but that it was also able to predict nuanced variations across individuals. Although we demonstrate its utility for modeling the relationship between structure and function in human visual cortex, our approach is flexible and well-suited for a range of other applications involving data structured in non-Euclidean spaces.

Hierarchical Organization of Corticothalamic Projections to the Pulvinar.

Signals from lower cortical visual areas travel to higher-order areas for further processing through cortico-cortical projections, organized in a hierarchical manner. These signals can also be transferred between cortical areas via alternative cortical transthalamic routes involving higher-order thalamic nuclei like the pulvinar. It is unknown whether the organization of transthalamic pathways may reflect the cortical hierarchy. Two axon terminal types have been identified in corticothalamic (CT) pathways: the types I (modulators) and II (drivers) characterized by thin axons with small terminals and by thick axons and large terminals, respectively. In cats, projections from V1 to the pulvinar complex comprise mainly type II terminals, whereas those from extrastriate areas include a combination of both terminals suggesting that the nature of CT terminals varies with the hierarchical order of visual areas. To test this hypothesis, distribution of CT terminals from area 21a was charted and compared with 3 other visual areas located at different hierarchical levels. Results demonstrate that the proportion of modulatory CT inputs increases along the hierarchical level of cortical areas. This organization of transthalamic pathways reflecting cortical hierarchy provides new and fundamental insights for the establishment of more accurate models of cortical signal processing along transthalamic cortical pathways.

Deep Neural Networks for Modeling Visual Perceptual Learning.

Understanding visual perceptual learning (VPL) has become increasingly more challenging as new phenomena are discovered with novel stimuli and training paradigms. Although existing models aid our knowledge of critical aspects of VPL, the connections shown by these models between behavioral learning and plasticity across different brain areas are typically superficial. Most models explain VPL as readout from simple perceptual representations to decision areas and are not easily adaptable to explain new findings. Here, we show that a well -known instance of deep neural network (DNN), whereas not designed specifically for VPL, provides a computational model of VPL with enough complexity to be studied at many levels of analyses. After learning a Gabor orientation discrimination task, the DNN model reproduced key behavioral results, including increasing specificity with higher task precision, and also suggested that learning precise discriminations could transfer asymmetrically to coarse discriminations when the stimulus conditions varied. Consistent with the behavioral findings, the distribution of plasticity moved toward lower layers when task precision increased and this distribution was also modulated by tasks with different stimulus types. Furthermore, learning in the network units demonstrated close resemblance to extant electrophysiological recordings in monkey visual areas. Altogether, the DNN fulfilled predictions of existing theories regarding specificity and plasticity and reproduced findings of tuning changes in neurons of the primate visual areas. Although the comparisons were mostly qualitative, the DNN provides a new method of studying VPL, can serve as a test bed for theories, and assists in generating predictions for physiological investigations.SIGNIFICANCE STATEMENT Visual perceptual learning (VPL) has been found to cause changes at multiple stages of the visual hierarchy. We found that training a deep neural network (DNN) on an orientation discrimination task produced behavioral and physiological patterns similar to those found in human and monkey experiments. Unlike existing VPL models, the DNN was pre-trained on natural images to reach high performance in object recognition, but was not designed specifically for VPL; however, it fulfilled predictions of existing theories regarding specificity and plasticity and reproduced findings of tuning changes in neurons of the primate visual areas. When used with care, this unbiased and deep-hierarchical model can provide new ways of studying VPL from behavior to physiology.

Revealing Detail along the Visual Hierarchy: Neural Clustering Preserves Acuity from V1 to V4.

How primates perceive objects along with their detailed features remains a mystery. This ability to make fine visual discriminations depends upon a high-acuity analysis of spatial frequency (SF) along the visual hierarchy from V1 to inferotemporal cortex. By studying the transformation of SF across macaque parafoveal V1, V2, and V4, we discovered SF-selective functional domains in V4 encoding higher SFs up to 12 cycles/°. These intermittent higher-SF-selective domains, surrounded by domains encoding lower SFs, violate the inverse relationship between SF preference and retinal eccentricity. The neural activities of higher- and lower-SF domains correspond to local and global features, respectively, of the same stimuli. Neural response latencies in high-SF domains are around 10 ms later than in low-SF domains, consistent with the coarse-to-fine nature of perception. Thus, our finding of preserved resolution from V1 into V4, separated both spatially and temporally, may serve as a connecting link for detailed object representation.

Compressive Temporal Summation in Human Visual Cortex.

Combining sensory inputs over space and time is fundamental to vision. Population receptive field models have been successful in characterizing spatial encoding throughout the human visual pathways. A parallel question, how visual areas in the human brain process information distributed over time, has received less attention. One challenge is that the most widely used neuroimaging method, fMRI, has coarse temporal resolution compared with the time-scale of neural dynamics. Here, via carefully controlled temporally modulated stimuli, we show that information about temporal processing can be readily derived from fMRI signal amplitudes in male and female subjects. We find that all visual areas exhibit subadditive summation, whereby responses to longer stimuli are less than the linear prediction from briefer stimuli. We also find fMRI evidence that the neural response to two stimuli is reduced for brief interstimulus intervals (indicating adaptation). These effects are more pronounced in visual areas anterior to V1-V3. Finally, we develop a general model that shows how these effects can be captured with two simple operations: temporal summation followed by a compressive nonlinearity. This model operates for arbitrary temporal stimulation patterns and provides a simple and interpretable set of computations that can be used to characterize neural response properties across the visual hierarchy. Importantly, compressive temporal summation directly parallels earlier findings of compressive spatial summation in visual cortex describing responses to stimuli distributed across space. This indicates that, for space and time, cortex uses a similar processing strategy to achieve higher-level and increasingly invariant representations of the visual world.SIGNIFICANCE STATEMENT Combining sensory inputs over time is fundamental to seeing. Two important temporal phenomena are summation, the accumulation of sensory inputs over time, and adaptation, a response reduction for repeated or sustained stimuli. We investigated these phenomena in the human visual system using fMRI. We built predictive models that operate on arbitrary temporal patterns of stimulation using two simple computations: temporal summation followed by a compressive nonlinearity. Our new temporal compressive summation model captures (1) subadditive temporal summation, and (2) adaptation. We show that the model accounts for systematic differences in these phenomena across visual areas. Finally, we show that for space and time, the visual system uses a similar strategy to achieve increasingly invariant representations of the visual world.

Invariant Temporal Dynamics Underlie Perceptual Stability in Human Visual Cortex.

An inherent limitation of human visual system research stems from its reliance on highly controlled laboratory conditions. Visual processing in the real world differs substantially from such controlled conditions. In particular, during natural vision, we continuously sample the dynamic environment by variable eye movements that lead to inherent instability of the optical image. The neuronal mechanism by which human perception remains stable under these natural conditions remains unknown. Here, we examined a neural mechanism that may contribute to such stability, i.e., the extent to which neuronal responses remain invariant to oculomotor parameters and viewing conditions. To this end, we introduce an experimental paradigm in which intracranial brain activity, a video of the real-life visual scene, and free oculomotor behavior were simultaneously recorded in human patients. Our results reveal, in high-order visual areas, a remarkable level of neural invariance to the length of eye fixations and lack of evidence for a saccade-related neuronal signature. Thus, neuronal responses, while showing high selectivity to the category of visual images, manifested stable "iconic" dynamics. This property of invariance to fixation onset and duration emerged only in high-order visual representations. In early visual cortex, the fixation onset was accompanied with suppressive neural signal, and duration of neuronal responses was largely determined by the fixation times. These results uncover unique neuronal dynamics in high-order ventral stream visual areas that could play an important role in achieving perceptual stability, despite the drastic changes introduced by oculomotor behavior in real life.

Visual motion serves but is not under the purview of the dorsal pathway.

Visual motion processing is often attributed to the dorsal visual pathway despite visual motion's involvement in almost all visual functions. Furthermore, some visual motion tasks critically depend on the structural integrity of regions outside the dorsal pathway. Here, based on numerous studies, I propose that visual motion signals are swiftly transmitted via multiple non-hierarchical routes to primary motion-dedicated processing regions (MT/V5 and MST) that are not part of the dorsal pathway, and then propagated to a multiplicity of brain areas according to task demands, reaching these regions earlier than the dorsal/ventral hierarchical flow. This not only places MT/V5 at the same or even earlier visual processing stage as that of V1, but can also elucidate many findings with implications to visual awareness. While the integrity of the non-hierarchical motion pathway is necessary for all visual motion perception, it is insufficient on its own, and the transfer of visual motion signals to additional brain areas is crucial to allow the different motion perception tasks (e.g. optic flow, visuo-vestibular balance, movement observation, dynamic form detection and perception, and even reading). I argue that this lateral visual motion pathway can be distinguished from the dorsal pathway not only based on faster response latencies and distinct anatomical connections, but also based on its full field representation. I also distinguish between this primary lateral visual motion pathway sensitive to all motion in the visual field, and a much less investigated optic flow sensitive medial processing pathway (from V1 to V6 and V6A) that appears to be part of the dorsal pathway. Multiple additional predictions are provided that allow testing this proposal and distinguishing between the visual pathways.

Credibility judgments in web page design - a brief review.

Today, more than ever, knowledge that interfaces appearance analysis is a crucial point in human-computer interaction field has been accepted. As nowadays virtually anyone can publish information on the web, the credibility role has grown increasingly important in relation to the web-based content. Areas like trust, credibility, and behavior, doubled by overall impression and user expectation are today in the spotlight of research compared to the last period, when other pragmatic areas such as usability and utility were considered. Credibility has been discussed as a theoretical construct in the field of communication in the past decades and revealed that people tend to evaluate the credibility of communication primarily by the communicator's expertise. Other factors involved in the content communication process are trustworthiness and dynamism as well as various other criteria but to a lower extent. In this brief review, factors like web page aesthetics, browsing experiences and user experience are considered.

Remapping Attention Pointers: Linking Physiology and Behavior.

Our eyes rapidly scan visual scenes, displacing the projection on the retina with every move. Yet these frequent retinal image shifts do not appear to hamper vision. Two recent physiological studies shed new light on the role of attention in visual processing across saccadic eye movements.

Feedforward and feedback frequency-dependent interactions in a large-scale laminar network of the primate cortex.

Interactions between top-down and bottom-up processes in the cerebral cortex hold the key to understanding attentional processes, predictive coding, executive control, and a gamut of other brain functions. However, the underlying circuit mechanism remains poorly understood and represents a major challenge in neuroscience. We approached this problem using a large-scale computational model of the primate cortex constrained by new directed and weighted connectivity data. In our model, the interplay between feedforward and feedback signaling depends on the cortical laminar structure and involves complex dynamics across multiple (intralaminar, interlaminar, interareal, and whole cortex) scales. The model was tested by reproducing, as well as providing insights into, a wide range of neurophysiological findings about frequency-dependent interactions between visual cortical areas, including the observation that feedforward pathways are associated with enhanced gamma (30 to 70 Hz) oscillations, whereas feedback projections selectively modulate alpha/low-beta (8 to 15 Hz) oscillations. Furthermore, the model reproduces a functional hierarchy based on frequency-dependent Granger causality analysis of interareal signaling, as reported in recent monkey and human experiments, and suggests a mechanism for the observed context-dependent hierarchy dynamics. Together, this work highlights the necessity of multiscale approaches and provides a modeling platform for studies of large-scale brain circuit dynamics and functions.

What insights can fMRI offer into the structure and function of mid-tier visual areas?

Inferring neural responses from functional magnetic resonance imaging (fMRI) data is challenging. Even if we take advantage of high-field systems to acquire data with submillimeter resolution, we are still acquiring data in which a single datum summarizes the responses of tens of thousands of neurons. Excitation and inhibition, spikes and subthreshold membrane potential modulations, local and long-range computations, and tuned and nonselective responses are mixed together in one signal. With a priori knowledge of the underlying neural population responses, careful experiment design allows us to manipulate the experiment or task design so that subpopulations are selectively modulated, and our experiments can reveal those tuning functions. However, because we want to be able to use fMRI to discover new kinds of tuning functions and selectivity, we cannot limit ourselves to experiments in which we already know what we are looking for. Broadly speaking, analyses that rely on classification of responses that are distributed across the local neural population [multi-voxel pattern analyses (MVPA)] offer the ability to discover new kinds of information representation and selectivities in neural subpopulations. There is, however, no way to determine how the information discovered with MVPA or other analyses is related to the underlying neuronal tuning functions. Therefore, we must continue to rely on behavioral, computational, and animal models to develop theories of information representation in mid-tier visual cortical areas. Once encoding models exist, fMRI can be powerful for testing these a priori models of information representation. As an aide in developing these models, an important contribution that fMRI can make to our understanding of mid-tier visual areas is derived from connectivity analyses and experiments that study information sharing between visual areas. This ability to quantify localized population average responses throughout the brain is the strength we can best leverage to discover new properties of local and long-range neural networks.

Differential magnetic resonance neurofeedback modulations across extrinsic (visual) and intrinsic (default-mode) nodes of the human cortex.

Previous advances in magnetic resonance imaging allow the analysis of blood oxygen level-dependent signals in real time, thus opening the possibility of feeding an index of these signals back to scanned human participants. However, it is still not known to what extent different cortical networks may differ in their sensitivity to such internally generated neurofeedback (NF). Here, we compare NF efficacy across six cortical regions including: early and high-order visual areas and the posterior parietal lobe, a prominent node of the default mode network (DMN). Our results reveal a consistent difference in NF activation across these areas. Sham controls ruled out a role of attention/arousal in these effects. These differences are suggestive of a relationship to the relative reliance on intrinsic information, moving from early visual cortex (lowest) to the DMN (highest). Interestingly, the visual parahippocampal place area showed NF activation closer to the DMN node. The results are compatible with the notion of the DMN as an intrinsically oriented system.

Delayed signal transmission in area 17, area 18 and the posteromedial lateral suprasylvian area of aged cats.

To investigate the effect of senescence on signal transmission, we have compared the visual response latency and spontaneous activity of cells in the lateral geniculate nucleus (LGN), area 17, area 18 and posteromedial lateral suprasylvian area (PMLS) of young and old cats. We found that LGN cells in old cats exhibit largely normal visual response latency. In contrast, all the other three areas exhibited significant aging-related delays in the visual response latency. On average, PMLS showed most pronounced delays among these three areas. Area 18 slowed more than area 17, but this was not significant. The degradation of signal timing in the visual cortex might provide insight into neuronal response mechanism underlying perception slowing during aging.

Fluctuations of visual awareness: combining motion-induced blindness with binocular rivalry.

Binocular rivalry (BR) and motion-induced blindness (MIB) are two phenomena of visual awareness where perception alternates between multiple states despite constant retinal input. Both phenomena have been extensively studied, but the underlying processing remains unclear. It has been suggested that BR and MIB involve the same neural mechanism, but how the two phenomena compete for visual awareness in the same stimulus has not been systematically investigated. Here we introduce BR in a dichoptic stimulus display that can also elicit MIB and examine fluctuations of visual awareness over the course of each trial. Exploiting this paradigm we manipulated stimulus characteristics that are known to influence MIB and BR. In two experiments we found that effects on multistable percepts were incompatible with the idea of a common oscillator. The results suggest instead that local and global stimulus attributes can affect the dynamics of each percept differently. We conclude that the two phenomena of visual awareness share basic temporal characteristics but are most likely influenced by processing at different stages within the visual system.

The role of pulvinar in the transmission of information in the visual hierarchy.

VISUAL RECEPTIVE FIELD (RF) ATTRIBUTES IN VISUAL CORTEX OF PRIMATES HAVE BEEN EXPLAINED MAINLY FROM CORTICAL CONNECTIONS: visual RFs progress from simple to complex through cortico-cortical pathways from lower to higher levels in the visual hierarchy. This feedforward flow of information is paired with top-down processes through the feedback pathway. Although the hierarchical organization explains the spatial properties of RFs, is unclear how a non-linear transmission of activity through the visual hierarchy can yield smooth contrast response functions in all level of the hierarchy. Depending on the gain, non-linear transfer functions create either a bimodal response to contrast, or no contrast dependence of the response in the highest level of the hierarchy. One possible mechanism to regulate this transmission of visual contrast information from low to high level involves an external component that shortcuts the flow of information through the hierarchy. A candidate for this shortcut is the Pulvinar nucleus of the thalamus. To investigate representation of stimulus contrast a hierarchical model network of ten cortical areas is examined. In each level of the network, the activity from the previous layer is integrated and then non-linearly transmitted to the next level. The arrangement of interactions creates a gradient from simple to complex RFs of increasing size as one moves from lower to higher cortical levels. The visual input is modeled as a Gaussian random input, whose width codes for the contrast. This input is applied to the first area. The output activity ratio among different contrast values is analyzed for the last level to observe sensitivity to a contrast and contrast invariant tuning. For a purely cortical system, the output of the last area can be approximately contrast invariant, but the sensitivity to contrast is poor. To account for an alternative visual processing pathway, non-reciprocal connections from and to a parallel pulvinar like structure of nine areas is coupled to the system. Compared to the pure feedforward model, cortico-pulvino-cortical output presents much more sensitivity to contrast and has a similar level of contrast invariance of the tuning.