Cross-fixation interactions of orientations suggest high-to-low-level decoding in visual working memory.

Sensory encoding (how stimuli evoke sensory responses) is known to progress from low- to high-level features. Decoding (how responses lead to perception) is less understood but is often assumed to follow the same hierarchy. Accordingly, orientation decoding must occur in low-level areas such as V1, without cross-fixation interactions. However, a study, Ding, Cueva, Tsodyks, and Qian (2017), provided evidence against the assumption and proposed that visual decoding may often follow a high-to-low-level hierarchy in working memory, where higher-to-lower-level constraints introduce interactions among lower-level features. If two orientations on opposite sides of the fixation are both task relevant and enter working memory, then they should interact with each other. We indeed found the predicted cross-fixation interactions (repulsion and correlation) between orientations. Control experiments and analyses ruled out alternative explanations such as reporting bias and adaptation across trials on the same side of the fixation. Moreover, we explained the data using a retrospective high-to-low-level Bayesian decoding framework.

Visual Detection and Decoding Skills of Aerial Photography by Adults with Autism Spectrum Disorder (ASD).

Despite challenges in social communication skills people with ASD often display strengths in visual processing. Aerial photography analysis is an occupation reliant on strong visual processing skills that matches this unique profile. We investigated basic-vision and "real-life" visual tasks in 20 cognitively-able young adults with ASD and 20 typically-developed (TD) "gamers". Basic-vision tests included Visual-Search, Embedded-Figures, and Vigilance; "real-life" tests included aerial-photograph detection and identification. Groups performed equally well, and did not differ significantly on any tasks. The study demonstrates strong visual skills in people with ASD in basic and "real-life" settings, and supports the idea that they may be well suited for employment in occupations that demand high visual perception skills such as aerial photography analysis.

Perception-to-Image: Reconstructing Natural Images from the Brain Activity of Visual Perception.

The reappearance of human visual perception is a challenging topic in the field of brain decoding. Due to the complexity of visual stimuli and the constraints of fMRI data collection, the present decoding methods can only reconstruct the basic outline or provide similar figures/features of the perceived natural stimuli. To achieve a high-quality and high-resolution reconstruction of natural images from brain activity, this paper presents an end-to-end perception reconstruction model called the similarity-conditions generative adversarial network (SC-GAN), where visually perceptible images are reconstructed based on human visual cortex responses. The SC-GAN extracts the high-level semantic features of natural images and corresponding visual cortical responses and then introduces the semantic features as conditions of generative adversarial networks (GANs) to realize the perceptual reconstruction of visual images. The experimental results show that the semantic features extracted from SC-GAN play a key role in the reconstruction of natural images. The similarity between the presented and reconstructed images obtained by the SC-GAN is significantly higher than that obtained by a condition generative adversarial network (C-GAN). The model we proposed offers a potential perspective for decoding the brain activity of complex natural stimuli.

A study of decoding human brain activities from simultaneous data of EEG and fMRI using MVPA.

Neuroscientists have investigated the functionality of the brain in detail and achieved remarkable results but this area still need further research. Functional magnetic resonance imaging (fMRI) is considered as the most reliable and accurate technique to decode the human brain activity, on the other hand electroencephalography (EEG) is a portable and low cost solution in brain research. The purpose of this study is to find whether EEG can be used to decode the brain activity patterns like fMRI. In fMRI, data from a very specific brain region is enough to decode the brain activity patterns due to the quality of data. On the other hand, EEG can measure the rapid changes in neuronal activity patterns due to its higher temporal resolution i.e., in msec. These rapid changes mostly occur in different brain regions. In this study, multivariate pattern analysis (MVPA) is used both for EEG and fMRI data analysis and the information is extracted from distributed activation patterns of the brain. The significant information among different classes is extracted using two sample t test in both data sets. Finally, the classification analysis is done using the support vector machine. A fair comparison of both data sets is done using the same analysis techniques, moreover simultaneously collected data of EEG and fMRI is used for this comparison. The final analysis is done with the data of eight participants; the average result of all conditions are found which is 65.7% for EEG data set and 64.1% for fMRI data set. It concludes that EEG is capable of doing brain decoding with the data from multiple brain regions. In other words, decoding accuracy with EEG MVPA is as good as fMRI MVPA and is above chance level.

Feedback from higher to lower visual areas for visual recognition may be weaker in the periphery: Glimpses from the perception of brief dichoptic stimuli.

Eye movements bring attended visual inputs to the center of vision for further processing. Thus, central and peripheral vision should have different functional roles. Here, we use observations of visual perception under dichoptic stimuli to infer that there is a difference in the top-down feedback from higher brain centers to primary visual cortex. Visual stimuli to the two eyes were designed such that the sum and difference of the binocular input from the two eyes have the form of two different gratings. These gratings differed in their motion direction, tilt direction, or color, and duly evoked ambiguous percepts for the corresponding feature. Observers were more likely to perceive the feature in the binocular summation rather than the difference channel. However, this perceptual bias towards the binocular summation signal was weaker or absent in peripheral vision, even when central and peripheral vision showed no difference in contrast sensitivity to the binocular summation signal relative to that to the binocular difference signal. We propose that this bias can arise from top-down feedback as part of an analysis-by-synthesis computation. The feedback is of the input predicted using prior information by the upper level perceptual hypothesis about the visual scene; the hypothesis is verified by comparing the feedback with the actual visual input. We illustrate this process using a conceptual circuit model. In this framework, a bias towards binocular summation can arise from the prior knowledge that inputs are usually correlated between the two eyes. Accordingly, a weaker bias in the periphery implies that the top-down feedback is weaker there. Testable experimental predictions are presented and discussed.

Decoding and encoding of visual patterns using magnetoencephalographic data represented in manifolds.

Visual decoding and encoding are crucial aspects in investigating the representation of visual information in the human brain. This paper proposes a bidirectional model for decoding and encoding of visual stimulus based on manifold representation of the temporal and spatial information extracted from magnetoencephalographic data. In the proposed decoding process, principal component analysis is applied to extract temporal principal components (TPCs) from the visual cortical activity estimated by a beamforming method. The spatial distribution of each TPC is in a high-dimensional space and can be mapped to the corresponding spatiotemporal component (STC) on a low-dimensional manifold. Once the linear mapping between the STC and the wavelet coefficients of the stimulus image is determined, the decoding process can synthesize an image resembling the stimulus image. The encoding process is performed by reversing the mapping or transformation in the decoding model and can predict the spatiotemporal brain activity from a stimulus image. In our experiments using visual stimuli containing eleven combinations of checkerboard patches, the information of spatial layout in the stimulus image was revealed in the embedded manifold. The correlation between the reconstructed and original images was 0.71 and the correlation map between the predicted and original brain activity was highly correlated to the map between the original brain activity for different stimuli (r=0.89). These results suggest that the temporal component is important in visual processing and manifolds can well represent the information related to visual perception.