Dissociable Cognitive Systems for Recognizing Places and Navigating through Them: Developmental and Neuropsychological Evidence.

Recent neural evidence suggests that the human brain contains dissociable systems for "scene categorization" (i.e., recognizing a place as a particular kind of place, for example, a kitchen), including the parahippocampal place area, and "visually guided navigation" (e.g., finding our way through a kitchen, not running into the kitchen walls or banging into the kitchen table), including the occipital place area. However, converging behavioral data - for instance, whether scene categorization and visually guided navigation abilities develop along different timelines and whether there is differential breakdown under neurologic deficit - would provide even stronger support for this two-scene-systems hypothesis. Thus, here we tested scene categorization and visually guided navigation abilities in 131 typically developing children between 4 and 9 years of age, as well as 46 adults with Williams syndrome, a developmental disorder with known impairment on "action" tasks, yet relative sparing on "perception" tasks, in object processing. We found that (1) visually guided navigation is later to develop than scene categorization, and (2) Williams syndrome adults are impaired in visually guided navigation, but not scene categorization, relative to mental age-matched children. Together, these findings provide the first developmental and neuropsychological evidence for dissociable cognitive systems for recognizing places and navigating through them.SIGNIFICANCE STATEMENT Two decades ago, Milner and Goodale showed us that identifying objects and manipulating them involve distinct cognitive and neural systems. Recent neural evidence suggests that the same may be true of our interactions with our environment: identifying places and navigating through them are dissociable systems. Here we provide converging behavioral evidence supporting this two-scene-systems hypothesis - finding both differential development and breakdown of "scene categorization" and "visually guided navigation." This finding suggests that the division of labor between perception and action systems is a general organizing principle for the visual system, not just a principle of the object processing system in particular.

Transient Oscillations of Neural Firing Rate Associated With Routing of Evidence in a Perceptual Decision.

To form a perceptual decision, the brain must acquire samples of evidence from the environment and incorporate them in computations that mediate choice behavior. While much is known about the neural circuits that process sensory information and those that form decisions, less is known about the mechanisms that establish the functional linkage between them. We trained monkeys of both sexes to make difficult decisions about the net direction of visual motion under conditions that required trial-by-trial control of functional connectivity. In one condition, the motion appeared at different locations on different trials. In the other, two motion patches appeared, only one of which was informative. Neurons in the parietal cortex produced brief oscillations in their firing rate at the time routing was established: upon onset of the motion display when its location was unpredictable across trials, and upon onset of an attention cue that indicated in which of two locations an informative patch of dots would appear. The oscillation was absent when the stimulus location was fixed across trials. We interpret the oscillation as a manifestation of the mechanism that establishes the source and destination of flexibly routed information, but not the transmission of the information per se Significance Statement It has often been suggested that oscillations in neural activity might serve a role in routing information appropriately. We observe an oscillation in neural firing rate in the lateral intraparietal area consistent with such a role. The oscillations are transient. They coincide with the establishment of routing, but they do not appear to play a role in the transmission (or conveyance) of the routed information itself.

Activity in the human superior colliculus associated with reaching for tactile targets.

The superior colliculus (SC) plays a major role in orienting movements of eyes and the head and in the allocation of attention. Functions of the SC have been mostly investigated in animal models, including non-human primates. Differences in the SC's anatomy and function between different species question extrapolations of these studies to humans without further validation. Few electrophysiological and neuroimaging studies in animal models and humans have reported a role of the SC in visually guided reaching movements. Using BOLD fMRI imaging, we sought to decipher if the SC is also active during reaching movements guided by tactile stimulation. Participants executed reaching movements to visual and tactile target positions. When contrasted against visual and tactile stimulation without reaching, we found increased SC activity with reaching not only for visual but also for tactile targets. We conclude that the SC's involvement in reaching does not rely on visual inputs. It is also independent from a specific sensory modality. Our results indicate a general involvement of the human SC in upper limb reaching movements.

A sensory-motor decoder that transforms neural responses in extrastriate area MT into smooth pursuit eye movements.

Visual motion drives smooth pursuit eye movements through a sensory-motor decoder that uses multiple parallel neural pathways to transform the population response in extrastriate area MT into movement. We evaluated the decoder by challenging pursuit in monkeys with reduced motion reliability created by reducing coherence of motion in patches of dots. Our strategy was to determine how reduced dot coherence changes the population response in MT. We then predicted the properties of a decoder that transforms the MT population response into dot coherence-induced deficits in the initiation of pursuit and steady-state tracking. During pursuit initiation, decreased dot coherence reduces MT population response amplitude without changing the preferred speed at its peak. The successful decoder reproduces the measured eye movements by multiplication of 1) the estimate of target speed from the peak of the population response with 2) visual-motor gain based on the amplitude of the population response. During steady-state tracking, the decoder that worked for pursuit initiation failed to reproduce the paradox that steady-state eye speeds do not accelerate to the target speed despite persistent image motion. It predicted eye acceleration to target speed even when monkeys' eye speeds were steady at well below the target speed. To account for the effect of dot coherence on steady-state eye speed, we postulate that the decoder uses sensory-motor gain to modulate the eye velocity positive feedback that normally sustains perfect steady-state tracking. Then, poor steady-state tracking persists because of balance between eye deceleration caused by low positive feedback gain and acceleration driven by MT.NEW & NOTEWORTHY By challenging a sensory-motor system with degraded sensory stimuli, we reveal how the sensory-motor decoder transforms the population response in extrastriate area MT into commands for the initiation and steady-state behavior of smooth pursuit eye movements. Conclusions are based on measuring population responses in MT for multiple target speeds and different levels of motion reliability and evaluating a decoder with a biologically motivated architecture to determine the decoder properties that create the measured eye movements.

Prior Knowledge Uses Prestimulus Alpha Band Oscillations and Persistent Poststimulus Neural Templates for Conscious Perception.

Prior knowledge has a profound impact on the way we perceive the world. However, it remains unclear how the prior knowledge is maintained in our brains and thereby influences the subsequent conscious perception. The Dalmatian dog illusion is a perfect tool to study prior knowledge, where the picture is initially perceived as noise. Once the prior knowledge was introduced, a Dalmatian dog could be consciously seen, and the picture immediately became meaningful. Using pictures with hidden objects as standard stimuli and similar pictures without hidden objects as deviant stimuli, we investigated the neural representation of prior knowledge and its impact on conscious perception in an oddball paradigm using electroencephalogram (EEG) in both male and female human subjects. We found that the neural patterns between the prestimulus alpha band oscillations and poststimulus EEG activity were significantly more similar for the standard stimuli than for the deviant stimuli after prior knowledge was provided. Furthermore, decoding analysis revealed that persistent neural templates were evoked after the introduction of prior knowledge, similar to that evoked in the early stages of visual processing. In conclusion, the current study suggests that prior knowledge uses alpha band oscillations in a multivariate manner in the prestimulus period and induces specific persistent neural templates in the poststimulus period, enabling the conscious perception of the hidden objects.SIGNIFICANCE STATEMENT The visual world we live in is not always optimal. In dark or noisy environments, prior knowledge can help us interpret imperfect sensory signals and enable us to consciously perceive hidden objects. However, we still know very little about how prior knowledge works at the neural level. Using the Dalmatian dog illusion and multivariate methods, we found that prior knowledge uses prestimulus alpha band oscillations to carry information about the hidden object and exerts a persistent influence in the poststimulus period by inducing specific neural templates. Our findings provide a window into the neural underpinnings of prior knowledge and offer new insights into the role of alpha band oscillations and neural templates associated with conscious perception.

Pinging the brain to reveal the hidden attentional priority map using encephalography.

Attention has been usefully thought of as organized in priority maps - putative maps of space where attentional priority is weighted across spatial regions in a winner-take-all competition for attentional deployment. Recent work has highlighted the influence of past experiences on the weighting of spatial priority - called selection history. Aside from being distinct from more well-studied, top-down forms of attentional enhancement, little is known about the neural substrates of history-mediated attentional priority. Using a task known to induce statistical learning of target distributions, in an EEG study we demonstrate that this otherwise invisible, latent attentional priority map can be visualized during the intertrial period using a 'pinging' technique in conjunction with multivariate pattern analyses. Our findings not only offer a method of visualizing the history-mediated attentional priority map, but also shed light on the underlying mechanisms allowing our past experiences to influence future behavior.

Weighted Geometric Mean (WGM) method: A new chromatic adaptation model.

The human visual system has undergone evolutionary changes to develop sophisticated mechanisms that enable stable color perception under varying illumination. These mechanisms are known as chromatic adaptation, a fundamental aspect of color vision. Chromatic adaptation can be divided into two categories: sensory adaptation, which involves automatic adjustments in the visual system, such as retinal gain control, in response to changes in the stimulus, and cognitive adaptation, which depends on the observer's knowledge of the scene and context. The geometric mean has been suggested to be the fundamental mathematical relationship that governs peripheral sensory adaptation. This paper proposes the WGM model, an advanced chromatic adaptation model based on a weighted geometric mean approach that can anticipate incomplete adaptation as it moves along the Planckian or Daylight locus. Compared with two other chromatic adaptation models (CAT16 and vK20), the WGM model is tested with different corresponding color data sets and found to be a significantly improvement while also predicting degree of adaptation (sensory and cognitive adaptation) in a physiologically plausible manner.

High-Throughput Binocular Pattern Electroretinograms in the Mouse.

The pattern electroretinogram (PERG) reflects the electrical activity of retinal ganglion cells (RGC) and has become the most used technology to assess RGC function in experimental models of glaucoma and optic neuropathies. We describe a novel method for obtaining user-friendly, robust PERG simultaneously from each eye using asynchronous binocular stimulation and one-channel acquisition of signals recorded from a subcutaneous needle in the snout.

Cross-Subject Transfer Method Based on Domain Generalization for Facilitating Calibration of SSVEP-Based BCIs.

In steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), various spatial filtering methods based on individual calibration data have been proposed to alleviate the interference of spontaneous activities in SSVEP signals for enhancing the SSVEP detection performance. However, the time-consuming calibration session would increase the visual fatigue of subjects and reduce the usability of the BCI system. The key idea of this study is to propose a cross-subject transfer method based on domain generalization, which transfers the domain-invariant spatial filters and templates learned from source subjects to the target subject with no access to the EEG data from the target subject. The transferred spatial filters and templates are obtained by maximizing the intra- and inter-subject correlations using the SSVEP data corresponding to the target and its neighboring stimuli. For SSVEP detection of the target subject, four types of correlation coefficients are calculated to construct the feature vector. Experimental results estimated with three SSVEP datasets show that the proposed cross-subject transfer method improves the SSVEP detection performance compared to state-of-art methods. The satisfactory results demonstrate that the proposed method provides an effective transfer learning strategy requiring no tedious data collection process for new users, holding the potential of promoting practical applications of SSVEP-based BCI.

Role of ON and OFF Visual Pathways in Rod- and Cone-Driven Flicker Responses.

PURPOSE: To evaluate the effects of various retinal neurotransmitters on temporal resolution, particularly, on the Critical Flicker Fusion Frequency (CFF), which has been previously applied in ophthalmic pathophysiologic research. METHODS: A binocular physiologic electroretinogram was performed on adult mice. Animals in the control group were injected in the right eye with 1 µL of phosphate-buffered saline (PBS). Animals in the experimental group were injected in the left eye with 1 µL of PBS and in the right eye with 1 µL of PBS to which different molecules were added: 2-amino-4-phosphonobutyric acid (APB), Glutamate, γ-aminobutyric acid (GABA), 6,7-dinitroquinoxaline-2,3-dione (DNQX), Bicuculline, Glycine, and 4-(2-hydroxyethyl)-1-piperazineethanesulfonic acid (HEPES). Initially, rod response was recorded and later the cone response. RESULTS: APB suppressed the rod-driven, but not the cone-driven flicker response. The other agents severely affected the lower flickering frequency response amplitude, in particular, at 3 Hz. The threshold of CFF was lowered from 50 Hz to 40 Hz after applying APB, Glycine, and HEPES. GABA remarkably enhanced rod-driven and cone-driven flicker response at 3 Hz, whereas Glutamate and GABA/Glutamate only did in rod-driven flicker response. CONCLUSIONS: Both ON and OFF visual pathways were implied in cone-driven response, but only the ON visual pathway appears to play a relevant role in rod-driven flicker response. Flicker response seems to be enhanced by horizontal cells both in rod-driven and cone-driven response. In addition, due to the greater sensitivity of the flicker at low frequencies, it is suggested that pathophysiological studies should be carried out at said frequencies.

Correlation between Cognitive Performance and Interhemispheric Asymmetry of EEG Alpha Rhythm during Lateralized Presentation of Visual Information against the Background of Optical Stimulation.

The effect of lateralized optical stimulation with a frequency of 10 Hz on the effectiveness of cognitive task performance (n-back test) was studied in 33 healthy subjects (right-handed men). Test visual information was presented to the right or left visual hemifield under normal conditions and against the background of optical stimulation with a frequency of 10 Hz. The absolute values of the spectral power of the high (10-13 Hz) subrange of alpha-rhythm of EEG (SPα2) were calculated. When test information was sent to the right hemisphere against the background of stimulation, an increase in task performance was revealed in subjects with low SPα2. This was accompanied by an increase in SPα2 in some cortical areas of the contralateral (left) hemisphere and, as a result, an increase in left-side dominance of SPα2. The findings indicate the possibility of using lateralized optical stimulation to improve cognitive task performance, in particular, by changing the interhemispheric asymmetry of the EEG alpha2-rhythm.

Stimulus edges induce orientation tuning in superior colliculus.

Orientation columns exist in the primary visual cortex (V1) of cat and primates but not mouse. Intriguingly, some recent studies reported the presence of orientation and direction columns in the mouse superficial superior colliculus (sSC), while others reported a lack of columnar organization therein. Using in vivo calcium imaging of sSC in the awake mouse brain, we found that the presence of columns is highly stimulus dependent. Specifically, we observed orientation and direction columns formed by sSC neurons retinotopically mapped to the edge of grating stimuli. For both excitatory and inhibitory neurons in sSC, orientation selectivity can be induced by the edge with their preferred orientation perpendicular to the edge orientation. Furthermore, we found that this edge-induced orientation selectivity is associated with saliency encoding. These findings indicate that the tuning properties of sSC neurons are not fixed by circuit architecture but rather dependent on the spatiotemporal properties of the stimulus.

Self-motion perception and sequential decision-making: where are we heading?

To navigate and guide adaptive behaviour in a dynamic environment, animals must accurately estimate their own motion relative to the external world. This is a fundamentally multisensory process involving integration of visual, vestibular and kinesthetic inputs. Ideal observer models, paired with careful neurophysiological investigation, helped to reveal how visual and vestibular signals are combined to support perception of linear self-motion direction, or heading. Recent work has extended these findings by emphasizing the dimension of time, both with regard to stimulus dynamics and the trade-off between speed and accuracy. Both time and certainty-i.e. the degree of confidence in a multisensory decision-are essential to the ecological goals of the system: terminating a decision process is necessary for timely action, and predicting one's accuracy is critical for making multiple decisions in a sequence, as in navigation. Here, we summarize a leading model for multisensory decision-making, then show how the model can be extended to study confidence in heading discrimination. Lastly, we preview ongoing efforts to bridge self-motion perception and navigation per se, including closed-loop virtual reality and active self-motion. The design of unconstrained, ethologically inspired tasks, accompanied by large-scale neural recordings, raise promise for a deeper understanding of spatial perception and decision-making in the behaving animal. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Metacognition in the audiovisual McGurk illusion: perceptual and causal confidence.

Almost all decisions in everyday life rely on multiple sensory inputs that can come from common or independent causes. These situations invoke perceptual uncertainty about environmental properties and the signals' causal structure. Using the audiovisual McGurk illusion, this study investigated how observers formed perceptual and causal confidence judgements in information integration tasks under causal uncertainty. Observers were presented with spoken syllables, their corresponding articulatory lip movements or their congruent and McGurk combinations (e.g. auditory B/P with visual G/K). Observers reported their perceived auditory syllable, the causal structure and confidence for each judgement. Observers were more accurate and confident on congruent than unisensory trials. Their perceptual and causal confidence were tightly related over trials as predicted by the interactive nature of perceptual and causal inference. Further, observers assigned comparable perceptual and causal confidence to veridical 'G/K' percepts on audiovisual congruent trials and their causal and perceptual metamers on McGurk trials (i.e. illusory 'G/K' percepts). Thus, observers metacognitively evaluate the integrated audiovisual percept with limited access to the conflicting unisensory stimulus components on McGurk trials. Collectively, our results suggest that observers form meaningful perceptual and causal confidence judgements about multisensory scenes that are qualitatively consistent with principles of Bayesian causal inference. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Visuo-vestibular heading perception: a model system to study multi-sensory decision making.

Integrating noisy signals across time as well as sensory modalities, a process named multi-sensory decision making (MSDM), is an essential strategy for making more accurate and sensitive decisions in complex environments. Although this field is just emerging, recent extraordinary works from different perspectives, including computational theory, psychophysical behaviour and neurophysiology, begin to shed new light onto MSDM. In the current review, we focus on MSDM by using a model system of visuo-vestibular heading. Combining well-controlled behavioural paradigms on virtual-reality systems, single-unit recordings, causal manipulations and computational theory based on spiking activity, recent progress reveals that vestibular signals contain complex temporal dynamics in many brain regions, including unisensory, multi-sensory and sensory-motor association areas. This challenges the brain for cue integration across time and sensory modality such as optic flow which mainly contains a motion velocity signal. In addition, new evidence from the higher-level decision-related areas, mostly in the posterior and frontal/prefrontal regions, helps revise our conventional thought on how signals from different sensory modalities may be processed, converged, and moment-by-moment accumulated through neural circuits for forming a unified, optimal perceptual decision. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions.

Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Crossmodal visual predictions elicit spatially specific early visual cortex activity but later than real visual stimuli.

Previous studies have indicated that crossmodal visual predictions are instrumental in controlling early visual cortex activity. The exact time course and spatial precision of such crossmodal top-down influences on the visual cortex have been unknown. In the present study, participants were exposed to audiovisual combinations comprising one of two sounds and a Gabor patch either in the top left or in the bottom right visual field. Event-related potentials (ERPs) were recorded to these frequent crossmodal combinations (standards) as well as to trials in which the visual stimulus was omitted (omissions) or the visual and auditory stimuli were recombined (deviants). Standards and deviants elicited an ERP between 50 and 100 ms of opposite polarity known as the C1 effect commonly associated with retinotopic processing in early visual cortex. By contrast, a C1 effect was not observed in omission trials. Spatially specific omission and mismatch effects (deviants minus standards) started only later with a latency of 230 ms and 170 ms, respectively. These results suggest that crossmodal visual predictions control visual cortex activity in a spatially specific manner. However, visual predictions do not modulate visual cortex activity with the same timing as visual stimulation activates these areas but rather seem to involve distinct neural mechanisms. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

The role of conflict processing in multisensory perception: behavioural and electroencephalography evidence.

To form coherent multisensory perceptual representations, the brain must solve a causal inference problem: to decide if two sensory cues originated from the same event and should be combined, or if they came from different events and should be processed independently. According to current models of multisensory integration, during this process, the integrated (common cause) and segregated (different causes) internal perceptual models are entertained. In the present study, we propose that the causal inference process involves competition between these alternative perceptual models that engages the brain mechanisms of conflict processing. To test this hypothesis, we conducted two experiments, measuring reaction times (RTs) and electroencephalography, using an audiovisual ventriloquist illusion paradigm with varying degrees of intersensory disparities. Consistent with our hypotheses, incongruent trials led to slower RTs and higher fronto-medial theta power, both indicative of conflict. We also predicted that intermediate disparities would yield slower RTs and higher theta power when compared to congruent stimuli and to large disparities, owing to the steeper competition between causal models. Although this prediction was only validated in the RT study, both experiments displayed the anticipated trend. In conclusion, our findings suggest a potential involvement of the conflict mechanisms in multisensory integration of spatial information. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Metacognitive awareness in the sound-induced flash illusion.

Hundreds (if not thousands) of multisensory studies provide evidence that the human brain can integrate temporally and spatially discrepant stimuli from distinct modalities into a singular event. This process of multisensory integration is usually portrayed in the scientific literature as contributing to our integrated, coherent perceptual reality. However, missing from this account is an answer to a simple question: how do confidence judgements compare between multisensory information that is integrated across multiple sources, and multisensory information that comes from a single, congruent source in the environment? In this paper, we use the sound-induced flash illusion to investigate if confidence judgements are similar across multisensory conditions when the numbers of auditory and visual events are the same, and the numbers of auditory and visual events are different. Results showed that congruent audiovisual stimuli produced higher confidence than incongruent audiovisual stimuli, even when the perceptual report was matched across the two conditions. Integrating these behavioural findings with recent neuroimaging and theoretical work, we discuss the role that prefrontal cortex may play in metacognition, multisensory causal inference and sensory source monitoring in general. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Early Visual Deprivation Impairs Functional Development of the Visual Ventral Stream.

Purpose: To probe the dynamic alternations of neural networks in real-time visual processing after visual deprivation (VD) removal. Methods: A prospective cross-sectional study was conducted. Twenty children with a history of early binocular VD caused by congenital cataracts and 20 matched typically developing (TD) children were enrolled. The event-related potential (ERP) data were obtained via high-density electroencephalography. ERP data were analyzed based on three components (P1, N170, and P2), three test conditions (objects, human faces, and Chinese characters), and peak time and region of interest (ROI) chosen on a grand average head map collapsed from the averaged waveform of each group. Source localization and alpha power spectrum density were applied to define the functional pattern of brain areas and evaluate the attention function. Results: The VD group showed significantly lower P1 amplitudes than the TD group under all conditions in peak ROIs, which were situated in the left occipito-temporal region. For both VD and TD groups, there were strong N170 effects in the character and human face conditions in the component's peak ROIs. Furthermore, source mapping indicated that the VD group generally showed significantly lower activation in the visual cortex and ventral stream, whereas the beyond network areas (mostly frontal areas) intensively participated in functional compensation in the VD group. The VD group showed significant poststimulus alpha desynchronization in object recognition. Conclusions: Our research described the mechanisms of visual networks after early binocular VD removal. Our findings may provide a new basis for the poor visual recovery after early binocular VD removal and offer clues for visual recovery strategies.