A retinotopic code structures the interaction between perception and memory systems.

Conventional views of brain organization suggest that regions at the top of the cortical hierarchy processes internally oriented information using an abstract amodal neural code. Despite this, recent reports have described the presence of retinotopic coding at the cortical apex, including the default mode network. What is the functional role of retinotopic coding atop the cortical hierarchy? Here we report that retinotopic coding structures interactions between internally oriented (mnemonic) and externally oriented (perceptual) brain areas. Using functional magnetic resonance imaging, we observed robust inverted (negative) retinotopic coding in category-selective memory areas at the cortical apex, which is functionally linked to the classic (positive) retinotopic coding in category-selective perceptual areas in high-level visual cortex. These functionally linked retinotopic populations in mnemonic and perceptual areas exhibit spatially specific opponent responses during both bottom-up perception and top-down recall, suggesting that these areas are interlocked in a mutually inhibitory dynamic. These results show that retinotopic coding structures interactions between perceptual and mnemonic neural systems, providing a scaffold for their dynamic interaction.

Examining the latent structure and correlates of sensory reactivity in autism: a multi-site integrative data analysis by the autism sensory research consortium.

BACKGROUND: Differences in responding to sensory stimuli, including sensory hyperreactivity (HYPER), hyporeactivity (HYPO), and sensory seeking (SEEK) have been observed in autistic individuals across sensory modalities, but few studies have examined the structure of these "supra-modal" traits in the autistic population. METHODS: Leveraging a combined sample of 3868 autistic youth drawn from 12 distinct data sources (ages 3-18 years and representing the full range of cognitive ability), the current study used modern psychometric and meta-analytic techniques to interrogate the latent structure and correlates of caregiver-reported HYPER, HYPO, and SEEK within and across sensory modalities. Bifactor statistical indices were used to both evaluate the strength of a "general response pattern" factor for each supra-modal construct and determine the added value of "modality-specific response pattern" scores (e.g., Visual HYPER). Bayesian random-effects integrative data analysis models were used to examine the clinical and demographic correlates of all interpretable HYPER, HYPO, and SEEK (sub)constructs. RESULTS: All modality-specific HYPER subconstructs could be reliably and validly measured, whereas certain modality-specific HYPO and SEEK subconstructs were psychometrically inadequate when measured using existing items. Bifactor analyses supported the validity of a supra-modal HYPER construct (ωH = .800) but not a supra-modal HYPO construct (ωH = .653), and supra-modal SEEK models suggested a more limited version of the construct that excluded some sensory modalities (ωH = .800; 4/7 modalities). Modality-specific subscales demonstrated significant added value for all response patterns. Meta-analytic correlations varied by construct, although sensory features tended to correlate most with other domains of core autism features and co-occurring psychiatric symptoms (with general HYPER and speech HYPO demonstrating the largest numbers of practically significant correlations). LIMITATIONS: Conclusions may not be generalizable beyond the specific pool of items used in the current study, which was limited to caregiver report of observable behaviors and excluded multisensory items that reflect many "real-world" sensory experiences. CONCLUSION: Of the three sensory response patterns, only HYPER demonstrated sufficient evidence for valid interpretation at the supra-modal level, whereas supra-modal HYPO/SEEK constructs demonstrated substantial psychometric limitations. For clinicians and researchers seeking to characterize sensory reactivity in autism, modality-specific response pattern scores may represent viable alternatives that overcome many of these limitations.

Scene Perception and Visuospatial Memory Converge at the Anterior Edge of Visually Responsive Cortex.

To fluidly engage with the world, our brains must simultaneously represent both the scene in front of us and our memory of the immediate surrounding environment (i.e., local visuospatial context). How does the brain's functional architecture enable sensory and mnemonic representations to closely interface while also avoiding sensory-mnemonic interference? Here, we asked this question using first-person, head-mounted virtual reality and fMRI. Using virtual reality, human participants of both sexes learned a set of immersive, real-world visuospatial environments in which we systematically manipulated the extent of visuospatial context associated with a scene image in memory across three learning conditions, spanning from a single FOV to a city street. We used individualized, within-subject fMRI to determine which brain areas support memory of the visuospatial context associated with a scene during recall (Experiment 1) and recognition (Experiment 2). Across the whole brain, activity in three patches of cortex was modulated by the amount of known visuospatial context, each located immediately anterior to one of the three scene perception areas of high-level visual cortex. Individual subject analyses revealed that these anterior patches corresponded to three functionally defined place memory areas, which selectively respond when visually recalling personally familiar places. In addition to showing activity levels that were modulated by the amount of visuospatial context, multivariate analyses showed that these anterior areas represented the identity of the specific environment being recalled. Together, these results suggest a convergence zone for scene perception and memory of the local visuospatial context at the anterior edge of high-level visual cortex.SIGNIFICANCE STATEMENT As we move through the world, the visual scene around us is integrated with our memory of the wider visuospatial context. Here, we sought to understand how the functional architecture of the brain enables coexisting representations of the current visual scene and memory of the surrounding environment. Using a combination of immersive virtual reality and fMRI, we show that memory of visuospatial context outside the current FOV is represented in a distinct set of brain areas immediately anterior and adjacent to the perceptually oriented scene-selective areas of high-level visual cortex. This functional architecture would allow efficient interaction between immediately adjacent mnemonic and perceptual areas while also minimizing interference between mnemonic and perceptual representations.

A retinotopic code structures the interaction between perception and memory systems.

Conventional views of brain organization suggest that the cortical apex processes internally-oriented information using an abstract, amodal neural code. Yet, recent reports have described the presence of retinotopic coding at the cortical apex, including the default mode network. What is the functional role of retinotopic coding atop the cortical hierarchy? Here, we report that retinotopic coding structures interactions between internally-oriented (mnemonic) and externally-oriented (perceptual) brain areas. Using fMRI, we observed robust, inverted (negative) retinotopic coding in category-selective memory areas at the cortical apex, which is functionally linked to the classic (positive) retinotopic coding in category-selective perceptual areas in high-level visual cortex. Specifically, these functionally-linked retinotopic populations in mnemonic and perceptual areas exhibit spatially-specific opponent responses during both bottom-up perception and top-down recall, suggesting that these areas are interlocked in a mutually-inhibitory dynamic. Together, these results show that retinotopic coding structures interactions between perceptual and mnemonic neural systems, thereby scaffolding their dynamic interaction.

Visual processing in genetic conditions linked to autism: A behavioral study of binocular rivalry in individuals with 16p11.2 deletions and age-matched controls.

Close phenotypic characterization of individuals with genetic conditions linked to autism provides a promising approach to navigating the heterogeneity of autism spectrum conditions. The current study investigated sensory processing in individuals with a rare genetic event that is highly penetrant for autism, 16p11.2 deletions, using a well-characterized visual paradigm, binocular rivalry, which is thought to be a non-invasive index of excitatory/inhibitory balance in the visual cortex. We characterized rivalry dynamics in 45 adolescent and adult individuals (19 individuals with 16p11.2 deletions, 26 age-matched neurotypical controls). We found that binocular rivalry perceptual transition rates were significantly slower for individuals with 16p11.2 deletions, relative to controls. Importantly, these results could not be accounted for by differences in motor response latencies or perceptual decision criteria, which were matched between groups. Results should be interpreted with caution given the unmatched psychometric features between groups, such as IQ. Future studies should study visual processing in other genetic groups linked to autism beyond 16p to understand the specificity of these findings. These results highlight the importance of characterizing sensory functions in individuals with genetic alterations associated with autism.

Active visual search in naturalistic environments reflects individual differences in classic visual search performance.

Visual search is a ubiquitous activity in real-world environments. Yet, traditionally, visual search is investigated in tightly controlled paradigms, where head-restricted participants locate a minimalistic target in a cluttered array that is presented on a computer screen. Do traditional visual search tasks predict performance in naturalistic settings, where participants actively explore complex, real-world scenes? Here, we leverage advances in virtual reality technology to test the degree to which classic and naturalistic search are limited by a common factor, set size, and the degree to which individual differences in classic search behavior predict naturalistic search behavior in a large sample of individuals (N = 75). In a naturalistic search task, participants looked for an object within their environment via a combination of head-turns and eye-movements using a head-mounted display. Then, in a classic search task, participants searched for a target within a simple array of colored letters using only eye-movements. In each task, we found that participants' search performance was impacted by increases in set size-the number of items in the visual display. Critically, we observed that participants' efficiency in classic search tasks-the degree to which set size slowed performance-indeed predicted efficiency in real-world scenes. These results demonstrate that classic, computer-based visual search tasks are excellent models of active, real-world search behavior.

Examining the Latent Structure and Correlates of Sensory Reactivity in Autism: A Multi-site Integrative Data Analysis by the Autism Sensory Research Consortium.

Background Differences in responding to sensory stimuli, including sensory hyperreactivity (HYPER), hyporeactivity (HYPO), and sensory seeking (SEEK) have been observed in autistic individuals across sensory modalities, but few studies have examined the structure of these "supra-modal" traits in the autistic population. Methods Leveraging a combined sample of 3,868 autistic youth drawn from 12 distinct data sources (ages 3-18 years and representing the full range of cognitive ability), the current study used modern psychometric and meta-analytic techniques to interrogate the latent structure and correlates of caregiver-reported HYPER, HYPO, and SEEK within and across sensory modalities. Bifactor statistical indices were used to both evaluate the strength of a "general response pattern" factor for each supra-modal construct and determine the added value of "modality-specific response pattern" scores (e.g., Visual HYPER). Bayesian random-effects integrative data analysis models were used to examine the clinical and demographic correlates of all interpretable HYPER, HYPO and SEEK (sub)constructs. Results All modality-specific HYPER subconstructs could be reliably and validly measured, whereas certain modality-specific HYPO and SEEK subconstructs were psychometrically inadequate when measured using existing items. Bifactor analyses unambiguously supported the validity of a supra-modal HYPER construct (ω H = .800), whereas a coherent supra-modal HYPO construct was not supported (ω H = .611), and supra-modal SEEK models suggested a more limited version of the construct that excluded some sensory modalities (ω H = .799; 4/7 modalities). Within each sensory construct, modality-specific subscales demonstrated substantial added value beyond the supra-modal score. Meta-analytic correlations varied by construct, although sensory features tended to correlate most strongly with other domains of core autism features and co-occurring psychiatric symptoms. Certain subconstructs within the HYPO and SEEK domains were also associated with lower adaptive behavior scores. Limitations: Conclusions may not be generalizable beyond the specific pool of items used in the current study, which was limited to parent-report of observable behaviors and excluded multisensory items that reflect many "real-world" sensory experiences. Conclusion Psychometric issues may limit the degree to which some measures of supra-modal HYPO/SEEK can be interpreted. Depending on the research question at hand, modality-specific response pattern scores may represent a valid alternative method of characterizing sensory reactivity in autism.

Brief Report: Differences in Naturalistic Attention to Real-World Scenes in Adolescents with 16p.11.2 Deletion.

Sensory differences are nearly universal in autism, but their genetic origins are poorly understood. Here, we tested how individuals with an autism-linked genotype, 16p.11.2 deletion ("16p"), attend to visual information in immersive, real-world photospheres. We monitored participants' (N = 44) gaze while they actively explored 360° scenes via headmounted virtual reality. We modeled the visually salient and semantically meaningful information in scenes and quantified the relative bottom-up vs. top-down influences on attentional deployment. We found, when compared to typically developed control (TD) participants, 16p participants' attention was less dominantly predicted by semantically meaningful scene regions, relative to visually salient regions. These results suggest that a reduction in top-down relative to bottom-up attention characterizes how individuals with 16p.11.2 deletions engage with naturalistic visual environments.

Evaluating the efficacy of multi-echo ICA denoising on model-based fMRI.

fMRI is an indispensable tool for neuroscience investigation, but this technique is limited by multiple sources of physiological and measurement noise. These noise sources are particularly problematic for analysis techniques that require high signal-to-noise ratio for stable model fitting, such as voxel-wise modeling. Multi-echo data acquisition in combination with echo-time dependent ICA denoising (ME-ICA) represents one promising strategy to mitigate physiological and hardware-related noise sources as well as motion-related artifacts. However, most studies employing ME-ICA to date are resting-state fMRI studies, and therefore we have a limited understanding of the impact of ME-ICA on complex task or model-based fMRI paradigms. Here, we addressed this knowledge gap by comparing data quality and model fitting performance of data acquired during a visual population receptive field (pRF) mapping (N = 13 participants) experiment after applying one of three preprocessing procedures: ME-ICA, optimally combined multi-echo data without ICA-denoising, and typical single echo processing. As expected, multi-echo fMRI improved temporal signal-to-noise compared to single echo fMRI, with ME-ICA amplifying the improvement compared to optimal combination alone. However, unexpectedly, this boost in temporal signal-to-noise did not directly translate to improved model fitting performance: compared to single echo acquisition, model fitting was only improved after ICA-denoising. Specifically, compared to single echo acquisition, ME-ICA resulted in improved variance explained by our pRF model throughout the visual system, including anterior regions of the temporal and parietal lobes where SNR is typically low, while optimal combination without ICA did not. ME-ICA also improved reliability of parameter estimates compared to single echo and optimally combined multi-echo data without ICA-denoising. Collectively, these results suggest that ME-ICA is effective for denoising task-based fMRI data for modeling analyzes and maintains the integrity of the original data. Therefore, ME-ICA may be beneficial for complex fMRI experiments, including voxel-wise modeling and naturalistic paradigms.

Reduced social attention in autism is magnified by perceptual load in naturalistic environments.

Individuals with autism spectrum conditions (ASC) describe differences in both social cognition and sensory processing, but little is known about the causal relationship between these disparate functional domains. In the present study, we sought to understand how a core characteristic of autism-reduced social attention-is impacted by the complex multisensory signals present in real-world environments. We tested the hypothesis that reductions in social attention associated with autism would be magnified by increasing perceptual load (e.g., motion, multisensory cues). Adult participants (N = 40; 19 ASC) explored a diverse set of 360° real-world scenes in a naturalistic, active viewing paradigm (immersive virtual reality + eyetracking). Across three conditions, we systematically varied perceptual load while holding the social and semantic information present in each scene constant. We demonstrate that reduced social attention is not a static signature of the autistic phenotype. Rather, group differences in social attention emerged with increasing perceptual load in naturalistic environments, and the susceptibility of social attention to perceptual load predicted continuous measures of autistic traits across groups. Crucially, this pattern was specific to the social domain: we did not observe differential impacts of perceptual load on attention directed toward nonsocial semantic (i.e., object, place) information or low-level fixation behavior (i.e., overall fixation frequency or duration). This study provides a direct link between social and sensory processing in autism. Moreover, reduced social attention may be an inaccurate characterization of autism. Instead, our results suggest that social attention in autism is better explained by "social vulnerability," particularly to the perceptual load of real-world environments.

A network linking scene perception and spatial memory systems in posterior cerebral cortex.

The neural systems supporting scene-perception and spatial-memory systems of the human brain are well-described. But how do these neural systems interact? Here, using fine-grained individual-subject fMRI, we report three cortical areas of the human brain, each lying immediately anterior to a region of the scene perception network in posterior cerebral cortex, that selectively activate when recalling familiar real-world locations. Despite their close proximity to the scene-perception areas, network analyses show that these regions constitute a distinct functional network that interfaces with spatial memory systems during naturalistic scene understanding. These "place-memory areas" offer a new framework for understanding how the brain implements memory-guided visual behaviors, including navigation.

Current Promises and Limitations of Combined Virtual Reality and Functional Magnetic Resonance Imaging Research in Humans: A Commentary on Huffman and Ekstrom (2019).

Real-world navigation requires movement of the body through space, producing a continuous stream of visual and self-motion signals, including proprioceptive, vestibular, and motor efference cues. These multimodal cues are integrated to form a spatial cognitive map, an abstract, amodal representation of the environment. How the brain combines these disparate inputs and the relative importance of these inputs to cognitive map formation and recall are key unresolved questions in cognitive neuroscience. Recent advances in virtual reality technology allow participants to experience body-based cues when virtually navigating, and thus it is now possible to consider these issues in new detail. Here, we discuss a recent publication that addresses some of these issues (D. J. Huffman and A. D. Ekstrom. A modality-independent network underlies the retrieval of large-scale spatial environments in the human brain. Neuron, 104, 611-622, 2019). In doing so, we also review recent progress in the study of human spatial cognition and raise several questions that might be addressed in future studies.

Active vision in immersive, 360° real-world environments.

How do we construct a sense of place in a real-world environment? Real-world environments are actively explored via saccades, head turns, and body movements. Yet, little is known about how humans process real-world scene information during active viewing conditions. Here, we exploited recent developments in virtual reality (VR) and in-headset eye-tracking to test the impact of active vs. passive viewing conditions on gaze behavior while participants explored novel, real-world, 360° scenes. In one condition, participants actively explored 360° photospheres from a first-person perspective via self-directed motion (saccades and head turns). In another condition, photospheres were passively displayed to participants while they were head-restricted. We found that, relative to passive viewers, active viewers displayed increased attention to semantically meaningful scene regions, suggesting more exploratory, information-seeking gaze behavior. We also observed signatures of exploratory behavior in eye movements, such as quicker, more entropic fixations during active as compared with passive viewing conditions. These results show that active viewing influences every aspect of gaze behavior, from the way we move our eyes to what we choose to attend to. Moreover, these results offer key benchmark measurements of gaze behavior in 360°, naturalistic environments.

Regional balance between glutamate+glutamine and GABA+ in the resting human brain.

Models of healthy brain function and psychiatric conditions assume that excitatory and inhibitory activity are balanced in the human brain at multiple spatial and temporal scales. In human neuroimaging, concentrations of the major excitatory (glutamate) and inhibitory (γ-aminobutyric acid, GABA) neurotransmitters are measured in vivo using magnetic resonance spectroscopy (MRS). However, despite the central importance of E/I balance to theories of brain function, a relationship between regional glutamate and GABA levels in the human brain has not been shown. We addressed this question in a large corpus of edited MRS data collected at 19 different sites (n â€‹= â€‹220). Consistent with the notion of E/I balance, we found that levels of glutamate+glutamine (Glx) and GABA+ were highly correlated (R â€‹= â€‹0.52, p â€‹= â€‹2.86 x 10-14). This relationship held when controlling for site, scanner vendor, and demographics. Controlling for neurochemicals associated with neuronal density and metabolism (i.e. N-acetylaspartate and creatine) significantly reduced the correlation between GABA+ and Glx, suggesting that the levels of GABA+ and Glx may be critically linked to regional metabolism. These results are consistent with the notion that excitation and inhibition are balanced in the human brain.

The limits of color awareness during active, real-world vision.

Color ignites visual experience, imbuing the world with meaning, emotion, and richness. As soon as an observer opens their eyes, they have the immediate impression of a rich, colorful experience that encompasses their entire visual world. Here, we show that this impression is surprisingly inaccurate. We used head-mounted virtual reality (VR) to place observers in immersive, dynamic real-world environments, which they naturally explored via saccades and head turns. Meanwhile, we monitored their gaze with in-headset eye tracking and then systematically altered the visual environments such that only the parts of the scene they were looking at were presented in color and the rest of the scene (i.e., the visual periphery) was entirely desaturated. We found that observers were often completely unaware of these drastic alterations to their visual world. In the most extreme case, almost a third of observers failed to notice when less than 5% of the visual display was presented in color. This limitation on perceptual awareness could not be explained by retinal neuroanatomy or previous studies of peripheral visual processing using more traditional psychophysical approaches. In a second study, we measured color detection thresholds using a staircase procedure while a set of observers intentionally attended to the periphery. Still, we found that observers were unaware when a large portion of their field of view was desaturated. Together, these results show that during active, naturalistic viewing conditions, our intuitive sense of a rich, colorful visual world is largely incorrect.

Slower Binocular Rivalry in the Autistic Brain.

Autism has traditionally been regarded as a disorder of the social brain. Recent reports of differences in visual perception have challenged this notion, but little evidence for altered visual processing in the autistic brain exists. We have previously observed slower behaviorally reported rates of a basic visual phenomenon, binocular rivalry, in autism [1, 2]. During rivalry, two images-one presented to each eye-vie for awareness, alternating back and forth in perception. This competition is modeled to rely, in part, on the balance of excitation and inhibition in visual cortex [3-8], which may be altered in autism [2, 9-14]. Yet direct neural evidence for this potential marker of excitation/inhibition (E/I) balance in autism is lacking. Here, we report a striking alteration in the neural dynamics of binocular rivalry in individuals with autism. Participants viewed true and simulated frequency-tagged binocular rivalry displays while steady-state visually evoked potentials (SSVEPs) were measured over occipital cortex using electroencephalography (EEG). First, we replicate our prior behavioral findings of slower rivalry and reduced perceptual suppression in individuals with autism compared with controls. Second, we provide direct neural evidence for slower rivalry in autism compared with controls, which strongly predicted individuals' behavioral switch rates. Finally, using neural data alone, we were able to predict autism symptom severity (ADOS) and correctly classify individuals' diagnostic status (autistic versus control; 87% accuracy). These findings clearly implicate atypical visual processing in the neurobiology of autism. Down the road, this paradigm may serve as a non-verbal marker of autism for developmental and cross-species research.

GABAergic Inhibition Gates Perceptual Awareness During Binocular Rivalry.

Binocular rivalry is a classic experimental tool to probe the neural machinery of perceptual awareness. During rivalry, perception alternates between the two eyes, and the ebb and flow of perception is modeled to rely on the strength of inhibitory interactions between competitive neuronal populations in visual cortex. As a result, rivalry has been suggested as a noninvasive perceptual marker of inhibitory signaling in visual cortex, and its putative disturbance in psychiatric conditions, including autism. Yet, direct evidence causally implicating inhibitory signaling in the dynamics of binocular rivalry is currently lacking. We previously found that people with higher GABA levels in visual cortex, measured using magnetic resonance spectroscopy, have stronger perceptual suppression during rivalry. Here, we present direct causal tests of the impact of GABAergic inhibition on rivalry dynamics, and the contribution of specific GABA receptors to these dynamics. In a crossover pharmacological design with male and female adult participants, we found that drugs that modulate the two dominant GABA receptor types in the brain, GABAA (clobazam) and GABAB (arbaclofen), increase perceptual suppression during rivalry relative to a placebo. Crucially, these results could not be explained by changes in reaction times or response criteria, as determined through rivalry simulation trials, suggesting a direct and specific influence of GABA on perceptual suppression. A full replication study of the GABAB modulator reinforces these findings. These results provide causal evidence for a link between the strength of inhibition in the brain and perceptual suppression during rivalry and have implications for psychiatric conditions including autism.SIGNIFICANCE STATEMENT How does the brain accomplish perceptual gating? Here we use a direct and causal pharmacological manipulation to present insight into the neural machinery of a classic illusion of perceptual awareness: binocular rivalry. We show that drugs that increase GABAergic inhibition in the brain, clobazam (GABAA modulator) and arbaclofen (GABAB modulator), increase perceptual suppression during rivalry relative to a placebo. These results present the first causal link between GABAergic inhibition and binocular rivalry in humans, complementing classic models of binocular rivalry, and have implications for our understanding of psychiatric conditions, such as autism, where binocular rivalry is posited as a behavioral marker of disruptions in inhibitory signaling in the brain.