The Human Posterior Superior Temporal Sulcus Samples Visual Space Differently From Other Face-Selective Regions.

Neuroimaging studies show that ventral face-selective regions, including the fusiform face area (FFA) and occipital face area (OFA), preferentially respond to faces presented in the contralateral visual field (VF). In the current study we measured the VF response of the face-selective posterior superior temporal sulcus (pSTS). Across 3 functional magnetic resonance imaging experiments, participants viewed face videos presented in different parts of the VF. Consistent with prior results, we observed a contralateral VF bias in bilateral FFA, right OFA (rOFA), and bilateral human motion-selective area MT+. Intriguingly, this contralateral VF bias was absent in the bilateral pSTS. We then delivered transcranial magnetic stimulation (TMS) over right pSTS (rpSTS) and rOFA, while participants matched facial expressions in both hemifields. TMS delivered over the rpSTS disrupted performance in both hemifields, but TMS delivered over the rOFA disrupted performance in the contralateral hemifield only. These converging results demonstrate that the contralateral bias for faces observed in ventral face-selective areas is absent in the pSTS. This difference in VF response is consistent with face processing models proposing 2 functionally distinct pathways. It further suggests that these models should account for differences in interhemispheric connections between the face-selective areas across these 2 pathways.

Moving Beyond the Keypress: As Technology Advances, so Should Psychology Response Time Measurements.

Decades of research in cognitive psychology have largely relied on simple key or button presses to quantify human behavior. While many valuable discoveries have been made, a richer response modality may reveal more information regarding the different processes that underlie complex human behavior. This study provides a proof of concept for using a touch-and-swipe response method to separate response time into two components to extract more meaningful behavioral insights. Across several analyses, the two components were consistently shown to be separable, independent measurements of behavior. Furthermore, evaluating these isolated response time components improved inferential power and clarity of behavioral patterns. The touch-and-swipe response method is simple and easy-to-use, and it shows promise for more accurately targeting mechanisms of interest.

Differential Representations of Perceived and Retrieved Visual Information in Hippocampus and Cortex.

Memory retrieval is thought to depend on interactions between hippocampus and cortex, but the nature of representation in these regions and their relationship remains unclear. Here, we performed an ultra-high field fMRI (7T) experiment, comprising perception, learning and retrieval sessions. We observed a fundamental difference between representations in hippocampus and high-level visual cortex during perception and retrieval. First, while object-selective posterior fusiform cortex showed consistent responses that allowed us to decode object identity across both perception and retrieval one day after learning, object decoding in hippocampus was much stronger during retrieval than perception. Second, in visual cortex but not hippocampus, there was consistency in response patterns between perception and retrieval, suggesting that substantial neural populations are shared for both perception and retrieval. Finally, the decoding in hippocampus during retrieval was not observed when retrieval was tested on the same day as learning suggesting that the retrieval process itself is not sufficient to elicit decodable object representations. Collectively, these findings suggest that while cortical representations are stable between perception and retrieval, hippocampal representations are much stronger during retrieval, implying some form of reorganization of the representations between perception and retrieval.

Differential Sampling of Visual Space in Ventral and Dorsal Early Visual Cortex.

A fundamental feature of cortical visual processing is the separation of visual processing for the upper and lower visual fields. In early visual cortex (EVC), the upper visual field is processed ventrally, with the lower visual field processed dorsally. This distinction persists into several category-selective regions of occipitotemporal cortex, with ventral and lateral scene-, face-, and object-selective regions biased for the upper and lower visual fields, respectively. Here, using an elliptical population receptive field (pRF) model, we systematically tested the sampling of visual space within ventral and dorsal divisions of human EVC in both male and female participants. We found that (1) pRFs tend to be elliptical and oriented toward the fovea with distinct angular distributions for ventral and dorsal divisions of EVC, potentially reflecting a radial bias; and (2) pRFs in ventral areas were larger (∼1.5×) and more elliptical (∼1.2×) than those in dorsal areas. These differences potentially reflect a tendency for receptive fields in ventral temporal cortex to overlap the fovea with less emphasis on precise localization and isotropic representation of space compared with dorsal areas. Collectively, these findings suggest that ventral and dorsal divisions of EVC sample visual space differently, likely contributing to and/or stemming from the functional differentiation of visual processing observed in higher-level regions of the ventral and dorsal cortical visual pathways.SIGNIFICANCE STATEMENT The processing of visual information from the upper and lower visual fields is separated in visual cortex. Although ventral and dorsal divisions of early visual cortex (EVC) are commonly assumed to sample visual space equivalently, we demonstrate systematic differences using an elliptical population receptive field (pRF) model. Specifically, we demonstrate that (1) ventral and dorsal divisions of EVC exhibit diverging distributions of pRF angle, which are biased toward the fovea; and (2) ventral pRFs exhibit higher aspect ratios and cover larger areas than dorsal pRFs. These results suggest that ventral and dorsal divisions of EVC sample visual space differently and that such differential sampling likely contributes to different functional roles attributed to the ventral and dorsal pathways, such as object recognition and visually guided attention, respectively.

Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images.

Numerous factors have been reported to underlie the representation of complex images in high-level human visual cortex, including categories (e.g. faces, objects, scenes), animacy, and real-world size, but the extent to which this organization reflects behavioral judgments of real-world stimuli is unclear. Here, we compared representations derived from explicit behavioral similarity judgments and ultra-high field (7T) fMRI of human visual cortex for multiple exemplars of a diverse set of naturalistic images from 48 object and scene categories. While there was a significant correlation between similarity judgments and fMRI responses, there were striking differences between the two representational spaces. Behavioral judgements primarily revealed a coarse division between man-made (including humans) and natural (including animals) images, with clear groupings of conceptually-related categories (e.g. transportation, animals), while these conceptual groupings were largely absent in the fMRI representations. Instead, fMRI responses primarily seemed to reflect a separation of both human and non-human faces/bodies from all other categories. Further, comparison of the behavioral and fMRI representational spaces with those derived from the layers of a deep neural network (DNN) showed a strong correspondence with behavior in the top-most layer and with fMRI in the mid-level layers. These results suggest a complex relationship between localized responses in high-level visual cortex and behavioral similarity judgments - each domain reflects different properties of the images, and responses in high-level visual cortex may correspond to intermediate stages of processing between basic visual features and the conceptual categories that dominate the behavioral response.

Privileged Functional Connectivity between the Visual Word Form Area and the Language System.

The visual word form area (VWFA) is a region in the left occipitotemporal sulcus of literate individuals that is purportedly specialized for visual word recognition. However, there is considerable controversy about its functional specificity and connectivity, with some arguing that it serves as a domain-general, rather than word-specific, visual processor. The VWFA is a critical region for testing hypotheses about the nature of cortical organization, because it is known to develop only through experience (i.e., reading acquisition), and widespread literacy is too recent to have influenced genetic determinants of brain organization. Using a combination of advanced fMRI analysis techniques, including individual functional localization, multivoxel pattern analysis, and high-resolution resting-state functional connectivity (RSFC) analyses, with data from 33 healthy adult human participants, we demonstrate that (1) the VWFA can discriminate words from nonword letter strings (pseudowords); (2) the VWFA has preferential RSFC with Wernicke's area and other core regions of the language system; and (3) the strength of the RSFC between the VWFA and Wernicke's area predicts performance on a semantic classification task with words but not other categories of visual stimuli. Our results are consistent with the hypothesis that the VWFA is specialized for lexical processing of real words because of its functional connectivity with Wernicke's area.SIGNIFICANCE STATEMENT The visual word form area (VWFA) is critical for determining the nature of category-related organization of the ventral visual system. However, its functional specificity and connectivity are fiercely debated. Recent work concluded that the VWFA is a domain-general, rather than word-specific, visual processor with no preferential functional connectivity with the language system. Using more advanced techniques, our results stand in stark contrast to these earlier findings. We demonstrate that the VWFA is highly specialized for lexical processing of real words, and that a fundamental factor driving this specialization is its preferential intrinsic functional connectivity with core regions of the language system. Our results support the hypothesis that intrinsic functional connectivity contributes to category-related specialization within the human ventral visual system.

Visual Search: You Are Who You Are (+ A Learning Curve).

Not everyone is equally well suited for every endeavor-individuals differ in their strengths and weaknesses, which makes some people better at performing some tasks than others. As such, it might be possible to predict individuals' peak competence (i.e., ultimate level of success) on a given task based on their early performance in that task. The current study leveraged "big data" from the mobile game, Airport Scanner (Kedlin Company), to assess the possibility of predicting individuals' ultimate visual search competency using the minimum possible unit of data: response time on a single visual search trial. Those who started out poorly were likely to stay relatively poor and those who started out strong were likely to remain top performers. This effect was apparent at the level of a single trial (in fact, the first trial), making it possible to use raw response time to predict later levels of success.

Task context impacts visual object processing differentially across the cortex.

Perception reflects an integration of "bottom-up" (sensory-driven) and "top-down" (internally generated) signals. Although models of visual processing often emphasize the central role of feed-forward hierarchical processing, less is known about the impact of top-down signals on complex visual representations. Here, we investigated whether and how the observer's goals modulate object processing across the cortex. We examined responses elicited by a diverse set of objects under six distinct tasks, focusing on either physical (e.g., color) or conceptual properties (e.g., man-made). Critically, the same stimuli were presented in all tasks, allowing us to investigate how task impacts the neural representations of identical visual input. We found that task has an extensive and differential impact on object processing across the cortex. First, we found task-dependent representations in the ventral temporal and prefrontal cortex. In particular, although object identity could be decoded from the multivoxel response within task, there was a significant reduction in decoding across tasks. In contrast, the early visual cortex evidenced equivalent decoding within and across tasks, indicating task-independent representations. Second, task information was pervasive and present from the earliest stages of object processing. However, although the responses of the ventral temporal, prefrontal, and parietal cortex enabled decoding of both the type of task (physical/conceptual) and the specific task (e.g., color), the early visual cortex was not sensitive to type of task and could only be used to decode individual physical tasks. Thus, object processing is highly influenced by the behavioral goal of the observer, highlighting how top-down signals constrain and inform the formation of visual representations.

A new neural framework for visuospatial processing.

The division of cortical visual processing into distinct dorsal and ventral streams is a key framework that has guided visual neuroscience. The characterization of the ventral stream as a 'What' pathway is relatively uncontroversial, but the nature of dorsal stream processing is less clear. Originally proposed as mediating spatial perception ('Where'), more recent accounts suggest it primarily serves non-conscious visually guided action ('How'). Here, we identify three pathways emerging from the dorsal stream that consist of projections to the prefrontal and premotor cortices, and a major projection to the medial temporal lobe that courses both directly and indirectly through the posterior cingulate and retrosplenial cortices. These three pathways support both conscious and non-conscious visuospatial processing, including spatial working memory, visually guided action and navigation, respectively.

Evaluating the correspondence between face-, scene-, and object-selectivity and retinotopic organization within lateral occipitotemporal cortex.

The organization of human lateral occipitotemporal cortex (lOTC) has been characterized largely according to two distinct principles: retinotopy and category-selectivity. Whereas category-selective regions were originally thought to exist beyond retinotopic maps, recent evidence highlights overlap. Here, we combined detailed mapping of retinotopy, using population receptive fields (pRF), and category-selectivity to examine and contrast the retinotopic profiles of scene- (occipital place area, OPA), face- (occipital face area, OFA) and object- (lateral occipital cortex, LO) selective regions of lOTC. We observe striking differences in the relationship each region has to underlying retinotopy. Whereas OPA overlapped multiple retinotopic maps (including V3A, V3B, LO1, and LO2), and LO overlapped two maps (LO1 and LO2), OFA overlapped almost none. There appears no simple consistent relationship between category-selectivity and retinotopic maps, meaning category-selective regions are not constrained spatially to retinotopic map borders consistently. The multiple maps that overlap OPA suggests it is likely not appropriate to conceptualize it as a single scene-selective region, whereas the inconsistency in any systematic map overlapping OFA suggests it may constitute a more uniform area. Beyond their relationship to retinotopy, all three regions evidenced strongly retinotopic voxels, with pRFs exhibiting a significant bias towards the contralateral lower visual field, despite differences in pRF size, contributing to an emerging literature suggesting this bias is present across much of lOTC. Taken together, these results suggest that whereas category-selective regions are not constrained to consistently contain ordered retinotopic maps, they nonetheless likely inherit retinotopic characteristics of the maps from which they draw information.

Global motion perception deficits in autism are reflected as early as primary visual cortex.

Individuals with autism are often characterized as 'seeing the trees, but not the forest'-attuned to individual details in the visual world at the expense of the global percept they compose. Here, we tested the extent to which global processing deficits in autism reflect impairments in (i) primary visual processing; or (ii) decision-formation, using an archetypal example of global perception, coherent motion perception. In an event-related functional MRI experiment, 43 intelligence quotient and age-matched male participants (21 with autism, age range 15-27 years) performed a series of coherent motion perception judgements in which the amount of local motion signals available to be integrated into a global percept was varied by controlling stimulus viewing duration (0.2 or 0.6 s) and the proportion of dots moving in the correct direction (coherence: 4%, 15%, 30%, 50%, or 75%). Both typical participants and those with autism evidenced the same basic pattern of accuracy in judging the direction of motion, with performance decreasing with reduced coherence and shorter viewing durations. Critically, these effects were exaggerated in autism: despite equal performance at the long duration, performance was more strongly reduced by shortening viewing duration in autism (P < 0.015) and decreasing stimulus coherence (P < 0.008). To assess the neural correlates of these effects we focused on the responses of primary visual cortex and the middle temporal area, critical in the early visual processing of motion signals, as well as a region in the intraparietal sulcus thought to be involved in perceptual decision-making. The behavioural results were mirrored in both primary visual cortex and the middle temporal area, with a greater reduction in response at short, compared with long, viewing durations in autism compared with controls (both P < 0.018). In contrast, there was no difference between the groups in the intraparietal sulcus (P > 0.574). These findings suggest that reduced global motion perception in autism is driven by an atypical response early in visual processing and may reflect a fundamental perturbation in neural circuitry.

Tunnel vision: sharper gradient of spatial attention in autism.

Enhanced perception of detail has long been regarded a hallmark of autism spectrum conditions (ASC), but its origins are unknown. Normal sensitivity on all fundamental perceptual measures-visual acuity, contrast discrimination, and flicker detection-is strongly established in the literature. If individuals with ASC do not have superior low-level vision, how is perception of detail enhanced? We argue that this apparent paradox can be resolved by considering visual attention, which is known to enhance basic visual sensitivity, resulting in greater acuity and lower contrast thresholds. Here, we demonstrate that the focus of attention and concomitant enhancement of perception are sharper in human individuals with ASC than in matched controls. Using a simple visual acuity task embedded in a standard cueing paradigm, we mapped the spatial and temporal gradients of attentional enhancement by varying the distance and onset time of visual targets relative to an exogenous cue, which obligatorily captures attention. Individuals with ASC demonstrated a greater fall-off in performance with distance from the cue than controls, indicating a sharper spatial gradient of attention. Further, this sharpness was highly correlated with the severity of autistic symptoms in ASC, as well as autistic traits across both ASC and control groups. These findings establish the presence of a form of "tunnel vision" in ASC, with far-reaching implications for our understanding of the social and neurobiological aspects of autism.

Slower rate of binocular rivalry in autism.

An imbalance between cortical excitation and inhibition is a central component of many models of autistic neurobiology. We tested a potential behavioral footprint of this proposed imbalance using binocular rivalry, a visual phenomenon in which perceptual experience is thought to mirror the push and pull of excitatory and inhibitory cortical dynamics. In binocular rivalry, two monocularly presented images compete, leading to a percept that alternates between them. In a series of trials, we presented separate images of objects (e.g., a baseball and a broccoli) to each eye using a mirror stereoscope and asked human participants with autism and matched control subjects to continuously report which object they perceived, or whether they perceived a mixed percept. Individuals with autism demonstrated a slower rate of binocular rivalry alternations than matched control subjects, with longer durations of mixed percepts and an increased likelihood to revert to the previously perceived object when exiting a mixed percept. Critically, each of these findings was highly predictive of clinical measures of autistic symptomatology. Control "playback" experiments demonstrated that differences in neither response latencies nor response criteria could account for the atypical dynamics of binocular rivalry we observed in autistic spectrum conditions. Overall, these results may provide an index of atypical cortical dynamics that may underlie both the social and nonsocial symptoms of autism.

Deconstructing visual scenes in cortex: gradients of object and spatial layout information.

Real-world visual scenes are complex cluttered, and heterogeneous stimuli engaging scene- and object-selective cortical regions including parahippocampal place area (PPA), retrosplenial complex (RSC), and lateral occipital complex (LOC). To understand the unique contribution of each region to distributed scene representations, we generated predictions based on a neuroanatomical framework adapted from monkey and tested them using minimal scenes in which we independently manipulated both spatial layout (open, closed, and gradient) and object content (furniture, e.g., bed, dresser). Commensurate with its strong connectivity with posterior parietal cortex, RSC evidenced strong spatial layout information but no object information, and its response was not even modulated by object presence. In contrast, LOC, which lies within the ventral visual pathway, contained strong object information but no background information. Finally, PPA, which is connected with both the dorsal and the ventral visual pathway, showed information about both objects and spatial backgrounds and was sensitive to the presence or absence of either. These results suggest that 1) LOC, PPA, and RSC have distinct representations, emphasizing different aspects of scenes, 2) the specific representations in each region are predictable from their patterns of connectivity, and 3) PPA combines both spatial layout and object information as predicted by connectivity.

Centering cognitive neuroscience on task demands and generalization.

Cognitive neuroscience seeks generalizable theories explaining the relationship between behavioral, physiological and mental states. In pursuit of such theories, we propose a theoretical and empirical framework that centers on understanding task demands and the mutual constraints they impose on behavior and neural activity. Task demands emerge from the interaction between an agent's sensory impressions, goals and behavior, which jointly shape the activity and structure of the nervous system on multiple spatiotemporal scales. Understanding this interaction requires multitask studies that vary more than one experimental component (for example, stimuli and instructions) combined with dense behavioral and neural sampling and explicit testing for generalization across tasks and data modalities. By centering task demands rather than mental processes that tasks are assumed to engage, this framework paves the way for the discovery of new generalizable concepts unconstrained by existing taxonomies, and moves cognitive neuroscience toward an action-oriented, dynamic and integrated view of the brain.

Assessing the interaction between working memory and perception through time.

Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.

Standard experimental paradigm designs and data exclusion practices in cognitive psychology can inadvertently introduce systematic "shadow" biases in participant samples.

Standard cognitive psychology research practices can introduce inadvertent sampling biases that reduce the reliability and generalizability of the findings. Researchers commonly acknowledge and understand that any given study sample is not perfectly generalizable, especially when implementing typical experimental constraints (e.g., limiting recruitment to specific age ranges or to individuals with normal color vision). However, less obvious systematic sampling constraints, referred to here as "shadow" biases, can be unintentionally introduced and can easily go unnoticed. For example, many standard cognitive psychology study designs involve lengthy and tedious experiments with simple, repetitive stimuli. Such testing environments may 1) be aversive to some would-be participants (e.g., those high in certain neurodivergent symptoms) who may self-select not to enroll in such studies, or 2) contribute to participant attrition, both of which reduce the sample's representativeness. Likewise, standard performance-based data exclusion efforts (e.g., minimum accuracy or response time) or attention checks can systematically remove data from participants from subsets of the population (e.g., those low in conscientiousness). This commentary focuses on the theoretical and practical issues behind these non-obvious and often unacknowledged "shadow" biases, offers a simple illustration with real data as a proof of concept of how applying attention checks can systematically skew latent/hidden variables in the included population, and then discusses the broader implications with suggestions for how to manage and reduce, or at a minimum acknowledge, the problem.

Real-world scene representations in high-level visual cortex: it's the spaces more than the places.

Real-world scenes are incredibly complex and heterogeneous, yet we are able to identify and categorize them effortlessly. In humans, the ventral temporal parahippocampal place area (PPA) has been implicated in scene processing, but scene information is contained in many visual areas, leaving their specific contributions unclear. Although early theories of PPA emphasized its role in spatial processing, more recent reports of its function have emphasized semantic or contextual processing. Here, using functional imaging, we reconstructed the organization of scene representations across human ventral visual cortex by analyzing the distributed response to 96 diverse real-world scenes. We found that, although individual scenes could be decoded in both PPA and early visual cortex (EVC), the structure of representations in these regions was vastly different. In both regions, spatial rather than semantic factors defined the structure of representations. However, in PPA, representations were defined primarily by the spatial factor of expanse (open, closed) and in EVC primarily by distance (near, far). Furthermore, independent behavioral ratings of expanse and distance correlated strongly with representations in PPA and peripheral EVC, respectively. In neither region was content (manmade, natural) a major contributor to the overall organization. Furthermore, the response of PPA could not be used to decode the high-level semantic category of scenes even when spatial factors were held constant, nor could category be decoded across different distances. These findings demonstrate, contrary to recent reports, that the response PPA primarily reflects spatial, not categorical or contextual, aspects of real-world scenes.

Disentangling visual imagery and perception of real-world objects.

During mental imagery, visual representations can be evoked in the absence of "bottom-up" sensory input. Prior studies have reported similar neural substrates for imagery and perception, but studies of brain-damaged patients have revealed a double dissociation with some patients showing preserved imagery in spite of impaired perception and others vice versa. Here, we used fMRI and multi-voxel pattern analysis to investigate the specificity, distribution, and similarity of information for individual seen and imagined objects to try and resolve this apparent contradiction. In an event-related design, participants either viewed or imagined individual named object images on which they had been trained prior to the scan. We found that the identity of both seen and imagined objects could be decoded from the pattern of activity throughout the ventral visual processing stream. Further, there was enough correspondence between imagery and perception to allow discrimination of individual imagined objects based on the response during perception. However, the distribution of object information across visual areas was strikingly different during imagery and perception. While there was an obvious posterior-anterior gradient along the ventral visual stream for seen objects, there was an opposite gradient for imagined objects. Moreover, the structure of representations (i.e. the pattern of similarity between responses to all objects) was more similar during imagery than perception in all regions along the visual stream. These results suggest that while imagery and perception have similar neural substrates, they involve different network dynamics, resolving the tension between previous imaging and neuropsychological studies.