A Unifying Model for Discordant and Concordant Results in Human Neuroimaging Studies of Facial Viewpoint Selectivity.

Recognizing faces regardless of their viewpoint is critical for social interactions. Traditional theories hold that view-selective early visual representations gradually become tolerant to viewpoint changes along the ventral visual hierarchy. Newer theories, based on single-neuron monkey electrophysiological recordings, suggest a three-stage architecture including an intermediate face-selective patch abruptly achieving invariance to mirror-symmetric face views. Human studies combining neuroimaging and multivariate pattern analysis (MVPA) have provided convergent evidence of view selectivity in early visual areas. However, contradictory conclusions have been reached concerning the existence in humans of a mirror-symmetric representation like that observed in macaques. We believe these contradictions arise from low-level stimulus confounds and data analysis choices. To probe for low-level confounds, we analyzed images from two face databases. Analyses of image luminance and contrast revealed biases across face views described by even polynomials-i.e., mirror-symmetric. To explain major trends across neuroimaging studies, we constructed a network model incorporating three constraints: cortical magnification, convergent feedforward projections, and interhemispheric connections. Given the identified low-level biases, we show that a gradual increase of interhemispheric connections across network-layers is sufficient to replicate view-tuning in early processing stages and mirror-symmetry in later stages. Data analysis decisions-pattern dissimilarity measure and data recentering-accounted for the inconsistent observation of mirror-symmetry across prior studies. Pattern analyses of human fMRI data (of either sex) revealed biases compatible with our model. The model provides a unifying explanation of MVPA studies of viewpoint selectivity and suggests observations of mirror-symmetry originate from ineffectively normalized signal imbalances across different face views.

Emotion and anxiety interact to bias spatial attention.

Emotional expressions are an evolutionarily conserved means of social communication essential for social interactions. It is important to understand how anxious individuals perceive their social environments, including emotional expressions, especially with the rising prevalence of anxiety during the COVID-19 pandemic. Anxiety is often associated with an attentional bias for threat-related stimuli, such as angry faces. Yet the mechanisms by which anxiety enhances or impairs two key components of spatial attention-attentional capture and attentional disengagement-to emotional expressions are still unclear. Moreover, positive valence is often ignored in studies of threat-related attention and anxiety, despite the high occurrence of happy faces during everyday social interaction. Here, we investigated the relationship between anxiety, emotional valence, and spatial attention in 574 participants across two preregistered studies (data collected in 2021 and 2022; Experiment 1: n = 154, 54.5% male, Mage = 43.5 years; Experiment 2: n = 420, 58% male, Mage = 36.46 years). We found that happy faces capture attention more quickly than angry faces during the visual search experiment and found delayed disengagement from both angry and happy faces over neutral faces during the spatial cueing experiment. We also show that anxiety has a distinct impact on both attentional capture and disengagement of emotional faces. Together, our findings highlight the role of positively valenced stimuli in attracting and holding attention and suggest that anxiety is a critical factor in modulating spatial attention to emotional stimuli. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Pupil Size Is Sensitive to Low-Level Stimulus Features, Independent of Arousal-Related Modulation.

Similar to a camera aperture, pupil size adjusts to the surrounding luminance. Unlike a camera, pupil size is additionally modulated both by stimulus properties and by cognitive processes, including attention and arousal, though the interdependence of these factors is unclear. We hypothesized that different stimulus properties interact to jointly modulate pupil size while remaining independent from the impact of arousal. We measured pupil responses from human observers to equiluminant stimuli during a demanding rapid serial visual presentation (RSVP) task at fixation and tested how response amplitude depends on contrast, spatial frequency, and reward level. We found that under constant luminance, unattended stimuli evoke responses that are separable from changes caused by general arousal or attention. We further uncovered a double-dissociation between task-related responses and stimulus-evoked responses, suggesting that different sources of pupil size modulation are independent of one another. Our results shed light on neural pathways underlying pupillary response.

Representations in human primary visual cortex drift over time.

Primary sensory regions are believed to instantiate stable neural representations, yet a number of recent rodent studies suggest instead that representations drift over time. To test whether sensory representations are stable in human visual cortex, we analyzed a large longitudinal dataset of fMRI responses to images of natural scenes. We fit the fMRI responses using an image-computable encoding model and tested how well the model generalized across sessions. We found systematic changes in model fits that exhibited cumulative drift over many months. Convergent analyses pinpoint changes in neural responsivity as the source of the drift, while population-level representational dissimilarities between visual stimuli were unchanged. These observations suggest that downstream cortical areas may read-out a stable representation, even as representations within V1 exhibit drift.

A unifying model for discordant and concordant results in human neuroimaging studies of facial viewpoint selectivity.

Our ability to recognize faces regardless of viewpoint is a key property of the primate visual system. Traditional theories hold that facial viewpoint is represented by view-selective mechanisms at early visual processing stages and that representations become increasingly tolerant to viewpoint changes in higher-level visual areas. Newer theories, based on single-neuron monkey electrophysiological recordings, suggest an additional intermediate processing stage invariant to mirror-symmetric face views. Consistent with traditional theories, human studies combining neuroimaging and multivariate pattern analysis (MVPA) methods have provided evidence of view-selectivity in early visual cortex. However, contradictory results have been reported in higher-level visual areas concerning the existence in humans of mirror-symmetrically tuned representations. We believe these results reflect low-level stimulus confounds and data analysis choices. To probe for low-level confounds, we analyzed images from two popular face databases. Analyses of mean image luminance and contrast revealed biases across face views described by even polynomials-i.e., mirror-symmetric. To explain major trends across human neuroimaging studies of viewpoint selectivity, we constructed a network model that incorporates three biological constraints: cortical magnification, convergent feedforward projections, and interhemispheric connections. Given the identified low-level biases, we show that a gradual increase of interhemispheric connections across network layers is sufficient to replicate findings of mirror-symmetry in high-level processing stages, as well as view-tuning in early processing stages. Data analysis decisions-pattern dissimilarity measure and data recentering-accounted for the variable observation of mirror-symmetry in late processing stages. The model provides a unifying explanation of MVPA studies of viewpoint selectivity. We also show how common analysis choices can lead to erroneous conclusions.

Natural scene sampling reveals reliable coarse-scale orientation tuning in human V1.

Orientation selectivity in primate visual cortex is organized into cortical columns. Since cortical columns are at a finer spatial scale than the sampling resolution of standard BOLD fMRI measurements, analysis approaches have been proposed to peer past these spatial resolution limitations. It was recently found that these methods are predominantly sensitive to stimulus vignetting - a form of selectivity arising from an interaction of the oriented stimulus with the aperture edge. Beyond vignetting, it is not clear whether orientation-selective neural responses are detectable in BOLD measurements. Here, we leverage a dataset of visual cortical responses measured using high-field 7T fMRI. Fitting these responses using image-computable models, we compensate for vignetting and nonetheless find reliable tuning for orientation. Results further reveal a coarse-scale map of orientation preference that may constitute the neural basis for known perceptual anisotropies. These findings settle a long-standing debate in human neuroscience, and provide insights into functional organization principles of visual cortex.

Layer-specific, retinotopically-diffuse modulation in human visual cortex in response to viewing emotionally expressive faces.

Viewing faces that are perceived as emotionally expressive evokes enhanced neural responses in multiple brain regions, a phenomenon thought to depend critically on the amygdala. This emotion-related modulation is evident even in primary visual cortex (V1), providing a potential neural substrate by which emotionally salient stimuli can affect perception. How does emotional valence information, computed in the amygdala, reach V1? Here we use high-resolution functional MRI to investigate the layer profile and retinotopic distribution of neural activity specific to emotional facial expressions. Across three experiments, human participants viewed centrally presented face stimuli varying in emotional expression and performed a gender judgment task. We found that facial valence sensitivity was evident only in superficial cortical layers and was not restricted to the retinotopic location of the stimuli, consistent with diffuse feedback-like projections from the amygdala. Together, our results provide a feedback mechanism by which the amygdala directly modulates activity at the earliest stage of visual processing.

Task-related hemodynamic responses in human early visual cortex are modulated by task difficulty and behavioral performance.

Early visual cortex exhibits widespread hemodynamic responses in the absence of visual stimulation, which are entrained to the timing of a task and not predicted by local spiking or local field potential. Such task-related responses (TRRs) covary with reward magnitude and physiological signatures of arousal. It is unknown, however, if TRRs change on a trial-to-trial basis according to behavioral performance and task difficulty. If so, this would suggest that TRRs reflect arousal on a trial-to-trial timescale and covary with critical task and behavioral variables. We measured functional magnetic resonance imaging blood-oxygen-level-dependent (fMRI-BOLD) responses in the early visual cortex of human observers performing an orientation discrimination task consisting of separate easy and hard runs of trials. Stimuli were presented in a small portion of one hemifield, but the fMRI response was measured in the ipsilateral hemisphere, far from the stimulus representation and focus of spatial attention. TRRs scaled in amplitude with task difficulty, behavioral accuracy, reaction time, and lapses across trials. These modulations were not explained by the influence of respiration, cardiac activity, or head movement on the fMRI signal. Similar modulations with task difficulty and behavior were observed in pupil size. These results suggest that TRRs reflect arousal and behavior on the timescale of individual trials.

Spatiotopic and retinotopic memory in the context of natural images.

Neural responses throughout the visual cortex encode stimulus location in a retinotopic (i.e., eye-centered) reference frame, and memory for stimulus position is most precise in retinal coordinates. Yet visual perception is spatiotopic: objects are perceived as stationary, even though eye movements cause frequent displacement of their location on the retina. Previous studies found that, after a single saccade, memory of retinotopic locations is more accurate than memory of spatiotopic locations. However, it is not known whether various aspects of natural viewing affect the retinotopic reference frame advantage. We found that the retinotopic advantage may in part depend on a retinal afterimage, which can be effectively nullified through backwards masking. Moreover, in the presence of natural scenes, spatiotopic memory is more accurate than retinotopic memory, but only when subjects are provided sufficient time to process the scene before the eye movement. Our results demonstrate that retinotopic memory is not always more accurate than spatiotopic memory and that the fidelity of memory traces in both reference frames are sensitive to the presence of contextual cues.

Spared perilesional V1 activity underlies training-induced recovery of luminance detection sensitivity in cortically-blind patients.

Damage to the primary visual cortex (V1) causes homonymous visual-field loss long considered intractable. Multiple studies now show that perceptual training can restore visual functions in chronic cortically-induced blindness (CB). A popular hypothesis is that training can harness residual visual functions by recruiting intact extrageniculostriate pathways. Training may also induce plastic changes within spared regions of the damaged V1. Here, we link changes in luminance detection sensitivity with retinotopic fMRI activity before and after visual discrimination training in eleven patients with chronic, stroke-induced CB. We show that spared V1 activity representing perimetrically-blind locations prior to training predicts the amount of training-induced recovery of luminance detection sensitivity. Additionally, training results in an enlargement of population receptive fields in perilesional V1, which increases blind-field coverage and may support further recovery with subsequent training. These findings uncover fundamental changes in perilesional V1 cortex underlying training-induced restoration of conscious luminance detection sensitivity in CB.

Population Models, Not Analyses, of Human Neuroscience Measurements.

Selectivity for many basic properties of visual stimuli, such as orientation, is thought to be organized at the scale of cortical columns, making it difficult or impossible to measure directly with noninvasive human neuroscience measurement. However, computational analyses of neuroimaging data have shown that selectivity for orientation can be recovered by considering the pattern of response across a region of cortex. This suggests that computational analyses can reveal representation encoded at a finer spatial scale than is implied by the spatial resolution limits of measurement techniques. This potentially opens up the possibility to study a much wider range of neural phenomena that are otherwise inaccessible through noninvasive measurement. However, as we review in this article, a large body of evidence suggests an alternative hypothesis to this superresolution account: that orientation information is available at the spatial scale of cortical maps and thus easily measurable at the spatial resolution of standard techniques. In fact, a population model shows that this orientation information need not even come from single-unit selectivity for orientation tuning, but instead can result from population selectivity for spatial frequency. Thus, a categorical error of interpretation can result whereby orientation selectivity can be confused with spatial frequency selectivity. This is similarly problematic for the interpretation of results from numerous studies of more complex representations and cognitive functions that have built upon the computational techniques used to reveal stimulus orientation. We suggest in this review that these interpretational ambiguities can be avoided by treating computational analyses as models of the neural processes that give rise to measurement. Building upon the modeling tradition in vision science using considerations of whether population models meet a set of core criteria is important for creating the foundation for a cumulative and replicable approach to making valid inferences from human neuroscience measurements.

Differential impact of endogenous and exogenous attention on activity in human visual cortex.

How do endogenous (voluntary) and exogenous (involuntary) attention modulate activity in visual cortex? Using ROI-based fMRI analysis, we measured fMRI activity for valid and invalid trials (target at cued/un-cued location, respectively), pre- or post-cueing endogenous or exogenous attention, while participants performed the same orientation discrimination task. We found stronger modulation in contralateral than ipsilateral visual regions, and higher activity in valid- than invalid-trials. For endogenous attention, modulation of stimulus-evoked activity due to a pre-cue increased along the visual hierarchy, but was constant due to a post-cue. For exogenous attention, modulation of stimulus-evoked activity due to a pre-cue was constant along the visual hierarchy, but was not modulated due to a post-cue. These findings reveal that endogenous and exogenous attention distinctly modulate activity in visuo-occipital areas during orienting and reorienting; endogenous attention facilitates both the encoding and the readout of visual information whereas exogenous attention only facilitates the encoding of information.

Task-related activity in human visual cortex.

The brain exhibits widespread endogenous responses in the absence of visual stimuli, even at the earliest stages of visual cortical processing. Such responses have been studied in monkeys using optical imaging with a limited field of view over visual cortex. Here, we used functional MRI (fMRI) in human participants to study the link between arousal and endogenous responses in visual cortex. The response that we observed was tightly entrained to task timing, was spatially extensive, and was independent of visual stimulation. We found that this response follows dynamics similar to that of pupil size and heart rate, suggesting that task-related activity is related to arousal. Finally, we found that higher reward increased response amplitude while decreasing its trial-to-trial variability (i.e., the noise). Computational simulations suggest that increased temporal precision underlies both of these observations. Our findings are consistent with optical imaging studies in monkeys and support the notion that arousal increases precision of neural activity.

What do across-subject analyses really tell us about neural coding?

A key challenge in human neuroscience is to gain information about patterns of neural activity using indirect measures. Multivariate pattern analysis methods testing for generalization of information across subjects have been used to support inferences regarding neural coding. One critical assumption of an important class of such methods is that anatomical normalization is suited to align spatially-structured neural patterns across individual brains. We asked whether anatomical normalization is suited for this purpose. If not, what sources of information are such across-subject cross-validated analyses likely to reveal? To investigate these questions, we implemented two-layered feedforward randomly-connected networks. A key feature of these simulations was a gain-field with a spatial structure shared across networks. To investigate whether total-signal imbalances across conditions-e.g. differences in overall activity-affect the observed pattern of results, we manipulated the energy-profile of images conforming to a pre-specified correlation structure. To investigate whether the level of granularity of the data also influences results, we manipulated the density of connections between network layers. Simulations showed that anatomical normalization is unsuited to align neural representations. Pattern similarity-relationships were explained by the observed total-signal imbalances across conditions. Further, we observed that deceptively complex representational structures emerge from arbitrary analysis choices, such as whether the data are mean-subtracted during preprocessing. These simulations also led to testable predictions regarding the distribution of low-level features in images used in recent fMRI studies that relied on leave-one-subject-out pattern analyses. Image analyses broadly confirmed these predictions. Finally, hyperalignment emerged as a principled alternative to test across-subject generalization of spatially-structured information. We illustrate cases in which hyperalignment proved successful, as well as cases in which it only partially recovered the latent correlation structure in the pattern of responses. Our results highlight the need for robust, high-resolution measurements from individual subjects. We also offer a way forward for across-subject analyses. We suggest ways to inform hyperalignment results with estimates of the strength of the signal associated with each condition. Such information can usefully constrain ensuing inferences regarding latent representational structures as well as population tuning dimensions.

Layer-Specific Contributions to Imagined and Executed Hand Movements in Human Primary Motor Cortex.

The human ability to imagine motor actions without executing them (i.e., motor imagery) is crucial to a number of cognitive functions, including motor planning and learning, and has been shown to improve response times and accuracy of subsequent motor actions [1, 2]. Although these behavioral findings suggest the possibility that imagined movements directly influence primary motor cortex (M1), how this might occur remains unknown [3]. Here, we use a non-blood-oxygen-level-dependent (BOLD) method for collecting fMRI data, called vascular space occupancy (VASO) [4, 5], to measure neural activations across cortical laminae in M1 while participants either tapped their thumb and forefinger together or simply imagined doing so. We report that, whereas executed movements (i.e., finger tapping) evoked neural responses in both the superficial layers of M1 that receive cortical input and the deep layers of M1 that send output to the spinal cord to support movement, imagined movements evoked responses in superficial cortical layers only. Furthermore, we found that finger tapping preceded by both imagined and executed movements showed a reduced response in the superficial layers (repetition suppression) coupled with a heightened response in the deep layers (repetition enhancement). Taken together, our results provide evidence for a mechanism whereby imagined movements can directly affect motor performance and might explain how neural repetition effects lead to improvements in behavior (e.g., repetition priming).

Visual temporal frequency preference shows a distinct cortical architecture using fMRI.

Studies of visual temporal frequency preference typically examine frequencies under 20 Hz and measure local activity to evaluate the sensitivity of different cortical areas to variations in temporal frequencies. Most of these studies have not attempted to map preferred temporal frequency within and across visual areas, nor have they explored in detail, stimuli at gamma frequency, which recent research suggests may have potential clinical utility. In this study, we address this gap by using functional magnetic resonance imaging (fMRI) to measure response to flickering visual stimuli varying in frequency from 1 to 40 Hz. We apply stimulation in both a block design to examine task response and a steady-state design to examine functional connectivity. We observed distinct activation patterns between 1 Hz and 40 Hz stimuli. We also found that the correlation between medial thalamus and visual cortex was modulated by the temporal frequency. The modulation functions and tuned frequencies are different for the visual activity and thalamo-visual correlations. Using both fMRI activity and connectivity measurements, we show evidence for a temporal frequency specific organization across the human visual system.

Stimulus vignetting and orientation selectivity in human visual cortex.

Neural selectivity to orientation is one of the simplest and most thoroughly-studied cortical sensory features. Here, we show that a large body of research that purported to measure orientation tuning may have in fact been inadvertently measuring sensitivity to second-order changes in luminance, a phenomenon we term 'vignetting'. Using a computational model of neural responses in primary visual cortex (V1), we demonstrate the impact of vignetting on simulated V1 responses. We then used the model to generate a set of predictions, which we confirmed with functional MRI experiments in human observers. Our results demonstrate that stimulus vignetting can wholly determine the orientation selectivity of responses in visual cortex measured at a macroscopic scale, and suggest a reinterpretation of a well-established literature on orientation processing in visual cortex.

Specific Visual Subregions of TPJ Mediate Reorienting of Spatial Attention.

The temporo-parietal junction (TPJ) has been associated with various cognitive and social functions, and is critical for attentional reorienting. Attention affects early visual processing. Neuroimaging studies dealing with such processes have thus far concentrated on striate and extrastriate areas. Here, we investigated whether attention orienting or reorienting modulate activity in visually driven TPJ subregions. For each observer we identified 3 visually responsive subregions within TPJ: 2 bilateral (vTPJant and vTPJpost) and 1 right lateralized (vTPJcent). Cortical activity in these subregions was measured using fMRI while observers performed a 2-alternative forced-choice orientation discrimination task. Covert spatial endogenous (voluntary) or exogenous (involuntary) attention was manipulated using either a central or a peripheral cue with task, stimuli and observers constant. Both endogenous and exogenous attention increased activity for invalidly cued trials in right vTPJpost; only endogenous attention increased activity for invalidly cued trials in left vTPJpost and in right vTPJcent; and neither type of attention modulated either right or left vTPJant. These results demonstrate that vTPJpost and vTPJcent mediate the reorientation of covert attention to task relevant stimuli, thus playing a critical role in visual attention. These findings reveal a differential reorienting cortical response after observers' attention has been oriented to a given location voluntarily or involuntarily.

Spontaneous microsaccades reflect shifts in covert attention.

Microsaccade rate during fixation is modulated by the presentation of a visual stimulus. When the stimulus is an endogenous attention cue, the ensuing microsaccades tend to be directed toward the cue. This finding has been taken as evidence that microsaccades index the locus of spatial attention. But the vast majority of microsaccades that subjects make are not triggered by visual stimuli. Under natural viewing conditions, spontaneous microsaccades occur frequently (2-3 Hz), even in the absence of a stimulus or a task. While spontaneous microsaccades may depend on low-level visual demands, such as retinal fatigue, image fading, or fixation shifts, it is unknown whether their occurrence corresponds to changes in the attentional state. We developed a protocol to measure whether spontaneous microsaccades reflect shifts in spatial attention. Human subjects fixated a cross while microsaccades were detected from streaming eye-position data. Detection of a microsaccade triggered the appearance of a peripheral ring of grating patches, which were followed by an arrow (a postcue) indicating one of them as the target. The target was either congruent or incongruent (opposite) with respect to the direction of the microsaccade (which preceded the stimulus). Subjects reported the tilt of the target (clockwise or counterclockwise relative to vertical). We found that accuracy was higher for congruent than for incongruent trials. We conclude that the direction of spontaneous microsaccades is inherently linked to shifts in spatial attention.