Characterizing Spatiotemporal Population Receptive Fields in Human Visual Cortex with fMRI.

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time-varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants (both females and males). We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (1) from early to later areas within a visual stream, spatial and temporal windows of pRFs progressively increase in size and show greater compressive nonlinearities, (2) later visual areas show diverging spatial and temporal windows across streams, and (3) within early visual areas (V1-V3), both spatial and temporal windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses using fMRI.

Asymmetries around the visual field: From retina to cortex to behavior.

Visual performance varies around the visual field. It is best near the fovea compared to the periphery, and at iso-eccentric locations it is best on the horizontal, intermediate on the lower, and poorest on the upper meridian. The fovea-to-periphery performance decline is linked to the decreases in cone density, retinal ganglion cell (RGC) density, and V1 cortical magnification factor (CMF) as eccentricity increases. The origins of polar angle asymmetries are not well understood. Optical quality and cone density vary across the retina, but recent computational modeling has shown that these factors can only account for a small percentage of behavior. Here, we investigate how visual processing beyond the cone photon absorptions contributes to polar angle asymmetries in performance. First, we quantify the extent of asymmetries in cone density, midget RGC density, and V1 CMF. We find that both polar angle asymmetries and eccentricity gradients increase from cones to mRGCs, and from mRGCs to cortex. Second, we extend our previously published computational observer model to quantify the contribution of phototransduction by the cones and spatial filtering by mRGCs to behavioral asymmetries. Starting with photons emitted by a visual display, the model simulates the effect of human optics, cone isomerizations, phototransduction, and mRGC spatial filtering. The model performs a forced choice orientation discrimination task on mRGC responses using a linear support vector machine classifier. The model shows that asymmetries in a decision maker's performance across polar angle are greater when assessing the photocurrents than when assessing isomerizations and are greater still when assessing mRGC signals. Nonetheless, the polar angle asymmetries of the mRGC outputs are still considerably smaller than those observed from human performance. We conclude that cone isomerizations, phototransduction, and the spatial filtering properties of mRGCs contribute to polar angle performance differences, but that a full account of these differences will entail additional contribution from cortical representations.

A visual encoding model links magnetoencephalography signals to neural synchrony in human cortex.

Synchronization of neuronal responses over large distances is hypothesized to be important for many cortical functions. However, no straightforward methods exist to estimate synchrony non-invasively in the living human brain. MEG and EEG measure the whole brain, but the sensors pool over large, overlapping cortical regions, obscuring the underlying neural synchrony. Here, we developed a model from stimulus to cortex to MEG sensors to disentangle neural synchrony from spatial pooling of the instrument. We find that synchrony across cortex has a surprisingly large and systematic effect on predicted MEG spatial topography. We then conducted visual MEG experiments and separated responses into stimulus-locked and broadband components. The stimulus-locked topography was similar to model predictions assuming synchronous neural sources, whereas the broadband topography was similar to model predictions assuming asynchronous sources. We infer that visual stimulation elicits two distinct types of neural responses, one highly synchronous and one largely asynchronous across cortex.

A population receptive field model of the magnetoencephalography response.

Computational models which predict the neurophysiological response from experimental stimuli have played an important role in human neuroimaging. One type of computational model, the population receptive field (pRF), has been used to describe cortical responses at the millimeter scale using functional magnetic resonance imaging (fMRI) and electrocorticography (ECoG). However, pRF models are not widely used for non-invasive electromagnetic field measurements (EEG/MEG), because individual sensors pool responses originating from several centimeter of cortex, containing neural populations with widely varying spatial tuning. Here, we introduce a forward-modeling approach in which pRFs estimated from fMRI data are used to predict MEG sensor responses. Subjects viewed contrast-reversing bar stimuli sweeping across the visual field in separate fMRI and MEG sessions. Individual subject's pRFs were modeled on the cortical surface at the millimeter scale using the fMRI data. We then predicted cortical time series and projected these predictions to MEG sensors using a biophysical MEG forward model, accounting for the pooling across cortex. We compared the predicted MEG responses to observed visually evoked steady-state responses measured in the MEG session. We found that pRF parameters estimated by fMRI could explain a substantial fraction of the variance in steady-state MEG sensor responses (up to 60% in individual sensors). Control analyses in which we artificially perturbed either pRF size or pRF position reduced MEG prediction accuracy, indicating that MEG data are sensitive to pRF properties derived from fMRI. Our model provides a quantitative approach to link fMRI and MEG measurements, thereby enabling advances in our understanding of spatiotemporal dynamics in human visual field maps.

Modeling visual performance differences 'around' the visual field: A computational observer approach.

Visual performance depends on polar angle, even when eccentricity is held constant; on many psychophysical tasks observers perform best when stimuli are presented on the horizontal meridian, worst on the upper vertical, and intermediate on the lower vertical meridian. This variation in performance 'around' the visual field can be as pronounced as that of doubling the stimulus eccentricity. The causes of these asymmetries in performance are largely unknown. Some factors in the eye, e.g. cone density, are positively correlated with the reported variations in visual performance with polar angle. However, the question remains whether these correlations can quantitatively explain the perceptual differences observed 'around' the visual field. To investigate the extent to which the earliest stages of vision-optical quality and cone density-contribute to performance differences with polar angle, we created a computational observer model. The model uses the open-source software package ISETBIO to simulate an orientation discrimination task for which visual performance differs with polar angle. The model starts from the photons emitted by a display, which pass through simulated human optics with fixational eye movements, followed by cone isomerizations in the retina. Finally, we classify stimulus orientation using a support vector machine to learn a linear classifier on the photon absorptions. To account for the 30% increase in contrast thresholds for upper vertical compared to horizontal meridian, as observed psychophysically on the same task, our computational observer model would require either an increase of ~7 diopters of defocus or a reduction of 500% in cone density. These values far exceed the actual variations as a function of polar angle observed in human eyes. Therefore, we conclude that these factors in the eye only account for a small fraction of differences in visual performance with polar angle. Substantial additional asymmetries must arise in later retinal and/or cortical processing.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields.

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Principles of intensive human neuroimaging.

The rise of large, publicly shared functional magnetic resonance imaging (fMRI) data sets in human neuroscience has focused on acquiring either a few hours of data on many individuals ('wide' fMRI) or many hours of data on a few individuals ('deep' fMRI). In this opinion article, we highlight an emerging approach within deep fMRI, which we refer to as 'intensive' fMRI: one that strives for extensive sampling of cognitive phenomena to support computational modeling and detailed investigation of brain function at the single voxel level. We discuss the fundamental principles, trade-offs, and practical considerations of intensive fMRI. We also emphasize that intensive fMRI does not simply mean collecting more data: it requires careful design of experiments to enable a rich hypothesis space, optimizing data quality, and strategically curating public resources to maximize community impact.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields.

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Characterizing spatiotemporal population receptive fields in human visual cortex with fMRI.

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants. We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (i) from early to later areas within a visual stream, spatial and temporal integration windows of pRFs progressively increase in size and show greater compressive nonlinearities, (ii) later visual areas show diverging spatial and temporal integration windows across streams, and (iii) within early visual areas (V1-V3), both spatial and temporal integration windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses in the human brain using fMRI.

Cortical magnification in human visual cortex parallels task performance around the visual field.

Human vision has striking radial asymmetries, with performance on many tasks varying sharply with stimulus polar angle. Performance is generally better on the horizontal than vertical meridian, and on the lower than upper vertical meridian, and these asymmetries decrease gradually with deviation from the vertical meridian. Here, we report cortical magnification at a fine angular resolution around the visual field. This precision enables comparisons between cortical magnification and behavior, between cortical magnification and retinal cell densities, and between cortical magnification in twin pairs. We show that cortical magnification in the human primary visual cortex, measured in 163 subjects, varies substantially around the visual field, with a pattern similar to behavior. These radial asymmetries in the cortex are larger than those found in the retina, and they are correlated between monozygotic twin pairs. These findings indicate a tight link between cortical topography and behavior, and suggest that visual field asymmetries are partly heritable.

A non-invasive, quantitative study of broadband spectral responses in human visual cortex.

Currently, non-invasive methods for studying the human brain do not routinely and reliably measure spike-rate-dependent signals, independent of responses such as hemodynamic coupling (fMRI) and subthreshold neuronal synchrony (oscillations and event-related potentials). In contrast, invasive methods-microelectrode recordings and electrocorticography (ECoG)-have recently measured broadband power elevation in field potentials (~50-200 Hz) as a proxy for locally averaged spike rates. Here, we sought to detect and quantify stimulus-related broadband responses using magnetoencephalography (MEG). Extracranial measurements like MEG and EEG have multiple global noise sources and relatively low signal-to-noise ratios; moreover high frequency artifacts from eye movements can be confounded with stimulus design and mistaken for signals originating from brain activity. For these reasons, we developed an automated denoising technique that helps reveal the broadband signal of interest. Subjects viewed 12-Hz contrast-reversing patterns in the left, right, or bilateral visual field. Sensor time series were separated into evoked (12-Hz amplitude) and broadband components (60-150 Hz). In all subjects, denoised broadband responses were reliably measured in sensors over occipital cortex, even in trials without microsaccades. The broadband pattern was stimulus-dependent, with greater power contralateral to the stimulus. Because we obtain reliable broadband estimates with short experiments (~20 minutes), and with sufficient signal-to-noise to distinguish responses to different stimuli, we conclude that MEG broadband signals, denoised with our method, offer a practical, non-invasive means for characterizing spike-rate-dependent neural activity for addressing scientific questions about human brain function.