Temporal dynamics of short-term neural adaptation across human visual cortex.

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses between lower and higher visual brain areas and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Feedback scales the spatial tuning of cortical responses during visual memory.

Perception, working memory, and long-term memory each evoke neural responses in visual cortex, suggesting that memory uses encoding mechanisms shared with perception. While previous research has largely focused on how perception and memory are similar, we hypothesized that responses in visual cortex would differ depending on the origins of the inputs. Using fMRI, we quantified spatial tuning in visual cortex while participants (both sexes) viewed, maintained in working memory, or retrieved from long-term memory a peripheral target. In each of these conditions, BOLD responses were spatially tuned and were aligned with the target's polar angle in all measured visual field maps including V1. As expected given the increasing sizes of receptive fields, polar angle tuning during perception increased in width systematically up the visual hierarchy from V1 to V2, V3, hV4, and beyond. In stark contrast, the widths of tuned responses were broad across the visual hierarchy during working memory and long-term memory, matched to the widths in perception in later visual field maps but much broader in V1. This pattern is consistent with the idea that mnemonic responses in V1 stem from top-down sources. Moreover, these tuned responses when biased (clockwise or counterclockwise of target) predicted matched biases in memory, suggesting that the readout of maintained and reinstated mnemonic responses influences memory guided behavior. We conclude that feedback constrains spatial tuning during memory, where earlier visual maps inherit broader tuning from later maps thereby impacting the precision of memory.

Conservation of cortical crowding distance across individuals in human V4.

Crowding is the failure to recognize an object due to insufficient spacing, which slows daily tasks such as reading and search. Across 49 observers, we found large variations in psychophysical crowding distance and retinotopic map size. These measures covary, conserving a 1.4-mm cortical crowding distance (threshold object spacing on the cortical surface) in the human V4 map, but not V1-V3. This links the spacing limit of visual recognition to overall V4 size.

Spatial attention alters visual cortical representation during target anticipation.

Attention enables us to efficiently and flexibly interact with the environment by prioritizing some image features in preparation for responding to a stimulus. Using a concurrent psychophysics- fMRI experiment, we investigated how covert spatial attention affects responses in human visual cortex prior to target onset, and how it affects subsequent behavioral performance. Performance improved at cued locations and worsened at uncued locations, relative to distributed attention, demonstrating a selective tradeoff in processing. Pre-target BOLD responses in cortical visual field maps changed in two ways: First, there was a stimulus-independent baseline shift, positive in map locations near the cued location and negative elsewhere, paralleling the behavioral results. Second, population receptive field centers shifted toward the attended location. Both effects increased in higher visual areas. Together, the results show that spatial attention has large effects on visual cortex prior to target appearance, altering neural response properties throughout and across multiple visual field maps.

Temporal dynamics of short-term neural adaptation across human visual cortex.

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses across the human visual hierarchy and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Normalization by orientation-tuned surround in human V1-V3.

An influential account of neuronal responses in primary visual cortex is the normalized energy model. This model is often implemented as a multi-stage computation. The first stage is linear filtering. The second stage is the extraction of contrast energy, whereby a complex cell computes the squared and summed outputs of a pair of the linear filters in quadrature phase. The third stage is normalization, in which a local population of complex cells mutually inhibit one another. Because the population includes cells tuned to a range of orientations and spatial frequencies, the result is that the responses are effectively normalized by the local stimulus contrast. Here, using evidence from human functional MRI, we show that the classical model fails to account for the relative responses to two classes of stimuli: straight, parallel, band-passed contours (gratings), and curved, band-passed contours (snakes). The snakes elicit fMRI responses that are about twice as large as the gratings, yet a traditional divisive normalization model predicts responses that are about the same. Motivated by these observations and others from the literature, we implement a divisive normalization model in which cells matched in orientation tuning ("tuned normalization") preferentially inhibit each other. We first show that this model accounts for differential responses to these two classes of stimuli. We then show that the model successfully generalizes to other band-pass textures, both in V1 and in extrastriate cortex (V2 and V3). We conclude that even in primary visual cortex, complex features of images such as the degree of heterogeneity, can have large effects on neural responses.

The Bouma law accounts for crowding in 50 observers.

Crowding is the failure to recognize an object due to surrounding clutter. Our visual crowding survey measured 13 crowding distances (or "critical spacings") twice in each of 50 observers. The survey includes three eccentricities (0, 5, and 10 deg), four cardinal meridians, two orientations (radial and tangential), and two fonts (Sloan and Pelli). The survey also tested foveal acuity, twice. Remarkably, fitting a two-parameter model-the well-known Bouma law, where crowding distance grows linearly with eccentricity-explains 82% of the variance for all 13 × 50 measured log crowding distances, cross-validated. An enhanced Bouma law, with factors for meridian, crowding orientation, target kind, and observer, explains 94% of the variance, again cross-validated. These additional factors reveal several asymmetries, consistent with previous reports, which can be expressed as crowding-distance ratios: 0.62 horizontal:vertical, 0.79 lower:upper, 0.78 right:left, 0.55 tangential:radial, and 0.78 Sloan-font:Pelli-font. Across our observers, peripheral crowding is independent of foveal crowding and acuity. Evaluation of the Bouma factor, b (the slope of the Bouma law), as a biomarker of visual health would be easier if there were a way to compare results across crowding studies that use different methods. We define a standardized Bouma factor b' that corrects for differences from Bouma's 25 choice alternatives, 75% threshold criterion, and linearly symmetric flanker placement. For radial crowding on the right meridian, the standardized Bouma factor b' is 0.24 for this study, 0.35 for Bouma (1970), and 0.30 for the geometric mean across five representative modern studies, including this one, showing good agreement across labs, including Bouma's. Simulations, confirmed by data, show that peeking can skew estimates of crowding (e.g., greatly decreasing the mean or doubling the SD of log b). Using gaze tracking to prevent peeking, individual differences are robust, as evidenced by the much larger 0.08 SD of log b across observers than the mere 0.03 test-retest SD of log b measured in half an hour. The ease of measurement of crowding enhances its promise as a biomarker for dyslexia and visual health.

Polar angle asymmetries in visual perception and neural architecture.

Human visual performance changes with visual field location. It is best at the center of gaze and declines with eccentricity, and also varies markedly with polar angle. These perceptual polar angle asymmetries are linked to asymmetries in the organization of the visual system. We review and integrate research quantifying how performance changes with visual field location and how this relates to neural organization at multiple stages of the visual system. We first briefly review how performance varies with eccentricity and the neural foundations of this effect. We then focus on perceptual polar angle asymmetries and their neural foundations. Characterizing perceptual and neural variations across and around the visual field contributes to our understanding of how the brain translates visual signals into neural representations which form the basis of visual perception.

Precise Spatial Tuning of Visually Driven Alpha Oscillations in Human Visual Cortex.

Neuronal oscillations at about 10 Hz, called alpha oscillations, are often thought to arise from synchronous activity across occipital cortex, reflecting general cognitive states such as arousal and alertness. However, there is also evidence that modulation of alpha oscillations in visual cortex can be spatially specific. Here, we used intracranial electrodes in human patients to measure alpha oscillations in response to visual stimuli whose location varied systematically across the visual field. We separated the alpha oscillatory power from broadband power changes. The variation in alpha oscillatory power with stimulus position was then fit by a population receptive field (pRF) model. We find that the alpha pRFs have similar center locations to pRFs estimated from broadband power (70-180 Hz), but are several times larger. The results demonstrate that alpha suppression in human visual cortex can be precisely tuned. Finally, we show how the pattern of alpha responses can explain several features of exogenous visual attention. Significance Statement: The alpha oscillation is the largest electrical signal generated by the human brain. An important question in systems neuroscience is the degree to which this oscillation reflects system-wide states and behaviors such as arousal, alertness, and attention, versus much more specific functions in the routing and processing of information. We examined alpha oscillations at high spatial precision in human patients with intracranial electrodes implanted over visual cortex. We discovered a surprisingly high spatial specificity of visually driven alpha oscillations, which we quantified with receptive field models. We further use our discoveries about properties of the alpha response to show a link between these oscillations and the spread of visual attention.

Asymmetries in the discrimination of motion direction around the visual field.

The discriminability of motion direction is asymmetric, with some motion directions that are better discriminated than others. For example, discrimination of directions near the cardinal axes (upward/downward/leftward/rightward) tends to be better than oblique directions. Here, we tested discriminability for multiple motion directions at multiple polar angle locations. We found three systematic asymmetries. First, we found a large cardinal advantage in a cartesian reference frame - better discriminability for motion near cardinal reference directions than oblique directions. Second, we found a moderate cardinal advantage in a polar reference frame - better discriminability for motion near radial (inward/outward) and tangential (clockwise/counterclockwise) reference directions than other directions. Third, we found a small advantage for discriminating motion near radial compared to tangential reference directions. The three advantages combine in an approximately linear manner, and together predict variation in motion discrimination as a function of both motion direction and location around the visual field. For example, best performance is found for radial motion on the horizontal and vertical meridians, as these directions encompass all three advantages, whereas poorest performance is found for oblique motion stimuli located on the horizontal and vertical meridians, as these directions encompass all three disadvantages. Our results constrain models of motion perception and suggest that reference frames at multiple stages of the visual processing hierarchy limit performance.

Comparing retinotopic maps of children and adults reveals a late-stage change in how V1 samples the visual field.

Adult visual performance differs with angular location -it is better for stimuli along the horizontal than vertical, and lower than upper vertical meridian of the visual field. These perceptual asymmetries are paralleled by asymmetries in cortical surface area in primary visual cortex (V1). Children, unlike adults, have similar visual performance at the lower and upper vertical meridian. Do children have similar V1 surface area representing the upper and lower vertical meridian? Using MRI, we measure the surface area of retinotopic maps (V1-V3) in children and adults. Many features of the maps are similar between groups, including greater V1 surface area for the horizontal than vertical meridian. However, unlike adults, children have a similar amount of V1 surface area representing the lower and upper vertical meridian. These data reveal a late-stage change in V1 organization that may relate to the emergence of the visual performance asymmetry along the vertical meridian by adulthood.

Mnemonic Content and Hippocampal Patterns Shape Judgments of Time.

Our experience of time can feel dilated or compressed, rather than reflecting true "clock time." Although many contextual factors influence the subjective perception of time, it is unclear how memory accessibility plays a role in constructing our experience of and memory for time. Here, we used a combination of behavioral and functional MRI measures in healthy young adults (N = 147) to ask the question of how memory is incorporated into temporal duration judgments. Behaviorally, we found that event boundaries, which have been shown to disrupt ongoing memory integration processes, result in the temporal compression of duration judgments. Additionally, using a multivoxel pattern similarity analysis of functional MRI data, we found that greater temporal pattern change in the left hippocampus within individual trials was associated with longer duration judgments. Together, these data suggest that mnemonic processes play a role in constructing representations of time.

Perception and memory have distinct spatial tuning properties in human visual cortex.

Reactivation of earlier perceptual activity is thought to underlie long-term memory recall. Despite evidence for this view, it is unclear whether mnemonic activity exhibits the same tuning properties as feedforward perceptual activity. Here, we leverage population receptive field models to parameterize fMRI activity in human visual cortex during spatial memory retrieval. Though retinotopic organization is present during both perception and memory, large systematic differences in tuning are also evident. Whereas there is a three-fold decline in spatial precision from early to late visual areas during perception, this pattern is not observed during memory retrieval. This difference cannot be explained by reduced signal-to-noise or poor performance on memory trials. Instead, by simulating top-down activity in a network model of cortex, we demonstrate that this property is well explained by the hierarchical structure of the visual system. Together, modeling and empirical results suggest that computational constraints imposed by visual system architecture limit the fidelity of memory reactivation in sensory cortex.

Variability of the Surface Area of the V1, V2, and V3 Maps in a Large Sample of Human Observers.

How variable is the functionally defined structure of early visual areas in human cortex and how much variability is shared between twins? Here we quantify individual differences in the best understood functionally defined regions of cortex: V1, V2, V3. The Human Connectome Project 7T Retinotopy Dataset includes retinotopic measurements from 181 subjects (109 female, 72 male), including many twins. We trained four "anatomists" to manually define V1-V3 using retinotopic features. These definitions were more accurate than automated anatomical templates and showed that surface areas for these maps varied more than threefold across individuals. This threefold variation was little changed when normalizing visual area size by the surface area of the entire cerebral cortex. In addition to varying in size, we find that visual areas vary in how they sample the visual field. Specifically, the cortical magnification function differed substantially among individuals, with the relative amount of cortex devoted to central vision varying by more than a factor of 2. To complement the variability analysis, we examined the similarity of visual area size and structure across twins. Whereas the twin sample sizes are too small to make precise heritability estimates (50 monozygotic pairs, 34 dizygotic pairs), they nonetheless reveal high correlations, consistent with strong effects of the combination of shared genes and environment on visual area size. Collectively, these results provide the most comprehensive account of individual variability in visual area structure to date, and provide a robust population benchmark against which new individuals and developmental and clinical populations can be compared.SIGNIFICANCE STATEMENT Areas V1, V2, and V3 are among the best studied functionally defined regions in human cortex. Using the largest retinotopy dataset to date, we characterized the variability of these regions across individuals and the similarity between twin pairs. We find that the size of visual areas varies dramatically (up to 3.5×) across healthy young adults, far more than the variability of the cerebral cortex size as a whole. Much of this variability appears to arise from inherited factors, as we find very high correlations in visual area size between monozygotic twin pairs, and lower but still substantial correlations between dizygotic twin pairs. These results provide the most comprehensive assessment of how functionally defined visual cortex varies across the population to date.

Non-neural factors influencing BOLD response magnitudes within individual subjects.

To what extent is the size of the blood-oxygen-level-dependent (BOLD) response influenced by factors other than neural activity? In a re-analysis of three neuroimaging datasets (male and female human participants), we find large systematic inhomogeneities in the BOLD response magnitude in primary visual cortex (V1): stimulus-evoked BOLD responses, expressed in units of percent signal change, are up to 50% larger along the representation of the horizontal meridian than the vertical meridian. To assess whether this surprising effect can be interpreted as differences in local neural activity, we quantified several factors that potentially contribute to the size of the BOLD response. We find relationships between BOLD response magnitude and cortical thickness, curvature, depth and macrovasculature. These relationships are consistently found across subjects and datasets and suggest that variation in BOLD response magnitudes across cortical locations reflects, in part, differences in anatomy and vascularization. To compensate for these factors, we implement a regression-based correction method and show that after correction, BOLD responses become more homogeneous across V1. The correction reduces the horizontal/vertical difference by about half, indicating that some of the difference is likely not due to neural activity differences. We conclude that interpretation of variation in BOLD response magnitude across cortical locations should consider the influence of the potential confounding factors of thickness, curvature, depth and vascularization.SIGNIFICANCE STATEMENTThe magnitude of the BOLD signal is often used as a surrogate of neural activity, but the exact factors that contribute to its strength have not been studied on a voxel-wise level. Here, we examined several anatomical and measurement-related factors to assess their relationship with BOLD signal magnitude. We find that BOLD magnitude correlates with cortical anatomy, depth and macrovasculature. To remove the contribution of these factors, we propose a simple, data-driven correction method that can be used in any functional magnetic resonance imaging (fMRI) experiment. After accounting for the confounding factors, BOLD magnitude becomes more spatially homogenous. Our correction method improves the ability to make more accurate inferences about local neural activity from fMRI data.

What has vision science taught us about functional MRI?

In the domain of human neuroimaging, much attention has been paid to the question of whether and how the development of functional magnetic resonance imaging (fMRI) has advanced our scientific knowledge of the human brain. However, the opposite question is also important; how has our knowledge of the brain advanced our understanding of fMRI? Here, we discuss how and why scientific knowledge about the human and animal visual system has been used to answer fundamental questions about fMRI as a brain measurement tool and how these answers have contributed to scientific discoveries beyond vision science.

Temporal Dynamics of Neural Responses in Human Visual Cortex.

Neural responses to visual stimuli exhibit complex temporal dynamics, including subadditive temporal summation, response reduction with repeated or sustained stimuli (adaptation), and slower dynamics at low contrast. These phenomena are often studied independently. Here, we demonstrate these phenomena within the same experiment and model the underlying neural computations with a single computational model. We extracted time-varying responses from electrocorticographic recordings from patients presented with stimuli that varied in duration, interstimulus interval (ISI) and contrast. Aggregating data across patients from both sexes yielded 98 electrodes with robust visual responses, covering both earlier (V1-V3) and higher-order (V3a/b, LO, TO, IPS) retinotopic maps. In all regions, the temporal dynamics of neural responses exhibit several nonlinear features. Peak response amplitude saturates with high contrast and longer stimulus durations, the response to a second stimulus is suppressed for short ISIs and recovers for longer ISIs, and response latency decreases with increasing contrast. These features are accurately captured by a computational model composed of a small set of canonical neuronal operations, that is, linear filtering, rectification, exponentiation, and a delayed divisive normalization. We find that an increased normalization term captures both contrast- and adaptation-related response reductions, suggesting potentially shared underlying mechanisms. We additionally demonstrate both changes and invariance in temporal response dynamics between earlier and higher-order visual areas. Together, our results reveal the presence of a wide range of temporal and contrast-dependent neuronal dynamics in the human visual cortex and demonstrate that a simple model captures these dynamics at millisecond resolution.SIGNIFICANCE STATEMENT Sensory inputs and neural responses change continuously over time. It is especially challenging to understand a system that has both dynamic inputs and outputs. Here, we use a computational modeling approach that specifies computations to convert a time-varying input stimulus to a neural response time course, and we use this to predict neural activity measured in the human visual cortex. We show that this computational model predicts a wide variety of complex neural response shapes, which we induced experimentally by manipulating the duration, repetition, and contrast of visual stimuli. By comparing data and model predictions, we uncover systematic properties of temporal dynamics of neural signals, allowing us to better understand how the brain processes dynamic sensory information.

Structural covariance and heritability of the optic tract and primary visual cortex in living human brains.

Individual differences among human brains exist at many scales, spanning gene expression, white matter tissue properties, and the size and shape of cortical areas. One notable example is an approximately 3-fold range in the size of human primary visual cortex (V1), a much larger range than is found in overall brain size. A previous study (Andrews et al., 1997) reported a correlation between optic tract cross-section area and V1 size in post-mortem human brains, suggesting that there may be a common developmental mechanism for multiple components of the visual pathways. We evaluated the relationship between properties of the optic tract and V1 in a much larger sample of living human brains by analyzing the Human Connectome Project 7 Tesla Retinotopy Dataset (including 107 females and 71 males). This dataset includes retinotopic maps measured with functional MRI (fMRI) and fiber tract data measured with diffusion MRI (dMRI). We found a negative correlation between optic tract fractional anisotropy and V1 surface area (r = -0.19). This correlation, though small, was consistent across multiple dMRI datasets differing in acquisition parameters. Further, we found that both V1 size and optic tract properties were correlated among twins, with higher correlations for monozygotic than dizygotic twins, indicating a high degree of heritability for both properties. Together, these results demonstrate covariation across individuals in properties of the retina (optic tract) and cortex (V1) and show that each is influenced by genetic factors.SIGNIFICANCE STATEMENT:The size of human primary visual cortex (V1) has large inter-individual differences. These differences do not scale with overall brain size. A previous post-mortem study reported a correlation between the size of the human optic tract and V1. In this study, we evaluated the relationship between the optic tract and V1 in living humans by analyzing a neuroimaging dataset that included functional and diffusion MRI data. We found a small, but robust correlation between optic tract tissue properties and V1 size, supporting the existence of structural covariance between the optic tract and V1 in living humans. The results suggest that characteristics of retinal ganglion cells, reflected in optic tract measurements, are related to individual differences in human V1.

Linking individual differences in human primary visual cortex to contrast sensitivity around the visual field.

A central question in neuroscience is how the organization of cortical maps relates to perception, for which human primary visual cortex (V1) is an ideal model system. V1 nonuniformly samples the retinal image, with greater cortical magnification (surface area per degree of visual field) at the fovea than periphery and at the horizontal than vertical meridian. Moreover, the size and cortical magnification of V1 varies greatly across individuals. Here, we used fMRI and psychophysics in the same observers to quantify individual differences in V1 cortical magnification and contrast sensitivity at the four polar angle meridians. Across observers, the overall size of V1 and localized cortical magnification positively correlated with contrast sensitivity. Moreover, greater cortical magnification and higher contrast sensitivity at the horizontal than the vertical meridian were strongly correlated. These data reveal a link between cortical anatomy and visual perception at the level of individual observer and stimulus location.

Mapping spatial frequency preferences across human primary visual cortex.

Neurons in primate visual cortex (area V1) are tuned for spatial frequency, in a manner that depends on their position in the visual field. Several studies have examined this dependency using functional magnetic resonance imaging (fMRI), reporting preferred spatial frequencies (tuning curve peaks) of V1 voxels as a function of eccentricity, but their results differ by as much as two octaves, presumably owing to differences in stimuli, measurements, and analysis methodology. Here, we characterize spatial frequency tuning at a millimeter resolution within the human primary visual cortex, across stimulus orientation and visual field locations. We measured fMRI responses to a novel set of stimuli, constructed as sinusoidal gratings in log-polar coordinates, which include circular, radial, and spiral geometries. For each individual stimulus, the local spatial frequency varies inversely with eccentricity, and for any given location in the visual field, the full set of stimuli span a broad range of spatial frequencies and orientations. Over the measured range of eccentricities, the preferred spatial frequency is well-fit by a function that varies as the inverse of the eccentricity plus a small constant. We also find small but systematic effects of local stimulus orientation, defined in both absolute coordinates and relative to visual field location. Specifically, peak spatial frequency is higher for pinwheel than annular stimuli and for horizontal than vertical stimuli.